There are already planes that use a single wrist control for pitch and roll, such as the Lancair Columbia 300. If the control forces are light enough, there's no reason why you couldn't add yaw control as well. Fly-by-wire or hydraulic assist could handle the control forces but that would add a lot of expense to the plane.

There are also modifications to existing production planes that allow handicapped pilots to fly using only hand controls. It isn't quite as simple as your diagram but it works. Then there was the Ercoupe which had a control wheel for pitch and roll with no rudder pedals. The rudders were tied in with the ailerons. Thousands of those planes were built and many are still flying.