Picture for a moment what happens if you try to throw an arrow backwards. With the fletching at the front, the arrow is dynamically unstable in every axis and will swap ends rapidly.

Now, picture that poor Airbus. When it lost the vertical stabilizer and rudder, it lost the very thing that makes the plane dynamically stable in the yaw axis. Losing that much vertical surface area would mean the plane would want to skew in yaw, a decidedly undesireable maneuver in an airliner.

Before it lost the vertical stabilizer, the plane was already in an 8 to 10 degree left sideslip, probably as a result of the 747's wingtip vortices. When the stabilizer broke off, the plane almost certainly would've accelerated sharply in the yaw axis to the left. But wait, it gets worse. Swept wing aircraft, when yawed sharply, also tend to roll to the same side as the yaw. It's easy to understand why if you draw a top-view picture of a swept wing plane. When the plane yawed to the left, the wing on that side swung further to the rear, meaning less lift could be generated. It's possible the airflow on the left wing was partially blanked by turbulence from the fuselage, further reducing lift. At the same time, the right wing swung forward, more directly into the airflow. This means the right wing would be generating considerably more lift than the left wing, causing the roll to the left. It's quite likely the rolling moment was stronger than anything the ailerons could handle.

Those engines each weighted several thousand pounds and are on pylons that just were not designed for sharp yaw and roll rotations. Gyroscopic effects from large masses in the engines (fan rotors and turbine blades) might've been a factor as well. It isn't at all hard to understand why those engines came off shortly after the plane lost the vertical stabilizer and rudder. From that second on, the plane was hopelessly doomed.