Do a quick calculation
Assume that a piece of debris breaking loose at, say, Mach 1, will almost immediately acclerate, relative to the Shuttle, to a similar velocity (relative to us on the ground, of course, the piece of debris is decelerating to near-zero velocity).
Now, figure how far that piece will move laterally (relative to the centerline of the stack) under gravitational acceleration.