Viewports
I know you don't want to put your annotations in model space, since you'd have to create a different UCS for each view, but if you do them on a different layer for each view, you can command a layer to be visible only in a certain viewport by freezing or thawing it only in that floating viewport. Doesn't affect visibility in the other viewports.
A kludge, I know.