Accelleration to near-light speed causes time dilation. It's not the physical separation.

if you accelarate one astronaut and stop him at a new destination, then accellerate the other and also bring him to the new destination, both clocks are in synch (but out of synch with Earth). If you take one away then bring him back, they are out of synch because only one party experienced the acceleration.

It's about the integrated time-history of acceleration.

Gravitational acceleration also has a small time effect (e.g. gps clocks need to be corrected vs earth surface clocks over long periods of time)