Don't forget the other reason RTGs are so tough
The military also needed the nuclear bombs in their missiles to survive atmospheric entry as they went into space before coming down on a terrestrial target. Thus the nuclear material had to survive extreme conditions in order to blow up at its destination and not before then.
RTG casings are a result of this.