You can certainly use an external device for the DATA
Things like email account, browser (with cache), personal file stores, password information etc. The firewall can live there, too, but all these things have to be called from the O/S, so far as I know.
You could maybe do it if you set the drive up on the ultimate destination computer but it wouldn't be a portable installation, merely a removable one.