Turning off your phone will keep it from howling at you (for both tests and The Actual End of the World). Assuming it's really off, and not just asleep. Most phones, you hold down the power button until a menu pops up giving you a choice of sleep, restart, and power off. You want power off.

Turning off the power should be enough to keep it from being used to track you. I don't buy into the paranoid belief that the NSA can turn on your phone and surveil you at any time. Something like that, across multiple vendors and product lines, would leave tracks.

You kinna violate the laws of physics, so a phone that could be turned on remotely would require the phone to never truly turn off, as it would be consuming power to keep the receiver on to listen for the naughty activation signal. You'd notice your battery discharging with the power off.

A Faraday cage is a little over the top, but then there are enough paranoids in the world that I'm sure you can buy some kind of metallic mesh bag for you phone somewhere. But keep in mind that a true Faraday cage is grounded, making it not exactly portable.

The truly paranoid open up their phones and remove both the battery and the SIM card. Pretty easy to do, actually, and probably more practical than a Faraday cage.

Good luck, ER.