For some turbopump driven engines that don't have to be restarted, an electrically activated pyrotechnic device is activated to spin up the turbines. They used this same technique to start a lot of jet engines back in the 1950s. It's simple and it works.

From this source:

Most modern rocket engines use high-speed turbine pumps to drive the propellants into the combustion chamber at a very fast rate. The turbopumps are initially spun up using a "starter", an electrically activated solid-fuel charge that spins up the drive turbine for the turbopumps.

For example, the Russian 11D33 engine uses this technique.

The more modern Fastrac engine used a different technique.

The engine is started with a hypergolic igniter - a starter fluid that spontaneously ignites when oxygen is fed to the chamber. Once the kerosene is injected, the engine is running. The propellants are then supplied to the gas generator and thrust chamber assembly for mixing and burning.

The engine uses a gas generator cycle, which burns a small amount of kerosene and oxygen to provide gas to drive the turbine and then exhausts the spent fuel. That's the same cycle that was used on the Saturn rockets, but much more simplistic than the Shuttle engine system.

Quite a few engine designs use a secondary combustion chamber - the gas generator mentioned above - to drive the turbopump. Most if not all rockets still pressurize the propellant tanks to a degree even with turbopumps, so feeding propellant to the gas generator isn't a problem. You open the valves to let propellant into the gas generator, ignite it, and use the combustion gases to drive the turbopump. The exhaust gases can then be dumped overboard or fed into the combustion chamber.

The problem with the pyrotechnic idea is that it doesn't work so well with restartable engines like the RL-10. That engine uses hydrogen peroxide to spin up the turbopumps. Highly concentrated peroxide is fed into a chamber with a catalyst, decomposes into steam, and uses that for the turbine.

Some interesting discussion on turbopumps can be found here.

You can probably find a lot of good info at the excellent Encyclopedia Astronautica website. It seems to be running very slow this morning or I would've given you more links.