Wouldn't intergalactic space be that empty...? I was looking at that Atlas of the Universe site Marcia pointed to earlier and there are some spaces as large as 250 million LYs across, like the Bootes Void (next to the Bootes Supercluster of Galaxies).

250m LYs... Isn't that like 1.4x10^23 miles (much more than billion billion miles)?

Sure, a billion billion would be 2.0x10^18 miles in diameter sphere, you could probably fit lots of them in this Void you are talking about. Problem is why do you think this area has nothing in it? Is it only because called Bootes Void?? It could be called that because some cloud is blocking the view, I don't know?
Assuming the label is correct. What mechanism would create this large volume of space which not only appears to lack anything in it but actually has nothing the size of a piece of dust in it?
You are living in a Universe which has huge objects (stars) which spewing forth massive quantities of matter (solar wind), over billions of years- in which the bigger ones explode violently (throwing mass of millions of Earths into surrrounding space) in a relatively short period of time (tens-hundreds millions of years).
Now, it's true that in comparison to these stars, space is huge (really, really big:).
If you claimed that there wasn't enough matter in this void to make a couple dozen stars, this might be believable, maybe. If you said the chance of hitting a piece of dust when zooming through the Bootes Void in your interstellar spaceship was nearly "impossible" that could be believable [but that's mainly due to the fact your spaceship even though it could be the size of a planet, is really, really small in comparison to the surrounding area of space- there could be a million Moon sized objects in this area and still the chance of hitting them would nearly impossible],
I would say the chance of there being not one grain of dust in a 2.0x10^18 miles in diameter sphere anywhere is the Bootes Void is unlikely. I would say it's more likely that you would find some alien ship or an answered to a New York Times crossword puzzle.