Unless somebody locates an anti-matter mine, anti-matter is a lot like hydrogen: a way to store energy, not create it.

Fusion reactions, depending on what you're using for "fuel" (standard easy reaction uses deuterium and tritium) create varying amounts of neutrons, which can make surrounding materials radioactive, and sometimes also alters their structural properties. Materials for such reactors need to be carefully chosen.

There are fusion reactions that generate fewer neutrons (deuterium-He3 is one) but the reaction is much harder to create.

I am not bothered by the waste products. The half-lives of the waste products are far less than for fission materials, and the amounts involved in even hundreds of reactors are easily handled and stored in a few square miles. More importantly, the waste products have no use as bomb material.