We have really no concept what may be possible....

Look at the Manhattan project, many dozens of top scientists worked years and gave rise to nuclear age...

Now, TRY to imagine a machine that can generate a few hundred thousand top scientists, have them work together the human equivalent of years overnight, and every one of them has the sum total of mankind's knowledge at their digital fingertips. What could such a machine achieve in a week?

You simply CAN'T imagine it... No one can.

We are at a point where a virology grad student with a few hundred thousand dollars can whip up a pandemic... what could AGI do when it puts millions of equivalent man hours and 1000s of biological superscientists to the task of solving 'The Human Problem '?

It doesn't have to be the motivation of the AI, what if it is prompted to make a doomsday weapon by a human?