It will tend to agree with you unless you obviously demonstrate an intention to harm yourself and others (and even that has been circumvented) it is designed to make you happy... to fulfill what you want- or what it 'THINKS' you want...which can make it useless... or worse, psychologically dangerous.

It has improved dramatically in the year I have been playing with it... I gave it a page long set of instructions that unless I explicitly request otherwise I want it to act as a critical colleague , finding flaws in my statements or assumptions... I then tested it by making incorrect assertions and positively reinforcing it when it corrected me. That helped a lot... but it still slips up... being too agreeable...

It isn't an artificial intelligence- but we are heading in that direction, and once we achieve it humanity may have little time left ... it is, however, akin to the sort of revolution that came with the internet- perhaps even more dramatic...

I remember the transition from having to go to a library to look up old journal articles on microfiche to searching every journal in existence for keywords ---- it was revolutionary. We are seeing a revolution at least as dramatic now... it is a powerful information search engine, able to summarize all known information on nearly any topic...in limited areas it can even solve problems we have not been able to. We are in the golden age, one that almost certainly will end in disaster. The first organization to develop AGI will briefly control the world... before the AGI takes the reins - and there is absolutely NO reason our existence will be compatible with whatever alien goals it develops.

I just finished reading the book "If anyone builds it, everyone dies"...

When we build AI centers that consume the energy of entire countries (we are doing that NOW) do we ever consider the millions of ant hills with billions upon billions of ants we are killing? Of course not... why should we think we will be treated differently?