I’ve noticed the same thing with ChatGPT when I ask it to write some code. On first glance, it seems awesome and like it solved your question with ease. However, when you try to go forward and use it, or just look a bit more closely, you realize it has used classes or functions that don’t exist in the library it imports and it made up something. Or, it’s just wrong and not solving the question. This seems fine, but if you go three or four rounds trying to correct it with prompts, and it still gets it wrong, you start to look for other answers.

It’s often helpful for getting headed in the right direction and it has saved me some time, but it doesn’t write flawless code. It needs to be reviewed and corrected by a human.