OpenAI's chatbot ChatGPT has passed a standard test for creativity with flying colors. Its results matched the top 1% of original creative human thinkers, revealed a new study by the University of Montana (UM).
Using the Torrance Tests of Creative Thinking (TTCT), Dr. Erik Guzik, assistant clinical professor in UM's College of Business, together with his team, administered the tool to OpenAI's ChatGPT. The Chat Generative Pre-Trained Transformer offered up eight responses.
Twenty-four University of Montana students also took the test, and all these scores were compared with 2,700 students across the US who took the TTCT in in 2016.
Not only were the chatbot's responses just as creative as those of the topmost creative humans who took the test, but the artificial intelligence-powered language model did better than the majority of all students.
TTCT uses three picture-based exercises to assess the following mental characteristics:
Fluency
Originality
Elaboration
Resistance to closure (ability to tolerate ambiguity and come up with a creative response)
Abstractness of titles
Creative strengths checklist
It offers two different assessments - verbal and figural. ChatGPT scored in the top percent for fluency and for originality. It only ceded its position somewhat on flexibility which measures a variety of different types and categories of ideas.
"We had all been exploring with ChatGPT, and we noticed it had been doing some interesting things that we didn’t expect. Some of the responses were novel and surprising. That’s when we decided to put it to the test to see how creative it really is... For ChatGPT and GPT-4, we showed for the first time that it performs in the top 1% for originality. That was new," said Guzik.
Guzik was cited as voicing surprise that ChatGPT excelled at “generating original ideas” – typically attributed to human imagination. Analyzing previous research on GPT-3, he said the chatbot had not done as well when it came to tasks requiring original thinking. Tremendous strides in AI have allowed the more advanced GPT-4 to up the game.