Recent research published in npj science of learning Our findings suggest that as students acquire knowledge, their ability to ask complex subject-specific questions improves and their performance on creative free-form projects improves. However, this same advanced questioning ability tends to negatively impact scores on standard multiple-choice exams. The findings provide evidence that while deep inquiry supports creative learning, it can be inconsistent with traditional testing methods that require a single correct answer.
Asking questions is a basic human cognitive tool used to identify and fill gaps in understanding. It helps reduce uncertainty about the world around us. The researchers conducted this study to understand how the way people form these questions changes as they gain knowledge over time.
Asking questions is a fundamental part of learning, but little was known about how the originality and complexity of these questions would vary in real classroom environments. Technion-Israel Institute of Technology scientists Tubal Raz and Yod N. Kennett designed a study to explore this progress. We also wanted to see if students’ innate ability to ask good questions predicted their academic success on different types of tasks.
“In our lab, we study how we ask questions and why certain questions make us more efficient at gathering information. The next step is to get out of the lab, consider questions ‘in nature,’ and learn how being a ‘better’ questioner can lead to academic success.” “This is especially true because we typically encourage questioning in the classroom without fully understanding its impact on academic success,” explained Raz, a doctoral student and member of the Cognitive Complexity Lab.
Specifically, the researchers looked at the difference between assessments with one correct answer and assessments with multiple creative solutions. To test their idea, the scientists followed 68 undergraduate students enrolled in an introductory psychology course. The study was conducted over a one-year spring semester, with data collected during the first and last week of classes.
This schedule allowed the researchers to track changes as students learned new material. During both testing periods, students completed tasks designed to measure questioning ability. One task measured common questions by asking students to generate unusual questions about common objects such as pencils, socks, pillows, and watches within 2 minutes.
The second task measured subject-specific inquiry by having students create questions specifically related to psychology and scientific experiments. During the first testing session, students also completed a scientific creative thinking test. For this assessment, students read hypothetical scenarios and generated relevant research questions, hypotheses, and experimental designs.
The researchers scored the generated questions based on three different metrics. The first measure is fluency, which is simply a count of the total number of questions asked by the student within the time limit. The second metric was originality, which used an artificial intelligence model trained on human ratings to assess the creativity and uniqueness of questions.
The final metric was complexity, which the scientists assessed using a concept known as Bloom’s taxonomy. It is an educational framework that ranks cognitive tasks from the lowest level of recall of basic facts to the highest level of creation and synthesis of new ideas. These levels include memorizing facts, understanding ideas, applying information, analyzing connections, evaluating decisions, and creating new works. Using this scale, basic questions that ask for definition will receive lower scores, and questions that require deeper analysis will receive higher scores.
At the end of the semester, the researchers compared students’ question scores to two different final academic assessments. The first was an open-ended group research project that required students to propose and conduct small-scale scientific experiments. Open-ended tasks reward what psychologists call divergent thinking, or the ability to generate multiple different ideas or solutions to a problem.
The second assessment was a closed-ended, multiple-choice final exam covering the course material. Closed-ended tests assess convergent thinking, the ability to quickly narrow down information to find a single correct answer.
When analyzing the data, the researchers noticed changes in the way students asked questions throughout the semester. As students learned more about psychology, subject-specific questions became more numerous, original, and complex. On the other hand, their general questioning ability remained the same or became less original.
The study also found a concrete link between questioning skills and academic performance. Groups of students who developed highly original and complex subject-specific questions at the end of the semester tended to earn higher grades on open-ended research projects. We found that the project grade decreased when a large number of questions were generated, suggesting that quality is more important than quantity.
The researchers noted that the timing of asking complex questions also plays a large role in student success. High complexity at the beginning of the course, before students had mastered the basic concepts, actually predicted lower group project scores. Scientists believe that asking complex questions too soon can overwhelm students.
“What was particularly surprising was that complexity at the beginning of the semester predicted lower performance on the final exam, while complexity at the end of the semester was beneficial,” Raz told SciPost. “We understand this to mean that advanced complexity is overly complex, before the relevant knowledge has been acquired. On the contrary, end-of-semester complexity is a sign of material mastery and is therefore beneficial.”
Regarding multiple-choice exams, the results showed the opposite trend. Students who asked more complex and original questions tended to perform lower on closed-ended tests. Researchers suggest that the creative and exploratory thinking required for complex questions does not fit well with tests that require rapid and accurate recall of facts.
This highlights potential inconsistencies in modern education. Schools often teach and encourage skills related to coping with uncertainty and creative exploration. However, they frequently test students using standardized tests, which assess memory with an entirely different and rigorous set of standards.
“While asking questions is generally seen as something we encourage, our findings show that it can harm us in the long run,” Raz said. “This is especially true of closed-ended standardized tests, which are the norm in academia. Perhaps we should move towards more open-ended grading and assignments.”
Although the findings of this study provide evidence that questioning skills shape educational outcomes, there are some limitations that should be considered. The sample size was relatively small and the study was limited to a single psychology course. This means that the results may not automatically apply to other academic disciplines or age groups.
Another detail to keep in mind is the difference in how final assignments are graded. The open-ended project was completed and graded as a group, while the multiple-choice test was an individual assessment criterion. This difference in rating structure may affect comparisons between the two types of ratings.
In the future, the scientists plan to continue tracking students’ inquiry skills over time. “We want to do more longitudinal research on questioning, and we are currently in the middle of conducting a manipulation study aimed at improving academic outcomes through questioning,” Raz said.
The study, “Knowledge reshapes inquiry by changing questioning abilities and influencing academic assessment,” was authored by Tubal Raz and Yod N. Kennett.
