Traditional intelligence and literacy tests can be fundamentally flawed because they force test takers to select a single answer rather than allowing them to express their level of confidence over a variety of options. When people are given financial incentives and can distribute their answers based on confidence, women actually score higher than men. The study was published in the Journal of Political Economy.
For decades, psychologists and economists have used multiple-choice tests to measure cognitive abilities. These ratings score answers as strictly correct or incorrect. Glenn W. Harrison of Georgia State University, Don Ross of University College Cork, and J. Todd Swarthout of Georgia State University suspected that this format missed important elements of human cognition. Knowing how strongly to believe in an answer is a skill in itself.
Researchers point out that standard formats force people to hide their thought processes. If someone is reasonably confident in their answer but recognizes that they could still be wrong, a rigid format won’t capture that nuance. The test format demands absolute certainty, even if one has a healthy skepticism.
To address this, the team looked at the Raven Advanced Progressive Matrices test. In this assessment, candidates are presented with a grid of shapes with one piece missing and asked to identify the pattern. It is widely used to measure fluid intelligence, which is the ability to solve new logical problems without relying on prior knowledge.
“Measuring intelligence requires identifying and measuring an individual’s subjective confidence that their answers to test questions are correct,” the researchers wrote. They noted that existing tests do not fully achieve this goal.
The standard version of this puzzle allows test takers unlimited time and there is no financial incentive. The researchers created a computerized version that offered monetary rewards for correct answers. They divided participants into different groups to test how the structure of the task affected participants’ performance.
In the baseline group, participants used the classic version for a flat fee of $5. In another group, participants received a reward based on their correct answer, but were still forced to choose only one answer. A third group experienced a fundamentally different test structure.
These participants were given 80 digital tokens to allocate to eight possible answers. If you are completely sure, you can place all 80 tokens on one choice and get rewards of up to $2 per puzzle. If you are unsure, you can spread your tokens across multiple possible answers to guarantee a smaller payout.
This token system measures what researchers call trustworthiness. In this context, confidence does not mean optimism. It refers to the accuracy of a person’s beliefs. A person who places 10 tokens on every answer will not know which shape is correct, so he can safely avoid the risk.
When we combined financial incentives with the ability to express varying degrees of trust, the results changed dramatically. In the traditional format, female participants had lower scores than male participants. When participants were able to allocate tokens based on their confidence, women outperformed men.
The data showed that female participants were better at calculating the risk of their answers and distributing tokens efficiently. Knowing when you are unsure is a core part of cognition. Researchers believe this risk assessment is a fundamental element of fluid intelligence.
The researchers also changed the order of the puzzles. Common tests start with easy puzzles and gradually progress to more difficult questions. Researchers call this sequence a structured progression, meaning it’s an environmental cue that helps humans think.
When the researchers randomized the order of the puzzles to vary their difficulty, overall performance decreased. The performance gap between the group forced to choose one answer and the group allowed to use tokens was even wider. This confirms that the ability to express uncertainty has a clear cognitive advantage when faced with unpredictable problems.
This finding regarding gender prompted researchers to reconsider other areas where men appear to have an advantage. They looked at research on competitiveness. Past behavioral research suggests that women shy away from competitive environments such as workplace tournaments and favor flat payment schedules.
Researchers replicated these experiments using a token system and found that women made mathematically correct risk management choices. Participants had to solve logic problems within a time limit and had to choose either a guaranteed payout for each correct answer or a tournament format where only the top performers received large prizes.
Men were more likely to choose competitive tournaments, even if it meant financial loss. It turns out that men are overly optimistic about their chances of winning. Women achieved better financial outcomes because they assessed risks more accurately and chose safer compensation structures.
The team also considered financial literacy tests. Standard research reports that women choose the “I don’t know” option much more often than men when asked financial questions. For this reason, women’s financial literacy is considered to be low.
The researchers asked participants standard questions about calculating purchasing power based on interest rates and inflation rates. When the researchers allowed participants to use tokens to answer questions, they found that women were more open about not being completely sure. Their actual knowledge bias was small and not statistically significant.
Many women distributed tokens widely. This means they were aware of their lack of accurate knowledge and guarded their bets accordingly. This behavior shows an intelligent awareness of uncertainty. People who know they’re guessing are more likely to seek out a financial advisor or textbook to learn the correct answer.
Someone who puts all their tokens on a wildly inaccurate answer represents a much greater risk. The researchers pointed out that these people were completely confident in their false knowledge. These are the people most likely to make devastating financial decisions without outside help.
The authors note that research findings on motivation may include variables that are difficult to separate. Participants may bring personal motivations to the laboratory that interact with the financial incentives provided by the experimenter.
Future research could attempt to separate these personal drives from monetary rewards to see how they influence token distribution. The researchers also plan to further examine data that suggests Black participants similarly perform significantly better when they are allowed to express their confidence through a token system.
The study, “Gender, Self-Confidence, Intelligence, Competitiveness, and Mistakes in Literacy,” is by authors Glenn W. Harrison, Don Ross, and J. Todd Swarsoth.
