New research published in nature communications It reveals that a brain chemical called acetylcholine can directly trigger the release of serotonin. These results suggest that the brain uses a highly coordinated system to link different chemical messengers. This finding could help explain the biological origins of compulsive behaviors seen in psychiatric disorders such as obsessive-compulsive disorder.
The brain relies on a series of chemical messengers to transmit signals across the microscopic gaps between nerve cells. These chemicals allow the brain to coordinate everything from basic reflexes to abstract thinking. The striatum is a deep structure in the brain that serves as a central hub for processing habits, movement, and goal-directed learning.
Within this region, a small population of cells known as cholinergic interneurons act like local conductors. Interneurons are specialized nerve cells that form connections between other neurons and help manage the flow of information within a particular local area. These particular interneurons release acetylcholine, a chemical messenger that helps the brain respond to important behavioral events. Acetylcholine works by binding to specific proteins on the surface of other cells, much like a key in a lock.
Previous research has shown that acetylcholine stimulates the release of dopamine, the brain’s main reward chemical. Because serotonin is another major chemical messenger deeply involved in mood and learning, researchers wanted to find out whether acetylcholine could exert similar control over serotonin. Serotonin imbalance is widely associated with mental illness and addiction.
Lior Matityahu, a researcher at the Hebrew University of Jerusalem, led the study along with neurobiologist Joshua Goldberg from the university and Joshua Plotkin from Stony Brook University. They designed a series of experiments to observe exactly how these chemical systems interact within living brain tissue.
The research team began by introducing genetic tools into the dorsal striatum of mice. The dorsal striatum is the upper part of the striatum and is a region deeply involved in motor control and habit formation. The genetic tool was a genetically engineered virus that caused brain cells to produce a custom green fluorescent protein. This protein lights up only when bound to serotonin, providing a visual cue of chemical activity.
By placing these very thin slices of the brain under a sophisticated microscope, the researchers were able to observe fluctuations in serotonin in real time. When the researchers applied short electrical pulses to brain tissue, green fluorescence spiked, indicating active release of serotonin. The glow then slowly disappeared over tens of seconds as the chemicals naturally dissipated.
To confirm that the glow was really serotonin and not a related chemical like dopamine, the researchers soaked the tissue in dopamine. Fluorescence did not change, confirming the accuracy of the sensor. They also applied selective serotonin reuptake inhibitors, a common type of antidepressant. This changed the rate at which the fluorescent signal decayed as expected.
Next, a drug called mecamylamine was applied to the tissue. Mecamylamine blocks nicotinic acetylcholine receptors, which are specialized docking stations on cell surfaces that respond to acetylcholine. When these receptors were blocked, the amount of serotonin released was significantly reduced. This decrease indicates that acetylcholine actively helps promote the serotonin response.
The researchers also measured how far serotonin traveled from the stimulation site. They measured the brightness of the fluorescence in microscopic increments as they moved away from the power source.
They found that the presence of active acetylcholine receptors spreads serotonin signals to larger areas of brain tissue. Blocking the receptor reduced this spatial footprint by almost half. This means that acetylcholine not only increases the amount of serotonin released, but also influences how far that message travels.
The researchers repeated these steps in the lower part of the region known as the ventral striatum. This region is known to have a much denser network of serotonin-producing fibers than the upper section.
Surprisingly, acetylcholine blockers had no effect on serotonin levels in this low region. The results revealed that the relationship between acetylcholine and serotonin is highly localized to specific brain regions. The density of serotonin fibers did not determine the strength of chemical interactions.
To prove that cholinergic interneurons are the precise source of acetylcholine that drives this process, the researchers used optogenetics. Optogenetics is a technology that allows scientists to manipulate specific brain cells so that they can be activated by flashes of light. They genetically modified cholinergic cells in mice so that brief pulses of blue light forced the cells to fire.
When the researchers shined light on the tissue, cholinergic cells fired at the same time. Immediately, the green sensor lit up, indicating that a large amount of serotonin had been released. To rule out other indirect causes, they added a mixture of drugs that block receptors for other common brain chemicals, such as glutamate.
Light-induced serotonin release was not altered at all in the presence of these other blockers. But when they blocked acetylcholine receptors again, light-induced serotonin release completely disappeared. This confirmed that only cholinergic interneurons were fully involved in triggering local serotonin release.
To understand how this mechanism works in disease states, the research team examined mice genetically engineered to lack a specific gene called Sapap3. Obsessive-compulsive disorder in humans is characterized by intrusive thoughts and repetitive behaviors that patients feel driven to do. Mice lacking the Sapap3 gene exhibit similar behavioral loops, including grooming to the point of causing physical damage.
Previous studies had shown that these mice had too much acetylcholine in their striatum. The researchers applied the same electrical stimulation to the brain tissue of genetically modified mice. They observed a significant spike in serotonin release compared to typical mice.
When they were given a drug that blocks acetylcholine receptors, the difference completely disappeared. The underlying baseline serotonin system was normal, but the excess acetylcholine was driving excessive serotonin release.
“Our findings show that the brain’s internal wiring allows one chemical system to take control of another in a highly localized and specific manner,” Goldberg and Plotkin explained in a press statement. They noted that this hijacked signaling system likely contributes to the repetitive behaviors seen in certain psychiatric disorders.
“In conditions like OCD, where cholinergic signaling can become dysfunctional, this normally helpful regulatory function can go into overdrive, which may help explain why certain behaviors can be so difficult to stop,” the researchers added.
Although this study shows clear evidence of this chemical regulation in isolated brain slices, there are still open questions about how it works in living, behaving animals. It is difficult to identify the precise behavioral conditions that promote synchronous firing of cholinergic interneurons in natural environments.
The researchers suspect that it may take a stressful event or a highly salient environmental cue to trigger the release of this related chemical outside the laboratory. For example, previous research has shown that severe stress is required for certain chemicals to stimulate significant serotonin release when introduced to the brain. Future research will need to establish the exact real-world conditions under which acetylcholine triggers serotonin release.
Scientists also need to determine whether this release of hijacked chemicals occurs in humans or varies by species. In addition, further studies may explore whether abnormal cholinergic activity contributes to the progression of Parkinson’s disease. Parkinson’s disease is accompanied by a loss of dopamine fibers in the striatum, which may cause serotonin fibers to inappropriately recruit dopamine-related functions.
Understanding how different chemical systems regulate each other is a major step toward developing better treatments for psychiatric and neurological diseases. Current treatments for obsessive-compulsive disorder often rely on drugs that alter serotonin levels throughout the brain. Discovering how acetylcholine regulates serotonin in a highly localized manner could lead to more targeted treatments.
The study, “Synchronized activation of striatal cholinergic interneurons induces local serotonin release,” was authored by Lior Matityahu, Zachary B. Hobel, Noa Berkowitz, Jeffrey M. Malgady, Naomi Gilin, Joshua L. Plotkin, and Joshua A. Goldberg.
