For patient after patient seeking to cure chronic back pain, the experience is years of frustration. Whether they strive to treat their aching muscles, bones and ligaments through physical therapy, massage or rounds of surgery, relief is often elusive - if the pain has not been made even worse. Now a new working hypothesis explains why: persistent back pain with no obvious mechanical source does not always result from tissue damage. Instead, that pain is generated by the central nervous system (CNS) and lives within the brain itself.

I caught my first whiff of this news about eight years ago, when I was starting the research for a book about the back-pain industry. My interest was both personal and professional: I'd been dealing with a cranky lower back and hip for a couple of decades, and things were only getting worse. Over the years, I had tried most of what is called 'conservative treatment' such as physical therapy and injections. To date, it had been a deeply unsatisfying journey.

Like most people, I was convinced that the problem was structural: something had gone wrong with my skeleton, and a surgeon could make it right. When a neuroscientist I was interviewing riffed on the classic lyric from My Fair Lady, intoning: 'The reign of pain is mostly in the brain,' I was not amused. I assumed that he meant that my pain was, somehow, not real. It was real, I assured him, pointing to the precise location, which was a full yard south of my cranium.

Like practically everyone I knew with back pain, I wanted to have a spinal MRI, the imaging test that employs a 10-ft-wide donut-shaped magnet and radio waves to look at bones and soft tissues inside the body. When the radiologist's note identified 'degenerative disc disease', a couple of herniated discs, and several bone spurs, I got the idea that my spine was on the verge of disintegrating, and needed the immediate attention of a spine surgeon, whom I hoped could shore up what was left of it.

Months would pass before I understood that multiple studies, dating back to the early 1990s, evaluating the usefulness of spinal imaging, had shown that people who did not have even a hint of lower-back pain exhibited the same nasty artefacts as those who were incapacitated. Imaging could help rule out certain conditions, including spinal tumours, infection, fractures and a condition called cauda equina syndrome, in which case the patient loses control of the bowel or bladder, but those diagnoses were very rare. In general, the correlation between symptoms and imaging was poor, and yet tens of thousands of spinal MRIs were ordered every year in the United States, the United Kingdom and Australia.

Very often, the next stop was surgery. For certain conditions, such as a recently herniated disc that is pressing on a spinal nerve root, resulting in leg pain or numbness coupled with progressive weakness, or foot drop, a nerve decompression can relieve the pain. The problem is that all surgeries carry risks, and substantial time and effort is required for rehabilitation. After a year, studies show, the outcomes of patients who opt for surgery and those who don't are approximately the same.

More invasive surgeries carry greater risks. Lumbar spinal fusion - surgery meant to permanently anchor two or more vertebrae together, eliminating any movement between them - is recognised as particularly hazardous. Even when the vertebral bones fuse properly, patients often do not get relief from the pain that sent them to the operating room. Beyond that, fusion surgery often results in 'adjacent segment deterioration', requiring a revision procedure.

In the US, about 80,000 spine procedures fail each year, and one in five patients returns for another operation. Typically, second, third and fourth attempts have an even lower chance of success, and patients continue to require painkillers over the long term. Even the procedures that surgeons deem successful, because the bones fuse and look perfect on a scan, are often unhelpful to patients. In one study, two years after spinal fusion, patients' pain had barely been reduced by half, and most patients continued to use painkillers. Given such unimpressive outcomes, the cost of treating back pain is unacceptably high. Spine surgery costs a fortune, but other approaches, including epidural steroid injections, physical therapy and chiropractic treatment, are also expensive.

Including direct medical expenses and indirect expenses such as lost earnings, spine care costs the US about $100 billion a year. In the UK, that tab is about £10.6 billion (c$13.6 billion). In Australia, it's A$1.2 billion (c$950 million). Many of these costs derive from the loss of productivity, as people take time off from work. Others result from the devastation wrought by addiction to prescription opioids. In Australia, between 1992 and 2012, prescription opioid dispensing increased 15-fold, and the cost to the Australian government increased more than 32-fold.

Pain falls into four basic categories. There's nociceptive pain, the normally short-lived kind you feel when you accidentally slam your finger in the car door. There's inflammatory pain, a response to damage or infection, resulting in a rush of small proteins called inflammatory cytokines to the site of the casualty. That pain has a habit of spreading, to affect everything in the vicinity. Beyond that, there's neuropathic pain, known as 'radiculopathy'. It results, usually, from an insult to a nerve, culminating in burning, tingling or shock-like sensations that travel the length of the affected nerve (sciatic pain is a good example).

When any of those three types of pain sticks around long after the inciting injury has healed - or in the absence of any noxious stimulus - the patient can be said to be suffering from 'central sensitisation'. Central sensitisation is a condition in which even mild injury can lead to a hyperactive and persistent response from the central nervous system.

The CNS includes the dorsal root ganglia, containing the cell bodies of sensory neurons that allow information to travel from the peripheral sites to the spinal cord and the brain. The peripheral nervous system (PNS) consists of the nerves beyond the brain and the spinal cord, serving all parts of the body that the CNS does not, comprising roughly 40 miles of nerve fibres, if they were laid out, end to end.

'As pain becomes more centralised,' wrote Clifford Woolf, a neurologist and neurobiologist at Harvard Medical School, 'it becomes increasingly more difficult and less relevant to identify the initial source.'

More than three centuries ago, the French philosopher, mathematician and natural scientist René Descartes advanced the heretical idea that pain was not a punishment from God, nor a test or trial to be endured, for which prayer was the only intervention. Instead, he said, pain existed as a mechanical response to physical damage. His work Treatise of Man would not be published until after he died (some say because he feared persecution by Christian authorities, for whom the threat of pain was a useful recruitment tool). But when the volume finally emerged, Descartes posited the existence of 'hollow tubules' that allowed messages he described as 'animal spirits' to travel on a dedicated somatosensory pathway, from the afflicted site to the brain. The intensity of pain, Descartes believed, rose with the severity of tissue damage. In the absence of such damage - a shattered bone, a wound, a burn - pain ought not to exist.

But of course, it did.

In the mid-1960s, two scientists, the Canadian psychologist Ronald Melzack and the British neurobiologist Patrick Wall, both then working at the Massachusetts Institute of Technology, set out to answer the question of how pain could persist in the absence of an injury. It was mostly guesswork. It would be years before neuroimaging would allow them to view the structure of a living human brain.

In their landmark article 'Pain Mechanisms: A New Theory' (1965), published in the journal Science, they considered the pathophysiology of chronic pain, based on post-mortem studies, surgical notes, neurofeedback and patients' reports of their experiences. Ultimately, the two scientists described the 'gate control theory of pain', hypothesising that nerve cells in the spinal cord acted as gates, flipping open to allow pain messages to pass through, or closing to prevent such messages from reaching the brain. At times, the scientists posited, the gates became stuck in the open position, allowing pain messages to flow unabated. It was that last little bit - the notion that messages would travel unceasingly, from the PNS to the CNS - that sparked Clifford Woolf's interest in how pain was generated, and how it could be silenced.

In 1983, Woolf was a young anaesthesiologist with a PhD in neurobiology. As a post-doc, he had worked in Wall's laboratory, which by that time had moved to University College London. There he observed post-mortem cellular and molecular changes in brain tissue in subjects who had suffered from chronic pain when they were alive.

Later, he had access to high-powered neuroimaging in the form of functional magnetic resonance imaging, or fMRI. This neuroimaging could measure changes in the brain's blood flow, volume, oxygen or glucose mechanism, allowing Woolf to see how the brain responded to pain in a living subject. Woolf thus began to explore the many ways in which neurons in different brain regions communicate; how they form a greater number of synapses, linking regions that are not normally hot-wired to work in concert; and how those neural changes lead to the perception of pain. He saw that the regions of the brain that responded to acute, experimental pain were different from the regions that were involved in chronic pain. Over the next three decades, Woolf explored the relationship between specific gene phenotypes and chronic pain, looking for potential targets for drug therapy. It would be slow-going, in part because pharmaceutical companies were profitably selling opioid analgesics. When, in the mid-2000s, the efficacy and safety of opioids began to be questioned, Woolf's work took on new vigour.

By then, the neuroscientist A Vania Apkarian, a professor of physiology, anaesthesiology and physical medicine at Northwestern University's Feinberg School of Medicine in Chicago, was well into his own study of what happens to specific regions of the brain under the onslaught of chronic pain. For two decades, in his provocatively named Pain and Passions Lab, where his group works with both rodents and humans, Apkarian's focus has been on pain's cognitive consequences.

'When we started this research in 1999,' Apkarian said, 'very few people believed that pain was more than nerves sending a signal into one part of the brain.' With grants from the National Institutes of Neurological Disorders and Stroke - part of the National Institutes of Health (NIH), Apkarian demonstrated that instead of simply responding to externally generated discomfort, under siege the brain itself would begin to generate the pain. 'The official definition of chronic pain,' Apkarian wrote in the journal Pain Management, 'is that it persists past the completion of injury-related healing processes.'

Brain activity in subjects with chronic pain was different from the nociception (perception of harm) evident in patients with experimentally induced pain, for instance, a hot poker placed on a sensitive part of the arm. While nociceptive-provoked pain activated primarily sensory regions - the ones that would cause you to yank your arm out of harm's way - Apkarian's group observed that chronic pain activated the prefrontal cortex and the limbic regions of the brain. The prefrontal cortex dictates higher-level thinking, including goal-setting and decision-making, while the limbic regions, including the hippocampus and the nucleus accumbens, govern memory, motivation and pleasure.

In a revelation that set the international media abuzz, Apkarian's group found that the anatomy of the human brain in patients who suffered from chronic pain was abnormal. In those who had suffered for five years, both the hippocampus and the prefrontal cortex were structurally transformed, sacrificing 5 to 11 per cent of their grey matter density. That was important because the prefrontal cortex, in concert with the hippocampus, dictates how optimistic or depressed patients feel about their prospects, how well they can cope and make decisions about treatment. There's still a great deal of work to do in this area but, wrote Apkarian, 'the concept is that the continued, unrelenting pain impacts limbic structures in the brain that in turn entrain the cortex to reflect both the suffering and coping strategies that develop in chronic-pain patients.'

Subsequently, more than 50 studies, most from other investigators, have documented regional decreases in grey matter density, volume or thickness. Beyond that, the neuronal network of the remaining grey matter is rearranged, in patterns that are specific to chronic-pain conditions. That means, for instance, that the brain of a person with lower-back pain will look different from that of a person with a repetitive-stress injury. It's still unknown, Apkarian adds, 'the extent to which the observed brain reorganisation is a causal response to the condition or a predisposing factor'.

At least one other aspect of brain activity is transformed in people with chronic pain. The nucleus accumbens' role is to monitor the brain's reward circuit, thus governing feelings of pleasure and motivation. According to the scientists at Stanford University who studied the nucleus accumbens in mice, the brain structure is involved in 'computing the behavioural strategies that prompt us to seek out or avoid things that can affect our survival '.

In chronic-pain patients, Apkarian's researchers observed, the nucleus accumbens and the medial prefrontal cortex (which, once again, mediates decision-making) become unusually chatty.

This much-enhanced level of communication between the two regions represents a profound reorganisation of neuronal connections. It's possible that this chattiness might correspond with chronic-pain patients' reluctance to follow self-care protocols such as exercise. The heightened communication might also drive a tendency to select interventions that seem 'easy', but often are not, and in hindsight might be damaging. It's easy to see why that would be. In the absence of any sense that hard work will be rewarded, or that things will get better, it's difficult to summon the energy to follow through.

There's much debate about what sparks the complex neurobiological sequence that results in central sensitisation and, puzzlingly, why this occurs in some people but not in others. Both environmental and hereditary factors are likely involved, Woolf has found. His lab at Boston Children's Hospital is focused on identifying human genes with a link to 'dramatic familial pain phenotypes' - extreme pain disorders that run in families - and could offer insight into more typical chronic-pain conditions.

Woolf's lab has identified a haplotype (an inherited DNA variation) that matches up with high sensitivity to pain from sciatica, osteoarthritis and lumbar disc degeneration. 'It is becoming clearer,' observed Woolf and his co-authors in a paper in the Journal of Pain in 2016, 'that the development of chronic low back pain may occur because of a combination of genetically based susceptibility factors as well as local pathological risk factors.' In other words, whether your intervertebral disc is going to rupture has much to do with your physiology and physical condition. But how much it's going to bother you, and for how long, is likely to be a matter of genetic predisposition. Woolf's group is currently working on a process that allows them to genetically reprogram skin cells to turn into pain-sensing nerve cells, which can be studied in a petri dish. Woolf hopes that once the nerve cells are established, they will be valuable for pre-screening patients to see who has the physical and biochemical traits that make it likely they will develop chronic pain.

Scientists now recognise that there are gene variations that 'wire' certain people for suffering, and variations that leave others unscathed. The enzyme catechol-O-methyltransferase (COMT) is essential to the production of several stress-related neurotransmitters, including dopamine, norepinephrine and epinephrine, each of which is involved in modulating mood and cognition. One variant of COMT produces a slower-acting enzyme that leaves a flood of dopamine intact within the synapse, a condition that is associated with a very high level of stress. People who inherit that slow-acting COMT variant can be especially emotional and pain-sensitive. Intriguingly, unless they choose more even-keeled partners, it's likely that their progeny will share their tendency towards pain sensitivity.

Research on this topic is still sparse, but scientists suspect that this shared genetic background, rather than any identifiable pathology, is the reason that pain hypersensitivity often runs in families, and it's common to hear stories of multiple family members who suffer from similar chronic back-pain conditions. Woolf's lab found that the gene GCH1 controls the production of the chemical BH4, a precursor of serotonin. Those without the protective variety of BH4 feel a great deal of pain, but about 15 per cent of the population carry the 'bulletproof' version of the GCH1 gene, which leaves them remarkably impervious to pain. At least one study has shown that patients with this pain-busting biology recover much more successfully from spine surgery than their ultrasensitive brethren.

An excellent article on Mosaic, a digital magazine produced by the Wellcome Trust in the UK, quotes professor Irene Tracey, head of the Nuffield Department of Clinical Neurosciences at the University of Oxford. The most significant change in evaluating chronic pain, observes Tracey, is the understanding that chronic pain is a different animal from nociceptive pain. 'We always thought of it as acute pain that just goes on and on - and if chronic pain is just a continuation of acute pain, let's fix the thing that caused the acute, and the chronic should go away,' she said. 'That has spectacularly failed. Now we think of chronic pain as a shift to another place, with different mechanisms, such as changes in genetic expression, chemical release, neurophysiology and wiring. We've got all these completely new ways of thinking about chronic pain. That's the paradigm shift in the pain field.'

One explanation for the phenomenon of central sensitisation is that when an injury has afflicted some aspect of the peripheral nervous system, neurons in the central nervous system can also become agitated. This bumped-up signal-to-noise ratio can result in increased activation of calcium channels, the molecular pores that govern the flow of calcium ions across the cell membrane. This boosts the number of chemical messages travelling between nerve cells. Certain vulnerable neurons can also get a dose of NMDA (N-methyl-D-aspartate), opening more calcium channels, and sending even more messages whirling around the CNS. One class of drug now being evaluated in laboratory studies, an NMDA antagonist, could one day be useful in treating central sensitisation by blocking the excess 'chatter' that flies between overwhelmed neurons.

A final hypothesis suggests that central sensitisation reflects a type of neurobiological learning disorder: essentially, the brain is misinterpreting pain messages, which are never dismissed, but continue to travel endlessly from PNS to CNS, leaving the brain unable to set a new course. Some researchers have remarked that central sensitisation can be understood as a form of classical conditioning: just as the Russian physiologist Ivan Pavlov conditioned his dogs to salivate when a bell was paired with food, and then to salivate when the bell alone was heard, the body that has learned to experience pain in response to insult or injury continues to experience it in response to inconsequential stimuli.

Recent research has revealed what many patients know all too well: chronic back pain is often accompanied by other types of pain, including headaches, other musculoskeletal disorders, temporomandibular joint disorders, fibromyalgia, irritable bowel syndrome and chronic fatigue syndrome. People who develop central sensitisation can also find light, noise or smells unusually disturbing, or display hypervigilance. Anxiety, stress and depression are problems for an estimated 30 to 45 per cent of patients with chronic back pain, and an even higher percentage of back-pain patients who experienced early childhood adversity.

One would think that opioid analgesics would be helpful in calming an agitated and dysregulated nervous system, but this premise has been debunked. In fact, to the contrary, long-term use of opioid analgesics, especially high-dose extended release drugs such as OxyContin and methadone, have been associated with the development of a particular type of central sensitisation called 'opioid-induced hyperalgesia', resulting in abnormal sensitivity to pain.

Despite Apkarian and Woolf's decades-long efforts, it is likely to be years before physicians can use targeted compounds to treat the neurobiological mechanisms that lead to central sensitisation. 'A huge clinical challenge remains to identify these mechanisms from the individual pain patient phenotype and to then target the molecular mechanism with a specific treatment,' Woolf says.

It's easy to see why progress has been slow: to make money in medicine, the common wisdom holds that it's necessary to incise, prescribe, implant or inject. Pain science, dealing with complex neurological function, doesn't readily allow for those kinds of interventions.

Historically, NIH has dedicated only 1 per cent of its research budget to pain science-related investigations. And until recently, painkiller manufacturers saw no reason to invest in very speculative research, thus unwisely diluting their shareholders' earnings. But with opioid treatment on the skids, and profits sinking, finding new therapeutic targets is suddenly very attractive.

Drug targets are still on the horizon. But many pain psychologists and rehab specialists believe that central sensitisation can be successfully treated with a combination of cognitive behavioural therapy (CBT) and graded, non-pain-contingent exercise. The good news is that several labs have now shown that, after a patient's pain has been properly treated, three months of CBT can substantially reverse pain-induced changes in grey matter.

While researching my book Crooked: Outwitting the Back Pain Industry and Getting on the Road to Recovery (2017), I listened to hundreds of back-pain patients explain their chronic pain: they spoke of degenerative disc disease, herniated discs, pinched nerves, sciatica, spondylolisthesis, scoliosis and spinal stenosis. But I never encountered a single patient who described his or her struggle in terms of central sensitisation, or had heard of terms commonly used in behavioural psychology, such as 'guarding' (walking with an attention-getting limp) or 'fear-avoidant behaviour' (eschewing activities that might tax back muscles, thereby making them progressively weaker) or 'pain catastrophising' (ruminating over how severe the condition is likely to become, ruining any hope of a productive future).

As a practice, CBT provides graded exposure to feared stimuli. That means if you're afraid of spiders or flying, you dull your terror by facing down the arachnid or the take-off and landing, safely and repetitively. With back-pain patients, the fear of pain might seem life-threatening. This idea is often implanted by healthcare practitioners who caution patients, unnecessarily, to 'be careful,' and to 'spare their backs'. The job is to let patients know that, in the case of chronic back pain, hurt does not typically mean harm; that in fact, if you wish to get past the terror, you are going to have to follow pain deep into its lair.

Depending on the kind of chronic-pain rehab programme you enter, you might find yourself hauling around a plastic milk-crate filled with steel bricks, or engaging in water aerobics, or doing reps with free weights, or pushing an industrial sled (a heavy aluminium rectangle equipped with sliders on its base and sturdy handles on either end) or playing a game of beach volleyball, all under the close but generally unsympathetic supervision of someone who understands how bodies work and has seen it all before. The grimacing, the groaning, the odd body mechanics - all of them must go. Strengthening must follow. And when it does, the patient is rewarded with a sense of mastery over his or her own body, and no longer feels like a helpless victim.

About the author

Cathryn Jakobson Ramin is an independent journalist, essayist, and author. Her work has appeared in The New York Times, The New Yorker, Scientific American, and NPR, among others. Her latest book is Crooked: Outwitting the Back Pain Industry and Getting on the Road to Recovery (2017). She lives in California.