Last year I was invited to give a speech at a scientific conference that examined the role of the microbiome in autoimmune disease – concepts I describe in this Current Opinion in Rheumatology journal article. Our research team had also developed an immunostimulatory treatment for autoimmune disease based off concepts in the paper. Doctors in at least a dozen countries were using the treatment with their patients, often with success.

I didn’t discuss this treatment in my speech, but made the following statement during the last twenty seconds of the talk: “We have developed an immunostimulatory treatment that patients are using in conjunction with their doctors. If you’re interested in any of our case histories find me later.”

After this remark, several of the researchers in the audience looked unhappy. Later the session chair clarified the reason (which I suspected): our treatment had not been validated by a randomized controlled trial (RCT) and therefore I should not have mentioned it. In other words, patient case histories (obtained from their respective physicians) should have had no place at the meeting.

For this reason, apparently, few researchers approached me after the speech. However, a line of doctors and patient advocates did begin to form near my chair. They were interested in the case histories and I gave them some basic information.

My experience at the meeting provides evidence of a divide I note with increasing frequency. There is a growing gap between members of the medical community who require that formal trials be conducted before even discussing a treatment and researchers/doctors who are willing to incorporate patient reported feedback into their recommendations.

Perhaps I see this situation with clarity because I am both a researcher and a patient. I obtained a PhD in microbiology based on formal, evidence-based training. I learned a great deal in the classroom but no part of this education involved interacting with patients.

However, in 2004, I became ill with ME/CFS, a poorly understood but extremely serious illness. My experience with the illness is typical. In 2004 few mainstream researchers believed the illness was real, largely because standard blood tests do not pick up signs of the disease. I was subsequently told that my symptoms were “all in my head” and referred to many psychiatrists (useless!). This eventually forced me into the realm of “alternative” medicine where I became my own advocate and effectively my own doctor.

Consider where this situation placed me in 2010. At the time my PhD education strongly encouraged me to view disease through the lens of evidence-based medicine – an approach to scientific or medical inquiry that emphasizes data obtained from randomized controlled trials (RCTs) or related formal trials. Doctors and researchers are urged to base their recommendations and insights on these “strong” sources of evidence. In contrast, evidence-based medicine posits that expert opinion (derived from experience in the clinic) and/or case controlled trials (studies that observe patient behavior) are “weak” forms of evidence that should play a lesser role in informing clinical care or research initiatives. Patient-reported feedback is not intended to inform strict evidence-based practice at all. In fact, RCTs are considered to be the gold standard of an “evidence-based” approach largely because they help eliminate “biases” associated with patient feedback.

Around the same time, however, efforts to address my own health issues had led me to become a member of several online forums for patients with ME/CFS and other related illnesses. In addition, I led a support group meeting for patients with inflammatory disease. This meant that although my formal training did not encourage it, I was frequently exposed to patient feedback. The patients I knew were continually testing myriad treatment and lifestyle adaptations in the hopes of improving their health. Their reactions and experiences with these interventions were not captured in formal trials.

My PhD thesis focused on the role of the microbiome in autoimmune disease and, over time, I naturally began to develop hypotheses (formal insights) on the topic. These hypotheses were deeply rooted in the scientific literature and evidence-based trials (I read dozens of journal articles a week). Yet I must admit that they were additionally informed by insights I gained from regularly interacting with patients.

For example, in 2008 I presented a poster at the “Days of Molecular Medicine” conference in Karolinska, Sweden. The talk examined the higher prevalence of autoimmune disease in women, a reality documented in formal trials. While doing research for the talk, I realized that the symptom “brain fog” was repeatedly described in women suffering from autoimmune conditions. But it was often poorly defined. “Brain fog” was also often made to sound fairly innocuous, like a minor inconvenience to the patient.

I read papers on the topic, but also decided to interview several women with ME/CFS about their “brain fog” symptoms. By that time in Europe ME/CFS was considered to be an autoimmune disease, so these women qualified as volunteers.

The following are excerpts from three interviews (segments of these were published in a subsequent poster):

Emma, age 52, ME/CFS: “Sometimes when people are talking, it is as if with some words they are speaking in a foreign language. I hear the words, but they don’t make sense. It feels like a form of age-related deafness, having to do with the inability to comprehend due to the loss of stereophonic sound (if that’s what it is called–I really can’t remember), but seems to be mainly word specific…Once I couldn’t remember my last name when the new receptionist in my Doctor’s office asked me for it; I feigned deafness while I got out my checkbook to look up my name.

Cassandra, age 21, ME/CFS: “At my worst, I was unable to watch television (even if I could stand to watch it which was rare) I couldn’t process what was going on or follow a storyline or, for example, why one character would respond to another character with a particular phrase…I could barely read words, or I could a little – but how the word I had read related to the last one I could not get my head around. I also had difficulty understanding people who would speak to me.. their sentences were like a bunch of words jumbled in the air.. I would pick up about every fourth or fifth word, and try to piece them together, like when you’re learning a new language. It got to a point where I was lying down flat on my back every day trying to focus on my roof and using my hands to block out light because the more color I saw, or the more clutter, the harder my brain had to work just to see.

Susan, age 36, ME/CFS: “In 2001 I spent eight months staring at a wall. I went from being a company executive to barely being able to read above a fourth grade level. I couldn’t focus/concentrate on anything. My short-term memory turned into about 15 seconds (my nickname was “Memento Girl” after the movie “Memento.”) I’d have to write notes immediately after someone told me something or I’d forget. I also had to write myself notes to remind me where I was driving because I’d forget and get lost. Ultimately, I had to stop driving because I’d also forget that the notes were on the seat next to me and it’d take me ages to find my way home.”

These interviews blew me away. I was surprised that what these women were describing was classified as “brain fog.” Their actual symptoms seemed much more profound than the word implies. My thinking changed in two significant ways.

First, I looked at myself. I had/have ME/CFS, but had never thought too seriously about my own cognition. At the time I was extremely focused on my profound flu-like symptoms. I was always a strong student, which seemed to suggest my brain wasn’t “fogged.” Now I began to wonder about some of my tendencies that others labelled personality traits. I could never, ever, remember people’s names. It was a running joke among my friends but often I was secretly concerned about the issue. In an effort to compensate for my shortcomings, I would take a person’s name, think of something it rhymed with, and then think of a movie/TV character with the name. I would repeat these words over and over in my head in an effort to remember. Still, this tactic (which I had developed unconsciously) often failed.

I was also considered to be “extremely irresponsible” in the sense that I would constantly lose things (I still do this today). Family and friends also joked about the fact that I couldn’t go anywhere without leaving something behind. Now I began to think that “brain fog” might be impacting this behavior. By the time I left a social gathering the sensory overload of light and sound would leave me struggling to even speak, let alone remember my belongings. The other women I had interviewed had verified these feelings of sensory overload. Maybe I wasn’t that irresponsible… perhaps at least some of my forgetfulness was related to my illness?

Over the next few months, I made more connections between my “personality” and my illness, all of which pushed me towards investigating new treatments aimed at helping neurological symptoms. Some of these helped me improve. Also, to be honest, I began to feel a little less crazy. I started to cut myself a break when I couldn’t remember something. It was nice.

However, the “brain fog” interviews also changed my science-based thinking about ME/CFS in general. At the time, few people were discussing ME/CFS in the context of neurological dysfunction. I was now convinced that there was a strong neurological component to the illness that was often understudied. For example, I became very interested in the results of a 2011 study on CFS that few other researchers seemed to be discussing. The study by Schutzer and team at New Jersey Medical School examined the cerebrospinal fluid of patients with ME/CFS. Cerebrospinal fluid is a clear body fluid that bathes the brain and spine. The team found that subjects with ME/CFS harbored a strikingly different mix of proteins in the cerebrospinal fluid when compared to healthy controls. In fact, 738 of the 2,783 of proteins detected by the team were unique to patients with ME/CFS. If the study was correct, it provided good evidence that ME/CFS profoundly impacts the brain as well as the body.

This new information changed the way I presented the disease to others. The Schutzer study became the cornerstone of hypotheses I advanced in this journal article on ME/CFS. I started my talks about ME/CFS with the slide to the right (data from the Schutzer study). Doing so forced the viewer to immediately acknowledge that patients with ME/CFS differ markedly from their healthy counterparts. While I had long argued that ME/CFS is driven by chronic infectious agents, I started to look more closely at microbes able to persist in the brain. I began to study the the gut-brain axis and the vagus nerve – topics I still investigate today.

Several years later, Mady Hornig at Columbia University reported a second analysis of proteins in the cerebrospinal fluid of patients with ME/CFS. Her findings demonstrated the same trend as Schutzer and team. In 2014 researchers at Stanford University documented clear brain abnormalities in patients with ME/CFS, thanks to the use of fMRI technology. Another study by researchers at Emory University School of Medicine also used fMRI technology to show that patients with ME/CFS have had less activation of the basal ganglia – a part of the brain that impacts motor function. The study’s lead author, Andrew Miller, stated, “The study supports the idea that an immune response to viruses could be associated with fatigue by causing inflammation in the brain.” Some researchers were surprised by this hypothesis, but in my case it backed insights that I had already published.

Karl Deisseroth seeks out patient reported feedback

Because patient-reported feedback has played a large role in my development as a researcher, I was both comforted and fascinated to read a recent New Yorker article on Karl Deisseroth, a psychiatrist and a neuroscientist in the Bioengineering Department at Stanford University. Deisseroth is an extremely prolific and accomplished scientist. He helped pioneer the field of optogenetics, and he is currently leading projects that may redefine how the brain is understood to function. Yet what interests me most about Deisseroth is the manner in which many of his ideas are informed by patient experience.

The New Yorker article describes a psychiatric appointment between Deisseroth and Sally, a patient with severe depression who has not been helped by any standard treatments. Before meeting Deisseroth seven years prior, Sally had spent much of her adult life in bed and had twice attempted suicide.

Once Sally starts seeing Deisseroth, he has a surgeon implant a vagal-nerve stimulator (VNS) in her left collarbone. The experimental device sends “bursts” of electricity into the vagus nerve, which carries the signal into areas of the brain. Doctors think some of the affected areas may regulate mood but the exact reason for the effectiveness of the device is not understood. Nevertheless, in Sally’s case, use of the device under Deisseroth’s guidance has “transformed her life” and made her symptoms much more manageable.

Deisseroth is studying how these vagal-nerve stimulators impact the brain. Yet as far as I can tell, he is not conducting a randomized controlled trial on their efficacy. Perhaps this is because the level of electrical stimuli he administers to each patient via the device is extremely personalized. Under such conditions, he allows patient feedback to guide his thinking on the technology.

Deisseroth tells the New Yorker: “Sally’s response to the treatment was good evidence for the efficacy of VNS. But it also provided valuable insight for…(my) work as a neuroscientist. When I’m sitting in front of a patient and hearing what they’re feeling, it concentrates the mind wonderfully…It’s a source of hypothesis, a source of ideas.”

In fact, Deisseroth places great value in the exact words his patients use to describe how they feel. For example, the New Yorker introduces us to Henry, another of Deisseroth’s patients with severe depression. Henry describes the worst depths of his depression directly to Deisseroth, stating that everything could fill him with hopelessness and dread. “It could be an object,” he says, pointing at a desk, “like that piece of paper. It bothers me in some unimaginable fashion.”

Deisseroth responds: “That’s a great phrase… Just looking at an object and it making you feel bad. I’ve never heard any patient say that. That’s a great, crystalline description of how it just touches everything: perception, action, and feeling.” When Deisseroth is pressed on his excitement over Henry’s feedback, he explains, “That’s very useable…I can think about doing experiments in animals now with that.”

This use of patient reported feedback to fuel scientific inquiry is an exception to the rule. As I’ve mentioned, most researchers do not regularly interact with patients or seek out their experiences. Instead, they are trained to base their hypotheses and medical recommendations on the results of formal trials.

There are several reasons why Deisseroth may be willing to allow patient-reported feedback to play a vital role in his research. For one thing, Deisseroth’s own brain functions differently from that of the average person. He has a photographic memory and an amazing, innate ability to rapidly internalize complex information on a wide variety of topics. For example, he once finished two long books over the course of an hour. Perhaps, then, Deisseroth’s day-in, day-out experience of living with his “different” brain has made him more open to the feedback of people whose cognition is altered due to illness.

Deisseroth also continually searches for inspiration beyond the confines of strict scientific inquiry. His initial dream in life was to become a writer so he has made a point of taking creative writing classes in addition to his science/medical courses. According to the New Yorker, he remains an avid reader of fiction and poetry and is polishing a book of short stories and essays. Deisseroth subsequently believes that creative writing and scientific inquiry are connected. He states, “In writing, it’s seeing the truth—trying to get to the heart of things with words and images and ideas. And sometimes you have to try to find unusual ways of getting to it.”

This means that, according to the magazine, Deisseroth chose to pursue a PhD in neuroscience because he was “motivated by a desire to better understand human nature.” He tells the New Yorker: “I didn’t come in by asking, ‘How many bits per second can flow through a pathway?’ … I came in—maybe from the literature exposure—wanting to know where feeling came from. How you could be uplifted by words. How imagination worked.”

This means that while Deisseroth initially pursued training as a neurosurgeon at Stanford, he changed his mind after doing a mandatory four-week rotation in psychiatry. One of his patients was a man with schizophrenia, whom he treated with strong mood-stabilizing and antipsychotic medications. Yet despite these interventions the patient remained too ill to leave the psych ward. According to the New Yorker, Deisseroth was both disappointed and fascinated. “It was the unknown that grabbed me,” he says. “I knew how far we were from a glimmer of understanding.”

During his residency, Deisseroth subsequently struggled to reconcile his laboratory research with the suffering patients he talked with on the ward. One of his mentors, Stanford neuroscientist Rob Malenka recalls, “He’d spend all day seeing patients, then rush over to my lab and spend four or five hours running experiments.”

The problem of bias

In the New Yorker article, Deisseroth does not mention the word “bias” when discussing his patients. In contrast, other members of the medical community are often quick to point out the many forms of bias (prejudice) that can impact the accuracy of patient-reported feedback. For example, the placebo effect: in some cases a patient might deem a treatment helpful simply because the person harbors the expectation that it will be helpful. Or reporting bias: some patients may selectively suppress information that might impact their response to treatment (for example past medical conditions, smoking, sexual experiences).

In this environment, information obtained from patient case histories is frequently described as “anecdotal evidence.” The term has a negative connotation. An anecdote is a story or an experience that lacks a firm basis in reality. For example, the sentence, “The plural of anecdote isn’t data”, is commonly invoked by members of the evidence-based research community. In other words, even a series of “anecdotal” cases – in which patients report the same or similar reaction to a treatment – is not considered a valid source of evidence.

It is absolutely true that many forms of bias can and do impact patient-reported feedback. However, I think that the above mindset is too strict. I prefer an approach similar to Deisseroth’s: we should trust patients to the greatest degree possible and use critical thinking to consider possible bias in specific cases. This stands in contrast to an “evidence-based” approach which starts with an inherent distrust of all patients.

Take Sally’s case from the New Yorker. Like Deisseroth, we could view her feedback as a means of validating the effectiveness of VNS therapy. Or we could think, “Sally’s reaction might be biased. She is probably so excited to be around a medical ‘superstar’ like Deisseroth that she told him whatever he wanted to hear.” Or “Sally is so desperate to get better that she’s exaggerating her positive reaction to VNS therapy.”

While these later biases are possible, in my opinion, patients do not display these behaviors as often as many researchers would suggest. In fact, the notion that patients cannot regularly make sense of their own treatment is rooted in the same belief system that promotes the concept of psychosomatic illness. Some of these ideas can be traced back to Freud, who believed that the “unconscious mind” prevents individuals from relating the true motivations driving their behavior. Women are prone to “hysteria”, hypochondriacs run rampant, and somatoform disorders (in which the mind somehow creates symptoms of physical illness) are common. However, in 2016, these notions are extremely outdated. Today the vast majority of “psychosomatic” conditions are understood to be real diseases that can be studied at the molecular level. Similarly then, we may need to re-evaluate whether many of the long-held “biases” associated with patient reported feedback still have a basis in 21st century medicine.

In fact, in my own experience, patients often act in ways opposite to the biases levelled against them. For example, most patients I know regularly admit when a treatment doesn’t work, instead of “clinging” to hope. Or they have succeeded in researching their own illness to the point where they can discuss it with clarity. English clergyman William Dodd once said, “Depend upon it, sir, when a man knows he is to be hanged in a fortnight, it concentrates his mind wonderfully.” It’s true: the constant struggle of living with a disease forces one to think about it constantly, yet often in a proactive fashion.

A story recently published online in ProPublica provides an amazing example of how patients can often offer a valuable perspective on their illness. The article tells the story of Jill Viles, a woman who, in her early teens, began studying medicine on her own in an effort to better understand why her muscles were wasting away. After decades of research, she correctly diagnosed herself with two different rare genetic diseases. She additionally made a connection that the research community would have missed without her help: Despite their vastly different levels of muscle mass, she and Olympian Priscilla Lopes-Schliep have the same rare genetic muscular disease.

If you look at the picture to the right, you can see how radically different Priscilla and Jill look at first glance (the woman to the right is Jill). However, Jill was so familiar with the subtle ways in which her disease had impacted her body that she could observe a similar dynamic in photos of Priscilla. For example, both women tend to drape their clothing in the same fashion in an effort to cover up areas of missing fat. After analyzing photos of Priscilla, Jill told ProPublica , “It was just unmistakable. It’s like a computer that can analyze a photo and get a match and be 100 percent sure that’s the same shoulder, that’s the same upper arm. I see the same veins, I see them branching this way. You just know and it’s hard to convey, how could you just know. But I knew we were cut from the same cloth. A very rare cloth.”

At the end of the article, Heidi Rehm, a geneticist at the Harvard Medical School, concedes that people with a rare disease in their family will often have seen more cases and different manifestations of the disease than their doctor. “I think there is a cultural change,” she says. “Physicians are recognizing the very important role of the patient in being not only an advocate for themselves, but really a source of relevant information.”

Rehm is right. Evidence-based medicine emphasizes a “building block” approach to the development of novel hypotheses. New insight is expected to gradually accumulate as data from an increasing number of clinical trials better illuminates a topic. While this approach functions in many cases, medicine must also consider hypotheses derived from other less conventional sources. This is especially urgent when the prevalence of most chronic disease is on the rise and few treatments exist for the majority of these conditions. Patient insight and experience can serve as a fruitful starting point for these new ideas. Those who embrace belief systems that categorically dismiss patient feedback will subsequently miss many valuable clues and opportunities.