Doing Your Own Research?

What You Need to Know About Data Voids

Yuichiro Chino

I am often asked in interviews what I think about people researching health information online. I think it’s wonderful that there is a massive repository of health information available for everyone. The problem is that the Internet is a tool, and like many tools if used correctly, it can be amazing. However, when used incorrectly, it can be harmful. Think of a car. It can be marvelous…a way that we can see friends, have a faster commute to work (depending on where you live), or take a road trip that builds amazing core memories. And yet, if driven on the wrong side of the road or while intoxicated, it is a dangerous and possibly lethal weapon. 

With medicine, the Internet is just like the car in our analogy. The line between marvelous and harmful can sometimes be almost impossible to spot, so learning how to search online is critical if you want to use the Internet to get someplace with your health and not crash into a proverbial tree. 

Here is an example, which is the event that spurred me to learn more about how people search for information and to make educating about this point a part of my platform. I was on tour for The Vagina Bible, and after the talk, which centered a lot on misinformation, when it came time for the Q and A, a woman stood up and asked about the O-Shot. You see, she had had it, there had been complications, and she was still suffering. How could it be, she wondered? She had done her due diligence and researched the O-shot online, and everything she had read was glowing. She had purchased The Vagina Bible, and I was the first source that she found explaining this was an unstudied, potentially risky procedure that is not recommended. 

How could it be that the Internet was so wrong? It wasn’t until she found an “old-school” form of information delivery, a book, that she was informed, albeit too late. I wondered, too, because I had written about it on my old blog. And yet, when I went home after the event and did my own search, the piece that I had written was on page three of my search, meaning it was essentially invisible.

Welcome to a Data Void

Sending people out to do “their own research” just doesn’t work because the Internet is filled with good information right alongside misinformation, and unfortunately, distinguishing between the two can be challenging, even for an expert. When I research online, I start with the medical literature. I make sure the journals involved aren’t predatory and check what relevant medical societies might have to say. I also look at financial bias, and I evaluate the data for its accuracy and how it fits into the body of knowledge. When I read content that I’ve found in the press, blogs, or on social media, I cross-reference what I’ve already found in the literature, and I check to see how accurately journals may have been cited. I can only do that for my own field of expertise because I am already an expert. Without that expertise, it’s very easy to end up in an eco-chamber where misinformation is mistaken for facts. This has been demonstrated in studies, which tell us when people look up information online, paradoxically, they often become less informed. And one contributing factor is data voids. 

A data void exists when there is no or very little reliable information on a topic. Data voids are easy to find in so-called alternative health care because when a procedure isn’t recommended or is even downright ridiculous, like the O-Shot, people don’t write research papers on the subject, and hence, there are few, if any, legitimate news articles. This means there is nothing accurate, or at least very little, for people to find when they search. 

In addition, search engine optimization can create what appears like a data void, meaning those who promote disinformation are so adept and so prolific that their sites come up first, and valid content ends up on page three or four of a search, which means it may as well not exist. Suppressing negative articles about your product vs countering them with facts is an explicit strategy of scammers. It is much more difficult to factually counter a negative article about a scammy product like the O-shot because, well, the science isn’t on their side. What these folks lack in science, they make up for in search engine optimization skills. In this case, physician licensees of the O-shot write articles and testimonials on their websites that promote the O-shot and link back to the original O-shot page. This raises the ranking of the O-shot information in the eyes of the search engine because lots of seemingly legitimate websites link to it. This leads to more traffic, which then raises the rank more and so on into a never-ending self-referencing network. 

As this self-referential network of misinformation rises in rank, any factual article correcting it has to go somewhere. And that somewhere is down, down, down the rankings, past the second page into the Internet’s hinterland. Job well done… and much less effort than producing high-quality research. 

A study published in Nature this year details several experiments looking at the accuracy of information that is found online and specifically highlights some of the issues with data voids. Participants were given news articles to validate, and one article that was used was titled  “U.S. faces engineered famine as COVID lockdowns and vax mandates could lead to widespread hunger, unrest this winter.” When “engineered famine” was used to validate the article, 63 percent of the results were unreliable. Since there was no engineered famine, this isn't surprising. Reliable sources aren’t going to write about an “engineered famine” unless it is in the context of a conspiracy theory, and then the number of articles about the “conspiracy theory of engineered famine” will be far less than the number of articles and links to the original content generated in creating the conspiracy theory. 

In medicine, it seems there is a new scam or super sketchy therapy daily, which lends itself perfectly to data voids. These “topics” aren’t written about in valid journals because they never followed the scientific method in the first place. Alternative medicine loves data voids because when you do the search, lots of sites come up, so at first glance, it looks legit when, in reality, it’s just site after site repeating the same disinformation.

Let’s go back to the O-Shot. If you Google O-shot, this is what you get. 

Most of the sources on this page are sites that profit from the O-shot. There is one post from Healthline that basically functions as an aggregator of content that I would advise against using. For this post, they decided to “learn more” by talking with providers who do the O-Shot, so a glorified advertisement masquerades as health content. There is a post from the American Society of Plastic Surgeons, which is a legitimate organization, but their providers profit from this procedure, and there is no scientific information to be found on this site. 

It’s no surprise that the AI summary of this search is basically an advertisement:

What if you decide to refine the search a little and use “O-Shot ACOG”? Adding ACOG (American College of Obstetrics and Gynecology) is a way to optimize the search and get better results.

Sadly, it’s not much better. The ACOG Committee Opinion on cosmetic procedures is the first hit, but it doesn’t specifically mention the O-Shot. The rest of the hits are promotions, a PR piece from a site that publishes press releases, and a low-quality study.  

Interestingly, the AI summary was much better as the AI was smart enough to apparently give preference to information from ACOG (American College of Obstetrics and Gynecology). 

What if you are a very discerning consumer of medical information online and use “O-Shot Dr. Jen Gunter” or “O Shot Vajenda” as your search term” Then, you get a very different result, and the first results give you accurate information about the procedure and the problematic backstory. 

How Do You Deal with a Data Void? 

The first is to know they exist and that they can easily lead you into an eco chamber, like the first video above, which makes you think a procedure or supplement is valid when, in fact, no data exists. 

The second is to be mindful of your search term. When you use specific terms, like “the O-Shot,” you are less likely to get a diverse array of results than if you search “treatments for orgasmic dysfunction.” Results from that second search would not make someone think the O-Shot is a valid treatment!

The third is to use medical guidelines or other reliable sources to fact-check because you need a source that is free from bias to help put whatever else that you have found in perspective. This is easy with something like menopause hormone therapy but admittedly is harder with sketchy procedures with no valid publications, especially when actual MDs offer them. Basically, these procedures are the rumors of medicine and rarely float to the level where they get included in guidelines. There are just too many of them for the guidelines to carve out warnings for all the scams.

Finally, lateral reading is important, which means evaluating the credibility of the information. That can be challenging with medical misinformation because some of it comes from medical professionals, who we like to think we can trust. My previous post on evaluating menopause content online has many examples of how to evaluate content from medical professionals, so while there might be a data void on the O-Shot if you see the person recommending it also recommends thermograms; they are not a reliable source of information. Also, considering financial bias, looking at our first search, we see that almost every site that recommends the O-Shot is from a provider who profits from the procedure. You want diverse sources, so if you can’t find information from a source that doesn’t have a profit motive, that should raise a red flag that you are in a data void and should step back and reconsider your search. 

References

Aslett, K., Sanderson, Z., Godel, W. et al. Online searches to evaluate misinformation can increase its perceived veracity. Nature 625, 548–556 (2024). https://doi.org/10.1038/s41586-023-06883-y