Six Steps to Evaluating Conflicting Menopause Information Online
Dr. X says one thing. Dr. Y says the opposite. Who’s right?
It’s not uncommon for me to write a Substack or make a video, then get a question or reply along the lines of, “Dr. Y recently said the opposite.”
How do you, as a consumer of health information, know which is the right answer? There are two possibilities. The first is that the topic is not settled, and so there may be two or more valid answers given the state of the medical evidence. The second is that one person aligns with the bulk of the high-quality evidence, and the other does not.
Here are some steps that should help point you in the right direction when Dr. X says one thing and Dr. Y says the opposite:
Check the Guidelines 📋
What do the menopause societies say? Expert guidelines are supposed to involve rigorous fact-checking, consensus, and disclosure of biases. Here are the guidelines that I consider when evaluating evidence online, having spent a lot of time reading them and the references. (This doesn’t mean other guidelines are wrong, I just know these very well).
The 2022 Menopause Society Hormone Therapy Guidelines
The 2021 Menopause Society Position Statement on Osteoporosis
A Practitioner’s Toolkit for Managing Menopause (from Monash University, really accessible and thorough)
The 2023 Menopause Society Guidelines on Non-hormone therapy
Is one doctor aligned with the guidelines, or are both? If it’s both, you have your answer (there are two or more ways to approach the problem). If only Dr. X is aligned with the guidelines, and Dr. Y is not. Then move on to step 2.
Check the References 🔍
Does Dr. Y have references? If the answer is no, that’s two strikes (one, against the guidelines and two, nothing to back it up), and this is a two-strikes-you’re-out situation.
If the answer is yes, check the dates of the references. Studies from 2 years or more before the guidelines were published will have been considered when the guidelines were developed. For example, if we are looking at guidelines from 2022 and Dr. Y quotes papers from before 2018/2019, those papers, if they are high quality, will almost certainly have been considered by the experts who wrote the guidelines and their results would have been included or rejected when writing those guidelines. If the information from the older references was not enough to change the guidelines, we should be questioning why we should be using them they probably shouldn’t be trotted out for decision making now without some major re-analysis of the literature.
What if Dr. Y’s references are more recent? Well, that’s a different story, because new data may have been published since the guidelines were written. Keep in mind that, sadly, being published does not mean the study is worthwhile. Garbage gets published all the time, even in what we like to think of as good journals. On Instagram this week, someone was defending doing hormone tests to regularly check estradiol levels and referenced an abysmal paper that, in my opinion, should never have been published. You can read my take on that paper here, and the post has a lot of information on levels. And also remember, a study may be good, but may still not be good enough to change clinical guidelines. For example, it may be cross-sectional or retrospective and that is not enough to over-ride other higher quality studies.
If the papers being quoted are relatively recent and you suspect they may not have been used in creating the guidelines, check whether they are clinical trials or opinion pieces. Here we’re specifically looking for clinical trials. A new clinical trial might change things (I say “might” because everything needs to be evaluated in the context of the available literature). When there is a new clinical trial, it’s worth Googling the name or topic of the trial and “The Menopause Society,” “The Australasian Menopause Society,” “The British Menopause Society,” or the “American College of Obstetrics and Gynecology.” If this were a pivotal clinical trial, the medical societies would almost always have a press release. Here is an example of a pivotal trial that changed recommendations: the NEJM study on partner treatment for recurrent BV. Here is my Google search, and voila, an updated guideline.
You can play around and use search terms like “Menopause Society new study hormone levels MHT osteoporosis,” or whatever term interests you.
Research the Doctor 🧑🔬
Is Dr. Y of the dissenting opinion, a noted researcher in the field? By “researcher” I don’t mean a physician who is reviewing someone else’s literature and then giving their opinion (for example, like I am doing here), I mean the person doing the actual studies. Go to PubMed, enter their name, and look for their research. You can even click on clinical trials and randomized controlled trials in the sidebar to narrow your search. If Dr. Y is a noted researcher in the field, then their dissenting opinion from guidelines should be further investigated. Perhaps they were excluded from the guidelines due to to political reasons? However, if they are doing a lot of rat studies, that should never be translated directly to clinical recommendations for humans!
Beware of Compounded Hormones and Supplements 🚫💊
Does this person recommend compounded hormones or supplements? These products are usually promoted with extreme acts of cherry-picking the literature. I always think, if someone is cherry-picking in one area, how can I trust their take in another area? Compounded hormone products are not recommended as they are difficult to compound, and absorption is erratic. If Dr. Y believes compounded hormones are the way to go, I have zero interest in anything they have to say given safe pharmaceutical versions exist. Also, I’m not interested in hearing dissenting opinions from people who sell supplements for menopause, because claiming these products are effective for menopause or special is, in my opinion, predatory.
Look for Bias 💰
What does Dr. Y, our dissenting doctor, gain from this information? This can be hard to suss out, but if the information is about supplements, estrogen face cream, or hormone levels, for example, and they sell/charge for that in their clinic or online store, that raises concerns about bias. For example, when someone is arguing with me about the benefits of hormone testing in managing MHT and I see this kind of promotion on their website, I’ve no interest in continuing the conversation.
Ask yourself, “Does this recommendation make my care more expensive?” If so, it’s worthwhile taking a much closer look. Remember, providing content (whether evidence based or not) that promotes treatments, products, or services the author subsequently profits from is not advocacy, it’s marketing.
Find a Trusted Fact-Checker 📚
While I stand by everything that I have written here and in The Menopause Manifesto, if mine is one of the opinions you are considering, you can also find another source. Having more than one trusted source is a good idea. Also, I do hope you run my statements through the fact-checking exercise above. But let me recommend an up-to-date resource that was just published and was written by some people I consider to be titans in the field: Menopause: What Your OB/GYN Wants You To Know. This is the kind of unbiased, fact-checked reference that represents an incredible command of the literature and decades upon decades of clinical experience.
I love this rundown, thank you! It's a great guideline for so many other spaces as well.
It is not the ‘Australasian Menopause Society’s Practitioner’s Toolkit’. It is the Menopause Practitioner’s Toolkit produced by the Women’s Health Research Program, which is headed by Professor Susan Davis. You are accessing it on the AMS website (but it is available from other places, too).