Not How Science Works - Adventures with the Placebo Effect


I'd like to put you in my shoes with a short analogy.  Trust me, it'll be worth it.  Johnathan Swift had an idea to eat babies in order to solve poverty.  If you haven't read this short piece of satire go ahead and follow the link.  I'll wait.

Now imagine you're talking to someone who sincerely doesn't understand the problem with cannibalizing children for profit.  This is hard, as a thought experiment, because you get a lot of time to think that you otherwise wouldn't have in the normal flow of a normal conversation.  You might imagine you'll give clever responses and woo them with enlightened understanding.  In my experience, I think the conversation will be more like the following.

You: "Because it's wrong to eat babies.  Everybody knows this thing."

Them: "Why is it wrong?  Obviously everybody doesn't know it, and there's this guy who thinks it's a great idea.  It seems to work in principle.  Why do you think it's wrong?  Can you give me some specifics?"

Specifics?  It violates so many foundational principles of human society and basic ethics all at the same time that it's impossible to pick just one.  You want me to tell you why it's wrong to implement a plan to eat babies?  Because it's wrong to eat babies!

You try a different tack.

You: "Okay maybe everyone doesn't know it's wrong, but this is a serious violation of basic ethical principles.  People who know anything about ethics are fully aware of this concept.  It's clear this guy who came up with this idea is either not serious or doesn't understand basic ethics.  

"Please understand that I'm not trying to be vague about specifics here.  It's just that there are so many problems with this whole concept it's difficult for me to focus on just one thing.  It would take hours to exhaustively go over in detail why this is so wrong, and you're asking for me if there are one or two little problems with it.  It's a catastrophic failure of ethical thinking, proposing to solve a long-term problem by creating a dystopian society."

Them: "You're just not open to new ideas.  This proposal would clearly work.  Think of all the problems it would solve!  You're attacking someone else as not knowing much about ethics, but maybe they know new principles of ethics you're just not aware of."

They reject your offer to explain - in detail - why it's wrong to eat babies, and leave believing infanticide is fully justified if it's targeted at poor people.

This sort of thing happens to me often enough that I've spent a lot of time trying to figure out how to avoid the final outcome above.  Not with cannibalism specifically.  Actually it would be easier if it was always the same thing.  It's the fact that each time is different that catches me off my guard.

Sometimes I have friends who get into some strange bit if pseudoscience.  They ask me what I know about Esoteric Idea, and when I say I've never heard about it they get excited.  Here they are, ahead of the curve telling a scientist about some new theory!  This must be at the bleeding edge of discovery.  They tell me about it, anticipating they'll get confirmation from an expert.

Except there's a reason I've never heard of it before: it's obviously wrong.  I immediately reject it.  This is the opposite of what they were expecting.  They think Esoteric Idea is real and justified and they're leading the wave of human understanding on health/beauty/nutrition/etc.  Soon everyone will realize how true Esoteric Idea is.  After all, the explanation makes sense!

There's a disconnect between me and my friends when this happens. To them Esoteric Idea sounds like the kind of solid scientific idea that is probably right, but my training tells me there's no way it could be.  The reasons for my objection are too numerous to explain to myself, let alone my friend, in the five seconds between their question and my response.

The problem is that after the ridiculous claim is made my friend wants to know whether it's true as a matter of brief factual analysis.  For some claims this is easy.  For example, if they said they love pistachio pudding because of the little bits of pistachio, I can point to the label on the box and show them they're actually bits of almond.  I can then let them smell some almond extract and within sixty seconds they'll understand that pistachio pudding is really just green almond pudding.  It might not be an immediately obvious fact, but it's simple.  Most of these misunderstandings about biology are not simple.

Biology is complex, and it's within that web of complexity that the uninitiated might think they've identified some detail that's poorly understood when the opposite is the case.  This is made the more difficult because the guy who is peddling Esoteric Idea usually hijacks normal terminology and applies magical thinking to it to produce crazy results.

Recently, I discovered a simple way to articulate this feeling of underlying wrongness I get when I'm objecting to this kind of popular science myth. I'll outline some examples below, but as a tl;dr I made a helpful meme to illustrate the epiphany:



That's not your skin works:

I like to joke with my wife that all her "creams and ointments" are really just the placebo effect.  While she's getting ready I'll ask her, "How long can it take to apply placebo?"

I can't speak to the possible effects of secret formulas and the like, so I'll shy away from giving a blanket statement that no company has or ever will discover the Secret To Youthful Skin.  But some of the ways they try to market this stuff is absurd.  It fits into the category of 'obviously wrong to anyone who knows about this stuff'.  Yet it's so commonly believed I want to try and explain why this stuff is so absurd to a biologist.

Let's start with the various structural proteins they put into hair and skin care products: keratin, collagen, elastin, etc.  There are plenty of hair care products that advertise how your brittle hair needs more keratin.  If you use their product the keratin will infuse into the cells and make your hair strong again.

The claim that keratin is a vital structural protein in hair is accurate.  What's misleading is the idea that slathering shampoo filled with keratin on your hair is going to get more keratin in there.  That line of reasoning is similar to saying, "Cars drive because of the gas in their engines.  Your car can't go anywhere now because it's out of gas.  To fix that, let's pour a hundred gallons of gas all over the hood."  That's obviously wrong because you need the gas to get inside the car in order for it to work.  The car isn't designed to just pick up fuel along the side of the road.  You have to add more the old fashioned-way - into the gas tank.  The same is true of hair.

Your hair grows when living cells at the root of your hair, called keratinocytes, produce massive amounts of keratin and store them inside themselves.  They then die and add themselves like a link on a chain in a long, connected strand.  By the time they get far enough out that you can see them all the cells are dead.  There's nothing living left to "fortify".  Going back to the car analogy, keratin shampoo (or collagen cream or whatever) is like pouring gas on the hood of a car with no engine.  There are too many things wrong with the whole idea to take it seriously.

It's not the dead cells we're targeting, it's the root!

Okay, let's talk about that for a second because this is often the second half of the sales pitch.  First, we'll fix your dead, frizzy hair with natural proteins and words that make it sound like hair is alive (no hair you've ever seen is).  After 'revitalizing' and 'rejuvinating' your hair, we're going to 'fortify' the hair at the root.  This will keep it 'vibrant' and 'healthy'.  Notice all the life-related references?  That's not how hair works.  It's supposed to die.  If we had shampoo that could bring your hair back to life - or keep it from dying - that would be truly spectacular.  And you should probably not touch that kind of magic because it really is tantamount to creating zombie hair.  Nobody would buy a product that promises to make you into medusa.

But the idea of adding keratin to living cells makes a certain kind of sense.  After all, if you add it while it's alive, won't that help keep your hair stronger later on?  Let's assume some small amount of keratin makes its way down into the hair follicle when you wash your hair.  There's no reason to assume keratin is going to make its way into the hair cells themselves.  Why would it?  The whole point of cells that line the outside of your body is to keep the outside world from getting in.

There are lots of reasons all your body's cells want to ensure they do not under any circumstances take in foreign proteins.  This is exactly like downloading and opening email attachments from Nigerian princes looking to give you money.  Not only is your body resistant to using any of that collagen, keratin, etc.  Your body is actively trying to avoid it!  The idea that it would ever do so just because you apply some expensive conditioner is not how biology works.

And why do we think those keratinocytes need us to apply extra keratin, when they can just make it on their own?  Again, going back to the car analogy, this is like pouring gasoline on the hood of the car when there's a full tank of gas and the car is humming along down the road.  For the same reasons you don't want any kind of foreign liquid that rains down on your car to get into the gas tank, your hair doesn't want any of the stuff in your creams and ointments to get inside the hair cells.  The only thing you want in your gas tank is what you put there intentionally, on your own.  The only thing your body want in your hair cells is what it puts there itself.

Your hair cells haven't heard any of the advertisements.  They don't want your expensive proteins.  They're not listening anyway, because they're all dead.

Okay we're in, now what?

Probiotics are a similar, but maybe a little more subtle, example of this same phenomenon.  People seem to know there are bacteria in your gut (loads of them!) and that which types of bacteria are most common is important for lots of things generally related to being healthy.  That part of the sales pitch is on solid scientific ground.  In this way it's exactly like many of these health fads: it's based on new and exciting research really is on the bleeding edge of biology.

Lots of people study the balance in gut bacteria, including a good friend of mine.  We used to work near each other at the same lab bench.  She loved learning about the balance of bacteria in your gut and how it affects human health in all kinds of ways.  There are literally thousands of different kinds of bacteria in your gut, and the balance among them all has been directly linked to health and disease.  It's a rich field of research, some of which even suggests that changing this bacterial balance could be enough to help people overcome things like asthma, allergies, and the like.

But bacteria in your gut don't work anything like how the yogurt companies advertise.  When I see yogurt marketed as 'specially formulated' with active cultures of bacteria, I cringe.  They sell their product full of 'good bacteria' and tell you it'll improve your heath.  How?  Well it's in the yogurt, so once you eat it those bacteria will be in you.  And you'll get healthy.  Simple, right?

No.  That's not how biology works.  That's not how bacterial growth works.

First, there's the problem that the bacteria have to get through the acidic environment of the stomach in order to get into the gut.  Some of the bacteria in your probiotics yogurt are probably going to survive the deadly journey.  Not many, but a few will make it.  Of course, since they just went through a harrowing ordeal, they may be primed to react a little differently than how they reacted under perfect laboratory growing conditions (when you stress them heavily, bacteria go into survival mode, which produces unpredictable results, at best).

Okay, say they take time to recover and most of the bacteria will die in transit, but once they get accustomed to their new home, won't the surviving bacteria start making a difference?

Probably not.  If the problem was that you don't have those bacteria in the first place, maybe introducing them to the gut would work.  Think of it like introducing frogs to Australia.  Australia had other frogs before these ones, but it had never seen a species quite like the Cane Toad.  Once they arrived, the amphibians pretty much took over.  You didn't need to keep bringing ships over with more frogs to keep their numbers up.  The same idea applies to bacteria.  If you didn't have any of these good bacteria in your gut, one cup of yogurt should be enough to put them there.  But those bacteria are literally everywhere.  There's no chance you haven't been exposed to all the good bacteria necessary to keep you healthy long before you ever heard about probiotic yogurt.

The problem isn't that you don't have any good bacteria.  You do.  The problem is that they aren't well balanced against the rest of the bacterial community.  Your gut has somewhere in the ballpark of 100 quadrillion bacteria in it.  (Nobody is going to count them all, so we have to estimate.)  Say a 'healthy' gut is at least 50% of these "good" bacteria, with a smattering of other kinds of bacteria in there.  You look at your own gut and realize it only has 25% of those bacteria you want.  You don't have enough good bacteria!  That's okay, you can fix this by eating a yogurt that has 10 trillion good bacteria in it.  99% of them die in transit, but 100 billion good bacteria make it through.  They add to the pool of what was there, helping to restore the balance.  Right?

100 billion sounds like a lot, but in sheer numbers it's not that much.  The difference between the 25 quadrillion good bacteria you had before you drank the yogurt and what you have after is negligible.  For every new bacterium you ate there were already 250,000 already present.  To 'restore balance' through yogurt, you'd have to eat so many of them you'd give yourself diabetes.

And it still wouldn't work.  That's because this isn't just a problem of dose.  If we could use a teleporter and zap the right amount of good bacteria into your gut that won't fix the problem long-term.  We're still ignoring the reason for the imbalance in the first place.

Those Australian frogs didn't spread because a certain number were placed in Australia and that's just what any species will do in a new environment.  The spread because they were uniquely able to exploit the environment they found themselves in.  Take a group of polar bears to the Sahara and you're going to end up with a bunch of dead polar bears.  How does this work with bacteria inside you?  Say the good bacteria like to eat nutrients found in beans, animal fats, or Brussels sprouts; while the bad bacteria thrive by eating sugar.  Your probiotics yogurt dumped a bunch of good bacteria into the equivalent of the Sahara.  They're liable to starve to death.  Meanwhile you just fed a huge meal for all those bad bacteria you're trying to keep from taking over.  The problem may get worse, not better, because of your intervention.

How to avoid science marketing scams

Here's the problem.  Some concepts from biology are really interesting from a theoretical perspective, but entirely useless from a practical perspective (so far).  It is often the case that we understand the rudimentary principles behind something but we can't find any way to convert that knowledge into practical benefits for the public.  That's because when we study something in the lab we usually start by taking it apart to see what it does.  We then publish paper describing what that piece does, often outside the context of the whole person.  It's that context that matters most, but a trained biologist adds it back naturally in their head without having to think about it.

This can be confusing to someone outside the field because they don't have the context to understand what's really being claimed.  For example, say you read about how damaging oxidation is to cells, and how important antioxidants are.  There are hundreds of peer-reviewed scientific papers about this concept.  And besides, free-radicals just sound bad.  They're unstable, after all!  You could easily be fooled into believing the elimination of free-radicals and oxidation from cells is of universal importance.

But your immune system actively creates free-radicals in order to fight disease.  They're a normal part of functioning biology, and if you got rid of them entirely you'd break a lot of systems.  They're bad in certain contexts, but in others they're important to have around.  To go in and eliminate all oxidation products in your body would be like randomly removing beams from inside the Statue of Liberty.  Those beams are in there for a reason.  Just because one beam in the wrong place might keep people from getting to the top doesn't mean the solution is to tear them all out.  Something is going to collapse if we try that.  Fortunately, eating 'antioxidants' for their vague health benefits is similar to slathering keratin all over your skin cells.  Your body isn't about to take in a bunch of garbage it doesn't need.

Scientists study oxidation and antioxidants looking for clues on how to fight disease.  By now, you should read that and realize you can't get the effects we're studying by just dumping antioxidants all over yourself any more than you could get gas in a car by spraying it on the hood.  There's a lot of stuff scientists are working on that isn't useful yet.  That's because we haven't figured out how to go from theory to application.  That step is a really tough problem.

It's possible one day we will discover how to create a special type of artificial collagen that binds to external structures on hair and strengthens it.  Or maybe we'll discover that eating certain types of figs boosts the competitive advantage for good gut bacteria.  (I'd like that to be true.  Figs are tasty.)  When that day comes, expect the products to contain caveats like, "don't use too much of this hair cream, or you'll end up with stiff hair that's hard to cut or style."  Or perhaps, "only eat one pill of this fig extract, or your gut bacteria will over-correct out of balance."  If there's no warning about over-using the product - in other words, if there's no dose-dependent effect - there's usually no effect at all.

The best way to tell when one of these products is really just peddling snake oil disguised as science is that they plaster the scientific label all over it - to the exclusion of everything else.  If a company is trying to teach you scientific concepts like probiotics, structural proteins, oxidation chemistry, and the like you can bet they're trying to prime your placebo response as much as they can.  If you think it works better, you'll probably be convinced it really is.

Their alternative would be to sell you a product that actually works and rely on efficacy to convince you.  Indeed, a good rule of thumb is to just ignore the sciency talk in advertisements and focus instead on whether the product works better than the competition.  Maybe it doesn't and you don't have to pay for all that expensive keratin in your shampoo.

There are many more examples of this science-as-snake-oil concept, but I think the best example of all these principles at work is the vitamins and supplements industry.  That's probably because they've been doing this the longest.  If I can take 10 tablets of 1000mg vitamin C and the only side effect is the citric acid aftertaste, they're probably not doing me any good.  Meanwhile, if I take iron and it gives me constipation, at least I know it's doing something.  I should probably avoid taking too much, and I should only take it if I suspect I'm a little anemic - something my doctor and lab tests can help me calibrate.  Meanwhile I can buy a pill or a drink with super-massive doses of vitamins with no warnings not to overdo it, and no target I'm trying to reach.  I'm supposed to just take it to 'feel better' like the people in the photos.  That's a real placebo at work.

People ask me, "how do I know if Product X is based on good science without becoming a scientist myself?"  First, stay away from general promises like how much it will make you "feel better".  You wouldn't go into your car with a wrench, tightening and loosening things at random, if you didn't know what you were doing.  And you certainly wouldn't do it when your car was working just fine.  If you have a problem with the gears not shifting right, you'd take it to a mechanic.  You don't need to be a mechanic to keep your car running well, you just have to listen to the basic advice mechanics give.  Same thing goes for your body.

Second, go back to that rule of thumb from earlier: when a product works, it doesn't matter how much research went into making it happen. What matters are the results.

I recently saw an ad for a mattress that claimed scientific research helped them develop a better material.  Since that's the same thing other mattress companies often claim, I was unimpressed.  They used it quite prominently in their advertising.  NASA something something, polymer something something, whatever.  Then they demonstrated the properties of the new material using raw eggs. Whether you're a scientist or not, the raw eggs are more meaningful for a reason.

All the 'new material' angle tells me is that I probably won't see similar results from a company that doesn't have exclusive rights to this material.  It doesn't tell me if the material is an improvement. In the end, I looked into the reviews and bought the mattress.  This wasn't because they made claims based on Science, but because they demonstrated unique performance. Performance should guide judgement, and that's what ultimately sold me on a mattress I really like.  If they have only rely on science to sell it to you, it's probably just placebo.

Comments

  1. A Modest Proposal was written by Jonathan Swift not Thomas Moore.

    ReplyDelete
    Replies
    1. Ha of course! Thanks for catching that. I fixed it above.

      Delete

Post a Comment

Popular posts from this blog

Reverse Engineering Life

Cancer update: precision oncology comes into its own