In our previous episode, we learned that more than two hundred and fifty billion dollars a year is spent in the U.S. on advertising globally, the figure is nearly 600 billion dollars. That's more than half a trillion dollars on advertising. Because of the digital revolution, television advertising has lost some of its primacy. But TV still accounts for roughly a third of ad spending in the U.S. The Super Bowl alone brings in more than 300 million dollars.

And how effective is all that TV advertising? I mean, how good is it at actually selling the products it is telling you to buy? The conventional wisdom says it's got to be effective. Why else would companies spend so much money on it? But the data? Well, the data tell a different story. Here's what we heard last week from Anna Tuckerman's. She is a marketing professor at Northwestern University and she recently co-authored a massive study on the efficacy of TV advertising.

This means that doubling the amount of advertising would lead to about a one percent increase in sales.

So your research argues that TV advertising is about 15 to 20 times less effective than the conventional wisdom says? Yes, that's right. There are, not surprisingly, objections to this research, especially from the marketing industry.

For instance, they will point to the brand building aspect of advertising. It's not just about short term sales, they'll say, or the game theory aspect. That is, if you don't advertise your product in your rivals, do, where does that leave you? Still, any company that spends even thousands of dollars on TV ads, much less millions or billions, would have to be sobered by Anna Tuchman's findings.

With TV advertising always so inefficient or didn't lose its luster recently with the arrival of digital giants like Google and Facebook.

We don't know the answer to that question, but we do know is that people are spending more time online than ever before and that digital advertising holds the promise of matching advertisers precisely to the people who want their products at the end of the day.

Everything about marketing strategy is around segmentation.

That's the former chief marketing officer of Unilever, Keith Wead, more relevant to our to the audience.

The more interested they'll be in your message, the more interested in your message, the more likely they are to buy your product.

And thanks to the cookies on your phone or computer, such a cute name for such a powerful tracking device.

You are constantly telling the companies who installed those cookies exactly what you are interested in.

You also tell them where you live, what you wear, and listen to and eat what kind of people you hang out with, which God, if any, you believe in which political party you hate less than the other.

Yes, the thinking behind advertising has changed radically with the arrival of digital and data.

The Internet has made it almost too easy to sell to us and sell to us. They do. Last year, advertisers spent 123 billion dollars on Internet ads in the U.S. It's just less than half the total ad spending across all media. That's how Facebook and Alphabet, the parent company of Google, have become two of the most valuable companies in the world. More than 80 percent of Google's revenue comes from advertising. More than 98 percent of Facebook's revenue comes from advertising.

With so many advertisers spending so many billions, they must be getting a healthy return on their investment. Right. So digital advertising must be effective, right?

Today on Freakonomics Radio, a hard look at the hard to find evidence around digital advertising.

Oh, ads definitely work, but we can't tell you how or why or give you any evidence for it. From Stitcher and other productions, this is Freakonomics Radio, the podcast that explores the hidden side of everything.

Here's your host, Stephen Dubner. Steve to Dallas is a professor at the School of Business at the University of California, Berkeley. Yes. And I teach and do research in economics.

A lot of his research is on the economics of the Internet. Back in 2011, he started doing some work with eBay.

I was asked to hire and lead a team of economists and work with a variety of the businesses within eBay.

The hiring of prominent academic economists is a long standing practice at tech firms. Hal Varian, also from Berkeley, has worked for years as chief economist at Google. John List from the University of Chicago has done similar work, first at Uber and now at Lyft.

So here was Steve Dedalus at eBay, and at one point one of the directors there realized that as economists we should know something about econometrics, which is the statistics of measuring economic activity. And they wanted to see if we could vet a consultant that they recently hired a consulting firm to do quantitative marketing analytics and help eBay figure out how to spend their scarce marketing dollars.

Let me just say that scarce here is a relative term, that the time eBay was spending about a billion dollars a year in a variety of marketing activities.

OK, so it makes sense that eBay would want to know how effectively that money was being spent. More than half of it was going toward Internet ads. And this outside consulting firm was going to analyze the efficacy of those ads.

And the goal was to speak with this consulting company and see if they're going to use solid and vetted ways of measuring these returns to advertising.

To be fair, measuring the efficacy of advertising can be really hard. Last week, my Freakonomics friend and co-author Steve Levitt told us about trying to help a big box retailer measure the efficacy of their TV ads. One issue is that they really only advertise three times a year for Father's Day, Black Friday and Christmas.

And so you, of course, have a correlation between when you're advertising on TV and when you're selling things. But it's not necessarily or even primarily because of the ad. It's because the company knows when the big selling days are and they target the ads around it.

So he's not the causal part, the sales that wouldn't have happened absent the advertising. It's just a really hard problem.

But within that firm, Levitt found the executives were convinced it was the ads that caused the sales.

Steve, tell us again, this is the kind of wrong analysis that many people end up falling into when they are not that conscientious about the difference between causation and correlation to Dellis knew this would be a hard problem for eBay as well.

And that's what these outside consultants were supposed to figure out.

So I got in a car with them and very quickly was able to confirm that what they're doing was quite wrong.

Oh, just so you know, consulting firms also like to hire academic economists to do this kind of work so you can maybe see where this is going to.

Dellis is listening intently on the call, which was on a landline, which will make sense in a second, and went to Dellis suggested that the consultants proposed methodology would not be able to untangle correlation and causation.

They responded using a whole bunch of jargon, especially the term proprietary transformation functions, proprietary meaning.

That's our secret transformation functions. Meaning, who knows?

Tudela said he still didn't understand how these consultants were going to measure the thing that needed to be measured the marginal effect of the ads that eBay was spending millions on.

Then the head of the company replied by saying that to do the marginal measurement, they're going to use Legrange multipliers. Well, I paused for a second because I know what Legrange multipliers are. I used to teach this stuff and I couldn't understand what they're trying to do here. And that's when the dime dropped. They are trying to avoid jargon me. So I replied by saying, well, we all know that the LeoGrande multipliers measure the shadow values of constraints in an optimization problem.

So it would really help me if you explain to me what is your objective function and what are your constraints after a short pause.

And this is where I have to take my hat off to the founder of that consulting company, he immediately responded with the only and best answer he could give, which was, Steve, are you driving now? Because I can't hear you. You're breaking up. To Dallas took his concerns back to his bosses at eBay. He proposed a different way to understand the impact of the online ads eBay was buying. He offered to run some randomized experiments to a researcher.

That's the gold standard.

And there was not any buy in.

But coincidentally, but coincidentally, one of the advertising and marketing teams wanted to renegotiate a deal with Yahoo!

And Microsoft network, namely the Bing search engine that is eBay wanted to renegotiate the terms under which they bought search ads on Bing, and before doing that, they ceased all payments for brand keyword advertising.

That's because eBay was trying to create some leverage heading into their renegotiation. OK, but now we need a primer on keyword advertising and how you actually buy online ads. Right?

So just like you could imagine the auction for a piece of art at Sotheby's where the auctioneer puts up this Picasso and says, how much are people willing to pay and then give it to the highest bidder? Something similar is happening every time someone searches for anything on Google or Yahoo! Or Bing or any other search engine.

And you have companies that are bidding on different kinds of keywords, companies that want to advertise online come up with a list of words that they think will give them the best chance of connecting with people who are searching for what the companies are selling. These could be branded keywords like eBay or Verizon or non branded keywords like stiletto heels or cable TV.

If I'm in the insurance industry and someone is searching car insurance on Google, I want to be there, but I don't want to be there when someone is searching for felafel, unless maybe you've got a specialty line of felafel insurance.

And once someone types car insurance, the second they click enter.

All these computers are basically running these auctions instantaneously.

This is done with the aid of sophisticated algorithms. So they have all the bids they allocated to the winner. And that's how the ad pops up.

When Dedalus was working for eBay, the company was in the practice of buying brand keyword ads, which meant that if you did an online search for eBay, the top result before all the organic search results was a paid ad for eBay.

The Federal Trade Commission requires such ads to be clearly labeled as ads, although some of the labeling is pretty subtle. Go do a Google search for something like best running shoes and you'll see what I mean.

Anyway, these paid eBay ads that Telus is talking about now, this is what the consulting company said had the highest bang for the buck, which, of course, made no sense from a common sense perspective, because if I'm already searching for eBay, I know exactly where I want to go. So the fact that the ad intercepts me and I click on it is just coincidental. And it cannibalizes what would have been a free click on the organic search.

So when eBay is planning to renegotiate their deal with being turned off, their brand keyword advertising, that gave us what's called a natural experiment.

This is what economists dream about, a change in a variable that affords a real opportunity to separate correlation from causation.

We could measure visits and we could measure purchases and we could see whether there was any drop in clicks and purchases. And not surprisingly, all the search that was taken away from the ads just ended up coming for free through the organic search because right below the ad was the free link to eBay. Once we had those results, I went to the chief financial officer of eBay North America and showed him the analysis to which he responded, OK, you guys were right, what do we do next?

And that gave us the open door to design.

More sophisticated experiments to Dallas wound up running two experiments, along with fellow economists Thomas Blake and Chris NASCO. The first one essentially mirrored the natural experiment.

They turned off all brand keyword search ads, which means that if someone searches for eBay, eBay will not surface. And ad the conjecture that we had, which I can't see any other conjecture, is that if there is no competition for keyword eBay, then once you remove that ad, the organic search is going to be the first thing that people see. And that means that they're going to go to eBay directly without having eBay pay any fees to the search engine for the advertising.

And that is precisely what they found, surprise, surprise. Other researchers have found even more pronounced results. Imagine that an eBay competitor like Amazon enters an ad auction for the keyword eBay and wins at auction. This means that when you search for eBay, the top link you'll get is an Amazon link. In such cases, researchers found most people just ignore the Amazon link and move on directly to eBay, which means brand keyword advertising. Whether it's your brand or someone else's is a waste of money.

Exactly.

Imagine you're a restaurant owner and you want to hand out coupons in order to get people to come to your restaurant. And if you want to measure the returns on that coupon, you really need to know how many people would come without the coupon. Because here's the thing. For every person who would have come anyway, you're losing money. The analogy, in my view, of brand keyword advertising is handing out the coupons inside the restaurant.

The second experiment to Dallas and his colleagues ran used non branded keywords, just the names of things that people might be looking to buy online, like guitar or boots or picture frame.

Yes.

So for non branded search, we actually had no idea what the results are going to be. Because here, if I am searching, for example, a studio microphone, I'm sure that on eBay I might find a variety of used ones. But if I'm not thinking about eBay and I just search for studio microphone, if eBay doesn't pay an ad, they might not even show up on the first page. And by the way, the automated machines at eBay that we're doing the online bidding, they had a basic library of close to 100 million different combinations of keywords because eBay has practically everything you could imagine for sale on the site.

So we really had no idea what the returns for the non branded searches would be.

The experiment they designed used a broader strategy. That's the same method we heard about in our earlier episode on TV advertising. This takes advantage of what are called DMAs or designated market areas, what most of us think of as media markets. It turns out that most people living on one side of a market border are socioeconomically and demographically similar to the people living on the other side.

When two similar populations are served, different ads, which happens all the time, you can isolate the impact of the advertising on their actual purchases.

There are 210 DMAs in the U.S. and we took a third of these DMAs and we turned off all paid search advertising.

This was an extremely blunt experiment where we're saying what would happen if we didn't advertise at all?

And to our surprise, the impact on average was pretty much zero.

Did you catch that?

They turned off all their keyword search ads, then measured actual sales and the impact on average was pretty much zero.

And what was eBay's existing belief about paid search advertising?

The company believed that roughly five percent of sales were driven by paid search advertising, meaning that they believe that if you would pull the plug on advertising, sales would drop by five percent. What we found was that sales dropped by about half a percent. So that's an order of magnitude less. And it was not statistically different from zero.

But maybe it's still worth it to gain even that half a percent. Now we have to know what the advertising costs and measure the return on investment.

When you did the return on investment for every dollar that eBay spends, eBay believe that for every dollar they're spending, they're getting roughly a dollar and a half back, meaning 50 cents of net profits. And what we showed is that on average, they're losing more than 60 cents on every dollar.

So how did these results go over? Well, the president of eBay, who later became the CEO, he cut the paid search marketing budget immediately by one hundred million dollars a year.

So what happened next, you might think, what with capitalism being the hyper competitive market, optimizing perfect information ecosystem it's supposed to be, you might think that other companies, once they learned about this eBay research, would cut their online ad spending or at least commissioned their own research to test the theories.

So did they.

Excellent question. There was a lot of chatter online after our experiments became public suggesting that folks at eBay don't know what they're doing. And paid search advertising works wonderfully if you know how to do it. But of course, that was backed with no data and no analysis.

In other words, the digital ad community did not rush to replicate the results. Now, given the opportunity to save millions of dollars that the eBay research showed was being wasted, why wouldn't other companies at least poke their own data a little harder?

Well, I think there are many reasons. Let's start with the way in which this industry is structured. You could think of four different actors here. There's the customer, which is the company or the person who wants to advertise in order to get business. And then you have three players sitting on the other side of this market. One' is the publishers. That would be Google, that would be The New York Times or any other place where the ad appears in front of people.

The other are the people who create ads. And then finally, a smaller part of the industry are these analytics companies that, like that company they hired, are trying to help companies spend this money.

And if you think of all these three players on the other side of the fence, no one there has an incentive to basically open this Pandora's box.

Even within the company that's buying the ads, the incentives can be complicated.

Steve Levitt, again, if you think about it, no chief marketing officer is ever going to say, hey, I don't know, maybe ads don't work, let's just not do them and see what happens. So don't get me wrong, I'm not implying that advertising doesn't work, I'm implying that we don't have a very good idea about how well it works.

Steve to Dallas agrees the potential for digital advertising would seem especially large given its ability to micro target consumers.

And targeting really is key because one of the lessons we learned from the experiments at eBay was that people who never shopped on eBay, they were very much influenced by having eBay ads for non brand keywords, you know, guitar chair, studio microphone.

And if eBay would be able to better target ads to customers that are not frequent customers, that's where you would get the real bang for the buck. So as companies become more sophisticated, they could try to engage in these kind of experiments to focus attention on different customer segments in order to see where they get the highest returns on advertising.

By and large, I don't see that happening.

A big part of it is the naivete on the side of these customers customers, meaning the companies who are buying these keyword ads, the one actor sitting alone on their side of the fence across from the agencies, the publishers, the ad tech firms.

And one of the things that I try hard to do is to give people enough information so that they wouldn't be able to do the job themselves.

But if someone is trying to sell them snake oil, they'll smell something is not working here.

Coming up after the break, is the digital advertising economy a bubble?

I do think this is very parallel to the kind of psychology that had driven bubbles in the past. Also, if you like Freakonomics Radio and you've gotten this far, so presumably you don't hate it. You may also like the two other shows that recently launched within the Freakonomics Radio network. One is called No Stupid Questions. The other people I mostly admire. I am proud to say that No Stupid Questions, which launched in May, is already coming up on five million downloads.

And people I mostly admire, which launched only at the end of summer, is coming up on its first million. So go listen and subscribe.

No stupid questions and people I mostly admire you can find them wherever you listen to podcasts. And now here's an ad that somebody probably cooked up just for you.

We reached out to Facebook and Google with some questions about their ad business and to get a response to the research we've been discussing today, which argues that paid search advertising is substantially less effective than the conventional wisdom holds. We got no reply from Facebook. The Google representatives wrote back to say, quote, Advertisers invest money in search ads because they work. They also sent some internal Google Research to back up their claim. Separately, we received an unsolicited email from Hal Varian, the chief economist at Google.

He attached a long list of research papers that assert the efficacy of search advertising, as well as advertising on YouTube, which is owned by Google. Much of the research variance sent was done by Google analysts. It offers a robust defense of the status quo, the online ad ecosystem. Google has built off of their search capability is quite literally a license to print money. Alphabet, Google's parent company, has a market capitalization of nearly one point two trillion dollars.

Last year, 83 percent of their revenue came from advertising.

So it would probably behoove all of us to know a little bit more about how this ecosystem functions, whether it's as effective as Google says it is, or as ineffective as researchers like Steve Dedalus told us earlier about his research at eBay. eBay believe that for every dollar they're spending, they're getting 50 cents of net profits. And what we showed is that on average, they're losing more than 60 cents on every dollar.

So, OK, let's try to understand this ecosystem better. First, we will need a guide. Sure, my name is Tim Quong H.W. Anji, my day job is I'm a research fellow at the Center for Security and Emerging Technology at Georgetown.

And before that, I was previously a global head of public policy for AI and Machine Learning at Google.

Wang recently published a book called Subprime Attention, Crisis Advertising and the Time Bomb at the Heart of the Internet. It's about how big tech monetizes our attention.

When I started to do research by very naturally started to talk to a couple of friends who work at these big tech companies. And it was a little bit like talking to someone who works in national security or the intelligence community or something like that, because they would be like, oh adds definitely work, but we can't tell you how or why or give you any evidence for it.

Google would plainly dispute that there is no evidence for whether online ads work. Tim Wong recognizes he is tilting at windmills here, trillion dollar windmills.

But in fact, he first grew skeptical about online advertising while still working at Google. He began reading trade journals and going to conferences.

And I had this fascinating experience with one of these keynotes at this conference was given by Nico Newman, who basically is a big ad critic.

Nico Neumann is a marketing professor at the Melbourne Business School in Australia, and he presented to really fascinating studies that his lab had done. The first one was looking into the quality of data used in the ad tech industry, basically demonstrating in many cases it was incredibly inaccurate. And the second one was he took dead aim at the hype cycle around A.I. that exists in ad tech right now, where people are saying, if you have this latest machine learning, you have this A.I., you'll be able to do targeting in a way that you never, ever were able to do before.

And Niko's lab did some experiments that demonstrated that in many cases, machine learning was finding people who would have bought the product anyways.

Bringing this message to an ad tech conference is a bit like bringing a safety pin to a balloon conference that looked around being like we.

So would people be angry? And it was just total dead air.

No one responded, no one engaged with it.

And it got me really interested in thinking about, like, is there a bubble here, a bubble like the dotcom bubble or the subprime bubble or the tulip bubble?

Because this is exactly the kinds of behaviour that occur in other financial bubbles where the red lights are flashing. But everybody in the industry just refuses to take a look at the real data.

Wang began thinking about how bubbles happen.

So the origins of every bubble come in this gap that occurs in a marketplace.

On one hand, you have people who believe that an asset, whether it's collateralized debt obligations or advertising inventory, is extremely valuable.

And then on the other hand, what you have is declining asset value. So in the subprime mortgage crisis, we believed that mortgages were always going to just pay out regularly forever. Right. When it actually turned out that the package of mortgages were actually a terrible asset, they were toxic and about to go belly up.

So how can one justify a parallel with digital advertising? I think the first piece is really the big question of do people ever see ads at all? So Google actually did a fascinating study not too long ago, which concluded that close to 60 percent of ads on the Internet are never, ever even seen the ad is delivered.

But that just ends up in some dumb part of the page, below the fold. It's along a sideline.

But what about the precise targeting? The digital ads are supposed to offer a 2019 study. This one, done by three academic researchers, addressed this question by measuring the impact of a user's cookies. Those, remember, are the tracking codes that most of us allow to roam our computers and phones in exchange for all the free information we get from companies like Google and Facebook. This study found that when a user's cookies were unavailable, ad revenues only dropped by about four percent.

Why would cookies be so ineffective?

Tim Wong argues that people pay a lot less attention to online ads, and they used to.

People often forget that when banner ads were first launched on the Internet, there click through rate was like 50 percent, completely mind bending. Right. And it just continued to fall and fall and fall. And now it's like zero point zero one 2.0, three percent.

Some estimates of clickthrough rates are higher than what Huang cites here.

That said, precise measurement is hard because there are so many bots clicking on ads, a whole other problem with the digital ad universe.

But no matter how you measure it, click through. Rates have fallen a lot as the novelty wears off, habituation sets in, and an ad that might have once grabbed your attention becomes invisible or worse, annoying. People increasingly don't want ads.

So ad blocking, for example, is really. Increasing over time, and I think these factors not being able to see adds the questions about the effectiveness of ads and the rise of things like ad blocking, bring into question whether this thing that we think is so valuable is actually worth as much as we think it is.

But if there's such a big gap between the perceived and real value of digital advertising, why are Google and Facebook worth so much money? Look at it this way. There are a couple trillion reasons why Tim Hwang might be wrong, but he doesn't think so. His theory is that digital advertising is grotesquely overvalued because it is still so hard to measure. And one reason it's hard to measure is that the marketplace is exceedingly opaque.

So there's a fascinating incident that I always think about, which is one of the last times that Mark Zuckerberg was called up to Congress. And one of the questions that he got from one of the senators was, well, how do you guys make money?

How do you sustain a business model in which users don't pay for your service?

And Mark Zuckerberg was like, Senator, we've run ads.

And at the time, a lot of the chatter on Twitter was like, ha ha, look at the super old senator. He doesn't know anything about the Internet.

But it's true that even if you talk to people in the tech industry and you're like, OK, level with me.

Joe Engineer, how do ads work on the Internet? It's kind of a rumor like we know this is how the business model works, but no one can really explain how it works in detail. So when I say advertising, a lot of people normally think of like Mad Men. Right. But it really looks like what the Nasdaq looks like, which is largely automated system that moves millions and millions and billions of pieces of ad inventory on a daily basis.

As Steve Tudela explained earlier, most ad inventory is sold by auctions, which are run by algorithms operating at phenomenal speed.

This is one contributor to the capacity of the industry. For instance, it can be hard to figure out why certain ads end up on certain pages. If you are a family friendly brand like Disney, you don't want your ad popping up on a YouTube video showing a terrorist beheading.

This has been a source of chronic embarrassment to the ad industry.

The prevention of this kind of ad mismatch is known as brand safety.

And despite the greatest efforts at trying to eliminate the risk of brand safety from the ad market, people by and large just haven't been able to.

Again, the industry itself would disagree.

We asked Google how they ensure an ad doesn't show up on a page promoting misinformation or a conspiracy theory. Here's their reply.

We have strict policies that govern what kind of content we place ads on. And if we find a page or website that violates our policies, we take immediate action.

In 2019, Keith Wead, the former marketing boss at Unilever, helped create a consortium called the Global Alliance for Responsible Media, which pushes for better ad controls to protect brand safety.

As they say, everyone wants a well functioning Internet and everyone wants it to have a positive impact on the world and not to have some of the issues we're wrestling with right now. I think the path has not been easy so far this September.

After months of advertiser boycotts, Facebook, Twitter and YouTube agreed to adopt a common set of definitions for hate speech and develop tools to let advertisers have more control over where their ads show up.

But Tim Wong thinks the long standing opacity of the online ad marketplace is just one reason we might be in a digital advertising bubble.

I think a second thing is a little bit like in the subprime mortgage crisis. You do have people who have very perverse incentives, I think, to push the effectiveness of online ads. That's the ad agencies, the ad platforms themselves, the people who run ad technology. All these people, I think, have a very strong incentive to say, no, this stuff is way better than earlier generations of advertising, and this is why you should use it.

If you've been listening closely, you will notice this is the exact same problem Steve Levitt talked about regarding the TV ad ecosystem. Human beings generally make decisions based on self-interest.

No chief marketing officer is ever going to say, hey, I don't know, maybe ads don't work, let's just not do them and see what happens.

Or as the author Upton Sinclair once wrote, it is difficult to get a man to understand something when his salary depends on his not understanding it.

So there is a common practice which is not very well disclosed in the ad industry, whereby an ad tech company will basically offer ad inventory at a cheap price to the advertising agency.

The agency, remember, is paid by you, the client who hired them to help you sell what you're selling and then the agency.

Turn around and say you should really use this ad tech product and sell it at a higher price. And what are the worries about this is that it changes the incentives. Right. Which is typically the ad agency should be working on behalf of the client.

But in these cases, they're very perverse incentives to push up distribution of a message that may not otherwise be rational or even useful to them.

All of these issues and all the new empirical evidence we've been discussing about the ineffectiveness of advertising has persuaded Tim Wong that, yes, the online ad marketplace is a bubble and it might soon pop. In fact, the deflation may have already begun.

A few years back, Procter & Gamble, which is one of the largest advertisers in the world, decided that they would run a little experiment. They were going to take about 200 million dollars of their digital ad spending and just cut it out of their budget to see what happens.

Procter and Gamble said they were doing this because of concerns over brand safety and the proliferation of bots, which can pollute the data on ad impressions.

And the end result was fascinating. Basically, they said that there was no noticeable impact on their bottom line. Again, the ad industry will have a lot of explanations for why this might be or for why there is a lot of value in advertising beyond short term sales figures.

But Procter and Gamble is a big player, even if they are wrong, even to a small degree, they're the ones whose money drives the advertising ecosystem. What would happen if this turned into a mass movement among advertisers? One shouldn't underestimate the size and reach of the advertising ecosystem.

The sports you watch on TV supported by ads, the journalism you consume, supported by ads, at least much of it.

Google Maps and Google Drive.

And, well, Google supported by ads as well as Facebook and Instagram and Twitter, nearly everything else you consume online and don't pay for, including this podcast and just about every other podcast you listen to. Advertising is also important for Tim Wong, whose day job is researching artificial intelligence and machine learning.

Some of the most cutting edge research in the world is being funded by ads, right?

If you think machine learning an ad is going to have a huge impact, a huge impact in the world, that is from autonomous travel to medical research and diagnosis, you may want to think about what it means that most of it is subsidized through this ad infrastructure.

So there's lots and lots of links to the economy that are not always obvious, but I think are worth thinking through because it points out how widespread a downturn could be.

So if there is an advertising bubble, can it be deflated in a slow, controlled way to avoid a massive economic unraveling?

Tim Wong thinks so. But change won't come from the industry players. They have too much incentive to keep selling. It's got to be driven by the buyers.

One of the groups that has the most to lose are people who theoretically might be wasting a lot of money on advertising.

But Wang does not think the buyside pressure will be enough, because I think one of the biggest problems in the space is that there's no objective third party evaluator of some of these claims claims that is about ad efficacy.

And so what I advise is kind of a punk rock. NBER, the NBER is the National Bureau of Economic Research. What does Wongs punk rock version look like?

It's basically a research group that is willing to be a little bit of a troll to the advertising industry. And so, again, how do we throw off the veil, reduce the opacity in the marketplace? You really need a dedicated group of people who are doing good research on this front. What you want is a handbrake where you can slowly bring down the momentum in the market so that it can deflate without exploding.

There are, of course, many people and institutions already doing research on advertising spending. But let's be honest, most of them have someone's thumb on the scale. And conventional wisdom isn't the only thing that someone like Tim Wong is challenging.

It's also the billions of advertising dollars that drive trillions of dollars of market value.

So it takes some courage to suggest that those billions and trillions may not be kosher.

One of my favorite arguments that people are using right now is, you know, companies wouldn't put money into this if it didn't work. So isn't that proof that the ads actually work?

That, in fact, is exactly what we heard earlier from the Unilever veteran, Keith Weed.

The fact that Coke and Dove and Ford have been around for decades and the fact that companies like Unilever spend billions suggests that maybe advertising does work, which is kind of this crazy circular mind maze, if you think about it.

But I do think that, again, this is very parallel to the kinds of psychology that had driven market bubbles in the past.

One reason to suspect that ads do work well is the underlying assumption that firms like Unilever who buy so much advertising are as Econ. one on one textbooks tell us profit maximizers. So why would they waste so much money?

Any economist that tells you that firms are profit maximizing is not ever worked with firms.

That again is Steve Levitt.

The realistic picture is that firms are composed of people and all of the foibles and shortcomings that people exhibit in their everyday life. They bring those to work with them.

We ask Steve to Dellis, the Berkeley economist who worked for a time at eBay, what he thought of Levitt's take on the nonprofit maximizing behavior of allegedly profit maximizing firms.

As an economist hearing you say that causes my stomach to hurt.

But at the same time, I know that you are absolutely correct to Dellis by the end of his time at eBay had come to think that his cynical view of advertising.

Didn't go far enough. He recalls the time eBay asked him to measure the efficacy of affiliate advertising.

Think of bloggers who put in links to, say, company websites. Well, we worked closely with the senior director in charge of spending the money on that. And after a two hour intensive meeting, we figured out a way to do that, to do that, meaning to measure whether these affiliate links were really worth buying.

He turned to me and said, you know, Steve, if your results look as bad as they did for paid search, I'm not going to believe your numbers.

Now, I was obviously shocked because it made me realize that religion and not science is what's winning this battle. But then I realized that it's something a lot more profound and for which I actually have a lot of compassion if you're working on something for 10, 20, 25 years. This is part of your identity and this is part of what you believe in. And if I'm going to prove that what you thought works so well in ways that you don't quite understand because you're not a statistician or a kind of attrition and you have to take it at face value, what are you going to believe your gut that tells you that what you've been doing for the last 20 years is really influential or some egghead academic that's showing you a bunch of equations that you don't understand and is claiming that you're wrong.

This digital advertising issue is just part of a bigger conversation about the power of modern technology companies.

For their first few decades, they were pretty much given free reign, but now they are facing scrutiny over the breadth and depth of their power, power both seen and unseen.

The U.S. government has brought a major antitrust case against Google, Facebook, YouTube, Twitter and frankly, thousands of digital platforms and repositories stand accused of promoting misinformation and or mishandling user information.

Given all that, society probably deserves a better answer than a lot of companies pay a lot of money for advertising, so it must work.

I think the question isn't necessarily do you want an Internet with ads or without ads?

That, again, is Tim Wong.

The question is, do you want an Internet that's just based on a huge monoculture that's largely funded through ads? Were the most powerful companies use ads and where VCs don't choose to invest if you don't use ads. Right. And I humbly suggest like, no, the kind of Internet that I want to see is Internet that has a bigger diversity of business models and where ads don't suck all the air out of alternative business models. I just think that's a much more robust market.

I think it's a more stable market over time. And I think it's one that leads to much better outcomes socially. And so I think that's my vote, which is an Internet that accepts many different ways of making money.

We should also say that just as there are different ways of making money, there are many, many, many different forms and styles and purposes of advertising. Over these past two episodes, we have talked about two of the biggest ad marketplaces, TV and digital. But there are, of course, many other advertising channels.

And within all these channels are countless variations, local ads versus regional or national ads, calls to action versus brand building ads about price versus ads, about quality ads for new products or services versus established ones.

Ads meant to inspire or entertain versus ads just deliver information personally. Those are the ads I respond to most learning some information I didn't know about some product or service that sounds useful or fun for me or my family.

I can think of at least eight or 10 things I bought after first learning about them by voicing the advertisements that go on this podcast. Our business partners have told us that one reason the podcast industry has grown so much lately is because advertising is particularly effective on podcasts, given the intimacy of the medium.

I have no idea if this is really true, but I want to believe it is.

Why wouldn't I? It's my livelihood.

So you could say this puts me on both sides of the fence. True believer and skeptic.

At the same time, one might say this is a sign of intellectual laziness to move right out of Upton Sinclair.

But I'm going to go with F. Scott Fitzgerald instead. Here's how he once put it. The test of a first rate intelligence is the ability to hold two opposed ideas in mind at the same time and still retain the ability to function.

Assuming I retain that ability for at least a bit longer, we will be back next week with another episode of Freakonomics Radio. Until then, take care of yourself and if you can, someone else do. Freakonomics Radio is produced by Stitcher and Dubner Productions. This episode was produced by Daphne Chen. Our staff also includes Allison Craig Lowe, Mark McCluskey, Greg Rippin, Zach Buchinsky, Mary Duke and Matt Hickey. Our intern is immaterial and we had help this week from James Foster.

Our theme song is Mr. Fortune by The Hitchhiker's. All the other music was composed by Louis Scare. You can get Freakonomics Radio on any podcast app.

If you want the entire back catalog, use the Stitcher app or go to Freakonomics Dotcom. Freakonomics Radio can also be heard on many NPR stations around the country. As always, thanks for listening. And I will sort of measure the success of the book based on how much it pisses everybody off, and on that front, I'm glad to report that we definitely have had ADTECH people being like, this guy's an idiot. That's just music to my ears.

Stitcher.