A collectivist, economic perspective on AI

Welcome to the LSE events podcast by the London School of Economics and Political Science. Get ready to hear from some of the most influential international figures in the social sciences.

Hello. Good evening, everyone, and welcome to LSC. Well, welcome also to those joining us online through LSC Live. I'm Mila Wojnovich. I'm professor of data science with head of department of the department statistics.

It's my pleasure to chair the this evening's lecture hosted by our department. Maybe a couple of notes. This evening evening is being live streamed and recorded. And the hashtag hashtag is LSEvents. And you're all invited to a drinks reception later afterwards, after the talk in the staff common room.

So please do join. I'll say maybe a few things just to give a background for the lecture. So much of the recent conversation about AI has focused on building a single, ever more capable machine. Tonight's speaker asked us to widen the lens to think not only about individual agents, but but the collective it operates within. So the people, institutions, markets, and the incentives that surround it that brings together statistics, computer science, and economics.

So which makes it a very LSE question. Professor Michael Jordan, he's a director of research at India Paris and distinguished professor emeritus at the University of California Berkeley for over 4 decades. He has been 1 of the leading figures in computer science and statistics, and his work has shaped how modern machine learning is done. He's a member of the US National Academy of Sciences and a foreign member of both the Royal Society and the Chinese Academy of Sciences. Our discussion is professor Maximilian Kazi, professor of economics at the University of Oxford, where he coordinates the machine learning and economics group.

His research ranges from economic inequality and basic income to the intersection of machine learning and economics, so an ideal vantage point for tonight's discussion. Professor Jordan will speak 1st, followed by a response from Professor Kazi, and then we'll open up to questions from the room and online. So with this, please join me in welcoming Professor Michael Jordan.

Alright. Thank you very much. It's a delight to be here. So something is happening in our era, and I don't know exactly what, and I'm not going to try to define any of the terms in that title. I think starting with definitions is a very bad idea at the present time.

Things are changing and old ideas are probably not going to apply. And probably the only thing in my title I think I'd stand behind is the word A. This is not the perspective, this is A perspective. But anyway, I hope you'll get an idea of what I mean by some of these terms as we go. And again, let's not start with any preconceived notion.

Certainly, AI, which now is the chatbot, it's terrible terminology. It's not a thing. It's not a technology. It's in some sense a way of thinking, but even that's not very helpful. And it's certainly not a well defined technology like we've seen in the past, but something is happening in our era.

So let's get our hands on kind of what that might be. 1st So of all, I don't think it's good to start with a technology. We've developed this thing, like a nuclear reactor, it'll do this, and what's the implications of that? It's not that. You really want to start with what are the kind of problems we're trying to solve in our era that need technological support.

We have 7,000,000,000 people on the planet, we need some support. And the kind of problems that we often have to worry about are things like transportation and finance and commerce and healthcare and so on. If you start with that, you'd say, well, who are the stakeholders? Well, there's people and there's devices and there's decisions and there's blah blah blah. We have this big set of things that interact in all kinds of complicated ways.

In the past, it was often designed very ad hoc or maybe there's a little bit of economic theory that had some crossing curves to tell you how to operate and so on. But now data informs all of this, and there's flows among people, there's different self interests and all that. So it becomes a very complicated, dynamical system. AI is sort of that. Right?

AI is not a mathematical idea. It's not optimization. It's not data. It's it's something had to do with these systems and how they operate and, what what are the consequences of that. All right.

So if you want to see that little diagram on the right, it's kind of the scope of the discussion. We have a huge federated network of people and machines. So those nodes in there, some of them are people, some of them are machines. There's not 1 big machine that everyone talks to. That's not a very helpful way to think, the super intelligence that we'll all get all the answers from.

Rather, it's all interlaced with all of us, and it knows certain things and it doesn't know other things. It can't know everything. It can't. We'll get into that more as we go. Right?

And so some of those are people, and they're going to be interacting in complex ways with everything around them, which includes some machines, includes other people and all that. And it's a very complicated environment to be in. But think about that person and then think about the overall goal of the system. The goal is to have everybody have good health care or everybody have good transportation and services. That's really the better level to think at.

But then you've got to go back and think about the individual person. What are they faced with? And I'm going to use the word market throughout this talk because I think that's about the best thing I can think of. When I go into a brand new market, say, Rio De Janeiro, I know nothing about anything around me. It's very complicated, all kinds of things hitting me, but I have some goal, And the overall thing is operating in some way to transfer goods among people, and I sort of start to trust it.

But I have to kind of think on my feet. Alright? And that's not a bad metaphor for where we're at right now. The overall system should work well even though each individual doesn't really quite know what they're doing. That sounds like economics to me.

K? But it's also a lot of statistics because data is flowing everywhere, and you got to analyze it on the fly and all that. That doesn't sound like economics to me. Okay? And a lot of computer science because all this data is on these networks that have been built up by computer science.

Okay. Let's talk about this individual now. Okay? The overall goal is not to build a super intelligence to solve all the world problems. The overall goal is to do transportation or healthcare with that.

Keep that in your mind. Alright? I think it's always helpful to think about a specific example there, but let's go up above that. Now think about an individual inside of that. They're faced with all these things around them.

Other people, data is flowing, they can ask queries and get answers, and they have to operate in that environment. Okay? Now it's very noisy or stochastic. Okay? Now when things are noisy like that, you have to make a decision, you worry about making bad decisions.

Right? And 1 way you make bad decisions is everything is just so variable. It's like the stock market. Another way is things are biased, and that's the more important problem for our day. It's not that things are so variable, it's that things are biased around us.

Is biased good or bad? Mostly kind of good. Bias means someone has an opinion, maybe they thought about it, they have a bias towards this answer because it's the better answer. But it's not always good. So let's dive into what it means.

Alright, so as I emphasized, there's multiple perspectives on this big system, economics perspective, statistics perspective, computer science perspective. Let's start with a statistical perspective on bias. Now statistics, bias was usually viewed as a bad thing. If your estimator has bias, make it unbiased somehow. Right?

That went away at some point, and it's really going away now. Our estimators are always terribly biased. We have to cope with that, not just wish it away. Alright. So I'm going take an example to sort of focus attention here.

This is AlphaFold, Nobel winning prize project done here in London. It will take a sequence of amino acids and predict the structure of the folded protein that results. And it does it better than anybody, better than in humans, better than anybody. It's an early example of an intelligent system for a specific task, not for all tasks, but for a good... And did a great job.

Alright. So with AlphaFold, you can put in any old sequence you want and get out a predictive structure, and it's really good. So those structures can almost be treated as data. You can do science with them, and that's often called AI for science. And in fact, if you look at structures that are tested in the lab, there's many fewer of them.

There's about a 100,000 because it's laborious. And so you've only got a 100,000 gold standard examples, whereas AlphaFold can generate hundreds of millions. Alright. Okay. So let's think about it.

Mostly, this is a good thing, but it can be... It is problematic. So this is a very busy slide, but what I want to convey is the 1st thing up here on the 1st of all. This was a study done in 2004, some scientists were studying whether proteins that had a lot of quantum fluctuation in them were actually used by biology. What does this mean?

Well, proteins are these things that we ask is mostly they like to fold up into a little globule thing, and they act like enzymes or structures in cells. But sometimes there's some quantum fluctuations, things don't bind quite right, and little pairs fall off, you know, come off of them. K? You might think that biology would say that's a bad protein, let's not use that. Or you might think that biology is clever, it might have found a way to use them.

And probably the latter is going to be... Is true. If you put many of these together, all these hairs could kind of link up with each other and form a little grid, and you can put that in a membrane and that leaks in and out of the cell. Probably what what is really true. In 2004, people were interested in studying that.

So they took the entire set of 10,000 at that time, known protein structures, and they did a little statistical test on this whole big corpus. What does that mean? They formed a little 2 by 2 table, 0 1 0 1. 0 meant no quantum fluctuations, 1 meant quantum fluctuations. The other dimension was used in the cell, yes or no.

Okay. That's called phosphorylization. Okay? And they took that 10,000 data and they reduced to this little table, and they calculated what's called an odds ratio. You took the diagonal divided by the off diagonal.

If that odds ratio is equal to 1, there's no association between these 2 things. Okay? Meaning that quantum fluctuation protein is probably not used in the cell. If it's bigger than 1, there's an association. Alright.

They studied this and they got that odds ratio, it was some number, it was bigger than 1 but not very much. And a good statistician doesn't just take the number, they form a little confidence interval. Alright? Under repeated samples, how big would my confidence be? And that confidence interval was big enough that it covered 1.

Meaning you couldn't reject the idea that there's no association. So that's what they said in that paper. 20 years later, people come along and say, well, now we've got alpha fold. It doesn't limit us to just those 10,000 proteins, we can just generate as many as we want. So let's just generate all these up here, 200,000,000.

We put that into the same little 2 by 2 table, an amazing reduction of data. We get the off diagonal against the diagonal, we get our odds ratio, and we get a confidence interval. Turned out it was wildly bigger than 1, and so they could reject hypothesis. There was the quantum fluctuation proteins where you'd be used in the cell. Okay?

That sounds like real progress. That sounds like the way to do science. And those are so many proteins here relative to here that this just you could have they didn't even use this. It just washes out. You don't need the real data.

AI for science can just use AI to generate data. Okay, here's the problem. All right, there's another sort of slightly busy slide, but this is my statistics slide. Okay, Here's the odds ratio, there it is actually defined if you care. 1 would mean no association, that's called the null hypothesis.

You get data and you get a number. In this case, it turned out to be 3 and a confidence interval around that. This confidence interval is very narrow, meaning you're quite sure of yourself. You're saying the true odds ratio is around 3 based on using alpha fold. Right?

That's way bigger than 1, so they could reject the null hypothesis. Here's the problem. We redid this entire experiment and we took 0.5 of the data and we held it aside and we calculated the ground truth in a big Monte Carlo run-in that 0.5. And then we took the other data and we redid the whole experiment. So here's the kind of proxy for ground truth.

The odds ratio in that 0.5 looked at the records around 2. And here's the problem, which is that the alpha fold generated confidence interval is not even close to covering 2. It should be covering the truth. It's not. This is bias and it's large bias.

Alright? So we did this. This is a paper in science and many other domains, astronomy, some econometrics problems, a whole bunch of them. This happened all the time. The machine learning model would generate a prediction and a confidence interval if you did your work, and that confidence interval will be extremely narrow.

The machine is quite sure of itself. And if you've interacted with alpha... With... Not AlphaFold, but ChatGPT, you can sort of get that sense. ChatGPT is quite sure of itself.

Alright? The problem is that it's very far from the truth. If you say forget alpha fold, let's just go back to classical statistics, we have now not just 10,000 proteins, we've a 100,000. It takes time, but we're up to a 100,000. Get a confidence rule based on that, that's the gray region.

So it's much bigger, it covers the truth like a good statistician will promise you it will do, so it's not biased. Alright? But it covers 1 also. So we still, after 20 years, can't reject the null hypothesis. Science was slow.

Alright. So this is a problem and it happens all the time. Alright. And and... But in my line of work, you try to find a problem, you try to solve it.

And we developed a method we call prediction powered inference. What it basically does is it takes AlphaFold and uses it as a data source, but it says AlphaFold is probably biased for my problem. Not over all problems, but over my problem, it's probably biased. What I'm going do is gather a little bit of gold standard data for my problem, maybe a 100 data points. Don't need much.

Now if you just added that into AlphaFold, it would wash out completely. Prediction matter is a different technique. It's a clever statistical way of taking a 100 good data points and then taking all of AlphaFold and merging the 2 in a smart way. So what happens is that this confidence interval shifts over to cover the truth and it gets bigger. Alright.

And so the green region is the PPI interval, and you see it covers the truth as it's probably will, is supposed to do, and it's much narrower than the classical statistical 1, and it doesn't cover 1, so you can finally reject the null hypothesis. So I'm not going describe how PPI works in this talk, but it's a paper in science, and there's like 2 pages of mathematics that tells us it's not that complicated. On the other hand, it didn't occur in previous, you know, decades of statistics. It's a bit of a new idea. Now I think this is going to happen all the time.

Here's another example that shows you where it hap... Another example where it happens. This is a particular econometrics problem of predicting a regression coefficient of income, and there's a machine learning model, and machine learning model is extremely sure of itself, again, and not even close to the truth, even further. Alright? So it's so biased that when you do the PPI correction of it, you cover the truth, but you're also not that sure of yourself, not that much better than the classical 1.

This is good. This is you're being honest. You say, got a machine learning model which pretends it's really sure of itself, but when I correct it appropriately, I debias it, I'm clear that it's a bad model. And the bad model comes in the size of this this conference interval. Now why is this happening?

Isn't AlphaFold really, really good? Yeah. It is really, really good. But here's the here's the problem. Here's a particular protein.

That's the prediction from AlphaFold, all nice and folded up. Here's the actual experimental structure. We didn't know that before we did the study, but we went looking and we found this. Right? What's happening is there's quantum fluctuation here.

Why is this happening? Why is alpha fold so bad? Well, there's not a lot of data in the training dataset of these quantum fluctuation proteins because people weren't all that interested in them. K? The dataset is full of everything that people already studied a lot in the past.

The problem is that scientists are rarely interested in doing queries about what happened in the past. They want to be on the edge of knowledge. So they will ask questions that are hard ones on the edge of knowledge. On the edge of knowledge, you'll get a very strong prediction with very small error bar. Error bar goes as 1 over square root of the number of data points.

If you have 200,000,000, 1 over square root of 200,000,000 is really, really small, so you get these tiny confidence intervals. You're very sure of yourself. But if you're on the edge of knowledge, you're probably going be quite wrong. That's what's happening. And I hope you agree that that's not going to be uncommon.

Scientists are always on the edge of knowledge, but not just scientists. You and me, when we go into any kind of market, we're trying to do something new. We're going be on the edge of knowledge. Our little LLM will be biased. Alright.

So a good way to think about this is that this is a problem, be aware of it, take the advantage of the LLM or the chatbot or whatever, but don't trust it completely. Always add in a little bit of local knowledge that you have that the LLM doesn't have. Now, if you then provide it to the LLM, eventually it'll get better, but waiting is a fool's errand because you always want to be on the edge of knowledge. Okay. So that's the statistical problem.

There's a solution which we call PPI, which I encourage you to look at if you're interested. Alright. Now let's turn to an economics perspective on bias. So the statistician's perspective is basically, it's something you could somehow correct for if you did smart things, and variance could go away if you just got enough samples. Alright?

Economics has that perspective too in econometrics, get a lot of samples and correct for things and all that. But also economics has another perspective which has to do with information asymmetry. Okay? So let's talk about again a concrete problem. Here's our friend, the FDA.

It's a regulatory body in The US that tests whether a new drug is good and should go to market or not. Okay? And this cost many, many tens of millions of dollars or euros or pounds or however you want for each therapeutic area of medicine and over the many phases of trials every year, huge societal investment. Why does it cost so much money? Well, these are randomized controlled trials.

You take 20,000 people, 0.5 of them you randomly give the medication that's proposed, the other 0.5 you give them sugar water and you compare how good they are after the treatment. Okay? And it costs a lot of money to do that with 20,000 people many many times and so on, but it works. The vaccines and so on that you've all profit from are are the example. Alright.

Now is this system foolproof? Is it really working well? It gets better over time, but there's problem. Alright. So again, a little bit of a technical statistician slide here, but let's just simplify and say there are...

A new drug is coming in. So I'm the FDA. My job is to test that drug, that proposed medication. Should it go to market? Yes or no.

So I'm being a statistician, and I'm going do a test by getting 10,000 people, and 0.5 of them get the medication, 0.5 of them don't. It'll have randomness in it, and I'll control all that. Now, if the drug is actually not good, bad means that it doesn't necessarily harm people. It's kind of been screened for that. It means it's just not effective.

It doesn't do you any good. Alright? Good means it really helps you. So I'd like to decide if it's good, I really want it to go to market, otherwise not. So I get random data, and so I have to make a decision that's probabilistic.

So my probability of approval in this case should be small. That's called the false positive rate. Want it to be, like, 0.05 or less. If it's actually a good drug, my probability of should be big, like 0.8. That's the power.

I want that to be a big number. And status agents are really good at making those numbers really work out even in very complicated situations with brand new medications, different kinds of people, all kinds of confounding factors and all that. They really do a good job with all that. K? The problem is that still lots of bad drugs go to market.

How can that be? Well, there's an economic side to it. Let's consider 2 cases. Case 1, a small profit is to be made. It costs you 20,000,000 to run the trial, and who pays that?

Well, now we have to identify there's multiple players in this game, not just the FDA. The medication that's being proposed comes from a pharmaceutical company. They have their own interests. They want to make money. They want to make a good drug, go to market, good reputation too, but they want to make money.

Okay? So they have to pay for the clinical trial. They pay the $20,000,000 if they want to play the game, right? Then if they're improved in this case, let's imagine they would make $200,000,000 in a year, so a small niche drug of some kind. Now if you take these numbers and these numbers and put them all together, the CEO can do a calculation.

She can say, if my drug is actually not good, I don't know that, but let's imagine counter factually that it's not a good drug, then my expected profit will be minus 10,000,000. Just... You can do the calculation yourself. She will look that number and say, okay, people in my company, let's only send candidates into the regulatory agency if we're really convinced they're very good. Otherwise, we're going to be losing money.

Now how do they know it's really good? No one's done the clinical trial yet. How can they know? Well, this is what a Bayesian would call a prior. They've had some experience with this kind of class of medications.

They put their best engineers on it, so on and so forth. Okay? So they can do that calculation and realize they should only send a handful that are really good to the FDA. Now the FDA doesn't have that prior knowledge. The company is not motivated to tell them this, but it is a fact that the FDA is getting mostly good candidates.

And then if they do the good statistical stuff up here, only good drugs will go to market. Problem is that case 2 is more common. Case 2, you cost $20,000,000 to run the trial and if you're approved maybe you make $2,000,000,000 more like ibuprofen or something, All right. In this case, you do the same calculation and the CEO will realize my expected profit even if my drug is no good is 80,000,000 positive. How can that be?

Well, it's this number right here, false positive rate. If I flood the FDA with tons of candidates, some of them will sneak through if they're not any good, and those will make me a lot of money, and they'll be on the market for a couple of years until people realize they're not really helping. That's what's happening all the time. So we've got an incentive misalignment here. Okay?

You could say to the company, you shouldn't be sending in all your candidates, only sending the good ones. And the company will say, we're doing our best. Who are you to tell us what to do? It's your job to screen these things. Alright?

And they'll look at the candidates and they'll say, well, this we're not so sure about this 1, but it's not a very risky bet. Let's just go ahead and try it. And maybe we're wrong. Maybe it's actually good. We should let the FDA figure that out.

So you could debate whether it's an ethical issue or not and whether profits are the right mechanism, but this is what happens. This is what companies do. Then it's partly because they're unsure. They have to... They're putting their bets out like everybody else.

Anyway, long story short, there's a problem here. Statistics is being done and economics is being neglected, and this is microeconomics. All right? And what is happening here is there is a form of information asymmetry. All right?

So in economics language, we have a principle, that's the FDA. They want to do a job, which is to get a good medication to go to market. Agent, that's the pharmaceutical company. They want to do a good job, which is to brand and find new brand new molecules that have medical effects and get them out to market and make money, stay in business. They know things that the principal doesn't know.

If the principal says, tell us all of your knowledge, they will lie. They will say, here's my candidate, it's really good. Trust me. Because then they'll say, okay, we'll test it and maybe, you know, blah blah blah. So this happens all the time.

If you're an economist, you recognize this information asymmetry. If I'm trying to hire 1 of you to to paint my house, I will go to you and say, you know, here's here's my house. How long is it going to take you? And how good are you? Alright.

And you know in your head it's going take 10 hours and you say it's going take 100 hours. Okay? But I know that you're going to do that. Alright? You're incentivized to exaggerate, so I need to build a little market here.

I've got to maybe not just ask you but ask some other people, then I've to have some little incentive structure. I've to maybe do an auction. That's a mechanism. Alright? So...

And why am I doing that? Because there's an information asymmetry. You know how long it'll take and I don't. But I want to get the job done. You're not the 1 trying to get my house painted.

Okay? So I got to set some prices and you've got to give me some information, but there'll always be this asymmetry. So this is not a statistical source of uncertainty. This will be, just do this again and again and again, and eventually I'll figure out the truth. That's not happening here.

Could ask you the same question again and you'll keep lying. Hundred hours. Hundred hours. So I've to develop an economic mechanism. Now, I've got to do this in the context of a statistical test.

That's different from classical economics. Classical economics is that you work, I pay you, you make money, I get something done. Revenue, welfare. Here, the goal is statistical data analysis, but in an incentivized way. Okay.

So let's call this a contract theory problem. We want this person to write a contract for the agent, so the agent is incentivized to send in mostly good drugs. How could we possibly do that? Well, there's this thing called the theory of incentives, and 1 branch of that is contract theory, the other branch is auction theory. And these correspond to 2 branches of game theory.

They're kind of the inverses of Nash equilibria and Stack Overg equilibria, if you know what that all means. And contracts are ways of trying to take account the fact that customers have different willingness to pay. Alright. So I like to tell this story thinking about the airlines. When I was young, you go to get on a plane, like in the eighties, and you pay 1 price for the entire plane, like going to a movie theater.

Okay? €100. Alright? And of course, what happened is that some days the plane would be empty because not enough people were interested, and other days the plane would be aligned after that, and the and the airplanes was not making much money. Revenue was was low.

And so airlines started to go out of business and many of them did. Pan Am and Eastern and Bran have all went out of business because of this kind of issue. In some countries, the National Alliance didn't go out of business like France, they just because Air France could not be allowed to go out of business, the government supported them. Right. So how would you fix this?

Well, what's happening? Well, on any given day, certain people really want to go from London to Rome. Okay? They have a high willingness to pay. They pay 1000 euros to go there.

Okay? Because there's some family reason or some other... They really got to get there. They'll pay a lot. Right?

Other people, no. Okay? Now, if I looked at them and I said, I just look at you, you, I can tell, have a high willingness to pay. I'll charge you €900. You'll say, okay, that gives me a little surplus, I'll pay.

And I didn't do that to you because I see you're not so willing to go, I'll say, you know, a €100 for you. You'll say, that's great, I'll pay. The airplane will be full now, but I'll get some high paying customers and some low. Alright. So a lot of people filled the airplane, that's called high social welfare, and the company made enough money to keep in business.

That's high revenue. Here's the problem. I can't look at you and decide how much you're willing to pay. That's an economics problem. Now, the LLM AI people would have you believe that eventually the LMs will be so good that they can make a prediction.

That's really good. They can look at you and say, based on your browsing history, based on everything I can see about you and the skin, you know, the sweat on forehead, I can tell you're willing to pay 1000 euros. Alright. That's bullshit. Alright.

The LM will never know these kind of things about us nor do we want them to. I know I want to go to Rome because I just got a phone call that suggested I should down be down there for some family reason. I don't want the LM to know that and the LM will Most things about willingness to pay, the LM will not know. AI will not solve the prediction problem in full generality in this kind of information asymmetry problem. Okay.

So what's the poor airline to do? They can't predict this. They can help. They can make it. But even if they can predict it, they don't really want to give different prices to different customers.

That's illegal. So what do they do? Well, you know what the answer is. They make this artificial little menu of different options, price, service, price, service. Different prices for different levels of service.

Right? And really what matters is there's a range of prices. The services are... There's this thing called business class that some of the students won't know about, but you go on and you can be 1st in line, you get a little glass of red wine, you have a big seat, and you feel really great about yourself. Alright?

But the students also feel good because they only paid a €100 and they're on the same airplane. So everyone feels good, that's high social welfare. And the airline got everybody on the plane. And here's the thing, high willingness to pay doesn't mean you're rich. It doesn't mean something that can necessarily predict about.

It just means on that day, you really have a high willingness to pay. So what's needed is that there's a population of people, and on some days, there's always a few who have high willingness to pay and a few that don't. And then I set a contract where I cover all of those different population with different levels of prices, and everybody can come in and find something for themselves. And the next day, it'll be different people, but the overall population will be good. So I hope you, if you didn't study this before, you get the flavor here.

All right? And this, of course, did spread to the airlines, it saved the airlines. You may not like business class and economy class, it's elite and all that, but it makes the airlines be in business. It's efficient. Alright.

Now how does that help with our FDA problem? Okay. Well, we want to have a menu that we give to the pharmaceutical company, not just 1 number. What's that going to mean? Well, we're to look at your...

The pharmacy is going say, well, there's different licensing terms. You could roll this out through the whole the whole country or on a limited case or you could... There's various caps and and, you know, various profit, you know, caps and various licensing terms we can give to you. Alright? And so it'll be kind of little bit of a cost benefit.

Now you can look at that list of licensing terms and also be a way to analyze data. You maybe you'll take not so much data, maybe you'll take more data that will cost more, but, know, you look at all this with your lawyer and you'll say, well, that... I I... My current candidate is is a drug that I really believe in. Now, don't...

No 1 else knows that, but I really believe in it. So I'm willing to bet that it's going to be good, and so I'll take very risky licensing terms, but high reward. High risk, high reward. Someone else will look at it and say, no, my candidate, I'm not so sure. I wanna do a little bit of a bet, but I don't want high risk.

Low reward, low risk. As long as all those options are covered, enough candidates will come in and they'll see something in there for them, and now the differences will allow us to trade off false positive against false negatives. We'll be able to not do revenue, we'll do statistical error control. Okay. So that may have been a little hard to follow, right?

But just to say, this can be made into a mathematical theory and we've done that. And what we basically do is we write down an economic goal, which is make sure that there's incentive compatibility. People who have something to offer are incentivized to play the game. They're not incentivized to walk away. All right?

The statistical goal is what's new, is that I want make sure the false positives are controlled and the power of detection is maximized. I want have a good statistical outcome. I want to put good drugs to market. That's not what you see in classical economics. But if you put those 2 things together, you can develop what we call a statistical contract.

And all that is, is what I just said. I give you a contract. It has a list of options of how your drug will be tested and how it will be rolled out if it's approved. And they'll have different costs and different options. Everybody gets to look at that same contract and they opt in and they pick an option.

Yes or no? If they picked an option and they opted in, they pay 20,000,000, all right, and then you gather data and that feeds into that contract is now evaluated on that data and you get your payoff. Okay. So that ultimately made into mathematics. And this theorem says that a cyclical contract is synthetic compatible and controls the falsified right if and only if these functions which are gathering data are what are called non negative supermartingales.

So here's the big math term in this talk. Probably some of you know what that means, others don't. It is a way to gather data over time such that if the null is true, things tend to decay away to 0, and if in fact the alternative is true, things tend to explode exponentially. K? So in statistics, these are called e values instead of p values.

It's a new way of thinking about statistics. And what we've proved in this theorem is that there's a relationship between statistics and economics at the foundations. Incentive compatibility and controlled errors if and only if non native supermarket. It's a way to gather data that is incentive compatible. Okay?

Alright. So long story short, the principle, the FDA can get their job done, they can regulate a market without knowing everything about anybody, without making perfect predictions. They could use predictions, but not perfect ones. And there's an incentivization to come in and play the game if you have something to offer to the market. And you're going be evaluated fairly and the overall market will tend to put good things on the market.

We've talked to the FDA about this and this is an ongoing discussion. Okay? Now, I use the FDA as an example here. I think it's a very good 1, but it's any situation in which there's an information asymmetry and there's large groups of agents on the 2 sides. It doesn't have to be just 1 principle.

Can be multiple principles, multiple agents. And there is an overall goal of making a good statistical outcome, okay, that all of us would share. We agree good drugs should go to market. Right? And we don't want people just to be saying, you're unethical if you don't, you know, do, you know, since and so forth.

We want the mechanism to to deliver that outcome. Okay. So that's what I wanted to say about bias, both statistical and economic. And I want to give you a couple of examples of how this is being used in my research and in my life in industry to build real markets that actually work in interesting ways. And so we're going to talk about learning where market design and this will be less technical.

I won't say a big word like not being super smart in this part talk. But hopefully, you'll find it interesting because this is what actually drove all the research. Here's my favorite 1. This drove a lot of my thinking. Music.

So I'm also a musician, and I'm very aware that musicians in this day and age don't make any money. A few superstar ones like Beyonce make a ton of money, billions, literally. Most musicians who put music on the Internet that most people actually listen to, turns out the data show that 90% of all songs listened to today were by artists you never heard of and they were written in the last few months. Okay? That's the actual market.

But it's not a market. People are not getting social value out of it other than song, but the artists aren't getting rewarded. They're not getting paid. They don't have a job. Alright.

So it's a broken system. So there must be some incentives that are missing here. Alright. The current market, what it does is that I write a song and I really am good at it. I put it on the Internet.

Spotify streams it, they make some money off of advertising or subscriptions, and they probably don't give me any money back. Ethically, should, they kind of do just to make themselves look good. It's not a really, really clear market. All right? They make enough money though, they make a lot of money, that they can pay a few people a huge amount.

And those people say this is a great system and we're all happy with it. But it's not really working for most people. Anyway, we have worked in the last 10 years towards a company that's called United Masters. It's been existing for 10 years. Steve Stout is a friend of mine.

He is a legendary hip hop producer in The U. S. He is the CEO. And it is a 3 way market. There are people who make music.

They put it on the Internet. There are people that listen to it, you and me. And then Critical Gears is 0.333 node, which is brands. Alright? So Steve knows he's famous.

He knows a lot of people who are CEOs of these major corporations. And he went to them and said, for example, FIFA is 1 of them. He went to FIFA and said, occasionally, FIFA, you need music, don't you? Because you play a clip of Ronaldo doing this or that, there's often music behind that. What are you how are you getting your music currently?

And they say, well, we go to the record companies and we pay them. You're paying a lot? Yeah. We're paying a lot. He said, well, no, United Masters could do this instead.

What we're going to do is have a recommendation system. We do data prediction that for a particular sports entity, a good musician might be this guy right here. He's sort of this music that sounds sports like. And so whenever you need a song, we're going to maybe send that person's music to you. Okay?

That will then probably bring in a certain demographic because they musician. You like that demographic, don't you? These are all adaptive prediction systems along each 1 of these edges. Right? Now it's a market.

So when that musician is viewed as being successful with FIFA, the NBA also likes that demographic, looks at that, says, we're going to reach out to the musician and maybe incentivize them to write some songs for us, work with us. So that's happening. There are currently over 2,000,000 musicians signed up exclusive contracts with United Masters, not with record companies. So this is actually a working system. It's in The US, very popular there, and it's going into Africa and Brazil as we speak.

Behind this is, in fact, data flows. You wouldn't know which musician is popular with this demographic unless you did this recommendation system here, and you wouldn't know how to recommend them to brands unless you knew what brands are related to which demographic. So this is all designed with these market principles in mind, and it is a working 3 way market. Right now, critically here, money is at play here. Where is the money coming from?

Well, you don't just link these 2 and then hope that money will arise, because then you go to advertising and all the negatives that are associated with that. Instead, you bring in some money that have people who have an actual goal to have some money. They then put it into the market and then it gets allocated hopefully in a good way. All right. That's my favorite example, it's a working system.

Here's another kind of thought experiment. This is not a working company, this is a data analysis with Ali Rizafala. It's a modeling analysis. So this model is a situation which we have users, platforms and data buyers. So what is this?

Users are you and me, and we often will interact with platforms, for example, to do payment. So Mastercard will interact with them and they will do a payment for us. When we interact with them, they get data from us. All right. That's good or bad.

We're losing some privacy, right? But we're getting back something which is they make a better and better service as the more we interact with them. So that's been the kind of implicit contract for all of us for all these years. So that's a positive feedback loop, data service, data service, data service. Okay.

The problem is that these companies often don't make enough money off of their service to stay in business. Now Google does, but Mastercard doesn't. Okay? So what they do is they take all the gathered data and they sell some of it to 3rd party data buyers over here. Why do people want to buy data?

Well, you kind of can guess. They want to do market research, figure out what kind of new restaurant would be good in London. So they would look at some data. All right. Now the problem is that and so this data goes here and now money comes into play.

This is paid for. The problem is that the user has just lost something, privacy in a big way. Instead of just having a little loss to some company that gives them a service, they're losing it arbitrarily to someone who could use the data however they want. So a rational user here would look at that and say, if I care about my privacy, maybe I don't, but if I do, I should walk away from this deal. The market should unravel.

And that's not really happening yet, partly just because of ignorance. People don't know about all these things. K? But they will. So how could you how could you fix this?

Alright. Well, as you probably know, there's a mechanism that helps align incentives called differential privacy where I can add noise to data. If I add noise to data, it's less easy for you to tell something about me. Who would add noise to data? Well, the users probably wouldn't.

The platform is good though. Right now, you could decide how much noise to add to the data. And here's where you start might thinking, well, adding noise today that protects the customer, we're the government, we should decide how much noise to add to data. Maybe it's a good idea, maybe not. The GDPR says sort of add a lot of noise to data basically, protect you a lot.

Alright? But now let's think about this more from a economics point of view, which is that these platforms could be free to choose their own level of differential privacy, how much noise to add. So let's suppose that Google says, we've already got a lot of customers, we're not going to add much noise, we don't care. Mastercard wants to be a competitor, they come in and say, we'll add a lot of noise to data, we'll protect your privacy, and you could audit them. Right?

So So now once they do that, those users that care about their privacy and against not everyone, some people don't care. But those who care would go to Mastercard now. Mastercard will get a lot more data and there would be this positive feedback loop, better service, better data and so on. All right. Of course, the problem is that the data buyers will look at that data and they'll say, oh, a lot of noise has been added to your data.

I'm not going to pay as much for that. All right. So there'll be a negative push on Mastercard and a positive 1 up here. Okay? And now you think about this, this is a dynamical system, it's going to have equilibria.

Okay? And you can calculate these equilibria, this is a stackable game, you can kind of flow this back and forth and calculate a fixed point of this dynamical system. And that's kind of 1 of the technical points I wanted to take away here. Machine learning and AI has often thought about optimization as optimization technology. Okay?

This is not optimization. This is equilibrium finding. And I think of equilibrium finding is a much more powerful way to approach information technology. Equilibrium finding often means there's conflicting incentives and when you put them all together, there'll be something going up, something going down, you have to figure out how it all settles. Okay.

Alright. So we did that in this situation. We made a model, and we'll get into details, and we wrote down the game behind it, and we did a calculation of an equilibrium for this. Alright. Now, you're the government.

You look at this overall system. You'd sort of say, is this doing the right thing for all people? Is it hurting some people? Whatever. You could say, well, it's not good.

I don't like it. I'm gonna make sure there's infinite data noise added. That would really protect people a lot. The problem is that might break the whole market. Okay?

Instead, you could say, maybe what I'm going do is have a few little knobs I could tune. So, for example, I could say, I'm not going to tell each platform how much noise to add, but I will have a lower bound on how much noise. I don't want... I want to protect the completely naive customer... Consumer.

Now if I do that, it'll change the equilibria. If I move that up a little more, it'll change even more. You can imagine sitting with a person in government office as an economist with data analysis doing that little thing and making that plot and see where the equilibrium go. And they say, oh, there it is. That's the that's the Goldilocks zone.

I want to be there. And so you set the parameters like that. What's happening now is you're regulating the equilibrium of an overall dynamical system. You're not regulating the algorithm and how it optimizes and all that. And I hope that distinction is coming through.

I've had that conversation with people in governments in Europe and US, and they tend to like this. Oh, regulate the algorithm? No. That's a bad idea. Don't understand the algorithm.

Regulate the equilibrium of the overall system? Yeah. I can sort of... I can see that. Okay.

And so, in fact, we did enough of a study here that we were able to look at the GDPR, which tends to add a lot of noise to data. Basically, it says you can't transmit certain kinds of data, and that actually is having a very negative effect on the economy in Western Europe. It is putting medium to small businesses out of business. The big companies are not hurt, but the medium and small ones are very much hurt by GDPR. Okay?

So instead of going to a full market, you know, everything goes, go to a kind of a partly regulated 1. That makes total sense and it's just not being talked about in this modern data world. It should be. Okay. My last little point, how am I doing on time?

Okay. Is to now ask 1 of the big questions is, okay, you're going to put economics together with statistics together with computer science, you can now build systems that have got some notion of how do I collect data even when it's incentivized and so the overall objective is somehow a good working system. That starts as I like that, its appeal. But now you can also think about what are the metrics? And of course, this is a huge, huge question.

I know my discussant Max works a great deal on this and he'll have, I'm sure, great things to say about it. It ties into political economy, it ties into history, ties into just everything. Alright. I want to just add a little bit to that dialogue, which again is an economics perspective. 1st of all, Serena, this was her thesis, and these are collaborators that have contributed a great deal to this this line of work.

Alright. So let's remind ourselves, I started the talk, that these machine learning or AI systems are not just a technology, they're an ecosystem of people and computers all mixed in together. Right? And there's different stakeholders. They have different goals, but there's probably some things they could agree on, and you want to kind of figure out ways to incentivize those things they agree on.

Right? And so Serena studied health care in particular where the the stakeholders are hospitals, patients, doctors, and of course, the government. Alright. And when you start to talk about a metric in a domain like this, you might pick something that someone obviously cares about. So there's an example that she studied in New York where the hospitals were...

Needed to be evaluated. The government decided they would be doing evaluations of hospitals. And so 1 of the obvious things you want a hospital to do is keep people alive. So what about making mortality rate a metric? Okay.

You're a good hospital if you have low mortality rate. So they did that in about 2010. Alright. And I think you can sort of imagine what starts to happen. When that becomes the metric, this is called Goodhart's law, what the hospital started to do the next year is only admit patients who are not too sick.

That happened. Alright. And I can't think you can imagine this happening in all kinds of places. So now what do you do? Well, you could argue and have the legislature talk about what's the right metric.

Maybe that would go somewhere, probably would, but not clear. Alright. Another way to think about this is as a principal agent problem. Alright. So mortality rate is a bad metric.

It's too easily gained. Who would know that? Well, the government maybe knows that if they think about it, but they often don't think about these things. They don't think 2 steps ahead. Who knows that?

Well, certainly, the hospitals know that and immediately realize. So the hospitals have... It's an asymmetry of information. The hospitals have information about what would be a good metric and what would be a bad metric. A bad metric is 1 they know they could game easily.

A good metric is 1 they know that actually gets at the problem and I can't game that very easily. They would know that. The regulator would love to know that, but they can't just ask the agent, the hospital, because the hospital will lie. They will tell them the metric they know could be gamed. Again, it's kind of unethical, but it's what human beings do.

So how can you deal with this? Well, it's a principal agent problem. We know how to think about principal agent problems. And so this specific principal agent problem is often called the agency game. And so, this is, again, a part of economics, it's part of contract theory, it's related but not the same.

And an example of this is, you know, a principal might be someone who wants to get a taxi or ride share, and the agent is the person who's driving the car. So I'm gonna put out a bid. I need to go back to Bloembury tonight, and so I'm gonna put a bid. And the agent receives my bid and now has to decide how to price this. Now the agent may know something about the local traffic conditions or maybe they know there's a riot going on up there, I guess that's Paris, not London.

And so they should change the price accordingly. And I don't... Maybe don't know that, but they know that. Now the question is, should they tell me that? It's an asymmetry of information.

They know something I don't know and that affects the actual price. Should they tell me? That's what an agency game is all about. Alright. And I'm gonna kinda skip to the chase here, but let's imagine that there's this extra variable, the local congestion, the agent knows about that this 1 doesn't.

And the question is, should they transmit that here versus to there? And you can say, well, ethically, they should, but again, it's not the right way to think about these problems necessarily. Okay? And so the agent, what do they do? They think this through.

If I reveal that, I'll get more often higher compensation in high cost settings, but of course, in low cost settings, I won't. I think that through a little bit, and I got to write down some equations and sort of think about what's the best thing to do. Because if I conceal it, I will get paid more in lower cost settings and that's called information ring. I'm getting something for free and people like to do that. And so Serena worked this out in her thesis and basically there's a theorem that states, and here's now a big economics word, but agent prefers to conceal x if the price elasticity for high cost x is close enough to that of low cost x.

K? So so again, this is an economics concept being brought into data analysis. Put the 2 together, you can get something that actually works.

Hi. I'm interrupting this event to tell you about another awesome LSE podcast that we think you'd enjoy. LSE IQ asks social scientists and other experts to answer 1 intelligent question, like, why do people believe in conspiracy theories? Or can we afford the super rich? Come check us out.

Just search for LSE IQ wherever you get your podcasts. Now back to the event.

Okay. So you can now start to design some how do we get out a good metric for a system. And, of course, this is not the whole story. You just don't put this down and it comes up with the right answer. But it's a start, and then you could have legislative, you have discussion on top of this.

But you want this kind of mechanism to be present. Okay. Alright. So I've said the word market a lot. And just to be clear, for me, market is a wonderful thing, basically.

It goes back tens of thousands of years. Humans developed it without any mathematics, without any person telling them what to do. In Mesopotamia, some people came in with something across the desert. They had some oil. Someone else needed the oil for this, and they traded, and everybody was happier.

And humans have developed this again and again and again in many kind of ways. Sometimes the market has failures and you have to think about regulating, someone's got to think about that, but basically, human prosperity has been built on markets. Alright? And I want to be very broad about what we mean by a market because again, no preconceptions. We're going forward in history.

We've these new ideas and everything is flowing around. What kind of markets will emerge? The music market is an example. That to me is a market, and it makes music go to people and musicians get paid. That's what I want.

But I always want money to be part of the market story. Okay? So 1 of the most famous markets, Nobel Prize winning, was for kidney exchange. Okay? If my brother has a bad kidney, I might be willing to give him 1 of my kidneys, but maybe we don't have the same blood type.

So I've got to give my kidney to somebody else who has somebody else they would give their kidney to and all the types we could match in a little cycle. Alright? So what we have is a swapping of kidneys that gives social welfare going up. Okay? Also, matching markets are like students match to universities.

You don't pay to go to a university or to be admitted. Alright? But you wanna match people. I prefer that university. I prefer this 1, and you want everybody to get something as close a match as they can.

Right? That is a prediction. I'm making a prediction about which 1 I prefer. They're predicting which students will be the best for them, and those all get immersed in an overall system that hopefully makes it such that it treats as many people as fairly as possible. You can't be perfect, but you try to be transparent and open about it.

To me, markets have these positive characteristics. They can make allocations that are transparent and open and tend to be more fair than the alternative. K. Moreover, markets don't just, do this with, centralized mathematics. What they do is they find not optima, they find equilibria.

And as I've kind of hopefully emphasized enough in this talk, equilibria tend to be a better way to think about problems involving social life, but also now machine learning algorithms supplanting social life than just pure optimization. Okay? Next, what do markets do? They make availability of our resources more stable. We need stability.

We're 7,000,000,000 people. And if things start to go unstable, it can be bad. Alright? And just think about what markets do in that regard. If I wanna open a pizza restaurant here in Downtown London, I need tomatoes.

In the old days, I might have to go out to the countryside and get some tomatoes, and then some days I'd get them and some days I want, and so some days I'd have pizza and some days not. But now there's this kind of supermarket. It's called Sainsbury. I go there and there's tomatoes every day, no matter what the weather, no matter what... And that's true in every city around the world.

Alright? So I can now plan, the uncertainty is reduced, that there will be tomatoes and I can leave my pizza restaurant and therefore I go from there. So we talked a lot about uncertainty in this talk and you tend to think about that as a statistician's domain, but economists also, by having markets, reduce uncertainty, that's a good thing. Alright. Last 2 slides, and I'm done.

Statistics, economics, and computer science, all are allies here. And in academic world, they've made pairwise alliances. Economics meets statistics, that's called econometrics. But not a lot of computer science in econometrics, not a lot of mechanisms and algorithms. Statistics meets computer science, that's machine learning.

But not a lot of economics in classical machine learning or AI. Economics meets computer science, that's algorithmic game theory, design of auctions and so on, not a lot of data and statistics there. The point is that in the center of that green triangle is where all the real problems lie. Okay? So for the younger people in the audience, you need to learn all 3 of these perspectives in your brain and then live in the green world and develop new ideas that are pertinent for this emerging green world.

Okay. And so for me, AI, which I've not used hardly at all in this talk, but every 50 years or so, a new engineering discipline emerges that changes human lives. 50 years ago, it was chemical engineering. Okay? Chemical engineering, you can argue whether it good or bad.

I came up in the sixties, was viewed as a bad thing. It created pollution and so on. Barba, papa. But, you know, chemical engineering created medicines, it created plastics, it created clothes, it created all kinds of things. It's a good for humanity.

Where did it come from? Well, in the thirties or forties, there was no chemical engineering, but there was quantum mechanics, and there was fluid flow, and so on. So people knew what happened in a test tube if you put some molecules together. But the idea you could build a factory, it would put a bunch of molecules together at scale and create enough product for all of humanity, that was wild. And they tried it, and sometimes it worked, sometimes it didn't, and a new field emerged to make it systematic.

And that field was called chemical engineering, and it had its own new ideas. 50 years before that, you had electrical phenomena. Maxwell's equations told you everything you needed about electrical phenomena. That was the science. But a new field needed to merge to build circuits on top of that or to build communication on the waves from Maxwell's equation or to build bring power to cities and so on.

That's called electrical engineering. So I think what's happening right now is we have proto ideas of prediction and data and economic ideas like utility and all that, and we're kind of throwing that together as kind of buzzwords into a big soup, and we're then putting trillions of dollars into making data centers support all that soup. And we're calling it superintelligence. We're just doing this crazy stuff. So I think 50 or 100 years ago, there was a lot of hyperbole, there was a lot of fear, and a lot of that stuff, but nothing like today.

And today, everyone is completely confused about what it's all about. Is it superintelligence? Is it money thrown away? Is it a bubble? Is it this that?

All that dialogue, I think, is so far away from the reality of what's happening. Really, I was trying to build working systems for real people on on principles that will need to still emerge. We don't have quantum mechanics or Maxwell's

interesting talk. I've been charged to provide a brief discussion. So I have like a few slides here. And we've heard about a whole bunch of interesting kind of more specific projects at this broad intersection of computer science, statistics, economics. What I want to do is kind of take a step back here and think a little bit about what's the broader agenda and where could it be going.

And so the way I understand what Michael is pushing for, and which I would very much agree with, is that it's a very intellectually productive endeavor to try and combine the insights of these 3 different fields. To take from computer science the kind of more algorithmic computational perspective. To take from statistics the more inferential perspective of using observation to learn about some underlying phenomena. And to take from economics various insights around thinking about the fact that there are multiple agents in the world to begin with. That these agents might have private information.

That there are strategic interactions between them and so on. I think there's a lot of questions that can and should be answered by combining these tools. But I would also add on top of what Michael has said here is that this is actually a feasible intellectual agenda, in the sense that those 3 disciplines are actually quite close to each other. They share a lot of the common underlying conceptual tools. They maybe don't always use the same terminology.

But a lot of what's happening in those 3 fields is built on the basic tools of probability theory, of optimization, And kind of combining them, decision theory. So decision theory is this idea that there's some objective. You might call it utility. You might call it reward or loss, depending on your field. And that based on whatever information you have, you're trying to optimize that objective or maybe the expectation of that objective or something like that.

I think, probably that's a great agenda. But I got a little more confused about reading through some of these projects is thinking through whose objective are we trying to solve here? And there are just multiple different answers that we could give to that question. But I think it's important and very useful to be clear about whose problem we're trying to solve here. We could think about solving the problems of tech companies.

How do you build like the next generation of an LLM or of some other AI system? How do you build like the next AlphaFold or something like that? You could take alternatively, the perspective of like a tech company or their engineers, but instead to take the perspective of the wider democratic public, or maybe regulators that are supposed to act for this democratic public. Thinking about how do we create the context and conditions for this technology to be beneficial in the broader social sense. Or 0.333 perspective would be asking more of a fundamental science question.

Like How should we do inference from data? How should we think about the process of collective learning? Depending whose perspective we take, we're going to ask different systems here. We might ask the question of how do we build AI systems that are generating more profits? Or we might ask how do we create regulation or context institutions that ensure that AI and related technologies are socially beneficial in a broader sense?

Or we might ask fundamental questions about what does intelligence mean? What does learning mean? What is science? How do we learn from data? Is it really just an individual decision problem?

Is it maybe more of a collective process? And how does the normative field of statistics impact this collective process of knowledge production? And so those are all potentially legitimate questions, but they are different questions. And I think it's helpful to be clear which 1 we are answering. I think this is also something where there might be more friction between these different disciplines.

Where, I mean it's not, not obvious clear cut, but maybe engineers are more inclined to think about the 1st of these. Maybe economists are often think, taking the perspective of the state or government. Asking what we should do in terms of regulation. Statisticians might be more inclined thinking about normatively what should we do when we do empirical science. So it's helpful to say which problem we are solving.

A 2nd point, we had a brief discussion about it before this talk, is it useful to use the market analogy or the market language to talk about what's happening with AI or with machine learning decision systems more broadly. And I think it might be more helpful to not necessarily use that terminology. By saying that, so if we think about what we conceive of as a market traditionally, or the way economists would define it, the canonical market is an exchange between private parties, sellers, buyers of some goods or services against money. It's done in a decentralized way. There's no centralized algorithm that's doing the coordination of these decisions.

And information is aggregated by prices. So everybody has their own private information. How much they want something. How much they're able to produce something and so on. And coordination is happening through a price price mechanism.

And that's quite different in a number of ways from what's happening with AI or algorithmic decision making systems more broadly. Which is very much not about a consensual decision between private parties that doesn't really concern anybody else and doing disaggregations through a price mechanism. Instead, what we have is that in a way the most important 1st step for producing AI, at least in the machine learning era, is accumulating data. And the data can come broadly from 2 broad domains. Would say 1 is information that's in the public domain.

Take anything from Wikipedia or Stack Overflow or archive or from news websites and so on. All this knowledge and information and art and literature and whatever else humanity has produced and kind of incorporate it and make it part of the product it's the relevant companies are producing. That's very similar to what we might call an enclosure of the commons. And the other kind of data is data that concerns individuals. How much are you willing to pay for something?

What wage would you be willing to accept? What ad you're going to click on? What news is going to keep you engaged? So those are all data about individuals. And there the concept of data externalities is extremely important.

They're pretty much all learning. It's not about you as an individual. It's about patterns across observations, patterns across individuals. And so any data that you are sharing on any of these platforms is not so much used to make decisions about you. It's used to make decisions about people who are similar to you in some ways.

And so anything you do there is concerning a lot of other people. Externalities are paramount in contrast to what we would think of as a market with secure property rights. That information is collected in a centralized way. It's not aggregated to a price mechanism, but it's incorporated in various databases of some form or other. And then there are algorithms that are making decisions in a centralized manner.

And so in many ways, this has a lot more resemblance to centralized economic planning, although with the technology of the 21st century. Which can be a good or a bad thing, but it's definitely different from what we would think of as a classical market. And so I think using the tools of economics to think about the systems is very important, thinking about the fact that there's many different actors. Using the language of markets might be more misleading, or trying to put it kind of round block into a square hole or something like that. And so it's a slightly different perspective.

So apologies for the shameless self promotion. But there's a book that came out last year where I also tried to take an economic perspective on AI, but from a slightly different angle. And so the overarching argument here is AI, at least the way it is constructed these days, is automated decision making using optimization. There's always some loss function, some reward function that these algorithms are optimizing. So all the different iterations of AI we've seen over the last few decades.

And so in that context, there's a lot of debates about if there's problems, conflicts around them, how do we fix them. And those debates are often framed in terms of optimization errors. But I think the much more 1st order issue is what's being optimized and who gets choose what's being optimized. And so that relates very much to property rights and control rights and ultimately to who controls the inputs into AI, the training data, the compute chips, data centers, but also the expertise to build these systems and the energy to run the data centers. And the ultimate solution to deal with problems here, I think, is not so much construct markets of some form or other, but we really need to think about how do we build democratic control of the objectives of these systems, where the control needs to exerted by those who are being impacted by the decisions of AI systems.

From the perspective, but also I think from Michael's perspective then, you can ask what's the way forward. And I think there's a huge amount of interesting research questions to be asked exactly at the intersection that Michael was describing between economics, computer science, statistics and adjacent fields. So here's a few domains I think where we might talk about those. 1 is if you buy kind of this perspective that we should aim for democratic control of AI objectives, you can ask. But let's imagine that we had achieved such democratic control.

Now how should we build these systems? So I think there's a question of how do we engineer such systems if they serve objectives broadly of those who are being impacted by them. Then there's kind of the step before how do we get to something like democratic control. Which includes both all kinds of policy, legal, regulatory, economic questions in many different domains from transparency legislation, interoperability requirements, tax standards, privacy legislation of various forms, intellectual property rights, competition law, and so on, that all impact who controls the data, who controls the inputs into these systems, and thereby impacts what objectives are being optimized. But there's also technical questions that are very interesting.

There's a lot of potential now with things like open weight models, distillation of commercial models, post training of these open weight models to serve local objectives of local public, nonprofit actors, to not be dependent on the objectives built in by big tech companies. And there's the kind of more foundational science questions on how do we build a better foundation for statistics as the methodology of science. Where I think a lot of the current foundations are ultimately going back to some form of single agent decision theory. If you have a loss function, if you build estimators and tests and other things, experimental designs and so on, to minimize these losses. When you open any Foundations of Statistics textbook, you're not going to find any discussion of the fact that there's more than 1 person in the world, basically.

Where I think, think Michael would agree with that, a more productive foundation might be to think about statistics, not so much about decision making, but really a selective communication. We look at large data sets. We produce summaries of some form or other. We communicate these summaries to other people, and then there's some kind of collective learning process that's going on. But once we think about selective communication and all these things like we saw in the FDA example, for instance, come up.

All these discussions that have been going on in recent years around things like p hacking, publication bias, pre registration and so on. Many other issues that come into view once you take account of the fact that science is a collective endeavor and not just individual decision making. Thank you.

So thank you both for thought provoking talks. And can we open up the floor for questions? Please, anyone?

Hi. I apologize for this, but I'm just really curious to see your slide about market failures because you skipped past it. The slide you had about market failures. Yes. You you fast forwarded past it.

So I'm just very curious to hear your opinion about what are the the market failures that you were... You would have talked about if you had more time.

If you don't have an economics background, 1st of all, just read economics. You know, there's just tons of examples of market failures throughout history. Sometimes they're, you know, oligopolies. Sometimes they're monopolies. Sometimes there's market unraveling, just so many examples.

And most of them are still alive in our era. And so I kinda wanted to take some of them and bring them forward to our era. But just to give examples, think music right now is, is broken. People... Young people who are 16 to 25 who are really good at making music are not able to make a job, have a living.

And that's 1 of many, many such grabs in the creative arts. Alright. So you can just say, well, it's too bad. That's the end of that. Or you can make...

You know, we all collect taxes and we give it to the musicians. And then, you know, you can go down those kind of routes. But really, it requires new incentive structures and new ways of recognizing there's a demand and a supply. And how do I connect them? And so when you start doing that, you could build it in ways that would be bad.

It wouldn't work. People would walk away. You you would get market failures. Some... You know, Beyonce has all the money and no 1 else has any kind.

You know? So it really... You know, markets, wanted to kind of give the positive spin though. They... You know, before we had any mathematics or any Nobel Prize winners or any of that or the LSE or whatever like that, we had working markets throughout every, you know, society.

Sometimes they did great things for humanity. They built all of the... You know, the Romans built all the stuff with big little markets. Sometimes they had slave trades. So, you know, it's just the good and bad.

Good and bad of humanity. It's all there. But not thinking about it is a... It was even worse. So I like to give the example of, you know, I talk about these engineering fields.

Think about mechanical engineering even before. Mechanical engineering, it's based on Newton's laws. 1 of the forces in Newton's laws is gravity. Right? You could say gravity is a very bad thing.

Why? Well, kills... How many people die every year from gravity? They fall off buildings. They fall out of the sky.

They... You know, there's just tons of deaths from gravity. So we should not use gravity in mechanical engineering. We could have pulleys and and ropes and all kinds of ways, and we could build all kinds of things without ever talking about gravity. And that wouldn't be better.

No. It wouldn't. Right? So markets are a powerful force. Humans come together.

They have information asymmetries. They have things to trade. They will do it. Alright? When they start doing it, then the bad side of humanity will often take over.

Hence, you have to regulate them. Right? But not taking that responsibility of thinking that through and say, what are the failures and how can we go? You know, that would be a mistake. And I think that, you know, I think with Max, I'm kind of...

You know, I I think the word markets are good, I just... Because I don't know a better 1. But nonetheless, social phenomena meets data. It's just not being talked about in Silicon Valley. It's not being talked about by journalists.

It's all about who has the most money and who's gonna... Who China win versus The US and where are the jobs. There's no more jobs left and just all the terror and all the hype and the hysteria. I mean, maybe I'm reading the wrong places, but, you know, that's what I get. And do...

The simple, oh, it's actually market. People can trade again at... You know, there's there's... And so, yes, I I didn't really answer your question, but, you know, just... You know, basically, any market I can think of where creative works are being produced right now, there's a failure.

And does that mean it's the end of it? No. It means we have the wrong incentives in place. Partly. I wanna just ask, how many of you are, 30 and below?

Okay. A good chunk. Some of you are lying. I know. You're incentivized to say something different here.

How many... Of those of you raised your hand, how many of you are more terrified by kinda where we're at right now or more more excited? More terrified? More excited? Okay.

It's 0.5 and 0.5. Maybe that's just good. How many of you feel like you really kinda know what's going on versus you'd have no clue? Really think you got it? Have really no clue.

Yeah. And you're not alone. And I don't... But I don't think in, you know, 50 years ago that young people who were going into chemical engineering were quite so clueless. They had an idea what it was doing and all that.

And I I think it's a it's a tragedy right now that we're not having the right words being used or dialogue. Our journalists are going after existential risk that's gonna kill all humanity because Nobel Prize winners say that. You know? I'm really sorry to see that kind of thing happening right now in our world. And so just finding the right words sometime is you know, instead of kill all humanity, market failure.

Maybe it's a better word to use than kill all humanity. Just just saying. Stand up.

As a student interested in the topic of the economics behind AI, where would you recommend I start in researching further about the topic?

1st of all, I've learned all my... Everything I've learned in the last 5 years by going on, you know, really smart lectures on YouTube. You know, really. It's just amazing. That's 1 of the beauties of our era is this thing has arisen, like Wikipedia and so on.

And there's so many great lectures. So I actually, believe it or not, would start there and then, you know, get some textbooks and teach yourself. And then get together with some like minded young students and have a reading group and learn about it and never give up. Never, never stop. And also people are now saying it's the end of mathematics because some pure math has really become easy for Chad GBT.

It's a big deal, and it's... It changed all. But most math is not giving up. It's not going away at all, quite to the contrary. So learn, all the mathematical concepts that lie behind that, you know, fixed point theory and probability theory and, you know, topology and blah blah blah.

The more you learn of that cuff, the better a thinker you'll be and be able... You'll see how to put things together. And then emphasize the human side. People often ask me, what jobs are gonna disappear? I'm not a labor economist, but I can tell you that many, like, you know, legal scholar goes into the archives and finds relevant patent cases.

That's gonna go away. Person who interacts with another human being in a teaching situation or in a health care or caregiving situation, that's not going away. Quite the contrary. So combine very strong mathematics with a social, you know, individual human awareness. Those 2 together are...

Make you a powerful force for the future. But don't be... You can't just learn it all in school. Schools are way behind right now. They're teaching stuff from the past and not doing a very good job of realizing all that's changed.

So use the tools to teach yourself as fast as you can and do it along like minded people. Max, you wanna add to that?

I mean, maybe it's slightly more specific if you go to my website. I've... We have this reading group that's been running for the last few years, which has a lot of links to papers around the economics of AI and to Jason's questions.

Anyone else? Please.

Hello. I'm a student of innovation policy, and 1 of the conversations we were having today was economics is all about trade offs and choosing which for which. In case of AI, a huge trade off that we're currently facing is that data centers consume a lot of water. They heat up the environment. And so when you're talking about equilibrium, what is the conversation when thinking about equilibrium, when when to choose AI to do what and whatnot, given

that?

That's a great question. Part of the reason I give these these talks is that I I don't believe that that's the right model. I... We we say AI requires this and that. It's just...

No. No. It doesn't. You know? So there is a bet being made right now that by taking all the world's text that you and I produced that they stole from the Internet and and then predicting the next word and doing that at a huge scale, that'll...

It'll not just be as good as it is now, which is amazing. It'll be so much better that it'll it'll... It can solve any problem. Okay? And the bet is that trillions of dollars will will will be needed to reveal all of that.

Okay? Max had on 1 of his slides the open weights models. So here's an alternative technological vision, which is that once someone builds a big box that really does a good job of predicting, you know, just about as good as it is now, but a little bit better, stop. It won't necessarily then know my willingness to pay for a while ago. It won't know a lot of things that are really relevant to him.

It's just... Who cares? It shouldn't know that. Okay? Now think of you're building a company that does, you know, I don't know, climate change research.

You've got ChadGPT. You've got open... You know, you got protein prediction. You got climate weather prediction. You got all these boxes.

Don't put them into 1 big mega box. Make them each be a tool. Okay? And now add your own knowledge to it locally. For that, you may take an open weights model from China.

You took all the effort that they put into it. You take it, and you add a little bit like PPI or you add it in your own way and use it as a tool to do your own stuff locally. You don't need a huge data center for that. I think that of the 10,000 companies already kind of using AI and getting some productivity gains and all, it'll be more that, alright, than building yet another huge big box that took in... You know, took massive iterations to do these things.

If you may... The the terminology is often now pre training versus inference. It's kinda terrible terminology as usual. Pre training needs data centers, massive, to take all this massive data to train the model. Inference stuff just says, given a fixed model, give me the answer.

Much, much cheaper. Don't need a big data center. Okay? But we're in an exuberant phase right now where... And you go to The US and they say, well, yeah, we realized that we are probably overspending on this.

It's probably crazy. But we don't have a better idea because we don't know what we're doing. And... But we know the Chinese are doing it, and they probably know what they're doing. So we better do it too.

And those are the, again, the kind of immature sophomoric ideas that are dominating our world right now. But it didn't have be that way. So, yeah, data centers are gonna plateau because the cost of benefit is clearly now not good. Okay? And, you know, the...

And it doesn't... But it's not I guess, is the point. You know? Again, open weights is 1 way to think about it. There'll be others.

At some point, the the cost will have been paid, and you could amortize over that for the future.

Maybe just to add to that, the classic economic answer, which I think is the right 1 in this case, which is just let them pay for the environmental damage they are causing and then different decisions will be made.

Up on the top, we haven't heard from you.

Thank

you for the talk. My question was on the contracts. You described how you guys calculate the equilibrium for where you think it will end up. My question is in real world markets, people often come up with exploits or kind of things that weren't seen backwards looking. So how do you make sure that the path that you take to the equilibrium isn't heavily exploited or even that you kind of end up at a different equilibrium than where you thought you were going to?

Fantastic question. So that's the best answer I can give to 1 of earlier questions about what should I study. You know, study all... That's a kind of a 20 year problem. And if you work on that, you know, you will you will help.

You'll change the world. That's what happened in all the other previous engineering technologies. Chemical engineering, those plants could have exploded all the time. They could have damaged huge numbers of things. People thought it through, and they put in the right kind of the guardrails.

They thought it through a few steps ahead. They didn't always work. But right now, we're living in a world where it was just been had ad hoc put together. The Internet was, in my era, put together just so we could communicate among friends. Okay?

And now it became the whole thing where the banking industry was built on and everything else, and now the bad people came out in droves. And then AI is helping them find out even more exploits. Okay? That's the world we live in. So what are we gonna do?

Well, number 1, a whole new security field has got to emerge that does it right. Slowly drifts towards something that's much, much more sandbox y, much more structured, like you do in a real company. You don't let your agents go out over the Internet if you're a real company and do horrific things. Those companies are not real companies. They're 25 year old naive people with a with a mission, messianic people.

Okay? So build more robust systems, and then think through the... When you think of a market, it needs to be also robust. Markets are somewhat naturally robust. But the ones that economists often built...

Have built have been making all lots of assumptions, the rational agent assumption, blah blah blah. Data analysis can help you with that. Your agent isn't just a rational assumption. Data informed how you design your agent and then do robust versions of that. But it's like...

It... That's what... You know, like I said, those engineering disciplines took 20 years. Chemical engineering took 20 years to kind of emerge. That exactly is the kind of thing to work on and not on superintelligence or whatever, but work on...

Yeah. What if an adversary comes in and does this? How bad could that be? Oh, let's think that through. Let's build something.

The banking system, you can go to an ATM all around the world and get on money. 7,000,000,000 people can do that. And somehow, yeah, people rob and steal a little bit, but a little bit, relatively. It works really well because amazingly good engineers thought it through about all the bad things that could happen, and most of them were kind of shut off. We don't have that style of thinking right now, unsadly.

This whole AI buzzword has created this mythology of boom versus bust that makes people not worry about the... Now I think most people and most of the companies that are a little bit more mature are taking this stuff in and using it in intelligent and responsible ways. But overall, yeah, markets can have failures, and they can be unstable, and they can do all these kind of things. But, know, again, it's a powerful tool like gravity. I like to use the Chinese model.

When I was young, China was communist, you know, avowedly. They really were. They did a market experiment in Shenzhen in the eighties. It was a wild success. It led to China's prosperity.

And so think about that for a minute. You know, the the Chinese ideology of of communism was overturned, really. It's now a very much market based economy in China with high regulation, authoritarian, you name it, but it's a market economy because it... They tried it, it worked. Now when they do AI, it's probably not gonna be quite as wild and crazy because they're much more worried about stability and bad players and all that.

And they do it in their way, but you can think about it in in your way. So, anyway, thanks for asking that question. It's a great question.

I guess we... So just we take 1 more answer, and then we conclude. Please, the gentleman in the blue. Apologies.

Nowadays, I feel like AI is being or like growing rapidly in so many different areas.

But I can't hear you very well. Try again.

Wow. AI is growing rapidly in so many different areas and even being exploited in some. Do you think we'll ever get to a point where AI should be regulated? And if so, how exactly

would be? Oh, we're way past that point. Yeah. AI should be regulated. There's no doubt about that.

It's a very powerful thing. Like, gravity should be regulated. You can't just let things fall.

Yeah.

Alright. The question is how? Yeah. And my talk was about that, really, at some level. Naive regulation of the algorithm should never do this probably is gonna have secondary consequences and unintended consequences that have got to be thought about.

And so I gave some ways of thinking about regulation with the FDA example that to me are more sensible starting places. They aren't the answer to the question. Right? And I like the word democracy. Just let me kinda...

Maybe we can close just by me, you know, I was I was glad Max came because he's a leader and thinking through how democracy is both hindered and helped and challenged by the current era. And so we need democracy and discussion and all that. And right now, we're kinda broken. We don't discuss very well. We don't have these kind of conversations very often.

AI, quote unquote, could help. In fact, I think of ChatGPT as not a entity, a bot, an intelligence. I think of that as all of humanity. When I ask a question about good restaurants in Downtown London, Chad, GVT will give me some great answers. They're not coming from its intelligence.

It's coming from humans that put in some data. If I ask a question about direct topology, it's a different set of humans. I can ask 8,000,000,000 and I'll get answers from all of humanity coming through that channel. Alright? And so you could imagine democracies could kinda then be empowered by having a channel where we actually can much more reasonably communicate with each other.

And there are people thinking about that. That's... I think that's another important thing to work on. How could democracies... Right now, democracies, you've got some old rich guy sitting off in some place who you'll never ever talk to.

And a bunch of them get together, and they talk... And they don't know what they're talking about, and they make some rules. It's really... It's the best we have, like, you know, Churchill said, but it's, it's it's endangered, right now. Although...

And and AI, it's only hurt. Right? The the fake news, the fake images that we're seeing right now, it's only gonna get worse. So you look at that, you say, oh, well, stop AI. You could also say, no.

No. Let's think about ways to... Could... AI is just a channel to communicate. Could we use that channel to make our democracies better?

Just my short kind of view on this, I'm not a... Again, go read all the democracy theorists starting with Athens, is that people do well in groups of 200. They don't build in groups of, you know, 1000000000. If have a group of 1000000000, everyone looks at the rich person. Groups of 200, you know, you can start to have some interaction and you can start to make some good ideas and get some consensus.

And then you merge that with another group of 200, you start to build up a hierarchy of things. And there are people... Think that Taiwan is kinda famous for doing this, using technology to help build hierarchies of decision making. But, blending AI with democracy is hugely important for making better, overall decisions about the course of this. You know?

But, again, economically savvy democracy.

Yeah. I guess it's time here all the time slightly, but still, Max, maybe do you want to say anything? Just

I mean, yes. Regulation, absolutely necessary. And, again, I think the 1st order question is thinking about all kinds of different domains of regulation, how they impact ultimately the control over the means of predictions of the training data, data for inference time, compute energy. Because control over these resources will determine where the technology is going to go and what it's going to do to society. So it's not just about high risk, low risk.

It's really thinking about how control over these resources is distributed in society.

Cavity, sorry. So just be closing, sir. Thanks again for both speakers. You.

Thank you for listening. You can subscribe to the LSE events podcast on your favorite podcast app and help other listeners discover us by leaving a review. Visit lse.ac.uk/events to find out what's on next. We hope you join us at another LSE event soon.