Christopher Bishop has been at the forefront of Microsoft’s artificial intelligence applications for some time since he runs the company’s AI for Science research unit, which applies the powerful technology to the natural sciences.
Bishop sees the mission of the lab, which was founded in 2022, as accelerating scientific discovery using the technology. His team studies everything from how AI models can help discover new materials to how they can assist weather forecasting by predicting changes in the atmosphere.
In this conversation with the Financial Times’ AI editor Madhumita Murgia, he explains why he believes scientific discovery will prove to be the single most important application of the technology.
Madhumita Murgia: Why did Microsoft found the AI for Science lab in 2022, with you heading it up in Cambridge? What’s its goal?
Christopher Bishop: The mission of the lab is to accelerate scientific discovery with AI by science within the natural sciences. Think chemistry, physics, biology and related areas like astronomy. The field is not new. My career started off in physics and then plasma physics for fusion, and I moved into the field of neural [networks] 35 years ago.
What became clear to me is that the deep-learning revolution has increased the capability of machine learning and, therefore, increased the potential for it to impact scientific discovery. So, in the year or so leading up to the formation of the team, we had a number of projects scattered around Microsoft Research in relevant areas. It was clear we wanted to accelerate this.
I suggested pulling together a team, combining a mix of existing projects, bringing those together under one umbrella and then growing with some new hires, to make this a focal point.
My view is that scientific discovery will prove to be the single most important application of artificial intelligence.
AI Exchange
This spin-off from our popular Tech Exchange series of dialogues will examine the benefits, risks and ethics of using artificial intelligence, by talking to those at the centre of its development
Scientific discovery is so fundamental to human progress. It’s about gaining a better understanding of the world, in order that we can improve the human condition, whether it’s agriculture, industry, drug discovery, tackling healthcare, new forms of energy, sustainable forms of energy or tackling climate change.
MM: You talked about moving from physics to AI. AI scientist Geoffrey Hinton won the [2024] Nobel Prize for physics, which some people found an unexpected classification as the two fields are quite separate. Hinton wasn’t a physicist to begin with. How did you move from physics to AI?
CB: As a teenager, I was fascinated by the idea of artificial intelligence. I was fascinated by the brain. The brain continues to be the biggest unsolved mystery in the universe. We understand the universe, yet there’s this kilo and a half of jelly that you can hold in your hand. It’s largely a complete mystery [but] is capable of incredible feats of information processing and creativity.
Understanding that and how we can recreate something along those lines in a machine, how we can amplify the capabilities of the human brain using artificial forms of intelligence, this is intellectually very exciting. But at the time, I found the field of artificial intelligence uninteresting because it was about how to construct rules that you can program into a computer which make the computer [seem] intelligent.
That, for me, was deeply unsatisfactory. It never appealed to me. And along came Geoff and others, and they were mavericks in a way. They were proposing this field — it was called connectionism at the time, or neural [networks] now — that was, at least loosely, modelled on the idea of the brain.
I found that intellectually fascinating. That felt like a path to intelligence. You could never write a set of rules that would make a machine intelligent, but here was a path to artificial intelligence. I found that inspiring. When I look back, I must have been very courageous, because I had a respectable, and successful, career as a theoretical physicist.
I walked away into what, at the time, was seen as a pretty flaky field. It wasn’t proper, mainstream computer science, it wasn’t mainstream physics, but it was very inspiring; 35 years later, that looks like quite a good decision.
MM: You’ve been in this field for 35 years. It’s evolved a lot in that time. What have been the key moments of inflection — changing points that have stood out to you as having transformed the field? What has that journey looked like from the point of view of those inflections?
CB: If you look at it from 50,000ft, there have been three phases. The first phase was on a much smaller scale than today, but a lot of excitement around neural nets [was] coming out of folks like Geoff Hinton and others in the field. The thing I think I brought to the field was recognising that, although these are inspired by neurobiology, what we were doing was statistics, albeit complex, non-linear statistics.
Then we found that these networks could solve interesting problems, but they were limited. They didn’t really have the performance, the accuracy, for real-world applications. They could do fun things in the laboratory, and it was impressive, but they kind of ran out of steam.
So, in the second phase, the field of neural nets went into the background. A lot of people got interested in other approaches.
The big breakthrough came around in 2012 — and again Geoff was instrumental in that: the development of deep learning. It was a breakthrough in our ability to train networks with many layers of processing. That was transformational, and so that’s really the beginning of the modern era. Problems that had eluded us for a decade or more suddenly became relatively much easier to solve.
Not only that — the same technology that led to a breakthrough in computer vision has led to a breakthrough in speech recognition. Then we started to apply it to other fields.
The rest is history. That’s the curve we’ve been on, driven by that fundamental ability to train models that are very deep, because when you have a model that has many layers of processing, it’s extremely general. Now you can apply it to a whole host of different areas.
MM: When did you start to become impressed by what language models could do and to believe that they might be a next stage in the evolution of these systems?
CB: I was privileged, because I was one of the relatively small number of people in Microsoft who was given early access to [Open AI’s model] GPT-4 while it was confidential. It was an extraordinary moment to play with GPT-4. Back then [in 2023 when it was released], when nobody — or very few people — had ever seen technology like this, to realise that it was a major step forward in the ability to generate languages.
It’s extremely good at generating human language, surprisingly good. But the second thing . . . the shocking thing, is that we had a system here which, for the first time, can actually reason. It didn’t just generate relevant text. It understood what was going on and could reason.
Now, its understanding, of course, is a shallower level than human understanding, but I liken it to [the first time the Wright brothers flew a powered aeroplane in] 1903, standing on Kill Devil Hills at Kitty Hawk, and watching a couple of bicycle mechanics struggle into the air in this contraption.
You could have looked at it and said: ‘I’m not very impressed with that’. It only flew 120ft. Or you could say: ‘Wow, this is the beginning of a new era’. It was that feeling — it was like the hairs on the back of my neck standing up and [me] thinking, for the first time in my life, I’m interacting with a machine that’s showing — it’s sometimes called — the sparks of artificial intelligence. A long way to go for human-level intelligence and beyond, but . . . like a first encounter, in a sense, at a personal level. That was a remarkable moment.
For me, [using GPT-4] was more visceral. It was less about, we’ve run this benchmark and, look, it’s so much better than last year’s benchmark. It was something quite different: rather qualitative and just realising that you could have had a conversation with GPT-3 before, and you would have had nice paragraphs and a nice conversation. But here . . . just the first time, you realised you were dealing with something that’s qualitatively different.
MM: Which aspects of science do you think have been most changed by AI and where are you seeing practical progress?
CB: The thing that really excites me is the differences between a large language model (LLM) used to help other forms of knowledge work and the nature of science.
When you think about scientific discovery, let’s imagine you’re a pharmaceuticals company and you’re trying to develop a drug to tackle a particular disease. You’ve got a protein that you’re trying to target, and . . . the space of organic molecules that you need to explore is about 10 to the power 60.
It’s a gargantuan space, and you need to explore that space to find one or two molecules that will bind with the target; that can be absorbed into the body; metabolise correctly; they’re not toxic; they can be synthesised, and all the rest of it. So, you’re trying to find that needle in a haystack. That’s not done by one person in an afternoon. That’s a team of dozens or hundreds of people working for many years.
As a scientist, in an ideal world, you would have read every paper that’s ever been written, and absorbed and internalised it. That’s impossible for a human being, but that’s something a large language model can do.
But it’s much more than that. An important aspect of science is that it involves experimentation. Fundamentally, it’s about evidence and about conducting experiments. In something like molecular research, you do lots of experiments. You’re getting the results of the experiments and you’re refining those hypotheses, and you’re going around that loop, often many times.
You go around that iterative process, but the steps are being accelerated by AI — very dramatically. In a sense, that’s the biggest news of what’s real today, versus what may or may not happen in the future.
MM: Why is Microsoft interested in investing in science? What’s exciting, in terms of progress, that will help companies like Microsoft?
CB: Microsoft has a leadership position in artificial intelligence. We have tremendous infrastructure and [are] building that out right now. Then the question is, where can that bring benefit? My view — but I think it’s one that’s shared broadly within the company — is that scientific discovery is an area that will see significant acceleration and disruption through AI.
Secondly, it’s of tremendous value to society that it underpins human development — it’s that fundamental. Then, what is Microsoft’s role as a company? It’s to accelerate the work and empower the work of others.
So, we think about drug discovery. We think about materials design, trying to develop batteries, photoelectric cells, methods for capturing CO₂, and so on. There are many organisations around the world doing this. We believe that AI will be a big accelerant. We believe we have world-leading AI technology. To be able to bring that to customers and partners aligns with Microsoft’s mission of empowering others. Something we would love to do in AI for Science is, through the research advances, to be able to create tools that can be used by scientists, possibly looking at broad ranges of applications.
MM: Do you see the current paradigm, where AI models, particularly LLMs, need more [computing power], data and bigger models to bring subsequent breakthroughs as a big gamble? Or are there alternative ways in which we’re going to get to more powerful AI systems that are going to bring us to all these other applications?
CB: Machine learning is a rich field. We’ve seen a particular framework that’s been very successful — the LLM and the particular scaling, architecture approach and so on. Because it’s been very successful, there’s been a lot of attention attracted to that. First of all, there are many different variants that can be explored for this. It’s not a fixed architecture.
There are so many different approaches and, of course, the field feels a lot bigger now. A lot more people are working in this area — more attention, more interest in the field. Just as in the days when neural nets ran out of steam and we saw that Cambrian explosion in that second phase of neural computing, I think there’s a tremendous amount of creativity that can be unlocked.
And so we start to see the need for complementary approaches or variations of current techniques. There’s a tremendous space to explore. The concept of a neural net is a general one, and we’ve seen certain architectures that work extremely well. We use LLM-like technology. For example, we have generative models for drugs. We’ve recently published some work called the TamGen molecular generator, which has produced a new potential drug molecule for tackling tuberculosis that’s 100 times more effective at binding to the target protein than the previous molecule.
It’s trained on the language of molecules. It can generate three-dimensional small organic molecules that are potential drug candidates. But we also use other architectures where appropriate. So it’s about the right machine-learning tool for the job.
Building those laws of physics into the architecture leads you to something that’s rather different in structure from, say, GPT-4. It remains to be seen how far this scaling law takes us. But even if, at some point, we find that that’s not the optimal path to pursue, there are many alternatives to explore as well.
MM: What will we see in the domain of AI in science over the next two to five years?
CB: The one thing that is now clear is this ability to take things that we knew how to do through lots and lots of [computing power], and we can now do them much faster, such as weather forecasting. This idea of an emulator, this thing we call the fifth paradigm, as it were, turns out to be very robust. We’ve seen it in many different scenarios. It’s a general purpose template, if you like, that we can apply in lots of different places.
In the next couple of years, we’ll see significant advances in this, probably in a number of different domains. Very likely we’ll see those landing in practical ways that scientists can use.
Source link
