Nobel Prize in Physics 2024 to J.J. Hopfield and to G.E. Hinton
Reporter: Aviva Lev- Ari, PhD, RN
UPDATED on 10/19/2024
The Nobel Prize Committee for Physics caught the academic community off-guard by handing the 2024 award to John J. Hopfield and Geoffrey E. Hinton for their foundational work in neural networks.
The pair won the prize for their seminal papers, both published in the 1980s, that described rudimentary neural networks. Though much simpler than the networks used for modern generative AI like ChatGPT or Stable Diffusion, their ideas laid the foundations on which later research built.
Even Hopfield and Hinton didn’t believe they’d win, with the latter telling The Associated Press he was “flabbergasted.” After all, AI isn’t what comes to mind when most people think of physics. However, the committee took a broader view, in part because the researchers based their neural networks on “fundamental concepts and methods from physics.”
“Initially, I was surprised, given it’s the Nobel Prize in Physics, and their work was in AI and machine learning,” says Padhraic Smyth, a distinguished professor at the University of California, Irvine. “But thinking about it a bit more, it was clearer to me why [the Nobel Prize Committee] did this.” He added that physicists in statistical mechanics have “long thought” about systems that display emergent behavior.
Hopfield first explored these ideas in a 1982 paper on neural networks. He described a type of neural network, later called a Hopfield network, formed by a single layer of interconnected neurons. The paper, which was originally categorized under biophysics, said a neural network could retain “memories” from “any reasonably sized subpart.”
Hinton expanded on that work to conceptualize the Boltzmann machine, a more complex neural network described in a 1985 paper Hinton co-authored with David H. Ackley and Terrence J. Sejnowski. They introduced the concept of “hidden units,” additional layers of neurons which exist between the input and output layers of a neural network but don’t directly interact with either. This makes it possible to handle tasks that require a more generalized understanding, like classifying images.
So, what’s the connection to physics?
Hopfield’s paper references the concept of a “spin glass,” a material in which disordered magnetic particles lead to complex interactions. Hinton and his co-authors drew on statistical mechanics, a field of physics that uses statistics to describe the behavior of particles in a system. They even named their network in honor of Ludwig Boltzmann, the physicist whose work formed the foundation of statistical mechanics.
And the connection between neural networks and physics isn’t a one-way street. Machine learning was crucial to the discovery of the Higgs boson, where it sorted the data generated by billions of proton collisions. This year’s Nobel Prize for Chemistry further underscored machine learning’s importance in research, as the award went to a trio of scientists who built an AI model to predict the structures of proteins.
While Hopfield and Hinton authored influential papers, their contributions to machine learning were cemented by their continued work, and both won multiple awards before the Nobel Prize. Among others, Hopfield won the Boltzmann Medal in 2022; Hinton received the IEEE Frank Rosenblatt Award in 2014, the IEEE James Clerk Maxwell Medal in 2016, and the Turing Award in 2018 (that last one alongside Yann LeCun and Yoshua Bengio).
Smyth saw Hopfield’s efforts first-hand as a student at the California Institute of Technology. “Hopfield was able to bring together mathematicians, engineers, computer scientists, and physicists. He got them in the same room, got them excited about modeling the brain, doing pattern recognition and machine learning, unified by mathematical theories he brought in from physics.”
In 2012, Hinton co-founded a company called DNNResearch with two of his students; Ilya Sutskever, who later co-founded OpenAI, and Alex Krizhevsky. Together, the trio collaborated on AlexNet, a hugely influential neural network for computer vision. Hinton also taught at the University of Toronto, where he continued to champion machine learning.
Navdeep Jaitly, now a deep learning researcher at Apple, said Hinton inspired new generations of engineers and researchers. In Jaitly’s case, the influence was direct; Jaitly studied under Hinton at the University of Toronto.
“I came in with experience in statistical modeling,” says Jaitly, “but Hinton still managed to entirely change how I think about problem solving. In terms of his contributions to machine learning, his methods are central to almost everything we do.”
SOURCE
https://spectrum.ieee.org/nobel-prize-in-physics
UPDATED on 10/13/2024
Website: https://www.bbc.co.uk/newsnight
https://youtu.be/MGJpR591oaM?si=6DhusxGt_B8dUpT_
10/8/2024
John J. Hopfield
Princeton University, NJ, USA
Born 1933 in Chicago, IL, USA. PhD 1958 from Cornell University, Ithaca, NY, USA. Professor at Princeton University, NJ, USA.
Geoffrey E. Hinton
University of Toronto, Canada
Born 1947 in London, UK. PhD 1978 from The University of Edinburgh, UK. Professor at University of Toronto, Canada.
was announced on 10/8/2024 in Stockholm, Sweden.
“for foundational discoveries and inventions that enable machine learning with artificial neural networks”
They trained artificial neural networks using physics
This year’s two Nobel Laureates in Physics have used tools from physics to develop methods that are the foundation of today’s powerful machine learning. John Hopfield created an associative memory that can store and reconstruct images and other types of patterns in data. Geoffrey Hinton invented a method that can autonomously find properties in data, and so perform tasks such as identifying specific elements in pictures.
John Hopfield invented a network that uses a method for saving and recreating patterns. We can imagine the nodes as pixels. The Hopfield network utilises physics that describes a material’s characteristics due to its atomic spin – a property that makes each atom a tiny magnet. The network as a whole is described in a manner equivalent to the energy in the spin system found in physics, and is trained by finding values for the connections between the nodes so that the saved images have low energy. When the Hopfield network is fed a distorted or incomplete image, it methodically works through the nodes and updates their values so the network’s energy falls. The network thus works stepwise to find the saved image that is most like the imperfect one it was fed with.
Geoffrey Hinton used the Hopfield network as the foundation for a new network that uses a different method: the Boltzmann machine. This can learn to recognise characteristic elements in a given type of data. Hinton used tools from statistical physics, the science of systems built from many similar components. The machine is trained by feeding it examples that are very likely to arise when the machine is run. The Boltzmann machine can be used to classify images or create new examples of the type of pattern on which it was trained. Hinton has built upon this work, helping initiate the current explosive development of machine learning.
Reference
Popular science background: They used physics to find patterns in information (pdf)
Scientific background: “For foundational discoveries and inventions that enable machine learning with artificial neural networks” (pdf)
SOURCE
https://www.nobelprize.org/prizes/physics/2024/press-release/
2012pharmaceutical
sjwilliamspa
Leave a Reply