Artist’s rendition of a computer system based on light that could jumpstart the power of machine-learning programs like ChatGPT. Blue sections represent the micron-scale lasers key to the technology. Credit: Ella Maru Studio
ChatGPT has made headlines around the world with its ability to write essays, email, and computer code based on a few prompts from a user. Now an MIT-led team reports a system that could lead to machine-learning programs several orders of magnitude more powerful than the one behind ChatGPT. The system they developed could also use several orders of magnitude less energy than the state-of-the-art supercomputers behind the machine-learning models of today.
In a recent issue of
Toward the Future
In the paper, the team also cites “substantially several more orders of magnitude for future improvement.” As a result, the authors continue, the technique “opens an avenue to large-scale optoelectronic processors to accelerate machine-learning tasks from data centers to decentralized edge devices.” In other words, cell phones and other small devices could become capable of running programs that can currently only be computed at large data centers.
Further, because the components of the system can be created using fabrication processes already in use today, “we expect that it could be scaled for commercial use in a few years. For example, the laser arrays involved are widely used in cell phone face ID and data communication,” says Zaijun Chen, first author, who conducted the work while a postdoc at MIT in the Research Laboratory of Electronics (RLE) and is now an assistant professor at the University of Southern California.
Says Dirk Englund, an associate professor in MIT’s Department of Electrical Engineering and Computer Science and leader of the work, “ChatGPT is limited in its size by the power of today’s supercomputers. It’s just not economically viable to train models that are much bigger. Our new technology could make it possible to leapfrog to machine-learning models that otherwise would not be reachable in the near future.”
He continues, “We don’t know what capabilities the next-generation ChatGPT will have if it is 100 times more powerful, but that’s the regime of discovery that this kind of technology can allow.” Englund is also leader of MIT’s Quantum Photonics Laboratory and is affiliated with the RLE and the Materials Research Laboratory.
A Drumbeat of Progress
The current work is the latest achievement in a drumbeat of progress over the last few years by Englund and many of the same colleagues. For example, in 2019 an Englund team reported the theoretical work that led to the current demonstration. The first author of that paper, Ryan Hamerly, now of RLE and NTT Research Inc., is also an author of the current paper.
Additional coauthors of the current Nature Photonics paper are Alexander Sludds, Ronald Davis, Ian Christen, Liane Bernstein, and Lamia Ateshian, all of RLE; and Tobias Heuser, Niels Heermeier, James A. Lott, and Stephan Reitzensttein of Technische Universitat Berlin.
Deep neural networks (DNNs) like the one behind ChatGPT are based on huge machine-learning models that simulate how the brain processes information. However, the digital technologies behind today’s DNNs are reaching their limits even as the field of machine learning is growing. Further, they require huge amounts of energy and are largely confined to large data centers. That is motivating the development of new computing paradigms.
Optical Neural Networks and Their Potential
Using light rather than electrons to run DNN computations has the potential to break through the current bottlenecks. Computations using optics, for example, have the potential to use far less energy than those based on electronics. Further, with optics, “you can have much larger bandwidths,” or compute densities, says Chen. Light can transfer much more information over a much smaller area.
However, current optical neural networks (ONNs) have significant challenges. For example, they use a great deal of energy because they are inefficient at converting incoming data based on electrical energy into light. Further, the components involved are bulky and take up significant space. while ONNs are quite good at linear calculations like adding, they are not great at nonlinear calculations like multiplication and “if” statements.
In the current work, the researchers introduce a compact architecture that, for the first time, solves all of these challenges and two more simultaneously. That architecture is based on state-of-the-art arrays of vertical surface-emitting lasers (VCSELs), a relatively new technology used in applications including lidar remote sensing and laser printing. The particular VCELs reported in the Nature Photonics paper were developed by the Reitzenstein group at Technische Universitat Berlin. “This was a collaborative project that would not have been possible without them,” Hamerly says.
Logan Wright, an assistant professor at