Tech

AI scaling problem? MIT proposes optical sub-photon deep learning at the edge


Photonic equation against colorful background.

MIT scientists used special photodetectors to execute an AI equation simply by shining a light beam at a low-power client device. This method can be especially useful in deep space.

Tiernan Ray / ZDNET

One of the most pressing concerns for the industrial application of artificial intelligence is a way of running programs on small computing devices with very little processing power, very little memory, and possibly a limit on available power, in the case of batteries.

The so-called competitive market for AI was a large area at the end of the year, with startups get tens of millions of venture capital to find the chip and software. Exceptional effort has led to tools developed specifically for machine learning forms of AI, such as TinyML Initiative from Google.

Those two paths represent two philosophies: Either make edge devices more powerful or downsize AI programs to use less computation.

Also: The real goal of AI may no longer be intelligence

There could be a third approach, which is to try and balance more carefully the work done on constrained devices and by what means. That’s the plan to come out in October by MIT researchers in the academic journal Science.

Researcher Alexander Sludds and colleagues at MIT’s Electronics, Computer Science, and Artificial Intelligence Research Laboratory and Lincoln Laboratory, in partnership with Nokia and NTT . Researchdeveloped a system that uses photonics to transmit data to a client device where it can be computed in the optical domain in a much more energy efficient manner.

Their network setup, which they call Netcast, can perform the basic operation of manipulating the weights or parameters of a deep neural network, using about 10 femtoJoules of power, or 10 fJ, according to they, “are three orders of magnitude lower than is possible in existing digital CMOS” – that is, standard semiconductor chips.

A femtoJoule, written as a decimal point, followed by 14 zeros and 1, is a quarter of a quarter, with being a very small fraction of a joule, the joule being the amount of electricity to run a 1-watt device for one second.

A tiny, very small fraction of a watt is a huge and important energy savings because many edge devices, the authors note, will have a total power budget in milliwatts, or thousandths of a watt, compared to Common computing devices use tens or hundreds of watts. Netcast’s femtoJoule operation brings the show below what has so far been a “stubborn bottleneck near 1 pJ,” aka a picoJoule, or trillionth of a joule.

Also: AI edge chip market is on fire, fueled by ‘incredible’ VC funding

The key to Netcast is how to reduce the work the client has to do for the underlying neural network operation to get within that 10 femtoJoule budget.

The neural network makes predictions by passing some input data to its parameters or weights and multiplying the input by the weights. That operation, the product of an input vector and a parameter matrix, is called a cumulative multiplication or MAC operation, and neural network programs do tons of them per second when multiple weights of each network layer are assigned. apply to input .

The biggest power for most neural networks in general is getting data from RAM memory chips and accessing the network. That’s a problem because the neural weights are usually stored in RAM, so any active layer of the MAC may require multiple outbound trips over the PCIe bus to RAM and possibly even to the network line card to remote memory storage.

Therefore, the key to Netcast is how to minimize memory access and network traffic for client devices.

Netcast concept diagram with explanation below it

Sludds et al.

The solution is an existing photonics technology called wavelength division multiplexing. Using WDM, as it is commonly called, multiple pieces of data can be sent over a fiber line simultaneously by assigning each piece of data its own wavelength of light so that the multiple pieces of data share the total radiation spectrum. available in fiber optic. WDM is a very mature, solid technology used in all modern telecommunications networks to increase fiber optic data transmission capacity; it forms the backbone of the Internet.

Each row of the matrix can be encoded on one wavelength of light and then “broadcast” to the client, so that the multi-wavelength WDM signal can send the entire weight matrix or even multiple matrices. . At the client device, the optical receiver recovers the wavelength-encoded data and combines it with the input data to perform matrix multiplication in the optical rather than electrical domain. The product can then be electrically stored on local RAM after being converted from the optical signal.

Sludds and team write that this leads to a significant simplification of the components required in the edge client.

This architecture minimizes the components that operate at the client, requiring only a single optical transceiver modulator, a digital-to-analog converter (DAC), and an analog-to-digital converter. (ADC).”

The authors built a real-life version of Netcast running over 84 kilometers of fiber using WDM at 2.4 terabits per second, running from MIT’s main campus to Lincoln Labs and back. Their test of the system was to perform predictions on a classical machine learning task, MNIST database of handwritten characters. Images of handwritten characters are entered into the neural network, and the network has to perform the image recognition task, determining which character each picture represents.

“Using 1,000 local test images, we demonstrate computation accuracy of 98.7%, comparable to the model’s baseline accuracy of 98.7%,” they report.

The authors go even further. Anticipating deployment in satellite and other exotic languages, they worked to come up with photodetectors, called integrated receivers, that could operate with very small numbers of photons.

“Netcast applications, including deployment in empty space for drones or spacecraft, can operate in deep-photon-deficient environments,” they wrote. One version of their built-in receivers that can detect the result of an active MAC operation uses only a fraction of the femtoJoule, called attoJoule, requires only 100 photons for the MAC operation.

But the authors go even further. They were able to get to a theoretical limit of Netcast, where each Mac requires less than one photon to be detected. Using what is known as a superconducting nanowire single-photon detector (SNSPD), they create a receiver that can measure the results of each MAC with less than one photon of information.

Sludds and team write: “This result may at first seem surprising because less than one photon per MAC is counterintuitive. “We can better understand this measurement by noting that at the time of our reading we performed a vector-vector product with M = 100 MAC. Each MAC can have less than one photon in it, but the signal is measured will have many photons in it.”

The effects on computers can be profound.

“Performing computations with less than one photon per MAC,” they write, “could enable a new class of computational systems that protect both client input and server weight data” according to the report. perspective of data privacy. It could also make calculations on spacecraft more reliable. “Weight data from a directional base station can be transmitted to the spacecraft and classified on board, before the results are transmitted back to Earth.”

All of Netcast’s parts today can be manufactured in any standard semiconductor chip factory, Sludds and the team note.

In their conclusion, they write, “Our approach removes a fundamental bottleneck in edge computing, enabling high-speed computing on deployed sensors and drones.”

news7f

News7F: Update the world's latest breaking news online of the day, breaking news, politics, society today, international mainstream news .Updated news 24/7: Entertainment, Sports...at the World everyday world. Hot news, images, video clips that are updated quickly and reliably

Related Articles

Back to top button