On April 17th, local time, chip giant Intel announced the creation of the world's largest neurosynaptic system, boasting 11.5 billion neurons and 128 billion synapses, with speeds up to 200 times faster than the human brain.
The large neuromorphic system, codenamed Hala Point, was initially deployed at Sandia National Laboratories, utilizing Intel's Loihi 2 processor. It is designed to support research into future brain-inspired artificial intelligence (AI) and address challenges related to the efficiency and sustainability of current AI technologies.
Hala Point has propelled Intel's first-generation large-scale research system, Pohoiki Springs, with architectural improvements that achieve over a tenfold increase in neuron capacity and up to twelve times the performance.
Mike Davies, Director of Intel's Neuromorphic Computing Lab, stated, "The computational costs of today's AI models are growing at an unsustainable rate. The industry needs entirely new approaches that can scale. That's why we developed Hala Point, which combines deep learning efficiency with novel neuromorphic learning and optimization capabilities. We hope that research with Hala Point will enhance the efficiency and adaptability of large-scale AI technologies."
Hala Point integrates 1152 Loihi 2 chips and over 2300 embedded x86 processors.
Loihi 2 neuromorphic processors form the foundation of Hala Point, applying brain-inspired computing principles such as asynchronous, event-based spiking neural networks (SNN), integrated memory and computation, as well as sparse and constantly changing connections to achieve orders of magnitude improvements in energy efficiency and performance. Neurons communicate directly with each other rather than through memory, thereby reducing overall power consumption.
△Loihi 2 Chip
Loihi 2, based on Intel 4 technology, features a core area of 31mm² and integrates 128 Neuromorphic Cores (each with 192KB of cache) along with 6 low-power Intel X86 cores. Thanks to significant improvements in the fabrication process, Loihi 2 boasts a million neurons, a 7.8-fold increase over the first generation, although the synapse count slightly decreases to 120 million.
Loihi 2 can allocate up to 4096 variable states based on neuron model requirements. These enhancements enable Loihi to achieve processing speeds ten times faster than the first-generation Loihi.
Hala Point has packaged 1152 Loihi 2 processors produced on the Intel 4 process node into a microwave-sized six-rack-unit data center enclosure. The system supports up to 1.15 billion neurons and 128 billion synapses distributed across 140,544 neuromorphic computing cores, with a maximum power consumption of 2,600 watts. It also includes over 2,300 embedded x86 processors for auxiliary computation.
Hala Point integrates processing, memory, and communication channels into a massively parallelized architecture, offering a total memory bandwidth of 16 PB/s, inter-core communication bandwidth of 3.5 PB/s, and chip-to-chip communication bandwidth of 5 TB/s. The system can handle over 3.8 quadrillion 8-bit synapses and more than 2.4 quadrillion neural operations per second.
Applied to spiking neural network models, Hala Point can execute its full capacity of 1.15 billion neurons, outpacing the human brain by 20 times in speed and up to 200 times at lower capacities. While not intended for neuroscientific modeling, its neuron capacity roughly matches that of the cerebral cortex of an owl or a macaque monkey.
Systems based on Loihi can perform AI inference and solve optimization problems, boasting energy efficiency 100 times higher and speeds 50 times faster than traditional CPU and GPU architectures. By leveraging sparse connections of up to 10:1 and event-driven activity, early results from Hala Point indicate it can achieve efficiency of up to 15 TOPS/W in deep neural networks without the need for batched input data collection, a common optimization of GPUs that can significantly delay processing of real-time incoming data, such as video from cameras.
While still under research, future neurally plausible LLMs capable of continual learning could save gigawatt-hours of energy by eliminating the need for periodic retraining on ever-growing datasets.
Intel has announced that Hala Point is the first large-scale neuromorphic system to demonstrate state-of-the-art computational efficiency on mainstream AI workloads.
Characterization indicates that it can support up to 200 trillion operations per second, or 200 petaops, when executing traditional deep neural networks, surpassing the efficiency of 150 trillion 8-bit operations per second per watt (TOPS/W). This rivals and surpasses the levels achieved by architectures built on graphics processing units (GPUs) and central processing units (CPUs).
The unique capabilities of Hala Point can enable future real-time continual learning for AI applications such as scientific and engineering problem-solving, logistics, smart city infrastructure management, large language models (LLMs), and AI agents.
Hala Point's Role and Significance
Researchers at Sandia National Laboratories plan to use Hala Point for advanced brain-scale computing research. The organization will focus on addressing scientific computing challenges in device physics, computer architecture, computer science, and informatics.
"Collaborating with Hala Point has enhanced our Sandia team's ability to tackle computational and scientific modeling challenges. Conducting research with systems of this scale will enable us to keep pace with the advancements of AI across domains from commercial to defense and fundamental science," said Craig Vineyard, the lead of Sandia National Laboratories' Hala Point team.
Currently, Hala Point is a research prototype poised to enhance the capabilities of future commercial systems. Intel anticipates that these learnings will lead to tangible advancements, such as LLMs being able to continually learn from new data. These advancements are expected to significantly alleviate the unsustainable training burdens associated with widespread AI deployment.
The recent trend of expanding deep learning models to tens of trillions of parameters has exposed daunting sustainability challenges for AI and underscores the necessity of innovation at the lowest hardware architecture levels.
Neuromorphic computing represents a novel approach that draws insights from neuroscience, integrating memory and computation with highly refined parallelism to minimize data movement. Loihi 2 demonstrated orders of magnitude improvements in efficiency, speed, and adaptability for emerging small-scale edge workloads in results presented at this month's International Conference on Acoustics, Speech, and Signal Processing (ICASSP).
Building upon its predecessor, Pohoiki Springs, Hala Point has undergone significant enhancements, now bringing neuromorphic performance and efficiency boosts to mainstream traditional deep learning models, particularly those handling real-time workloads like video, speech, and wireless communication. For instance, Ericsson Research is applying Loihi 2 to optimize the efficiency of telecommunications infrastructure, as emphasized at this year's Mobile World Congress.
According to Intel, the next step involves delivering Hala Point to Sandia National Laboratories, marking the inaugural deployment of a new series of large-scale neuromorphic research systems that Intel plans to share with its research collaborators. Further development will enable neuromorphic computing applications to overcome power and latency constraints, limiting real-world real-time deployment of AI capabilities.
Intel, along with the ecosystem comprising over 200 Intel Neuromorphic Research Community (INRC) members including leading academic groups, government labs, research institutions, and companies, is committed to pushing the boundaries of brain-like AI and transitioning this technology from research prototypes to industry-leading commercial products in the coming years.
