Intel recently hosted the Vision 2024 Annual Industry Innovation Conference, which showcased several highlights. Among them were the next-generation AI accelerator Gaudi 3, touted to significantly surpass NVIDIA's H100, the brand-new upgrade of Xeon 6 processors, and the next-generation ultra-low-power processor Lunar Lake, which boasts a substantial increase in AI computing power. These announcements captured significant attention.
However, for AI developers, the AI industry, and especially for enterprise AI, there was another major development at this conference:
Intel, along with numerous industry giants, launched an open enterprise AI platform to drive innovation in enterprise AI applications. Concurrently, they are advancing enterprise AI high-speed interconnect network innovation through the Unified Ethernet Coalition (UEC) and a series of AI-optimized Ethernet solutions.
When it comes to large-scale AI deployment, two names immediately come to mind for many:
One is OpenAI, dominating with its large models and algorithms; the other is NVIDIA, a typical representative of hardware power and ecosystem.
However, whenever NVIDIA is mentioned, along with its ecosystem represented by CUDA, its consistent closed approach is often criticized. Jim Keller, revered as a chip development guru, has been highly dissatisfied with NVIDIA's practices, condemning CUDA as not a moat but a swamp, and suggesting that proprietary interconnect standards like NVLink should be abandoned.
To be honest, NVIDIA AI not only "far exceeds" in hardware performance but also capitalizes heavily on its CUDA ecosystem, built over more than a decade and countless dollars, becoming its "monopolistic" and immensely profitable ace in the hole.
However, times are changing. Whether it's enterprises or developers, nobody wants to be confined to a small circle; they prefer to move forward freely in an open and shared world. This presents a great opportunity for other vendors to catch up with or even surpass NVIDIA.
Intel has seized upon this trend and demand. It has announced a joint creation of an enterprise AI open platform with numerous industry partners such as Anyscale, Articul8, DataStax, Domino, Hugging Face, KX Systems, MariaDB, MinIO, Qdrant, RedHat, Redis, SAP, VMware, Yellowbrick, and Zilliz, aiming to assist enterprises in driving AI innovation.
It will provide a complete platform for enterprise AI from bottom to top, with the underlying hardware based on Intel's XPU concept, covering AI compute hardware in various fields such as cloud, data centers, edge, and PCs.
Built upon this is a standardized and scalable infrastructure ecosystem, a secure and reliable software ecosystem, and an open and convenient application ecosystem, all of which are open to the entire industry.
This initiative harnesses the collective strength of the entire industry, aiming to develop open and multi-vendor AIGC systems. Through RAG (Retrieve-Augment-Generate) technology, it offers first-class deployment convenience, performance, and value.
For enterprises with a large number of proprietary data sources running on standard cloud infrastructure, RAG can help enhance their capabilities through open large language models, thereby accelerating the application of AIGC in enterprises.
According to Intel, adhering to open source principles and leveraging them to drive the rapid development of an open AI ecosystem is crucial.
Li Ying, Vice President of Intel and General Manager of Intel China Software and Advanced Technology Business Unit, stated in an interview, "In the traditional model, several leading companies form an open alliance with clear division of labor, more of a matter of choice. Now, with the changes brought by AI large models, the first integration of open source with the entire industry's technological innovation explosion has occurred. Today, both open source and closed source are present simultaneously, no longer a choice, but a natural process of evolution and development."
Dr. Dai Jinqian, Intel Fellow, Global Chief Technology Officer of Big Data Technologies, and Dean of Big Data Analytics and Artificial Intelligence Innovation Institute, also pointed out that an open ecosystem allows for mutual promotion of innovation within the same ecosystem, fostering interoperability in new application scenarios.
The entire industry is gradually realizing that building an AIGC (Artificial Intelligence and General Computing) application requires more than just a large model. Some of the most advanced system solutions are actually akin to building an AI system to address problems.
In this enterprise AI open platform, a key focus for Intel is accelerating the development of an open AI software ecosystem. By building foundational software, Intel aims to provide convenience for developers and help large enterprises simplify and deepen their AI development and deployment at scale.
According to Li Ying, a crucial aspect for Intel in fostering an open AI software ecosystem is how to accelerate enterprise AI development through software and integrate existing cloud architectures with future AI architectures based on large models and data.
Software plays a significant role in this process, and from the perspective of the entire software stack, Intel is one of the very few top-tier companies that can genuinely provide optimization and technology across various levels through software.
At the same time, Intel has been vigorously promoting AI-based software innovation, with oneAPI being the most typical example, with downloads exceeding 1 million.
Thirdly, Intel focuses on helping developers improve development efficiency, with a crucial part being the Intel Developer Cloud platform.
This platform not only enables developers to access the latest Xeon and Gaudi AI accelerators early but also ensures compatibility of various open-source frameworks and components in the same environment, thereby enhancing development efficiency and optimizing user experience.
It's also worth mentioning that Intel is actively contributing technology, innovation, and expertise to the open-source community, advancing open standards.
For example, PyTorch has gradually become a standard AI framework, with Intel being a significant contributor to PyTorch and joining the PyTorch Foundation as a distinguished member.
In addition to optimizing PyTorch itself, some of Intel's technological innovations have been integrated into the PyTorch open-source framework, enabling more companies and developers to benefit and making the entire AI software framework more open and accessible.
Furthermore, in the realm of open-source Chinese Linux communities like openEuler and Dragonfly, Intel's contributions are in sync with international standards and even leading in certain areas compared to other Linux distributions and developments worldwide.
Let's take a look back at the three major compute products unveiled by Intel this time.
The Gaudi 3 AI accelerator is built on TSMC's 5nm process and features 8 matrix engines, 64 tensor cores, 96MB SRAM cache, 1024-bit 128GB HBM2E memory (bandwidth of 3.7TB/s), along with 16 PCIe 5.0 channels, 24 2000GbE networks, and JPEG/VP9/H.264/H.265 decoders.
It comes in two power levels: 600/900W, offering OAM-compatible mezzanine cards, general-purpose boards, and PCIe expansion cards.
Compared to its predecessor, Gaudi 3 boasts twice the FP8 AI compute (1835TFlops), four times the BF16 AI compute, twice the network bandwidth, and 1.5 times the memory bandwidth.
Intel also claims a 50% lead in inference performance over NVIDIA H100 LLM and 40-70% faster training times, with energy efficiency leading by up to 2.3 times.
Additionally, thanks to powerful and convenient development tools, developers only need to change a minimum of 3 lines of code to port other AI applications onto Gaudi 3.
The new Xeon 6 series consists of two branches, with Sierra Forest released in the second quarter, marking the biggest transformation in the history of Xeon processors, featuring the first-ever adoption of Energy Efficient Cores (E-cores).
It focuses on efficiency, ideal for high-density, scalable workloads, supporting up to 288 cores and 288 threads.
According to official statements, Sierra Forest promises a 2.4x improvement in energy efficiency compared to the 2nd Gen Xeon, while rack density could increase by up to 2.7 times.
On the other hand, Granite Rapids follows a traditional Pure Performance Core (P-core) design, emphasizing performance optimization, suitable for compute-intensive applications and high-intensity AI workloads.
It introduces software support for MXFP4 data format, capable of running the Llama 2 model with 700 billion parameters, offering up to 6.5 times reduction in token latency compared to the 4th Gen Xeon.
The next-generation ultra-low-power Core i9 Ultra processor, codenamed Lunar Lake, will boast AI performance exceeding 100TOPS (100 trillion operations per second), tripling the capabilities of the current generation Core i9 Ultra Meteor Lake!
Specifically, the NPU unit alone will deliver approximately 45TOPS of computing power, quadrupling the current capacity, meeting Microsoft's definition of the next-generation AI PC requirements.
It's fair to say that Intel possesses the most comprehensive AI infrastructure to date, covering hardware acceleration across the cloud, data center, edge, and client XPU, complemented by network solutions, development tools, and extensive ecosystem partnerships. With the establishment of an open enterprise AI platform, Intel is poised to excel in AI training, inference, and AIGC domains.
