Intel Gaudi 3 AI Chip (Image Source: Intel Official Website)
There's a new development regarding the "China-exclusive edition" AI chip of Intel's Gaudi 3.
According to the Titanium Media App on April 15th, chip giant Intel recently revealed in a 24-page "Gaudi 3 AI Accelerator White Paper" on its official website that it will launch two "exclusive edition" AI chip products of Gaudi 3 for sale in China.
Specifically, these include two hardware form-factor accelerator cards: one is the HL-328 OAM-compatible mezzanine card, expected to be released on June 24th this year; the other is the HL-388 PCIe accelerator card, expected to be launched on September 24th this year. Based on estimates of core count, operating frequency, TDP, and other parameters, compared to the international version of Gaudi 3, the performance of the "China-exclusive edition" HL-328 chip may be reduced by around 92%.
At the American Intel Vision 2024 conference held on April 9, Intel unveiled its next-generation Gaudi 3 AI accelerator chip, fabricated using TSMC's 5nm process, boasting a whopping 400% increase in BF16 AI computing capability. Moreover, compared to Nvidia's H100 GPU, Intel's Gaudi 3 AI chip exhibits a 40% improvement in model training speed, a 50% boost in inference speed, a 50% average performance increase, and a 40% average efficiency improvement, all at a fraction of the cost of the H100.
Regarding the news of the Gaudi 3 China Special Edition, Titanium Media App has reached out to Intel's US headquarters for further details via email. However, as of the time of publication, Intel has not responded.
It's worth noting that over recent times, the US government has been progressively tightening export controls on semiconductor and AI chip technologies to China.
On August 9, 2022, President Biden signed the "Chip and Science Act," totaling a staggering $280 billion. With provisions including a hefty $52.7 billion industrial subsidy and clauses aimed at curbing competition, the act aims to facilitate the "reshoring" of chip manufacturing to the US mainland. The legislation prohibits subsidized US and allied companies from establishing or expanding advanced process chip factories in China and other concerning countries for a period of 10 years.
On October 7, 2022, and October 17, 2023, the US Department of Commerce's Bureau of Industry and Security (BIS) issued consecutive export controls on advanced semiconductors and computing devices to China, aiming to prevent the import of AI semiconductor products by China. Consequently, several GPU and AI chip products from Nvidia, AMD, and Intel can no longer be exported to China, including high-end gaming graphics cards like the RTX 4090, which are also restricted.
In December 2023, the BIS announced an investigation into mature process node semiconductor supply chains, targeting China's chip semiconductor industry.
On March 29, 2024, the BIS updated its export restriction measures, including adopting a "case-by-case review" policy for US exports of AI semiconductor products to China. This entails comprehensive scrutiny of technical levels, customer identities, compliance plans, etc. Additionally, some AI PC chips are included in the export control measures, resulting in broader restrictions on the sale of advanced AI chips and semiconductor equipment from companies like Nvidia and AMD to China, effective April 4.
On April 11, the US Department of Commerce updated the latest Entity List in the Federal Register, adding six Chinese companies, including Sitonholy (Tianjin) Co., Ltd., Nvidia's largest AI chip distributor in China.
In fact, under the initial round of export controls by the US BIS, as early as July 2023, Intel released the "China Special Edition" AI chip Gaudi 2 based on the 7nm process. Compared to the international version of Gaudi 2, the performance difference of the acceleration card launched for the Chinese market was not significant. However, the number of integrated Ethernet RDMA ports was reduced from 24 to 21 to comply with US chip export control regulations (see Titanium Media App's previous article: "Intel releases China Special Edition 7nm AI chip Gaudi 2, reducing RDMA interfaces to comply with export control regulations").
Nevertheless, Titanium Media App learned that Intel's Gaudi 2 China Special Edition only sold a few dozen OAM products. Since the performance of Intel's Gaudi 2 "China Special Edition" products exceeded the export control measures, they ultimately could no longer be sold in the Chinese market after the updated BIS 1017 regulation last year.
Around December of last year, Intel attempted to develop a variant of Gaudi 2, known internally as Gaudi 2 HL-225B or Gaudi 2C AI chip, hoping to regain permission to sell to mainland China. However, it wasn't until March of this year that the BIS announced updated export restrictions.
Following the release of Gaudi 3 on April 10, Intel has once again developed special edition products, aiming to compete vigorously in the Chinese market with the 5nm Gaudi 3 AI chip in the future, providing an "alternative choice" for Nvidia's products to AI and cloud customers.
In terms of specific hardware specifications, compared to the international version, the China Special Edition of Gaudi 3 features the same 96MB SRAM on-chip memory, 128GB HBM2e high-bandwidth memory with a bandwidth of 3.7TB/s, PCIe 5.0 x16 interface, and decoding standards. However, due to US export control regulations on AI chips, the comprehensive computational performance (TPP) of such high-performance AI needs to be below 4800 to be exported to China. This means that the 16-bit performance of the China Special Edition Gaudi 3 cannot exceed 150 TFLOPS.
Compared to the 1835 TFLOPS achieved by the international version of Gaudi 3 (FP16/BF16), the China Special Edition Gaudi 3 will likely need to significantly reduce the number of cores and operating frequency, resulting in an approximately 92% decrease in AI performance to comply with US export control requirements.
Furthermore, due to the reduced AI performance of the China Special Edition Gaudi 3 products, their TDP (thermal design power) is expected to be substantially lower, with estimated TDPs of 450 watts for both OAM cards and PCIe cards, while the original PCIe card (HL-338) has a TDP of up to 600 watts, and the original OAM cards (HL-325L, HL-335) have a TDP of up to 900 watts. Overall, the AI performance of the two "special edition" Gaudi 3 products launched by Intel specifically for the Chinese market may be comparable to NVIDIA's "China Special Edition" AI accelerator card, the H20. The latter, compared to the H100, has an overall performance reduction of about 80%, reaching around 148 TFLOPS of FP16/BF16 performance, slightly below the export-controlled limit of 150 TFLOPS.
Currently, NVIDIA's H20 AI chip has been sampled to customers in mainland China, but leading Chinese AI companies such as Baidu and Alibaba have shown lukewarm reactions. Industry insiders attribute this mainly to the low performance and high price of the H20, which has dampened corporate purchasing intentions.
US Commerce Secretary Gina Raimondo previously stated at a public forum that the US cannot allow China to access these cutting-edge chips and cannot allow China's chip technology to surpass that of the US. The US government does not permit NVIDIA to sell the "most complex, highest performing" AI chips to China to prevent China from being capable of training cutting-edge AI models.
Additionally, there are reports that the US government is also preparing a list of Chinese advanced chip manufacturing facilities that will be prohibited from receiving critical tools, making it easier for US companies to prevent technology from flowing into China. This list may be released in the coming months. Meanwhile, the US is increasing pressure on allies including the Netherlands, Japan, Germany, and South Korea, urging them to further tighten restrictions on China's access to semiconductor technology.
In response to the US actions, Foreign Ministry spokesperson Zhao Lijian stated on April 11th, "We firmly oppose the US' abuse of export control tools such as the Entity List to suppress Chinese companies and urge the US to stop politicizing, weaponizing, and tooling economic and technological issues. China will take necessary measures to firmly defend the legitimate rights and interests of Chinese companies."
(Originally published on the Titanium Media App, Author: Lin Zhijia)
