On April 12th, Intel unveiled its latest generation AI accelerator, the Gaudi 3. Originally, the U.S. government wouldn't allow its sale to China, but Intel surprised by concurrently preparing a China-specific version!
Built on TSMC's 5nm process, the Gaudi 3 boasts 8 matrix engines, 64 tensor cores, 96MB SRAM cache, 1024-bit 128GB HBM2E memory (3.7TB/s bandwidth), along with 16 PCIe 5.0 lanes, 24 2000GbE networks, JPEG/VP9/H.264/H.265 decoders, available in three forms: OAM-compatible mezzanine cards, general-purpose boards, and PCIe expansion cards.
Intel claims it offers twice the FP8 AI compute power of its predecessor, four times the BF16 AI compute power, network power consumption doubled with 600W bandwidth, and memory bandwidth increased by 1.5 times. In comparison to NVIDIA H100 LLM, it leads with 50% better inference performance and 40% faster training times.
The inaugural version of Gaudi 3, coded HL-325L in its OAM form, has been shipped with a power consumption of 900W and air-cooled heat dissipation.
Its China-exclusive version, codenamed HL-328, is set to launch in June with halved power consumption at 450W. It retains HMB memory, cache, and decoders, but with the reduced power consumption, its computational power is expected to be roughly halved as well.
In October, a liquid-cooled version, HL-335, supporting dual-path parallelism, will be released, though it won't be available for sale in China and there won't be a special edition.
A PCIe expansion card version, air-cooled, will ship in September with the full-power edition coded HL-338 and the China-exclusive edition coded HL-388, both limited to 450W power consumption.
The universal board version, designated HLB-325, is also in the pipeline, but specific specifications and shipping dates are currently undisclosed.
When it comes to China-exclusive AI accelerators, NVIDIA was among the pioneers with products like the H800 and A800. However, these were eventually banned from sale. Recently, they've introduced new models like the H20, L20, L2, and RTX 4090D, all of which are available for purchase.
Similarly, AMD developed special versions like the Instinct MI309 and MI388X, but faced setbacks from the US government, citing concerns over their computational power being too robust. Nevertheless, they're gearing up for an upgrade to the MI350 by the end of the year.
