Recent market rumors suggested that major Chinese internet giants might significantly reduce orders for NVIDIA's H20 compute cards in the future. It was rumored that this was mainly due to some companies requiring those purchasing H20 to report and provide reasons for not using alternative solutions, leading to concerns about order cuts for H20.
This news seems to have impacted NVIDIA's stock price as well. Last Friday, after reaching a new high of $974, NVIDIA's stock price experienced a sharp pullback, dropping from a 5% increase to a 5% decrease in a single day. While this was primarily due to options trading, it may also be partly related to these rumors.
Recently, some institutions conducted interviews with leaders of NVIDIA's marketing department regarding this matter. Here are the specific responses:
Q: How does the performance of an H20 cluster compare to the Ascend 910?
A: There are bottlenecks between server cards and between servers. The inter-card bandwidth of the 910 is half that of the H20, and the inter-card interconnect performance is halved.
For server-to-server interconnects, NVIDIA has an IB switch with one and a half times the bandwidth of competing solutions in China, which suffer from significant computing power loss.
Q: H20's NVLINK is 900GB/s, while H800 is only 400GB/s. Can this be achieved?
A: NVLINK is not high-end for NVIDIA; it's quite mature. The H series can all support it.
With such a large network bandwidth support, more cards can be purchased to achieve it. The transmission speed of NVLINK can be ahead. This is something that other competitors can't achieve; currently, it's at 400G, and with eight cards connected, it's only 200.
Q: Have there been instances of significant order reductions in China recently?
A: As of yesterday, orders for H20 in China have remained stable, and the marketing department has not yet received reports of significant order reductions.
Q: Could you please introduce the advantages and disadvantages of H20?
A: As the global leader in AI chips, NVIDIA is the most mature in both hardware and software ecosystems. H20 has high cache and high bandwidth but lacks computational performance, mainly due to the requirements and restrictions of regulations.
While the computational performance of the H20 card is slightly inferior to the 910, its cache and bandwidth are much higher, with twice the bandwidth of the 910.
With NVLINK architecture, H20 can completely surpass the 910 through multi-card usage and stacking modes, and even surpass the H100.
Q: How's business expansion in the Chinese market?
A: H20, a customized version tailored for China, has been launched with significant emphasis on the Chinese market. We've initiated the promotion of H20, entrusting the strongest domestic technical team in China. During the event, there will be a network cluster of thousands of H20 units, showcasing truly impressive results.
Key Updates on NVIDIA:
H20:
Designed exclusively for domestic training purposes, with confirmed quantities of around 400,000 units. As acceptance of this card increases among domestic internet and other enterprises, there's substantial room for growth.
Price reduction: Initially priced at $23,000-$25,000 in January, dropped to $12,000-$14,000 in February, mainly due to pressure from the 910b model.
Single-card performance exceeds 60% of the 910b, with computational power reaching 20% of the H100.
Superior cluster performance, benefiting primarily from NVIDIA NVLINK technology.
B100:
Currently the most high-end training card available, boasting over three times the performance of the H100.
Limited supply since July '24, with mass production becoming the mainstay for training cards starting '25.
Expected pricing between $36,000-$39,000, based on H100 computational power and customer price acceptance, not exceeding $40,000.
Comes in low-power/high-power variants, with corresponding cooling methods being air-cooling/liquid-cooling, at a ratio of 60% liquid-cooling to 40% air-cooling.
NVIDIA Post-2025:
Estimated total training card volume around 5 million units by '25, with projections of 2 million B100s, 1 million H200s, and 1 million H100s.
'23-'25 will mark a peak period for NVIDIA training cards. By '26, with AI applications becoming widespread, the growth rate of AI servers will gradually level off. This trend will prompt general-purpose servers to gradually recover from the impact of AI servers and reverse the previous negative growth trend. It's anticipated that by '27, the general-purpose server market will resume positive growth.
Shifts in Dynamics:
Anticipated positive growth in the general-purpose server market by '27.
No impact on AI server foundries; the dynamics between inference and training will balance out, keeping the total volume stable.
Potential impact on card foundries as the growth rate of training cards slows in '25, prompting these foundries to seek other production capacity partnerships.
