NVIDIA CEO Reveals GPT-6 Astra Trained Using 100,000 Blackwell GPUs

NVIDIA CEO Jensen Huang disclosed that the training of GPT-6 Astra utilized over 100,000 Grace Blackwell GPUs, forming a high-speed interconnected cluster via NVLink72. The next batch will feature 400,000 GPUs, although the specific models were not specified. Leading model companies in China still show a significant gap in computing power, with ByteDance currently the closest, connecting approximately 36,000 B200 GPUs. ByteDance's domestic clusters primarily use the Hopper architecture's H20 and H800. Kimi obtained around 20,000 Hopper GPUs through Alibaba, specifically the H200 model, but Alibaba denied this claim. The H200 belongs to the previous generation of Hopper, while the B200 has been upgraded to Blackwell. The GB200 combines Grace CPU and Blackwell GPU, with NVL72 capable of placing 72 GPUs in the same high-speed interconnect domain. DeepSeek has not disclosed the complete training hardware for V4, but leaked information indicates the company had about 20,000 H-equivalent computing power in May, with Huawei providing approximately 16,000 units of the 950 capacity. Liang Wenfeng stated that this batch of domestic cards is equivalent to 4,000 Nvidia B-series cards, sufficient for training the current generation of models. The Blackwell used by Astra is not NVIDIA's latest generation; Vera Rubin has entered full-scale production, and NVIDIA estimates that the number of GPUs required for training large MoE models with Rubin can be reduced to a quarter of that needed for Blackwell. Currently, no Chinese company has disclosed a case of using 100,000 advanced GPUs of the same generation for a single model training, highlighting the gap between Chinese and American AI companies in terms of both quantity and generation of cards.

This content is provided for general informational purposes only and doesn't constitute financial, investment, legal, or tax advice. Any events, rewards, online promotions, or related information mentioned herein should not be considered a recommendation, solicitation, or invitation to purchase, sell, trade, or otherwise deal in any crypto assets. Crypto assets are highly volatile and may result in loss. The availability of WEEX services, products, and related events may vary by region. You are responsible for ensuring that your participation is in accordance with applicable local laws and regulations.