Field Trial of Multi-Datacenter Distributed Training for LLM Based on Bandwidth Convergence and Two Parallel Strategies over 120km High-reliability 800Gbit/s C+L OTN

We have conducted 175 billion parameters, 1024 GPUs large language model training with up to $99.41 \%$ (Pipeline parallel, PP) and $98.95 \%$ (Data parallel, DP) training efficiency in two distributed datacenters with an interconnection distance of 120 km carried by $800 \mathrm{Gbit} / \mathrm{s} \mathrm{C}$ plus L WDM in the field-deployed high-reliability optical transport network.