Recently, on April 2nd, MLCommons unveiled the benchmark results for AI inference in MLPerf v4.0. Among them, the Intel Emerald Rapids 5th Gen Xeon processors demonstrated outstanding performance, showcasing a significant improvement compared to the 4th Gen Sapphire Rapids.
Up to now, Intel remains the sole vendor to submit MLPerf CPU test results. Since 2020, they have been submitting results based on the 4th Gen Xeon processors, and now, the 5th Gen Xeon has also joined the fray.
Specifically, the 5th Gen Intel Xeon, after hardware and software optimizations, exhibits an average performance improvement of 1.42x compared to the 4th Gen Xeon in MLPerf v3.1 benchmarks.
For instance, for software-optimized models like GPT-J with features such as continuous batching, the performance improvement is approximately 1.8x.
With additional optimizations like MergedEmbeddingBag and Intel AMX accelerator, DLRMv2 demonstrates a test performance improvement of around 1.8x, achieving an accuracy of 99.9%.
Intel is collaborating with OEM vendors such as Cisco, Dell, Inspur, AMD, and WeRide Tech to submit MLPerf test results based on their respective products.
