The era of artificial intelligence (AI) has arrived. The key to AI is the tight integration of hardware and software, as well as improvements in performance and computing power. However, the world's data center and network infrastructure is under tremendous pressure due to the combination of growing data demands and increasingly challenging power constraints.
The volume of connected devices and data continues to grow, which means more power and higher performance. Coupled with the decline of Moore's Law, the cost and complexity of system-on-chip (SoC) design continues to rise. As a result, the industry expects efficient, specialized processing products for innovation.
Recently, at the Arm Tech Symposia annual technology conference in Shenzhen, Mohamed Awad, senior vice president and General manager of the Infrastructure Division of Arm, told reporters, "In recent years, Arm has transformed itself into a computing platform company, and at the same time, it is committed to IP licensing business. We are also focused on bringing flexibility of choice to our partners through complete and comprehensive solutions, helping them bring products to market faster and allowing them to focus more time and energy on developing innovative solutions to deliver more new products and solutions that bring differentiation."
As we all know, China is one of the very important markets for Arm. Innovation from the Chinese market is also one of the reasons for Arm's continuous development and business success. "We believe that Arm provides the foundational platform that enables eco-partners to continue to innovate and build superior products and solutions." "In China, it is very important to accelerate innovation through continuous efforts. It is this spirit of continuous innovation that has led Arm and our eco-partners to achieve remarkable results and bring new products to market."
Today, the total shipment of ARM-based chips by Chinese partners has reached 30 billion. The Chinese market alone will ship nearly four chips for every person on the planet. Arm currently has nearly 400 technology licensing customers in China. In addition, Arm has been working with ARM Technology to further assist local partners.
According to Mohamed Awad, the Arm Total Computing Solution, Arm Neoverse platform, Arm Corstone and SOAFEE, which have been launched by Arm in recent years, are examples of how Arm's computing platforms are widely used in the market. "These platforms enable Arm to deliver a complete, integrated solution to its ecosystem partners, enabling customers to quickly adopt and launch their solutions." "He introduced.
Meanwhile, Arm remains committed to its IP licensing business. "We want to give our partners more freedom to develop their products in the way that works best for them - we can provide our partners with a complete solution to help them get products to market quickly, and they can also choose individual Arm IP, all of which are free to choose according to their needs." That's what makes Arm unique." Mohamed Awad noted.
Infrastructure is one of the unique highlights of Arm and can be a very good explanation of Arm's development, especially some of the changes that are currently taking place in artificial intelligence (AI). "At Arm, we believe that innovation is the only way to continuously advance technology." Mohamed Awad added.
Traditional system architecture in infrastructure
As shown in the figure above, in a traditional server system architecture, the left side is memory, which is then connected to a generic off-the-shelf CPU, which in turn manages multiple accelerators. "This traditional architecture is the only architecture available in the market in the past, but the problem with this architecture is that the common off-the-shelf CPU and the interface between the accelerators directly limits the final performance level of the product, because all accelerators have to access additional memory through a CPU, so there is no memory consistency. This means that the performance of accelerators cannot be fully utilized and cannot support the needs of the era of generative AI." Mohamed Awad noted.
Facing the new application requirements, modern system architecture has emerged in the field of infrastructure
Now there are modern system architectures, take one of them as an example (see figure above). Each CPU can be connected to a separate accelerator, which will have strong memory consistency. "With this architecture, the designer can better design the CPU according to the actual scenario and use case of the user, so as to maximize the efficiency of the accelerator." Only by specifically understanding what the final use case is and designing the CPU according to your use scenario can you achieve better efficiency and achieve the best performance of the product. In this case, only through such a new architectural design can we better support the market needs of future infrastructure, including generative AI." Mohamed Awad stressed.
The NVIDIA GH200 Grace Hopper superchip uses Arm Neoverse with NVIDIA Gpus to meet AI performance needs
In fact, this architecture is the one used by NVIDIA's GH200 Grace Hopper superchip. "NVIDIA is working with Arm because they want the flexibility Arm technology provides to design the chips they need to further optimize their systems while taking full advantage of Arm's powerful software ecosystem." In this architecture, the combination of 72 Arm Neoverse cores, plus Gpus from NVIDIA, improves AI performance by 10 times compared to systems based on x86 architecture." Mohamed Awad said.
Why do hyperscale cloud service providers such as Amazon Cloud Services (AWS) and Alibaba have to do their own CPU design, and they all choose to cooperate with Arm? The answer is simple: by working with Arm, they are able to build and optimize solutions based on their own use cases and infrastructure, explains Mohamed Awad.
From hyperscale cloud providers to startups, empower innovation on the Arm platform
"We are very proud of our partnership with NVIDIA. At the same time, we are also very happy to cooperate with local manufacturers in China. It can be seen that Alibaba's Yitian 710 is a CPU built on the Arm Neoverse platform. In addition, like Hongjun Microelectronics, Yuxian Microelectronics and Yunbao Intelligence, these three are local start-ups from China, they are based on the Arm platform to build server SoC solutions and cloud computing and data center DPU solutions. Lenovo, on the other hand, is more about deploying ARM-based solutions to 5G. So you can see that by working with a strong local partner in China, we will further promote Neoverse's growth in China."
Arm Neoverse Computing Subsystem (CSS)
Now, Arm is taking further steps to help customers accelerate the deployment of systems based on the Neoverse platform. The Arm Neoverse Computing Subsystem (CSS) provides a complete subsystem that Arm partners can adopt directly or on demand, accelerating the time-to-market process.
Compared to standalone IP, CSS will enable the Arm ecosystem to create specialized chips at lower cost, less risk, and in less time.
As the first generation of the Arm Neoverse CSS, the Arm Neoverse CSS N2 integrates the Neoverse N2 platform and optimizes power, performance and area (PPA) through a verified configuration. The Neoverse CSS N2 leverages the per-watt performance and efficiency of the Neoverse N2 platform, is optimized based on an advanced 5nm process, and is delivered to partners in the form of a customizable computing subsystem to accelerate time to market. This will open up new and higher degrees of freedom for innovation that can be leveraged as they are or customized, providing partners with opportunities to differentiate in storage, IO, acceleration, physical topology, and other areas. These are custom chip solutions for workload optimization.
With Neoverse CSS, the partners' engineering resources will be able to focus on SoC and system-level innovation. In the case of one partner, Neoverse CSS has helped their project go from concept to implementation (launching Linux on more than 100 core systems) in just 13 months. Another partner saved more than 80 people/year of engineer time with Neoverse CSS. It also enables the integration of dome-specific dedicated accelerators, including AI, and delivers market-leading performance per watt for cloud-native workloads.
What's unique about the Arm ecosystem is the ability to drive a change in custom chips. To take full advantage of the benefits of dedicated processing, an architecture must provide the flexibility to innovate and deliver diverse and differentiated solutions, while maintaining access to a robust software ecosystem. Flexibility and specialization have always been at the heart of Arm, and are part of Arm Neoverse's design philosophy, which matches decades of software investment and careful development of Arm architectures.
Arm's partners have used the unique balance between customization and standardization to achieve several industry innovations, including the first CPU with a total memory bandwidth of more than 1 terabyte per second, the first CPU with more than 100 cores, and the first CPU to introduce DDR5 and PCIe Gen5 technologies. These innovations are accelerating momentum across key industry markets, from wireless networking to cloud computing.
The introduction of Neoverse CSS is a technical milestone for Arm and another innovation in the way Arm offers the Neoverse platform. By reducing design costs and accelerating time to market, it ensures that more partners have access to custom chips. Neoverse CSS opens a new era of custom chips in infrastructure, while also expanding Arm Neoverse's portfolio to provide trusted CSS to the Arm ecosystem and help ecosystem partners focus on differentiation and innovation. The results of using Arm Neoverse's designs over the past few years are redefining infrastructure. With the introduction of CSS, more partners will be able to leverage Arm's technology to create customized infrastructure.
"All in all, Neoverse CSS enables our partners to further reduce investment, accelerate the accessibility of our solutions across the ecosystem, and accelerate time to market for our partners' products." Mohamed Awad said.
In addition, the Cobalt 100 CPU recently released by Microsoft is designed based on Neoverse CSS. There are also Chinese partners who are developing based on CSS.
Microsoft Azure Cobalt 1
Building on Neoverse CSS, Arm also introduced Arm Total Design, which further combines the power of the ecosystem to simplify the development process of custom chips and make delivery easier. The Arm Full Design now further enhances the availability of the Neoverse CSS ecosystem to partners, helping them further accelerate their solution development. This means that whether it is an application-specific integrated circuit (ASIC) design company, IP vendors, EDA tool providers, foundries and firmware developers, Arm can further support and help their product development.
"We continue to help our partners build the computing solutions of the future from the traditional edge to the ARM-based data center. We help our partners further accelerate time to market, simplify the development process, and invest more energy in innovation." Mohamed Awad concluded.
About US
Heisener Electronic is a famous international One Stop Purchasing Service Provider of Electronic Components. Based on the concept of Customer-orientation and Innovation, a good process control system, professional management team, advanced inventory management technology, we can provide one-stop electronic component supporting services that Heisener is the preferred partner for all the enterprises and research institutions.
