Is China silently working to be self-sufficient in AI memory chips?

• China has determined to catch up in the manufacturing of high-grade AI memory chips.
• The production of HBM/AI memory chips does not necessarily require cutting-edge lithography technology.
• That makes it possible for China to produce its own versions even without the latest equipment.

A strong growth in artificial intelligence (AI) server shipments since generative AI emerged with ChatGPT has driven demand for high bandwidth memory (HBM), the memory chips tailored for AI processors. So much so that HBM is now a major battleground between Samsung Electronics and SK Hynix, two out of the three only players in the field.

But China wants to change that — in fact, it wants to be able to rely on its domestic production for AI memory chips. With the AI era in full swing, the global demand for HBM products, including China, could increase dramatically. Added to which, Beijing’s restrictions on Micron, which had a 15% share of China’s server DRAM market, primarily benefited its South Korean competitors.

For context, South Korea’s SK Hynix and Samsung Electronics control 90% of the global market in HBM chips, crucial components needed to train AI systems such as OpenAI’s ChatGPT. As of last year, SK Hynix had a 50% share of the global HBM market, followed by Samsung at 40% and US rival Micron Technology trailing at 10%.

Both Korean companies have unveiled plans to double their HBM chip production next year even as they scale back elsewhere in the memory sector amid a supply glut that has slashed operating profits. SK Hynix became the first to mass produce cutting-edge fourth-generation HBM3 chips, which are used in Nvidia’s H100 graphic processing units (GPUs).

A brief explanation of how HBM is adopted

HBM vertically stacks memory chips, like floors in a skyscraper, effectively shortening the distance information travels. These memory towers die to connect to a CPU or GPU through an ultra-fast interconnect called the “interposer.” Nvidia set a new industry standard by using HBM chips to speed up data transfers between GPUs and memory stacks, according to TrendForce. 

According to the company, Nvidia’s much-sought-after H100 GPU features an HBM3 memory system, delivering three terabytes per second of memory bandwidth. SK Hynix shared in its promotional materials that HBM technology was a “prerequisite for Levels four and five of driving automation in autonomous vehicles.” 

Last week, the company announced that it had successfully developed HBM3E, the next-generation high-end DRAMs for AI applications, and provided samples to a customer for product performance evaluations. Mass production is scheduled for the first half of 2024, with clients such as US chip firms AMD and Nvidia reportedly lining up for the new product. 

TrendForce forecasted that demand for HBM chips will grow by almost 60% this year. China though will have only its local productions to rely on, which puts it in a race against time.

Is China officially in the AI memory chips race?

As it is, Beijing’s ambitions of joining the world’s top memory chipmakers and replacing imports with local products in the domestic market have increasingly been at risk as the world’s leading players adopt cutting-edge technologies. At the same time, China’s ability to catch up is hampered by US sanctions.

Not long ago, China was seen as catching up quickly with international suppliers in advanced 3D NAND flash and dynamic random access memory (DRAM) memory chips. But now the gap is widening daily in the era of ChatGPT, as China’s Yangtze Memory Technologies Corporation (YMTC) and Changxin Memory Technologies (CXMT) have been unable to press ahead with catch-up efforts, due to the US export restrictions.

China is reportedly exploring ways to produce its own HBM as part of its self-sufficiency push. “While it will be an uphill battle to catch up with global leaders like SK Hynix, Samsung Electronics, and Micron Technology given the impact of Washington’s sanctions, the Chinese government has determined that the country must become self-sufficient in HBMs, even though it may take years,” the South China Morning Post (SCMP) reported.

Quoting Industry sources, the SCMP said China’s top DRAM maker, ChangXin Memory Technologies, is the country’s best hope for HBMs, but it may take up to four years to bring products to the market. But, If CXMT or other Chinese chipmakers decide to go ahead, they will inevitably be hindered by using less advanced technologies to manufacture the powerful DRAM chips, the sources added.

The upside is that industry insiders believe that despite their high performance, producing HMB chips does not necessarily require cutting-edge lithography technology such as extreme ultraviolet (EUV) tools. In short, China can make its versions, even without the latest equipment. 

Due to the need to integrate several chips vertically, high-density packaging technologies such as through-silicon-via (TSV) are required. Still, as the SCMP noted, China has relatively advanced players in this field, such as Jiangsu Changjiang Electronics Technology. Ultimately, China is known for keeping its cards close its chest regarding its latest technologies, leaving the outside world to guess how far it is progressing in Beijing’s self-sufficiency drive. 

So – can China catch up in HBM manufacturing? Only time will tell – and it’s likely the rest of the world will only find out on China’s timetable.