First in {industry} to launch 8-high 24GB HBM3 Gen2 with bandwidth over 1.2TB/s and superior energy effectivity enabled by superior 1β course of node
Micron Know-how, Inc. (Nasdaq: MU), at present introduced it has begun sampling the {industry}’s first 8-high 24GB HBM3 Gen2 reminiscence with bandwidth better than 1.2TB/s and pin pace over 9.2Gb/s, which is as much as a 50% enchancment over at the moment transport HBM3 options. With a 2.5 occasions efficiency per watt enchancment over earlier generations, Micron’s HBM3 Gen2 providing units new information for the vital synthetic intelligence (AI) knowledge heart metrics of efficiency, capability and energy effectivity. These Micron enhancements scale back coaching occasions of enormous language fashions like GPT-4 and past, ship environment friendly infrastructure use for AI inference and supply superior complete value of possession (TCO).
The inspiration of Micron’s high-bandwidth reminiscence (HBM) answer is Micron’s industry-leading 1β (1-beta) DRAM course of node, which permits a 24Gb DRAM die to be assembled into an 8- excessive dice inside an industry-standard bundle dimension. Furthermore, Micron’s 12-high stack with 36GB capability will start sampling within the first quarter of calendar 2024. Micron gives 50% extra capability for a given stack top in comparison with present aggressive options. Micron’s HBM3 Gen2 performance-to-power ratio and pin pace enhancements are vital for managing the acute energy calls for of at present’s AI knowledge facilities. The improved energy effectivity is feasible due to Micron developments similar to doubling of the through-silicon vias (TSVs) over aggressive HBM3 choices, thermal impedance discount by a five-time enhance in metallic density, and an energy-efficient knowledge path design.
Micron, a confirmed chief in reminiscence for two.5D/3D-stacking and superior packaging applied sciences, is proud to be a associate in TSMC’s 3DFabric Alliance and to assist form the way forward for semiconductor and system improvements. As a part of the HBM3 Gen2 product improvement effort, the collaboration between Micron and TSMC lays the muse for a clean introduction and integration in compute programs for AI and HPC design functions. TSMC has obtained samples of Micron’s HBM3 Gen2 reminiscence and is working carefully with Micron for additional analysis and assessments that can profit clients’ innovation for the next-generation HPC utility.
The Micron HBM3 Gen2 answer addresses growing calls for on the earth of generative AI for multimodal, multitrillion-parameter AI fashions. With 24GB of capability per dice and greater than 9.2Gb/s of pin pace, the coaching time for big language fashions is lowered by greater than 30% and ends in decrease TCO. Moreover, the Micron providing unlocks a major enhance in queries per day, enabling skilled fashions for use extra effectively. Micron HBM3 Gen2 reminiscence’s best-in-class efficiency per watt drives tangible value financial savings for contemporary AI knowledge facilities. For an set up of 10 million GPUs, each 5 watts of energy financial savings per HBM dice is estimated to avoid wasting operational bills of as much as $550 million over 5 years.
“Micron’s HBM3 Gen2 know-how was developed with a deal with unleashing superior AI and high-performance computing options for our clients and the {industry},” stated Praveen Vaidyanathan, vice chairman and basic supervisor of Micron’s Compute Merchandise Group. “One vital criterion for us has been the convenience of integrating our HBM3 Gen2 product into our clients’ platforms. A totally programmable Reminiscence Constructed-In Self Take a look at (MBIST) that may run on the full specification pin pace positions us for improved testing functionality with our clients, creates extra environment friendly collaboration and delivers a sooner time to market.”
“On the core of generative AI is accelerated computing, which advantages from HBM excessive bandwidth with power effectivity,” stated Ian Buck, vice chairman of Hyperscale and HPC Computing at NVIDIA. “We now have a protracted historical past of collaborating with Micron throughout a variety of merchandise and are desirous to be working with them on HBM3 Gen2 to supercharge AI innovation.”
Micron developed this breakthrough product by leveraging its international engineering group, with design and course of improvement in the USA, reminiscence fabrication in Japan and superior packaging in Taiwan. Right this moment’s announcement is one other milestone in Micron’s know-how management within the {industry}. Micron beforehand introduced its 1α (1-alpha) 24Gb monolithic DRAM die-based 96GB DDR5 modules for capacity-hungry server options and at present launched the1β 24Gb die-based 24GB HBM3 providing. Within the first half of calendar 2024, the corporate plans to make obtainable its 1β 32Gb monolithic DRAM die-based 128GB DDR5 modules. These choices reveal Micron’s modern know-how improvements for AI servers.
