Kneron takes purpose at GPU scarcity with its neural processing unit (NPU) replace

With issues a few international scarcity of GPUs for AI, edge AI startup Kneron sees a possibility for its neural processing unit (NPU) expertise as a aggressive various.

Kneron in the present day is asserting its newest KL730 NPU, with the corporate claiming that it presents as much as 4 occasions extra vitality effectivity than its prior fashions. The brand new chip can be function constructed to assist speed up GPT, transformer-based AI fashions.

Kneron’s silicon is basically focused at edge purposes, reminiscent of autonomous autos and medical and industrial purposes, though the corporate additionally sees potential for enterprise deployments. Kneron advantages from the backing of Qualcomm and Foxconn and has deployments with Quanta in edge servers.

“An NPU has extra cores in contrast with a GPU,” Kneron founder and CEO Albert Liu advised VentureBeat. “The cores are extra environment friendly and they’re extra centered with nuanced connectivity.

The expertise inside Kneron’s NPUs

Liu argued {that a} GPU will not be a purpose-built machine for AI. 

“GPU {hardware} was particularly designed for gaming, and proper now it’s simply Nvidia making an attempt to brainwash all of us making an attempt to say that solely a GPU can do AI,” stated Liu.

Nvidia’s GPU expertise is, after all, market main and is the premise on which fashionable giant language fashions (LLMs) and generative AI are constructed. Liu doesn’t assume it’s going to all the time be that approach, he stated, and he’s hopeful his firm will carve out an expanded market footprint as organizations more and more search for methods to satisfy AI calls for.

Kneron’s chips use a reconfigurable AI structure to speed up AI, which is a distinct structure than what’s utilized in a GPU. With the KL730, the structure has additionally been particularly optimized for GPT’s transformer-based AI fashions.

Kneron well-established within the NPU market

The KL730 isn’t Kneron’s first chip optimized for transformers — the corporate introduced the KL530 silicon two years in the past, which had that functionality. The unique use case for the transformer mannequin in Kneron’s silicon was to assist autonomous car producers. Liu stated that transformer fashions may be very useful with actual time temporal correlation detection use circumstances. 

What wasn’t clear in 2020, not less than to Liu, was that transformers would grow to be extensively used for enabling LLMs and generative AI. To assist meet the wants of LLMs, Liu stated that his firm has made its AI chip bigger for GPT fashion purposes.

“The reconfigurable AI structure can dynamically change the construction contained in the chip to help nearly any type of new mannequin,” Liu stated.

The cascading energy of the KL730

With the brand new KL730, Kneron has made some dramatic efficiency enhancements to its NPU silicon.

Liu stated that the KL703 has higher efficiency than prior generations and can be clustered. As such, if a single chip isn’t sufficient for a selected use case, a number of KL703s may be clustered collectively in a bigger deployment.

Whereas Kneron’s silicon is basically used for inference use circumstances in the present day, Liu is hopeful that the power to mix a number of KL730s collectively will allow broader use of the expertise for machine studying (ML) coaching as nicely.

“For server purposes, Kneron already has clients like Naver, Chunghwa Telecom and Quanta,” stated Liu. “Foxconn is one in every of our strategic buyers and they’re carefully working with us for AI servers.”