Qualcomm president and CEO Cristiano Amon speaks at a information convention throughout CES 2022 in Las Vegas, Nevada, U.S. January 4, 2022.
Steve Marcus | Reuters
Qualcomm on Tuesday introduced two new chips which are designed to run AI software program — together with the massive language fashions, or LLMs, which have captivated the know-how business — with out having to hook up with the web.
Interest in AI purposes has exploded since the Stable Diffusion picture generator and OpenAI’s ChatGPT had been launched in late 2022. Both so-called “generative AI” purposes require plenty of processing energy, and to this point, they’ve primarily been run in the cloud on highly effective and power-hungry Nvidia graphics processors.
The new silicon consists of Qualcomm’s X Elite chip for PCs and laptops and the Snapdragon Series 8 Gen 3 for high-end Android phones.
The pace at which a smartphone chip processes AI fashions may symbolize a new characteristic battleground between high-end Android phones from corporations resembling Asus, Sony and OnePlus versus Apple’s iPhones, that are additionally getting new AI features on an annual basis.
Qualcomm’s newest Snapdragon chip can carry out AI duties considerably faster than final yr’s processor, dropping the time to generate a picture from 15 seconds on final yr’s chip to lower than a second, a Qualcomm govt mentioned in an interview.
“If somebody would go to purchase a telephone right now, they might say, how briskly is the CPU, how a lot reminiscence do I get on this factor? What does the digital camera seem like?” mentioned Alex Katouzian, Qualcomm senior vp for cellular. “Over the subsequent two or three years, individuals are going to say, what sort of AI functionality am I going to have on this?”
Generative AI in your pocket
The AI increase has boosted Nvidia inventory however has largely bypassed Qualcomm, regardless that its smartphone chips are shipped in large volumes and have included AI parts known as NPUs since 2018.
Qualcomm’s NPUs have been used to enhance images and different options. Now, Qualcomm mentioned, its smartphone chip can deal with the greater AI fashions used in generative AI — as many as 10 billion parameters. That’s nonetheless lower than a few of the largest AI fashions, resembling OpenAI’s GPT3, which has about 175 billion parameters.
Qualcomm executives mentioned these sorts of AI fashions can run on gadgets if the chips are quick sufficient and outfitted with sufficient reminiscence. They mentioned it makes extra sense to run language fashions domestically relatively than in the cloud as a result of it may be sooner and is extra personal. Qualcomm mentioned its chips can run a model of Meta‘s Llama 2 mannequin and that it expects its shoppers — smartphone makers — to develop their very own fashions, too. Qualcomm can also be growing its personal AI fashions, it mentioned.
Qualcomm supplied an instance of a tool operating the free Stable Diffusion AI model, which may generate photos based mostly on a string of phrases. It additionally demonstrated a associated capability to develop or fill in components of images utilizing AI.
Last yr’s Qualcomm chip, the Gen 2, ran the identical mannequin efficiently, however it took 15 seconds to crunch all the numbers to create a picture of, for instance, a cat at the seashore. This yr, its new chip can do it in half a second, which Katouzian mentioned may enormously enhance responsiveness for AI purposes resembling private assistants.
Future purposes, resembling a private voice assistant, may use an AI mannequin on the gadget itself for straightforward queries, operating on the gadget’s chip, and can ship more durable inquiries to a beefier pc in the cloud, Qualcomm mentioned.
Qualcomm mentioned this is the reason it really works so intently with Microsoft to verify its chips are optimized for AI software program.
“The extra of those gadgets get used on the edge for operating AI capabilities, the much less cash they need to spend on the Azure cloud, operating tremendous costly inference capabilities,” Katouzian mentioned. “All of that stuff can get offloaded. A hybrid state of affairs the place an offload of the cloud onto the consumer gadgets at the edge offers them an enormous benefit.”
This yr’s high smartphone chip from Qualcomm, Snapdragon Series 8 Gen 3, will begin showing in “premium” Android gadgets costing greater than $500 early subsequent yr from manufacturers together with Asus, Sony and OnePlus, Qualcomm mentioned. The options on the high-end chip finally trickle right down to different gadgets.
Qualcomm X Elite
Qualcomm’s new PC chip, the X Elite, is predicated on Arm and will compete with Intel‘s x86 chips for use in laptops and desktops.
It makes use of know-how from the firm’s acquisition of Nuvia, which was began by former Apple engineers and is at the coronary heart of a legal dispute with Arm. Laptops based mostly on the chip utilizing what Qualcomm calls Oryon cores are anticipated to hit the market in the center of subsequent yr. Qualcomm mentioned it beats Apple’s M2 Max chip in efficiency whereas utilizing much less energy.
Don’t miss these CNBC PRO tales: