Despite vendor enthusiasm, analysts have been clear they do not think 2024 is going to be the year that sees PCs with dedicated hardware for AI workloads enter the mainstream.
But as software catches up to the new hardware, it may be time for the industry to take these devices seriously and take a deeper look at how they can add value through AI-enhanced computers.
At least, this was the main goal of HP’s Imagine AI event, where it launched its flagship Copilot+ device – the OmniBook Ultra 14. In reality, the spotlight was really on the independent software vendors who will actually drive the migration toward the next generation of PCs.
HP brought out a string of independent software vendors (ISVs) who are developing solutions that will take advantage of the new processing power on AI PCs to run workloads locally.
The focus was firmly on demonstrating how the AI software ecosystem is well on its way to becoming mature enough to merit businesses considering AI-ready devices in their upcoming hardware refreshes to take advantage of these new capabilities.
In an ISV panel discussing how CIOs can deliver on the promise of AI, Fred Balsinger, group program manager of Windows Silicon at Microsoft, said models were previously too large and cumbersome to consider running them at the edge, but the rate of innovation in the industry has changed this reality.
“But recently, [with] the speed at which the AI industry has been evolving, we’re learning how to get those models just so much smaller. We had techniques around formatting the machine learning models, around quantizing them, around the weights, how we can actually get them small enough,” he explained.
“Now we’re at a point in the industry where we can have truly meaningful models and we can run them on the edge.”
Balsinger broke down why organizations would want to be able to run AI workloads locally, and where he sees where PCs equipped with AI accelerators can truly add value for enterprise.
Local models bring new security for AI tools
One area in which businesses will be able to derive new value from AI-enhanced PCs, according to Balsinger, is security.
He noted that by running models on the device’s neural processing unit (NPU), enterprise users will no longer have to worry about their data leaving their machine, a common concern around using AI models in industries with particular concerns around data protection, such as healthcare or finance.
Speaking to ITPro, Rachel Cossar, CEO at AI communication coaching service Virtual Sapiens, said this was always a core concern for her organization, which gives real time feedback on how a user is presenting themselves in meetings.
“There are multiple levels of trust that you’re trying to establish. One of the biggest ones for us is that your likeness will never leave your device. We found that people are understandably very sensitive about that, they don’t like the idea of having their image and their video wandering around the ether.”
Running models locally, therefore, presents Virtual Sapiens with a convincing proposition to risk averse users where it can ensure their private video never leaves the device.
Cossar noted local processing was always in the company’s vision due to these security considerations, but hardware limitations had constrained the company’s ability to deploy Virtual Sapiens as widely until this point.
“We actually have always envisioned using local, on-device processing. We made that decision from a security perspective as well as an economic one, but it’s been a struggle,” she explained.
“It’s been a struggle to meet consumers where they are with their devices because we’ve been eating such high levels of GPU/CPU. This gives us a much more efficient option to run seamlessly and smoothly for our users while they’re doing other things.”
NPUs will free up other processors
Balsinger highlighted this new availability on board AI accelerators can provide, and how important this will be for ISVs to drive AI innovation further, while avoiding exorbitant cloud hosting costs.
“Introducing a new processor allows them to take some of their workloads that typically compete for resources on a GPU, for example, move it over to the NPU, all of a sudden, the rest of the system is more available to the software. So now you can more efficiently use a machine while relying on the NPU to inference in a more effective way.”
Manuel Terrazza, CEO and founder of Loccus.ai, an audio deepfake detection tool that uses AI to provide an authenticity score to indicate the likelihood of the audio being an impersonation, told ITPro the inclusion of NPUs on new devices was the ‘perfect match’ for the company.
“What we’re doing with HP is to leverage the whole power of the NPU to add a layer of protection in voice communication channels. It’s the perfect combination of very powerful knowledge but running locally very close to the edge. From privacy, real time experience, and an efficient inference process, it's the perfect match.”
Terraza added that since transitioning from a cloud hosted model, he was pleasantly surprised to see the efficacy of the detection tool did not suffer when using smaller models running locally.
“The whole process of adapting our models to run on the NPU surprisingly showed us there is no compromise in accuracy,” he explained.
“In reality, we kept the same address and, because we are running one model on the NPU, we can even get faster inference processes and a more real-time experience. Right now we get a score every second, so we just need one second of volume to provide a score without any compromise in accuracy.”
Living up to the personal computer name
HP has already introduced a number of AI software features that will be included for free with its devices and leverage the new NPUs. These include live captions for translating speech in real-time, Poly DirectorAI smart video conferencing solution, or its AI Companion chatbot.
These features by themselves will not drive every business to overhaul their hardware, HP admitted, but the flexibility they can offer will make these systems truly personal to every user.
Speaking to ITPro, Guayente Sanmartin, SVP and division president of commercial systems & displays solutions at HP, said she doesn't think there will be on ‘killer app’ that will force businesses to make the switch, but the power of theses devices will be their ability to personalize the experience to whatever use case the enterprise needs them for.
“I don’t think there is going to be the killer or unique app that is going to work for every company and every worker. For every persona there are going to be different use cases.”
“Now, these AI PCs are going to be able to personalize the experience for you and me differently, so the way I’m working is different to you, the tools that I work [with], how I work, [whether] I do multitasking or not, the PC will adapt to the way you work,” she asserted.
“That is what I think is the big difference, it’s not their specific use case, it’s the personalization of your work style”.
