Meta has unveiled the latest iteration of its Llama model, Llama 3 which it says trumps other open source contenders on performance and looks set to challenge the biggest names in AI.
Llama 3 comes in two parameter sizes, 8B and 70B, both of which beat out models of a similar size such as Google’s Gemma 7B, and Gemini Pro 1.0, as well as Mistral 7B.
Trained on two recently announced custom-built 24K GPU clusters that feature a training dataset seven times larger than that used for Llama 2, the firm boasts these models will take “data and scale to new heights”.
On MMLU, a common benchmark for AI models, Llama 3 8B leads the way with 68.4% over Gemma’s 53.3% and Mistral’s 58.4%, while Llama 3 70B has an MMLU score of 82% that just overtakes Gemini’s score of 81.9% and Sonnet’s score of 79%.
Llama 3 supports an 8K context length that doubles the capacity of Llama 2 and the new model reportedly “excels” at language nuance, contextual understanding, and complex translation or dialogue generation tasks.
It can also handle multi-step tasks “effortlessly” according to the firm, while Meta’s “refined post-training processes” have lowered false refusal rates and boosted diversity in the model’s answers.
"The release of Llama 3 by Meta with its advanced capabilities is poised to set a new benchmark in the open-source generative AI market,” Moses Guttmann, CEO and co-founder of ClearML, told ITPro.
“By pushing the limits of what open-source models can achieve, Llama 3 challenges other contributors to elevate their own offerings, accelerating innovation across the board,” he added.
Meta also teased a more powerful model that it says is still in training called Llama 3 400B+, which looks to be competitive with the biggest proprietary models at benchmarks of 86.1% MMLU and 84.1% HumanEval, a benchmark that tests code generation capabilities.
By way of comparison, Claude 3 Opus operates at 86.8% MMLU, just 0.7% ahead, while OpenAI’s GPT-4 operates at 86.4% MMLU.
Of course, Meta has yet to release the 400B+ version of Llama 3, though these figures do point to an overarching acceleration in open source as a competitor to closed source.
“These latest advancements reaffirm our belief in the potential of open source AI to rival closed-source alternatives,” Guttmann said.
Proprietary models could fall by the wayside for enterprise use
Though OpenAI’s GPT-4 is still the frontrunner in terms of performance, Llama 3 may signal a sea change in the enterprise’s approach to AI, according to CEO and co-founder of SambaNova Systems Rodrigo Liang.
RELATED WHITEPAPER
“Enterprises don’t need to build their own models, because the open-source community is more innovative than any single organization,” Liang told ITPro.
“A community working together will always be more powerful than an individual company, and leveraging this community is the best way for any organization to keep up with a rapidly evolving AI landscape,” he added.
Liang suggests that the ethos of innovation in the open source community will seem a more attractive landscape for the enterprise AI user, and Meta’s latest announcement seems to further solidify this position.
“With each passing week, the trajectory becomes clearer, affirming that open source is the optimal path forward,” Guttmann said.
-
Bigger salaries, more burnout: Is the CISO role in crisis?In-depth CISOs are more stressed than ever before – but why is this and what can be done?
-
Cheap cyber crime kits can be bought on the dark web for less than $25News Research from NordVPN shows phishing kits are now widely available on the dark web and via messaging apps like Telegram, and are often selling for less than $25.
-
Meta executive denies hyping up Llama 4 benchmark scores – but what can users expect from the new models?News A senior figure at Meta has denied claims that the tech giant boosted performance metrics for its new Llama 4 AI model range following rumors online.
-
The DeepSeek bombshell has been a wakeup call for US tech giantsOpinion Ross Kelly argues that the recent DeepSeek AI model launches will prompt a rethink on AI development among US tech giants.
-
Meta won't release multimodal AI models in Europe due to "unpredictable" privacy regulationsNews The "unpredictable nature" of EU regulations have prompted Meta to scrap plans to release its models across the region
-
A new LLM jailbreaking technique could let users exploit AI models to detail how to make weapons and explosives — and Claude, Llama, and GPT are all at riskNews LLM jailbreaking techniques have become a major worry for researchers amid concerns that models could be used by threat actors to access harmful information
-
Three open source large language models you can use todayNews Enterprises are flocking to open source large language models, many of which have become highly popular - here’s three you might want to try out
-
An open source challenger to GitHub Copilot? StarCoder2, a code generation tool backed by Nvidia, Hugging Face, and ServiceNow, is free to use and offers support for over 600 programming languagesNews StarCoder2 offers code generation support for over 600 programming languages, and it’s free to use
-
Google’s new ‘Gemma’ AI models show that bigger isn’t always betterNews Smaller AI models are clearly the hot new commodity as Google unveils two new lightweight models, Gemma 2B and Gemma 7B
-
Hugging Face just launched an open source alternative to OpenAI’s custom GPT builder, and it’s freeNews Hugging Face looks primed to position its new assistant creation platform as a direct competitor with OpenAI’s GPT Store, but how does it compare?
