Meta has unveiled its most capable ‘open-source’ model yet, which it promises will outperform the leading private models from OpenAI, Google, and Anthropic.
Llama 3.1 is significantly more complex than the Llama 3 models released earlier this year, with its largest variant boasting 405 billion parameters and was trained with over 16,000 Nvidia H100s.
This marks the first ‘frontier-level’ open source model, according to Meta, with improved 70 billion and 8 billion parameter models distilled from the larger variant also now available, which Meta claims are the leading open-source models for their sizes.
In a statement accompanying the release of Llama 3.1, Mark Zuckerberg, founder and CEO at Meta, said the open source model ecosystem is quickly ‘closing the gap’ on their private counterparts, claiming he expects future Llama models to surpass the closed-source competition in 2025.
“Last year, Llama 2 was only comparable to an older generation of models behind the frontier. This year, Llama 3 is competitive with the most advanced models and leading in some areas. Starting next year, we expect future Llama models to become the most advanced in the industry. But even before that, Llama is already leading on openness, modifiability, and cost efficiency.”
In the post, Zuckerberg reiterated his claim that open source is the future of AI, projecting it will follow a similar path to open source operating system, as it competed with closed source versions of Unix in its early days.
The flexibility and affordability of Linux were soon complemented by improvements in its security and performance, driving the platform’s rise to becoming the industry-standard operating system for cloud computing and mobile devices, and Zuckerberg stated he thinks AI will follow a similar trajectory.
Meta was also bullish on the success of its in-house AI assistant, Meta AI, which it has made available in more countries and languages, as well as adding some new features including its Imagine image generation capability.
In an Instagram post on 23 July, Zuckerberg said Meta AI is on track to reach the firm’s target of becoming the most used AI assistant in the world by the end of 2024.
A significant step forward, but ‘open source’ concerns remain
Victor Botev, CTO and cofounder at Iris.ai, said Llama 3.1 is a big leap in terms of expanding the level of access people have to powerful AI systems, eliminating some of the barriers to entry for those without deeper pockets.
“The open-source nature of Llama 3.1 405b represents a significant step forward in democratizing access to AI technology. Meta is enabling researchers and developers worldwide to explore, innovate, and build upon state-of-the-art language AI without the barriers of proprietary APIs or expensive licensing fees.” he explained.
“This approach emphasizes transparent development, fosters collaboration and accelerates progress in the field, potentially leading to breakthroughs that benefit society as a whole.”
Some have criticized Meta’s use of the label ‘open source’ for their Llama models, however, preferring to use ‘free to use’ as Meta places restrictions on particularly large enterprises.
Organizations with hundreds of millions of users will need to seek approval from Meta before they can take advantage of the Llama 3.1, and Meta still refuses to share the model’s training data or the code used to train it.
As such, the open source label may have more to do with attracting users to its platform and securing a supply of data to continue training its model on, which Meta can eventually monetize down the line.
Speaking to ITPro, Theo Priestley, technology futurist, keynote speaker, and author, said it is important to remember Meta is not being totally transparent when it refers to its Llama model as open source.
“The idea of [Llama 3.1] being truly open source is a red herring, eventually everything flows back to Meta HQ.”
Priestley added that there are serious concerns around the governance of these open-weight models, and Meta may be using the open source label to abrogate some of the responsibility for managing how they are used.
“Open source AI is a bigger problem because it becomes unregulated and unmanageable, unsustainable and unethical. It's the absolution of corporate responsibility towards AI by saying 'well it's your problem now' to anyone who uses it and builds upon it.”
In a post on X, Jeffrey Ladish, executive director at Palisade Research warned that his research showed the 70 billion parameter version of Llama 3 could be stripped of its safety guard rails in minutes for a negligible cost, adding that he expects the same to be true for Llama 3.1.
Ladish reminded users that the decision to release the weights of the Llama models cannot be undone.
