Meta has released Llama 3, its latest open-source large language model, alongside a new AI assistant powered by the technology. Meta is rolling out the AI assistant across its major platforms, including Facebook, Instagram, WhatsApp, and Messenger, in select countries.
Meta has introduced two open-source Llama 3 models: an 8-billion parameter version and a 70-billion parameter version. Both models are freely available to developers and can be accessed through major cloud providers.
The Llama 3 models were trained on over 15 trillion tokens, a significant increase from Llama 2’s training data. The training dataset is seven times larger than its predecessor, with four times more code included.
Key improvements in Llama 3 include:
- A new tokenizer with an expanded vocabulary of 128,256 tokens, up from 32,000 in Llama 2
- Grouped-Query Attention (GQA) for more efficient processing of longer contexts
- Reduced false refusals, allowing the model to respond to more queries appropriately
Meta AI’s new AI assistant built on Llama 3, is being rolled out to users in several countries, including Australia, Canada, Ghana, Jamaica, Malawi, New Zealand, Nigeria, Pakistan, Singapore, South Africa, Uganda, Zambia, and Zimbabwe.
Meta is also training a massive 400-billion parameter model, expected to be released later this year. They have introduced new trust and safety tools, including Llama Guard 2, Code Shield, and CyberSec Eval 2.
With the release of Llama 3 and its integration into Meta’s AI assistant, Meta aims to compete with other leading AI models and assistants in the market. The open-source nature of Llama 3 is expected to drive further innovation in the AI community and potentially lead to the development of new, powerful models based on this technology.
