According to a report by The Information citing a Meta employee, the tech giant is set to unveil two smaller versions of its Llama 3 large language model (LLM) next week, ahead of the official release of Llama 3 later this summer.
The launch of these two smaller models is expected to pave the way for the grand debut of Llama 3. Meta introduced Llama 2 back in July last year, and since then, several companies including Google, Elon Musk's xAI, and Mistral have released their own open-source large language models, intensifying the competition.
Llama 3 is positioned directly against OpenAI's GPT-4, which has emerged as a powerful multimodal model capable of handling longer texts and supporting image inputs.
The report suggests that the official version of Llama 3 will also support multimodal processing, enabling it to understand and generate both text and images simultaneously, a capability that the initial two smaller versions will lack.
Generally, smaller models come with lower costs and faster processing speeds, particularly significant in today's landscape where running large models can be prohibitively expensive. Smaller models are also conducive for developers aiming to deploy AI software on mobile devices.
Meta has previously rolled out three versions of Llama 2, with the largest one boasting 700 billion parameters, while the other two versions have 130 billion and 70 billion parameters respectively.
According to a previous article by Hard AI, the largest version of Llama 3 could potentially exceed 140 billion parameters.
In Llama 3, Meta aims to address the issue of Llama 2 being too conservative in responding to controversial topics. Researchers plan to relax the restrictions on large models in this regard, enabling them to engage more with users by providing background information rather than simply refusing to answer.
