Microsoft has released a press release announcing Phi-3 Mini, the company’s first small language models (SLMs), which developers describe as ‘The most cost-effective and efficient of small language models’, they are lightweight. More cost-effective than large language models due to smaller parameter sizes. This parameter refers to the complexity of the set of instructions that the AI can understand.
Ronen Eldan, a machine learning researcher at Microsoft Research, revealed the inspiration for the Phi-3 when he was reading a bedtime story to his daughter. He had a question in his head. “How did you learn these words? And how do you understand the connections between words?”
This has led Microsoft Research to think about developing an AI that would be easy for a 4-year-old to understand using a set of words. This has led to the creation of small language models that will help more people access AI.
Although large language models (LLMs) are the standard for finding answers to complex questions, But small language models have many of the same functions that large language models do, such as understanding language, writing code, or doing low-complexity math problems. And it’s suitable for organizations with limited data resources. Microsoft claims that the Phi-3 Mini has 3.8 billion parameters. It’s as powerful as a model twice its size.
“The trend we are going to see going forward is not a move from big to small models. It is a move away from reliance on one type of model to a combination of models based on user needs to achieve the best results,” said Solina Yadav, product manager for Generative AI at Microsoft.
However, Phi-3 is not the industry’s first small language model. Because previously competitors like Google launched Gemma with 2 million parameters and 7 million parameters earlier this year. which specializes in processing language tasks, or Claude 3 Haiku, which can quickly summarize research documents, both content and chart data. Llama-3, a size of 8 million meta parameters, released last week. were classified in the same category It may be used to help write code.
For use cases They are smaller and more cost-effective due to the low processing power inherent to small language models. This type of model is suitable for use on personal devices such as smartphones or laptops. Eric Boyd, vice president of Microsoft Azure AI Platform, told The Verge that Phi-3 will work better with applications that It requires customization to specific use cases. This is something that many companies are looking for due to the small amount of internal data.
Phi-3 is currently available to developers on Azure, Hugging Face and Ollama, with next versions Phi-3 Small (7B Parameters) and Phi-3 Medium (14B Parameters) coming in the near future.
Image: SOPA Images/Getty Images
refer:
Tags: Phi3 Microsofts smallest model
