StableLM mascot. Image: © Stable Diffusion XL
StableLM is now available on GitHub for developers to use and adapt for generating text and code as Stability seeks to make AI accessible.
Stability AI, the start-up behind the AI-powered text-to-image generator Stable Diffusion, has launched the first of a suite of large language models to take on the likes of OpenAI’s GPT-4.
Collectively known as StableLM, the language models are open source, meaning that developers can freely inspect, use and adapt them for commercial or research purposes, subject to the terms of the licence.
London-based Stability AI said the Alpha version of the model is available in 3bn and 7bn parameters, with 15bn to 65bn parameter models to follow. By comparison, OpenAI’s GPT-3 has 175bn parameters.
“With the launch of the StableLM suite of models, Stability AI is continuing to make foundational AI technology accessible to all,” the start-up wrote in an announcement on its website.
“Our StableLM models can generate text and code and will power a range of downstream applications. They demonstrate how small and efficient models can deliver high performance with appropriate training.”
StableLM builds on Stability AI’s earlier language model work with non-profit research hub EleutherAI. These language models were trained on an open-source dataset called The Pile, which encompasses information from a range of sources including Wikipedia, Stack Exchange and PubMed.
Stability AI’s latest language model is trained on a significantly larger version of The Pile, with 1.5trn tokens of content.
“The richness of this dataset gives StableLM surprisingly high performance in conversational and coding tasks, despite its small size of 3 to 7bn parameters,” the company said.
Available now on GitHub and Hugging Face, a platform for hosting AI models and code, StableLM is the result of Stability AI’s plan to create AI technology that is “transparent, accessible, and supportive”.
“Language models will form the backbone of our digital economy, and we want everyone to have a voice in their design.”
Earlier this year, Stability AI was sued by image hosting giant Getty Images for “brazen infringement” of its intellectual property “on a staggering scale”.
The start-up allegedly copied more than 12m photos – along with captions and metadata – from Getty Images’ collection without obtaining permission or compensating them.
10 things you need to know direct to your inbox every weekday. Sign up for the Daily Brief, Silicon Republic’s digest of essential sci-tech news.
Image: © Stable Diffusion XL
