Image: © ImageFlow/Stock.adobe.com
The new quantised models are 56pc smaller and use 41pc less memory when compared to the full-size models released last month.
Meta has released compact versions of its lightweight Llama 3.2 1B and 3B models that are small enough to run effectively on mobile devices.
The Facebook owner, in an announcement yesterday (24 October), said that these new “quantised” models are 56pc smaller and use 41pc less memory when compared to the original 3.2 models released last month.
Meta says you can use the 1B or 3B models for on-device applications such as summarising a discussion from your phone or calling on-device tools such as calendar.
The new models “apply the same quality and safety requirements” as the original Llama 3.2 1B and 3B, while processing information two to three times faster, the company claimed.
The new versions were quantised using two techniques – one that prioritises accuracy in low-precision environments and one that prioritises portability while aiming to retain quality.
“These models offer a reduced memory footprint, faster on-device inference, accuracy and portability – all while maintaining quality and safety for developers to deploy on resource-constrained devices,” the announcement read.
Users can download and deploy these new model versions onto mobile CPUs that the company has built in “close collaboration” with other industry leaders in this space, it said.
These lightweight models are part of the text-only series of 1B and 3B models, which are available in the EU.
The multimodal models, 11B and 90B, which can process multiple formats, such as text, images, audio and video, are not available in the EU. In the summer, Meta said that it will not release these models in the EU because of the bloc’s “unpredictable” regulatory environment.
The month before this announcement, the company rolled back on plans to train its large language models using public content shared by adults on Facebook and Instagram, following intensive discussion with the Irish Data Protection Commission.
Privacy advocacy group Noyb expressed serious concerns about the plan, alleging that Meta’s intention to use AI training material sourced from public and licenced data that could include personal information would breach the GDPR.
Don’t miss out on the knowledge you need to succeed. Sign up for the Daily Brief, Silicon Republic’s digest of need-to-know sci-tech news.
