Apple has introduced a new open-source language model as part of its ongoing efforts to contribute to the broader AI ecosystem. This latest model, part of the DCLM (dataComp for Language Models) project, is a 7 billion parameter model that has been made publicly available, including all its weights, training code, and datasets.
The new DCLM model has demonstrated impressive performance, outperforming Meta’s Mistral-7B in benchmarks and approaching the capabilities of similar models from Meta and Google. Despite its relatively small size and a context window of only 2,000 tokens, the model achieves a 63.7% accuracy on standard 5-shot evaluation benchmarks.
Apple’s decision to release this model as fully open-source is a significant move. Apple has made the entire framework, including training logs, multiple checkpoints, and pre-training configurations, publicly available. This approach is designed to empower the open research community, allowing researchers and developers to explore, modify, and improve the models without the constraints of proprietary datasets or code.
