One of the key challenges in developing and deploying deep learning models lies in their size and computational requirements. However, Neural Magic tackles this issue head-on through a concept called compound sparsity.

Compound sparsity combines techniques such as unstructured pruning, quantisation, and distillation to significantly reduce the size of neural networks while maintaining their accuracy. 

“We have developed our own sparsity-aware runtime that leverages CPU architecture to accelerate sparse models. This approach challenges the notion that GPUs are necessary for efficient deep learning,” explains Bogunowicz.

Bogunowicz emphasised the benefits of their approach, highlighting that more compact models lead to faster deployments and can be run on ubiquitous CPU-based machines. The ability to optimise and run specified networks efficiently without relying on specialised hardware is a game-changer for machine learning practitioners, empowering them to overcome the limitations and costs associated with GPU usage.

When asked about the suitability of sparse neural networks for enterprises, Bogunowicz explained that the vast majority of companies can benefit from using sparse models.

By removing up to 90 percent of parameters without impacting accuracy, enterprises can achieve more efficient deployments. While extremely critical domains like autonomous driving or autonomous aeroplanes may require maximum accuracy and minimal sparsity, the advantages of sparse models outweigh the limitations for the majority of businesses.

Looking ahead, Bogunowicz expressed his excitement about the future of large language models (LLMs) and their applications.

“I’m particularly excited about the future of large language models LLMs. Mark Zuckerberg discussed enabling AI agents, acting as personal assistants or salespeople, on platforms like WhatsApp,” says Bogunowicz.

One example that caught his attention was a chatbot used by Khan Academy—an AI tutor that guides students to solve problems by providing hints rather than revealing solutions outright. This application demonstrates the value that LLMs can bring to the education sector, facilitating the learning process while empowering students to develop problem-solving skills.

“Our research has shown that you can optimise LLMs efficiently for CPU deployment. We have published a research paper on SparseGPT that demonstrates the removal of around 100 billion parameters using one-shot pruning without compromising model quality,” explains Bogunowicz.

“This means there may not be a need for GPU clusters in the future of AI inference. Our goal is to soon provide open-source LLMs to the community and empower enterprises to have control over their products and models, rather than relying on big tech companies.”

As for Neural Magic’s future, Bogunowicz revealed two exciting developments they will be sharing at the upcoming AI & Big Data Expo Europe.

Firstly, they will showcase their support for running AI models on edge devices, specifically x86 and ARM architectures. This expands the possibilities for AI applications in various industries.

Secondly, they will unveil their model optimisation platform, Sparsify, which enables the seamless application of state-of-the-art pruning, quantisation, and distillation algorithms through a user-friendly web app and simple API calls. Sparsify aims to accelerate inference without sacrificing accuracy, providing enterprises with an elegant and intuitive solution.

Neural Magic’s commitment to democratising machine learning infrastructure by leveraging CPUs is impressive. Their focus on compound sparsity and their upcoming advancements in edge computing demonstrate their dedication to empowering businesses and researchers alike.

As we eagerly await the developments presented at AI & Big Data Expo Europe, it’s clear that Neural Magic is poised to make a significant impact in the field of deep learning.

You can watch our full interview with Bogunowicz below:

(Photo by Google DeepMind on Unsplash)

Neural Magic is a key sponsor of this year’s AI & Big Data Expo Europe, which is being held in Amsterdam between 26-27 September 2023.

Swing by Neural Magic’s booth at stand #178 to learn more about how the company enables organisations to use compute-heavy models in a cost-efficient and scalable way.

The post Damian Bogunowicz, Neural Magic: On revolutionising deep learning with CPUs appeared first on AI News.

This strategic move aims to make generative AI accessible to organisations of all sizes, allowing them to develop, possess, and safeguard their own generative AI models using their own data. 

The acquisition, valued at ~$1.3 billion – inclusive of retention packages – showcases Databricks’ commitment to democratising AI and reinforcing the company’s Lakehouse platform as a leading environment for building generative AI and LLMs.

Naveen Rao, Co-Founder and CEO at MosaicML, said:

“At MosaicML, we believe in a world where everyone is empowered to build and train their own models, imbued with their own opinions and viewpoints — and joining forces with Databricks will help us make that belief a reality.

We started MosaicML to solve the hard engineering and research problems necessary to make large-scale training more accessible to everyone. With the recent generative AI wave, this mission has taken centre stage.

Together with Databricks, we will tip the scales in the favour of many — and we’ll do it as kindred spirits: researchers turned entrepreneurs sharing a similar mission. We look forward to continuing this journey together with the AI community.”

MosaicML has gained recognition for its cutting-edge MPT large language models, with millions of downloads for MPT-7B and the recent release of MPT-30B.

The platform has demonstrated how organisations can swiftly construct and train their own state-of-the-art models cost-effectively by utilising their own data. Esteemed customers like AI2, Generally Intelligent, Hippocratic AI, Replit, and Scatter Labs have leveraged MosaicML for a diverse range of generative AI applications.

The primary objective of this acquisition is to provide organisations with a simple and rapid method to develop, own, and secure their models. By combining the capabilities of Databricks’ Lakehouse Platform with MosaicML’s technology, customers can maintain control, security, and ownership of their valuable data without incurring exorbitant costs.

MosaicML’s automatic optimisation of model training enables 2x-7x faster training compared to standard approaches, and the near linear scaling of resources allows for the training of multi-billion-parameter models within hours. Consequently, Databricks and MosaicML aim to reduce the cost of training and utilising LLMs from millions to thousands of dollars.

The integration of Databricks’ unified Data and AI platform with MosaicML’s generative AI training capabilities will result in a robust and flexible platform capable of serving the largest organisations and addressing various AI use cases.

Upon the completion of the transaction, the entire MosaicML team – including its renowned research team – is expected to join Databricks.

MosaicML’s machine learning and neural networks experts are at the forefront of AI research, striving to enhance model training efficiency. They have contributed to popular open-source foundational models like MPT-30B, as well as the training algorithms powering MosaicML’s products.

The MosaicML platform will be progressively supported, scaled, and integrated to provide customers with a seamless unified platform where they can build, own, and secure their generative AI models. The partnership between Databricks and MosaicML empowers customers with the freedom to construct their own models, train them using their unique data, and develop differentiating intellectual property for their businesses.

The completion of the proposed acquisition is subject to customary closing conditions, including regulatory clearances.

(Photo by Glen Carrie on Unsplash)

See also: MosaicML’s latest models outperform GPT-3 with just 30B parameters

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The event is co-located with Digital Transformation Week.

The post Databricks acquires LLM pioneer MosaicML for $1.3B appeared first on AI News.