As we know, most AI chatbots currently in circulation require cloud computing. In addition to using very large amounts of data, these chatbots also require high GPU processing capabilities. Therefore, we often hear about large companies like Tesla, Twitter, and Silicon Valley giants buying GPUs from NVIDIA.
Aside from ChatGPT and its colleagues, there are also some AI chatbots and machine learning models that can run entirely locally. However, they require powerful GPUs and very high VRAM. ChatGPT can actually run on Raspberry Pi, which theoretically is not entirely local because it still sends the prompts it generates to the server to be processed.
AI Chatbots Can Run Entirely Locally Without Cloud, Including on iPhones
But what if there is a lightweight Large Language Model that can be run entirely locally on the device’s internal storage? It turns out there is, and the project is an open source software that you can try for free.
Named MLC LLM (Machine Learning Compilation), a Large Language Model software based on Meta’s LLaMA, Vicuna-7B-V1.1. LLM has been trained from March to April this year. Although it can run locally and without the cloud, the features and capabilities offered are not as complete and advanced as GPT 3.5.
AI MLC LLM Available for Almost All Platforms
This project is available for Linux, Windows, macOS, and iOS, while the Android version is not yet available. The developers also provide simple instructions on their website along with the source code on GitHub.
For now, LLM runs smoothly on iOS iPhones using 6GB of RAM, while those with less RAM still cannot run optimally. As for Windows PC usage, the software can run smoothly even on 6th generation Intel devices.
Because it is still new, MLC LLM cannot provide accurate answers yet. The information provided is still very vague. In the future, as it develops, this AI will become more advanced and able to answer questions accurately.
