Chapter #8: How to Use GPT4All to Run AI Models Locally on Linux

Using AI tools like ChatGPT online is easy - you just need an API key, and you’re good to go.

But sometimes, that’s not the best choice. If you work at a company where data is sensitive or private, sending it to an online AI could be risky. That’s because when you ask a question online, your data leaves your computer, goes to a remote server, and then the AI sends back the answer.

If you don’t want your information to leave your system, there’s a better option - use a local LLM (Large Language Model).

With local models like GPT4All, everything runs on your own machine. You don’t need the internet or an API key. Your data stays safe and private.

In this guide, we’ll show you how to install and use GPT4All on your system, so you can run AI tools completely offline.

What is GPT4All

GPT4All is a free and open-source tool that lets you run AI on your computer, using your files and documents, which means you can chat with an AI that understands your personal data, without needing the internet or sending anything to the cloud.

This kind of setup is called RAG (Retrieval-Augmented Generation). It’s a fancy way of saying: “the AI searches your files instead of using online services.”

With GPT4All, you can choose from different AI models. Just pick one, download it, and you're ready to go - all on your own machine. Since everything runs locally, your data stays 100% private and secure.

GPT4All works as a desktop app, but to run smoothly, your system should have:

• 💾 At least 8GB of RAM
• 🗂️ Around 10GB of free disk space (AI models are large)
• 🚀 A GPU (optional) for faster performance

It's a great way to bring powerful AI to your personal computer - no cloud, no APIs, just local magic.

Install GPT4All in Linux

To install GPT4All on Linux, you need to download the installer and make the file executable.

sudo chmod +x gpt4all-installer-linux.run

Now execute the installer to start the installation.

./gpt4all-installer-linux.run

It will show you a screen like this; click "Next".

Next, it will ask for the location to save the application files.

Next, it will show you the components to choose from; click "Next".

Next, accept the license and hit "Next".

The final step is to hit "up " to start the installation. It will also show information about the space the application will take up after installation.

It will take some time depending on your network connection, as it will download 600MB from the internet.

How to Use GPT4All

To open the application, navigate to the location where you installed it. Go to the bin directory, find the chat application, and click on it to start the application.

After the installation, using the application itself is simple and easy. But before that, we need to understand how it works.

The application works like this:

The LLMs take your query or prompt, go to the local file, search for the response, process the file, and give you a response.

Each model has a different response time depending on how it is trained and other factors.

The first screen you will see when you open the application for the first time looks like this:

This will ask you if you want to share your chats for development purposes or keep them private. Depending on your needs and use cases, you can choose "Yes" or "No."

After that, the home screen will look like this:

How to Add a Model to GPT4All

As mentioned before, in order to work with GPT4All, you need a model that will process the input and generate responses. The model should be downloaded and installed locally. GPT4All supports multiple models that you can choose from.

To add a model, go to "Add Models", where there are plenty of options to choose from, depending on your needs.

Choose the right model for you and download it.

Add Local Docs to GPT4All

Local docs or local files are the files you provide to the LLMs to perform a search and generate more accurate responses. These files can be of any type, and you have control over which file extensions to use.

Click on "Add Local Docs," enter a name, and choose the files to add.

Start Chatting with Local LLM

Once you have added local docs and a model, you can start chatting with the LLM and receive responses.

You also have the option to choose which model to use from the ones you downloaded and which local docs to use from the files you added.

Personalize Your GPT4All Experience

From the settings, you can control many things, from simple options like the theme to managing how models are used.

To change the theme, go to Settings → Application, where you can adjust the theme and font size:

Besides the theme and font settings, there is also a language option in case you want to change it.

From here, you can also choose the default model to use in case you have multiple models. At the bottom, there are some advanced settings, such as:

• Changing the path where models will be stored after downloading.
• Setting the number of threads to use.
• Choosing whether to share your data or not.

Another interesting setting to control is the file extensions allowed for local docs. To change this, go to Settings → Local Docs:

Add or remove file extensions as needed.

Conclusion

Running AI models locally gives you more control and privacy.

Using online APIs is easy and works well in many cases, but your data is sent to external servers, which may not be ideal for sensitive tasks.

To keep things private, you can use local LLMs (Large Language Models) that run directly on your computer - no internet or external servers needed.

In this guide, we showed you how to install and use GPT4All, a simple way to run AI locally and keep your data safe.