Installing a local LLM - Post 2 of 3

This post will discuss installing GPT4ALL from Nomic.io.

It worked reasonably easily.

I found it easier to install the app than the code.

1. Install Ubuntu linux version 22 (Jammy Jellyfish). 

a. Go to https://ubuntu.com/download 

b. Get the iso file

c. Copy it to a flash disk using Etcher (https://etcher.io) 

d. Insert flash disk, reboot, hit f2 on the keyboard; enter BIOS and tell your PC to boot off the flash disk. Take it from there. 

Step 2. Install gpt4all from github:


mkdir git

mkdir gpt4all

cd gpt4all

git clone https://github.com/ParisNeo/Gpt4All-webui

git clone https://github.com/nomic-ai/gpt4all


Step 3. Do this step

pip install 'pygpt4all==v1.0.1' --force-reinstall

sudo apt install libxcb-cursor0

Step 4. Install dev tools and pip

sudo apt-get install python3-dev

python3.10 -m pip install --upgrade pip

sudo apt install build-essential

cd Gpt4All-webui

python3.10 -m pip install -r requirements.txt


Step 5. Install hnswlib and chromadb

For some reason these do not seem to want to install unless you first upgrade pip and install build-essential which provides g++

cd git/gpt4all # if you are not already in that folder

sudo apt-get install python3-dev

python3.10 -m pip install --upgrade pip

sudo apt install build-essential

python3.10 -m pip install hnswlib

python3.10 -m pip install chromadb

then

python3.10 -m pip install chroma-migrate

chroma-migrate

python3.10 -m pip install -r requirements.txt


Step 6

Look for the app called 'chat' and run it. It should be in one of the git subfolders.

If that doesn't work, try get it precompiled from:


$ wget https://gpt4all.io/installers/gpt4all-installer-linux.run

$ chmod u+x gpt4all-installer-linux.run



Step 7: Run the installer:

$ ./gpt4all-installer-linux.run

$ sudo apt install libxcb-cursor0


However you still have to open it to download the LLM MODELS which are large files containing learning records of an AI that modeled/learnt a bunch of data. They are at least 2GB each so please do this on a fibre line. Different models have different levels of intelligence and knowledge. You have to experiment with them to see which are best. So far I think OpenAI's is best, but you can't get it. You have to make do with LLaMA from Facebook or Falcon or something else. 

Popular posts from this blog

Testing ChatGPT on coding - a script to make CSV to ICS

 So I gave ChatGPT a task of writing a CSV to ICS convertor, to take a spreadsheet of events and turn it into individual calendar file invitations. The idea is you can make a calendar with lots of event items in a spreadsheet, and then output ICS that you can just double-click to add to your calendar automatically. So here's the spreadsheet: "Event Name","Event Start Date","Event End Date","Event Start Time","Event End Time","event description or notes","URL applicable" "Meeting with John","2024-06-10","2024-06-10","09:00","10:00","Discuss project updates.","http://example.com" And here's the script ChatGPT made. It seems to get confused with things like Daylight Savings and/or Timezone but you can fix that manually. It took some debugging and I had to copy/paste my Mac's manpage for DATE into ChatGPT, but it got it eventually. The only
Read more

Ollama3 works - running a local LLM

Do this: curl -fsSL https://ollama.com/install.sh | sh ollama create llama3  It will then download the llama3 model, about 4.7 gb to run it: ollama run llama3  if you want it to read and summarise a textfile called textfile.txt: ollama run llama3 "Read the text file $(cat textfile.txt) and summarise the findings in one paragraph of no more than 300 words"  And here's a gui for it: https://github.com/amithkoujalgi/ollama-pdf-bot The system runs as a service (daemon) on Linux. CPU usage is high but not bad and response time much better than localGPT. It stores the model files (large LLM blobs) in  /usr/share/ollama/.ollama/models The models can be dowloaded from huggingface.io or https://ollama.com/library
Read more

What is a "token" , what is "temperature", and what is "n-shot"?

 TL;DR: token is a meaning unit; temperature is randomness. Token: In the context of AI and natural language processing (NLP), a "token" refers to the smallest unit of text or language that a model can understand. Tokens are usually words or subwords, but they can also be characters or even smaller units, depending on the tokenization method used. Word Tokens: In most cases, a token corresponds to a word in a sentence. For example, the sentence "ChatGPT is helpful" would be tokenized into three word tokens: ["ChatGPT", "is", "helpful"]. Subword Tokens: Some models use subword tokenization methods like Byte-Pair Encoding (BPE) or WordPiece to split words into smaller units. For example, "unhappiness" might be tokenized into ["un", "happiness"]. Character Tokens: In some cases, especially for languages with complex characters or when dealing with very short text, tokens can be individual characters. For ex
Read more