Installing a local LLM - Post 2 of 3

This post will discuss installing GPT4ALL from Nomic.io.

It worked reasonably easily.

I found it easier to install the app than the code.

1. Install Ubuntu linux version 22 (Jammy Jellyfish). 

a. Go to https://ubuntu.com/download 

b. Get the iso file

c. Copy it to a flash disk using Etcher (https://etcher.io) 

d. Insert flash disk, reboot, hit f2 on the keyboard; enter BIOS and tell your PC to boot off the flash disk. Take it from there. 

Step 2. Install gpt4all from github:


mkdir git

mkdir gpt4all

cd gpt4all

git clone https://github.com/ParisNeo/Gpt4All-webui

git clone https://github.com/nomic-ai/gpt4all


Step 3. Do this step

pip install 'pygpt4all==v1.0.1' --force-reinstall

sudo apt install libxcb-cursor0

Step 4. Install dev tools and pip

sudo apt-get install python3-dev

python3.10 -m pip install --upgrade pip

sudo apt install build-essential

cd Gpt4All-webui

python3.10 -m pip install -r requirements.txt


Step 5. Install hnswlib and chromadb

For some reason these do not seem to want to install unless you first upgrade pip and install build-essential which provides g++

cd git/gpt4all # if you are not already in that folder

sudo apt-get install python3-dev

python3.10 -m pip install --upgrade pip

sudo apt install build-essential

python3.10 -m pip install hnswlib

python3.10 -m pip install chromadb

then

python3.10 -m pip install chroma-migrate

chroma-migrate

python3.10 -m pip install -r requirements.txt


Step 6

Look for the app called 'chat' and run it. It should be in one of the git subfolders.

If that doesn't work, try get it precompiled from:


$ wget https://gpt4all.io/installers/gpt4all-installer-linux.run

$ chmod u+x gpt4all-installer-linux.run



Step 7: Run the installer:

$ ./gpt4all-installer-linux.run

$ sudo apt install libxcb-cursor0


However you still have to open it to download the LLM MODELS which are large files containing learning records of an AI that modeled/learnt a bunch of data. They are at least 2GB each so please do this on a fibre line. Different models have different levels of intelligence and knowledge. You have to experiment with them to see which are best. So far I think OpenAI's is best, but you can't get it. You have to make do with LLaMA from Facebook or Falcon or something else. 

Popular posts from this blog

Hallucinations/inaccuracies

 You have to test LLMs individually for hallucinations and inaccuracies. These below are attempts at summarising my first academic article. Useless! john@john-GF63-Thin-11SC:~/ai$ ./scripts/ollama_summarise_one.sh SAMPLES/hawaiiarticle.txt  Sure, here's the paragraph you requested: >The problem with some of the analyses of Libet is that they make it look  like the details were complicated. However, I do believe I agree with  McCrone that Libet's data is sloppier than people often realize. Glynn  made that point a long time ago and it was the main point of my plots in  the Tucson III Proceedings. john@john-GF63-Thin-11SC:~/ai$ ./scripts/ollama_summarise_one.sh SAMPLES/hawaiiarticle.txt  Sure, here is the paragraph you requested: >Libet showed that a thalamic stimulus requires a duration of about 250  msec to be felt whereas a 20 msec skin stimulus is adequate. This shows  that the intensity of the stimulus plays an important role in dete...
Read more

Access to model mistralai/Mistral-Nemo-Base-2407 is restricted. You must be authenticated to access it.

 If you get an error like Access to model mistralai/Mistral-Nemo-Base-2407 is restricted. You must be authenticated to access it. Then pip install huggingface_hub huggingface-cli login it will ask you for a token go to https://huggingface.co/settings/tokens it will have an option to generate a token make sure you click the checkboxes for APIs generate the token copy it go back to the terminal at the top paste.
Read more

Building my own LLM

Image
Yesterday I started building my own LLM because I want to avoid licensing issues for basic chatbot usages. It's super slow training these things. I won't post code here because I plan to commercialise it, but here's the progress screenshot: I've set it to use the following parameters. Sadly my GPU is rather small so it is taking ages. # Training hyperparameters TEMPERATURE=0.7 TOP_K=40 TOP_P=0.95 REPEAT_PENALTY=1.1 MAX_NEW_TOKENS=200 BLOCK_SIZE=32 BATCH_SIZE=8 NUM_EPOCHS=5 LEARNING_RATE=0.001 CHECKPOINT_INTERVAL=1 USE_GPU=true # File and directory settings MODEL_DIR=models/ CORPUS_DIR=repo/ LOG_DIR=logs/ BACKUP_DIR=bak/ TOKENIZER_PATH= # Internal settings EVAL_INTERVAL=100 EVAL_ITERS=100 SEED=1337 INPUT_FILE=input.txt
Read more