Installing a local LLM - post 3 of 3

This is a post on getting PrivateGPT to work.

1. Install git repo

Firstly let's assume you have already downloaded and installed PrivateGPT in your local git repository. If that means nothing to you, open your terminal (Linux) and type:

mkdir git

cd git

git clone https://github.com/imartinez/privateGPT


2. Make sure python and libraries are working

Mostly if you install the components it just works, however Ubuntu 22 comes with a version of g++ and friends that it doesn't like. So you have to use Python 3.10 specifically and then run these commands:

cd git/privateGPT

sudo apt-get install python3.10 # please only use this version

pip install 'pygpt4all==v1.0.1' --force-reinstall

sudo apt install libxcb-cursor0


sudo apt-get install python3-dev

python3.10 -m pip install --upgrade pip

sudo apt install build-essential

python3.10 -m pip install hnswlib

python3.10 -m pip install chromadb


then

python3.10 -m pip install chroma-migrate

chroma-migrate

python3.10 -m pip install -r requirements.txt



3. Test it

Copy some PDFs or text files into the folder "source_documents"

cp ~/*.pdf ~/*.txt source_documents/ 


run the ingest script to read the files:

python3.10 ingest.py


run the privateGPT script

python3.10 privategpt.py

Popular posts from this blog

Testing ChatGPT on coding - a script to make CSV to ICS

 So I gave ChatGPT a task of writing a CSV to ICS convertor, to take a spreadsheet of events and turn it into individual calendar file invitations. The idea is you can make a calendar with lots of event items in a spreadsheet, and then output ICS that you can just double-click to add to your calendar automatically. So here's the spreadsheet: "Event Name","Event Start Date","Event End Date","Event Start Time","Event End Time","event description or notes","URL applicable" "Meeting with John","2024-06-10","2024-06-10","09:00","10:00","Discuss project updates.","http://example.com" And here's the script ChatGPT made. It seems to get confused with things like Daylight Savings and/or Timezone but you can fix that manually. It took some debugging and I had to copy/paste my Mac's manpage for DATE into ChatGPT, but it got it eventually. The only
Read more

Ollama3 works - running a local LLM

Do this: curl -fsSL https://ollama.com/install.sh | sh ollama create llama3  It will then download the llama3 model, about 4.7 gb to run it: ollama run llama3  if you want it to read and summarise a textfile called textfile.txt: ollama run llama3 "Read the text file $(cat textfile.txt) and summarise the findings in one paragraph of no more than 300 words"  And here's a gui for it: https://github.com/amithkoujalgi/ollama-pdf-bot The system runs as a service (daemon) on Linux. CPU usage is high but not bad and response time much better than localGPT. It stores the model files (large LLM blobs) in  /usr/share/ollama/.ollama/models The models can be dowloaded from huggingface.io or https://ollama.com/library
Read more

What is a "token" , what is "temperature", and what is "n-shot"?

 TL;DR: token is a meaning unit; temperature is randomness. Token: In the context of AI and natural language processing (NLP), a "token" refers to the smallest unit of text or language that a model can understand. Tokens are usually words or subwords, but they can also be characters or even smaller units, depending on the tokenization method used. Word Tokens: In most cases, a token corresponds to a word in a sentence. For example, the sentence "ChatGPT is helpful" would be tokenized into three word tokens: ["ChatGPT", "is", "helpful"]. Subword Tokens: Some models use subword tokenization methods like Byte-Pair Encoding (BPE) or WordPiece to split words into smaller units. For example, "unhappiness" might be tokenized into ["un", "happiness"]. Character Tokens: In some cases, especially for languages with complex characters or when dealing with very short text, tokens can be individual characters. For ex
Read more