Installing a local instance of an LLM - Post 1 of 3

In this post I will basically mention two LLMs that I am going to install and thereafter comment on them.

Most AI apps seem to use python to run. So once you have installed 

1. Install Ubuntu linux version 22 (Jammy Jellyfish). 

a. Go to https://ubuntu.com/download 

b. Get the iso file

c. Copy it to a flash disk using Etcher (https://etcher.io) 

d. Insert flash disk, reboot, hit f2 on the keyboard; enter BIOS and tell your PC to boot off the flash disk. Take it from there. 

e. Install PHP 3.10

sudo apt-get install python3.10 


2. Install either or both of GPT4ALL or privateGPT.

GPT4ALL: https://github.com/nomic-ai/gpt4all

PrivateGPT: https://github.com/imartinez/privateGPT

For example

mkdir git

cd git

git clone https://github.com/imartinez/privateGPT

git clone https://github.com/nomic-ai/gpt4all


3. Once those are downloaded ...

cd git/privateGPT 


or

cd git/gpt4all

then

pip install 'pygpt4all==v1.0.1' --force-reinstall

sudo apt install libxcb-cursor0

python3.10 -m pip install -r requirements.txt

Note that this last step almost always fails with some error or dependency. I find that the following commands usually fix it.

Note also that the dependencies, LLM files and libraries are usually stored in ~/.cache/nomicai/ or similar.

4. Fix its nonsense

sudo apt-get install python3-dev

python3.10 -m pip install --upgrade pip

sudo apt install build-essential

python3.10 -m pip install hnswlib

python3.10 -m pip install chromadb

then

python3.10 -m pip install chroma-migrate

chroma-migrate

python3.10 -m pip install -r requirements.txt

Popular posts from this blog

Recent Experiments testing ChatGPT's limits and some useful prompts

Introduction ChatGPT has a lot of people excited, but it deserves this level of excitement. It has a number of faults which I list below - some with workarounds - but in general it is amazingly good. It is absolutely not for having fun chatting to a robot, it's really more like a brilliant coworker who is incredibly stupid at some quite mundane tasks, but incredibly good at more tricky tasks. Summary of findings so far.  1. API vs Interactive Version - a caution 2. Coding Strengths and Weaknesses - be very careful but very useful anyway 3. Shell Scripting - amazingly good, probably what it's best at 4. Text Summarisation - fiddly but useful 5. Spreadsheet Analysis - annoying, maybe do by hand 6. Multi-Step Instruction Handling - very annoying 7. Spellchecking and writing style/grammar - ok overall but fiddly 8. File Compatibility and Formatting - not great 9. APA References from DOIs - good 10. Reference Accuracy - bad, double-check its work 11. Tone and Style Adaptations - ex...
Read more

Testing ChatGPT on coding - a script to make CSV to ICS

 So I gave ChatGPT a task of writing a CSV to ICS convertor, to take a spreadsheet of events and turn it into individual calendar file invitations. The idea is you can make a calendar with lots of event items in a spreadsheet, and then output ICS that you can just double-click to add to your calendar automatically. So here's the spreadsheet: "Event Name","Event Start Date","Event End Date","Event Start Time","Event End Time","event description or notes","URL applicable" "Meeting with John","2024-06-10","2024-06-10","09:00","10:00","Discuss project updates.","http://example.com" And here's the script ChatGPT made. It seems to get confused with things like Daylight Savings and/or Timezone but you can fix that manually. It took some debugging and I had to copy/paste my Mac's manpage for DATE into ChatGPT, but it got it eventually. The only...
Read more

Some rough notes on FreedomGPT and PrivateGPT

 The following are rough notes. I'll tidy them up when I get the steps 100% right. https://www.freedomgpt.com/ https://huggingface.co/mosaicml/mpt-30b https://files.mistral-7b-v0-1.mistral.ai/mistral-7B-v0.1.tar use case: wordpress viruses; recognising one https://www.icloud.com/shortcuts/8bcc2979d6e34a3d90a48c3f77f96d68 openai - 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 hugg tok - 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 https://developer.nvidia.com/blog/nvidia-releases-open-source-gpu-kernel-modules/ https://github.com/eosphoros-ai/DB-GPT privateGPT fix: sudo apt-get install python3-dev python3.10 -m pip install --upgrade pip sudo apt install build-essential python3.10 -m pip install hnswlib python3.10 -m pip install chromadb then python3.10 -m pip install chroma-migrate chroma-migrate python3.10 -m pip install -r requirements.txt pip install pycc How to download the models: Visit the Code Llama repository in GitHub and follow the instructions in the README to run the down...
Read more