Some rough notes on FreedomGPT and PrivateGPT

 The following are rough notes. I'll tidy them up when I get the steps 100% right.


https://www.freedomgpt.com/

https://huggingface.co/mosaicml/mpt-30b

https://files.mistral-7b-v0-1.mistral.ai/mistral-7B-v0.1.tar


use case: wordpress viruses; recognising one


https://www.icloud.com/shortcuts/8bcc2979d6e34a3d90a48c3f77f96d68


openai - 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

hugg tok - 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


https://developer.nvidia.com/blog/nvidia-releases-open-source-gpu-kernel-modules/


https://github.com/eosphoros-ai/DB-GPT


privateGPT fix:

sudo apt-get install python3-dev

python3.10 -m pip install --upgrade pip

sudo apt install build-essential

python3.10 -m pip install hnswlib

python3.10 -m pip install chromadb

then

python3.10 -m pip install chroma-migrate

chroma-migrate

python3.10 -m pip install -r requirements.txt


pip install pycc



How to download the models:


Visit the Code Llama repository in GitHub and follow the instructions in the README to run the download.sh script.

When asked for your unique custom URL, please insert the following:

https://download2.llamameta.net/*?Policy=eyJTdGF0ZW1lbnQiOlt7InVuaXF1ZV9oYXNoIjoia3J1d3ZuODE2NDlxN3E1cHc2bm14NW82IiwiUmVzb3VyY2UiOiJodHRwczpcL1wvZG93bmxvYWQyLmxsYW1hbWV0YS5uZXRcLyoiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE2OTMxMjI3MTN9fX1dfQ__&Signature=UFLEAWXlW7ZBHjOT3zzMiUxN0dDutjoR9q0yaiXiUUPSAReWjLqfcDlRhI3sRi11ms7dmGx92CZAAyNEBTByADJvq0nF28YRlVC4ZQwBIUrpB81M1V7Dv4jds2yiK6RTVtBgz5UW1RUFKopf6rynUkBhc34UhaNWkxep7UMjH8FOv5nlIyTaJBC1d81x-LifRJzP0bD3zJ8qQ1v5ObYKFDgFmf9ltR6K-idMcqkInB%7EbEgf6QcbXiPr2lEML-QQNAxho91%7ECEN%7EM8MelFhY3TwZtqPyPF8BTVviyOd9VAfLU1ZujGSMCue-tb1blZH9c-JimwxZ3ptLJPWh-5Ald3w__&Key-Pair-Id=K15QRJLYKIFSLZ&Download-Request-ID=1369068316975498

Select which model weights to download




 sudo apt-get install python3.10-venv



wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin

sudo mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600

wget https://developer.download.nvidia.com/compute/cuda/12.2.2/local_installers/cuda-repo-ubuntu2204-12-2-local_12.2.2-535.104.05-1_amd64.deb

sudo dpkg -i cuda-repo-ubuntu2204-12-2-local_12.2.2-535.104.05-1_amd64.deb

sudo cp /var/cuda-repo-ubuntu2204-12-2-local/cuda-*-keyring.gpg /usr/share/keyrings/

sudo apt-get update

sudo apt-get -y install cuda


https://www.geeksforgeeks.org/running-python-script-on-gpu/


wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb

sudo dpkg -i cuda-keyring_1.1-1_all.deb

sudo apt-get update

sudo apt-get -y install cuda


sudo apt install nvidia-cuda-toolkit



ubuntu-drivers devices

apt install nvidia-driver-510

update-initramfs -u

reboot

apt install nvidia-cuda-toolkit

mkdir /usr/local/cuda

cd /usr/local/cuda

ln -s /usr/lib/nvidia-cuda-toolkit/bin bin

ln -s /usr/include include

ln -s /usr/lib/x86_64-linux-gnu lib64

ln -s /usr/share share

Popular posts from this blog

Recent Experiments testing ChatGPT's limits and some useful prompts

Introduction ChatGPT has a lot of people excited, but it deserves this level of excitement. It has a number of faults which I list below - some with workarounds - but in general it is amazingly good. It is absolutely not for having fun chatting to a robot, it's really more like a brilliant coworker who is incredibly stupid at some quite mundane tasks, but incredibly good at more tricky tasks. Summary of findings so far.  1. API vs Interactive Version - a caution 2. Coding Strengths and Weaknesses - be very careful but very useful anyway 3. Shell Scripting - amazingly good, probably what it's best at 4. Text Summarisation - fiddly but useful 5. Spreadsheet Analysis - annoying, maybe do by hand 6. Multi-Step Instruction Handling - very annoying 7. Spellchecking and writing style/grammar - ok overall but fiddly 8. File Compatibility and Formatting - not great 9. APA References from DOIs - good 10. Reference Accuracy - bad, double-check its work 11. Tone and Style Adaptations - ex...
Read more

Testing ChatGPT on coding - a script to make CSV to ICS

 So I gave ChatGPT a task of writing a CSV to ICS convertor, to take a spreadsheet of events and turn it into individual calendar file invitations. The idea is you can make a calendar with lots of event items in a spreadsheet, and then output ICS that you can just double-click to add to your calendar automatically. So here's the spreadsheet: "Event Name","Event Start Date","Event End Date","Event Start Time","Event End Time","event description or notes","URL applicable" "Meeting with John","2024-06-10","2024-06-10","09:00","10:00","Discuss project updates.","http://example.com" And here's the script ChatGPT made. It seems to get confused with things like Daylight Savings and/or Timezone but you can fix that manually. It took some debugging and I had to copy/paste my Mac's manpage for DATE into ChatGPT, but it got it eventually. The only...
Read more

Ollama3 works - running a local LLM

Do this: curl -fsSL https://ollama.com/install.sh | sh ollama create llama3  It will then download the llama3 model, about 4.7 gb to run it: ollama run llama3  if you want it to read and summarise a textfile called textfile.txt: ollama run llama3 "Read the text file $(cat textfile.txt) and summarise the findings in one paragraph of no more than 300 words"  And here's a gui for it: https://github.com/amithkoujalgi/ollama-pdf-bot The system runs as a service (daemon) on Linux. CPU usage is high but not bad and response time much better than localGPT. It stores the model files (large LLM blobs) in  /usr/share/ollama/.ollama/models The models can be dowloaded from huggingface.io or https://ollama.com/library
Read more