John's AI blog

So this is an update on my experimentation with LLMs. There are applications that let you access the models (LLMs) and models themselves. Each model is good at some things and bad at others. For example, ChatGPT is said to be bad at maths, whereas LLaMa is said to be better. Your mileage may vary.  So far, I've experimented with the following apps: - PrivateGPT (with a range of models) - LocalGPT (with a range of models) - ChatGPT (with GPT-3.5) - GPT4All (with a range of models) The following table documents the pro-con analysis: Depending on your purpose, I'd suggest you use ChatGPT if you have a mere goal of getting information. If however you need summaries of actual documents (a repository), I suggest you look at the others. GPT4All is easiest to use and install. However, I noticed that it really tended to hallucinate and give inaccurate answers. The way it works is that you install the app and then it asks you which LLM you want to download to answer queries. It supports ...

So apparently you need Cuda and Anaconda to use the GPU with python. As I explore more I'll add notes here. (For those reading, the GPU is just faster with this stuff). Unfortunately you have to have a specific type of Nvidia.  0. Background steps: a. Become Root: sudo su b. Install wget:  apt-get install wget 1. Install Cuda :  https://developer.nvidia.com/cuda-downloads  .  wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin sudo mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600 wget https://developer.download.nvidia.com/compute/cuda/12.2.2/local_installers/cuda-repo-ubuntu2204-12-2-local_12.2.2-535.104.05-1_amd64.deb sudo dpkg -i cuda-repo-ubuntu2204-12-2-local_12.2.2-535.104.05-1_amd64.deb sudo cp /var/cuda-repo-ubuntu2204-12-2-local/cuda-*-keyring.gpg /usr/share/keyrings/ sudo apt-get update sudo apt-get -y install cuda sudo apt install nvidia-cuda-toolkit If that doesn't work try: wget htt...

 TL;DR: token is a meaning unit; temperature is randomness. Token: In the context of AI and natural language processing (NLP), a "token" refers to the smallest unit of text or language that a model can understand. Tokens are usually words or subwords, but they can also be characters or even smaller units, depending on the tokenization method used. Word Tokens: In most cases, a token corresponds to a word in a sentence. For example, the sentence "ChatGPT is helpful" would be tokenized into three word tokens: ["ChatGPT", "is", "helpful"]. Subword Tokens: Some models use subword tokenization methods like Byte-Pair Encoding (BPE) or WordPiece to split words into smaller units. For example, "unhappiness" might be tokenized into ["un", "happiness"]. Character Tokens: In some cases, especially for languages with complex characters or when dealing with very short text, tokens can be individual characters. For ex...

So it seems that most AI things require Python. So what I will do in this blog as well is document peculiar things I discover about Python. For starters, it seems that if you use pip , the package manager, to install libraries and packages into python, you can get into a situation where two different AI apps need different versions of certain libraries. To do this, you use python's virtual environment, or python-venv. This post explains it quite nicely.  https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/