Ollama3 works - running a local LLM

Do this:

curl -fsSL https://ollama.com/install.sh | sh

ollama create llama3 

It will then download the llama3 model, about 4.7 gb

to run it:

ollama run llama3 

if you want it to read and summarise a textfile called textfile.txt:

ollama run llama3 "Read the text file $(cat textfile.txt) and summarise the findings in one paragraph of no more than 300 words" 

And here's a gui for it:

https://github.com/amithkoujalgi/ollama-pdf-bot

The system runs as a service (daemon) on Linux. CPU usage is high but not bad and response time much better than localGPT.

It stores the model files (large LLM blobs) in 

/usr/share/ollama/.ollama/models

The models can be dowloaded from huggingface.io or https://ollama.com/library

Popular posts from this blog

Testing ChatGPT on coding - a script to make CSV to ICS

 So I gave ChatGPT a task of writing a CSV to ICS convertor, to take a spreadsheet of events and turn it into individual calendar file invitations. The idea is you can make a calendar with lots of event items in a spreadsheet, and then output ICS that you can just double-click to add to your calendar automatically. So here's the spreadsheet: "Event Name","Event Start Date","Event End Date","Event Start Time","Event End Time","event description or notes","URL applicable" "Meeting with John","2024-06-10","2024-06-10","09:00","10:00","Discuss project updates.","http://example.com" And here's the script ChatGPT made. It seems to get confused with things like Daylight Savings and/or Timezone but you can fix that manually. It took some debugging and I had to copy/paste my Mac's manpage for DATE into ChatGPT, but it got it eventually. The only
Read more

Hallucinations/inaccuracies

 You have to test LLMs individually for hallucinations and inaccuracies. These below are attempts at summarising my first academic article. Useless! john@john-GF63-Thin-11SC:~/ai$ ./scripts/ollama_summarise_one.sh SAMPLES/hawaiiarticle.txt  Sure, here's the paragraph you requested: >The problem with some of the analyses of Libet is that they make it look  like the details were complicated. However, I do believe I agree with  McCrone that Libet's data is sloppier than people often realize. Glynn  made that point a long time ago and it was the main point of my plots in  the Tucson III Proceedings. john@john-GF63-Thin-11SC:~/ai$ ./scripts/ollama_summarise_one.sh SAMPLES/hawaiiarticle.txt  Sure, here is the paragraph you requested: >Libet showed that a thalamic stimulus requires a duration of about 250  msec to be felt whereas a 20 msec skin stimulus is adequate. This shows  that the intensity of the stimulus plays an important role in determining  the duration of the require
Read more