Can we have Datacentre AI... No, we have AI at Home.

[Dartz]

The "AI" we have at home

This is Colibri: https://github.com/JustVugg/colibri

It's some open source thing that lets you run big large language models offline on 'ordinary' hardware. It works by only holding the parts of the model that're actually being triggered in RAM, rather than all 300GB. The rest it streams from your disk. The default model it runs with is GLM which is also open-source (Unless Nvidia gets dickish with Hugginface)

It took 3 hours to generate a script that took Gemini 3 minutes to do - running at a blistering .24 tokens a second. But it ran - on a 5 year old laptop with 32GB of DDR4 RAM. And the script it spat out worked, and did exactly what I told it to do, logic errors and all. It'll run on 16 - potentially. But 32 is best. Also - it won't kill the SSD because reading is harmless.

Meanwhile, I know someone whose been able to justify buying a fully loaded Mac Studio to their manager to run these models locally because they were burning so many tokens just using these things to automate software testing and data generation.

One imagines once these things start getting unchained from the big datacentre model - then the big datacentres are going to be a little less valuable. 

I haven't given it  the ability to read or write files yet - I don't know how to do that. But it says a lot that it's gotten to the point where I can get it to work - and it's been a curious thing to play with and the first 'computer' thing that's amused me since I last ran Xubuntu. 

[classicdrogn]

Hm, interesting. I've been reading through various articles related to using AI coding with Godot engine due to as-yet-vague thoughts of makking another run at the same turn based tactical RPG I've poked at off and on for a decade, and found one line that particularly resonates with what I've seen as a whole:

The hype phase is over. What remains is a tool that works well within a defined scope and fails badly outside it.

That'sthe final conclusion of Vibe Coding Games: What Ships and What Breaks, with a couple of developer surveys cited to back it up. My own feeling from the reading agrees with the article; complexity is the enemy of generated coe, so either stick to super simple games like you could find in a classic arcade cabinet, or break it down into modular systems and integrate them the old fashioned way.

As a note for anyone else who might be curious: There is also a Colibri web browser, which was all the Ubuntu app store found when I checked there before using the link above. Very much not the same thing.

Also not mentioned in the Ziva docs, so if I do give it a go it'll probably be Ollama or LM Studio instead, since uding the Ziva addon from inside Godot reputedly works much better, for the quite logical reason that it lets the AI model directly interact with the engine to see the context and get error messages etc.