output is about 6 tokens/s with 16k context window i'm not having any issues since it went live this afternoon. it's not sparkling like opus 4.5/6 but gets the job done

i generally send it a voice note via telegram while driving and then check back in like 1-2 min, or it is sending me a reminder about something on our shared calendar. it's still faster than texting my buddy about making plans for this weekend or whatever.

I'm using a $200 used ~5 year old (from the ebay listing) HP EliteDesk 805 G6 DM Desktop Ryzen 5 PRO 4650GE 3.3GHz 32GB RAM 512GB SSD WiFi in cpu mode... you don't need a gpu to run single user local LLM... just a bunch of ram. This isn't 2022 anymore

It's about 5 tokens/second which is totally fine for an async assistant. 20 tokens/second is about the lower limit for usable in realtime. You can also set it up to use a smaller model for quick questions (what are the next 6 items on my calendar/to-do list?) and drop through to the bigger slower model for harder questions (can you add this feature to my internal ticketing system and redeploy?)

I ordered 64gb of ram about an hour ago and i'm planning on running either qwen 35B-A3B 8 bit or 122B-A10B 3 bit in fully offline mode.
 
  >the actual cost of 'running the AI.'
 
is a fixed $200 cost (ram upgrade) + electricity

I cancelled my subscription overnight, and I'm using the free credits they gave me to wrap up some things and transition away. I am not going to be locked into someone's walled garden again.

I'm skeptical this company is doing it properly (or even has their own models), but I think you could do this with two models.

The documenter is trained on all available data.

The coder is trained but without any copy left code.

Clean room reverse engineering actually seems like a place where AI will be extremely capable.

Used? Why stop? Others and I still use mutt!

With open source you can document a lot more than with closed source.

Generally if the implementors have seen the original then it's not clean room.

It seems to me if one AI writes documentation and then another that never saw the code in question writes code it'd be legit.

How is that illegal?

I don't like ads either, but I do like that they (at least for now) have a paid tier with no ads. If there was an option to use google services at some paid tier, without being part of their ad network, I'd probably pay it. But there isn't and llm is as good as search these days (in many cases anyways) so I'm happy to jump ship. Piss off, google.

Trump in his first term was willing to go all-in on human spaceflight to mars...until he realized he couldn't get it done before the end of his term. Trump has always been interested in space stuff...but only if it's achievable within his term. This seems like a play to keep contractors employed and skills sharp until the next administration is seated, which will hopefully be willing to invest in goals longer than 4 years.

Unfortunately everyone picked an opinion two years ago, when AI was genuinely garbage beyond some basic bash scripts or a top 1000 bug/question on stack exchange (which mostly overlap). AI started getting really good in Dec '24, particularly spring '25 and by August 2025 even the $20/mo tier of chatgpt was starting to get legit as OpenAI started to try catching up with (now market leader) Anthropic and their blessed claude code. The 4.5/4.6 models released this year are nothing short of incredible, and the Qwen 3.5 series of models are right behind the state of the art models. Google is doing some stuff too but I'm kind of done giving them my money.
 
In 2-3 years we'll have found all 20,000 top reasons LLMs hallucinate things and solved for 95% of them
 
Creatives rallied against LLMs but as has been proven, nobody actually cares about making funny pictures of , they just want to know that they can.