Sorry I meant to say parameters not tokens.

For reference Kimi 2.7 weights in at 1T tokens and 2 bit quants for it are in the 250GB to 350GB range. With 2.8T tokens I don't see it fitting.

This seems crazy to me. I guess it depends on how routing works inside of the model. From memory this one has almost 900 experts and 16 of them are selected on each pass. Maybe you can cache the most used layers and hope for the best but realistically I suspect you are still going to have to transfer tens of gigabytes per token and performance will be counted in seconds per token.

Would need a TB of RAM to run it effectively. Might be able to get away with a stupidly bit starved quant using as little as a half TB however would expect quality drop to be such you would be better off running a smaller model.

CPU inference isn't a huge deal /w only 50B active parameters unless expecting instant responses. For Intel hardware I think most 4th gen or later Xeons /w AMX are able to max out memory bandwidth. Realistically if you have 1TB of ram you are going to have at least an 8 channel system which means at least 300GB/s bandwidth. So maybe a dozen tokens a second performance.

Might be able to squeeze more performance using speculative decoding. I have no clue by how much.

As far as I know there are no other realistic options. GPUs /w 1TB total VRAM fall well outside of my pay grade. Apple's unified memory is popular for inference however as far as I know they don't have a 1TB SKU so you would have to get two 512GB systems and split the model between them.

A few years ago you would be able to spend less than $10k for the hardware... I don't even want to know what it costs today.

I don't think it matters all that much. Uncensored version of previous Kimi models are widely available. Given absurd model size you might have to wait some time for an uncensored K3.

This model has 2.8T parameters /w 50B active. They should focus on repeating layers or something to get the size down. Memory prices are too damn high.

I think batteries should not only be standardized but modular so people can purchase capacity or replacements as needed amongst a small number of different classes corresponding to size/mass/voltage/current. If I need lots of range or performance I purchase the maximum number of modules that will fit my vehicle. If not I might only purchase a single module accepting associated range and performance limits.

This would have the advantage of driving competition, innovation and cost reductions in the battery marketplace allowing for lighter and cheaper vehicles, redundancy or longer range vehicles as needed. People could buy a new car and reuse their existing battery instead of purchasing a new one saving huge sums of cash. They can replace or add modules as budget allows or needs change.

BMS should be integrated into each battery module with standardized interconnects and protocols for power, cooling and communication.

I don't understand how easily removable batteries in everything isn't globally required of all consumer products. All of this electronic shit eventually gets thrown in the trash with integrated rechargeable lithium ion batteries and ends up starting fires .. thousands of them each year throughout waste collection systems of Europe.

One thing I miss is the ability to talk to my phone via BT HFP to do basic tasks like make calls or send texts. Used to be able to do much of it decades ago on my blackberry yet with Android it was always Google malware or nothing. There are now amazing local full text STT IMEs and TTS... yet there does not seem to be anything anywhere that glues it all together.

The closest I know of is Dicio yet it blows up and doesn't work at all when the screen is off rendering it useless. Something that you can enter a URL to talk to an LLM of your choice would be nice too yet I would settle for just having the basics. This has always been baked into the profile and I never understood why it seems to be abandoned or there is very little interest. I've looked for years and it is always crickets.

Why? What was the point of the irrelevant statements?

You tell me. I have no clue where you are going with this.

The Israelis have not initiated hostilities in Lebanon and have stated they have no territorial or other designs on Lebanon. They have only responded to incoming fire from non-state actors (Hezbollah) in the form of rockets and missiles launched from Lebanon. Attacks from Hezbollah were being directly coordinated with the IRGC.

The Iranian regime is the same technology dressed in different cloths. While underlying details are different the mechanisms and effects are similar. Obviously nobody thinks the regime are literally Nazi's yet the dismissal as rhetorical completely misses the point.

This statement lacks any useful specificity.

The whole effort of design of software systems is ultimately the effective management of complexity. Complexity of features that provide real world value is the developers problem to manage. If "technical debt just keeps compounding" it is probably best to find a better developer.

Now things are getting stupid. Death to Israel is really just an inside joke. Ditto for asserting Israel should be wiped off the map and calling it a cancerous tumor of a state that should be removed from the region.

Well north of 90% fled. There are only a few thousand left. Certainly less than what remained in Nazi Germany during WWII.

And to think you've lectured others here about spouting nonsense.

The Japs were worse than the Nazis.

Are you sure?

Kill the Jews...check.
External expansionist aggression... check.
Delusions of world domination... check.
Internal repression... check.

Even if it looks like a Nazi, swims like a Nazi, and quacks like a Nazi... I suppose there is a chance it might not be a Nazi... yet that seems rather unlikely.

That mostly went out the window with Khomeini in the 80s. Ali openly admitted he was not qualified and was chosen because the IRGC thought he could be controlled. It is beyond question Mojtaba is comically unqualified (hereditary ascension... Khomeini must be rolling in his grave). He was only selected by the assembly of experts under duress. So no the clerics don't run shit and have very little power.. Iran is now more of a mafia state.

You obviously have not been paying attention to current events. The Lebanese just signed an agreement with the Israelis and the Americans in which Hezbollah who has been lobbing rockets into Israel from Lebanese territory would be disarmed and Israel would leave. The Lebanese people are fed up with Hezbollah's bullshit... from memory opinion polling 80% of Lebanon wants these fuckers disarmed. How can any state tolerate not having a monopoly on the use of violence? This is a basic requirement for any state.

So I'm a little confused here. Are the Lebanese misguided? Do they not understand Hezbollah is the only thing standing between Lebanon and genocide? As soon as Hezbollah is disarmed is that when the country is going to be leveled by the Israelis out of boredom...right? Is this what you honestly believe?

In Iraq Al Zaidi is cleaning house rounding up dozens of corrupt politicians swimming in Iranian money and putting the armed groups on notice. The shit that has been pulled out of backyard gardens and stashed away in homes is literally comic gold.

https://www.cbsnews.com/news/i...

The genocidal Israelis have also managed to mostly normalize relations with the Arab states and even delivered air defense systems to the UAE. Perhaps the systems are a Trojan horse and when UAE least expects it they will activate and blow up the UAE.

The Iranian people are amongst Israel's biggest fans which I'm sure pisses the living fuck out of you.

Syria has been taken out of play, Hamas is severely degraded and hopefully the situation in Yemen will improve as the Nazi regimes support for Houthis dries up.

Does it make you sad the regimes proxies are being systematically eroded? Do you support aggressive expansionist forever "revolution" that seeks to kill the Jews while oppressing and massacring its own people internally? Does oppression and the largest two day massacre since Babi Yar make you happy?