Follow Slashdot blog updates by subscribing to our blog RSS feed

 



Forgot your password?
typodupeerror
×

Comment Re:Flash is costly? (Score 5, Informative) 37

Creating the training dataset is the *last* step. I have dozens of TB of raw data which I use to create training datasets that are only a few GB in size. Of which I'll have a large number sitting around at any point in time.

Take a translation task. I start with several hundred gigs of raw data. This inflates to a couple terabytes after I preprocess it into indexed matching pair datasets (for example, if you have an article that's published in N different languages, it becomes (N * N-1) language pairs - so, say, UN, World Bank, EU, etc multilingual document sets greatly inflate). I may have a couple different versions of this preprocessed data sitting around at any point in time. But once I have my indexed matching pair datasets, I'll weighted-sample only a relatively small subset of it - stressing higher-quality data over lower quality and trying to ensure a desired mix of languages.

But what I do is nothing compared to what these companies do. They're working with common crawl. It grows at a rate of 200-300 TB per month. But the vast majority of that isn't going to go into their dataset. It's going to be markup. Inapplicable file types. Duplicates. Junk. On and on. You have to whittle it down to the things that are actually relevant. And in your various processing stages you'll have significant duplication. Indeed, even the raw training files... I don't know about them, but I'm used to working with jsons, and that adds overhead on its own. Then during training there's various duplications created for the various processing stages - tokenization, patching with flash attention, and whatnot.

You also use a lot of disk space for your models. It's not just every version of the foundation you train (and your backups thereof) - and remember that enterprise models are hundreds of billions to trillions of FP16 parameters in their raw states - but especially the finetune. You can make a finetune in like a day or so; these can really add up.

Certainly disk space isn't as big of a cost as your GPUs and power. But it is a meaningful cost. As a hobbyist I use a RAID of 6 20TB drives and one of 2 4TB SSDs. But that's peanuts compared to what people working with common crawl and having hundreds of employees each working on their own training projects will be eating up in an enterprise environment.

Comment Putting numbers into perspective (Score 3, Interesting) 113

This is all to produce a peak of 240k EVs per year. Production "starts" in 2028. It takes years for a factory to hit full production. Let's be generous and say 2030.

Honda sold 1,3 million vehicles in the US alone last year - let alone all of North America, including both Canada and Mexico. If all those EVs were just for the US it'd be 18% of their sales, but for all of North America, significantly less.

In short, Honda thinks that in 2030 only maybe 1/7th to 1/8th of its North American sales will be EVs. This is a very pessimistic game plan.

Comment Re:Gotta start somewhere (Score 5, Informative) 146

Ford made the Ford Ranger EV 1998 to 2002, then the Ford Focus Electric from 2011 to 2018 before switching to the Mach-E. They are not "new at it". They're just bad at it.

To be fair, I have a lot more hope for Ford than GM, as Farley seems to actually understand the critical importance of turning things around and the limited timeframes to do so, unlike GM, which still seems to only care about press.

Comment Re:Just bought... (Score 1) 165

I've never had a problem reading Chinese or Japanese books or watching movies. Yes, translation of idioms is always problematic, particularly from languages that are not related to our own, but a good translator can usually deal with that. For me, the problem with The Three Body Problem was the loopy plot, shallow characters and the author's abrupt genre jumping. I'm reasonably familiar with the Cultural Revolution and its profound effect on Chinese society, so ironically, reading the first chapter was the best part of the book. There was an interesting story there that wasn't a science fiction story.

Comment Re:How much is really delayed maintenance? (Score 1) 116

Copper is not "the last mile". It's the last five meters. If that. When people talk about "the grid", they're not talking about the wiring in your walls. Which you don't have to redo anyway for adding an EV. Nobody has to touch, say, your kitchen wiring to add an EV charger.

"The grid" is the wiring leading up to your house. Those conductors are alumium, not copper. Occasionally the SER/SEU cable will occasionally be copper, but even that's generally alumium these days. And that's only to the service connection point (not even to the transformer - to the point of handoff between grid-owned and the homeowner-owned, generally right next to the house), e.g. after the service drop line with overhead service that descends down to the building. The "last mile" is absolutely not copper. Approximately zero percent of modern grid-owned wiring is copper, and even the short customer-owned connection from the drop line into the house is usually alumium.

Grids are not copper. Period. This isn't the year 1890 here.

And no, grid operators don't make money selling power. They make money providing the grid through which power is sold.

I have never seen a single utility that charges a flat grid access fee to residential consumers, anywhere on Earth.

Distinction can be hard to grasp for someone utterly ignorant on the subject

Says a guy who thinks that there's a mile of copper leading up to your house.

Comment Re:How much is really delayed maintenance? (Score 5, Interesting) 116

The grid is not made of copper. You thought it was? Copper is for home wiring, if that. Up to that point, it's alumium, bundled with steel on major lines for tensile strength. Does it look like copper to you?

As for the article: grid operators don't build out grids on a lark. They do it to sell power, because they make money selling power. If people want to buy more power because they want to charge an EV, then that's more money available for them. EVs are a boon to grid operators. They're almost an ideal load. Most charging done at night, steady loads, readily shiftable and curtailable with incentives, etc. Daytime / fast charging isn't, but that's a minority. And except in areas with a lot of hydro, most regions already have the ample nighttime generation capacity; it's just sitting idle, power potential unsold. In short, EVs can greatly improve their profitability. Which translates to any combiation of three things:

1) More profits
2) A better, more reliable grid
3) Lower rates

    * ... depending on the regulations and how competitive of an environment it is.

As for the above article: the study isn't wrong, it's just - beyond the above (huge) problem - it is based on stupid assumptions. Including that there's zero incentives made for people to load shift when their vehicles charge, zero battery buffering to shift loads, and zero change in the distribution of generation resources over the proposed timeframe. All three of these are dumb assumptions.

Also, presenting raw numbers always leads to misleading answers. Let me rephrase their numbers: the cost is $7 to $26 per person per year. The cost of 1 to 5 gallons of gas per year at California prices..

Comment Re:Israel (Score 2) 118

Funny that to you, "Israel" and "Jews" are synonymous. As if all Jewish people unconditionally support all actions of the state of Israel, even those which are highly controversial within Israel itself.

This false synonymy creates an extremely harmful backlash. Stop doing it.

Comment Re:Titan or Bust! (Score 1) 70

Ukraine is not free

Give me a list of Ukrainian prime ministers since 2000, and compare it to a list of Russian presidents since 2000 . Thanks in advance.

Even before the conflict it was the poorest and most corrupt country in Europe

This is not even remotely true. Ukraine's Rule of Law Index in 2022 was 0,50; contrast with NATO members Turkey at 0,42 and Hungary at 0,52. And its scores were dragged down by the consequences of the war in Donbas.

with a military second in size in Europe only to Russia (hence the poverty)

Ukraine's percentage of GDP spent before the current invasion was 3,2%, and that was *with* the ongoing Donbas conflict . By contrast, the US, at peace, spends 3,45% of its GDP on the military. For some European contrasts:

Azerbaijan: 4,5%
Armenia: 4,3%
Russia: 4%
Greece: 3,7%

Before the 2014 Russian invasion, Ukraine's percentage of GDP spent on the military was 1,6%.

Comment Re:Terraforming on the same trip (Score 1) 70

ED: Just saw your second paragraph. But the things you speculate on are not exactly common on Titan, if they even exist on the surface at all (it's an icy crust ,not a rocky one). And either way, it'd be much easier with compounds other than methane.

And no, there doesn't seem to be meaningful amounts of nitrates in the atmosphere at least. You can see a list here. Nitrogen compounds are cyanide and nitrile compounds.

Slashdot Top Deals

8 Catfish = 1 Octo-puss

Working...