Forgot your password?
typodupeerror

Comment A definite mix (Score 2) 73

Salon was a definite mix. A left-wing bias, and some good analysis despite that. But then they also had people like Amanda Marcotte and Arthur Chu writing for them who frequently just wrote nasty attack pieces on anyone they didn't see as sufficiently left-wing, or worse people who came across as weird and nerdy. Look for example at the unpleasant attacks they wrote about Scott Aaronson. If Salon had had any serious ethical concerns or general care about moral approaches to things, they would have fired Chu simply for his own statements that what mattered was "winning" and not anything like an actual attempt at conversation or accuracy. So, yes, there was something unfortunate here. But most of the good parts of Salon died over a decade ago.

Comment Re:It's simpler than that, (Score 1) 81

it's a very popular feature on Macintosh computers. I worked for years as a service tech, and did hundreds of migrations. Apple has always made it very easy... I can remember plugging two computers together with a SCSI cable and running the migration assistant! I can still remember the days when windows users had to buy one of our "pc migration kits", that was a box with software and a special cable - USB-A on BOTH ends, and a plastic box in the middle. We sold quite a few of those.

Firewire on the mac was a breeze. Just launch the (included) migration assistant on the new computer, boot up the old computer in "firewire target disk mode" (would do that even if unbootable), connect computers with a firewire cable, and ALL your user data/account migrate, along with most or all of your fully installed apps. Apps with twitchy anti-piracy measures often needed to be reinstalled but not always. (and sometimes just required re-activating)

Apple did upgrade the hardware to do USB-target-disk-mode for awhile, but unfortunately Apple has dropped target disk mode in the newest systems. That's a shame, it was an INCREDIBLY useful tool for the technicians. I could plug in a hard drive that wouldn't even boot to another computer for remote repair or data recovery, never needed to pull a hard drive unless it was physically damaged. Nowadays you have the same migration assistant but it must be launched on both ends and you use an ethernet cable, so you can't just browse the other computer's hard drive if it won't boot.

Comment Jews taking the same position (Score 2) 72

Danya Ruttenberg, a prominent Conservative Rabbi wrote a detailed piece arguing that Jews should ask for exemptions from using AI https://www.lifeisasacredtext.com/the-jewish-case-for-ai-work-exemptions/. I found both the halachic (Jewish law) reasoning and elements of how American secular law handles such exemptions to be unconvincing. (I've sent Rabbi Ruttenberg's piece to a bunch of people as one of the better arguments for a moral problem with using AI in general, even as I find it unconvincing.) But it is notable there's a trend here. It will be interesting to see if any Christian clergy try to make a similar argument.

Comment That's not the point of turnitin (Score 2) 151

The last time I interacted with turnitin, I could easily see its shortcomings. That software is made to look for cheating and plagiarism. That works well for essays, but for math you're looking at a square peg - round hole problem. There are a lot of different ways to write a paragraph on an historical figure. There are only so many ways to solve a given practice math problem. It's easy to show that a history student copy / pasted from wikipedia, it's very difficult to show if a math student came up with the correct math answer on their own or googled it.

The proper way to asses learning in a math class is a live in-person test. Homework should be for the students to practice their learning, and if they choose to not learn while doing it that will be seen on the exam.

Comment Re:Good (Score 1) 34

The business case for LLMs is a bit broader than that. They don't need to replace that many workers if they make enough workers more efficient or more reliable (such as simply having LLMs look over drafts).

As things stand 'reliable' is not a word you associate with AI.

That's why it is using it to look over drafts. You take the AI output and then go through its suggested edits or concerns. Essentially it functions as a very fast, half-way decent proof-reader. For example, my spouse looks over a lot of drafts of things I write, but if I give her a 15 page paper, there's good work on the first 8 or 9 pages, probably a joke on page 10 about my notation, one or two comments on the next page, and then a few comments on the bibliography. Claude is not as good a proof-reader as she is, but Claude doesn't poop out after a few pages. And Claude is much faster. So reliability goes up without having to rely on the AI being reliable itself.

And sometimes it catches subtle things. For example, not too long ago, I gave a paper draft to Claude, and it noted in the bibliography that the journal volume number seemed unlikely given the year; it turned out that the volume was correct, but I had actually written the wrong journal down in the entry. Another time, Claude noted that one entry in a bibliography was from a "borderline predatory" (its words but it wasn't wrong) journal, and since I wasn't using it for much other than as part of a list of "hey, here's prior literature" and the paper in question didn't prove anything interesting, I discussed just dropping it from the citation list. Claude then without me asking, pulled up preprint I already had on the arXiv and noted it saw that paper cited there and suggested we remove the paper there also. I didn't do so, but this is the sort of thing that it can suggest. One still has the human make the decisions, but the input itself is useful. And for the most part, verification by a human for most issues is much faster than finding the issues one self.

That's what I was talking about. You need smaller models specialised for the specific scientific/engineering problem they'll be used for.

Not really. It turns out that specialized models do not do as well as large general models. This is essentially a variant of the Bitter Lesson https://en.wikipedia.org/wiki/Bitter_lesson. In this case, there's other reason for it also. There's a lot of reasoning science and math discussions in non-specialized training data. For example, one paper I wrote with two coauthors started as a Reddit discussion. But there's also just a lot of subtle interaction that specialized models don't necessarily get. Math is easier to understand if one has a good command of physics. Engineering is better if one has math and physics. Etc. And sometimes a technique gets directly pulled over. For example, in the large OpenAI paper drop, one of the graph theory problems relied on a trick involving a weighing on the graph that involved complex numbers designed to cancel, where the trick came from a physics context that did something similar. A math specialized model would have been unlikely to find and import that trick.

The distinction between a commodity or tech bubble is interesting but in both cases you can have a tech or commodity which is definitely useful but whose usefulness is being overestimated, and tech or commodity which has very limited utility and whose current value is mostly based on hype. Bitcoin and especially NFT are good examples of the latter type.

Right, and the line here isn't clear. One take on NFTs is that they are nominally a tech but functioned purely as a commodity bubble. Or that NFT is a tech but it was used to make commodities which had a bubble. And the line between tech and commodity was blurry historically. When Spain entered the "platinum age," that was based in part on how to make nearly pure platinum was a tech, but the platinum itself was a commodity.

How much computing you get out of a token is surely a purely business decision by the company selling the tokens. Theoretically there's nothing stopping them from making a single token give you unlimited lifetime access to their AI - probably wouldn't make much business sense but I don't know how much sense any of this is making atm. Tokens giving you more bang for the buck may mean the tech is improving or there's more infrastructure or it may just be due to a business strategy this particular company is pursuing.

Since per a token output seems to be getting better also for open-weight models, this suggests this is more than just a business decision. And some of it is specific changes; we know that larger models with more training data for example will often produce higher quality output. But also aside from scaling, there have been architectural improvements. For example, one insight that occurred was that having lower precision weights which allowed more weights was often a positive tradeoff. That lead to changes in both the software and hardware ends, as the GPUs were modified as a result. Similar, there's been changes in what functions are used for the feedforward function.

Comment Re:Good (Score 1) 34

The business case for LLMs is a bit broader than that. They don't need to replace that many workers if they make enough workers more efficient or more reliable (such as simply having LLMs look over drafts). There's also at this point, clear usage cases for scientific and mathematical research but that's using the very high end models, and admittedly is a pretty small market, not enough to justify massive expenditure at this level. The tulip example is interesting; what you are touching on their is the distinction between a "tech bubble" (like railroads or internet) and a "commodities bubble" (tulips, housing, Beanie Babies). Note that the degree to which the tulip bubble was real is questionable. See https://theconversation.com/tulip-mania-the-classic-story-of-a-dutch-financial-bubble-is-mostly-wrong-91413, and the line between a tech bubble and a commodity bubble itself can be blurry (see for example Bitcoin and NFTs). One of the difficulties here is that the LLMs keep improving, and the cost per token is going down, and the amount one can do per a token is going up. But that's happening in part due to massive investment in both infrastructure and research. How sustainable any of that is is unclear.

Comment Like so many other things... (Score 2) 6

Paypal used to be a useful tool that did useful things. Now it's just a way for rich people and con artists to extract money from others who apparently don't know better. It's infuriating how many nonprofits are losing truck loads of cash to paypal as the fees continue to climb. The ones I'm involved with switch to Zelle a while back for the majority of all transfers and that has been a huge step in the right direction though of course not without its own set of tax weirdness.

Comment Re:Good (Score 5, Interesting) 34

Sooner or later people are going to realize that AI is just another name for Linear Algebra and all this will come crashing down

There are at least four problems with this. First, linear algebra is from a math standpoint insanely powerful. For example, all of representation theory https://en.wikipedia.org/wiki/Representation_theory is using linear algebra to understand groups. Much of the study of modular forms https://en.wikipedia.org/wiki/Modular_form involves understanding the behavior of modular forms over SL_2(Z) which has heavy linear algebra elements to it. People who see linear algebra as somehow a weak thing are often people who took a single semester of it and didn't do much else with it.

Second, the idea that these AI systems are just linear algebra isn't accurate anyhow. While there's a lot of linear involved, a major part of how transforms like LLMs work is the intervening non-linear layers, which use ReLu https://en.wikipedia.org/wiki/Rectified_linear_unit, Softplus, or a similar function. So, summarizing this as just linear algebra isn't accurate. It is linear algebra with some spice.

Third, even if this were somehow just linear algebra, that shouldn't therefore be a reason to think that there's any limits to what is going on. At the end of the day, the empirical success is what matter, and systems which can do things like prove the quasi-Riemann Hypothesis or solve other major math problems are pretty impressive regardless of what is under the hood. So, if one does think this is just linear algebra, that's not a reason to dismiss the systems but a reason to increase your estimate of what can be done with mere linear algebra.

Fourth, slowing down connections like this may make crashes *less likely.* In particular, the main way we could have a serious bubble here is over-investment. Delays including power grid delays force bottlenecks on the investment level, and make that sort of problem less likely.

We may be in a bubble here. Certainly some elements look very bubble-like. In the late 1990s, during the dot-com boom, one had people making perfectly normal businesses but just tacking on "on the internet" to it and it would get millions of dollars of funding automatically. See for example pets.com . And we're seeing some of that now; we've had a whole bunch of businesses see their stock price shootup just because they mention AI. So we may be in a bubble, and it may crash now, or might crash never or decades from now. (Arguably the US had an 80 year automobile bubble that didn't crash until the late 2000s which then forced the auto bailout.) But the other thing to keep in mind is that a crash does not mean the tech goes away. In 1873, the US had a devastating collapse of the railroad bubble, and yet railroads continued to grow, with the amount of track not peaking until around 1910 https://transportgeography.org/contents/chapter5/rail-transportation-pipelines/rail-track-mileage-united-states/. Similarly, although the internet bubble burst, by 2015, one had more internet use than even the most internet hyping person in 1998 would have likely dreamed of. A bubble bursting does not mean the tech itself goes away.

Comment Re:What do you actually get from the newest consol (Score 1) 71

That sounds a bit circular there. The argument is to get the newest hardware to run the newest software, but the newest software is specifically written only to run on the newest hardware - actual benefits of the hardware be damned. Certainly Sony and Microsoft wouldn't be selling these newest consoles if there wasn't money to be made doing so (regardless of where the profit actually lies for them).

It's just a hugely diminishing return at this point from my perspective. 2nd generation consoles -> 3rd generation consoles was a tremendous leap. 3rd generation -> 4th generation was a tremendous leap as well. 4th generation -> 5th generation was a bit less of a leap than before. 5th generation -> 6th generation was still significant but still smaller than before. We're now in the 8th and 9th generation of consoles, and not only do many people find the 8th and 9th generation titles indistinguishable from each other, many people have a hard time seeing benefits of even the 9th generation over the 7th generation.

Obviously the whole bit still makes a ton of money. Obviously folks don't care what I think of it. I'm going to say it anyways. Maybe I'll go yell at a cloud after lunch.

Comment Re:I smell plateau (Score 4, Informative) 58

In my experience the new Frontier LLM models are in no way as much of an advance over their predecessors as their predecessors were.

I don't know what field you are in, but in my area, math, this is wildly not the case. They are much, much better. And the next line of the public frontier models are obviously even better judging from everything that we've seen drop from OpenAI and Anthropic. The large OpenAI release would by itself would have four or five results which would arguably be a Fields Medal if a human had done that, and quasi-RH which would if a human did it be almost definitely a Fields Medal result.

Slashdot Top Deals

Real wealth can only increase. -- R. Buckminster Fuller

Working...