Follow Slashdot stories on Twitter

 



Forgot your password?
typodupeerror
Check out the new SourceForge HTML5 internet speed test! No Flash necessary and runs on all devices. ×

Comment Re:FP16 isn't even meant for computation (Score 1) 55

So, one problem is that there is not always more data. In my field, we have a surplus of some sorts of data, but other data requires hundreds of thousands of hours of human input, and we only have so much of that to go around. Processing all of that is easy enough, getting more is not.

Also, by "effective", I should have made it clear that I meant "an effective overall solution to the problem", which includes all costs of training a wider, lower-precision network. This includes input data collection, storage and processing, all of the custom software to handle this odd floating point format, including FP16-specific test code and documentation, run time server costs and latency, any increased risks introduced by using code paths in training and , etc.

I'm not saying that I don't believe it's possible, I've just seen absolutely no evidence that this is a significant win in most or even a sizable fraction of cases, or that it represents a "best practice" in the field. Our own experiments have shown a severe degradation in performance when using these nets w/out a complete retraining, the software engineering costs will be nontrivial, and much of the hardware we are forced to run on does not even support this functionality.

As an analog, when we use integer based nets and switch between 16-bit and 8-bit integers, we see an unacceptable level of degradation, even though there is a modest speedup and we can use slightly larger neural nets. I'm very wary of anything with a mantissa much smaller than 16 bits for that reason--those few bits seem to make a significant difference, at least for what we're doing. We're solving a very difficult constrained optimization problem using markov chains in real time, and if the observational features are lower fidelity, the optimization search will run out of time to explore the search space effectively before the result is returned to the rest of the system. It's possible that the sensitivity of our optimization algorithm to input quality is the issue here, not the fundamental usefulness of FP16, but I'm still quite skeptical. If this were a "slam dunk", I'd expect to see it move through the literature in a wave like the Restricted Boltzmann Machine did.

Oh, and thank you for the like (great reading) and the thoughtful reply. Not always easy to find on niche topics online.

Comment Re:Exploitative by design? (Score 1) 153

It seems like these systems are exploitative by design, even if exploitation wasn't explicitly the goal. They're designed with every possible algorithm and available data to maximize labor output at the lowest possible cost. Individual workers are operating at extreme information asymmetry and against a system which does not negotiate and only offers a take it or leave it choice.

This is by far the best comment I've ever seen regarding this sort of algorithmic labor management.

Normally I'm all for this sort of thing--my company is a client and uses it to handle large bursts of data processing quickly--but the information symmetry argument is a powerful one. Also, there doesn't seem to be a lot of competition in this space, which might otherwise ameliorate a lot of the problems induced by the "take it or leave it" bargaining approach.

The analysis provided by the article is absurd, but yours seems to lead to the inescapable conclusion that some kind of regulation is necessary to prevent blatant exploitation. Maybe just reducing information asymmetry in some way, or requiring transparency in reports available to the public on the website regarding effective wages paid to workers as a fraction of the minimum and average wages of employees in their respective countries. Surely someone can find an answer to this.

Comment Re:FP16 isn't even meant for computation (Score 1) 55

Accidentally posted as anonymous coward, reposting under my actual name.

So they're all excited about the lowest-precision, smallest-size floating point math in IEEE 754?

FP16 is good enough for neural nets. Do you really think the output voltage of a biological neurons has 32 bits of precision and range? For any given speed, FP16 allows you to run NNs that are wider and deeper, and/or to use bigger datasets That is way more important than the precision of individual operations.

There's a lot of rounding error with FP16. The neural networks I use are 16-bit integers, which work much, much better, at least for the work I'm doing. Also, do you have a good citation that FP16 neural networks are, overall, more effective than FP32 networks, as you've described?

Comment Re:extraordinary claims (Score 1) 332

Both you and the author are wrong. Books are processed by individual humans, who are supposed to understand and follow company policy but may or may not do so. The most plausible scenario is simply that the author hit an overly zealous censor who misinterpreted what they're supposed to be looking out for, and that censoring references to Amazon is not official Apple policy. FWIW, I've had issues much like this with Amazon's book publishing arm CreateSpace: some manuscripts were rejected by processor A for dubious reasons, and then accepted when the resubmission was processed by guy B instead.
Wikipedia

Submission + - Wikitravel dumps Internet Brands, Wikimedia to host fork (gyrovague.com)

jpatokal writes: "Once pipped to do to Lonely Planet what Wikipedia did to Encyclopedia Britannica, Creative Commons licensed travel guide Wikitravel has languished ever since it was bought by a used car company in 2006. But the community has finally had enough: the Wikimedia Foundation has agreed in principle to host a fork, and pretty much all active users are jumping ship. Why did they do it, will the fork succeed and what's Internet Brands going to do about it?"
Yahoo!

Submission + - Hackers post 450K credentials apparently pilfered from Yahoo (cnet.com)

x0d writes: "Yahoo appears to have been the victim of a security breach that yielded more than 400,000 login credentials stored in plain text.
  The hacked data, posted to hacker site D33D Company, contained more than 453,000 login credentials and appears to have originated from the Web pioneer's network. The hackers, who said they used a union-based SQL injection technique to penetrate the Yahoo subdomain, intended the data dump to be a 'wake-up call.'"

Books

Submission + - Why E-Books Will Soon Be Obsolete (gyrovague.com)

jpatokal writes: E-book digital rights management (DRM) gets a lot of press on Slashdot and rightly so, but the crippling brain damage of e-book formats and distribution extends much further than that. Charlie Stross recently predicted that e-readers like Kindle will be obsolete in 2-3 years, but will HTML5 and PDFs replace e-books themselves within five, and destroy the publishing industry as we know it in the process?
Microsoft

Submission + - Microsoft Makes $300M Investment In New Barnes & Noble Subsidiary

suraj.sun writes: Barnes & Noble has found a new, major partner in its fight to get an edge over Amazon and Apple in the market for e-books and the devices being used to consume them: it is teaming up with Microsoft in what the two are calling a strategic partnership, name yet to be determined. It will come in the form of a new subsidiary of B&N that will include all of its Nook business as well as its educational College business. Microsoft is making a $300 million investment in the subsidiary, valuing the company at $1.7 billion in exchange for around 17.6 percent equity in the subsidiary.

The new company, referred to for the moment as Newco, will contain B&N’s digital business, as well as its College division. While Microsoft will take 17.6 percent, B&N will own 82.4 percent of the venture. And there is a legal twist to the deal, too: the two companies say they have definitely sorted out their patent litigation now: “Moving forward, Barnes & Noble and Newco will have a royalty-bearing license under Microsoft’s patents for its NOOK eReader and Tablet products,” the two write in the release below.
Slashdot.org

Rob "CmdrTaco" Malda Resigns From Slashdot 1521

After 14 years and over 15,000 stories posted, it's finally time for me to say Good-Bye to Slashdot. I created this place with my best friends in a run down house while still in college. Since then it has grown to be read by more than a million people, and has served Billions and Billions of Pages (yes, in my head I hear the voice). During my tenure I have done my best to keep Slashdot firmly grounded in its origins, but now it's time for someone else to come aboard and find the *future*. Personally I don't have any plans, but if you need to get ahold of me for any reason, you can find me as @cmdrtaco on twitter or Rob Malda on Google+. You could also update my mail address to be malda at cmdrtaco dot net. Hit the link below if you want to read some nostalgic saccharine crap that I need to get out of my system before I sign off for the last time.
NASA

NASA Discovers 7th Closest Star 137

Thorfinn.au says "Scientists using data from NASA's Wide-field Infrared Survey Explorer (WISE) have discovered the coldest class of star-like bodies, with temperatures as cool as the human body. Astronomers hunted these dark orbs, termed Y dwarfs, for more than a decade without success. When viewed with a visible-light telescope, they are nearly impossible to see. WISE's infrared vision allowed the telescope to finally spot the faint glow of six Y dwarfs relatively close to our sun, within a distance of about 40 light-years. 'WISE scanned the entire sky for these and other objects, and was able to spot their feeble light with its highly sensitive infrared vision,' said Jon Morse, Astrophysics Division director at NASA Headquarters in Washington. 'They are 5,000 times brighter at the longer infrared wavelengths WISE observed from space than those observable from the ground.'"
Real Time Strategy (Games)

Sports Bars Changing Channels For Video Gamers 351

dtmos wrote in to say that "This summer, StarCraft II has become the newest bar room spectator sport. Fans organize so-called Barcraft events, taking over pubs and bistros from Honolulu to Florida and switching big-screen TV sets to Internet broadcasts of professional game matches. As they root for their on-screen superstars, StarCraft enthusiasts can sow confusion among regular patrons... But for sports-bar owners, StarCraft viewers represent a key new source of revenue from a demographic—self-described geeks—they hadn't attracted before."
Security

Researchers Report Spike In Boot Time Malware 132

wiredmikey writes "In their most recent intelligence report, Symantec researchers pointed out a massive increase in the amount of boot time malware striking users, noting there have already been as many new boot time malware threats detected in the first seven months of 2011 as there were in the previous three years. Also known as MBR (master boot record) threats, the malware infect an area of the hard disk that makes them one of the first things to be read and executed when a computer is turned on. This enables the threats to effectively dodge many security defenses."

Slashdot Top Deals

THEGODDESSOFTHENETHASTWISTINGFINGERSANDHERVOICEISLIKEAJAVELININTHENIGHTDUDE

Working...