Slashdot is powered by your submissions, so send in your scoop

 



Forgot your password?
typodupeerror
Check out the new SourceForge HTML5 internet speed test! No Flash necessary and runs on all devices. Also, Slashdot's Facebook page has a chat bot now. Message it for stories and more. ×

Comment Re:FP16 isn't even meant for computation (Score 1) 55

So, one problem is that there is not always more data. In my field, we have a surplus of some sorts of data, but other data requires hundreds of thousands of hours of human input, and we only have so much of that to go around. Processing all of that is easy enough, getting more is not.

Also, by "effective", I should have made it clear that I meant "an effective overall solution to the problem", which includes all costs of training a wider, lower-precision network. This includes input data collection, storage and processing, all of the custom software to handle this odd floating point format, including FP16-specific test code and documentation, run time server costs and latency, any increased risks introduced by using code paths in training and , etc.

I'm not saying that I don't believe it's possible, I've just seen absolutely no evidence that this is a significant win in most or even a sizable fraction of cases, or that it represents a "best practice" in the field. Our own experiments have shown a severe degradation in performance when using these nets w/out a complete retraining, the software engineering costs will be nontrivial, and much of the hardware we are forced to run on does not even support this functionality.

As an analog, when we use integer based nets and switch between 16-bit and 8-bit integers, we see an unacceptable level of degradation, even though there is a modest speedup and we can use slightly larger neural nets. I'm very wary of anything with a mantissa much smaller than 16 bits for that reason--those few bits seem to make a significant difference, at least for what we're doing. We're solving a very difficult constrained optimization problem using markov chains in real time, and if the observational features are lower fidelity, the optimization search will run out of time to explore the search space effectively before the result is returned to the rest of the system. It's possible that the sensitivity of our optimization algorithm to input quality is the issue here, not the fundamental usefulness of FP16, but I'm still quite skeptical. If this were a "slam dunk", I'd expect to see it move through the literature in a wave like the Restricted Boltzmann Machine did.

Oh, and thank you for the like (great reading) and the thoughtful reply. Not always easy to find on niche topics online.

Comment Re:Exploitative by design? (Score 1) 153

It seems like these systems are exploitative by design, even if exploitation wasn't explicitly the goal. They're designed with every possible algorithm and available data to maximize labor output at the lowest possible cost. Individual workers are operating at extreme information asymmetry and against a system which does not negotiate and only offers a take it or leave it choice.

This is by far the best comment I've ever seen regarding this sort of algorithmic labor management.

Normally I'm all for this sort of thing--my company is a client and uses it to handle large bursts of data processing quickly--but the information symmetry argument is a powerful one. Also, there doesn't seem to be a lot of competition in this space, which might otherwise ameliorate a lot of the problems induced by the "take it or leave it" bargaining approach.

The analysis provided by the article is absurd, but yours seems to lead to the inescapable conclusion that some kind of regulation is necessary to prevent blatant exploitation. Maybe just reducing information asymmetry in some way, or requiring transparency in reports available to the public on the website regarding effective wages paid to workers as a fraction of the minimum and average wages of employees in their respective countries. Surely someone can find an answer to this.

Comment Re:FP16 isn't even meant for computation (Score 1) 55

Accidentally posted as anonymous coward, reposting under my actual name.

So they're all excited about the lowest-precision, smallest-size floating point math in IEEE 754?

FP16 is good enough for neural nets. Do you really think the output voltage of a biological neurons has 32 bits of precision and range? For any given speed, FP16 allows you to run NNs that are wider and deeper, and/or to use bigger datasets That is way more important than the precision of individual operations.

There's a lot of rounding error with FP16. The neural networks I use are 16-bit integers, which work much, much better, at least for the work I'm doing. Also, do you have a good citation that FP16 neural networks are, overall, more effective than FP32 networks, as you've described?

Comment This isn't a victory for Behring-Breivik. (Score 3, Insightful) 491

Someone once pointed out that hoping a rapist gets raped in prison isn't a victory for his victim(s), because it somehow gives him what he had coming to him, but it's actually a victory for rape and violence. I wish I could remember who said that, because they are right. The score doesn't go Rapist: 1 World: 1. It goes Rape: 2.

What this man did is unspeakable, and he absolutely deserves to spend the rest of his life in prison. If he needs to be kept away from other prisoners as a safety issue, there are ways to do that without keeping him in solitary confinement, which has been shown conclusively to be profoundly cruel and harmful.

Putting him in solitary confinement, as a punitive measure, is not a victory for the good people in the world. It's a victory for inhumane treatment of human beings. This ruling is, in my opinion, very good and very strong for human rights, *precisely* because it was brought by such a despicable and horrible person. It affirms that all of us have basic human rights, even the absolute worst of us on this planet.

Comment Find a local amateur astronomy club (Score 4, Interesting) 234

The best place to start is by connecting with other people who share the same passions as you. I've similarly been fascinated by astronomy since I was a young child. A few years ago I rekindled my interest by buying a telescope and joining a club in Pittsburgh. I learned more from a few casual conversations with members than I had in months and months of reading and practicing on my own. Now I practice astrophotography and engage in lots of educational and community outreach events, and I owe just about everything to the club I joined. Best of luck, and clear skies!
User Journal

Journal Journal: in which i am a noob all over again 17

I haven't posted a journal here in almost three years, because I couldn't find the button to start a new entry. ...yeah, it turns out that it's at the bottom of the page.

So... hi, Slashdot. I used to be really active here, but now I mostly lurk and read. I've missed you.

Comment Re:Intelligent (Score 1) 138

That's certainly an option that will be looked at if they can't revive the probe in time, but it might be tricky. That would just about double the expected mission lifetime. It's tough to say that the mission-critical hardware (batteries, most importantly) will still be serviceable in 18 months. The rapid day/night cycles of earth orbit put a lot of strain on electrical systems from constantly switching power supplies.

Comment Re:Intelligent (Score 5, Informative) 138

The location/position of the probe has been known almost from the beginning. It was never that they couldn't find it, the problem was that the booster pack that was supposed to send the probe on to Mars never fired and the probe wasn't responding to the Russian's radio commands. What has changed in the last day is that receivers here on Earth are finally picking up radio signals from the probe itself, indicating that it is still alive and at least theoretically operational. Telemetry hasn't yet been received, but now there is a possibility we can communicate with it and try to diagnose the failures it suffered. As for if it can be recovered, I've not heard a definitive answer on this. One source will say the window has already closed, another says it's open until sometime in December. The window can probably be extended if they have enough fuel to try some exotic gravity assist with the Earth or Moon, but if it hasn't already passed it will soon.
Education

Quantum Physics For Everybody 145

fiziko writes in with a self-described "blatant self-promotion" of a worthwhile service for those wishing to go beyond Khan Academy physics: namely Bureau 42's Summer School. "As those who subscribe to the 'Sci-Fi News' slashbox may know, Bureau 42 has launched its first Summer School. This year we're doing a nine-part series (every Monday in July and August) taking readers from high school physics to graduate level physics, with no particular mathematical background required. Follow the link for part 1."
Java

Sun's Project Darkstar Game Server Platform No More 82

sproketboy writes "Project Darkstar, an open source software platform from Sun labs that simplifies the development of horizontally scalable servers for online games, is being discontinued as of the Oracle acquisition. This project, mentioned a couple of years back on Slashdot, was a unique concept for building an application server specific to on-line gaming. Sadly they were so close at version 0.9.11 (which is still very stable). Hopefully the open source community can get involved and help continue work on this project."

Comment Re:Wash your hands! (Score 1) 374

This is good advice, and gives me an opportunity to speak to the community at large: some of us who go to cons and are in a position to shake tons of hands politely decline. It's not because we're being dicks, it's because we know it's a good way to substantially decrease our chances of catching and spreading any germs.

Comment Oh, cruel irony (Score 2, Interesting) 374

I played the PAX Pandemic game, where the Enforcers handed out stickers to attendees that read [Carrier] [Infected] or [Immune] (There was also a [Patient Zero].

I got the [Immune] sticker, and by the time I got home on Monday, it was clear that I had the flu. I've had a fever between 100 and 104 all week that finally broke last night, but I'm going to the doctor today because I think whatever I had settled into my lungs. I'll tell him about the H1N1 outbreak and get tested if he wants to run the test, but at this point I think it's safe to assume that I was [Immune] to the Pig Plague, but definitely [Infected] with the damn PAX pox.

Even though it's been a week of misery, it was entirely worth it, and I don't regret going to PAX for a single second.

Slashdot Top Deals

"What I've done, of course, is total garbage." -- R. Willard, Pure Math 430a

Working...