Please create an account to participate in the Slashdot moderation system

 



Forgot your password?
typodupeerror
×
Cloud

Ask Slashdot: Scientific Computing Workflow For the Cloud? 80

diab0lic writes "I have recently come into the situation where I need to run cloud computing on demand for my research. Amazon's EC2 Spot Instances are an ideal platform for this as I can requisition an appropriate instance for the given experiment {high cpu, high memory, GPU instance} depending on its needs. However I currently spin up the instance manually, set it up, run the experiment, and then terminate manually. This gets tedious monitoring experiments for completion, and I incur unnecessary costs if a job finishes while I'm sleeping, for example. The whole thing really should be automated. I'm looking for a workflow somewhat similar to this:
  1. Manually create Amazon machine image (AMI) for experiment.
  2. Issue command to start AMI on specified spot instance type.
  3. Automatically connect EBS to instance for result storage.
  4. Automatically run specified experiment, bonus if this can be parameterized.
  5. Have AMI automatically terminate itself upon experiment completion.

Something like docker that spun up on-demand spot instances of a specified type for each run and terminated said instance at run completion would be absolutely perfect. I also know HTCondor can back onto EC2 spot instances but I haven't really been able to find any concise information on how to set up a personal cloud — I also think this is slight overkill. Do any other Slashdot users have similar problems? How did you solve it? What is your workflow? Thanks!"

Comment Re:Not Surprising for HP (Score 1) 57

That. And everything else.

American cars are ugly, spend too much fuel, can't withstand bad roads, are too slow and steer too badly to be driven fast as well, and their quality sucks in general. That's why GM has a whole design/engineering team in Germany, to sell cars to the rest of the world.

Too bad that when the japanese started taking over the american market, their cars started to suffer from the same effects: too big, too shabby, too slow, too soft.

Comment Re:Barrier? (Score 1) 96

It's just speech. It's a milestone. It's not difficult to exceed one exaflops (the name stands for operations per second, it's not a plural) once you got to, say, 0.99 exaflops. Scientists like to talk in orders of magnitude. Right now we are in the tens of petaflops, but didn't get yet to hundreds. Tiahne-2 gets to 55 pflops, but its sustained speed is a bit bigger than half of that.

Problem is much more about how to get there. It's not just machinery. Is how to actually write and debug programs at that scale. As we cannot make the cores much faster than what we have today, the solution is to add more cores.

The added cores increase the stress on the network, and makes programming such thing much more difficult. Good luck debugging a race condition on one million processes.

Other problems arise from things as mundane as equipment breaking. Think that if you have a single broken memory chip during the execution of a program, the whole computation is either compromised or just lost. And with millions of cores, comes millions of motherboards, power supplies, I/O system, storage, all kinds of electronic components which are subject to problems.

So, while technically, it's not a barrier per se, this huge number of variables that makes things exponentially more complex than what we have today is indeed a barrier. As someone asked here, we cannot just make a cluster of tianhe-2s. The thing would be breaking all the time, spending so much electricity and manpower for maintenance that its uptime would be smaller than a windows 98 unpatched machine connected to an open network.

Comment Re:Information != benefit (Score 2) 96

Huge supercomputers have the advantage that they are efficient, when compared to projects such as those running "@home", and their interconnects allows them to solve problems that need strong communications between the computing elements. Such problems cannot be solved in an efficient way by this "@home" model, where a machine receives a work unit, computes it and returns the result for final aggregation.

Those interconnects can sum to as much as half the price of building a supercomputer.

When you mention the environmental rising costs, I suspect you mean the carbon footprint, caused by energy consumption for manufacturing and operating those machines. The costs are not negligible, granted, but they are probably not as big as that caused by the cars of the thousands of scientists who use such machines :-) This is especially true in US, where cars are horribly inefficient, public transport from the suburbs to research centers is spotty and distances are large.

I understand that these environmental costs are much smaller than the benefits given by the use of such machines. Remember that supercomputers are used to simulate things such as nuclear explosions, ballistics and radiation decay. The costs for the environment are certainly better than blowing atomic bombs around! Not to mention the gains in health research, for example.

So, yes, there is a HUGE demand for such behemoths, and they are much better than the alternative.

Comment Re:Interesting data, but doesn't support conclusio (Score 1) 173

you know why people don't even bother anymore to try buying stuff anymore?

Because the "pirate" download is easier, sometimes faster, and, especially in the case of videos, earlier and better (meaning without all these traillers, fbi screens on the beginning etc).

Besides, the industry only moved their slow butts for two reasons: this piracy they speak of and apple. If they could, they would be selling nicki minaj crap por 30 dollars each song.

Privacy

The Wall That Knows If You're a Criminal 119

Barence writes "A German company called Dermalog is showing off a wall-sized transparent display that can tell a person's age, mood and criminal intent simply by scanning their face. The system displays data about the user next to their face, and is a demonstration of a fraud-prevention system that matches criminal intent to certain characteristics. PC Pro's tester wasn't overly impressed. 'If the face was a good enough indicator of mood then it should have tagged me as "freaked out on business technological ennui," not simply "happy", and no police force would accept a description of someone as "aged between 45 and 75 — that's the gap between Daniel Craig and Jack Nicholson.'"

Comment Aerogarden (Score 2) 44

For this price, I can buy an aerogarden, which comes with the water deposit, water pumps, lights and even comes with some seeds inside an optimized growing medium and fertilizer, besides being an aeroponics system, with a lot of advantages over growing stuff on dirt.

Even for those pot growers this is not a good idea.

Slashdot Top Deals

One man's constant is another man's variable. -- A.J. Perlis

Working...