Slashdot is powered by your submissions, so send in your scoop

 



Forgot your password?
typodupeerror
×

Comment Re:MapReduce = map + reduce (Score 2, Informative) 166

Almost, but not quite. MapReduce has a slightly different format than just map() and reduce(). Here is the signature of map and reduce from a theoretical functional language:

map(): A* -> B*
reduce(): B* -> C

Whereas in MapReduce:

map: (K, V)* -> (K1, V1)*
reduce: (K1, (V1)*)* -> (K2, V2)*

I think that is mostly accurate. Read more accurate/detailed report in MapReduce revisited[PDF].

Comment Re:MapReduce (Score 4, Informative) 166

The individual functions map and reduce are quite standard. The innovation here is the systems work they've done to make it work on such a large scale. All the programmer needs to worry about is implementing the two functions, they don't have to worry about distributing the work, ensuring fault tolerance, or anything else for that matter. That is the innovation.

They mention in the article that if you try and sort a petabyte you WILL get hard disk and computer failures. Hell, you can only read a terabyte hard disk a few times before you encounter unrecoverable errors. The system for executing those maps and reduces is what is important here. The important parts are in the design details, such as dealing with stragglers. If you have 4000 identical machines, you won't necessarily get equal performance. If a few of those machines have a bit flipped and started without disk cache, they might see a huge decrease in read/write performance. The system needs to recognize this and schedule the work differently. That can make a huge difference in execution time. If you graph the percentile complete of a MR job, you'll often see that it quickly reaches 95% and then plateaus. The last 5% may take 20% of the time, and good scheduling is required to bring this time down.

But like I said, the innovation isn't in the idea of using a Map and Reduce function, it is the system that executes the work.

Space

The Quietest Sun 227

Orbity sends in a Boston Globe report on the unusual calm on the surface of the sun. The photos, many taken in more active solar times, are excellent — see the sequence from last year of a coronal mass ejection carrying away the tail of a comet. "The Sun is now in the quietest phase of its 11-year activity cycle, the solar minimum — in fact, it has been unusually quiet this year — with over 200 days so far with no observed sunspots. The solar wind has also dropped to its lowest levels in 50 years. Scientists are unsure of the significance of this unusual calm..." As if to be contrary, New Scientist mentions that the number of sunspots seem to be increasing.
Education

University Tries "One iPhone Per Student" 281

alphadogg writes to tell us that one freshman class has a little more than usual to be excited about. When students at Abilene Christian University showed up for their first days of class they were greeted with the choice of either a new iPhone 3g or an iPod Touch plus a package of custom web apps to use on them. "The hardware is part of the Texas university's pilot mobile learning project, which has been gestating for over a year. About 650 first-year students chose the iPhone, and about 300 the iPod Touch, which is a very similar device but without the 3G radio (both devices incorporate an 802.11g Wi-Fi adapter). ACU pays for the hardware, student (or their parents) select and pay for their monthly AT&T service plan."
The Almighty Buck

Fuel Efficiency and Slow Driving? 1114

vile8 writes "With the high gas prices and ongoing gas gouging in my hometown many people are trying to find a reasonable way to save gas. One of the things I've noticed is people driving exceptionally slow, 30mph in 45mph zones, etc. So I had to take a quick look and find out if driving slow is helpful in getting better mileage. I know horsepower increases substantially with wind resistance, but with charts like this one from truckandbarter.com it appears mileage is actually about the same between 27mph and 58mph or so. So I'm curious what all the drivers out there with the cool efficiency computers are getting ... of specific interest would be the hemis with MDS; how do those do with the cylinder shutoff mode at different speeds?" Related: are there any practical hypermiling techniques that you've found for people not ready to purchase a new car, nor give up driving generally?
Businesses

Ars Examines Outlandish "Lost To Piracy" Claims and Figures 380

Nom du Keyboard writes "For years the figures of $200 billion and 750,000 jobs lost to intellectual property piracy have been bandied about, usually as a cudgel to demand ever more overbearing copyright laws with the intent of diminishing of both Fair Use and the Public Domain. Now ARS Technica takes a look into origin and validity these figures and finds far less than the proponents of them might wish."
Microsoft

Microsoft Adding jQuery To Visual Studio 67

Tim Anderson writes "Microsoft's Scott Guthrie, Corporate VP of the .NET developer division, announced that the open source jQuery Javascript library will be integrated into Visual Studio, the main Windows development tool. Further, Microsoft will treat jQuery as a supported product within technical support contracts, and will use jQuery to build new controls for ASP.NET, its web platform."
The Internet

Sending Excess Load To the Cloud? 153

TristanBrotherton writes "Cloud computing seems to be a good choice for startups like ours, looking to scale easily with users. (We're providing a series of Web services, assets, and Web applications to users of our mobile client.) There are the obvious choices of Google, Amazon, and smaller shops like EngineYard. The biggest issue we have in choosing cloud computing to run our applications is trust in their robustness. If the provider goes down, we suffer. In traditional hosting environments we mitigate this with multiple sites / vendors. It's not really feasible to host on multiple compute services, so I wondered if a better option might be to set up a small (perhaps two servers) origin infrastructure in a traditional manner at a datacenter, running our applications, but then send excess load, or in the event of our origin servers failing, all load, to compute services. This would give us the best of both worlds. Has anyone done this, or had experience in designing Web applications to scale seamlessly across both environments? Is there particular load-balancing hardware we can use to do this?"

Feds Unwrap $15M For Corporate Energy Reduction 146

As hard as it is to imagine, coondoggie writes with news that the federal government just unveiled a new energy bill that will offer $15 million in assistance to retailers who help to build and adopt energy-efficient technologies. "The US Department of Energy (DOE) announced the first phase of $15 million awards to retailers Best Buy, JCPenney, John Deere, Macy's, SuperValu, Target, Toyota, and Whole Foods Market. Commercial Real Estate Firms such as CB Richard Ellis, Forest City Enterprises as well as the financials groups also saw some of the money. Along with the money the companies will have access to the DOE's Pacific Northwest National Laboratory (PNNL) and National Renewable Energy Laboratory (NREL) to design, build, tune and operate at least one new prototype building and to retrofit an existing building project."
The Courts

State of Kentucky Seizes Control of 141 Domain Names 505

ashmodai9 writes "In a rather interesting (read: insane) decision, a district judge in the State of Kentucky has awarded control of 141 online gambling domain names to the governor of the state. Most of these are hosted offshore, and very few are registered under US domain name registrars, let alone registrars in the State of Kentucky (are there any?). You can check out the press release here, and confirm that the Commonwealth of Kentucky does in fact now 'own' these domain names by performing a WHOIS search on any of the domains listed here."
Censorship

YouTube Bans Gun and Knife Videos In the UK 632

PatPending writes with a depressing excerpt from the UK's Metro: "The Google-owned video-sharing site YouTube has decided to introduce the ban [on weapons-related videos] for the UK only amid widespread unease about the increase in knife crime in the country. 'We recognise that there has been particular concern over videos in the UK that involve showing weapons with the aim of intimidation, and this is one of the areas we are addressing,' a YouTube spokesperson said. 'I would like to see other internet service providers follow suit to reinforce our message that violence will not be tolerated either on the internet or in the real world,' she said."
Graphics

Twilight of the GPU — an Interview With Tim Sweeney 286

cecom writes to share that Tim Sweeney, co-founder of Epic Games and the main brain behind the Unreal engine, recently sat down at NVIDIA's NVISION con to share his thoughts on the rise and (what he says is) the impending fall of the GPU: "...a fall that he maintains will also sound the death knell for graphics APIs like Microsoft's DirectX and the venerable, SGI-authored OpenGL. Game engine writers will, Sweeney explains, be faced with a C compiler, a blank text editor, and a stifling array of possibilities for bending a new generation of general-purpose, data-parallel hardware toward the task of putting pixels on a screen."

Slashdot Top Deals

Do you suffer painful elimination? -- Don Knuth, "Structured Programming with Gotos"

Working...