Slashdot stories can be listened to in audio form via an RSS feed, as read by our own robotic overlord.

 



Forgot your password?
typodupeerror

Slashdot videos: Now with more Slashdot!

  • View

  • Discuss

  • Share

We've improved Slashdot's video section; now you can view our video interviews, product close-ups and site visits with all the usual Slashdot options to comment, share, etc. No more walled garden! It's a work in progress -- we hope you'll check it out (Learn more about the recent updates).

×
Facebook

+ - Facebook open sources its Hadoop job scheduling enhancement, Corona

Submitted by stormspirit
stormspirit (2770655) writes "Today, Facebook open sources its scalable Map-Reduce job scheduling framework, Corona, as an enhancement for Hadoop. Facebook has been using Hadoop as MapReduce framework for years. According to the engineering notes, by early 2011, it started to hit the scalability wall of Hadoop. In particular, the job scheduling module became the bottleneck for large workload as there's only a single job tracker in Hadoop to both manage cluster resources and task trackers. Corona mainly divide the two responsibilities between a central resource manager and per task job tracker, allowing the module to scale. Other improvement includes: a pushed-based task assignment, dynamic resource management and better fairness guarantee. Corona is now deployed in all production systems with its design goals achieved."
This discussion was created for logged-in users only, but now has been archived. No new comments can be posted.

Facebook open sources its Hadoop job scheduling enhancement, Corona

Comments Filter:

IBM Advanced Systems Group -- a bunch of mindless jerks, who'll be first against the wall when the revolution comes... -- with regrets to D. Adams

Working...