Forgot your password?
typodupeerror
Facebook

+ - Facebook open sources its Hadoop job scheduling enhancement, Corona

Submitted by stormspirit
stormspirit (2770655) writes "Today, Facebook open sources its scalable Map-Reduce job scheduling framework, Corona, as an enhancement for Hadoop. Facebook has been using Hadoop as MapReduce framework for years. According to the engineering notes, by early 2011, it started to hit the scalability wall of Hadoop. In particular, the job scheduling module became the bottleneck for large workload as there's only a single job tracker in Hadoop to both manage cluster resources and task trackers. Corona mainly divide the two responsibilities between a central resource manager and per task job tracker, allowing the module to scale. Other improvement includes: a pushed-based task assignment, dynamic resource management and better fairness guarantee. Corona is now deployed in all production systems with its design goals achieved."
This discussion was created for logged-in users only, but now has been archived. No new comments can be posted.

Facebook open sources its Hadoop job scheduling enhancement, Corona

Comments Filter:

"A mind is a terrible thing to have leaking out your ears." -- The League of Sadistic Telepaths

Working...