stormspirit writes: Today, Facebook open sources its scalable Map-Reduce job scheduling framework, Corona, as an enhancement for Hadoop. Facebook has been using Hadoop as MapReduce framework for years. According to the engineering notes, by early 2011, it started to hit the scalability wall of Hadoop. In particular, the job scheduling module became the bottleneck for large workload as there's only a single job tracker in Hadoop to both manage cluster resources and task trackers. Corona mainly divide the two responsibilities between a central resource manager and per task job tracker, allowing the module to scale. Other improvement includes: a pushed-based task assignment, dynamic resource management and better fairness guarantee. Corona is now deployed in all production systems with its design goals achieved.
DEAL: For $25 - Add A Second Phone Number To Your Smartphone for life! Use promo code SLASHDOT25. Also, Slashdot's now on IFTTT. Check it out! Check out the new SourceForge HTML5 Internet speed test! ×