Microsoft To Open Source A Big Data Framework Called REEF

Summary: Microsoft has developed a big data technology that sits on top of Hadoop’s new YARN resource manager. Called REEF, it’s designed to let users build jobs that can maintain state even after they’re done, and that can grab data from wherever they need it.

Microsoft has developed a big data framework called REEF (a graciously simple acronym for Retainable Evaluator Execution Framework) that the company intends to open source in about a month. REEF is designed to run on top of YARN, the next-generation resource manager for Hadoop, and is particularly well suited for building machine learning jobs.

Microsoft Technical Fellow and CTO of Information Services Raghu Ramakrishnan explained REEF and Microsoft’s plans to open source it during a Monday morning keynote at the ACM Knowledge Discovery and Data Mining conference, taking place in Chicago.