Forgot your password?
typodupeerror

Comment Re: Ok, I give up (Score 3, Informative) 34

Flink and Spark address similar use cases and have similar APIs, but the technology under the hood differs a bit. Flink serializes data to memory and works a lot on binary representations. This also makes (partial) spilling to disk easier if the system runs out of memory. Flink's processing engine uses pipelined processing (like many DBMS) which allows for true streaming and batch processing in a single engine. Iterations are build-in operators in Flink which means that data continuously flows in cycles. A special iteration operator reduces the number of computations as iteration count increases which gives very good performance for certain use cases such as some ML algorithms.

Comment Re:Ok, I give up (Score 3, Informative) 34

Flink is a parallel data processing engine similar to Hadoop and Spark with some unique features: 1) combines realtime stream and batch processing, 2) features an DBMS-style optimizer, 3) in-memory processing which goes gracefully to disk if memory is scarce, 4) provides special operators for iterative processing, ... Check out http://flink.apache.org/ for details.

Submission + - The Apache Software Foundation's newest Top-Level Project: Apache Flink (sdtimes.com)

An anonymous reader writes: "Flink is an open-source Big Data system that fuses processing and analysis of both batch and streaming data. The data-processing engine, which offers APIs in Java and Scala as well as specialized APIs for graph processing, is presented as an alternative to Hadoop’s MapReduce component with its own runtime. Yet the system still provides access to Hadoop’s distributed file system and YARN resource manager.

The open-source community around Flink has steadily grown since the project’s inception at the Technical University of Berlin in 2009. Now at version 0.7.0, Flink lists more than 70 contributors and sponsors, including representatives from Hortonworks, Spotify and Data Artisans (a German startup devoted primarily to the development of Flink)."

Slashdot Top Deals

"Religion is something left over from the infancy of our intelligence, it will fade away as we adopt reason and science as our guidelines." -- Bertrand Russell

Working...