Average Ratings 0 Ratings

Total
ease
features
design
support

No User Reviews. Be the first to provide a review:

Write a Review

Average Ratings 0 Ratings

Total
ease
features
design
support

No User Reviews. Be the first to provide a review:

Write a Review

Description

Airflow is a community-driven platform designed for the programmatic creation, scheduling, and monitoring of workflows. With its modular architecture, Airflow employs a message queue to manage an unlimited number of workers, making it highly scalable. The system is capable of handling complex operations through its ability to define pipelines using Python, facilitating dynamic pipeline generation. This flexibility enables developers to write code that can create pipelines on the fly. Users can easily create custom operators and expand existing libraries, tailoring the abstraction level to meet their specific needs. The pipelines in Airflow are both concise and clear, with built-in parametrization supported by the robust Jinja templating engine. Eliminate the need for complex command-line operations or obscure XML configurations! Instead, leverage standard Python functionalities to construct workflows, incorporating date-time formats for scheduling and utilizing loops for the dynamic generation of tasks. This approach ensures that you retain complete freedom and adaptability when designing your workflows, allowing you to efficiently respond to changing requirements. Additionally, Airflow's user-friendly interface empowers teams to collaboratively refine and optimize their workflow processes.

Description

You determine the cluster size, node specifications, and a range of services, while Yandex Data Proc effortlessly sets up and configures Spark, Hadoop clusters, and additional components. Collaboration is enhanced through the use of Zeppelin notebooks and various web applications via a user interface proxy. You maintain complete control over your cluster with root access for every virtual machine. Moreover, you can install your own software and libraries on active clusters without needing to restart them. Yandex Data Proc employs instance groups to automatically adjust computing resources of compute subclusters in response to CPU usage metrics. Additionally, Data Proc facilitates the creation of managed Hive clusters, which helps minimize the risk of failures and data loss due to metadata issues. This service streamlines the process of constructing ETL pipelines and developing models, as well as managing other iterative operations. Furthermore, the Data Proc operator is natively integrated into Apache Airflow, allowing for seamless orchestration of data workflows. This means that users can leverage the full potential of their data processing capabilities with minimal overhead and maximum efficiency.

API Access

Has API No 

API Access

Has API Yes 

Screenshots View All

Screenshots View All

Integrations

Apache Druid Yes 
Apache Hive No 
Apache Zeppelin No 
DQOps Yes 
DataHub Yes 
Decube Yes 
Determined AI Yes 
DoubleCloud Yes 
Google Cloud Managed Service for Apache Airflow Yes 
LocalStack Yes 
Microsoft Purview Yes 
Netdata Yes 
Ray Yes 
Sifflet Yes 
Stackable Yes 
Stonebranch Yes 
Telmai Yes 
Yandex Data Proc Yes 
ZenML Yes 
intermix.io Yes 

Integrations

Apache Druid No 
Apache Hive Yes 
Apache Zeppelin Yes 
DQOps No 
DataHub No 
Decube No 
Determined AI No 
DoubleCloud No 
Google Cloud Managed Service for Apache Airflow No 
LocalStack No 
Microsoft Purview No 
Netdata No 
Ray No 
Sifflet No 
Stackable No 
Stonebranch No 
Telmai No 
Yandex Data Proc No 
ZenML No 
intermix.io No 

Pricing Details

No price information available.
Free Trial No 
Free Version No 

Pricing Details

$0.19 per hour
Free Trial Yes 
Free Version No 

Deployment

Web-Based Yes 
On-Premises No 
iPhone App No 
iPad App No 
Android App No 
Windows No 
Mac No 
Linux No 
Chromebook No 

Deployment

Web-Based Yes 
On-Premises No 
iPhone App No 
iPad App No 
Android App No 
Windows No 
Mac No 
Linux No 
Chromebook No 

Customer Support

Business Hours No 
Live Rep (24/7) No 
Online Support No 

Customer Support

Business Hours No 
Live Rep (24/7) Yes 
Online Support Yes 

Types of Training

Training Docs No 
Webinars No 
Live Training (Online) No 
In Person No 

Types of Training

Training Docs Yes 
Webinars Yes 
Live Training (Online) Yes 
In Person No 

Vendor Details

Company Name

The Apache Software Foundation

Founded

1999

Country

United States

Website

airflow.apache.org

Vendor Details

Company Name

Yandex

Founded

1997

Country

Russia

Website

cloud.yandex.com/en/services/data-proc

Product Features

Workflow Management

Access Controls/Permissions No 
Approval Process Control No 
Business Process Automation No 
Calendar Management No 
Compliance Tracking No 
Configurable Workflow No 
Customizable Dashboard No 
Document Management No 
Forms Management No 
Graphical Workflow Editor No 
Mobile Access No 
No-Code No 
Task Management No 
Third Party Integrations No 
Workflow Configuration No 

Product Features

Alternatives

Alternatives

JAMS Scheduler Reviews

JAMS Scheduler

JAMS Software
Amazon MWAA Reviews

Amazon MWAA

Amazon
esProc Reviews

esProc

Raqsoft
dbt Reviews

dbt

dbt Labs
CereVoice Me Reviews

CereVoice Me

CereProc