Redpanda Integrations

StarTree Cloud is a fully-managed real-time analytics platform designed for OLAP at massive speed and scale for user-facing applications. Powered by Apache Pinot, StarTree Cloud provides enterprise-grade reliability and advanced capabilities such as tiered storage, scalable upserts, plus additional indexes and connectors. It integrates seamlessly with transactional databases and event streaming platforms, ingesting data at millions of events per second and indexing it for lightning-fast query responses. StarTree Cloud is available on your favorite public cloud or for private SaaS deployment. StarTree Cloud includes StarTree Data Manager, which allows you to ingest data from both real-time sources such as Amazon Kinesis, Apache Kafka, Apache Pulsar, or Redpanda, as well as batch data sources such as data warehouses like Snowflake, Delta Lake or Google BigQuery, or object stores like Amazon S3, Apache Flink, Apache Hadoop, or Apache Spark. StarTree ThirdEye is an add-on anomaly detection system running on top of StarTree Cloud that observes your business-critical metrics, alerting you and allowing you to perform root-cause analysis — all in real-time.

Kubernetes (K8s), an open-source software that automates deployment, scaling and management of containerized apps, is available as an open-source project. It organizes containers that make up an app into logical units, which makes it easy to manage and discover. Kubernetes is based on 15 years of Google's experience in running production workloads. It also incorporates best-of-breed practices and ideas from the community. Kubernetes is built on the same principles that allow Google to run billions upon billions of containers per week. It can scale without increasing your operations team. Kubernetes flexibility allows you to deliver applications consistently and efficiently, no matter how complex they are, whether you're testing locally or working in a global enterprise. Kubernetes is an open-source project that allows you to use hybrid, on-premises, and public cloud infrastructures. This allows you to move workloads where they are most important.

Amazon Simple Storage Service (Amazon S3), an object storage service, offers industry-leading scalability and data availability, security, performance, and scalability. Customers of all sizes and industries can use Amazon S3 to store and protect any amount data for a variety of purposes, including data lakes, websites and mobile applications, backup, restore, archive, enterprise apps, big data analytics, and IoT devices. Amazon S3 offers easy-to-use management tools that allow you to organize your data and set up access controls that are tailored to your business, organizational, or compliance needs. Amazon S3 is built for 99.999999999% (11 9,'s) of durability and stores data for millions applications for companies around the globe. You can scale your storage resources to meet changing demands without having to invest upfront or go through resource procurement cycles. Amazon S3 is designed to last 99.999999999% (11 9,'s) of data endurance.

All sizes of companies can use object storage. You can store any amount of data. You can retrieve it as often and as often as you like. You can configure your data with Object Lifecycle Management to automatically transition to lower cost storage classes when it meets certain criteria, such as when it reaches a certain date or when you have stored a newer version. Cloud Storage offers a growing number of storage locations that you can store your data, with multiple redundancy options. You can choose where and how to store your data, regardless of whether you want to optimize for a split-second response or create a robust disaster recovery strategy. Two highly efficient online routes to Cloud Storage are Storage Transfer Service and Transfer Service. Both offer the speed and scalability you need to make data transfers faster. Our Transfer Appliance is a shippable storage device that can be used for offline data transfer.

Apache Kafka®, is an open-source distributed streaming platform.

AWS offers a wide range of services, including database storage, compute power, content delivery, and other functionality. This allows you to build complex applications with greater flexibility, scalability, and reliability. Amazon Web Services (AWS), the world's largest and most widely used cloud platform, offers over 175 fully featured services from more than 150 data centers worldwide. AWS is used by millions of customers, including the fastest-growing startups, large enterprises, and top government agencies, to reduce costs, be more agile, and innovate faster. AWS offers more services and features than any other cloud provider, including infrastructure technologies such as storage and databases, and emerging technologies such as machine learning, artificial intelligence, data lakes, analytics, and the Internet of Things. It is now easier, cheaper, and faster to move your existing apps to the cloud.

Docker eliminates repetitive, tedious configuration tasks and is used throughout development lifecycle for easy, portable, desktop, and cloud application development. Docker's complete end-to-end platform, which includes UIs CLIs, APIs, and security, is designed to work together throughout the entire application delivery cycle. Docker images can be used to quickly create your own applications on Windows or Mac. Create your multi-container application using Docker Compose. Docker can be integrated with your favorite tools in your development pipeline. Docker is compatible with all development tools, including GitHub, CircleCI, and VS Code. To run applications in any environment, package them as portable containers images. Use Docker Trusted Content to get Docker Official Images, images from Docker Verified Publishings, and more.

Google Cloud is an online service that lets you create everything from simple websites to complex apps for businesses of any size. Customers who are new to the system will receive $300 in credits for testing, deploying, and running workloads. Customers can use up to 25+ products free of charge. Use Google's core data analytics and machine learning. All enterprises can use it. It is secure and fully featured. Use big data to build better products and find answers faster. You can grow from prototypes to production and even to planet-scale without worrying about reliability, capacity or performance. Virtual machines with proven performance/price advantages, to a fully-managed app development platform. High performance, scalable, resilient object storage and databases. Google's private fibre network offers the latest software-defined networking solutions. Fully managed data warehousing and data exploration, Hadoop/Spark and messaging.

RisingWave is an open-source distributed SQL streaming database released under Apache 2.0 license. RisingWave is PostgreSQL-compatible, and allows users to process streaming data using standard SQL. Written in Rust and designed with cloud-native architecture, RisingWave can achieve 10X better performance and cost efficiency compared to conventional stream processing systems. RisingWave Cloud is a fully managed cloud service. Users can leverage RisingWave Cloud to process streaming data and serve analytical queries at ease.

Open-source monitoring solutions are able to power your alerting and metrics. Prometheus stores all data in time series. These are streams of timestamped value belonging to the same metric with the same labeled dimensions. Prometheus can also generate temporary derived times series as a result of queries. Prometheus offers a functional query language called PromQL, which allows the user to select and aggregate time series data real-time. The expression result can be displayed as a graph or tabular data in Prometheus’s expression browser. External systems can also consume the HTTP API. Prometheus can be configured using command-line flags or a configuration file. The command-line flags can be used to configure immutable system parameters such as storage locations and the amount of data to be kept on disk and in memory. . Download: https://sourceforge.net/projects/prometheus.mirror/

DeltaStream is an integrated serverless streaming processing platform that integrates seamlessly with streaming storage services. Imagine it as a compute layer on top your streaming storage. It offers streaming databases and streaming analytics along with other features to provide an integrated platform for managing, processing, securing and sharing streaming data. DeltaStream has a SQL-based interface that allows you to easily create stream processing apps such as streaming pipelines. It uses Apache Flink, a pluggable stream processing engine. DeltaStream is much more than a query-processing layer on top Kafka or Kinesis. It brings relational databases concepts to the world of data streaming, including namespacing, role-based access control, and enables you to securely access and process your streaming data, regardless of where it is stored.

The most comprehensive network observability solution available for modern data platforms. Provides unprecedented insights at any size. ElastiFlow enables organizations to achieve unprecedented levels in network performance, availability and security. ElastiFlow gives detailed information about network traffic, including IP addresses, ports and protocols, as well as the amount of data sent. This information allows network administrators gain a deeper understanding of the network's performance, and identify potential problems. ElastiFlow can be used to diagnose and troubleshoot network issues, such as congestion, packet loss, or high latency. Administrators can identify the root cause of a problem by analyzing network traffic and taking appropriate action. ElastiFlow allows organizations to improve their security posture and detect and respond more effectively to threats, while maintaining compliance with regulatory requirements.

Scalable Python framework designed to build real-time intelligent applications, data pipelines, and integrate AI/ML models

PeerDB is the fastest, easiest, and most cost-effective solution to replicate data between Postgres and data warehouses, queues and storage. Designed to scale at any size and tailored for data stores. PeerDB replays the schema messages using replication messages from Postgres replication slots. Alerts for slot expansion and connections. Native support for Postgres toast columns and large JSONB for IoT. Optimized query designs to reduce warehouse costs. Especially useful for Snowflake or BigQuery. Support for partitioned table via both publish. Transaction snapshotting and CTID scanning provide a consistent and fast initial load. High-availability with autoscaling, advanced logs, metrics, dashboards and monitoring dashboards.

PuppyGraph allows you to query multiple data stores in a single graph model. Graph databases can be expensive, require months of setup, and require a dedicated team. Traditional graph databases struggle to handle data beyond 100GB and can take hours to run queries with multiple hops. A separate graph database complicates architecture with fragile ETLs, and increases your total cost ownership (TCO). Connect to any data source, anywhere. Cross-cloud and cross region graph analytics. No ETLs are required, nor is data replication. PuppyGraph allows you to query data as a graph directly from your data lakes and warehouses. This eliminates the need for time-consuming ETL processes that are required with a traditional graph databases setup. No more data delays or failed ETL processes. PuppyGraph eliminates graph scaling issues by separating computation from storage.

Timeplus is an easy-to-use, powerful and cost-effective platform for stream processing. All in one binary, easily deployable anywhere. We help data teams in organizations of any size and industry process streaming data and historical data quickly, intuitively and efficiently. Lightweight, one binary, no dependencies. Streaming analytics and historical functionality from end-to-end. 1/10 of the cost of comparable open source frameworks Transform real-time data from the market and transactions into real-time insight. Monitor financial data using append-only streams or key-value streams. Implement real-time feature pipelines using Timeplus. All infrastructure logs, metrics and traces are consolidated on one platform. In Timeplus we support a variety of data sources through our web console UI. You can also push data using REST API or create external streams, without copying data to Timeplus.

The only fully-managed cloud data lakehouse that can ingest data from all of your sources in minutes, and support all of your query engines on a large scale. All for a fraction the cost. With the ease of fully managed pipelines, you can ingest data from databases and event streams in near-real-time. You can query your data using any engine and support all of your use cases, including BI, AI/ML, real-time analytics and AI/ML. Simple usage-based pricing allows you to cut your costs by up to 50% compared with cloud data warehouses and ETL software. With a fully-managed, highly optimized cloud service, you can deploy in minutes and without any engineering overhead. Unify all your data into a single source and eliminate the need for data to be copied between data lakes and warehouses. Apache Hudi, Apache Iceberg and Delta Lake all offer omnidirectional interoperability, allowing you to choose the best table format for your needs. Configure managed pipelines quickly for database CDC and stream ingestion.

ZooKeeper is a central service that maintains configuration information, naming, distributed synchronization and provides group services. Distributed applications use all of these services in some way. There is a lot of work involved in fixing bugs and other race conditions. These kinds of services are difficult to implement so applications often skimp on them at the beginning. This makes them fragile and difficult to manage in the face of change. Even if the services are implemented correctly, there can be multiple implementations that lead to complexity in management when the applications are deployed.

Waterstream transforms your Kafka-compatible platform to a fully-fledged MQTT brokerage. No code, no integration pipelines and no additional storage required to connect millions of clients to your data streaming platform. Waterstream creates a bidirectional layer that connects Kafka to MQTT clients. You can forget about managing external MQTT clusters and integration pipelines to code. Waterstream scales linearly. Its nodes are independent of each other for most operations. To support a growing number of clients, you can add more instances. Waterstream only requires Kafka in order to function. All the inherent persistence benefits of Kafka include high availability, high throughput and low latency.

Scale from 0 to millions of events every second. Arroyo is shipped as a single compact binary. Run locally on MacOS, Linux or Kubernetes for development and deploy to production using Docker or Kubernetes. Arroyo is an entirely new stream processing engine that was built from the ground-up to make real time easier than batch. Arroyo has been designed so that anyone with SQL knowledge can build reliable, efficient and correct streaming pipelines. Data scientists and engineers are able to build real-time dashboards, models, and applications from end-to-end without the need for a separate streaming expert team. SQL allows you to transform, filter, aggregate and join data streams with results that are sub-second. Your streaming pipelines should not page someone because Kubernetes rescheduled your pods. Arroyo can run in a modern, elastic cloud environment, from simple container runtimes such as Fargate, to large, distributed deployments using the Kubernetes logo.

Shift testing to one side and identify issues earlier when they are simpler and more cost-effective. Developers can do better integration testing, reduce dev cycles, and increase productivity. More reliable products will be possible with shorter and more thorough integration feedback cycles. Testcontainers Cloud allows developers to run reliable integration testing, with real dependencies defined and coded, from their laptops to the CI of their team. Testcontainers is an Open-Source framework that provides lightweight, throwaway instances of databases, message brokers and web browsers. It can also run just about any other application that can be run in a Docker Container. No need to create mocks or configure complex environments. Simply define your test dependencies in code and run your tests. Containers will be created and deleted.