Average Ratings 0 Ratings
Average Ratings 0 Ratings
Description
Deequ is an innovative library that extends Apache Spark to create "unit tests for data," aiming to assess the quality of extensive datasets. We welcome any feedback and contributions from users. The library requires Java 8 for operation. It is important to note that Deequ version 2.x is compatible exclusively with Spark 3.1, and the two are interdependent. For those using earlier versions of Spark, the Deequ 1.x version should be utilized, which is maintained in the legacy-spark-3.0 branch. Additionally, we offer legacy releases that work with Apache Spark versions ranging from 2.2.x to 3.0.x. The Spark releases 2.2.x and 2.3.x are built on Scala 2.11, while the 2.4.x, 3.0.x, and 3.1.x releases require Scala 2.12. The primary goal of Deequ is to perform "unit-testing" on data to identify potential issues early on, ensuring that errors are caught before the data reaches consuming systems or machine learning models. In the sections that follow, we will provide a simple example to demonstrate the fundamental functionalities of our library, highlighting its ease of use and effectiveness in maintaining data integrity.
Description
Scala seamlessly integrates both object-oriented and functional programming paradigms into a single, elegant high-level language. With its static type system, Scala minimizes the likelihood of errors in intricate applications, while its compatibility with JVM and JavaScript allows developers to create efficient systems that can leverage extensive libraries. The Scala compiler is adept in managing static types, meaning that in most instances, you don't need to specify variable types; its robust type inference handles this automatically. Structural data types in Scala are represented by case classes, which automatically provide well-defined methods for toString, equals, and hashCode, in addition to enabling deconstruction through pattern matching. Moreover, in Scala, functions are treated as first-class citizens, allowing for the creation of anonymous functions using a streamlined syntax. This versatility makes Scala an appealing choice for developers seeking a language that combines the best of both programming worlds.
API Access
Has API
No
API Access
Has API
Yes
Integrations
Ably
No
Apache NetBeans
No
Apache PredictionIO
No
BLACKBOX AI
No
Gemini 1.5 Pro
No
Gemini 2.0 Flash
No
Gemini 2.5 Flash
No
Gemini 2.5 Pro Preview (I/O Edition)
No
Gemini Code Assist
No
JetBrains Datalore
No
Integrations
Ably
Yes
Apache NetBeans
Yes
Apache PredictionIO
Yes
BLACKBOX AI
Yes
Gemini 1.5 Pro
Yes
Gemini 2.0 Flash
Yes
Gemini 2.5 Flash
Yes
Gemini 2.5 Pro Preview (I/O Edition)
Yes
Gemini Code Assist
Yes
JetBrains Datalore
Yes
Pricing Details
No price information available.
Free Trial
No
Free Version
No
Pricing Details
Free
Free Trial
No
Free Version
Yes
Deployment
Web-Based
Yes
On-Premises
No
iPhone App
No
iPad App
No
Android App
No
Windows
No
Mac
No
Linux
No
Chromebook
No
Deployment
Web-Based
Yes
On-Premises
Yes
iPhone App
No
iPad App
No
Android App
No
Windows
Yes
Mac
Yes
Linux
Yes
Chromebook
Yes
Customer Support
Business Hours
No
Live Rep (24/7)
No
Online Support
Yes
Customer Support
Business Hours
No
Live Rep (24/7)
No
Online Support
Yes
Types of Training
Training Docs
Yes
Webinars
No
Live Training (Online)
No
In Person
No
Types of Training
Training Docs
Yes
Webinars
No
Live Training (Online)
No
In Person
No
Vendor Details
Company Name
Deequ
Website
github.com/awslabs/deequ
Vendor Details
Company Name
Scala
Website
www.scala-lang.org