Average Ratings 0 Ratings

Total 0 out of 5 stars
ease 0 out of 5 stars
features 0 out of 5 stars
design 0 out of 5 stars
support 0 out of 5 stars

No User Reviews. Be the first to provide a review:

Write a Review

Average Ratings 0 Ratings

Total 0 out of 5 stars
ease 0 out of 5 stars
features 0 out of 5 stars
design 0 out of 5 stars
support 0 out of 5 stars

No User Reviews. Be the first to provide a review:

Write a Review

Description

evalship is quality control for LLM evals, delivered as a GitHub App. Setup is one line for a coding agent such as Claude Code or Codex: the agent reads your evals, finds the test cases that can't fail, and opens a pull request that wires them into CI. From then on, each pull request gets a comment naming the evals the change broke, the likely cause in the diff, the flaky evals left out, and any new AI call without an eval. When you want better results or lower bills, Optimize has your agent run experiments on your prompts and models, and evalship tells you which variants beat the current one beyond chance. You pick what ships. evalship sits next to Langfuse, Braintrust, LangSmith or promptfoo rather than replacing them, and your evals keep running on your own infrastructure.

Description

Pytest is an invaluable tool for enhancing your programming skills, as it simplifies the creation of both basic tests and complicated functional tests for various applications and libraries. The framework’s ability to provide detailed assertion introspection means you can rely solely on standard assert statements for all your testing needs. It offers thorough information regarding failed assertions, automatically identifies test modules and functions, and features modular fixtures that help manage both small and parameterized long-lived test resources effectively. Additionally, pytest can seamlessly execute unittest (including trial) and nose test suites, and it is compatible with Python versions 3.6 and above, as well as PyPy 3. Its rich plugin architecture boasts over 315 external plugins and is backed by a vibrant community of users. Furthermore, the maintainers of pytest, along with thousands of other packages, have partnered with Tidelift to provide commercial support and maintenance for the open-source dependencies integral to your projects. By leveraging pytest, you can save valuable time, minimize risks, and enhance the overall health of your codebase, all while ensuring that the developers of the specific dependencies you rely on are compensated for their work. This commitment to community and support truly sets pytest apart as a leader in the testing framework landscape.

API Access

Has API No 

API Access

Has API Yes 

Screenshots View All

Screenshots View All

Integrations

Braintrust Yes 
Captain No 
Codecov No 
Jest Yes 
Katalon Recorder No 
Katalon True Platform No 
LangSmith Yes 
Langfuse Yes 
Opik No 
Pynt No 
Qualflare No 
Roost.ai No 
TestQuality No 
VIKTOR No 
Vitest Yes 
Zebrunner No 
evalship No 
promptfoo Yes 
pytest Yes 
pytest-cov No 

Integrations

Braintrust No 
Captain Yes 
Codecov Yes 
Jest No 
Katalon Recorder Yes 
Katalon True Platform Yes 
LangSmith No 
Langfuse No 
Opik Yes 
Pynt Yes 
Qualflare Yes 
Roost.ai Yes 
TestQuality Yes 
VIKTOR Yes 
Vitest No 
Zebrunner Yes 
evalship Yes 
promptfoo No 
pytest No 
pytest-cov Yes 

Pricing Details

$19/month
Free for all public repositories, plus 1 private repository with 1 user. Team is $19/month (3 users, 2 private repositories), Pro $49/month (10 users, 5 private repositories) and Business $199/month (50 users, 25 private repositories). Yearly billing gets 2 months free. Paid plans come with a 7-day free trial.
Free Trial Yes 
Free Version Yes 

Pricing Details

No price information available.
Free Trial Yes 
Free Version No 

Deployment

Web-Based Yes 
On-Premises No 
iPhone App No 
iPad App No 
Android App No 
Windows No 
Mac No 
Linux No 
Chromebook No 

Deployment

Web-Based Yes 
On-Premises No 
iPhone App No 
iPad App No 
Android App No 
Windows No 
Mac No 
Linux No 
Chromebook No 

Customer Support

Business Hours No 
Live Rep (24/7) No 
Online Support Yes 

Customer Support

Business Hours No 
Live Rep (24/7) No 
Online Support Yes 

Types of Training

Training Docs Yes 
Webinars No 
Live Training (Online) No 
In Person No 

Types of Training

Training Docs Yes 
Webinars No 
Live Training (Online) No 
In Person No 

Vendor Details

Company Name

Garm Tech B.V.

Founded

2026

Country

Netherlands

Website

evalship.com

Vendor Details

Company Name

pytest

Founded

2004

Website

docs.pytest.org/en/6.2.x/

Product Features

Product Features

Functional Testing

Automated Testing No 
Interface Testing No 
Regression Testing No 
Reporting / Analytics No 
Sanity Testing No 
Smoke Testing No 
System Testing No 
Unit Testing No 

Software Testing

Automated Testing No 
Black-Box Testing No 
Dynamic Testing No 
Issue Tracking No 
Manual Testing No 
Quality Assurance Planning No 
Reporting / Analytics No 
Static Testing No 
Test Case Management No 
Variable Testing Methods No 
White-Box Testing No 

Alternatives

Langfuse Reviews

Langfuse

5.0 out of 5 stars

Alternatives

DeepEval Reviews

DeepEval

Confident AI
Galileo Reviews

Galileo

Cisco