1. [Home](https://www.g2.com/)
2. ...
3. [Test Management Tools](https://www.g2.com/categories/test-management)
4. [evalship Discussions](https://www.g2.com/products/evalship/discuss)

[
 ![Product Avatar Image](https://images.g2crowd.com/uploads/product/image/large_detail/large_detail_6098c9516fa381a6f9b2abaf4c2e37ce/evalship.png "Product Avatar Image")
](/products/evalship/reviews)

[

evalship

](/products/evalship/reviews)

0 ratings

evalship reveals the flaws in your LLM evals, then keeps watch on every pull request. Setup is one line for your coding agent: "Set up https://evalship.com/setup.md". The agent reviews every scorer, judge and prompt in your evals and finds their flaws, such as a judge that scores 1.0 when its call fails, or a privacy score too small to ever fail a reply. Each flaw comes with fix instructions. The agent then wires your evals into GitHub Actions and opens a setup pull request. On every pull request, evalship comments with the evals the change really broke and the likely cause, sets flaky evals aside by checking them against main's history, and flags prompt and LLM-call changes that no eval exercises. The evalship check can block a merge that breaks an eval. Your evals keep running in your own GitHub Actions with your own keys. evalship reads the results file from pytest, Vitest, Jest, promptfoo, Langfuse, Braintrust or LangSmith (JUnit XML, promptfoo JSON or evalship JSON), so there's no SDK to add, and it works next to the eval platform you already use. A dashboard per repo shows what needs attention and every eval's history, and a weekly report each Monday sums up the regressions caught and what your evals cost. Public repositories are free.

Show More

When users leave evalship reviews, G2 also collects common questions about the day-to-day use of evalship. These questions are then answered by our community of 850k professionals. Submit your question below and join in on the G2 Discussion.

* * *

### 0.0

Nps Score

### All evalship Discussions

Search

Most CommentedMost HelpfulCommon QuestionsNewest

All DiscussionsDiscussions with CommentsCommon QuestionsDiscussions without Comments

FilterFilter

Filter byExpand/Collapse 

Sort by

Most Commented

Most Helpful

Common Questions

Newest

Filter by

All Discussions

Discussions with Comments

Common Questions

Discussions without Comments

Sorry...

There are no questions about evalship yet.

## Start a New Software Discussion

Have a software question?

Get answers from real users and experts

[Start A Discussion](/products/evalship/discussions/new?entry_point=discussions%23product)

* * *

 ![Product Avatar Image](https://images.g2crowd.com/uploads/product/image/thumb_square/thumb_square_6098c9516fa381a6f9b2abaf4c2e37ce/evalship.png "Product Avatar Image")

### Have you used evalship before?

Answer a few questions to help the evalship community

[
Yes
](javascript:void(0))[
Yes
](https://www.g2.com/login?context=product_review&return_to=https%3A%2F%2Fwww.g2.com%2Fproducts%2Fevalship%2Fdiscuss%3Fsmall_ask%3Devalship)
No