I've used WhyLabs for a few weeks and I was extremely pleased with it!
I will evaluate some dimensions of the tool that summarize my experience with it.
Easy Data Ingestion:
The ingestion API is straightforward to use and supports multiple connectors such as BigQuery, Databricks, and Spark, making data importation easy. Whylabs' use of Data Profiling ensures fast and secure data processing, eliminating the need to upload entire datasets, and making all the process very secure, since your data doesn't leave your servers.
Reliable Data Features:
Whylabs delivers all standard feature metrics accurately. Tracking data and model drift is very straightforward using Monitors.
Also, the platform supports custom metrics creation during or after ingestion.
Grouping by variables (segments) works well but must be defined during ingestion. Then you can analyze dataset features and track model performance per segment.
Flexible Monitors:
The monitoring system in Whylabs is highly adaptable and user-friendly, covering multiple variables with ease.
Monitors are easy to set up via the UI or JSON import, with summarized notifications for each monitor, keeping users informed without overwhelming them.
Additionally, monitors are JSON serializable, which is very helpful since you can track them with version control.
User-Friendly Usability:
Whylabs have a clean and intuitive UI, simplifying navigation for users.
While some advanced features may require programming knowledge, most tasks can be accomplished within the UI.
Thanks to data profiling, Whylabs delivers speedy performance without compromising on accuracy.
Solid Documentation:
The documentation provided by Whylabs is comprehensive and easy to understand, enabling users to make the most of the platform.
Pricing:
It's simply cheaper than its competition while having top notch features.
Customer Support:
They are always very helpful, answering all our questions and having several calls showcasing us different uses cases directly on the platform.
Overall, Whylabs offers a straightforward, efficient and affordable solution for monitoring Machine Learning models, with easy data ingestion, reliable feature analysis, and flexible monitoring options. Review collected by and hosted on G2.com.
There are a few cons:
- Dashboards are in beta, and while functional, they lack polish in terms of user interface. They are working actively on this, so probably a few months after this review this may be already fixed.
- Defining groupings by variables must be done at ingestion time, limiting flexibility for post-ingestion analysis.
That being said, they are very open to feedback and they may change or add features based on your needs.
In our case, dashboards were important and they are working on them. Review collected by and hosted on G2.com.
Validated through a business email account
This reviewer was offered a nominal incentive as thanks for completing this review.
Invitation from G2 on behalf of a seller or affiliate. This reviewer was offered a nominal incentive as thanks for completing this review.
