Skip to content

examples.md

Latest commit

 

History

History
76 lines (53 loc) · 6.51 KB

examples.md

File metadata and controls

76 lines (53 loc) · 6.51 KB
Raw
description Code examples and tutorials.

{% hint style="info" %} You are looking at the old Evidently documentation: this API is available with versions 0.6.7 or lower. Check the newer version here. {% endhint %}

Quick Start

Check the short Quickstart examples here.

Get Started Tutorials

Introductory tutorials that walk you through the basic functionality step by step.

Title Guide Code
LLM Evaluation Tutorial Jupyter notebook
Data & ML Monitoring Tutorial Jupyter notebook
LLM Tracing Tutorial Jupyter notebook
Intro to Reports & Test Suites (OSS) Tutorial Jupyter notebook
Self-host ML monitoring Dashboard (OSS) Tutorial Jupyter notebook

Example Reports and Tests

Simple examples show different local evaluations (Metrics, Tests and Presets) for tabular data and ML.

Title Code example Contents
Evidently Test Presets Jupyter notebook Pre-built Test Suites on tabular data:
  • Data Drift
  • Data Stability
  • Data Quality
  • NoTargetPerformance
  • Regression
  • Classification (Multi-class, binary, binary top-K)
Evidently Tests Jupyter notebook
  • All individual Tests (50+) that one can use to create a custom Test Suite. Tabular data examples.
  • How to set test conditions and parameters.
Evidently Metric Presets Jupyter notebook All pre-built Reports:
  • Data Drift
  • Target Drift
  • Data Quality
  • Regression
  • Classification
Evidently Metrics Jupyter notebook
  • All individual metrics (30+) that one can use to create a custom Report.
  • How to set simple metric parameters.
Evidently LLM Metrics Jupyter notebook
  • Evaluations for Text Data and LLMs

For LLM and text metrics, check the LLM evaluation tutorial.

Tutorials - LLM

Title Tutorial
How to create LLM judge evaluator Tutorial
How to run regression testing for LLM products Tutorial

Tutorials - ML

To better understand the Evidently use cases, refer to the detailed tutorials accompanied by the blog posts.

Title Code example Blog post
Understand ML model decay in production (regression example) Jupyter notebook How to break a model in 20 days. A tutorial on production model analytics.
Compare two ML models before deployment (classification example) Jupyter notebook What Is Your Model Hiding? A Tutorial on Evaluating ML Models.
Evaluate and visualize historical data drift Jupyter notebook How to detect, evaluate and visualize historical drifts in the data.
Monitor NLP models in production Colab Monitoring NLP models in production: a tutorial on detecting drift in text data
Create ML model cards Jupyter notebook A simple way to create ML Model Cards in Python
Use descriptors to monitor text data Jupyter notebook Monitoring unstructured data for LLM and NLP with text descriptors

You can find more examples in the Community Examples repository.

How to examples

For code examples on specific functionality, check the How-To examples:

{% content-ref url="https://github.com/evidentlyai/evidently/tree/ad71e132d59ac3a84fce6cf27bd50b12b10d9137/examples/how_to_questions" %} How to guides {% endcontent-ref %}

Integrations

To see how to integrate Evidently in your prediction pipelines and use it with other tools, refer to the integrations.

{% content-ref url="../integrations/evidently-integrations.md" %} integrations/evidently-integrations.md {% endcontent-ref %}