Contents Menu Expand Light mode Dark mode Auto light/dark, in light mode Auto light/dark, in dark mode
⚠️ Legacy Documentation Notice
This legacy documentation is no longer updated. Visit our current documentation for the latest GenAI features.
Light Logo Dark Logo
Light Logo Dark Logo

Getting Started

  • Why Giskard?
  • Quickstart
    • 📚 LLM Quickstart
    • 📊 Tabular Quickstart
    • 🗣️ NLP Quickstart
    • 📸 Vision Quickstart

Guides

  • 📥 Install the Giskard Python Library
  • 🤖 Setting up the LLM Client
  • 🔍 Scan a model
    • 📚 LLM scan
    • 📊 Tabular model scan
    • 🗣️ NLP model scan
    • 📸 Vision model scan
    • Advanced scan usage
  • 🧰 RAG Evaluation Toolkit
    • 🎯 RAGET Testset Generation
    • 🥇 RAGET Evaluation
  • 🧪 Customize your tests
    • 👨‍🔬 Create tests
    • 🔪 Create data slices
    • 🔄 Create data transformations
  • 🔁 Integrate your tests
    • 🚀 Execute your test suite in your CI/CD pipeline
    • 🏃 MLflow
      • MLflow Example - LLM - Databricks
      • MLFlow Example - Tabular
    • 🐝 Weights & Biases
      • W&B Example - LLM
      • W&B Example - Tabular
    • 🧪 Pytest
      • Example script

Notebook Tutorials

  • LLM Tutorials
    • LLM Question Answering over the IPCC Climate Change Report
    • LLM Question Answering with Langchain, Qdrant and OpenAI
    • LLM Question Answering over the 2022 Winter Olympics Wikipedia articles
    • LLM product description from keywords
    • LLM Newspaper Comments Generation with LangChain and OpenAI
    • LLM Question Answering over the documentation with Langchain, FAISS and OpenAI
  • RAG Tutorials
    • RAG Evaluation Toolkit on an IPCC Climate Agent
    • RAG Evaluation Toolkit on a Banking Supervisory Process Agent
  • Tabular Tutorials
    • 📊 Tabular Quickstart
    • Breast cancer detection [XGBoost]
    • Customer churn prediction [LGBM]
    • German credit scoring [scikit-learn]
    • Drug classification [scikit-learn]
    • IEEE Fraud detection adversarial validation [LGBM]
    • Insurance charges prediction [LGBM]
    • M5 Sales prediction [LGBM]
    • Wage classification [scikit-learn]
  • NLP Tutorials
    • Twitter sentiment analysis using RoBERTa model [HuggingFace]
    • Airline tweets sentiment analysis [HuggingFace]
    • Amazon reviews classification [scikit-learn]
    • ENRON email classification [scikit-learn]
    • Fake/real news classification [tensorflow (keras)]
    • Regression on the hotel reviews [scikit-learn]
    • Medical transcript classification [scikit-learn]
    • Movie Review Sentiment Classification with DISTILL-BERT [scikit-learn + torch preprocessing]
    • Newspaper classification [PyTorch]
    • Tripadvisor reviews sentiment classification [HuggingFace]
  • Vision Tutorials
    • Face landmark detection
    • Object detection

Knowledge

  • LLM Vulnerabilities
    • Hallucination and Misinformation
    • Harmful Content Generation
    • Prompt Injection
    • Robustness
    • Output Formatting
    • Information Disclosure
    • Stereotypes and Discrimination
  • How does the LLM Scan work?
  • ML Model Vulnerabilities
    • Performance Bias
    • Unrobustness
    • Overconfidence
    • Underconfidence
    • Unethical behaviour
    • Data Leakage
    • Stochasticity
    • Spurious correlation
  • Catalogs
    • Tests
      • Classification tests
      • Regression tests
      • Text generation tests
    • Slicing functions
    • Transformation functions

Integrations

  • 🐙️ GitHub
    • 🚀 Execute your test suite in your CI/CD pipeline
  • 🏃 MLflow
    • MLflow Example - LLM - Databricks
    • MLFlow Example - Tabular
  • 🟩 NeMo Guardrails
    • NeMo Guardrails + Giskard scan
  • 🐝 Weights & Biases
    • W&B Example - LLM
    • W&B Example - Tabular
  • 🐶 DagsHub
  • 🤗 Hugging Face
    • 🔍Giskard Evaluator
    • 📤 Push a QATestset to the Hugging Face Hub
  • 📒 AVID
    • Reporting Giskard LLM Scans to AVID
  • 🧪 Pytest
    • Example script

API Reference

  • Models
    • Base model classes
    • Catboost models
    • Prediction function
    • HuggingFace models
    • Langchain models
    • Pytorch models
    • Sklearn models
    • Tensorflow models
  • Dataset
  • Model Scanner
    • Scan Report
    • Tabular & NLP Detectors
    • Detectors for LLM models
  • RAG Evaluation Toolkit
    • Testset Generation
    • Knowledge Base
    • Evaluation
    • Question Generation
    • Available Metric functions
  • Tests
    • Metamorphic tests
    • Statistical tests
    • Performance tests
    • Drift tests
    • LLM tests
    • Data quality tests
    • Stability tests
  • Slicing functions
  • Transformation functions
  • Test suite

Community

  • Discord community
  • GitHub community
  • Contribute to Giskard
    • How to configure local development environment
    • Giskard architecture
    • Configuration
Back to top
Edit this page

RAG Evaluation Toolkit¶

  • Testset Generation
    • generate_testset
    • QATestset
  • Knowledge Base
    • KnowledgeBase
    • Document
  • Evaluation
    • evaluate
    • CorrectnessEvaluator
  • Question Generation
    • SimpleQuestionsGenerator
    • ComplexQuestionsGenerator
    • DistractingQuestionsGenerator
    • SituationalQuestionsGenerator
    • DoubleQuestionsGenerator
    • ConversationalQuestionsGenerator
    • OutOfScopeGenerator
  • Available Metric functions
    • Correctness
    • RAGAS Metrics
    • Base Metric
Next
Testset Generation
Previous
Detectors for LLM models
Copyright © 2024, Giskard AI
Made with Sphinx and @pradyunsg's Furo