Objectives should connect to business metrics. If the goal is customer satisfaction, evaluation metrics need to predict whether users find responses helpful.

Step 2: Create evaluation datasets

Good evaluation requires good data. The dataset represents what the model will actually face in production.

Three ways to build datasets:

• Manual curation: Write test cases by hand. Time-consuming but gives complete control over what gets tested. Works well for critical edge cases or when starting from scratch. Start with 20-50 high-quality examples covering main use cases.
• Production logs: Pull real user interactions. This captures actual usage patterns but requires user data and may contain sensitive information. Clean the data first - remove personal information, filter out spam, fix any mislabeled examples.
• Synthetic generation: Use an LLM to create test cases automatically. Fast and scalable. The synthetic model reads context (documentation, product info, sample queries) and generates questions, answers, or scenarios. Quality varies - synthetic data needs human review to catch nonsense or repetitive patterns.

Best practices for dataset quality:

• Get diversity. Cover different topics, question types, complexity levels, and edge cases. If building a support chatbot, include questions about products, returns, technical issues, and account problems.
• Include hard cases. Add examples of the current system that fail. These show where to improve and track progress over time.
• Make it representative. The test set should match production distribution. If 60% of real queries are about shipping, roughly 60% of test cases should be too.
• Keep it updated. As the product changes or new issues emerge, add examples. Evaluation datasets aren't static.
• Size matters but don't overdo it. Start with 100-200 test cases. That's enough to measure performance and spot patterns. Add more as needed.

Step 3: Select evaluation metrics

Pick 3-5 metrics maximum. More than that creates noise and makes it hard to tell what matters.

• Match metrics to objectives. If the goal is reducing hallucinations, track Faithfulness and Correctness. If users complain about irrelevant answers, measure Answer Relevancy.
• Consider the architecture. RAG systems need retrieval metrics (Contextual Relevancy, Precision, Recall) plus generation metrics. Simple question-answering only needs generation metrics.
• Think about tradeoffs. Some metrics conflict. Optimizing for completeness might hurt conciseness. Pick what matters most for the use case.
• Test metrics on sample data. Run them on a few dozen examples. Do scores match human judgment? If a metric gives high scores to obviously bad outputs, don't use it.

Metric selection criteria:

• Accuracy - Does it measure what matters?
• Reliability - Do results stay consistent?
• Cost - LLM-based metrics burn through API credits fast
• Speed - Deterministic metrics run instantly; LLM judges take seconds per evaluation
• Interpretability - Can the team understand what the score means?

Don't get fancy. Start with simple metrics. Add more complex ones only if simple metrics miss important quality issues.

Step 4: Choose evaluation tools and framework

Multiple open-source tools exist. Pick based on what the system needs.

Popular options:

• DeepEval - Python library for LLM evaluation. Supports G-Eval, RAG metrics, and custom metrics. Integrates with pytest for automated testing.
• RAGAS - Built specifically for RAG systems. Measures retrieval quality and generation quality. Has metrics for faithfulness, answer relevancy, and context precision.
• Promptfoo - Command-line tool for prompt testing. Compares different prompts and models. Good for experimentation.
• LangSmith - Part of LangChain ecosystem. Tracks all LLM calls, logs, and evaluation runs. Strong debugging features.

What to consider:

• Integration - Does it work with the current stack? Check compatibility with the LLM framework, vector database, and deployment platform.
• Features - Does it support needed metrics? Can it handle the specific use case (RAG, agents, simple completion)?
• Cost - Open-source is free but might need more setup. Hosted platforms charge but handle infrastructure.
• Learning curve - How fast can the team start using it? Complex tools take time to learn.
• Community - Active communities mean better documentation, more examples, and faster bug fixes.

Most teams start with one tool and add others as needs grow. Don't build everything custom unless there's a very specific requirement.

Step 5: Implement and automate

Manual evaluation doesn't scale, LLM evaluation frameworks require automation. Automate the process so it runs on every change.

Integration with CI/CD:

Add evaluation as a pipeline step. When code gets pushed, tests run automatically. If evaluation scores drop below thresholds, the pipeline fails. This catches regressions before they reach production.

Example workflow:

• Developer changes prompt or model
• CI pipeline triggers
• Evaluation runs on the test dataset
• Scores get compared to baselines
• Pass: deploy; Fail: block deployment

Set up monitoring:

Track metrics in production. Log every LLM interaction along with quality scores. This reveals performance drift over time.

Watch for patterns. If metrics drop on specific topics or during certain hours, investigate. Maybe new product launches confused the model. Maybe unusual traffic patterns exposed edge cases.

Establish baselines:

Run an evaluation on the current system. These scores become the baseline. Future changes should beat the baseline or at least not make things worse.

Track baselines over time. As the system improves, update baseline scores. What was acceptable last month might not be good enough now.

Setting pass/fail thresholds:

Thresholds determine what's acceptable. Too strict means nothing passes. Too loose means bad outputs slip through.

How to set thresholds:

Start with human judgment. Have people review 50-100 outputs and label them pass/fail. Look at metric scores for examples humans marked as passing. The 10th percentile of passing examples becomes the threshold.

For example, humans mark 100 outputs. Of the 60 they labeled "pass," the lowest Faithfulness score was 0.75. Set threshold at 0.75.

Adjust based on tolerance. High-stakes applications (medical, financial) need stricter thresholds. Lower-stakes apps can be more permissive.

Consider false positives vs false negatives. A strict threshold blocks good outputs (false negatives). A loose threshold lets bad outputs through (false positives). Pick based on which mistake costs more.

Test different thresholds. Run evaluation with various cutoffs. See how many outputs pass at 0.7 vs 0.8 vs 0.9. Pick the threshold that balances quality and coverage.

Example threshold settings:

Customer Support Chatbot: - Answer Relevancy: 0.8 (must be relevant) - Faithfulness: 0.85 (accuracy is critical) - Toxicity: 0.1 (very low tolerance) - Coherence: 0.7 (some flexibility)  Code Generator: - Correctness: 0.9 (code must work) - Tool Correctness: 0.95 (wrong API calls break things) - Coherence: 0.6 (messy but functional is okay)

Review thresholds quarterly. As models improve or requirements change, thresholds need adjusting.

Build incrementally. Start with a basic evaluation on a small dataset. Add automation once manual testing works. Expand the dataset as patterns emerge. This beats trying to build everything perfect from day one.

The framework is ready. Next step: picking the right tools for the job

LLM evaluation framework tools and comparison

Dozens of evaluation tools exist. Most overlap in basics, but excel at different things. Here's what teams actually use.

DeepEval works well for general testing with strong pytest integration. RAGAS fits the RAG-specific evaluation. Promptfoo handles quick prompt testing via CLI. LangSmith suits LangChain users needing full observability. TruLens and Phoenix focus on production monitoring. MLflow integrates with existing ML pipelines.

Most teams use multiple tools - DeepEval for testing, LangSmith for monitoring, Promptfoo for quick checks. Pick based on architecture (RAG vs agents), workflow (CI/CD vs manual), and existing stack. All open-source options are free to start.

Conclusion

LLM evaluation isn't optional. Models that work in demos fail with real users. Without testing, teams guess instead of knowing.

The LLM evaluation framework process is straightforward:

• Define what matters for the application
• Build a test dataset that represents real usage
• Pick 3-5 metrics that address the biggest risks
• Use tools that fit the workflow
• Automate tests in CI/CD

Run tests manually first. Add automation once it works. Expand the dataset as patterns emerge. Teams that evaluate well ship faster and break less.

To learn more about large language models, take this free course: Intro to Large Language Models (LLMs).

Frequently asked questions

1. What is the best LLM evaluation tool?

No single best tool. DeepEval works for general testing. RAGAS fits RAG pipelines. Promptfoo handles quick prompt checks. LangSmith suits LangChain users. Pick based on use case and workflow.

2. What are the best metrics for LLM evaluation?

Answer Relevancy, Faithfulness, and Correctness cover basics. RAG systems add Contextual Relevancy and Contextual Precision. Customer-facing apps need Toxicity checks. Use 3-5 metrics that address real risks.

3. What is the F1 score in LLM evaluation?

F1 balances precision and recall: 2 × (precision × recall) / (precision + recall). Works for classification tasks but doesn't capture semantic quality. Modern evaluation uses relevancy, faithfulness, and coherence metrics instead.

4. What are the tools for LLM testing?

DeepEval (testing), RAGAS (RAG), Promptfoo (prompts), LangSmith (observability), TruLens (monitoring), MLflow (experiments), Arize Phoenix (traces). Most are open-source.

5. What is the difference between LLM Model Evals vs. LLM System Evals?

Model evaluation tests base LLMs using benchmarks (MMLU, HellaSwag, TruthfulQA). System evaluation tests complete applications including prompts, retrieval, and tools. Model evals pick which LLM. System evals verify the whole thing works.

'The Codecademy Team, composed of experienced educators and tech experts, is dedicated to making tech skills accessible to all. We empower learners worldwide with expert-reviewed content that develops and enhances the technical skills needed to advance and succeed in their careers.'

• What is LLM evaluation?
• Essential metrics for LLM evaluation
• Building your LLM evaluation framework
• LLM evaluation framework tools and comparison
• Conclusion
• Frequently asked questions