👋 Welcome! Start Here
This repository teaches you everything about evaluating agentic AI systems through hands-on, practical examples.
🎯 What You’ll Learn
You’ll learn how to:
- Understand agentic AI systems
- Design comprehensive evaluation frameworks
- Measure agent performance accurately
- Test tool usage and reasoning
- Ensure agent safety and reliability
- Evaluate multi-agent systems
- Build automated evaluation pipelines
- Monitor agents in production
📚 How This Repository is Organized
Learning Structure
- Topics are numbered (01, 02, 03…) - work through them in order
- Each topic is self-contained - has its own code, docs, and examples
- Builds incrementally - each topic builds on previous concepts
Key Files
| File | Purpose |
|---|---|
HOW_TO_START.md | 👉 START HERE - Step-by-step guide to begin learning |
LEARNING_PATH.md | Overview of all topics and learning approach |
README.md | Repository overview and quick reference |
01_agentic_ai_fundamentals/ | Your first agent and evaluation |
02_evaluation_frameworks/ | How to evaluate systematically |
| … | More topics as you progress |
🚀 Your First Steps
1. Read the Start Guide
cat HOW_TO_START.md
This has everything you need to begin.
2. Understand the Fundamentals
cd 01_agentic_ai_fundamentals
cat README.md
Learn what agentic AI is and how it works.
3. Run Your First Example
cd 01_agentic_ai_fundamentals
pip install -r requirements.txt
python examples.py
4. Test It
# The examples will show you agents in action
📖 Learning Topics
- Agentic AI Fundamentals - What agents are
- Evaluation Frameworks - How to evaluate
- Metrics and Benchmarks - Measuring performance
- Tool Use Evaluation - Testing tools
- Reasoning Evaluation - Testing reasoning
- Safety Evaluation - Ensuring safety
- Multi-Agent Evaluation - Testing interactions
- Real-World Testing - Production evaluation
- Automated Evaluation - Building pipelines
- Benchmark Datasets - Standard datasets
- Evaluation Tools - Tools and frameworks
- Production Monitoring - Ongoing evaluation
💡 Learning Approach
- Read the documentation
- Study the code
- Run the examples
- Modify and experiment
- Move to the next topic
✅ Prerequisites
- Python 3.9+
- Basic Python knowledge
- Understanding of LLMs
- (Optional) LangChain or similar framework
🎓 Ready to Start?
👉 Open HOW_TO_START.md and follow the step-by-step guide!
Questions? Check the README.md in each topic directory for detailed explanations.
Stuck? Read error messages carefully, check the docs, and experiment with simpler examples first.
Let’s learn! 🚀