BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//events//Events//EN
CALSCALE:GREGORIAN
X-WR-CALNAME:AI Book Club: Evals for AI Engineers
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-PUBLISHED-TTL:PT1H
BEGIN:VEVENT
UID:event-jjcncm97@ontown.app
DTSTAMP:20261110T093354Z
DTSTART:20270126T210000Z
SUMMARY:AI Book Club: Evals for AI Engineers
LOCATION:San Francisco\, United States
DESCRIPTION:AI Builders and Learners SF\nDecembers book is "Evals for AI En
 gineers"!\nThis is a casual-style event. Not a structured presentation on 
 topics. Sometimes\, the discussion even drifts away from the chapters\, bu
 t feel free to grab the mic to help steer it back.\n\nFeel free to join th
 e discussion even if you have not read the book chapters! :)\n​Want to d
 iscuss the contents during the reading week? Join the AI Slack group: http
 s://slack.flyte.org/\n​-------------------------------------------------
 \nAbout the book:\n\n• Title: Evals for AI Engineers\n• Authors: Shrey
 a Shankar\, Hamel Husain\n• Published: October 2026\n\nO'rielly platform
 : https://learning.oreilly.com/library/view/evals-for-ai/9798341660717/\nC
 hapters:\n\n1. Introduction\n2. LLMs and Evaluation Basics\n3. Error Analy
 sis\n4. Collaborative Evaluation Practices\n5. Implementing Automated Eval
 uators\n6. Evaluating Multi-Turn Conversations\n7. Evaluating Retrieval-Au
 gmented Generation\n8. Evaluating Tool Use and Complex Agents\n9. Continuo
 us Integration and Deployment for LLM Agents\n10. Interfaces for Human Rev
 iew\n11. Data Analysis for Traces\n12. Improving LLM Agents\n\nBook Descri
 ption\nStop using guesswork to find out how your AI applications are perfo
 rming. Evals for AI Engineers gives you a practical framework for systemat
 ically testing\, measuring\, and improving AI applications\, especially th
 ose built with LLMs. Written by Shreya Shankar and Hamel Husain\, AI engin
 eers with extensive experience across 35+ AI products and cutting-edge res
 earch\, this book focuses on the techniques teams actually need to build r
 eliable AI systems in production.\nIdeal for software engineers\, technica
 l product managers\, and technical leads\, this hands-on guide shows you h
 ow to build an evaluation process that improves alongside your product.\n\
 n• Find and prioritize failures through systematic error analysis and re
 presentative test sets\n• Build trustworthy evaluations using code based
  checks\, LLM judges\, human feedback\, and synthetic data\n• Improve pr
 oduction systems over time by monitoring regressions and optimizing qualit
 y\, latency\, and cost
URL:https://ontown.app/e/jjcncm97-ai-book-club-evals-for-ai-engineers/
END:VEVENT
END:VCALENDAR