AI Engineer World's Fair 2025

Quick introduction to ComfyUI and what's new followed by a QA session.

• CA Comfy Anonymous Original creator of ComfyUI

Why is Reinforcement Learning (RL) suddenly everywhere, and is it truly effective? Have LLMs hit a plateau in terms of intelligence and capabilities, or is RL the breakthrough they need? In this workshop, we'll dive into the fundamentals of RL, what makes a good reward function, and how RL can help create agents. We'll also talk about kernels, are they still worth your time and what you should focus on. And finally, we’ll explore how LLMs like DeepSeek-R1 can be quantized down to 1.58-bits and still perform well, along with techniques to maintain accuracy.

• DH Daniel Han CEO

Learn the foundations of GraphRAG, starting with knowledge graph construction and then common retrieval patterns.

• ZB Zach Blumenfeld Graph Data Science & AI Specialist, Neo4j

Ever wished your webhooks could think for themselves? Join us to discover how A2A agents can transform passive webhook endpoints into intelligent workflow processors. In this session, we'll show you how to build a system that automatically spawns AI Agents to handle incoming webhooks. Using Google's Agent-to-Agent framework and MCP, you'll learn how to create dynamic AI agents that respond to events, communicate with external services, and make decisions based on content analysis. See the future of workflow automation where webhooks don't just trigger actions—they trigger intelligence!

• DM Damien Murphy Founding Engineer

Do you want to learn how to serve models like DeepSeek and Qwen with SOTA speeds on launch day? SGLang is an open-source fast serving framework for LLMs and VLMs that generates trillions of tokens per day at companies like xAI, AMD, and Meituan. This workshop guides AI engineers who are familiar with serving models using frameworks like vLLM, Ollama, and TensorRT-LLM through deploying and optimizing their first model with SGLang, as well as providing guidance on when SGLang is the appropriate tool for LLM workloads.

• PK Philip Kiely Head of Developer Relations
• YZ Yineng Zhang Inference lead at SGLang

Accuracy scores and leaderboard metrics look impressive—but production-grade AI requires evals that reflect real-world performance, reliability, and user happiness. Traditional benchmarks rarely help you understand how your LLM will perform when embedded in complex workflows or agentic systems. How can you realistically and adequately measure reasoning quality, agent consistency, MCP integration, and user-focused outcomes? In this practical, example-driven talk, we'll go beyond standard benchmarks and dive into tangible evaluation strategies using various open-source frameworks like GuideLLM and lm-eval-harness. You'll see concrete examples of how to create custom eval suites tailored to your use case, integrate human-in-the-loop feedback effectively, and implement agent reliability checks that reflect production conditions. Walk away with actionable insights and best practices for evaluating and improving your LLMs, ensuring they meet real-world expectations—not just leaderboard positions!

• TJ Taylor Jordan Smith Senior Developer Advocate

We’re all hearing that AI agents will enable AGI, but they can’t yet reliably perform even basic computer tasks. It turns out that getting AI to click, type, and scroll is more challenging than getting it to generate code. How can we build general-purpose agents that can do anything we can do on a computer? This is our goal at the Amazon AGI SF Lab. In this talk, I’ll propose a new approach to agents that we call Useful General Intelligence. After describing how we’re solving the biggest challenges in computer use while enabling developers to access our tech in it’s earliest developmental stages, I’ll show real workflows that developers have built with Nova Act, our agentic model and SDK.

• DP Danielle Perszyk Cognitive Scientist, PhD

Want to take advantage of your data, but don't want to reinvent RAG infrastructure? Join our workshop and see how you can deploy Agentic RAG in minutes using Contextual AI's managed RAG solution. We'll explore how Contextual handles intelligent parsing and chunking of your data, retrieves information with state of the art accuracy, and generates responses with a multi layered set of guardrails against hallucinations. Together, we'll build an end-to-end Agentic RAG pipeline and demonstrate its integration with Claude Desktop via MCP, so you can see how this could plug into your existing ecosystem. By the end of this session, you'll have a functioning Agentic RAG prototype that you can easily customize and deploy to production for your specific use cases, even with complex, unstructured documents.

• NL Nina Lopatina Lead Developer Advocate
• RS Rajiv Shah Chief Evangelist

A no fluff, all tactics discussion. More AI engineers should build startups, the world needs more software. But there’s a way to raise VC and it’s hard to do it if you’ve never seen it done. We are going to walk through the exact playbook to raise your first round of funding. We will show you real pitch decks, real cold emails and real term sheets so when you go out to raise your first round of funding, you are setup to do it. Every AI Engineer should be equip to start their own company and this session makes sure raising $$$ is not going to be the blocker.

• DG Dani Grant CEO
• CT Chelcie Taylor Investor

This hands-on workshop will guide participants through the complete AI evaluation lifecycle using Braintrust, from initial prompt testing to production monitoring. Attendees will learn to build evaluation frameworks that ensure their AI applications perform reliably in real-world scenarios. Topics covered include both offline and online evaluation strategies, logging and feedback systems, and human review processes.

• DG Doug Guthrie Solutions Engineer, Braintrust

This workshop explores how USDC, AI, and smart contracts can streamline escrow by automating fund release based on task or process verification. By using AI to interpret off-chain signals such as document validation, delivery confirmations, or milestone completion, we can trigger secure, programmable USDC payouts without manual intervention. The result is a faster, trust-minimized escrow system ideal for services, trade, and gig economy use cases.

• CC Corey Cooper Developer Relations Manager @ Circle

One of the biggest challenges in building evals you can trust is building metrics that reliably measure goodness in your application; metrics that are highly accurate, rapid fast, and tunable to ground truth rater and user behavior. This workshop is inspired by decades of AI and machine learning development in Google Search, reinvented for the modern LLM stack by the Pi team over the past year. In this workshop you will learn how to: 1) Brainstorm and design custom metrics tailored to your specific application needs. 2) Identify which types of signals (natural language, code, other models) work best for your use case through rapid trial and error. 3) Combine & calibrate your metrics against ground truth data using real examples from your domain. 4) Use simple tools like Google Sheets for visualizing and analyzing your inputs and outputs with those metrics. 5) Integrate your scoring models into both online workflows like agent control and offline ones like model comparison and training evaluation.

• DK David Karam CEO

Coding agents are transforming how software gets built, tested, and deployed, but engineering teams face a critical challenge: how to embrace this automation wave without sacrificing trust, control, or reliability. In this 80 minute workshop, you’ll go beyond toy demos and build production-minded AI agents using Dagger, the programmable delivery engine designed for real CI/CD and AI-native workflows. Whether you're debugging failures, triaging pull requests, generating tests, or shipping features, you'll learn how to orchestrate autonomous agents that live in and around your codebase: from your laptop to your CI platform. We’ll guide you through: Building real-world agents with Dagger and popular LLMs (GPT, Claude, etc.) Programming agent environments using real languages (Go, Python, TypeScript) Executing agent workflows locally and in GitHub Actions, so you can bring them to production Using a composable runtime that ensures isolation, determinism, traceability, and repeatability Designing agents that automate and enhance debugging, test generation, code review, bug fixing, and feature implementation By the end of the workshop, you’ll walk away ready to build your own army of autonomous agents, working collaboratively across your codebase, locally and in CI, accelerating development without ceding control. Let’s build agents that don’t just talk, they ship!

• KP Kyle Penfound Solutions Engineer at Dagger
• JA Jeremy Adams Head of Ecosystem

The agent capabilities added to GitHub Copilot have enhanced its ability to act as a peer programmer. Copilot can now discover and generate code based on existing standards, run tests, recover from errors, and call tools using Model Context Protocol (MCP). This workshop will guide you through piloting Copilot's agent capabilities and how to best integrate with the most widely adopted AI coding assistant in the world. Key takeaways include: - Understanding how and when to bring agents into your software development workflow - Providing context through the use of custom instructions and prompt files to ensure consistency across your team - Discovering how MCP provides access to an additional set of external tools and capabilities that the agent can use - Configuring Copilot's agentic capabilities to take advantage of your custom MCP server - Recommended best practices to help your responsibly accelerate your development while maintaining code quality and governance

• CH Christopher Harrison Senior Developer Advocate

In this hands-on workshop, you will build a multimodal AI agent capable of processing mixed-media content—from analyzing charts and diagrams to extracting insights from documents with embedded visuals. Using MongoDB as a vector database and memory store, and Google's Gemini for multimodal reasoning, you will gain hands-on experience with multimodal data processing pipelines and agent orchestration patterns by implementing core components directly, using good ol' Python.

• AJ Apoorva Joshi Senior AI Developer Advocate, MongoDB

Voice AI Agents are being deployed today in a wide range of business contexts. For example: - handling an increasing variety of call center tasks, - collecting patient data prior to healthcare appointments, - following up on inbound sales leads, - coordinating scheduling and logistics between companies, and - answering the phone for nearly every kind of small business. On the consumer side, conversational voice (and video) AI is also starting to make its way into social applications and games. And developers are sharing innovative personal voice AI projects and experiments every day on GitHub and social media. Building production-ready voice agents is complicated. Many elements are non-trivial to implement from scratch. This workshop will start with an overview of the voice AI landscape today. - The models, APIs, and infrastructure are used for Voice AI applications that are operating at production scale. - How to write voice agent code that achieves ultra low latency conversation and enterprise-quality reliability. - What new models and tools are coming in the second half of 2025. Then we will shift to a hands-on format: build and deploy a voice agent. Engineers from Google and Daily will help you get set up with a starter kit repo for your intended use case, then help you extend that code to create your own, customized Voice AI application.

• SB Shrestha Basu Mallick Product lead, Gemini Developer API
• KK Kwindla Kramer CEO

In this workshop you will learn how to build multilingual Conversational AI agents that can automatically detect your user's spoken language and can seamlessly switch to their preferred language.

• T雷 Thor 雷神 Schaeff DX at ElevenLabs

Don't be intimidated. Modern AI can feel like magic, but underneath the hood are principles that web developers can understand, even if you don't have a machine learning background. In this workshop, we'll explore a complete GPT-2 inference implementation built entirely in Vanilla JS. This JavaScript translation of the popular "Spreadsheets-are-all-you-need" approach will let you debug and step through a real LLM line by line without the overhead of learning a new language, framework, or even IDE. All the major LLMs, including ChatGPT, Claude, DeepSeek, and Llama, inherit from GPT-2's architecture, making this exploration a solid foundation to understand modern AI systems and comprehend the latest research. While we won't have time to cover *everything*, you'll gain the essential knowledge to understand the key concepts that matter when building with LLMs, including how they: -Convert raw text into meaningful tokens - Represent semantic meaning through vector embeddings - Train neural networks through gradient descent - Generate text with sampling algorithms like top-k, top-p, and temperature This intense but beginner-friendly workshop is designed specifically for web developers diving into ML and AI for the first time. It’s your "missing AI degree" in just two hours. You'll walk away with an intuitive mental model of how Transformers work that you can apply immediately to your own LLM-powered projects.

• IA Ishan Anand AI Consultant and educator

Hands on Workshop on learning to use Gemini 2.5 Pro in combination with Agentic tooling and MCP Servers.

• PS Philipp Schmid AI Developer Experience

Vector search is only a feature. Search engines and information retrieval have retaken their position as the foundation of RAG. This workshop takes you through decades of research, what has been working for a long time, and how it got better with Machine Learning.

• PK Philipp Krenn Code and conference monkey

Our hands-on workshop will walk you through how to build your own Mixture of Agents (MoA) system using the fastest, and most capable open models available: Qwen3-32B and Llama 3.3-70B. MoA is an emerging architecture that combines the strengths of multiple large language models in a layered, agent-based design. This approach delivers superior performance by enabling specialized agents to collaborate across layers—outperforming today’s frontier models in both accuracy and efficiency. To ground this new paradigm in its roots, we’ll also explore how Mixture of Experts (MoE) architectures continue to push the boundaries of scale and specialization. Learn how Cerebras trains state-of-the-art MoEs from Daria Soboleva, Head Research Scientist.

• DK Daniel Kim Head of Growth, Cerebras
• DS Daria Soboleva Head Research Scientist, Cerebras

Join us for a hands-on workshop designed to demonstrate how VS Code and GitHub Copilot's expanding suite of AI features can match or even surpasses the benefits of other popular AI developer tools. We'll focus on practical scenarios to ensure immediate applicability and work through live demos of Copilot features such as: Code generation using Edits, Planning/problem solving using Chat, Inline terminal command generation, Boilerplate code generation using Agent mode, Improving boilerplate with custom instructions and then refactoring using Agent mode and Edits, Improving test generation and code reviews with custom instructions, as well as an Introduction to MCP. Pre Requisit - install GitHub Copilot and VS Code

• HK Harald Kirschner VS Code Team Member
• CH Christopher Harrison Senior Developer Advocate

Covering all forms of fine-tuning and prompt engineering, like SFT, DPO, RFT, prompt engineering / optimization, and agent scaffolding.

• IB Ilan Bigio Developer Experience

In this 2-hour workshop, participants will gain practical hands-on experience building sophisticated AI agents using Amazon's agent technologies. You'll learn to build agents that can navigate the web like humans, perform complex multi-step tasks, and leverage specialized tools through natural language commands. You’ll explore Amazon Nova Act for reliable web navigation, Model Context Protocol (MCP) for connecting agents to external data sources and APIs, and Amazon Bedrock Agents for orchestrating complex workflows. Through guided exercises, you'll create agents capable of retrieving information and taking action across web applications, all through natural language interactions. By the end of this workshop, you'll have the practical skills to build AI agents that can browse websites, interact with web interfaces, and solve multi-step problems by combining these powerful Amazon technologies.

• DL Duan Lightfoot AWS, Sr. Cloud Networking Developer Advocate

Advanced GraphRAG techniques apply graph ML and algorithms, wrapped into tidy notebooks.

• AC Alison Cossette Data Science Strategist, Advocate, Educator
• AK Andreas Kollegger GenAI Lead

We've all seen website chat bots which can look up an order or answer a basic question -- but what does it take to build autonomous agents which manage long, delicate processes like multi-day medical treatments? In this workshop, we'll explore a workflow Stride built in partnership with Avila (https://avilascience.com/) that helps patients self-administer medication regimens at home. The stack includes LangGraph/LangSmith, Claude, MCP, Node.js, React, MongoDB, and Twilio, and rests on a foundation of treatment "blueprints" which LLM-powered agents use to guide patients to good outcomes. You'll learn how to: -Build a hybrid system of code and prompts that leverages LLM decisioning to drive a web application, message queue and database -Design and maintain flexible agentic workflow blueprints, with no special tools (just Google Docs!) -Create an agent evaluation system, which uses LLM-as-a-judge to evaluate the complexity of each interaction and escalate to human support when needed We'll also talk about the prompt engineered guidelines and guardrails which helps agents adhere to protocol as much as possible, while gracefully handling curveballs from the patient. Please bring questions -- we look forward to sharing our learnings on how to make agentic systems like this work in the real world!

• DM Dan Mason Principal, Head of AI

Learn about learnprompting.org and HackAPrompt, the first guide on prompt engineering in the world and the first competition on prompt injection respectively. I will give a workshop on prompt engineering and ai red-teaming, then we will get into some live prompt hacking. Bring your computer to participate, for the chance to win $100,000+: https://www.hackaprompt.com

• SS Sander Schulhoff CEO

This hands-on workshop introduces Mastra.ai, a TypeScript framework that streamlines the development of agentic AI systems compared to traditional approaches using LangChain and vector databases. Participants will learn to build structured AI workflows with composable tools and reliable control, enabling them to create internal AI assistants that can handle requests like data cleaning, email drafting, and document summarization with minimal code. The session covers Mastra installation, running a local MCP server, defining tools and agents in TypeScript, using the Mastra playground, and implementing practical examples such as RAG setups and tool-chaining agents—all designed to equip attendees with the skills to develop scalable AI-driven internal tools based on sound software engineering principles rather than just experimental prompts.

• NN Nick Nisi Software developer and panelist on the JS Party podcast
• ZP Zack Proser Open source hacker. Dev Education at WorkOS

Agentic coding marks a new era in software development, where AI agents take on autonomous roles in coding tasks. The Windsurf IDE embodies this shift by integrating intelligent agents like Cascade, which maintain full codebase context to perform multi-file edits, run terminal commands, and suggest changes through tools like Supercomplete and Flows. In this session, we will explore features that allow developers to guide strategy while the AI handles execution, enhancing productivity and enabling more creative, high-level work.

• NB Numair Baseer Deployed Engineer, Windsurf

We'll walk through the differences between chained and speech-to-speech powered voice agents, how to approach them, best practices and transform a text-based agent into our first voice-enabled agent

• DK Dominik Kundel Developer Experience

Join us for a hands-on workshop where you'll learn practical strategies to evaluate AI applications throughout their lifecycle—from initial testing of prompts to ongoing monitoring in production. We’re excited to host Sarah Sachs, AI Lead at Notion, who will share insights into how Notion built their acclaimed Notion AI.

• CE Carlos Esteban Solutions Engineer, Braintrust

Attendees will learn to use an AI coding agent as a fast and intuitive part of navigating and working with complex, production-grade legacy code bases. We will drop directly into the code–written in assembly–that landed the1969 Apollo 11 astronauts on the moon and, through a series of challenges, locate parts of the code tied to key functionality. Using the agent to convert a key guidance computer algorithm into a more modern programming language, attendees will then compete to see whose code has what it takes to land on the moon.

• FB Forrest Brazeal Speaker and Workshop Organizer
• MB Matt Ball Empowering Developers with AI

In this workshop we will explore the integration of Colpali, a cutting-edge Vision based Retrieval Model, with voice synthesis for next-generation RAG systems. We'll demonstrate how Colpali's ability to generate multi-vector embeddings directly from document images bypasses traditional OCR and complex preprocessing, while adding voice output creates a more intuitive and accessible user experience. Attendees will see how this combination handles documents with mixed textual and visual information, leading to more efficient and accurate information retrieval with natural voice responses.

• SD Suman Debnath Principal Developer Advocate, AI/ML, AWS

GenAI is reshaping the product landscape, creating huge opportunities (along with new expectations) for product managers. Yet while prompt engineering and model tuning get the spotlight, one critical skill can get overlooked: rigorous evaluation. This talk will help PMs move beyond gut-feel “vibe checks” to adopt concrete, repeatable evaluation strategies for LLM-powered products. I'll break down essential eval methodologies, from human feedback and code-based checks to cutting-edge LLM-based evaluations. Drawing on real-world examples, I'll share a practical framework PMs can use to: -Confidently evaluate AI-driven features - Ground decisions in real, repeatable data - Build trust and delight through consistent quality

• AK Aman Khan Director of Product