Laurie Voss
An AI developer relations expert who currently leads developer relations at Arise.ai and has taught evaluation methodologies for agentic applications in a collaborative course with deeplearning.ai.
Timeline (last 90 days)
According to Laurie Voss, head of developer relations at Arise.ai, in a deeplearning.ai collaborative course, the failure modes of Agentic applications (such as fabricating information, calling the wrong tools, skipping steps) simply do not exist in traditional software development.
This course by Laurie Voss was launched in collaboration with deeplearning.ai, takes the form of hands-on labs, and covers the complete chain from trace analysis, test case construction, and evaluator and judge implementation, to making improvement decisions based on evaluation.
The improvement loop Voss describes is: use evaluation results to decide what to prioritize improving, and then after making changes use the same set of evaluations to confirm that the Agent has indeed gotten better.
Voss specifically mentioned that you also need to check whether the LLM judge itself is reliable; if the judge's judgments are unstable or biased, the improvement decisions made based on it are built on shifting sand.
Voss's methodological starting point is: do not design test sets out of thin air, but start from real failures found in Agent traces and turn failure cases into reusable test cases.
In the course, Laurie Voss clearly pointed out that building an evaluation system is not a nice-to-have add-on but a basic skill that AI engineers must master.
Ryan Carson argues that work is becoming "agent manager", and Laurie Voss talks about the rise of the product engineer.
Evaluations with Phoenix 101, hosted by Laurie Voss, will start from evaluation fundamentals in Turing Hall.
This observation by Laurie Voss is in line with the themes of generative AI and agent engineering that Simon Willison has long focused on.
In the article 'We are all Product Engineers now,' Laurie Voss argues that the cost of writing code has collapsed, and the cost of reviewing, fixing, and operating code is following suit.
4 more timeline events
All Facts (13)
According to Laurie Voss, head of developer relations at Arise.ai, in a deeplearning.ai collaborative course, the failure modes of Agentic applications (such as fabricating information, calling the wrong tools, skipping steps) simply do not exist in traditional software development.
50%UnverifiedThis course by Laurie Voss was launched in collaboration with deeplearning.ai, takes the form of hands-on labs, and covers the complete chain from trace analysis, test case construction, and evaluator and judge implementation, to making improvement decisions based on evaluation.
50%UnverifiedThe improvement loop Voss describes is: use evaluation results to decide what to prioritize improving, and then after making changes use the same set of evaluations to confirm that the Agent has indeed gotten better.
50%UnverifiedVoss specifically mentioned that you also need to check whether the LLM judge itself is reliable; if the judge's judgments are unstable or biased, the improvement decisions made based on it are built on shifting sand.
50%UnverifiedVoss's methodological starting point is: do not design test sets out of thin air, but start from real failures found in Agent traces and turn failure cases into reusable test cases.
50%UnverifiedIn the course, Laurie Voss clearly pointed out that building an evaluation system is not a nice-to-have add-on but a basic skill that AI engineers must master.
50%UnverifiedRyan Carson argues that work is becoming "agent manager", and Laurie Voss talks about the rise of the product engineer.
50%UnverifiedIn the article 'We are all Product Engineers now,' Laurie Voss argues that the cost of writing code has collapsed, and the cost of reviewing, fixing, and operating code is following suit.
50%UnverifiedThis observation by Laurie Voss is in line with the themes of generative AI and agent engineering that Simon Willison has long focused on.
50%UnverifiedThe costs of understanding requirements, defining the product, and polishing the experience exist on a per-piece-of-software basis and cannot be transferred or reused.
50%UnverifiedVoss argues that when writing, reviewing, fixing, and operating code all become cheap, the work that truly remains in software production is figuring out what people really want, defining it precisely, and making it delightful to use.
50%UnverifiedVoss believes that the demand for software has no ceiling and that the quantity of software will trend toward infinity.
50%UnverifiedVoss predicts that the cost of reviewing, fixing, and operating code will ultimately reach the same low point as writing code.
50%