Ellie

The first session of the OpenAI official API Bootcamp series was presented by Ellie from the Enablement team

When building API applications, one should first understand the real problem before considering whether to bring in more infrastructure, and technology choices should be driven by requirements rather than by how advanced the technology is

The three typical pitfalls of building API applications are vague use cases, obsession with a particular technology, and the lack of a clear validation method

The BCA diagnostic framework breaks down problems into three lanes — Behavior, Context, and Action — plus a Quality dimension that runs throughout

The Behavior layer has four control knobs: Prompting, structured outputs, model selection, and reasoning intensity — they are distinct knobs rather than ones that all need to be turned to the maximum

Model selection should involve running the same set of representative use cases through different candidate models, comparing classification accuracy, recall on sensitive cases, latency, and cost, with decisions made by evaluation rather than by model labels

Context requirements do not equal the need to build a retrieval system; RAG is only needed when finding the correct passage among tens of thousands of constantly changing documents

In the three-step function-calling process, the application (e.g., Acme) defines the capabilities, the model requests a call with structured parameters, and the application decides whether to execute it

Agents become useful only when the next approved step depends on what the system discovers along the way; scenarios with fixed steps are suited for Workflows, and simple question-and-answer is suited for Chatbots

A complete eval consists of four components — a dataset, the system under test, a scorer, and metrics — and during evaluation only one variable should be changed at a time while rerunning the same set of use cases