Matt
Head of Cybersecurity Engineering at OpenAI, leading the company's engineering and technical direction in the cybersecurity domain as well as related product strategy.
Timeline (last 90 days)
Matt is the developer of Docs Agent, an open-source documentation tool built on LangGraph that can automatically analyze GitHub repositories, find documentation gaps, and generate fix PRs.
In the demo evaluation, Docs Agent's documentation quality score was low and exposed a rendering prompt error, giving Matt a data basis to judge whether the code needed improvement or the evaluation method needed adjustment.
Matt believes that if you are trying to put a product into production or use it in an enterprise, evaluation is very important; but for independent developers, evaluation is worth the investment only when the agent really needs to perform well in a production environment.
Matt mentioned that if the model gets stronger tomorrow or the testing framework changes, the evaluation system also needs to iterate accordingly.
Matt observed that the top-performing students are without exception excellent communicators who write clear and detailed descriptions when submitting code changes; those with poor output often are not lacking in technical skill but are unable to explain the problem clearly to the agent.
Matt once tried to organize a custom video editor project into a massive specification document and then have the agent generate the code, which lasted several months with unsatisfactory results, and he ultimately abandoned the specification and turned his attention back to the code itself.
Matt (founder of aihero.dev) has roughly ten years of hobbyist plus eight years of professional coding experience, long believed that large models writing code was just a bubble, and after a certain model capability leap point came to believe he would never write a line of code himself again.
Matt puts forward a view contrary to the mainstream narrative: do not give the model the grandest tasks, but rather the most boring and trivial work (such as drafting replies and handling miscellaneous chores), because such tasks make output quality easy to measure and errors easy to correct.
Matt believes there is a severely underestimated learning curve with agents, and that stronger models will not eliminate this curve but only hide it deeper.
Matt suggests that people without a technical background learn the brakes rather than the accelerator first: at the start they should watch the agent's every move and understand each of its actions and the capabilities of its toolchain, then gradually expand to multi-agent setups, custom skills and external data source integration.
3 more timeline events
All Facts (13)
Matt mentioned that if the model gets stronger tomorrow or the testing framework changes, the evaluation system also needs to iterate accordingly.
50%UnverifiedMatt is the developer of Docs Agent, an open-source documentation tool built on LangGraph that can automatically analyze GitHub repositories, find documentation gaps, and generate fix PRs.
50%UnverifiedMatt believes that if you are trying to put a product into production or use it in an enterprise, evaluation is very important; but for independent developers, evaluation is worth the investment only when the agent really needs to perform well in a production environment.
50%UnverifiedMatt designed a skill called Grill Me that has the agent instead relentlessly interview the user, forcing out requirements, boundaries and implicit preferences.
50%UnverifiedMatt observed that the top-performing students are without exception excellent communicators who write clear and detailed descriptions when submitting code changes; those with poor output often are not lacking in technical skill but are unable to explain the problem clearly to the agent.
50%UnverifiedMatt (founder of aihero.dev) has roughly ten years of hobbyist plus eight years of professional coding experience, long believed that large models writing code was just a bubble, and after a certain model capability leap point came to believe he would never write a line of code himself again.
50%UnverifiedMatt puts forward a view contrary to the mainstream narrative: do not give the model the grandest tasks, but rather the most boring and trivial work (such as drafting replies and handling miscellaneous chores), because such tasks make output quality easy to measure and errors easy to correct.
50%UnverifiedMatt believes there is a severely underestimated learning curve with agents, and that stronger models will not eliminate this curve but only hide it deeper.
50%UnverifiedMatt suggests that people without a technical background learn the brakes rather than the accelerator first: at the start they should watch the agent's every move and understand each of its actions and the capabilities of its toolchain, then gradually expand to multi-agent setups, custom skills and external data source integration.
50%UnverifiedMatt says he now runs four agents simultaneously handling long-term tasks across different projects, with daily commits pushed to GitHub reaching 200 to 300, whereas in his hand-coding days 20 a day was considered good.
50%UnverifiedMatt believes the codebase is the agent's working environment, and the environment determines the output: putting a powerful model into a garbage codebase only yields garbage results.
50%UnverifiedIn the demo evaluation, Docs Agent's documentation quality score was low and exposed a rendering prompt error, giving Matt a data basis to judge whether the code needed improvement or the evaluation method needed adjustment.
50%UnverifiedMatt once tried to organize a custom video editor project into a massive specification document and then have the agent generate the code, which lasted several months with unsatisfactory results, and he ultimately abandoned the specification and turned his attention back to the code itself.
50%