The Second Material of Software Engineering: How AI Is Reshaping the Way We Build

Software Engineering Is Gaining a Second "Material"

At a conversation in New York, prominent tech blogger Gergely Orosz and WorkOS founder Michael Grinich put forward a thought-provoking idea: software engineering is gaining its "second material." For decades, the only material available for building software was deterministic code — logic written line by line by human engineers, predictable and reproducible. Now, with the maturation of large language models (LLMs), an entirely new, probabilistic material is being introduced into the builder's toolkit.

What makes this metaphor valuable is that it reframes AI from "yet another tool" into a foundational material. Much like construction evolved from relying solely on wood to incorporating reinforced concrete, the addition of a new material isn't a simple substitution — it fundamentally changes what can be built, how it's built, and who gets to participate in building it.

The Fundamental Difference Between the Two Materials

Deterministic vs. Probabilistic

The defining characteristic of traditional code is determinism: given the same input, you always get the same output. This predictability has been the cornerstone of software engineering methodologies, testing frameworks, and quality assurance processes for the past several decades. Engineers rely on this determinism to reason about system behavior, track down bugs, and guarantee reliability.

The second material introduced by LLMs is fundamentally probabilistic in nature. It excels at handling ambiguous, unstructured tasks that require "judgment," but the trade-off is that outputs are no longer fully predictable. The same prompt can produce slightly different results, and the same scenario may yield unexpected variations. This isn't a flaw — it's an inherent property of the material, just as concrete has compressive strength advantages alongside tensile limitations.

The probabilistic nature of LLMs stems from their underlying architecture: when generating each token, the model is essentially sampling from a probability distribution over its vocabulary rather than executing deterministic logic. The temperature parameter directly controls the intensity of this randomness — higher temperature produces more varied outputs, while temperature approaching zero makes the model behave closer to deterministic. This means developers can "tune" the degree of probabilism within a certain range, but cannot eliminate it entirely. Furthermore, even with identical prompts, model version upgrades, backend load balancing, or differences in inference frameworks can quietly alter outputs. This poses a fundamental challenge to traditional software testing paradigms that depend on precise reproducibility — you can't use unit tests to cover a component that is statistical in nature.

Complementary Strengths, Distinct Roles

An important implicit judgment in the conversation is that these two materials don't replace each other — they coexist in a complementary relationship. Deterministic code continues to handle the core logic that must be precise, reliable, and error-free; while the probabilistic AI material takes over problems that were previously difficult to solve elegantly with code alone — natural language understanding, content generation, fuzzy matching, and intent recognition.

The hallmark of truly excellent system design will be knowing which material to use where. Delegate the parts requiring strict guarantees to code, hand off the parts requiring flexible judgment to models, and design clear boundaries and validation mechanisms between the two.

What This Means for Builders

A Redefined Skill Set

When engineers gain an additional material, the capabilities they need to master expand accordingly. Beyond traditional programming and architecture skills, builders now need to understand how to "work" this new material: how to design prompts, how to constrain model outputs, how to evaluate the quality of probabilistic results, and how to build reliable user experiences amid uncertainty.

This is akin to a carpenter who only knows wood suddenly needing to learn how to pour concrete — the underlying engineering mindset transfers, but the specific techniques, intuitions, and failure modes all need to be built from scratch.

Evaluating the quality of probabilistic systems has spawned an emerging field: LLM Evaluation (LLM Eval). Unlike traditional testing, it typically relies on "model-evaluating-model" approaches, or uses human-annotated gold datasets to measure output accuracy, safety, and stylistic consistency. Prompt Engineering, as another core skill, studies how to guide models toward consistently producing expected results through instruction design, few-shot examples, and Chain-of-Thought techniques. These skills have yet to develop a mature methodological framework comparable to software engineering — much practical knowledge still circulates as experience rather than theory. This is precisely why "how to work with the new material" remains an open question today.

An Expansion of Product Possibilities

The most exciting thing about a new material is that it makes previously "unbuildable" products viable. Many features that depend on human judgment, language understanding, or creative generation were either impossible to implement before, or required enormous human labor costs. Now, these capabilities can be embedded directly into software.

This also means the competitive dimensions of products are shifting. As foundational capabilities become democratized through AI materials, differentiation will increasingly come from how the two materials are combined — and from finding the right balance between reliability and flexibility.

A Transformation Worth Watching

Understanding AI as software's "second material" is a deeper framework than thinking of it as an "AI tool" or "AI assistant." It reminds builders that what we're facing isn't a tool upgrade, but an expansion of the building paradigm itself.

For engineering teams, the critical question going forward is no longer "whether to use AI," but rather "in which parts of the system, which material, and how to make the two materials work together." The conversation between Gergely Orosz and Michael Grinich is a timely prompt for exactly this kind of shift.

It should be noted that this article is based on analysis of publicly available summaries of the conversation. Specific technical details and the complete discussion are subject to further elaboration from the original source material.