Getting Started with Physics-Informed Neural Networks (PINNs): Should You Major in Math, CS, or Physics?

A Seemingly Simple but Highly Representative Question

Recently, a student aspiring to research Physics-Informed Neural Networks (PINNs) posed a question on Reddit that cuts right to the heart of the matter: to pursue research in this field, should you choose math, computer science, or physics as your undergraduate major?

reddit source

This question may look like a simple major-selection dilemma, but it actually touches on a core tension in the intersection of scientific computing and deep learning — PINNs is inherently a convergence of three major disciplines, and knowledge from any single one is insufficient for in-depth research. Below, we'll analyze the value of each major from the perspective of PINNs' technical essence and offer more actionable advice.

What Are Physics-Informed Neural Networks (PINNs)?

To answer the question of which major to choose, we first need to understand what kind of knowledge PINNs actually requires.

The Origins and Development of PINNs

Physics-Informed Neural Networks were systematically proposed in 2019 by George Karniadakis' team at Brown University, representing a deep fusion of deep learning and scientific computing. Traditional scientific computing relies on numerical methods like finite elements and finite differences to discretize and solve partial differential equations, requiring mesh generation and incurring high computational costs. PINNs, on the other hand, leverage the universal approximation capability of neural networks, encoding PDE constraints directly into the loss function and optimizing through backpropagation — no explicit mesh needed. This paradigm shift has demonstrated unique advantages in inverse problem solving, data assimilation, and multi-physics coupling, particularly for high-dimensional problems and sparse data scenarios.

Core Idea

PINNs are a class of deep learning methods that embed physical laws (typically expressed as partial differential equations, PDEs) directly into the neural network's loss function. While traditional neural networks rely purely on data fitting, PINNs must not only fit observational data during training but also satisfy constraints from governing equations, boundary conditions, and initial conditions.

In other words, a PINN's loss function typically consists of two parts: one for data error, and another for the residual term — measuring how much the network output deviates from zero when substituted into the PDE. By using automatic differentiation to compute derivatives of the network output with respect to inputs at various orders, PINNs can solve both forward and inverse problems with virtually no labeled data.

Automatic Differentiation: The Technical Cornerstone of PINNs

Automatic Differentiation (AD) is the core technical enabler of PINNs, distinct from both numerical and symbolic differentiation. Numerical differentiation suffers from truncation and rounding errors, while symbolic differentiation faces expression swell. Automatic differentiation, based on the chain rule, tracks derivatives of each operation through a computational graph, efficiently computing derivatives of any order to machine precision. Modern deep learning frameworks like PyTorch's autograd, TensorFlow's GradientTape, and JAX's grad all implement automatic differentiation. In PINNs, AD allows us to directly compute partial derivatives of neural network outputs with respect to input coordinates, thereby constructing PDE residual terms — a level of flexibility that traditional numerical methods can hardly match.

Why PINNs Is an Interdisciplinary Field

From the definition above, it's clear that PINNs involves at least three layers of knowledge:

Physics Layer: Understanding the governing equations of the system under study, such as the Navier-Stokes equations in fluid dynamics, heat conduction equations, wave equations, etc. Partial Differential Equations (PDEs) are the mathematical language for describing continuous change in nature. From the Navier-Stokes equations for fluids, Maxwell's equations for electromagnetic fields, the Schrödinger equation in quantum mechanics, to the Black-Scholes equation for derivatives pricing — PDEs are everywhere. Traditional PDE solving requires numerical analysis techniques like the Finite Element Method (FEM) and Finite Volume Method (FVM), which are mature but computationally intensive. PINNs offer a data-driven alternative, particularly advantageous when boundaries are complex, dimensions are high, or complete physical models are lacking. Understanding PDE well-posedness, boundary condition types, and conservation laws is a prerequisite for using PINNs.
Mathematics Layer: PDE theory, numerical analysis, optimization theory, functional analysis, etc. — the foundation for understanding convergence and stability of the method. Theoretical analysis of PINNs is a current research hotspot. Despite PINNs performing well in practice, their theoretical guarantees remain relatively weak. Key issues include: non-convexity of the loss function causing optimization difficulties, spectral bias making high-frequency components hard to learn, and a lack of theoretical guidance for balancing multi-term loss weighting strategies. Error estimates from numerical analysis, Sobolev space theory, and functional analysis tools are being used to establish convergence proofs. For example, Mishra et al. (2021) proved a priori error bounds for PINNs under specific conditions, and De Ryck et al. (2022) analyzed residual decay rates. This theoretical work requires solid mathematical foundations and represents the dividing line between engineering applications and academic research.
Computation Layer: Neural network architectures, automatic differentiation frameworks (such as PyTorch, JAX), GPU acceleration, and training optimization techniques.

This is precisely the root of this student's dilemma — all three directions are indispensable.

Value Analysis of Three Majors for PINNs Research

Mathematics: The Deepest Theoretical Foundation

If your goal is to deeply understand the theoretical essence of PINNs, a mathematics major provides the most rigorous training. PDEs, numerical analysis, and optimization theory are core tools that PINNs research cannot do without. A large portion of cutting-edge papers in the PINNs field discuss convergence proofs, error estimates, regularization for ill-posed problems, and more — all of which demand solid mathematical foundations.

The weakness of a math major is that pure mathematics curricula often lack direct training in programming practice and physical modeling, which you'll need to supplement on your own.

Computer Science: The Strongest Engineering Capability

At the end of the day, PINNs is a machine learning method, and its implementation heavily depends on deep learning frameworks, automatic differentiation, and high-performance computing. A CS major provides the most direct training in machine learning, programming, and systems skills.

In practice, training a PINN often involves engineering challenges like gradient pathologies, convergence difficulties, and multi-objective loss balancing. Solving these problems relies heavily on understanding the underlying mechanisms of deep learning — exactly where CS training excels.

Physics: The Most Authentic Problem Context

A physics major lets you truly understand where the equations you're solving come from and what they physically mean. The value of PINNs lies in solving real scientific and engineering problems. Without understanding the physics, it's easy to fall into the trap of "running models for the sake of running models," unable to judge whether results are reasonable or identify the sources of error.

Physics programs also typically cover mathematical methods (mathematical physics) and some programming (computational physics), making it a relatively balanced choice among the three.

Major Selection Advice: Combining Your Primary Major with Self-Study

There's No Single Right Answer, but There Are Better Paths

Given PINNs' knowledge structure, math or physics as a primary major + computer science as a strong minor (or self-study track) is often the safer path. The reasoning is:

Machine learning and programming skills are relatively easier to supplement through self-study, open-source projects, and online courses, with the practical barrier dropping year by year;
Solid PDE theory and physical intuition, on the other hand, are very difficult to build through piecemeal self-study without systematic coursework.

If your ultimate goal is algorithm and theory research (e.g., improving PINNs convergence, proposing new architectures), a math major gives you the highest theoretical ceiling. If your goal is to apply PINNs to specific engineering/science problems (e.g., meteorology, materials science, biomedical engineering), then a physics or relevant engineering major + CS skills is the more practical combination.

The Key Is Building Cross-Disciplinary Competence

Regardless of which major you choose, what truly determines how far you can go in the PINNs field is your willingness to proactively study across disciplines. If you major in math, you should take machine learning and physical modeling courses as early as possible; if you major in CS, you must supplement numerical analysis and PDEs.

In reality, top researchers in the PINNs field are almost all "T-shaped talents" — with sufficient depth in one discipline and adequate breadth across the other two. The T-shaped talent concept originates from management theory, referring to individuals with both deep expertise (the vertical bar) and cross-disciplinary breadth (the horizontal bar). In the intersection of scientific computing and AI, this model is especially important. Training in a single discipline often creates knowledge blind spots: a pure math background may lack familiarity with deep learning engineering, a pure CS background may lack intuition for physics problems, and a pure physics background may have insufficient theoretical analysis capabilities. Successful PINNs researchers typically exhibit a T-shaped profile: building depth with one discipline as their "home base," then expanding into the other two dimensions through self-directed learning, cross-disciplinary collaboration, and project-driven exploration. Developing this capability requires initiative — auditing courses in other departments, participating in interdisciplinary seminars, reading papers from different fields — rather than passively waiting for curriculum arrangements.

Conclusion: Choose Your "Home Base," Then Extend Outward

This student's question fundamentally reflects the strong demand for interdisciplinary talent in the AI for Science era. AI for Science is an emerging research paradigm in recent years, aiming to accelerate scientific discovery with artificial intelligence. DeepMind's AlphaFold2 successfully predicted protein structures in 2020, marking a breakthrough for AI in fundamental science. Since then, applications have proliferated: machine learning force fields for molecular dynamics simulations, neural networks for quantum many-body problems, AI-assisted materials design, and more. PINNs is an important branch of this movement — it's not simple data fitting, but rather embedding physical laws as prior knowledge into models, achieving a unification of physical consistency and data-driven approaches. This fusion has given rise to Scientific Machine Learning (SciML) as an emerging interdisciplinary field, requiring researchers to simultaneously possess domain knowledge, mathematical theory, and computational capability.

PINNs is just a microcosm — the entire Scientific Machine Learning field is calling for researchers who can simultaneously command physical laws, mathematical tools, and computational implementation.

Rather than agonizing over which single major to choose, think of it as choosing your "home base" — you can start from any discipline, but you'll ultimately need to extend into the other two. Pick the subject you're most passionate about and most talented in as your primary major, then deliberately build a cross-disciplinary knowledge map. That's the right approach to entering PINNs research.