Building a Deep Learning Framework from Scratch in Pure NumPy: A New Lightweight DL Option

The deep learning framework ecosystem seems long since divided between the two giants PyTorch and TensorFlow, but some people still choose to start from scratch. Recently on Reddit, a lightweight deep learning framework built entirely from scratch in pure NumPy attracted attention—and more importantly, the project has been accepted by JOSS (Journal of Open Source Software), meaning it has passed academic peer review.

Why Would Anyone Rewrite a Deep Learning Framework in NumPy?

Implementing a deep learning framework from scratch in NumPy sounds like reinventing the wheel, but it is precisely the best path to understanding the underlying mechanisms of deep learning. PyTorch's automatic differentiation and TensorFlow's computation graphs—these abstraction layers make development efficient, but they also hide the details of matrix operations, gradient propagation, and backpropagation.

The NumPy version lays everything out in the open: every tensor operation and every gradient calculation step is explicit Python code. For teaching scenarios and learners, this transparency is something mainstream frameworks cannot provide. This is also an important reason why JOSS was willing to accept it—the academic value lies in readability and reproducibility, not performance metrics.

Automatic differentiation is the core mechanism of modern deep learning frameworks; it automatically computes gradients by recording the computation process. Mainstream frameworks typically use reverse-mode autodiff, which builds a computation graph during a forward pass and then traverses that graph in reverse, applying the chain rule. NumPy itself does not provide automatic differentiation, so when implementing a framework from scratch, one must manually build a computation graph or adopt a tape-based gradient recording approach. Understanding this makes it clear why rewriting a framework in NumPy is a deep engineering and teaching exercise—it requires the implementer to fully handle every link in gradient propagation.

What This Framework Can Do

As a lightweight implementation, its positioning is clear: it is not meant to replace PyTorch for production deployment, but to provide a readable, modifiable, and learnable complete deep learning stack. Based on the source material, the project covers the core components of a deep learning framework—tensor operations, automatic differentiation, and the model training workflow built on these foundational capabilities.

You may not have noticed that JOSS's acceptance criteria emphasize the academic contribution and documentation completeness of open-source software. This means the project not only has runnable code but also comes with sufficiently clear documentation and tests, allowing researchers to build on it for secondary development or teaching use.

Dual Value for Teaching and Research

For university courses and laboratories, such frameworks are ideal teaching tools. Students can read through the entire source code in a single semester and understand how backpropagation is derived from the chain rule into code; researchers can quickly modify the underlying implementation for specific algorithms without struggling with the abstraction layers of a massive framework.

JOSS (Journal of Open Source Software) is an academic journal focused on open-source research software, using a public peer-review process. Its acceptance criteria include: the software must have research value, have clear documentation, include test cases, and follow an open-source license. Unlike traditional algorithm or theory papers, JOSS review focuses on the usability, reproducibility, and documentation quality of the software itself. Therefore, a project being accepted by JOSS means it has reached a standard of engineering rigor and academic transparency that can be examined by peers, not merely that it is "code that runs."

The Realistic Boundaries of Lightweight Frameworks

It is important to recognize clearly that a framework implemented in NumPy cannot match the performance of a highly optimized PyTorch. There is no CUDA acceleration, no operator fusion, and no distributed training support—these are the natural shortcomings of lightweight solutions.

But this precisely defines its applicable boundaries: small-scale experiments, algorithm validation, teaching demonstrations, and customized research scenarios that require complete control over every line of code. In these scenarios, code transparency matters far more than training speed. For developers who want to deeply understand the essence of deep learning, the learning value provided by such a project far exceeds its practical value as a tool.

CUDA acceleration refers to using the parallel computing power of NVIDIA GPUs to accelerate matrix operations, and it is a key source of the performance advantage of frameworks such as PyTorch. Tensor operations in deep learning (such as matrix multiplication and convolution) are naturally suited to the massively parallel architecture of GPUs, and CUDA provides a programming interface for directly accessing GPUs. A pure NumPy implementation can only run on CPUs, and although NumPy's underlying BLAS libraries are optimized, they cannot leverage the thousands of GPU cores. In addition, compilation optimization techniques such as operator fusion can further reduce memory access overhead, and none of these can be implemented in a lightweight NumPy framework.

Implications for the Open-Source Ecosystem

In an era pursuing larger models and more compute, such a "small but complete" project reminds us: deep learning is not only engineering practice but also a science that needs to be understood. JOSS's acceptance provides a recognition mechanism for teaching-oriented open-source projects of this kind, and it also makes more people realize that implementing a framework from scratch is itself the best way to learn.

If you are learning the underlying principles of deep learning, or need a small, controllable experimental platform, this NumPy framework is worth adding to your toolbox.