[KongchangAI]
· 4 min read· 2,063 words

Data Scraping, AI Proofs, and Open Source Contracts: This Week's Deep Dive into the Tech World

Data Scraping, AI Proofs, and Open Source Contracts: This Week's Deep Dive into the Tech World

This week's tech discussions centered on data openness, mathematical verification, AI orchestration, and software durability.

The article opens with crawlers scraping three 3D model communities, noting the controversy isn't about copying but about lost contact—researchers' requests went unanswered, and the community received no weight feedback. It then discusses OpenAI's use of Lean formalization and public GitHub verification for complexity theory results, contrasting with the physics group's conservative approach; Brooks' Law doesn't fail in AI agent scenarios—communication and orchestration costs merely shift; six rhythm criteria for depression can be measured by phone, lowering the diagnostic threshold; a Debian developer rejects LLM review on DFSG grounds and proposes actionable acceptance criteria; and a 34-year-old Windows program's continued operation shows durability comes from commitments, not code. Throughout, one theme runs: openness is the precondition for systems to be inspected, inherited, and repaired.

When Crawlers Learn "Manners": Openness and Lost Contact in the 3D Model Community

A crawler calling itself "Trellis 2 Data Collection" swept through three 3D model communities—Oven Sunday Lab, SFM Lab, and SmartBass—over 34 hours in mid-October with 26,000 requests. It read nearly 19,000 project homepages and downloaded 1,027 complete files totaling 145GB. The key detail: it used an identifiable user agent throughout, attached a roblox.com email and an AWS address, and even scheduled lunch breaks and a decreasing request rate.

This wasn't a stealthy scrape but an organized data collection operation. The technical motivation is easy to understand: language models have trillions of tokens, while the largest open mesh dataset in the 3D field, Objiverse, has fewer than a million observations—many of which are untextured scans or CAD exports. What's truly scarce are rigged character models with bones, weights, and topology—exactly the cream that these three communities have accumulated over a decade.

For a specific reason.

The controversy isn't about "copying" but about "lost contact." The researcher sent two emails to that address, and roblox never responded. A person who made a request under their real identity, along with the institution behind them, received no reply. This creates a one-way valve: data flows from the community to the platform's models, while modelers cannot download, inspect, or run anything. The community's demand is clear—take the data, then release the weights. Objiverse became foundational precisely because it is open.

Objaverse is a large-scale 3D object dataset released by the Allen Institute for AI in 2022, containing about 800,000 3D models and their metadata scraped from platforms like Sketchfab by Objaverse 1.0, with later versions expanding beyond 800,000. It is often regarded as the "ImageNet moment" of 3D vision, because before it, publicly available 3D mesh data was small in scale and sparsely annotated. However, most Objaverse models are static scans or CAD exports, lacking skeletal rigging, skinning weights, and motion data, making them difficult to use directly for animation or embodied intelligence research. The article's mention that "the largest open mesh dataset has fewer than a million observations" refers to this scale bottleneck, and rigged character models with bones and weights, accumulated over decades in the three communities, are precisely the source of scarcity.

The "Publicness" of Mathematical Proofs Is Being Redefined

This week brought three new findings from mathematics, computer science, and physics. Most notable were three results announced simultaneously by OpenAI—one each in harmonic analysis, complexity theory, and quantum field theory—but with three different release strategies.

The complexity group's claim was especially bold: a deterministic algorithm that can exactly count the number of satisfying solutions for any conjunctive normal form formula, fast enough to overturn the Exponential Time Hypothesis (ETH). This hypothesis is the load-bearing wall for a large body of lower-bound results. The working group's first decision was to immediately release a natural-language proof, a Lean formal verification, and a GitHub repository, so that anyone with a laptop could download and run the verifier.

This echoes the "Chief Programmer Team" proposed by Harlan Mills in 1971 and supported by Brooks in Chapter 3 of The Mythical Man-Month: the key isn't who does it, but whether the structure can be publicly inspected. By contrast, the physics group chose internal writing and waited for release. The podcast's metaphor was apt: understanding is like a fermentation starter—not something you can possess, but a culture that must be handed to other kitchens to survive.

A jar on one counter is not a tradition.

One overlooked issue: two labs released similar boundaries within hours, one in a PDF and one on Google Drive, neither machine-verifiable. Yet theorems should be the most open human product—they don't care who proved them or on which page they first appeared. Verification still relies on trusting two organizations, a condition that could be fixed this month.

When AI Starts Writing Code: Does Brooks' Law Still Hold?

"Adding manpower to a late software project makes it later." Brooks wrote this in 1975 based on experience with IBM OS/360. Fifty years later, the question becomes: does this law still hold when the new additions are agent processes?

On the surface, agents have zero onboarding cost and zero rest time. But what Brooks really measured was communication paths—which grow with the square of the number of participants. Nine agents running on the same codebase without orchestration will overwrite each other at machine speed. Sequential tasks remain sequential. Someone still has to define "done," review output, catch fabrications, and stitch fragments together. The overhead has shifted, but it hasn't evaporated.

The "Chief Programmer Team" proposed by Mills in 1971—a surgeon writing code, supported by a deputy, administrator, toolsmith, tester, and language lawyer—was previously constrained by the scarcity of human labor for each support role. Today this organizational structure becomes buildable: one engineer plus a toolsmith process, a tester process, and a documentation process. The team structure drawn on paper 55 years ago is being rediscovered this month. But a caution: model weights are not the interesting artifact; the org chart is. And right now, most orchestration patterns live in private scaffolding and unshared configurations.

Brooks' Law comes from Fred Brooks' 1975 book The Mythical Man-Month, originally stated as "Adding manpower to a late software project makes it later." Its core argument is communication overhead in software engineering: the number of communication channels needed among n developers is roughly n(n-1)/2, growing with the square of the number of people, while task decomposition and integration themselves consume time. Brooks also distinguished between "decomposable tasks" and "sequential dependencies," the latter of which cannot be accelerated by parallelism. The article transfers this framework to AI agent scenarios, pointing out that although agents have no onboarding or rest costs, the costs of orchestration, review, and defining "done" remain, so communication overhead hasn't disappeared but shifted.

Chief Programmer Team is a software organizational structure proposed by Harlan Mills in 1971, later discussed by Brooks in Chapter 3 of The Mythical Man-Month. The idea is to form a small, specialized team around a chief programmer: the chief handles all core design and coding, a deputy assists and takes on some implementation, and roles such as administrator, toolsmith, tester, language lawyer, and documentation editor each perform their duties to reduce communication paths. Mills demonstrated its feasibility internally at IBM on projects like the New York Times clipping system. The article notes that this model was previously constrained by the scarcity of human labor for each support role, but AI agents can cheaply play toolsmith, tester, and other roles, making this 55-year-old paper structure buildable again—though the orchestration patterns themselves still largely reside in private scaffolding.

Diagnosis and Understanding: From Sleep Rhythms to Building a Reasoning Model from Scratch

A Sydney team analyzed 11,380 people in the Australian Genetics of Depression Study and found that one in four depression patients met six observable criteria: seasonal mood swings, delayed sleep midpoint, social jet lag, evening chronotype, long sleep when allowed, and morning fatigue. If the illness is a phase error rather than a neurotransmitter deficiency, then drugs that modulate serotonin may be working at the wrong level.

A comment hit the nail on the head: this amounts to announcing that "scientists have discovered night owls, and society is forcing them into a destructive schedule." The pathology may partly reside in the nine-to-five system. And the data needed for diagnosis—seasonality, two sleep midpoints, chronotype, sleep duration, morning fatigue—can be measured with a phone and a cheap actigraphy device. All six criteria can be measured at home, freeing diagnosis from laboratory dependence.

Now break the sync, not the individual clocks, just the phase relationship between them.

On another front, Sebastian Raschka released an 8-video, 11-hour course on "Building a Reasoning Model from Scratch," implementing evaluation, inference-time scaling, reinforcement learning with verifiable rewards, and distillation by hand in Python and PyTorch on small pre-trained models. Pre-training is skipped—that's the part that costs money; the inference layer is the part that costs understanding. And reasoning post-training is precisely the most tightly guarded link in the modern stack: labs publish architectures but withhold recipes. Now this recipe is written as readable PyTorch code, recorded on open weights. The barrier's disappearance is voluntary.

Debian's Honest Refusal and the Enduring Contract of Old Software

Debian developer Johannes Schauer stated that he will stop uploading new packages to the NEW queue until he can opt out of automated review. NEW is the queue where entirely new source packages wait for manual license and copyright checks before entering the archive. His reasoning is precise enough to be written as acceptance criteria: if the LLM has a bug, I can't fix it—the model is a binary blob. This is a DFSG argument expressed in Debian's own vocabulary.

The solution is actually clear: open weights instead of binary blobs, temperature set to zero with pinned revisions to ensure reproducible output, public prompts for auditability, and an opt-out field in debian/control. Someone summarized it all in one line: "Do not use LLM" in an upload suppresses review. That's an afternoon's work, maybe two. This demonstrates the discipline of criticism—a developer looks at a new tool in the pipeline and specifies point by point what would make it legitimate.

What does a new review actually need to be?

Another story is about durability. A developer searched for the oldest graphics program that runs on Windows 11 without an emulation layer, and the answer was CDPlayer.exe, linked on November 16, 1992—34 years old, still able to launch, draw a window, and play audio with a virtual CD drive. Reversi has been running since 1985, but only through an open-source project that reimplements the 16-bit layer. The vendor fulfilled the contract for 34 years, then stopped. Someone working in the open took over. Durability isn't a property of the code; it's a property of the commitments around it.

DFSG (Debian Free Software Guidelines) is an appendix to Debian's social contract for determining whether software qualifies as free software, and is the direct source of the Open Source Definition (OSD). Its core requirements include: free redistribution, provision of source code, permission for modifications and derivative works, no discrimination against persons or groups, no discrimination against fields of use, and rights that travel with the software. The key to DFSG is that it demands "fixability" and "auditability," not just free access. The article's mention that Schauer's objection is "a DFSG argument expressed in Debian's own vocabulary" refers precisely to the fact that an LLM as a binary blob cannot satisfy the requirements of modifiable source code and reproducible behavior, thus conflicting with the spirit of the DFSG.

The Tension Between Open and Closed

Several stories this week point to the same theme: openness isn't a moral posture but a precondition for whether a system can be inspected, inherited, and repaired. A crawler takes data but doesn't release weights—a one-way valve; mathematical proofs locked in PDFs degrade verification to trust; AI review tools are binary blobs whose defects can't be fixed; old Windows programs rely on the open-source community to survive after the vendor stops.

Conversely, Objiverse, Lean formalization, downloadable reasoning model courses, and public interface documentation show a different path: release after taking, verify after copying, take over after depending. This week's tech discussions superficially concern crawlers, mathematics, and emulators, but at their core they ask the same question—does what you build allow others to download, run, and continue?

One-line Summary

This week's tech discussions centered on data openness, mathematical verification, AI orchestration, and software durability, with the core question being whether systems can be inspected and inherited.

Paragraph Summary

The article opens with crawlers scraping three 3D model communities, noting the controversy isn't about copying but about lost contact—researchers' requests went unanswered, and the community received no weight feedback. It then discusses OpenAI's use of Lean formalization and public GitHub verification for complexity theory results, contrasting with the physics group's conservative approach; Brooks' Law doesn't fail in AI agent scenarios—communication and orchestration costs merely shift; six rhythm criteria for depression can be measured by phone, lowering the diagnostic threshold; a Debian developer rejects LLM review on DFSG grounds and proposes actionable acceptance criteria; and a 34-year-old Windows program's continued operation shows durability comes from commitments, not code. Throughout, one theme runs: openness is the precondition for systems to be inspected, inherited, and repaired.

#data scraping#3D models#OpenAI#Lean formal verification#Brooks' Law#AI agents#Debian#DFSG#open source#software durability
Share:

Related articles