From One Prompt to 39 Finished Products: Lessons from Building an Automated Pipeline

Uploading 40 assets to Adobe Stock in one batch, only to wake up the next morning to a screen full of rejection notices — the images themselves were fine. The problem was the CSV. One column header written incorrectly, and Adobe's uploader rejects the entire file without even glancing at the images. This developer distilled a simple truth from that experience: generating content with a prompt is the easy part. What actually turns ideas into products is everything that happens after generation.

Here's what he documented: which parts broke, how he fixed them, and the numbers behind it all.

Metadata Headers Are a Gate, Not a Suggestion

CSV upload requirements vary wildly across platforms. Adobe requires exactly 5 header columns, Vecteezy wants 4, and Dreamstime needs 15. Submit a file with the wrong structure to any of them, and validation fails immediately — no partial passes, no "close enough." The author lost an entire batch because one header column was missing.

The fix is boring but effective: give each platform exactly what it asks for — not one field more. His make_metadata.py script now generates all three formats from the same folder using --platform adobe, --platform vecteezy, or --platform dreamstime flags. The script falls back to capitalizing the filename as the title, but before submitting he always passes in a titles.json with real titles and keywords. A filename like asset-017.jpg won't survive human review.

If you're building something similar, don't try to make one "universal CSV." Build one per platform, follow the spec strictly, and validate column counts before uploading. Three formats, three headers, zero improvisation.

Duplicate Submissions Are the Fastest Way to Get Banned

The second mistake was entirely human. After a network hiccup, he re-ran an upload batch and ended up sending 12 assets twice. Duplicate submissions are exactly the kind of behavior that gets accounts flagged — and he had no record of what had already gone out.

Now every batch runs through a ledger script called upload_tracker.py. It's a plain-text ledger with three commands: pending lists incomplete items, done marks an item finished, and status gives an overview. It uses only the Python standard library — no dependencies to install. Since the ledger is checked before every submission, re-running a batch can never cause duplicates.

This lesson generalizes: any automation pipeline that touches an external account needs its own memory. Don't trust your 1 a.m. self to remember what was already sent.

"Idempotency" is the core concept here: the same operation, executed any number of times, produces the same result as executing it once. For an upload pipeline, idempotency means that even if the script is re-run after a network interruption, the system identifies already-submitted entries and skips them rather than blindly resending. The ledger script essentially maintains a local "submitted set," computing the set difference before each upload and only processing entries not yet in the set. In distributed systems this is known as the "at-least-once delivery + idempotent consumer" pattern — but it applies equally well to single-machine scripts. External platform review systems generally don't deduplicate on your behalf; the cost of duplicate submissions falls entirely on the submitter.

A 1-Core 2GB Server Forces Discipline

The entire pipeline runs on a single-core machine with 2GB of RAM and no GPU. That constraint shaped everything. The habits that made it work:

• Run only one process at a time. Parallelizing image tasks on a single core just causes them to compete and degrade performance.
• Call gc.collect() after each asset to prevent memory from climbing steadily across a 39-item run.
• Measure peak RSS, not average, using resource.getrusage(RUSAGE_SELF).ru_maxrss. Averages tell you nothing; peak RSS tells you exactly where a particular render will OOM.
• Spawn a subprocess for each heavy rendering step so memory is actually returned to the OS.
• Preview at 60 dpi, render final output at 300 dpi. He once spent an entire night debugging a memory spike, only to discover he'd been rendering inspection copies at full resolution.

None of this is clever. It's just what a small server forces out of you. Honestly, the constraint made the pipeline simpler, not worse.

RSS (Resident Set Size) refers to the actual physical memory a process currently occupies — unlike virtual memory, it reflects real RAM consumption. resource.getrusage(RUSAGE_SELF).ru_maxrss returns the peak RSS over the process's lifetime, in KB on Linux and bytes on macOS. OOM (Out Of Memory) refers to the mechanism by which the OS force-kills a process when memory is exhausted — the Linux kernel's OOM Killer selects a process to terminate when both physical memory and swap are depleted. Killed processes leave no meaningful error messages; they simply vanish silently, making OOM issues extremely hard to trace in logs. This is precisely where subprocesses help: when a child process exits, the OS reclaims all of its memory pages, whereas in a single long-running process, even calling gc.collect() may not actually return memory to the OS if it was allocated by C extensions (such as image processing libraries).

Turning Content into a Product Is Its Own Step

The author had a folder full of Markdown documents with real content but no way to make them look like products. The gap between "a document" and "a sellable PDF" is layout: headings, tables, lists, links. md2pdf.py bridges that gap with reportlab in a single command: python md2pdf.py content.md product.pdf "My Product Title".

Packaging works the same way. A bare zip without a cover image doesn't sell. pack_product.py bundles files and uses Pillow to render a 1280×720 cover and a 600×600 thumbnail, writing the zip, cover, and thumbnail into a _system folder that can be relocated via --dir or PACK_ROOT. The entire toolset uses only two third-party packages: Pillow and reportlab. Everything else is standard library.

Folder Structure Is the Real Product

The most underestimated thing: an asset library without a fixed structure turns into a junk drawer. The layout that eventually stabilized uses five folders: assets/ for raw output, output/ for submittable files and metadata CSVs, packs/ for finished zips and covers, research/ for notes and scrape logs, and scripts/ for tooling.

There's also one rule learned the hard way: never delete research/. He once cleared it out for the sake of "cleanup" and lost the source material for three future products. Scrape logs and notes are inventory, not garbage.

The Final Numbers

The pipeline ultimately produced 39 finished products and 856 MB of output, backed by roughly 19,000 lines of parent project code and over 300 pages of research supporting the scripting rules. All of it validated on that 1-core, 2GB, GPU-free machine. The four scripts described above form the packaging layer, running beneath a free orchestration stack — Hermes Agent handles task scheduling, and 9Router handles local LLM access at $0 cost.

The author admits he never set out to build a toolkit. He just wanted to stop losing batches. It wasn't until a third friend asked for "those CSV scripts" that he cleaned up the four scripts that actually did the work. No framework, no cloud account, runs on Python 3.9+; if you only need metadata and tracking, you don't even need pip install.

This case study is a useful reference for anyone building an automated content pipeline: what usually determines success or failure isn't the flashy generation step — it's the boring engineering details: header specs, idempotent ledgers, memory discipline, directory structure. Generating ten thousand images is easy. Reliably turning those ten thousand images into sellable products is the hard part.

The "Hermes Agent" and "9Router" mentioned in the article are components of a local LLM orchestration stack: the orchestration layer breaks high-level tasks into subtasks and schedules script execution, while the routing layer distributes requests across multiple locally running language models — processing prompts without incurring any API call costs. Typical alternatives for this kind of local stack include Ollama with LangChain, or directly calling llama.cpp's HTTP interface. The core value in all cases is reducing LLM access costs to near zero, so that metadata generation for each image can be retried any number of times without worrying about a mounting bill.