A 15-Column Dreamstime CSV and an Upload Tracker: Fixing Rejections and Double Posts

A Sunday afternoon was completely ruined by a rejection email that said nothing but "Invalid CSV header." I had 40 Dreamstime upload files stuck in the queue, and the table columns I had put together by hand were in the wrong order. Dreamstime's importer doesn't guess: it reads the header row, checks it item by item against the expected 15-column layout, and if even one column is missing or renamed, the whole file fails. That was the moment I stopped hand-editing CSVs and wrote make_metadata.py.

If you're pushing assets or templates to multiple platforms from the same pipeline, metadata format and re-run problems will find you earlier than anything else. Here's what I learned while fixing them.

Every platform has a fixed column count, and they're all incompatible

The three platforms I submit to have three different schemas, and none of them tolerates extra columns:

• Adobe Stock requires a 5-column CSV: title, keywords, category, licensed file, filename. A slight deviation in the header text and the entire upload errors out.
• Vecteezy requires 4 columns. Fewer fields, but it's just as strict about the header.
• Dreamstime is that 15-column format that cost me a Sunday. 15 columns, exact names, exact order.

My first script tried to be clever and generate a "universal" CSV with superset columns. It failed on every platform. What actually worked was boring: the script knows the exact header each platform requires, writes only that header, and nothing else. make_metadata.py ./my-assets --platform dreamstime --out dreamstime.csv --ext .jpg produces a file whose first line matches byte-for-byte what the importer checks. No extra columns, no reordered fields, no "helpful" additions.

One honest note: if there's no title file, the script converts the filename into a title format and uses that. Fine as a first pass, but you should replace it with real titles and keywords before submitting. Using a generic filename as a title is a fast track to soft quality rejections, which are harder to diagnose than a hard header failure.

Duplicate uploads are worse than rejections

A bad CSV gets rejected, you fix it, you move on. A duplicated batch is a different animal. Because a script was re-run, or a cron job fired while you were testing, the upload task runs twice, and the same asset goes live twice. On asset platforms, that kind of behavior is enough to get an account flagged, and a flagged account costs far more than a rejected file.

My pipeline runs in batches, and batches getting re-run is normal. That's the reality. So I wrote a second script, upload_tracker.py, which maintains a simple ledger of what has already been published. The workflow is three commands:

python upload_tracker.py pending items.txt   # which ones aren't done yet
python upload_tracker.py done "asset-001.jpg"
python upload_tracker.py status

The key habit is to mark done immediately after each successful upload, before starting the next one, not all at once at the end of the batch. If the task crashes at item 23, the ledger knows the first 22 are complete, and the re-run picks up from item 23. It's a plain-text ledger, standard library only, no database to corrupt or migrate. That simplicity is a feature, not a flaw. The ledger has to be more reliable than the thing it's tracking.

The two scripts run on the Python standard library alone

This matters more than it sounds. The whole pipeline runs on a 1 vCPU, 2 GB RAM server with no GPU. Every dependency you add is memory you don't have. make_metadata.py and upload_tracker.py use only the standard library, so there's nothing to install and nothing sitting in memory between runs. The entire toolkit has just two third-party packages: Pillow for cover rendering and reportlab for PDF output, and they live in separate scripts you only run at packaging time.

The same constraint shaped the rest of the pipeline's habits: one process at a time on a single core, gc.collect() after processing each asset, measuring peak RSS with resource.getrusage(RUSAGE_SELF).ru_maxrss instead of average usage, and rendering previews at 60 dpi only for checks, not for delivery.

resource.getrusage(RUSAGE_SELF).ru_maxrss is a process resource query exposed by Python's resource module. It returns the peak resident memory (RSS, in KB on Linux) of the current process and its reaped children, not the usage at a single instant. Its value is that memory problems often show up in short bursts — say, Pillow loading a large image or reportlab assembling a PDF — and averages flatten those spikes out, whereas ru_maxrss records the historical high-water mark, which is exactly the moment you need to catch. On a 1 vCPU / 2 GB server, knowing whether the peak is approaching the OOM line is more useful than knowing the average. Combined with calling gc.collect() after processing each asset, it avoids peak accumulation from reference cycles or large objects lingering. The usage is simple: import resource; print(resource.getrusage(resource.RUSAGE_SELF).ru_maxrss) — no third-party dependency, consistent with the pipeline's standard-library-only constraint.

Folder structure is part of the fix

These two scripts don't work well against a junk-drawer directory. The layout the pipeline assumes is fixed: assets/ for raw output, output/ for submission-ready files and CSVs, packs/ for finished archives, research/ for notes and logs, scripts/ for tools. Point make_metadata.py at output/, and it picks up exactly the files that are actually ready.

A mistake I made once and won't repeat: deleting research/ during a cleanup. Those scrape logs and notes were the raw material for three later products. Always keep it. Disk is cheaper than redoing research.

"Junk-drawer directory" here means raw output, submission-ready files, archives, notes, and scripts all mixed together in one directory. The problem isn't that it's ugly — it's undecidable: a script can't reliably tell from filename and extension alone which files are still drafts and which are finished. When make_metadata.py points at such a directory, it may write half-finished items into the CSV, so you upload them with wrong entries, get rejected on the platform side, and then have to trace it back — a multiplied cost. Once you split directories by lifecycle (assets is input, output is deliverable-ready, packs is final packaging, research is knowledge and logs, scripts is tools), each script only ever faces one clearly-defined state of file, and the decision logic changes from "guessing" to "reading." That's why folder conventions are part of the fix: they move some of the correctness out of human memory and into the file system itself.

What I'd tell my past self

The failure wasn't carelessness. Hand-maintaining a 15-column CSV across 40 assets is a process that simply can't survive a real batch. The fix was two small scripts that mechanically enforce platform rules instead of relying on me to remember them. Metadata headers come from the script, never from memory. Upload state comes from the ledger, never from "I think I already uploaded that."

Those two scripts, plus md2pdf.py for converting Markdown docs into styled PDFs and pack_product.py for bundling archives with 1280×720 covers and 600×600 thumbnails, are the entire toolset that packaged the project's 39 delivered products and 856 MB of output on that small server.

If any part of your week sounds familiar, this toolset grew out of that work.

One-line Summary

Two standard-library-only scripts mechanically enforce each platform's CSV header and maintain an upload ledger, eliminating duplicate uploads and manual errors.

Paragraph Summary

The author got an entire batch rejected after hand-maintaining Dreamstime's 15-column CSV header wrong, so he wrote make_metadata.py to have the script generate headers matching each platform's exact schema (Adobe Stock 5 columns, Vecteezy 4 columns, Dreamstime 15 columns) without extra columns. The real hazard wasn't rejection but duplicate uploads from re-runs — which can get an account flagged — so he wrote upload_tracker.py, using a plain-text ledger to mark done immediately after each successful upload so a re-run after a crash can resume from the breakpoint. Both scripts use only the Python standard library and run on a 1 vCPU / 2 GB GPU-less server; paired with a fixed folder structure and memory-control habits, they ultimately supported delivery of 39 products and 856 MB of output.