One Evening a Week to Nail Asset Uploads: Metadata, Covers, and Automated Duplicate-Post Prevention

Title

One Evening a Week to Nail Asset Uploads: Metadata, Covers, and Automated Duplicate-Post Prevention

Meta Description

A digital asset seller shares his one-evening-a-week upload automation system: scripts generate platform metadata CSVs, a plain-text ledger makes duplicate submissions structurally impossible, and cover packaging plus document conversion run on a 2GB low-memory server—keeping his account out of trouble.

Article Content

The biggest fear for a digital asset seller isn't a lack of inspiration—it's repetitive work and duplicate submissions. One developer shares his firsthand experience: last Tuesday, because he couldn't remember which files had already been uploaded, he nearly pushed the same batch of 40 files to Adobe again. Adobe doesn't warn about duplicates—it simply records it as a poor review history. Re-submitting already-published assets is a clear cause of accounts getting flagged. He caught it in time, but only by manually digging through his upload history. That evening he decided: tracking must be done by scripts, not by memory.

Metadata CSVs: Headers Are a Hard Requirement, Not a Suggestion

The three major asset platforms are extremely strict about the headers of metadata files. Adobe requires 5 columns, Vecteezy requires 4, and Dreamstime requires 15. Format errors don't get gentle warnings—they're rejected outright at upload time. Piecing things together manually in a spreadsheet will sooner or later result in wrong column order or incorrect delimiters, and the whole batch gets sent back.

The author's solution is make_metadata.py: point it at a folder, choose a platform, and it generates a CSV that matches that platform's header requirements. If no title file is provided, it automatically converts filenames into title format—but the author warns that these auto-generated titles still need to be replaced before submission, because reviewers really do look at them.

The time accounting is straightforward: manually building CSVs for three platforms for 40 assets used to eat up most of an evening; now it takes three commands and about a minute of compute.

Duplicate Submissions Are the Fastest Way to Get Your Account Flagged

The most common mistake in upload automation is running the same batch of jobs against the same channel twice. Asset sites treat duplicate submissions as a quality issue, and enough of them will trigger account review or restrictions. The author's fix is so simple it can't go wrong: upload_tracker.py is a plain-text ledger. Before starting a batch, run pending to see incomplete items; after each successful upload, mark it done. When re-running the batch script, check the ledger first—duplicate submissions go from "unlikely" to "structurally impossible."

His advice is direct: don't build a database, build a list you can't misread. The entire upload history is a single notepad file—readable, backup-able, and grep-able.

Why asset platforms treat duplicate submissions as a quality issue

Search ranking on asset libraries depends heavily on "new asset quality signals": approval rates, distribution of rejection reasons, and the duplication rate within the same account all feed into risk-control models. Re-submitting the same file not only wastes reviewer effort but is judged as "library padding" behavior—the account's approval rate gets dragged down, all subsequent uploads enter a slower manual review queue, and in serious cases upload permissions are restricted or the account is frozen.

A more insidious risk is near-duplicates: different resolutions or slightly color-adjusted versions of the same asset are highly similar at the perceptual-hash level and also get grouped into duplicate families. That's why the ledger should record not just "uploaded" but also an identifier for the asset (filename + content hash), so that re-running a batch after renaming still blocks duplicate submissions. A plain-text ledger is sufficient precisely because a single line of hash filename platform date is all it takes to make that determination.

Discipline on a 2GB Low-Memory Machine: Serial Processing, Prompt Cleanup, On-Demand Rendering

The entire pipeline runs on a server with 1 CPU core, 2GB of memory, and no GPU. This constraint shapes every operating habit.

• Never parallelize image tasks: on a single core, parallel doesn't run faster—it runs slower and crushes the machine.
• Call gc.collect() after every asset: free memory between projects.
• Use subprocesses for re-rendering steps: one subprocess per file, so memory is truly returned to the OS rather than pooled inside Python.
• Measure peak RSS, not average: use resource.getrusage(RUSAGE_SELF).ru_maxrss, because averages can't tell you what happened at the moment the task crashed.
• Render at on-demand resolution: use 60dpi instead of 300dpi for preview checks—the cost of checking is nearly zero; only final output pays the full-resolution cost. For AI image generation billed per image, re-rendering an entire batch because a high-resolution check failed is a real cost, not just a theoretical one.

Peak RSS and ru_maxrss

RSS (Resident Set Size) is the number of bytes of physical memory a process currently occupies, while ru_maxrss records the process's peak resident memory since startup. The reason to measure peak rather than average is that OOM (out-of-memory) kills a process based only on the instantaneous peak: a task that averages 300 MB but spikes to 1.9 GB while decoding a 3000x3000 image in a single frame looks completely safe on average, yet the kernel will terminate it at exactly that moment.

Watch out for the unit trap: on Linux, resource.getrusage(RUSAGE_SELF).ru_maxrss returns kilobytes (KB), while on macOS it returns bytes. A cross-platform script that divides by 1024² will be off by a factor of 1024. On a 2GB machine, logging this number—one line per asset—lets you see the memory climbing trend before a crash instead of afterward, guessing at a traceback.

Why gc.collect() and subprocesses actually return memory

Python's reference counting frees objects immediately when references are dropped, but circular references can only be handled by the generational garbage collector. That's why long-running batch processes show a "high water mark" where RSS only grows and never falls. Explicitly calling gc.collect() after each asset reclaims that batch's circular references and pushes memory back down to baseline.

But the more thorough approach is subprocesses: memory freed by CPython often isn't returned to the OS, but stays in pymalloc's internal pools for reuse. When a subprocess exits, the kernel reclaims its entire address space in one go—the only way to make RSS actually drop. The cost is process startup overhead and the inability to share caches, so use it only for steps like re-rendering where "a single call consumes a lot and the number of calls is limited." Batch lightweight tasks stay in the main process.

Packaging Is a Command, Not Manual Labor

Bare files don't sell. A zip with a 1280x720 cover and a 600x600 thumbnail is a product. The author used to spend 30 to 45 minutes per product making covers in an image editor—39 products meant a full week of spare time.

pack_product.py takes a folder, title, subtitle, and item count, and directly outputs the zip, cover, and thumbnail. The only third-party dependency is Pillow. His view is pragmatic: a cover isn't art—it's a consistency framework, and consistency is exactly what scripts excel at.

The same goes for documentation. md2pdf.py uses reportlab to convert Markdown into a styled PDF with headings, tables, lists, and links. One command. The document looks like a product because the styling is fixed—not because you still have good taste at 11 PM.

Folder Structure Matters More Than Tools

Without a fixed structure, directories turn into junk drawers. His five-level structure is: assets for raw output, output for submittable files and metadata, packs for finished zips and covers, research for notes and scrape logs, and scripts for tools.

He made an early mistake: during cleanup, he deleted the research folder. Those notes and logs are raw material for future products. He rebuilt what he could recover, and now nothing in that folder is ever deleted.

The Real Rhythm of One Evening a Week

One evening, about three hours. Generate this week's assets and review them, run three metadata commands, check the ledger with pending, package two or three products, mark everything done. The scripts have packaged 39 products so far—about 856 MB of output—all on that 2GB server. The entire parent pipeline is about 19,000 lines of code, but the day-to-day packaging layer is just four scripts and two pip installs.

The author doesn't pretend this is a fully automated business: reviewers still reject things, titles still need rewriting, and some weeks he skips the evening entirely. But last Tuesday's failure mode—re-running a batch because he forgot—is now structurally impossible.

The four scripts, the folder layout, and the memory notes are bundled as a starter kit, on the grounds that the packaging layer is the boring part that holds everyone back—it shouldn't take a year to get right.

One-line Summary

With four scripts and a plain-text ledger, the repetitive work and duplicate submissions in asset packaging and uploading become structurally impossible.

Paragraph Summary

After nearly pushing 40 files to Adobe twice, a digital asset seller decided to replace memory with scripts. The article presents a lightweight pipeline running on a single-core, 2GB, GPU-less server: make_metadata.py generates platform-compliant metadata CSVs (Adobe 5 columns, Vecteezy 4, Dreamstime 15); upload_tracker.py uses a plain-text pending/done ledger to make duplicate submissions structurally impossible; pack_product.py and md2pdf.py compress cover, thumbnail, and document packaging from 30–45 minutes of manual work into a single command. The author also stresses low-memory discipline—serial processing, gc.collect() after every asset, subprocesses for re-rendering, measuring peak RSS instead of average, on-demand resolution rendering—and a fixed five-level folder structure to keep directories under control.