Preventing Duplicate Submissions with an Upload Tracker: A Guide to Protecting Your Stock Image Account

Title

Preventing Duplicate Submissions with an Upload Tracker: A Guide to Protecting Your Stock Image Account

Meta Description

Duplicate submissions during bulk stock image uploads can get your account flagged. This article introduces an upload tracker built with only the Python standard library that uses ledger-style records and pending checks to prevent duplicate submissions on reruns, plus tips for optimizing pipelines on low-memory servers.

Article Content

When bulk uploading stock images, a single careless rerun can cause dozens of files to be submitted twice—at best clogging the review queue, at worst getting your account flagged by the platform. One developer shared his hard-learned lesson: on a Tuesday night, he accidentally ran his batch script twice, and by the next morning he found 84 duplicate submissions across two submission portals. To avoid manually withdrawing files ever again, he wrote a simple upload tracker, upload_tracker.py, and from then on checked his to-do list before every bulk upload.

Why Duplicate Submissions Are More Dangerous Than You Think

Many creators assume that one extra file won't matter, but from the reviewer's perspective the situation is entirely different. Adobe Stock's review process checks for duplicates and near-identical submissions, and Vecteezy's reviewers will also see the same file appear twice in the queue. Stock platforms pay per download, and reviewers compare submissions against existing content—duplicate uploads are easily judged as spam or keyword stuffing, which can lead to the entire account being reviewed rather than just the duplicate files. Manually withdrawing them is equally time-consuming—this developer spent about four hours cleaning up 84 files.

The core review logic of stock asset platforms is to prevent content duplication and keyword stuffing—that is, uploaders submitting identical or highly similar files repeatedly, or abusing keywords to gain exposure. Platforms typically maintain a fingerprint database of content already in their library (such as image hashes or perceptual hashes), and reviewers compare submitted content against existing assets for similarity. Duplicate submissions are treated as spam behavior that attempts to bypass normal competitive order, triggering account-level trust demotion rather than merely rejecting a single file. This demotion can manifest as lower priority in the review queue, restricted upload quotas, or even temporary account freezes, with recovery taking far longer than cleaning up the duplicate files themselves.

Ledger Instead of Memory: The Tracker's Core Logic

The author previously tracked uploaded files in text files, but manual updates were easily skipped late at night. The new approach automates recording and enforces checking: the tracker is a plain-text ledger offering three commands—pending lists unfinished files, done marks a file as uploaded, and status shows the overall state.

The key habit is this: the batch script no longer reads the raw file list directly but instead reads the output of pending. That way, on a rerun, already-uploaded filenames won't appear in the to-do list and naturally won't be resubmitted. Run done once after each successful upload, and take two seconds to glance at status before each batch. The entire tool uses only the Python standard library with no dependencies, and runs on any machine you can SSH into.

The pending/done/status design is essentially a minimal state machine: each file moves one-way between the "to upload" and "uploaded" states, and the ledger records the results of those transitions. Compared with a database, a plain-text ledger's advantages are zero dependencies, human readability, read/write access via SSH on any machine, and no corruption from process crashes. Changing the batch script's input source from the raw file list to the output of pending is an "idempotency" design—no matter how many times the script reruns, completed files are never processed again. This is exactly the key to avoiding duplicate submissions: not relying on operator memory but making system state the single source of truth.

The Metadata CSV Is the First Hurdle When Interfacing with Platforms

Before the tracker can do its job, the metadata must be correct. Each platform has strict requirements for the number and order of CSV columns, and header errors will be rejected by the upload form before files even reach review: Adobe Stock requires 5 columns, Vecteezy requires 4, and Dreamstime requires 15. A companion script, make_metadata.py, generates these exact headers from the same folder—no more, no less. Once the CSVs are generated from the same folder, the batch script becomes the single source of truth for "which files have been submitted"—and reruns are precisely what break that.

Stock platform CSV upload forms typically use strict column mapping: each column corresponds to a metadata field (such as title, description, keywords, category, or copyright info), and the column order and header names must exactly match the platform template—otherwise the form parser rejects the entire file outright and may not even specify which column is wrong. Adobe Stock's 5-column template typically includes filename, title, keywords, category, and releases info; Dreamstime's 15-column template breaks things down further into fields such as description, category, and model releases. Generating exact headers from the same folder with a script avoids the common issues of manual CSV editing—column misalignment, extra commas, or BOM encoding problems—which get amplified into whole-batch failures during bulk uploads.

Pipeline Survival Rules on a Small Server

The tracker itself is lightweight, but the surrounding pipeline is not. The author runs the entire workflow on a server with 1 vCPU, 2GB of memory, and no GPU, staying stable by following three habits: running only one process at a time (parallelism on a single core just drags everything down); calling gc.collect() after processing each asset; and measuring peak memory with resource.getrusage(RUSAGE_SELF).ru_maxrss rather than average usage, since peaks are what trigger OOM. The re-rendering step spawns a separate subprocess per file so memory is returned to the OS; previews render at 60 dpi instead of 300 dpi. As a text ledger, the tracker consumes almost no resources—a safety tool should be the cheapest link in the stack.

OOM (Out Of Memory) is the forced termination mechanism triggered by the Linux kernel when memory is exhausted, killing the process consuming the most memory—usually manifesting as a process vanishing without warning. ru_maxrss returns the peak resident set size (RSS) over the process's lifetime, in KB on Linux. The reason to measure peak rather than average is that OOM is triggered by instantaneous peaks, and averages mask memory spikes. gc.collect() is Python's manual trigger for the garbage collector; calling it proactively after processing large objects (such as image arrays) in a loop helps circular-reference objects be released promptly, lowering peak memory. Subprocess isolation (spawning a separate process per file) leverages the OS automatically reclaiming all memory when a process exits, which is more reliable than repeatedly allocating and freeing within a single process.

Folder Structure Makes Tracking Possible

The tracker depends on stable filenames. The author uses a fixed structure: assets/ holds raw output, output/ holds files to be submitted and metadata CSVs, packs/ holds finished compressed packages and covers, research/ stores notes and sources, and scripts/ holds tools. Each asset is one file in assets/, and its name carries all the way through to output/. If you rename between steps, the ledger becomes invalid—so settle on a name once and don't change it. The author also stresses keeping the research/ folder: he once lost the original source materials for a subsequent product by cleaning up his notes.

Toolchain and Practical Recommendations

This approach has been organized into a compliant metadata toolkit containing four scripts: make_metadata.py (exact Adobe 5-column, Vecteezy 4-column, and Dreamstime 15-column headers), upload_tracker.py (the ledger described above), md2pdf.py (converts Markdown to styled PDF), and pack_product.py (packages products and generates 1280×720 covers and 600×600 thumbnails). The entire toolkit has only two dependencies, Pillow and reportlab, while the metadata and tracking scripts need no dependencies at all.

If you regularly run bulk uploads, add a ledger this week. Manually cleaning up duplicate files was the "most boring four hours" of the author's project—and it's entirely preventable.

One-line Summary

Use a plain-text upload tracker and a fixed folder structure to eliminate duplicate submissions caused by bulk upload reruns at the root.

Paragraph Summary

The article recounts one developer's lesson after rerunning a bulk upload script produced 84 duplicate submissions across two stock asset platforms and took four hours to clean up manually. He therefore wrote upload_tracker.py using only the Python standard library, maintaining an upload ledger through three commands—pending, done, and status—and making the batch script read only the pending output so reruns no longer resubmit already-uploaded files. The article also stresses that metadata CSVs must be generated with the exact column counts per platform (Adobe 5, Vecteezy 4, Dreamstime 15), that on a small server with 1 vCPU and 2GB of memory you should process in a single process, call gc.collect() promptly, and measure peak memory, and that a stable folder structure and filenames should be maintained to keep the tracker effective long-term.