← Back to Blog

Referer You Cannot Set: Anti-Hotlink Image Downloads in MV3

October 202610 min read

"Completed 40 Images" — Zero Files

A user batch-downloads from an image-heavy site, gets a cheerful toast — "Completed 40" — opens the Downloads folder, and finds nothing. Not a partial failure: nothing at all, while the UI insists everything succeeded.

That combination of confident success reporting and total absence of output is worth taking seriously, because it usually means the code is counting intentions instead of results. It did here, and behind it were four separate layers of wrongness.

Layer 1: The Count Was a Lie

The batch pipeline mapped over the selected images with a try/catch per image and incremented a counter before attempting the work:

// Before: `total` is "images we planned to process"
return { total: images.length, message: `Completed ${total}` };

Every per-image failure was swallowed, so the reported number was the size of the input list. A downloader must report four distinct outcomes:

return {
  total,
  succeeded: succeeded.length,
  failed: failed.length,
  downgraded,   // requested conversion, delivered original format instead
  deduped: enablePHash ? images.length - processed.length : 0
};

downgraded is the field users never ask for and always need: it distinguishes "your AVIF request was delivered as PNG because this CDN cannot be re-encoded" from "silently gave you something else".

Layer 2: crossOrigin='anonymous' Killed the Pipeline on CDNs

Conversion and perceptual hashing both need pixel access, which requires a non-tainted canvas, which requires:

img.crossOrigin = 'anonymous';

On a CDN that does not return Access-Control-Allow-Origin, that attribute turns a perfectly loadable image into an onerror. The image is visible on the page — the page loaded it fine — but the extension's own request for it fails the CORS check, and the entire downstream chain (convert → compress → pHash → download) fails with it.

The fix is not to drop CORS and accept tainted canvases. It is to make conversion optional per image and degrade instead of dying:

Layer 3: The Thumbnails Were the Real Casualties

A subtler symptom appeared alongside: on anti-hotlink sites, cards vanished from the grid as soon as they rendered. Users could not even select the images they wanted.

The grid rendered <img src="https://cdn.example/...> from the side panel. That request is issued by the extension UI, not by the page, so it carries no page Referer. The CDN answered 403, the error handler removed the card, and the UI dutifully deleted the evidence.

This is where declarativeNetRequest looked like the answer — and where its boundaries start to matter.

Layer 4: chrome.downloads.download Does Not Accept a Referer

The plan was: add a DNR rule that appends the correct Referer to requests matching the CDN, then download normally. It works for the thumbnails. It does not work for the download itself, for three reasons that are all documented behaviour once you find them:

  1. The download request originates in the browser process, not the renderer. Passing headers: [{name: 'Referer', ...}] to chrome.downloads.download throws Unsafe request header name.
  2. downloads.download does not reject on HTTP 403. The call resolves with an id; the failure shows up only in onChanged. Without polling the item to completion, you will keep reporting success.
  3. An MV3 service worker has no URL.createObjectURL. The usual escape hatch — fetch the bytes, wrap in a Blob, download the blob URL — cannot run in the worker at all.
// Required, because the promise resolving only means "download started"
const dlId = await chrome.downloads.download({ url: payloadUrl, filename });
if (!(await C.waitDownloadDone(dlId))) throw new Error('download interrupted');

So the working architecture is: fetch in a UI context (side panel / tab), convert there if needed, create the blob URL there, and hand that URL to the downloads API. The service worker orchestrates rule registration and bookkeeping; it never owns the bytes.

DNR Rules Are a Shared, Finite Resource

Once DNR is in play, the real hazard is not capability but coordination. Two features need Referer rewrites — thumbnail rendering and downloading — and each tab may need a different value. Rules are global, ids are unique, and a naive writer replaces "its" rule wholesale, which silently removes the domains another feature depended on. The result is 403s that appear only after some unrelated action.

Two rules of thumb fixed it:

// Per-tab rule ids derived from the tab id: no in-memory map to lose
// when the service worker restarts, and cleanup is always possible.
const REFERER_RULE_BASE = 100000;
const refererRuleId = (tabId) => REFERER_RULE_BASE + (Number(tabId) || 0);

// Temporary rules (short-lived per-request work) get a disjoint range
// so the two families can never overwrite each other.
const TEMP_RULE_BASE = 1000; // TEMP_RULE_BASE + seq

What the Fix Looks Like From the Outside

How to Diagnose This Class of Bug

  1. Log what the browser actually did. For downloads, listen to chrome.downloads.onChanged and record state plus error; "resolved" is not "saved".
  2. Treat a status count as a contract. If the function returns a number of files, every failure path must decrement it. Add a test with a URL that 403s.
  3. Check document.location versus request origin. Any request your extension makes on the page's behalf is missing the page's Referer; that is not a bug in the site.
  4. Audit DNR rule ids with getDynamicRules() after each feature runs — if the count drops, someone replaced a shared rule.
  5. Remember the context boundaries: worker (no DOM, no object URLs), UI page (full DOM, blob URLs), browser process (downloads, immutable headers).

Checklist

  1. Report succeeded / failed / downgraded / deduped; never a planned-count.
  2. Make conversion and hashing optional per image; degrade to original bytes on CORS failure.
  3. Show a placeholder for 403 thumbnails instead of removing the card.
  4. Never try to set Referer via downloads.download; fetch bytes in a UI context and download a blob URL.
  5. Always verify completion with waitDownloadDone.
  6. Derive DNR rule ids from stable inputs and write unions of domains.
  7. Keep per-feature rule id ranges disjoint.

References

Related Articles