← Back to Blog

Perceptual Hashing in the Browser: Near-Duplicate Image Detection

October 2026~13 min read

Same Picture, Different Bytes

Cryptographic hashes (SHA-256 and friends) answer "are these two files identical?" For images, that is almost never the question you have. The same photo saved twice from different sources differs in every byte: JPEG quality 85 versus 92, a 2× resize, metadata stripped, an alpha channel added. SHA-256 sees two unrelated files; your eyes see the same picture.

Perceptual hashing closes that gap. The idea: compress the image's visual content — not its bytes — into a small fingerprint, typically 64 bits, such that similar-looking images produce fingerprints with small differences. "Similar" then becomes a numeric threshold on the Hamming distance between fingerprints, and duplicate detection becomes a distance query.

The classic construction is pHash: downscale to 32×32 grayscale, run a 2D DCT, keep the top-left 8×8 block of low-frequency coefficients, and threshold each against their median. Every step is a few dozen lines of JavaScript; there is no library gap to fill, and the whole pipeline — decode aside — costs well under a millisecond per image. This article builds it end to end, then deals with the two hard parts most tutorials skip: indexing thousands of 64-bit fingerprints, and being honest about what the hash cannot see.

Step 1: Decode, Flatten, Downscale

The first 90% of real-world wall-clock time is decoding. A 4000×3000 JPEG takes tens of milliseconds to decode no matter how fast the math that follows is — so the goal here is to touch full-resolution pixels as little as possible. createImageBitmap can resize during bitmap creation, which is strictly faster than drawing a full-size canvas and reading it back:

async function toGrayscale32(blob) {
  const bitmap = await createImageBitmap(blob, {
    resizeWidth: 32,
    resizeHeight: 32,
    resizeQuality: 'medium',
    imageOrientation: 'from-image',   // honor EXIF rotation deterministically
  });

  const canvas = new OffscreenCanvas(32, 32);
  const ctx = canvas.getContext('2d', { willReadFrequently: true });
  // Transparent pixels would read back as black — composite onto white first
  ctx.fillStyle = '#ffffff';
  ctx.fillRect(0, 0, 32, 32);
  ctx.drawImage(bitmap, 0, 0);
  bitmap.close();

  const { data } = ctx.getImageData(0, 0, 32, 32);
  const gray = new Float32Array(32 * 32);
  for (let i = 0; i < gray.length; i++) {
    const o = i * 4;
    // Rec. 709 luma weights — matches human brightness perception better
    // than a flat (r + g + b) / 3, and is what pHash implementations use.
    gray[i] = 0.2126 * data[o] + 0.7152 * data[o + 1] + 0.0722 * data[o + 2];
  }
  return gray;
}

Three details that each cost an afternoon if missed. The white composite: PNGs with transparency yield (0,0,0,0) pixels; drawing them straight onto a fresh canvas reads back as black, so a transparent logo hashes differently from the same logo on white — and differently again from the same logo on a dark page. Compositing onto white first makes the hash a function of the artwork, not of whatever alpha happened to be in the file. EXIF orientation: request 'from-image' explicitly rather than depending on platform defaults; the same photo with two different orientation tags should hash the same. Rec. 709 weights over a flat average: the green channel carries most luminance information for human vision, and weighting it 0.7152 gives visibly better discrimination on color-similar images.

Why 32×32 as the working size? The DCT keeps only the top-left 8×8 block, and a common rule for that transform is to start from 4× the target dimension. 32×32 is the size used by the reference pHash implementation and every port since, so it is also what gives you comparable thresholds with published guidance.

Step 2: The DCT, With One Precomputed Table

The discrete cosine transform converts the 32×32 spatial image into 32×32 frequency coefficients, where low frequencies live in the top-left corner. The 2D transform is separable — apply it to every row, then every column — which turns an O(n⁴) naive computation into two O(n³) passes of plain multiply-add.

The only real trick is the cosine table. Every coefficient's kernel is cos((2x + 1) · u · π / 64) — a function of pixel position and frequency only — so a 32×32 table of cosines can be computed once per application and reused for every image and every row and column:

// cos[(2x + 1) * u * PI / 64], precomputed once for the lifetime of the app
const COS = new Float32Array(32 * 32);
for (let u = 0; u < 32; u++) {
  for (let x = 0; x < 32; x++) {
    COS[u * 32 + x] = Math.cos(((2 * x + 1) * u * Math.PI) / 64);
  }
}

function dct2d(src) {
  const tmp = new Float32Array(32 * 32);
  const out = new Float32Array(32 * 32);

  // Pass 1: rows. src[y][x] -> tmp[y][u]
  for (let y = 0; y < 32; y++) {
    for (let u = 0; u < 32; u++) {
      let sum = 0;
      for (let x = 0; x < 32; x++) sum += src[y * 32 + x] * COS[u * 32 + x];
      tmp[y * 32 + u] = sum;
    }
  }

  // Pass 2: columns. tmp[y][u] -> out[v][u]
  // The DCT's normalization factor (2/32, plus 1/sqrt(2) for u=0) is a
  // constant scaling — pHash only compares relative magnitudes, so skipping
  // it changes nothing downstream.
  for (let x = 0; x < 32; x++) {
    for (let v = 0; v < 32; v++) {
      let sum = 0;
      for (let y = 0; y < 32; y++) sum += tmp[y * 32 + x] * COS[v * 32 + y];
      out[v * 32 + x] = sum;
    }
  }
  return out;
}

Count the work: 2 × 32³ ≈ 65,000 multiply-adds per image. On any machine from the last decade that is tens of microseconds of Float32Array math — which is worth knowing, because it means pHash never needs WebAssembly or a worker for the math. If your pipeline is slow, it is decoding, and the fix is a decode-side optimization (thumbnail first, smaller resize target, parallelism across workers), not a faster DCT.

Step 3: Median, Bits, and a 63-Bit Fingerprint

The top-left 8×8 block of the coefficient matrix holds the low frequencies — the coarse structure of the image. The DC term at (0,0) is the average brightness of the whole image and carries no structure, so it is excluded; that leaves 63 AC coefficients. Each one becomes one bit: 1 if it is at or above the median of all 63, 0 otherwise. The median comparison is what makes the fingerprint robust to uniform brightness and contrast changes — scaling an image's brightness scales all coefficients together, so the bit pattern survives.

function phash64(gray) {
  const dct = dct2d(gray);

  const low = [];                       // 63 low-frequency AC coefficients
  for (let v = 0; v < 8; v++) {
    for (let u = 0; u < 8; u++) {
      if (v === 0 && u === 0) continue;   // skip the DC term
      low.push(dct[v * 32 + u]);
    }
  }

  const sorted = [...low].sort((a, b) => a - b);
  const median = sorted[sorted.length >> 1];

  let hi = 0, lo = 0;                   // 64-bit fingerprint as two int32 halves
  for (let i = 0; i < low.length; i++) {
    if (low[i] >= median) {
      if (i < 32) hi |= (1 << i);
      else        lo |= (1 << (i - 32));
    }
  }
  return { hi: hi >>> 0, lo: lo >>> 0 };  // >>> 0 normalizes int32 to unsigned
}

Two packing conventions to note because implementations in the wild disagree. Some libraries include the DC coefficient in the median calculation but not in the bits; I exclude it from both. What matters is being consistent within one deployment — hashes from two different base algorithms are not comparable, even at the same bit width. The 63rd bit stays 0, so the fingerprint is really 63 effective bits; serializing as 16 hex characters keeps storage uniform:

const hex = h => h.hi.toString(16).padStart(8, '0')
               + h.lo.toString(16).padStart(8, '0');
// '9f2c4e1a0b3d5c7e' — 16 chars, one row per image in your database

Step 4: Hamming Distance and Calibration

Comparing fingerprints is XOR plus population count — the number of differing bits:

function popcount(x) {
  x -= (x >>> 1) & 0x55555555;
  x = (x & 0x33333333) + ((x >>> 2) & 0x33333333);
  x = (x + (x >>> 4)) & 0x0f0f0f0f;
  return (x * 0x01010101) >>> 24;
}

const hamming = (a, b) => popcount(a.hi ^ b.hi) + popcount(a.lo ^ b.lo);

Now the hard question: what distance means "duplicate"? There is no universal number, and articles that quote one without qualification are wrong about your data. What is true across datasets:

Calibrate empirically instead of guessing: sample a few thousand pairs from your actual library (or hash a folder of known variants), histogram the distances, and look for the valley between the "same image" cluster near 0 and the "different image" mass around 30. Put the threshold there. In my image-download workflows the valley sits at 5–7, and the default threshold is a user-visible setting, because the cost of a false positive (deleting a distinct image) and a false negative (a duplicate slips through) are both unacceptable to guess at.

What pHash Cannot Do

The failure modes are as important as the algorithm, and they are usually undersold:

When you need crop- or rotation-robust matching, pHash is the wrong tool; that problem belongs to feature-based matching (ORB, SIFT) which trades orders of magnitude more computation for geometric invariance. For the dominant real-world cases — re-encodes, resizes, screenshots, downloads from different sites — the 63-bit fingerprint is the right 65,000 multiply-adds.

Indexing a Whole Library: Banding Beats Brute Force

Comparing candidate images one at a time is fine for hundreds of files. At ten thousand it is a hundred million Hamming computations per batch — wasteful, because almost all pairs are obviously different. The problem is that Hamming distance is not a B-tree-queryable metric, so the fix has to be algorithmic. The standard one is banding, and it rests on a pigeonhole argument:

Split the 64-bit hash into B bands. If two hashes differ in at most B − 1 bits total, then at least one band is bit-for-bit identical between them.

With 4 bands of 16 bits, any pair within distance 3 shares at least one identical band. So you index images by each band's 16-bit value, and a query only has to check the union of the query's four posting lists — a few dozen candidates instead of the whole library:

const bandsOf = h => {
  const s = hex(h);
  return [s.slice(0, 4), s.slice(4, 8), s.slice(8, 12), s.slice(12, 16)];
};

// INSERT: for each band, push the image id into that band's bucket
// bands: Map<bandValue, Set<imageId>>
function index(hash, id, bands) {
  for (const b of bandsOf(hash)) {
    if (!bands.has(b)) bands.set(b, new Set());
    bands.get(b).add(id);
  }
}

// QUERY: union the 4 posting lists, then verify exact distance
function candidates(hash, bands) {
  const out = new Set();
  for (const b of bandsOf(hash)) {
    const bucket = bands.get(b);
    if (bucket) for (const id of bucket) out.add(id);
  }
  return out;
}

Two caveats, stated plainly. First, banding only finds pairs within B − 1 bits; querying with distance threshold 6 requires 7 bands of ~9 bits, and more bands means more false-positive candidates for the verification pass to filter. Second, identical bands are necessary but not sufficient — the verification pass with exact Hamming distance is still mandatory. If your threshold needs to float, a BK-tree over the distance metric is the more flexible structure at moderate library sizes; banding with a fixed threshold is the one that stays simple and fast.

Where do the hashes live? One row per image in your storage layer of choice — 16 hex characters plus an identifier — which is a rounding error next to the images themselves. If the app already embeds SQLite (a natural fit for local-first image tools), the band index maps directly to four indexed columns; compute the candidate set with a single OR query and verify in JavaScript.

Where It Runs: Workers and Progress

Decode dominates, so that is what to parallelize. A worker pool sized to navigator.hardwareConcurrency keeps the UI responsive while a batch imports, and Blob objects cross the worker boundary cheaply:

// hash-worker.js — one worker, many images
self.onmessage = async ({ data: { jobId, blob, imageId } }) => {
  try {
    const gray = await toGrayscale32(blob);
    self.postMessage({ jobId, imageId, hash: phash64(gray) });
  } catch (err) {
    self.postMessage({ jobId, imageId, error: err.message });
  }
};

Two production notes. Size the pool by cores but cap it — eight workers each holding a decoded 12-megapixel bitmap is roughly a gigabyte of transient memory, and the correct pool size for a batch importer on a mid-range laptop is often four. And report progress per completed image, not per percent of bytes: the user's mental model is "47 of 230 images", and a per-file counter also surfaces stuck decodes (a corrupt image that never calls back) far earlier than a bytes-progress bar ever would. Wrap the per-image decode in a timeout for the same reason.

Production Checklist

  1. Composite onto white before reading pixels — transparent PNGs must hash as art, not alpha.
  2. Request imageOrientation: 'from-image' on createImageBitmap for deterministic EXIF handling.
  3. Resize inside the decoder (resizeWidth/resizeHeight); never touch full-resolution ImageData in JS.
  4. Precompute the cosine table once; keep DCT math in Float32Array.
  5. Exclude the DC coefficient consistently — pick one convention and never mix fingerprints from different ones.
  6. Calibrate the distance threshold against your own library's histogram; make it a setting, not a constant.
  7. Detect degenerate low-variance images and route them to exact hashing.
  8. Index by bands for large libraries; always verify candidates with full Hamming distance.
  9. Hash in a worker pool sized to cores — capped around 4–8 — and close bitmaps eagerly.
  10. Timeout individual decodes and surface per-image progress.
  11. Store 16-hex-char fingerprints as ordinary rows; keep originals and hashes in the same transaction so the index can rebuild.

References

Related Articles