JavaScript Set Keeps Duplicate Objects: Deduplicate by ID

A Set of objects does not compare their fields#

MDN’s Set reference explains that object equality depends on identity. Two object literals with matching fields remain different entries. Adding the very same object reference twice produces one entry instead. Neither outcome consults the id property.

The fixture below verifies both cases before applying ID-based deduplication. This distinction matters when an API returns independently constructed records: the duplicate you mean is an application key collision, rather than the same JavaScript object appearing twice.

Choose the collision rule before shortening the implementation into a one-liner:

These are encounter-order rules. The last record in an array is not necessarily the newest by timestamp. If freshness is your requirement, compare a validated version or timestamp explicitly rather than assuming response order proves it.

Reproduce all three winner and ordering policies#

Save the following as deduplicate.mjs and run node deduplicate.mjs. The published fixture was executed with Node.js 24.13.0 on macOS during article preparation on October 20, 2026. It requires no dependencies.

1
import assert from 'node:assert/strict';
2

3
function uniqueById(rows, policy = 'first') {
4
  if (!Array.isArray(rows)) throw new TypeError('Expected an array');
5
  if (!['first', 'last', 'last-position'].includes(policy)) {
6
    throw new TypeError('Unknown duplicate policy');
7
  }
8
  const byId = new Map();
9
  for (const row of rows) {
10
    if (row === null || typeof row !== 'object' || Array.isArray(row)) {
11
      throw new TypeError('Expected a data record');
12
    }
13
    const id = row.id;
14
    if (!Object.hasOwn(row, 'id') ||
15
        !(typeof id === 'string' && id.length > 0 ||
16
          typeof id === 'number' && Number.isSafeInteger(id))) {
17
      throw new TypeError('Expected an own nonempty string or safe-integer id');
18
    }
19
    if (policy === 'first' && byId.has(id)) continue;
20
    if (policy === 'last-position') byId.delete(id);
21
    byId.set(id, row);
22
  }
23
  return [...byId.values()];
24
}
25

26
const rows = [
27
  { id: 'a', version: 'a-old' },
28
  { id: 'b', version: 'b-only' },
29
  { id: 'a', version: 'a-new' },
30
];
31
assert.equal(new Set(rows).size, 3);
32
assert.equal(new Set([rows[0], rows[0]]).size, 1);
33
console.log('Set: three separate objects remain; repeated reference collapses');
34

35
for (const [policy, expected] of [
36
  ['first', ['a-old', 'b-only']],
37
  ['last', ['a-new', 'b-only']],
38
  ['last-position', ['b-only', 'a-new']],
39
]) {
40
  const result = uniqueById(rows, policy);
41
  const versions = result.map(row => row.version);
42
  assert.deepEqual(versions, expected);
43
  assert.equal(rows.length, 3);
44
  assert.equal(rows[0].version, 'a-old');
45
  console.log(`${policy}: ${versions.join(', ')}`);
46
}
47

48
const mixed = uniqueById([
49
  { id: 1 }, { id: '1' }, { id: 0 }, { id: -0 },
50
]);
51
assert.deepEqual(mixed.map(row => row.id), [1, '1', 0]);
52
console.log('keys: number 1 differs from string 1; zero and negative zero collide');
53

54
for (const bad of [{}, { id: undefined }, { id: null }, { id: '' },
55
  { id: NaN }, { id: Infinity }, { id: {} }, { id: 1.5 }]) {
56
  assert.throws(() => uniqueById([bad]), TypeError);
57
}
58
assert.throws(() => uniqueById(new Array(1)), TypeError);
59
assert.deepEqual(uniqueById([]), []);
60
console.log('validation: invalid IDs and sparse input rejected; empty array accepted');
61

62
const retained = uniqueById(rows);
63
assert.notEqual(retained, rows);
64
assert.equal(retained[0], rows[0]);
65
retained[0].version = 'changed';
66
assert.equal(rows[0].version, 'changed');
67
console.log('references: new array, original record objects');
68
console.log('all assertions passed');

Expected output:

1
Set: three separate objects remain; repeated reference collapses
2
first: a-old, b-only
3
last: a-new, b-only
4
last-position: b-only, a-new
5
keys: number 1 differs from string 1; zero and negative zero collide
6
validation: invalid IDs and sparse input rejected; empty array accepted
7
references: new array, original record objects
8
all assertions passed

For the input sequence a-old, b-only, a-new, both last-wins policies keep a-new. Their different output order is intentional. The assertions check the exact sequence, rather than merely confirming that the output contains two records.

The implementation uses one local Map for each call. In first mode, an existing key skips replacement. In last mode, replacement keeps that key’s position. In last-position mode, deleting before setting gives the replacement a new position. The ECMAScript Map algorithms distinguish updating an existing entry from appending a new entry; the fixture exercises both paths.

Validate IDs before treating them as keys#

The helper rejects a missing, undefined, null, empty-string, object, fractional, infinite, or NaN ID. This is an application policy implemented by the guard, not a restriction imposed by Map. Without a missing-ID check, multiple records whose row.id is undefined can collapse into one entry, concealing malformed input.

A string ID and a numeric ID remain separate here: 1 differs from '1'. Zero and negative zero share a key, as the mixed-ID assertion demonstrates. Map’s documented key equality explains this boundary.

Do not add String(row.id) simply to make records collapse. First decide whether numeric and textual identifiers denote the same entity in your data contract. This helper does not trim strings, normalize case, or reject whitespace-only strings; add those checks if your identifier format requires them. It accepts zero, so a truthiness check such as if (!row.id) would enforce a different policy.

A sparse array also fails validation: the loop encounters an undefined entry instead of a data record. An empty dense array returns an empty result. The code assumes ordinary records whose field access does not execute application behavior; it does not sanitize getters or proxies. Parse external payloads into your accepted schema before applying the helper.

Deduplication does not create independent records#

The result array is new, but the selected records are the original objects. The final assertion deliberately mutates a retained record and observes the change through the source array. This is separate from the earlier checks that deduplication itself leaves the original length and first record intact.

If a caller must edit the output without changing the source, define a copying step for the retained data. A shallow object spread still shares nested objects; see the nested object spread and reference checks. Do not infer deep independence from the fact that the array itself is new.

For an ID-based result, verify three things at the call site: the chosen record for each collision, the required order, and whether shared record references are acceptable. The fixture covers those decisions for string and safe-integer IDs in Node.js. It does not benchmark large lists, test browser or framework state behavior, resolve composite keys, or define a deep-equality algorithm.

References:

• MDN: Set
• MDN: Map
• ECMAScript: Map.prototype.set