{"slug":"entity-resolution-product-matching","title":"Entity Resolution at Scale: Matching Products Across Amazon, Reddit, and RTINGS","excerpt":"\"AirPods Pro 2,\" \"Apple AirPods Pro (2nd Generation),\" \"AirPods Pro USB-C\" u2014 same product, three...","content":"\"AirPods Pro 2,\" \"Apple AirPods Pro (2nd Generation),\" \"AirPods Pro USB-C\" u2014 same product, three different names.\n\nEntity resolution u2014 figuring out that different strings refer to the same real-world thing u2014 is one of the hardest problems in product data engineering. At [SmartReview](https://www.aversusb.net), we match products across 50+ review sources, each with its own naming conventions, categorization, and data formats.\n\nHere's how we solved it without spending six months building a custom ML model.\n\n## The Problem Space\n\nConsider matching products across these sources:\n\n| Source | Product Name | Format |\n|--------|-------------|--------|\n| Amazon | Apple AirPods Pro (2nd Generation) - MagSafe Case (USB-C) | Full name + SKU details |\n| Reddit | AirPods Pro 2 | Colloquial shorthand |\n| RTINGS | Apple AirPods Pro 2nd Gen | Abbreviated formal |\n| YouTube | NEW AirPods Pro 2 USB-C Review | Title with marketing fluff |\n| Best Buy | Apple - AirPods Pro 2 - White | Brand-prefixed with color |\n\nAll five refer to the same product. A naive string match would treat them as five different products.\n\n## Our Three-Layer Approach\n\nWe use three complementary techniques, each catching matches the others miss.\n\n### Layer 1: Brand + Model Normalization\n\nThe first pass normalizes brand names and extracts model identifiers:\n\n```typescript\ninterface NormalizedProduct {\n  brand: string;           // \"apple\"\n  modelFamily: string;     // \"airpods pro\"\n  generation: string;      // \"2nd gen\"\n  variant: string;         // \"usb-c\"\n  rawName: string;         // original string\n}\n\nfunction normalizeProductName(raw: string): NormalizedProduct {\n  let name = raw.toLowerCase().trim();\n  \n  // Remove common noise\n  name = name\n    .replace(/\\b(new|latest|best|review|vs\\.?)\\b/g, \"\")\n    .replace(/[-u2013u2014]/g, \" \")\n    .replace(/\\s+/g, \" \")\n    .trim();\n\n  // Extract brand (lookup against known brand list)\n  const brand = extractBrand(name);\n  \n  // Extract generation markers\n  const genPatterns = [\n    /(?:gen(?:eration)?\\s*)(\\d+)/i,\n    /(\\d+)(?:st|nd|rd|th)\\s*gen/i,\n    /\\b(\\d+)\\b(?=\\s|$)/,  // trailing number often = generation\n  ];\n  const generation = extractPattern(name, genPatterns);\n\n  // Extract variant (color, connectivity, size)\n  const variant = extractVariant(name);\n\n  // What remains is the model family\n  const modelFamily = extractModelFamily(name, brand, generation, variant);\n\n  return { brand, modelFamily, generation, variant, rawName: raw };\n}\n```\n\nThis handles ~60% of matches u2014 the straightforward cases where brand + model + generation aligns.\n\n### Layer 2: Fuzzy String Matching\n\nFor the remaining 40%, we use Levenshtein distance with a category-aware threshold:\n\n```typescript\nfunction fuzzyMatch(\n  a: NormalizedProduct,\n  b: NormalizedProduct,\n  threshold: number = 0.85\n): boolean {\n  // Brand must match exactly (after normalization)\n  if (a.brand !== b.brand) return false;\n\n  // Compare model family with fuzzy matching\n  const similarity = stringSimilarity(\n    a.modelFamily,\n    b.modelFamily\n  );\n\n  if (similarity < threshold) return false;\n\n  // If generations are both present, they must match\n  if (a.generation && b.generation && a.generation !== b.generation) {\n    return false;\n  }\n\n  return true;\n}\n```\n\nThe key insight: **brand must match exactly, but model name can be fuzzy.** This prevents false positives like matching \"Sony WH-1000XM5\" with \"Sony WF-1000XM5\" (over-ear vs in-ear u2014 completely different products with similar names).\n\n### Layer 3: Cross-Reference Validation\n\nFor edge cases, we validate matches against external canonical sources:\n\n```typescript\nasync function crossReferenceValidate(\n  candidates: NormalizedProduct[]\n): Promise<ProductCluster[]> {\n  const clusters: ProductCluster[] = [];\n\n  for (const candidate of candidates) {\n    // Search for the product on a canonical source\n    const canonicalResults = await tavily.search(\n      `${candidate.brand} ${candidate.modelFamily} specifications`,\n      { searchDepth: \"basic\", maxResults: 3 }\n    );\n\n    // Extract canonical product identifier\n    const canonicalId = extractCanonicalId(canonicalResults);\n\n    // Group by canonical ID\n    const existing = clusters.find(c => c.canonicalId === canonicalId);\n    if (existing) {\n      existing.members.push(candidate);\n    } else {\n      clusters.push({\n        canonicalId,\n        canonicalName: candidate.rawName,\n        members: [candidate],\n      });\n    }\n  }\n\n  return clusters;\n}\n```\n\n## Handling the Hard Cases\n\n### Product Lines vs Individual Products\n\"Roomba\" could mean the brand, the product line, or a specific model (Roomba j7+). We use context clues:\n\n- If a review discusses specific features (\"self-emptying base\"), it's likely a specific model\n- If it's a general comparison (\"Roomba vs Roborock\"), it's the product line\n- We maintain a hierarchy: Brand u2192 Line u2192 Model u2192 Variant\n\n### Regional Name Differences\nThe same product sometimes has different names in different markets. The Samsung Galaxy S24 is called \"Galaxy S24\" everywhere, but some accessories have region-specific names. We maintain an alias table for known cases.\n\n### Discontinued vs Current Models\nWhen someone searches \"AirPods Pro vs Sony,\" do they mean the current or previous generation? We default to current unless the query specifies otherwise, but we keep both generations in our database with clear generation markers.\n\n## Performance at Scale\n\nOur entity resolution pipeline processes ~5,000 product mentions daily across all review sources:\n\n| Metric | Value |\n|--------|-------|\n| Products in canonical database | 12,000+ |\n| Daily new mentions processed | ~5,000 |\n| Match accuracy (spot-checked) | 94.2% |\n| False positive rate | 1.8% |\n| Processing time (full pipeline) | ~12 minutes |\n| Most common false positive | Generation confusion (XM4 vs XM5) |\n\nThe 1.8% false positive rate is acceptable because our trust score system (covered in [Part 5](https://dev.to/danie_rozin/building-a-product-trust-score-from-50-review-sources-3cpi)) catches anomalies u2014 if a \"product\" suddenly has wildly inconsistent ratings, it's likely a merge error.\n\n## Lessons Learned\n\n1. **Don't build ML first.** Our three-layer heuristic approach handles 94%+ of cases. ML would marginally improve accuracy but massively increase complexity.\n2. **Brand matching must be exact.** Fuzzy brand matching creates catastrophic false positives.\n3. **Generation numbers are treacherous.** \"AirPods 3\" and \"AirPods Pro 3\" are different products. Always match model family before generation.\n4. **Maintain a manual override table.** Some matches are just weird. \"Galaxy Buds2 Pro\" vs \"Galaxy Buds 2 Pro\" (note the space) u2014 keep a list of known aliases.\n5. **Log everything.** When a match seems wrong in production, you need the matching pipeline's decision trail to diagnose why.\n\n## What's Next\n\nWe're exploring embedding-based matching for the long tail u2014 products where our heuristics fail because names are too dissimilar. Early experiments with product description embeddings show promise for matching across languages.\n\nSee entity resolution in action on [aversusb.net](https://www.aversusb.net) u2014 every comparison page unifies data from multiple sources under a single canonical product identity.\n\n---\n\n*Part 8 of our \"Building SmartReview\" series. Previous: [Part 7: People Also Ask Content Discovery](https://dev.to/danie_rozin/how-we-use-people-also-ask-to-generate-40-of-our-comparison-content-ideas-49j8)*","category":"business","tags":["ai","webdev","dataengineering","tutorial"],"url":"https://www.aversusb.net/blog/entity-resolution-product-matching","publishedAt":"2026-06-25T11:21:34.576Z","updatedAt":"2026-06-25T11:21:34.577Z","articleSchema":{"@context":"https://schema.org","@type":"BlogPosting","@id":"https://www.aversusb.net/blog/entity-resolution-product-matching#article","headline":"Entity Resolution at Scale: Matching Products Across Amazon, Reddit, and RTINGS","description":"\"AirPods Pro 2,\" \"Apple AirPods Pro (2nd Generation),\" \"AirPods Pro USB-C\" u2014 same product, three...","abstract":"\"AirPods Pro 2,\" \"Apple AirPods Pro (2nd Generation),\" \"AirPods Pro USB-C\" u2014 same product, three...","url":"https://www.aversusb.net/blog/entity-resolution-product-matching","image":{"@type":"ImageObject","@id":"https://www.aversusb.net/blog/entity-resolution-product-matching#primaryImage","url":"https://www.aversusb.net/api/og?title=Entity%20Resolution%20at%20Scale%3A%20Matching%20Products%20Across%20Amazon%2C%20Reddit%2C%20and%20RTINGS&type=blog","contentUrl":"https://www.aversusb.net/api/og?title=Entity%20Resolution%20at%20Scale%3A%20Matching%20Products%20Across%20Amazon%2C%20Reddit%2C%20and%20RTINGS&type=blog","width":1200,"height":630,"caption":"Entity Resolution at Scale: Matching Products Across Amazon, Reddit, and RTINGS"},"thumbnailUrl":"https://www.aversusb.net/api/og?title=Entity%20Resolution%20at%20Scale%3A%20Matching%20Products%20Across%20Amazon%2C%20Reddit%2C%20and%20RTINGS&type=blog","contentReferenceTime":"2026-06-25T11:21:34.577Z","datePublished":"2026-06-25T11:21:34.576Z","dateCreated":"2026-06-25T11:21:34.576Z","dateModified":"2026-06-25T11:21:34.577Z","author":{"@type":"Organization","@id":"https://www.aversusb.net/#organization","name":"A Versus B"},"publisher":{"@type":"Organization","@id":"https://www.aversusb.net/#organization","name":"A Versus B"},"inLanguage":"en-US","isPartOf":{"@type":"WebSite","@id":"https://www.aversusb.net/#website"},"keywords":"ai, webdev, dataengineering, tutorial","articleSection":"business","wordCount":1011,"license":"https://creativecommons.org/licenses/by/4.0/","speakable":{"@type":"SpeakableSpecification","cssSelector":["h1",".article-excerpt",".article-intro","#article-summary"]},"accessMode":["textual"],"accessModeSufficient":[{"@type":"ItemList","itemListElement":["textual"]}],"isAccessibleForFree":true}}