{"slug":"aggregating-product-reviews-with-ai","title":"Aggregating 50K+ Product Reviews with AI: What We Learned","excerpt":"How we built a review aggregation pipeline that processes 50K+ reviews from Reddit, Amazon, RTINGS, and 45+ other sources — entity resolution, sentiment extraction, and the guardrails that keep AI honest.","content":"When someone searches \"AirPods Pro 2 vs Sony WF-1000XM5,\" they don't want another opinion piece. They want the aggregate truth — what do *thousands* of real users actually think, across every source that matters?\n\nThat's what we built at [aversusb.net](https://www.aversusb.net/). Here's how the review aggregation pipeline works under the hood.\n\n## The Challenge: Reviews Are Messy\n\nProduct reviews exist everywhere — Amazon, Reddit, YouTube comments, RTINGS, Wirecutter, G2, Trustpilot. The data is:\n\n- **Scattered** across 50+ sources with different formats\n- **Inconsistent** — \"AirPods Pro 2\" vs \"Apple AirPods Pro (2nd Generation)\" vs \"APP2\"\n- **Noisy** — spam reviews, promotional content, outdated info\n- **Unstructured** — free text with no standard schema\n\nOur job: turn this chaos into structured, reliable comparison data.\n\n## Step 1: Entity Resolution\n\nBefore you can aggregate reviews, you need to know *which product* each review is about. This sounds trivial. It's not.\n\n```typescript\n// These all refer to the same product:\nconst aliases = [\n  \"AirPods Pro 2\",\n  \"AirPods Pro (2nd gen)\",\n  \"Apple AirPods Pro 2nd Generation\",\n  \"APP2\",\n  \"AirPods Pro USB-C\",\n  \"airpods pro 2022\",\n];\n```\n\nOur approach:\n\n1. **Seed entities** from manufacturer specs (canonical names, model numbers, SKUs)\n2. **Fuzzy matching** with normalized strings — strip punctuation, lowercase, expand abbreviations\n3. **AI disambiguation** — for edge cases, Claude determines whether \"Galaxy Buds\" in a 2024 review means Buds3, Buds2 Pro, or Buds FE based on surrounding context\n4. **Manual overrides** — a small lookup table for known tricky cases\n\nEntity resolution accuracy improved from ~78% to ~96% after we added the AI disambiguation step.\n\n## Step 2: Multi-Source Collection\n\nWe pull reviews from multiple source types:\n\n| Source Type | Examples | What We Extract |\n|-------------|----------|-----------------|\n| Structured | Amazon, G2, RTINGS | Ratings, pros/cons |\n| Semi-structured | Reddit, forums | Opinions in context |\n| Expert | Wirecutter, Tom's Guide | Detailed test results |\n| Social | YouTube, Twitter | Sentiment signals |\n\nEach source type gets a different weight in our aggregate scoring:\n\n- **Expert reviews** (Wirecutter, RTINGS): 2x weight — standardized testing\n- **Verified purchase reviews** (Amazon): 1.5x weight\n- **Community discussions** (Reddit): 1x weight — great for real-world usage patterns\n- **Social mentions**: 0.5x weight — signal, not substance\n\n## Step 3: Attribute Extraction\n\nRaw reviews are free text. We need structured attributes.\n\nFor earbuds, our target schema:\n\n- Sound quality (1-5)\n- ANC effectiveness (1-5)\n- Comfort (1-5)\n- Battery life (hours, verified against specs)\n- Value for money (1-5)\n- Build quality (1-5)\n\nWe use Claude to extract these from review text:\n\n```typescript\nconst prompt = `\n  Extract product attribute ratings from this review.\n  Only extract attributes explicitly mentioned.\n  Return null for attributes not discussed.\n\n  Review: \"${reviewText}\"\n  Product: \"${productName}\"\n\n  Return JSON: { soundQuality, anc, comfort, batteryLife, value, buildQuality }\n`;\n```\n\nKey insight: **always return null for unmentioned attributes**, never infer. A review that says \"great sound\" but doesn't mention ANC should not generate an ANC score. This avoids hallucinated data contaminating the aggregate.\n\n## Step 4: Sentiment Aggregation\n\nWith structured attributes from thousands of reviews, we compute weighted aggregates:\n\n```typescript\nfunction aggregateRatings(reviews: ExtractedReview[]): AggregateScore {\n  const weighted = reviews.map(r => ({\n    ...r,\n    weight: SOURCE_WEIGHTS[r.source] * recencyFactor(r.date)\n  }));\n\n  // Only include attributes with 10+ data points\n  const attributes = ATTRIBUTE_KEYS.filter(attr =>\n    weighted.filter(r => r[attr] !== null).length >= 10\n  );\n\n  return attributes.reduce((acc, attr) => {\n    const valid = weighted.filter(r => r[attr] !== null);\n    const sum = valid.reduce((s, r) => s + r[attr] * r.weight, 0);\n    const totalWeight = valid.reduce((s, r) => s + r.weight, 0);\n    acc[attr] = Math.round((sum / totalWeight) * 10) / 10;\n    return acc;\n  }, {});\n}\n```\n\nThe `recencyFactor` gives newer reviews more weight — a 2024 review about firmware-updated ANC is more relevant than a 2022 launch-day review.\n\n## What We Learned\n\n### 1. Source diversity beats volume\n\n500 Amazon reviews + 50 Reddit threads + 5 expert reviews gives a more accurate picture than 5,000 Amazon reviews alone. Each source captures different usage patterns and user segments.\n\n### 2. Negative reviews are more informative\n\nUsers who rate a product 3/5 tend to write the most detailed, attribute-specific reviews. Five-star reviews often say \"love it!\" (not useful). One-star reviews are often about shipping/returns (not product quality). Three-star reviews are gold.\n\n### 3. AI extraction needs guardrails\n\nWithout explicit instructions to return null for unmentioned attributes, Claude will sometimes infer ratings from context. \"These earbuds have great ANC\" does NOT imply anything about comfort. We added validation that rejects any extraction where >80% of attributes are rated — that's a sign the model is filling in blanks.\n\n### 4. Freshness matters more than you think\n\nProduct firmware updates can dramatically change the experience. The AirPods Pro 2 got significantly better ANC via software update months after launch. Our recency weighting captures this — static aggregation would miss it.\n\n## Results\n\nOur aggregated comparison data now powers thousands of comparison pages on [aversusb.net](https://www.aversusb.net/). Each page shows:\n\n- Weighted aggregate ratings across all sources\n- Attribute-by-attribute breakdown\n- Source count and freshness indicators\n- Confidence scores (higher when more sources agree)\n\nThe average comparison page references data from 15+ review sources and 500+ individual reviews.\n\n## Try It\n\nBrowse comparisons at [aversusb.net](https://www.aversusb.net/) — click any comparison to see the aggregated review data in action. A good place to start: [AirPods Pro 2 vs Sony WF-1000XM5](/compare/airpods-pro-2-vs-sony-wf-1000xm5).\n\nYou may also like our deep dives on [ChatGPT vs Claude](/compare/chatgpt-vs-claude) and [Anthropic vs OpenAI](/compare/anthropic-vs-openai) for more on the AI models powering this kind of work.\n\n## Conclusion\n\nAggregating reviews at scale is less about volume and more about source diversity, careful entity resolution, and disciplined extraction. The biggest gains came from refusing to let the model guess — null values are a feature, not a bug. If you're building something similar, start with the boring parts (entity resolution, source weighting) before you reach for fancier modeling. That's where the accuracy lives.\n","category":"technology","tags":["ai","machine-learning","reviews","engineering","comparison"],"url":"https://www.aversusb.net/blog/aggregating-product-reviews-with-ai","publishedAt":"2026-05-28T00:00:00.000Z","updatedAt":"2026-05-28T11:04:07.594Z","articleSchema":{"@context":"https://schema.org","@type":"BlogPosting","@id":"https://www.aversusb.net/blog/aggregating-product-reviews-with-ai#article","headline":"Aggregating 50K+ Product Reviews with AI: What We Learned","description":"How we built a review aggregation pipeline that processes 50K+ reviews from Reddit, Amazon, RTINGS, and 45+ other sources — entity resolution, sentiment extraction, and the guardrails that keep AI honest.","abstract":"How we built a review aggregation pipeline that processes 50K+ reviews from Reddit, Amazon, RTINGS, and 45+ other sources — entity resolution, sentiment extraction, and the guardrails that keep AI honest.","url":"https://www.aversusb.net/blog/aggregating-product-reviews-with-ai","image":{"@type":"ImageObject","@id":"https://www.aversusb.net/blog/aggregating-product-reviews-with-ai#primaryImage","url":"https://www.aversusb.net/api/og?title=Aggregating%2050K%2B%20Product%20Reviews%20with%20AI%3A%20What%20We%20Learned&type=blog","contentUrl":"https://www.aversusb.net/api/og?title=Aggregating%2050K%2B%20Product%20Reviews%20with%20AI%3A%20What%20We%20Learned&type=blog","width":1200,"height":630,"caption":"Aggregating 50K+ Product Reviews with AI: What We Learned"},"thumbnailUrl":"https://www.aversusb.net/api/og?title=Aggregating%2050K%2B%20Product%20Reviews%20with%20AI%3A%20What%20We%20Learned&type=blog","contentReferenceTime":"2026-05-28T11:04:07.594Z","datePublished":"2026-05-28T00:00:00.000Z","dateCreated":"2026-05-28T00:00:00.000Z","dateModified":"2026-05-28T11:04:07.594Z","author":{"@type":"Organization","@id":"https://www.aversusb.net/#organization","name":"A Versus B"},"publisher":{"@type":"Organization","@id":"https://www.aversusb.net/#organization","name":"A Versus B"},"inLanguage":"en-US","isPartOf":{"@type":"WebSite","@id":"https://www.aversusb.net/#website"},"keywords":"ai, machine-learning, reviews, engineering, comparison","articleSection":"technology","wordCount":982,"license":"https://creativecommons.org/licenses/by/4.0/","speakable":{"@type":"SpeakableSpecification","cssSelector":["h1",".article-excerpt",".article-intro","#article-summary"]},"accessMode":["textual"],"accessModeSufficient":[{"@type":"ItemList","itemListElement":["textual"]}],"isAccessibleForFree":true}}