|
Problem:
A catalog had grown with one product page per color instead of one page per style. That split the SEO value, the review count, and the merchandising across hundreds of near duplicate pages. Merging them by hand was not possible at that scale, and a bad merge could break live URLs with real search traffic behind them.
Solution:
A Python pipeline that uses fuzzy title matching, SKU token grouping, and product keyword filtering to cluster hundreds of color duplicate pages into real product families. Executed as a staged rollout with a 301 redirect for every archived URL and a documented reason for every record that did not make the cut.
How it works:
The pipeline reads the full product catalog export, groups products by the second token in the SKU, and scores title similarity using difflib to handle wording differences and encoding issues across the catalog. The merge plan runs in three stages, a test batch, a retry batch, and the full rollout, each producing its own import, archive, and redirect file. After each merge, variant display order is normalized and a plain language summary is generated per merged group for the client to review before the next stage runs. Every record that did not merge lands in a separate file with a specific reason, not a silent drop. One bad merge that made it through, a SKU token collision between two genuinely different products, was caught from the review output and reversed with a targeted removal file before it affected live traffic.
Tools:
