Using Solr and Magento, it only takes me 45 minutes to run full text indexing against 50K enriched product records (color, weight, vendor, description, short description, title, nicknames, etc. - easily way more data than in a typical email for each product) And my box is not especially fast and does nothing in terms of clustering to improve performance. Now do that 13 times to arrive at approx 650k items, and it only takes approx 9 or 10 hours. Now you could run keyword searches against the entire lot to see if there is anything of interest. And that is BEFORE removing duplicates like Snowden suggested, or applying some Natural Language Processing algorithms, or any other relevant AI code... No conspiracy here, I think. Rather I think you see just who is truly out of touch with what modern tech can do.