The Sitemap Heist
Lift a competitor's entire programmatic-SEO engine off one public file.
Every company hands Google a map of its whole site. It has to. That is what a sitemap is for. Almost nobody treats that map as competitive intelligence, even though it sits one command away and spells out every page-generation strategy the company runs. This is the first play in the book because it is the purest. One file, thirty seconds, and you are holding a competitor's growth engine in your hands.
What to do: curl -s site.com/robots.txt | grep -i sitemap, then open the sitemaps it lists.
Why it works: The sitemap is the blueprint a company hands Google. Every programmatic template and URL pattern is in there, exposed.
Example: Zapier's robots.txt points to a two-service sitemap that runs to roughly 135,000 "Connect [App A] to [App B]" pages. Their entire programmatic-SEO engine, for free.
Walk it through
I ran this against zapier.com in July 2026. Here is exactly what came back.
1. Ask the site where its sitemaps live.
curl -s https://zapier.com/robots.txt
The robots file does two useful things at once. It lists every sitemap, and it tells you which crawlers the company cares about.
Eleven sitemaps came back. Two of them are the whole game. one-service is every "[App] integrations" page, and two-service is every "[App A] plus [App B]" pair. Look at the bottom of the file too. Zapier explicitly allows GPTBot, ClaudeBot, PerplexityBot, and Google-Extended, which tells you they want to be cited by AI answers, not only ranked by Google. That is a strategy decision sitting in a plain text file.
2. Open the two-service sitemap. It is an index, not a list.
It does not hand you 135,000 URLs in one file. It hands you a paginated index, ?p=2, ?p=3, and on and on, each child holding thousands of app-pair URLs. Count the pages, multiply by the URLs per page, and you have the size of the bet.
3. Open one page and read the template.
Every URL follows one pattern: /apps/[App A]/integrations/[App B]. Here is what zapier.com/apps/slack/integrations/google-sheets actually renders.
That is a complete, polished landing page. Headline, a trigger-and-action builder, proof points, a call to action. And Zapier has one for every pair of apps it supports. Nobody wrote 135,000 of these by hand. They wrote one template and filled it from a database of apps.
The read
Three numbers tell you almost everything about a competitor's SEO strategy, and all three are in the sitemap.
- Size is how big the bet is. 135,000 pages is a company betting a real slice of the business on programmatic search.
- Pattern is the template.
[App A] × [App B]is the exact page they generate, so you can generate the same shape for your own catalog. - Change over time is direction. Re-run this next month and diff the count. Growing means they are still pouring pages in. Flat means they have stopped, which is your opening.
Steal it
You do not need Zapier's catalog to run Zapier's play. You need any list that pairs off. If your product connects things, tools, cities, job titles, currencies, plan tiers, you have a two-service sitemap of your own waiting to be built. Pick the two variables, write one genuinely good template page, and generate the long tail. The company you just curled did exactly that, and you can read the whole strategy off one file before you write a line of your own.
Gotchas
- Filenames are often hashed. Zapier's sitemaps carry a random string in the path, so you will never guess
/sitemap.xml. Read robots.txt first. It is the only reliable index. - Being in the sitemap is not the same as ranking. A URL being generated does not mean it earns traffic. Before you copy a strategy wholesale, spot-check a handful of the competitor's pages in real search results to confirm the play is working, not just running.
- Respect the rules you are reading. This is public and meant to be served, but the same file lists the paths the company asks crawlers to leave alone. Reading the map is fair game. Hammering disallowed paths at scale is not.