Crawl Budget Analysis for Large Casino Platforms
When crawl budget is a real constraint for casino platforms, how to prove it, and the fixes Google documents.
An iGaming sitemap should list only the canonical, indexable URLs you want in search results, split by template so each file stays under 50,000 URLs and 50MB uncompressed. Google ignores priority and changefreq, and uses lastmod only when it is consistently accurate, so generate lastmod from real content changes.
A sitemap is a list of URLs you want search engines to know about. Google's sitemap overview is careful about its limits: submitting one "doesn't guarantee that all the items in your sitemap will be crawled and indexed." It is a discovery aid (internal links still matter; see orphan pages) and, in Google's canonicalization guide, "a weak signal that helps the URLs that are included in a sitemap become canonical."
On a casino or sportsbook platform the sitemap does real work, because inventory changes constantly. New games launch weekly, events open and close daily, and promotions come and go. Internal links reach these pages eventually; a sitemap with accurate dates tells Google about them sooner.
Google's build-sitemap page says: "All formats limit a single sitemap to 50MB (uncompressed) or 50,000 URLs." Beyond that, use a sitemap index file, which Google says you can submit as a single file. The sitemaps.org protocol adds that an index "may not list more than 50,000 Sitemaps".
We recommend splitting by template, not alphabetically or by date, because it makes monitoring possible. Search Console reports submitted and indexed counts per sitemap, so a file per template shows at a glance whether game pages or event pages are the ones falling out.
<!-- constructed example: sitemap index for casino.example -->
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<sitemap><loc>https://casino.example/sitemaps/games-1.xml</loc></sitemap>
<sitemap><loc>https://casino.example/sitemaps/providers.xml</loc></sitemap>
<sitemap><loc>https://casino.example/sitemaps/guides.xml</loc></sitemap>
<sitemap><loc>https://casino.example/sitemaps/promotions.xml</loc></sitemap>
</sitemapindex>
Google's instruction is short: "Include the URLs in your sitemap that you want to see in Google's search results", using "fully-qualified, absolute URLs". In practice that means each URL must pass all of these tests:
Leave out account pages, cashier flows, search result pages, filter combinations you block in robots.txt, and expired events or promotions. A sitemap full of redirects and 404s does not break anything, but it wastes crawling (see crawl budget) and makes the indexed-versus-submitted numbers useless for monitoring. Listing parameter variants also sends mixed canonical signals.
This is where most platform sitemaps go wrong. Google states that it "ignores <priority> and <changefreq> values" and uses <lastmod> only when it is "consistently and verifiably" accurate. The crawl budget guide also recommends lastmod for updated content.
Common mistakes on casino builds:
Generate lastmod from the content record: the time the game description, rules, RTP information, terms or main copy changed. Use the W3C Datetime format that sitemaps.org specifies. If you cannot produce a trustworthy date, leaving lastmod out is better than sending a misleading one.
Event pages live for days. If they deserve to be indexed at all, they need to enter the sitemap when they are published and leave it when the event ends. A separate, frequently regenerated events.xml keeps this churn away from stable game and guide sitemaps. When the event ends, decide on one policy and apply it consistently: a 301 to the competition page, or a 404 or 410. Google's crawl budget guide recommends "a 404 or 410 status code for permanently removed pages."
Each subdomain is a separate site for crawling; see sportsbook subdomain indexing.
By default a sitemap "affects only descendants of the parent directory" unless submitted in Search Console. For groups that host sitemaps centrally, Google says: "To submit cross-site sitemaps that are hosted in a single location, you can either use Search Console or robots.txt", and for Search Console you must have "verified ownership of all the sites that you will add in the sitemap." sitemaps.org documents the robots.txt route as pointing the robots.txt on one host to a sitemap on another.
For multi-market brands, hreflang annotations can live in the sitemap instead of the page head. Google's localized versions guide says each <url> must list every alternate "including itself", and that the HTML, header and sitemap methods are "equivalent from Google's perspective". Pick one method per site to avoid conflicts. Our hreflang guide covers the details.
Google's build-sitemap page lists Search Console, the Search Console API and a Sitemap: line in robots.txt as the ways to tell Google about a sitemap. The old sitemap "ping" endpoint was announced as going away in a June 2023 Search Central post and is no longer listed. Then monitor:
Not with Google. Its documentation says Google ignores both values. Spend the effort on an accurate lastmod instead, which Google uses when it is consistently and verifiably accurate.
No. List only URLs you want in search results. Remove events when they end and give the old URL a deliberate response, such as a redirect to the competition page or a 404 or 410.
Yes, if Google can confirm you control them. Google accepts cross-site sitemaps through Search Console when you have verified ownership of every site listed, or through a Sitemap line in each site's robots.txt.
Checked on October 25, 2026. Search engine documentation changes; the official page prevails over this summary.
Corrections: none since publication. Report an error to hello@casinocrawlsignal.online.