Crawlability/indexability: Your sitemap and robots.txt with each other explain to Google Anything you do and don’t want Google to crawl and index. For those who have many pages that have precisely the same facts, check out creating a redirect from non-chosen URLs to the URL that greatest represents that data. https://www.youtube.com/watch?v=hgyQy18ZcIs