How to Scrape Websites

If you’ve needed to extract information from a website, and ended up copy and pasting data from the webpage to your spreadsheet, you probably should have been scraping anything that took more than 2 minutes. Scraping a website is particularly useful if you need...
How to Create XML Sitemaps

How to Create XML Sitemaps

XML sitemaps are pretty well known within the SEO community but their purpose and value are often misunderstood, so let’s start with the basics. In this post, we’ll look at how to create XML sitemaps strategically. Shortly I’ll be posting a follow up...
How to Add the Canonical Tag to HTTP Headers

How to Add the Canonical Tag to HTTP Headers

Typically the canonical tag is deployed to the <head> section of a website, ex: <link rel="canonical" href="https://geoffkenyon.com/how-to-add-canonical-tag-to-http-headers"> This works fine for most use cases and there are many plugins for popular CMS...
How to Remove PDFs From Google Search

How to Remove PDFs From Google Search

Typically when you want to remove a webpage from Google, you can just add the meta robots noindex tag to the <head> section of the webpage. Once this is done you want to get Google to recrawl the webpage in order to pick up the noindex tag and then remove the...
How to Use Wildcards in Robots.txt

How to Use Wildcards in Robots.txt

The robots.txt file is used to control what content search engines are allowed to access on your site. This is great for controlling duplicate content and for directing your crawl budget to your most important pages. It is important to understand that if you you have...
Is Google Sitelinks Search Box Hurting You

Is Google Sitelinks Search Box Hurting You

On September 5th, Google announced their “new and improved” sitelinks search box (blog post announcement). I’m sure that this was a well intentioned action on Google’s part, but it may not be beneficial for you. This experience will steal branded traffic,...