Google Webmaster Tools manual: register and solve problems

Google Webmaster Tools is a collection of free tools that you use to solve problems with your website and make your website easier to find for Google. As a user, you do not get a preferred position, but you do gain insight into how Google views your website. Are there errors? Can Google find all pages? Which keywords will you be found on? Which pages are popular? Google Webmaster Tools (GWT) helps you with this. Why does Google do that? Because Google benefits from as many well-functioning websites as possible. In this blog I explain all the different parts as a kind ultimate Google Webmaster Tools manual... but first a short introduction video.
Go to https://www.google.com/webmasters/tools/home?hl=nl. For this you need a standard Google account. You add your site with the red button at the top right of the Add site.
The next step can be difficult: verifying the website. The recommended method of site verification is via Google Analytics, and in addition, Google offers four alternative methods:
So many ways, of which via Google Analytics simply the easiest works.
I use Bloeise.nl myself. If you type in www.Bloeise.nl, the browser jumps to Bloeise.nl. You can add both domain names (verification is the same for both and it has no further effect), but I recommend setting your preferred domain. You then indicate to Google that MET or WITHOUT "WWW" is your preference. If you do not do this and both sites work equally well, Google thinks that you have 2 equal sites running. Every page sees Google twice, and Google does not know which domain should be displayed when. All pagerank then spreads, and that is bad for your findability. Google calls this duplicate content.
Go to the top right to the gear, and click Site Settings.
When you set your preferred domain, Google will:
The Dashboard consists of:
Now we have added and verified a site. Google Webmaster Tools is for two things:
I first address problem solving by handling site messages, crawl errors, crawl statistics, robots.txt tester, sitemaps, URL parameters, and security issues. Not the most interesting material, but important to make your site run flawlessly. Another way is to click through GWT yourself, view each menu item, and if you notice, continue reading Google's explanation. Yet it doesn't hurt to be aware of everything that can go wrong ... before the time comes and your site will no longer be found.
You need site messages always read. After verifying a website, you will also receive an email for each site message. They are often only announcements ("preffered domain changed" for example), but Google also informs you if something is really wrong with your website, for example: malware, Google penalty, "Googlebot has no access to your site" (so your site is not displayed in Google), "increase in the number of server errors", increase in the number of 404 reports, etc. All site messages can be read afterwards as a logbook.
Does Google not get access to your site? Then follow the following tips:
Furthermore, the server can temporarily be out (from your domain provider or hosting provider) or Google can make a small mistake itself.
Google visits and indexes your website with a small program: Googlebot. Every problem that Googlebot sees, Google Webmaster Tools reports such as Site errors and URL errors. Site errors are crawler mirrors where Googlebot could not visit the entire domain: the entire web site then gives an error. They are therefore important to resolve immediately. URL errors are specific to a specific URL. Google expects that a link from one site to another will result in a properly functioning web page. Every time a link ends on a non-existent page, it gives an error code (Google calls it a response code). Every error code represents a different problem. Do you have many error codes? Then Google thinks your site is bad. And so regular indexing is not worth it.
The following crawl errors are possible:
It is important to know that a 404 can occur even if another page refers to a non-existent page, for example due to a spelling mistake. When you click on http: /www.bloeise.nl/dezepaginabestaatniet , that gives me a 404 message. However, when you rename a page, for example from http://www.bloeise.nl/facebookmarketing to http://www.bloeise.nl/facebook-marketing, then each deep link to the first version will generate a 404 notification. Unless you link the old URL to the new URL with a 301 reference. A 301 reference is made by the website administrator on the server, and is only the setting: http://www.bloeise.nl/facebookmarketing = http://www.bloeise.nl/facebook-marketing. Even if you delete a page and it is still being referenced, it will result in a 404 message that you resolve with a 301 reference. Is your website updated? Then insist that ALL your old links be redirected to new links. That way you will remain easy to find in Google.
The server says that the page does exist, but it is clear to the user that the page does not exist. In that case it also becomes clear to the Googlebot: that page is not there. Soft 404 messages can cause problems with indexing (wrong page is being indexed). To be solved by having the server give a 404 message. See further explanation Soft 404 notifications.
Googlebot cannot continue. This can be due to Flash or Javascript. These are programming languages with which you can do very special things (active content), but which Googlebot cannot read as well. Another reason may be that a redirect has been placed incorrectly: http://www.bloeise.nl/facebookmarketing refers to http://www.bloeise.nl/facebook-marketing, which again refers to http://www.bloeise.nl/facebookmarketing. See explanation of Google on The error "URLs not followed" to remedy.
The URL does exist, but is not seen by Googlebot. For example, when a page can only be viewed after logging in. There are a number of solutions: just make the page freely accessible, give Googlebot access as a user (quite a thing to set up) or inform Googlebot that this page does not have to be viewed. Enter that URL in robots.txt as disallow. Even then an access denied notification will be issued, but you have set it yourself. Another reason may be that the hosting provider has blocked access.
Just like crawl errors, Crawl statistics are under the Crawl menu item. Here you can see the Googlebot activity on your website for the past 90 days:
It often has peaks and troughs. But if there really is a trend, such as a sharp fall, then something may be wrong. For example, duplicate content has been discovered on your site or a growth in error messages. It can also be the result of one Google Panda update: the quality of your site is insufficiently qualified. The Googlebot works with a 'crawl budget': a limited amount of time, bandwidth and server space. A site with many problems and low quality receives less crawling budget. Also how often new content appears has an impact on the crawl budget.
I mentioned the problem that "disallow: /" can give: Googlebot does not index your entire website. Another problem may be that all your images are not indexed (already seen at Joomla websites) while people also search for images. Such searches are very specific, and can therefore also be for specific products. But back: what does robots.txt do? It is an instruction for Googlebot and other search engine crawlers on how to read your website. A card: visit these pages or directories (allow) or not. (disallow) It can still happen that an external site does refer to a page that is blocked in robots.txt and therefore can be found via Google. Then move that page, or make the content accessible only with a password, or add one noindex meta tag in the head section of the page.
Robots.txt tester lets you view and edit your own robots.txt file. CMS directories are typically on disallow. It is common to also refer to your sitemap with robots.txt.
A sitemap tells the Googlebot what is on your website: a kind of menu. This way you point Google in a structured way at your content. Sounds important? Is it too. Google comes along faster, finds content faster and more complete. A separate sitemap is also possible for images and video. Under the sitemaps section in GWT you can see how many web pages and images there are in your sitemap, and how many are indexed. The indexation may be slightly behind. At the top right you add a new sitemap and you can also test it first:
So how do you get a sitemap.xml file? Typically a task for the website builder, as part of delivering your website. The sitemap must be updated automatically. For WordPress users there is a handy free plugin for this, WordPress SEO by Yoast (indeed from a Dutchman named Joost).
Note: the XML Sitemap must be created according to a specific layout: the Sitemap Protocol. There are countless validators for it Validome.org. Do you have a very large website with many web pages? Then you can in one Sitemap index file refer to multiple sitemaps.
For completeness: URL parameters. See the text below: "Googlebot currently has no problems with the coverage of your site .."? Continue, nothing can be seen here.
Why am I saying that? URL parameters is only for if you really know what you are doing. Do you think that you really do something with this, then I recommend to consult several sources and specialists, for example Idesthost, before you take action. I haven't had to deal with it myself. With URL parameters you explain to Googlebot that multiple URLs actually belong together as one URL. When is that necessary? With a session ID, or when a sorting results in a new URL, while the information is really the same. For example, cycle a webshop, with the same information on the red bikes and black bikes page. Only the photo differs. That would yield duplicate content, which is undesirable for your Google findability. If you use URL parameters incorrectly, you will lose visibility in Google on pages that were not affected. Not for nothing that Google explicitly points out to you with the above message.
You can hear security issues from Google via Site Messages. A problem may be that your website is hacked and malicious software (malware) is being distributed. No that's not good. If everything is correct, you will receive the following message:
If you are pointed out a security issue, it means that Google users will see a warning at your URLs when they are displayed in Google. Nobody clicks on it anymore. The following steps are recommended:
The next part, Google Webmaster Tools: easier to find in Google, I put on a next page.
Practical list of sources to help you use GWT better:
Google Webmaster Tools Google+ page
Here is a list of other Google support services:
Utility for testing structured data | Use the Structured Data Testing Tool to check if Google can correctly process and display your structured data format in the search results. |
Helper for structured data formatting | Not sure how to get started with adding structured data formatting to your HTML? Try this user-friendly utility. |
Tester for e-mail markers | Validate the structured data content of an HTML e-mail with the e-mail marker tester. |
Google Places | Research shows that 97% of consumers search the internet for local businesses. Make sure that your company is easy to find with Google Places: a free local platform from Google. |
Google Merchant Center | The place where you can upload your product data to Google and make it available for Google Shopping and other Google services. |
PageSpeed Insights | Use PageSpeed Insights to find out how you can quickly create your web pages on all devices. |
Custom search | Use the power of Google to create a customized search experience for your own website. |
This is the second blog of a series about the Basic knowledge of Online Marketing
A question that I get regularly: how do you give someone access to Google Analytics? And…
Google Webmaster Tools is for everyone who wants to be found in Google In the previous ...
Matt Cutts works as head of the Google webspam team. He regularly answers questions from ...
Ask? Comments? Give your reaction: