Don't lose Google traffic? Customize Opent for JavaScript and CSS files
The way Google views your site is per Monday, October 27, 2014 changed thoroughly. If you do nothing with this, you run the risk of losing your search positions in Google and therefore visitors. And with that also sales. So what has Google changed and how do you fix that?
How Google sees your website
Google views every website with Googlebot: fully automatic software that clicks on all links and indexes the content. Googlebot previously only saw texts, links and tags such as alt-image (Google does not recognize any images). That meant that Googlebot also saw no formatting (C.S.S) and additional applications (JavaScript).
Google now looks like a user
Googlebot now views your website like a human user: including CSS files and Javascripts. See announcement of Google Webmaster Central. It's an adjustment in Google's Technical Webmaster Guidelines.
Robots.txt for Googlebot
Which part of your website Googlebot can and cannot see is in robots.txt. That is an instruction card for Googlebot, and all other bots from other search engines. This also states whether Googlebot can see CSS and Javascript on your site.
Do I have a robots.txt?
That is a good question. More people should ask that! Here's the fastest way:
Go to your website. Such as www.bloeise.nl. In the address bar after the URL, type: /robots.txt. Then you get it www.bloeise.nl/robots.txt Press enter.
YES, you can do this for any website. Also for Google itself: www.google.nl/robots.txt.
Check Google Webmaster Tools
Does the above not work? Do not panic.
- Go in Google Webmaster Tools to your website. Have you not yet set GWT? View my Google Webmaster Tools manual.
- Go in the menu on the left to Crawl and then to Fetch as Google.
- Leave the URL field blank and leave type on Desktop. Just press Retrieve and View.
Go and update Facebook, check your mail or get a cup of coffee.
- Google ready? You have the option 'submit to index'. However, you click on / below URL
Fetch as Google
You will then see how Googlebot sees your site. Example:
The first column shows the URL, in the 2nd column it type, in the third the reason, then possibly robots.txt tester.
In this example, Javascripts are blocked by robots.txt. That is not good, because Google says:
Disallowing crawling or Javascript or CSS files in your site's robots.txt directly harms how well our algorithms render and index your content and can result in suboptimal rankings.
View robots.txt
Don't have you yet Robots.txt seen? click on the link robots.txt tester or go to the menu in Google Webmaster Tools Crawl and then to robots.txt tester. You can now immediately see how Google reads your robots.txt file.
Customize robots.txt to allow JS and CSS
The steps to give Google access to JS and CSS are as follows:
- Use Fetch as Google as described above.
- Does the example look good to a user? Then Google sees it well.
- Does it not look good? Then go through the lines one by one on the Robots.txt link click behind it and adjust robots.txt and press to send to click. Check after each line if the modification makes the example readable.
Now you have fixed the problem for Google. For the time being, other search engines do not yet look at CSS or JS. Read about it here all the possibilities of robots.txt.
Are you stuck? Forward this blog to your own webmaster / web builder with the Share buttons below this blog. This is covered by a service contract, but if you have paid once for creating your website and then no longer, your web builder can count the time it takes to adjust. That really cannot be more than an hour, depending on your website, cms and knowledge of the web builder.
Just one warning!!!!! (5 exclamation marks !!!!) When you adjust robots.txt, make sure that you do not have any at disallow / let alone. Then you say to Googlebot: you are not allowed to view the entire site. Your entire site will no longer be crawled by Googlebot and will therefore no longer be displayed in Google search.
Robots.txt for WordPress: set / wp content / on allow
In an earlier version of this blog I gave a WordPress example and recommended the folder / wp-includes / from disallow to allow.
Daniël Mulder from OmniLeads was kind enough to point out to me that although this folder contains Javascript, Fetch as Google is reporting this, but this is background Javascript which hardly affects readability. It is about the disallow of the folder / wp content / to delete in robots.txt: this contains javascripts and CSS files that have an impact on the display for the user and therefore for Google. In a standard WordPress installation this folder is already on allow. In some articles it is explicitly stated that / wp-content / plugins / to disallow. Daniel says about this:All content under the folder / wp content is intended as public content or user content and plug-ins and themes are included. I wouldn't put the plug-ins dir on disallow just like the theme folder. This is because it contains content that influences the rendering. "
Keep up with your website
You must keep a website up to date. Just like a car or a house. If you don't, your website may suddenly no longer rank in Google, your traffic will decline and therefore your turnover. Bloeise helps you keep up.
Give your reaction: