We do a lot of SEO Audits here at Bowler Hat and one problem we see creeping up, again and again, are incorrectly indexed HTTPS pages.
This can present in a couple of ways but will most often mean that users clicking through from your site will see a yellow warning page before they reach your site which has one of two messages on that (scary looking) intermediate page.
Fortunately, there is a simple fix for those brave enough to dive into their .htaccess file and use some Mod Rewrite Wizardry!
Danger! Wrong Way! Turn back!
The messages you are likely to see are generally along the lines of:
- The site’s security certificate is not trusted!
- This is probably not the site that you are looking for!
- This connection is untrusted!
- Safari Can’t verify the identity of the website
There is some variance there but the core message is the same: site not trusted, proceed at risk. Likewise, the exact page will depend on the browser and OS but you will see something along the lines of:
The end result here is much the same and the browser can’t in good conscience advise the user to proceed to a site that is running on one URL but claiming to be another. The benchmark to use here is ‘would my mother click through to the page’ and as ever the answer is no so are you losing traffic and custom due to some incorrectly indexed pages on your site? What would happen if you were?
How does this happen?
Well, there are a bunch of ways this can happen but the likely candidates are:
- Shared hosting with a generic certificate
- An expired certificate
- Technical misconfiguration
Ultimately, what we see 99% of the time is that a shared hosting account has a generic HTTPS certificate on another domain that is presented when anyone visits one of your URLs via HTTPS instead of traditional HTTP. Now, this is not going to happen in the standard run of things but give it a try and change your URL to https://www.yoururl.co.uk and see if you get a screen like the above.
Users Implications
Now, this becomes a problem when a page gets indexed on an HTTPS URL. You may also then have one or more pages that become indexed on both HTTP and HTTPS. This creates a possible duplicate content scenario and even Webmaster Tools lets you add sites on both HTTP and https URLs so you can see how these could in principle be different sites.
The prompt for this article was actually stumbling across some changes caused by updates to the ever trusty Yoast WordPress SEO plugin – we ended up with a few problems due to the permalinks but also stumbled across one or two pages indexed as HTTPS rather than HTTP.
Here you can see our ‘What Google Wants‘ blog post is indexed as HTTP and HTTPS and is actually filtered from a query of our own site for that term (duplicate content is often filtered in this way). Not great and likely impacting the ability of the original article to rank itself (we had to repeat the search with the duplicates included to get this result).
So, if even us <sarcasm>god like SEO’s</sarcasm> can pick up issues hopefully you can see how easy things can go wrong and possibly cause you problems.
Have you got a problem?
First up – have you got a problem? If you do the simple site command and browse through the results you can see if you find any https results. If you have hundreds (thousands, millions) of pages then use a tool to scrape your indexed pages or register the HTTPS version with Webmaster Tools to save some time.
site:bowlerhat.co.uk
Browsing I see one HTTPS page so I can then search for that post:
site:bowlerhat.co.uk “what google wants”
I then have to click the ‘similar results’ link to get the filtered result in and see my two indexed articles.
A simple Fix – Redirect the HTTPS pages to HTTP
Fortunately, putting this right or putting a simple preventative measure in place is a 5-minute job (and certainly quicker than reading this blog post ). We can simply use some URL Rewriting to redirect any requests for HTTPS pages to the HTTP alternative. This will certainly prevent any HTTPS pages from getting indexed and deal with any existing problems given a little time.
The fix uses a Rewrite Rule in your .htaccess file to match all requests for an HTTPS page and permanently redirect (HTTP 301) them to the same page with an HTTP URL (protocol).
#Ensure rewriting engine is fired up RewriteEngine On
# deal with HTTPS indexed pages
RewriteCond %{ENV:HTTPS} on
RewriteRule (.*) http://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
A Simple Fix and Preventive Measure
In SEO an ounce of prevention is often worth a pound of cure and this approach will fix existing problems but will also protect against this becoming an issue. You will still have to watch out for hard coded links but the browsers tend to deal with this a lot better and often without the warning when the redirect is in place so it really is a simple win-win.
If you have any questions or how we can help then please get in touch, drop a comment and follow us on our social channels for more tips, advice and even the odd SEO joke or two (no, really =).
References and some more reading for the curious (geeky) amongst you:
