Audit the website for SEO issues #286

rviscomi · 2019-10-31T21:46:10Z

Since we have SEO experts on-hand it'd be great if we can make sure that we're following our own best practices on the website.

The version pushed to https://almanac.httparchive.org has most of the site structure except for the chapter content (the most important part) so if needed we can revisit the website when that's pushed in ~a week or we can audit the local website. The website is also missing translations, so we won't be able to check for that class of SEO issues yet.

CC @ymschaap @rachellcostello @AVGP @clarkeclark @andylimn @voltek62 @AymenLoukil @catalinred

AymenLoukil · 2019-10-31T21:48:17Z

Already started ;)

AymenLoukil · 2019-11-01T09:38:07Z

I made a first check pass and wrote down my findings in draft mode. I'll create an issue for each one + details on how to correct / enhance :)
Here is the draft document : feel free for every one who want to add things
https://docs.google.com/document/d/1k_HnRaS1portw8RzvA7CRQTTxzn4VcfkT28VFz3zl6M/edit?usp=sharing

tunetheweb · 2019-11-01T10:36:12Z

http => https 307 temporary redirect should be 301 one

This one is not quite correct. 307 is because of HSTS which the browser has cached. It’s an internal browser redirect added by your browser. If you use an online tool like https://wheregoes.com/ on http://almanac.httparchive.org then you’ll see it does a 302 to https://almanac.httparchive.org/ (this should probably be a 301) and then a further 302 https://almanac.httparchive.org/en/2019/ (which probably needs to stay as a 302 because next year it will redirect to the 2020 directory so it is a temporary redirect -albeit a year long temporary redirect).

AymenLoukil · 2019-11-01T11:16:39Z

Yeah, the point is http=>https should be 301 and not 302, 307 or any other redirection type.

AymenLoukil · 2019-11-01T11:18:08Z

The second 302 is not because 1 year redirect. It is mainly for language detection. When the translations will be added, if your accept-language is for example es, it should redirect to /es/2019 with 302.

rachellcostello · 2019-11-01T11:57:42Z

I ran a crawl of the site in its current state, and everyone can access the findings here.

In addition to the points in @AymenLoukil's Google doc:

Multiple H1 tags on the home page and methodology page (not a huge issue for SEO but would recommend maintaining one H1 per page and moving the rest to H2s / H3s etc for more clarity for screen readers)
Missing Twitter cards and Open Graph tags

I'll schedule another crawl when the chapters have been added in to look for any other issues, and am happy to run more as we go along and share those as and when needed.

catalinred · 2019-11-01T12:09:14Z

I made a first check pass and wrote down my findings in draft mode. I'll create an issue for each one + details on how to correct / enhance :)
Here is the draft document : feel free for every one who want to add things
https://docs.google.com/document/d/1k_HnRaS1portw8RzvA7CRQTTxzn4VcfkT28VFz3zl6M/edit?usp=sharing

@AymenLoukil In addition to your doc, the rel=nooopener for external links should be added only when you open another page using target=_blank - in this case within the contributors page.

tunetheweb · 2019-11-01T13:50:39Z

Yeah, the point is http=>https should be 301 and not 302, 307 or any other redirection type.

Agreed but my point is the 307 isn’t a real redirect served by the server. It’s a fake one generated by Chrome to represent the automatic upgrade to HTTPS that HSTS does (see here: https://www.seroundtable.com/googlebot-hsts-redirects-301-307-21405.html). So you can’t “fix” this (and in fact there is nothing to fix). But you are 100% right about the first 302 (that those clients who have not loaded the HSTS instruction will use - including GoogleBot) - which is basically the point you were making! 😀

AymenLoukil · 2019-11-04T11:23:33Z

Yeah, the point is http=>https should be 301 and not 302, 307 or any other redirection type.

Agreed but my point is the 307 isn’t a real redirect served by the server. It’s a fake one generated by Chrome to represent the automatic upgrade to HTTPS that HSTS does (see here: https://www.seroundtable.com/googlebot-hsts-redirects-301-307-21405.html). So you can’t “fix” this (and in fact there is nothing to fix). But you are 100% right about the first 302 (that those clients who have not loaded the HSTS instruction will use - including GoogleBot) - which is basically the point you were making!

Yeah yeah i confirm @bazzadp ! You right on the 307. I went too fast on this :)

rviscomi · 2019-11-04T23:35:22Z

I'll schedule another crawl when the chapters have been added in to look for any other issues, and am happy to run more as we go along and share those as and when needed.

@rachellcostello two chapters are currently live and ready for another audit: Markup and Performance.

rachellcostello · 2019-11-05T10:58:54Z

The updated crawl can be accessed here.

It confirms that the canonical tags are being picked up and the author link errors have been resolved.

Page reports for the Markup and Performance chapters aren't showing any issues, which is good!

tunetheweb · 2019-11-06T22:37:23Z

OK so at the moment the issues I see are:

No Sitemap - being tracked in Create a sitemap.xml file #318 about to be fixed with Auto generate sitemap #345
No alt for images - should be completed in each chapter edit.
Add nooopener for external links - more a security issue, than an SEO one and we don't hold secure information so not sure it's worth it.
Add native lazy loading for all the imgs / iframes - I like this, but not sure how feasible it is. More a performance recommendation than SEO (and yes I know good performance is good for SEO)
Static assets browser cache - checked a few of these out manually and looks fine to me
http => https 307 temporary redirect should be 301 one - @rviscomi not show how this is configured on production? Don't think it's a biggy as this will be https all the way.

So looks to me that once we add the Sitemap then hopefully there is no further work required here other than rerunning a scan when full site is launched. So let's hold it open until then but wanted to make sure there was no expectation that any other work was ongoing here. Let me know if there is!

AymenLoukil · 2019-11-06T22:42:13Z

Add native lazy loading for all the imgs / iframes
=> https://web.dev/native-lazy-loading/

tunetheweb · 2019-11-06T22:44:57Z

Add native lazy loading for all the imgs / iframes
=> https://web.dev/native-lazy-loading/

Raised #351 for this.

AymenLoukil · 2019-11-06T22:58:19Z

Add nooopener for external links - more a security issue, than an SEO one and we don't hold secure information so not sure it's worth it.

It is a security + performance issue :)

AymenLoukil · 2019-11-08T11:30:42Z

I'm running a new check to validate the new implementations :) @rviscomi @bazzadp

rviscomi · 2019-11-10T10:18:53Z

Hmm. Just got this email from Google Search Console:

Google has detected that some pages on your site have implemented the rel-alternate-hreflang tag incorrectly. In particular, there seems to be a problem with incorrect language and region codes, or incorrect bi-directional linking (if page A links with hreflang to page B, there must be a link back from B to A as well). Google uses the hreflang attributes to serve the correct language or regional URL to the right users in search results.

Until you correct the errors on the hreflang links, your website will not benefit from additional language or region targeting.

I also submitted the sitemap.xml file and it returned a few errors similar to this:

An invalid date was found. Please fix the date or formatting before resubmitting.

Line 52
Parent tag: url
Tag: lastmod

Visiting the file itself shows an error page: https://almanac.httparchive.org/sitemap.xml

AymenLoukil · 2019-11-10T10:26:05Z

For the sitemap

tunetheweb · 2019-11-10T10:41:55Z

OK will fix that with my sitemap fix PR.

tunetheweb · 2019-11-11T06:35:23Z

Any ideas if these are resolved now? Guessing the Sitemap is, but does it list hreflang issues in GSC so can see if fixed there (may not be until it’s next crawled)?

rviscomi · 2019-11-11T06:51:24Z

Not sure. I wasn't able to view the error anywhere other than the vague email. I'll keep an eye out for any more vague emails.

rviscomi · 2019-11-11T19:12:19Z

One of the great things about staffing CDS is that there are real live humans who work on Search Console here! They helped identify the hreflang issue we were seeing and pointed me to this support doc: https://support.google.com/webmasters/answer/189077?hl=en

International Targeting | Language > 'en' - no return tags
URLs for your site and alternate URLs in 'en' that do not have return tags.

Missing return links: If page X links to page Y, page Y must link back to page X. If this is not the case for all pages that use hreflang annotations, those annotations may be ignored or not interpreted correctly.

It's still kind of unclear what SC is complaining about, but maybe

<link rel="alternate" href="https://almanac.httparchive.org/en/2019/" hreflang="en" />

should be

<link rel="alternate" href="https://almanac.httparchive.org/en/2019/" hreflang="x-default" />

tunetheweb · 2019-11-11T19:14:00Z

Could it be you shouldn’t implement it unless you have more than one language?

AymenLoukil · 2019-11-11T20:11:09Z

It's still kind of unclear what SC is complaining about, but maybe

<link rel="alternate" href="https://almanac.httparchive.org/en/2019/" hreflang="en" />

should be

<link rel="alternate" href="https://almanac.httparchive.org/en/2019/" hreflang="x-default" />

No @rviscomi.

We implemented it right. Just as i said earlier, modify en-US to en. and paths must be absolute. That's it. Every page should reference itself + reference the other languages (if they exist). And if A reference B, B should also reference A.

The x-default one is for telling what's the default page (if we are an international company, it could be the language selector page). It is optional but we could add it.

Could you please resubmit the website in SC and ask for indexing. + submitting the Sitemap.xml ?

rviscomi · 2019-11-11T20:54:40Z

Ok so this <html lang="en-US"> should be this <html lang="en">?

tunetheweb · 2019-11-11T20:55:30Z

https://www.seroundtable.com/one-language-hreglang-google-23970.html

I really think we should add an if statement around this so it doesn't show until the second language is launched. It makes no sense to have it as of now.

tunetheweb · 2019-11-11T20:57:32Z

It's a two line change to base.html:

    {% if supported_languages | length > 1%}
    {% for l in supported_languages %}
    <link rel="alternate" href="https://almanac.httparchive.org{{ url_for(request.endpoint, **get_view_args(lang=l.lang_code)) }}" hreflang="{{ l.lang_code}}" />
    {% endfor %}
    {% endif %}

tunetheweb · 2019-11-11T21:01:46Z

Ok so this <html lang="en-US"> should be this <html lang="en">?

Both versions are fine: https://support.google.com/webmasters/answer/189077?hl=en.

However agree we should just do the language and not the country since we are unlikely to have en-GB or similar so this will be the only en version.

AymenLoukil · 2019-11-11T22:25:29Z

Ok so this <html lang="en-US"> should be this <html lang="en">?

Yes :)

AymenLoukil · 2019-11-11T22:26:43Z

https://www.seroundtable.com/one-language-hreglang-google-23970.html

I really think we should add an if statement around this so it doesn't show until the second language is launched. It makes no sense to have it as of now.

OK for that. When we implemented the hreflang tags, i thought that we will have at least another language at launch (es). So why not hide them until we publish new language.

tunetheweb · 2019-11-12T10:10:55Z

Ok so this <html lang="en-US"> should be this <html lang="en">?

Yes :)

Agree, as per here: https://support.google.com/webmasters/answer/189077?hl=en:

If you have several alternate URLs targeted at users with the same language but in different locales, it's a good idea also to provide a catchall URL for geographically unspecified users of that language. For example, you may have specific URLs for English speakers in Ireland (en-ie), Canada (en-ca), and Australia (en-au), but should also provide a generic English (en) page for searchers in, say, the US, UK, and all other English-speaking locations. It can be one of the specific pages, if you choose.

We have no en page so in theory, only US people will currently be shown that page! As we have no other English pages, we should remove the -US completely.

Included a fix for that in 5e9beba

In that, I say that we support region codes, but I'm not 100% sure we do since directory structure is based on language only, so probably more clean up we could do but that will do for now and can look again when we have more languages and/or regions as difficult to test with just the one we have now.

The x-default one is for telling what's the default page (if we are an international company, it could be the language selector page). It is optional but we could add it.

Included in 0810d2d

Could do with thoroughly testing this when first additional language goes live! Should we raise an issue for that separate to this one?

rviscomi · 2019-11-13T20:33:27Z

The hreflang issue appears to have been resolved.

@AymenLoukil do you feel comfortable closing this issue?

AymenLoukil · 2019-11-13T20:45:14Z

@rviscomi good!
I'm on my way back from London. Could I have more time to make global sanity / qa check?

rviscomi · 2019-11-13T20:46:17Z

Of course! Thank you!

AymenLoukil · 2019-11-14T15:12:13Z

Ok i did checked again the SEO aspects of the website.

Figures images don't have decriptive file names and no alt values. It could be better to give them good ones. Here is the list :

The rest LGTM

AymenLoukil · 2019-11-14T16:01:18Z

Also, contributors page is shuffling on reload which is not the best choice from SEO / crawl POV.

tunetheweb · 2019-11-25T21:26:14Z

Also, contributors page is shuffling on reload which is not the best choice from SEO / crawl POV.

Yeah I don't think it's a big deal, and it's nicer to our contributors so I say we accept this.

@AymenLoukil are we good to close this issues? Alt-tags is being tracked in #379 and rest have either been dealt with or can be accepted IMHO.

AymenLoukil · 2019-11-25T21:31:49Z

What about the figures files names ?

Also, contributors page is shuffling on reload which is not the best choice from SEO / crawl POV.

Yeah` I don't think it's a big deal, and it's nicer to our contributors so I say we accept this.

You create a separate issue ?

tunetheweb · 2019-11-25T21:36:19Z

What about the figures files names ?

I think it's a minor thing. And having done the last big bulk update not too inclined to do another!

I care much more about the content being SEO-friendly than the images anyway. Another item could be that we should watermark all the images with "© HTTP Web Almanac 2019" in case anyone uses them, but again I think that's one for next year unless anyone takes that on?

Also, contributors page is shuffling on reload which is not the best choice from SEO / crawl POV.

Yeah I don't think it's a big deal, and it's nicer to our contributors so I say we accept this.

You create a separate issue ?

Nope. I'm saying we accept it. This was a conscious decision and I don't see the big SEO lose with it to be honest.

AymenLoukil · 2019-11-25T21:45:11Z

I think it's a minor thing. And having done the last big bulk update not too inclined to do another!
I care much more about the content being SEO-friendly than the images anyway.

Visual search is so important and one of the 3 pillars of Google search announced on its 20th birthday. Images are part of the content and text is not more or less important. Having a descriptive/SEO file name is one of the basics of images SEO.

And having done the last big bulk update not too inclined to do another!

I understand. We could imagine make use of the figure title to slugify it for the image filename.
example : Figure 2. Distribution of JavaScript per page by device. => distribution-of-javascript-per-page-by-device.png

Nope. I'm saying we accept it. This was a conscious decision and I don't see the big SEO lose with it to be honest.

There is no big SEO loose here. The fact is we are making Google receive a different answer on each time it crawls the page. but it is ok.

tunetheweb · 2019-11-25T22:09:16Z

Visual search is so important and one of the 3 pillars of Google search announced on its 20th birthday. Images are part of the content and text is not more or less important. Having a descriptive/SEO file name is one of the basics of images SEO.

Personally I think the effort involved - particularly with the translations well underway now - far outweighs any benefit here.

To prove the point I just Googled with what I think might be a typical search request that one of the title might help with:

As you can see we are already as the 3rd and 5th search results. I really don't think there will be a noticeable improvement from this by renaming the image files. They are already loaded with context from text around them, alt attributes, figcaptions and aria descriptions. That's a wealth of SEO right there and the file names will add little beyond this but require a lot of effort and disruption of translations.

rviscomi · 2019-11-26T16:24:40Z

I think this issue is ok to close and we can open new issues for anything SEO related that comes up.

rviscomi added question Further information is requested development Building the Almanac tech stack SEO SEO related labels Oct 31, 2019

rviscomi added this to the SHIP IT! milestone Oct 31, 2019

rviscomi assigned AymenLoukil Oct 31, 2019

tunetheweb mentioned this issue Nov 2, 2019

SEO fixes for canonical and meta info (inc Meta Descriptions, OG, Twitter and Article Structured Data) #302

Merged

rviscomi mentioned this issue Nov 2, 2019

Audit the website for accessibility issues #303

Closed

5 tasks

tunetheweb mentioned this issue Nov 3, 2019

SEO Fixes part 2 #308

Merged

This was referenced Nov 4, 2019

Create a sitemap.xml file #318

Closed

Seo fixes3 #320

Merged

rviscomi mentioned this issue Nov 10, 2019

Audit the website for performance issues #413

Closed

5 tasks

tunetheweb mentioned this issue Nov 10, 2019

Sitemap date fix #419

Merged

rviscomi mentioned this issue Nov 11, 2019

Investigate 404 errors #452

Closed

2 tasks

tunetheweb mentioned this issue Nov 11, 2019

Hide rel="alternative" xreflang if only one language exists #474

Merged

rviscomi mentioned this issue Nov 14, 2019

Maintenance plan (please read) #488

Closed

rviscomi closed this as completed Nov 26, 2019

Audit the website for SEO issues #286

Audit the website for SEO issues #286

Comments

rviscomi commented Oct 31, 2019

AymenLoukil commented Oct 31, 2019

AymenLoukil commented Nov 1, 2019 • edited Loading

tunetheweb commented Nov 1, 2019

AymenLoukil commented Nov 1, 2019

AymenLoukil commented Nov 1, 2019

rachellcostello commented Nov 1, 2019

catalinred commented Nov 1, 2019 • edited Loading

tunetheweb commented Nov 1, 2019 • edited Loading

AymenLoukil commented Nov 4, 2019

rviscomi commented Nov 4, 2019 • edited Loading

rachellcostello commented Nov 5, 2019

tunetheweb commented Nov 6, 2019

AymenLoukil commented Nov 6, 2019

tunetheweb commented Nov 6, 2019

AymenLoukil commented Nov 6, 2019 • edited Loading

AymenLoukil commented Nov 8, 2019

rviscomi commented Nov 10, 2019 • edited Loading

AymenLoukil commented Nov 10, 2019

tunetheweb commented Nov 10, 2019

tunetheweb commented Nov 11, 2019

rviscomi commented Nov 11, 2019

rviscomi commented Nov 11, 2019 • edited Loading

tunetheweb commented Nov 11, 2019

AymenLoukil commented Nov 11, 2019 • edited Loading

rviscomi commented Nov 11, 2019

tunetheweb commented Nov 11, 2019

tunetheweb commented Nov 11, 2019

tunetheweb commented Nov 11, 2019

AymenLoukil commented Nov 11, 2019

AymenLoukil commented Nov 11, 2019

tunetheweb commented Nov 12, 2019 • edited Loading

rviscomi commented Nov 13, 2019

AymenLoukil commented Nov 13, 2019

rviscomi commented Nov 13, 2019

AymenLoukil commented Nov 14, 2019

AymenLoukil commented Nov 14, 2019 • edited Loading

tunetheweb commented Nov 25, 2019

AymenLoukil commented Nov 25, 2019

tunetheweb commented Nov 25, 2019 • edited Loading

AymenLoukil commented Nov 25, 2019

tunetheweb commented Nov 25, 2019 • edited Loading

rviscomi commented Nov 26, 2019

AymenLoukil commented Nov 1, 2019 •

edited

Loading

catalinred commented Nov 1, 2019 •

edited

Loading

tunetheweb commented Nov 1, 2019 •

edited

Loading

rviscomi commented Nov 4, 2019 •

edited

Loading

AymenLoukil commented Nov 6, 2019 •

edited

Loading

rviscomi commented Nov 10, 2019 •

edited

Loading

rviscomi commented Nov 11, 2019 •

edited

Loading

AymenLoukil commented Nov 11, 2019 •

edited

Loading

tunetheweb commented Nov 12, 2019 •

edited

Loading

AymenLoukil commented Nov 14, 2019 •

edited

Loading

tunetheweb commented Nov 25, 2019 •

edited

Loading

tunetheweb commented Nov 25, 2019 •

edited

Loading