🔥 Google unable to index pages

[jonsnowpt]

🐛 Introduction:

Google is unable to index the platform fixtures and other pages.

📝 Details:

Since the use of scores_lib, the platform has degraded its performance and created several unexpected issues; one of them of high severity is the inability to get correctly indexed, making the platform lose 90% of its traffic.

Error on Google:

The page is not usable on mobile devices.

Uncaught (in promise) Error: Rejected at wrsParams.serviceWorkers.navigator.serviceWorker.register (<anonymous>:13:648) at https://scores.betarena.com/pt/futebol/hellas-verona-bologna-18548086:59:30

🔍 Steps to Reproduce:

Use the search console and add any fixture link.

Also, if a search is made on Google:

"site:scores.betarena.com" is possible to see that the number of pages indexed in the last 24 hours is very low.

🤔 Expected Behavior:

There must be no message stating the pages cannot be used on mobile. Also, the preview should be the mobile view. Besides, the number of indexed pages must be much higher.

💥 Actual Behavior:

Lots of pages are not being indexed, and errors on the Search console.

[migbash]

NOTES

Can replicate the issue on DEV using the following Google Service

https://search.google.com/test/mobile-friendly

---

TESTING

✅ [DEV] https://betarena-scores-platform.herokuapp.com/
✅ [PROD] mirror | docker https://temp-betarena-scores-docker.herokuapp.com/
❌ [PROD] https://scores.betarena.com/

✅ [DEV] https://betarena-scores-platform.herokuapp.com/football/portugal/liga-portugal
✅ [PROD] docker https://temp-betarena-scores-docker.herokuapp.com/football/portugal/liga-portugal
❌ [PROD] https://scores.betarena.com/football/portugal/liga-portugal

❌ https://betarena-scores-platform.herokuapp.com/pt/futebol/hellas-verona-bologna-18548086
Full-Time Game
[?] Suddenly, error on [DEV] no longer seen and only not working on ❌ PROD (scores.betarena.com) vs ✅ DEV


Error
00:08.000
TypeError: Failed to fetch dynamically imported module: https://betarena-scores-platform.herokuapp.com/_app/immutable/chunks/7-8ce30b49.js
https://betarena-scores-platform.herokuapp.com/_app/immutable/start-c8bbede3.js:3

Error
00:08.000
Uncaught (in promise) TypeError: Failed to fetch dynamically imported module: https://betarena-scores-platform.herokuapp.com/_app/immutable/chunks/0-15bbc82d.js
https://betarena-scores-platform.herokuapp.com/_app/immutable/start-c8bbede3.js:8

Error
00:08.000
Uncaught (in promise) TypeError: Failed to fetch dynamically imported module: https://betarena-scores-platform.herokuapp.com/_app/immutable/chunks/0-15bbc82d.js
https://betarena-scores-platform.herokuapp.com/pt/futebol/hellas-verona-bologna-18548086:0

Error
00:08.000
Uncaught (in promise) Error: Rejected at wrsParams.serviceWorkers.navigator.serviceWorker.register (<anonymous>:13:648) at https://betarena-scores-platform.herokuapp.com/pt/futebol/hellas-verona-bologna-18548086:112:30
:12

✅ [DEV] https://betarena-scores-platform.herokuapp.com/football/vitria-sc-sporting-cp-18560334
❌ [PROD] https://scores.betarena.com/football/vitria-sc-sporting-cp-18560334
Tested when game was not yet started

✅ [DEV] https://betarena-scores-platform.herokuapp.com/pt/futebol/luton-town-middlesbrough-18544870
24/04/2022 21:44 | Live (at testing)
❌ [PROD] https://scores.betarena.com/pt/futebol/luton-town-middlesbrough-18544870

Uncaught (in promise) Error: Rejected at wrsParams.serviceWorkers.navigator.serviceWorker.register (<anonymous>:13:648) at https://scores.betarena.com/pt/futebol/luton-town-middlesbrough-18544870:59:30
:12

---

NOTES

1. ❌ A theory to the cause of the issues, is the possibility of the docker-compose continuous uptime pipeline we have configured with the frontend, which can ben keeping files from past deployments, causing a filesystem error, especially from the errors in (bullet point 3).
   Test required to factory reset the website, docker container.

2. ❌ Could be a cloudflare issue, as the DEV environment seems to be working correctly, for the tested links above. [⏸️ pending]
   Conclusion: disabling cloudflare has not done any visible change to the platform.

3. A code / sveltekit error

• Not found /_app/immutable/assets/xxx sveltejs/kit#9089
• adapter-node doesn't work correctly with paths.base sveltejs/kit#3726
• adapter-cloudflare: only including _app content in _routes.json sveltejs/kit#7640
• TypeError: Failed to fetch dynamically imported module vitejs/vite#11804
• Error during site building sveltejs/kit#9146
• https://stackoverflow.com/questions/63301353/service-worker-registeration-failed-when-indexed-in-google-search-console

4. ❌ A compile/bundle error with Vite
   Conclusion: tested with removing vite plugins and with the 7. compliment, on the latest sveltekit - vite - svelte versions.

5. 📌 A server/environment issue on Production deployment.
   Conclusion: As the attempt to replicate the issue using the DEV version on Heroku and a mirror of productions with an exact docker/container deployment on Heroku, has not shown any issues.

6. ❌ Secrets/environment issue.
   Conclusion: updating secrets to latest has not caused changes.

7. ❌ A pacakge.json version issue.
   Conclusion: updating packages versions to latest has not caused changes.

8. ❌ A robots.txt issue.
   Conclusion: tested with one of the DigitalOcean deployments.

9. ❌ A svelte.config.js issue
   Conclusion: tested with one of the DigitalOcean deployments.

---

NOTES

Able to replicate on docker deployment on DigitalOcean using Ubuntu versions 22.10-04 LTS x64 & 20.04 LTS x64, indicating that the ubuntu version is not at fault.
http://164.92.204.152:5055 - http://64.226.72.95:5055 - http://134.209.245.5:5055

Reverting to Feb 21 commit to -> main still shows the same error, of mobile being unusable.

---

NOTES

• https://support.google.com/webmasters/thread/212859428?hl=en&sjid=7957376067034146390-EU

Works fine with no issues on Bing URL Inspector.

---

NOTES

Applied a custom hack to preload with the tempalte (app.html) file the necessary CSS as a master override, and remove it once googlebot crawls the website, to improve the website score. (best).

[jonsnowpt]

Pages tested on weekends get the same no usable on mobile response:

https://scores.betarena.com/pt/futebol/real-madrid-almera-18545407

[jonsnowpt]

The errors persist:

[jonsnowpt]

The errors were settled after transferring to Heroku.