Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wrong server list is returned #50

Closed
garci66 opened this issue Jul 12, 2021 · 8 comments · Fixed by #63
Closed

wrong server list is returned #50

garci66 opened this issue Jul 12, 2021 · 8 comments · Fixed by #63

Comments

@garci66
Copy link

garci66 commented Jul 12, 2021

When using version 1.1.2, the --list command is not returning the correct server list. On the same host, just doing curl https://www.speedtest.net/speedtest-servers-static.php the list returns the proper servers (those close to me) but if I use the --list command, the closest server is 250km away. Which of course impacts the testing results.

This is somehow related to #37 which prevents me from running proper tests since I can't override for the servers closest to my location.

The test is using the pre-built binaries for linux x64 on a centos 7.7 machine. I can provide packet captures if needed,

One detail. This can't be always reproduced... which makes it even more confusing.

@showwin
Copy link
Owner

showwin commented Jul 19, 2021

Thank you for using speedtest-go!

speedtest-go also using https://www.speedtest.net/speedtest-servers-static.php to fetch server list as you may know. And there is nothing special trick in this process, like adjusting location information, just sorting server list by distance from your location. As a result, it may have slightly different server orders between --list option and curl https://www.speedtest.net/speedtest-servers-static.php. But even taking that into account, it is unlikely the 250km away server comes first 🤔

In my research, the result of curl https://www.speedtest.net/speedtest-servers-static.php could be change about every 30 seconds. This may because speedtest.net has a cache. In my thought, during the period the server list should be same between --list option and curl https://www.speedtest.net/speedtest-servers-static.php. This means when the distant server is selected by speedtest-go, there are no closer servers from curl https://www.speedtest.net/speedtest-servers-static.php if you do it right away.

In conclusion, we may have no control over this problem 😢. If you have any good idea or solution, please feel free to leave comment. Thank you!

@showwin showwin closed this as completed Jul 19, 2021
@garci66
Copy link
Author

garci66 commented Jul 19, 2021

Dear @showwin, do you know what user agent the speedtest-go client is using? maybe speedtest is doing some user-agent based filtering. Doing the curl request to the server list, i ALWAYS receive the same exact list. I tried 20-30 queries and all got the same closest servers. But with speedtest-go, i would say the list would change on every request, alternating between the close (good) list and the one with the remote servers.

Is there any way we could at least fail if using a --server command and the chosen server is not on the list? cause otherwise the user can inadvertently be running against a far away server when he's trying to get a "close one" based on the --list results.

@showwin showwin reopened this Aug 3, 2021
@showwin
Copy link
Owner

showwin commented Aug 4, 2021

@garci66 Sorry for late response 🙏
I wrote a test code 1a2a5d1 to verity that the server list change on every request, though such a situation did not occur. The test has passed. Despite requesting server list 100 times, I got the same result 🤔. If you could provide some code to reproduce the change of list on every request, that would be great.

@garci66
Copy link
Author

garci66 commented Aug 4, 2021 via email

@mikemadden42
Copy link

I also have some inconsistencies with the server list.

On a system in Las Vegas, Nevada I consistently get servers in Kansas & Oklahoma.

$ speedtest-go --list
Testing From IP: x.x.x.x, (Hulk) [x, -y]
[20794]    35.76km Hutchinson, KS (United States) by IdeaTek Telcom
[22144]   106.39km Holyrood, KS (United States) by H&B Communications
[43077]   193.39km Stillwater, OK (United States) by Centranet
[3753]   208.73km Wamego, KS (United States) by Wamego Telephone Company
[32726]   235.58km Edmond, OK (United States) by Oklahoma Christian University
[16619]   255.66km Oklahoma City, OK (United States) by Cox - Oklahoma City
[12377]   261.50km Lawrence, KS (United States) by Midco
[42552]   281.47km Shawnee, OK (United States) by CVEC Fiber
[18539]   292.56km Blue Hill, NE (United States) by Glenwood Telephone Membership Corp
[8925]   301.80km Joplin, MO (United States) by Sparklight

I do not see these issues with the Python client.

$ ./speedtest.py --list
Retrieving speedtest.net configuration...
 2426) Spacelink (San Diego, CA, United States) [1839.16 km]
29471) Janus Networks (San Diego, CA, United States) [1839.16 km]
17249) Wiline Networks (Los Angeles, CA, United States) [1881.35 km]
 3864) GeoLinks (Los Angeles, CA, United States) [1881.35 km]
 7190) Sharktech Inc. (Los Angeles, CA, United States) [1881.35 km]
18229) Starry, Inc. (Los Angeles, CA, United States) [1881.35 km]
18114) Atlantic Metro (Los Angeles, CA, United States) [1881.35 km]
44933) Cloudflare (Los Angeles, CA, United States) [1881.35 km]
43513) Huajuan (Los Angeles, CA, United States) [1881.35 km]
44349) PoloCloud (Los Angeles, CA, United States) [1881.35 km]

@briancline
Copy link

Yes, they seem to provide different server lists based on the user agent. I see vastly different server lists when running these two commands:

$ curl -v https://www.speedtest.net/speedtest-servers-static.php

$ curl -v https://www.speedtest.net/speedtest-servers-static.php -H "User-Agent: Mozilla/5.0 ($(uname -r); U; $(uname -p); en-us)"

@garci66
Copy link
Author

garci66 commented Sep 27, 2021

I am having similar issues once again. This is the list returned by "speedtest-go"

Testing From IP: 181.228.66.93, (Telecom Argentina S.A.) [-34.4727, -58.5243]
[33850]   495.08km Quarai (Brazil) by ZAP TCHE Telecom
[26641]   495.08km Quaraí (Brazil) by Fronteira Internet
[44378]   542.34km Uruguaiana (Brazil) by Fronteira Internet
[5405]   582.03km Alegrete (Brazil) by RedeConesul
[23388]   613.61km Itaqui (Brazil) by Gerensis Telecom
[35216]   677.16km São Pedro do Sul (Brazil) by São Pedro Net
[28085]   682.77km Santiago (Brazil) by Azecom
[25081]   825.97km Santo Cristo (Brazil) by Sygo Telecom
[25400]   829.54km Santa Rosa (Brazil) by Sygo Telecom
[34449]   852.94km Tucunduva (Brazil) by Log Internet

while plain curl -v https://www.speedtest.net/speedtest-servers-static.php
is returning the following:

<settings>
<servers><server url="http://speed.twireless.com.ar:8080/speedtest/upload.php" lat="-34.5304" lon="-58.5244" name="Munro" country="Argentina" cc="AR" sponsor="TuRed" id="40951"  host="speed.twireless.com.ar:8080" />
<server url="http://speedtest.telered.com.ar:8080/speedtest/upload.php" lat="-34.6036" lon="-58.3817" name="Buenos Aires" country="Argentina" cc="AR" sponsor="TeleRed" id="2186"  host="speedtest.telered.com.ar:8080" />
<server url="http://velocidad.telecentro.net.ar:8080/speedtest/upload.php" lat="-34.6036" lon="-58.3817" name="Buenos Aires" country="Argentina" cc="AR" sponsor="Telecentro" id="900"  host="velocidad.telecentro.net.ar:8080" />
<server url="http://speedtest1.convergia.com.ar:8080/speedtest/upload.php" lat="-34.6036" lon="-58.3817" name="Buenos Aires" country="Argentina" cc="AR" sponsor="Convergia Argentina" id="11480"  host="speedtest1.convergia.com.ar:8080" />
<server url="http://speedtestba.miinternet.cl:8080/speedtest/upload.php" lat="-34.6036" lon="-58.3817" name="Buenos Aires" country="Argentina" cc="AR" sponsor="Mi Internet SpA" id="38765"  host="speedtestba.miinternet.cl:8080" />
<server url="http://ar.sa.speedtest.i3d.net:8080/speedtest/upload.php" lat="-34.6036" lon="-58.3817" name="Buenos Aires" country="Argentina" cc="AR" sponsor="i3D.net" id="44792"  host="ar.sa.speedtest.i3d.net:8080" />
<server url="http://speedtest2.metrotel.com.ar:8080/speedtest/upload.php" lat="-34.6037" lon="-58.3816" name="Buenos Aires" country="Argentina" cc="AR" sponsor="Metrotel" id="18908"  host="speedtest2.metrotel.com.ar:8080" />
<server url="http://st1ram.personal.com.ar:8080/speedtest/upload.php" lat="-34.6500" lon="-58.5667" name="Ramos Mejia" country="Argentina" cc="AR" sponsor="Telecom Personal" id="10319"  host="st1ram.personal.com.ar:8080" />
<server url="http://speedtest-ramosmejia.solunet.com.ar:8080/speedtest/upload.php" lat="-34.6500" lon="-58.5667" name="Ramos Mejia" country="Argentina" cc="AR" sponsor="Grupo Solunet" id="36715"  host="speedtest-ramosmejia.solunet.com.ar:8080" />
<server url="http://speedtest-moreno-1.solunet.com.ar:8080/speedtest/upload.php" lat="-34.6340" lon="-58.7914" name="Moreno" country="Argentina" cc="AR" sponsor="Grupo Solunet" id="36796"  host="speedtest-moreno-1.solunet.com.ar:8080" />
</servers>
</settings>

(The curl with the custom UA above returns another list as well). Both curls return "close" servers, but the "custom" curl returns the best list (a server in my own ISP for example). The list from speedtest-go, all servers are in brazil... (im in argentina)

@fanek007
Copy link

fanek007 commented Dec 8, 2021

Hello @showwin,

first of all, thank you for your great job ! speedtest-go is a great solution !
Both URLs https://www.speedtest.net/speedtest-servers-static.php nor https://c.speedtest.net/speedtest-servers-static.php do not return correct servers. I'm in Czech Republic and I receive server located mostly in Poland. The behavior of these two URL is unpredictable.

I found out, that there is another URL, which returns server list in JSON format which seems to be correct.
It's mentioned in this discussion:
https://issueexplorer.com/issue/sivel/speedtest-cli/765

The URL is: https://www.speedtest.net/api/js/servers?engine=js&limit=10&https_functional=true

I'm not good at GO lang, so I would like to ask you @showwin or anybody else to help me with rewriting the code to accept and parse JSON format.

Thanks in advance
Milan

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants