401 Unauthorized / 404 Not Found / Cannot iterate over null

[pstadler]

Here we go again.
More information in #33. Fix available in branch acquire-yahoo-finance-session

[pcause]

tried that version and same jq error.

[amccombsahce]

I know nothing, but I edited ticker.sh and changed line API_ENDPOINT, and replaced "v7" with "v6", and that appears to return a result; I guess I just got lucky.

[pstadler]

Interesting. Will test things later. I‘m currently undecided what to do next. Last time the problem went away.

[pcause]

i just made it v6 and that worked

[pstadler]

released a new version using v6. thanks folks!

[pstadler]

It looks like the workaround using v6 stopped working. acquire-yahoo-finance-session is probably the way to go.

[justinodunn]

It looks like the workaround using v6 stopped working. acquire-yahoo-finance-session is probably the way to go.

Do you know if acquiring crumbs and cookies for the v6 url works? Or is it only working for v7?

Because if this isn't a crumb / cookie issue with v6, maybe they're just having problems in the backend and will most likely be fixed when the Yahoo team is awake.

[pstadler]

Yeah, I'm definitely waiting before releasing a fix.

[tomaszg7]

I didn't look at the code, but as far as I know https://github.com/premnirmal/StockTicker also uses Yahoo data and it is still working. Might be worthwhile to take a look there. It was broken just like this script last month when the issue first surfaced. Maybe they figured out some fix.

[pstadler]

Just merged #39
This won't work in some countries due to checks for cookie consent. Unfortunately, I can't reproduce it on my side, so I'd need your help to handle this case.

[g4570n]

Just merged #39
This won't work in some countries due to checks for cookie consent. Unfortunately, I can't reproduce it on my side, so I'd need your help to handle this case.

With this change it is working for me now. I am in south america.

[pcause]

fixed for me as well

[tomaszg7]

Just merged #39 This won't work in some countries due to checks for cookie consent. Unfortunately, I can't reproduce it on my side, so I'd need your help to handle this case.

Not fixed for me, as expected :) I'm still troubled by georestriction issue. Let me know if you have any idea how I could help.

[PSLLSP]

It still doesn't work for me. I cannot get file crump.txt, ticker.sh script fetch "Server error 500" (long file with css, etc to show error 500). When I try in my browser, I get code, like "C5CDSr9vMve", I assume that something like this should be in crump.txt. When I examine cookies.txt, I see just one entry, like
.yahoo.com TRUE / TRUE 1685652033 GUCS AXbWBsEE

I run Linux and I am in EU...

[jeandupont314]

Still no solution for me:
ticker.sh ASML.AS
jq: error (at :1): Cannot iterate over null (null)

I run Linux Mint 21 and I am in the EU too

[pcause]

working for me here in US.

[peterhorsley]

working for me in Australia.

[giuli007]

I was experiencing similar issues being reported here and on #33

Symptoms are that:

• the initial request (

  ticker.sh/ticker.sh

  Line 41 in 2ac6837

  curl --silent --output /dev/null --cookie-jar "$COOKIE_FILE" "https://finance.yahoo.com" \

  ) to https://finance.yahoo.com gets redirected, as mentioned here and doesn't set the cookie properly
• which means that the subsequent request (

  ticker.sh/ticker.sh

  Line 43 in 2ac6837

  curl --silent -b "$COOKIE_FILE" "https://query1.finance.yahoo.com/v1/test/getcrumb" \

  ) to get the crumb returns a 500 error page
• which is then used as a query parameter (even though it's a multiline HTML doc) on the request to get quotes (

  ticker.sh/ticker.sh

  Line 48 in 2ac6837

  curl --silent -b "$COOKIE_FILE" "$API_ENDPOINT&fields=$fields&symbols=$symbols&crumb=$(cat "$CRUMB_FILE")"

  ) and causes curl to just fail silently

After looking around a bit and reading this StackOverflow answer I noticed it suggests using https://fc.yahoo.com/ instead of https://finance.yahoo.com/ to get the cookie.

I tried simply changing that url on ticker.sh (and deleting the cached cookie/crumbs with rm -rf "${TMPDIR%/}/ticker.sh-$(whoami)") and it works!

index f2ffe6c..a84f5a1 100755
--- a/ticker.sh
+++ b/ticker.sh
@@ -38,7 +38,7 @@ fields=$(IFS=,; echo "${FIELDS[*]}")
 [ ! -d "$SESSION_DIR" ] && mkdir -m 700 "$SESSION_DIR"

 preflight () {
-  curl --silent --output /dev/null --cookie-jar "$COOKIE_FILE" "https://finance.yahoo.com" \
+  curl --silent --output /dev/null --cookie-jar "$COOKIE_FILE" "https://fc.yahoo.com" \
   -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8"
   curl --silent -b "$COOKIE_FILE" "https://query1.finance.yahoo.com/v1/test/getcrumb" \
     > "$CRUMB_FILE"

I don't know how reliable this is going to be in the long term, as I am not sure if yahoo makes any guarantee about this url not redirecting, but it seem to be an effective workaround for now and I just thought it would be worth sharing it. cc @pstadler

[pstadler]

@giuli007 wow, this is great! would you mind opening a PR?

[giuli007]

@giuli007 wow, this is great! would you mind opening a PR?

Sure! There you go #42

[pstadler]

Awesome, thanks a lot!

@tomaszg7 @PSLLSP @jeandupont314 could you please confirm whether the fix in #42 is working for you?

[tomaszg7]

It is. I just had to remember to delete old /tmp/ticker* which kept broken cookie from previous runs.

[jeandupont314]

Awesome, thanks a lot!

@tomaszg7 @PSLLSP @jeandupont314 could you please confirm whether the fix in #42 is working for you?

It works again, thanks!

[dropbrick]

Anyone getting this again ?

[tomaszg7]

I was getting empty output again, had to remove manually /tmp/ticker* again to fix it.

[dropbrick]

I am getting this again, inside /tmp/ticker.sh-username/crumb.txt I get "Too Many Requests"

[g4570n]

I am getting this again, inside /tmp/ticker.sh-username/crumb.txt I get "Too Many Requests"

the same thing happens here too... and I am getting empty output

[pstadler]

#45