[fetch] Testing time-dependent results, disabling cache

[gadicc]

Following on from #32, thanks @pudgereyem.

I'll add a way to disable using the cached result (probably an environment variable is easiest) but this still only solves half the problem. What we actually need is cached results from all the different possible times... a pain, I know -- but what we really want is for testing to really test everything, so we can be confident moving forward that EVERYTHING works all the time, without thinking that we need to run live tests at different times of the day.

[pudgereyem]

Hey @gadicc, yes I think environment variable is the easiest as well.

Adding time to the cache string

Oh, I got an idea regarding how we can test different responses due to the time of day, and cache them as well. Currently our test files look something like this;

it(`passed validation for search: ${search}`, async () => {
    const devel = `search-${search}.json`;
    await yf.search(search, {}, { devel });
});

..which will create a cached response for the search here; tests/http/search-${search}.json.

However, since the response from Yahoo depends on the time of day, we could factor in "time of day" to the cache string. I think hourly would be enough. So we could do something like this;

const hourOfTheDay = new Date().getHours(); // e.g 10
const amOrPm = hourOfTheDay >= 12 ? 'pm' : 'am';
const timeString = `${hourOfTheDay}${amOrPm}`; // --> '10am'

it(`passed validation for search: ${search}`, async () => {
    const devel = `search-${search}-${timeString}.json`;
    await yf.search(search, {}, { devel });
});

If tests where then to be run it would create cached responses for all the 24 hours of the day, like so:

- tests/http/search-AAPL-1am.json
- tests/http/search-AAPL-2am.json
...

I think the solution could be more fleshed out and improved upon, e.g how do we best automate this. But it seems like a good start to me. What do you think @gadicc?

[gadicc]

Hey @pudgereyem, this is a good idea in terms of effort, as we don't really have to do much planning or thinking to cover everything. The downside though is that we already have 200 tests, which run fairly quickly, but to have 200 * 24 = 4800 tests, when almost all of them are near identical, I think is a bad trade off.

I think the best thing to do, even though it's a lot more effort, is to run specific timed tests for the different states of the markets, which we'll need to figure out (however, it will be great to add to our documentation too). So, a test specifically for an LSE stock when the LSE is in different states. I can't think of any other time related dependence beyond market state, can you?

Having said all that, I'm not even sure if there's that much work here... as all we have to do figure out is the different market state values and which fields are dependent on that. As an example, as a result of testing the new FETCH_DEVEL env variable (see commit afcb397 above or note in README), I've added all the preMarket*?: number stuff, and additional marketState enum values (in 1c256c7 above). Thanks of course for raising this issue in the first place as I only noticed this because of it!

But my suspicious is that there's not that much more to discover. I'll run the live tests at a few different times of the day (you're welcome to do so too if you want) and we'll see what other kinds of things come up, which should give us a better idea of what direction to go. So leaving open for now.

[gadicc]

Going to close this since the FETCH_DEVEL=nocache code is done. Let's see what else comes up and then we'll decide on further strategy for time-dependent results.