-
Notifications
You must be signed in to change notification settings - Fork 14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
more frequent updates? #5
Comments
@kylebakerio you will get JSON output if you use I am against additional JSON files because I only will be maintaining On the other hand, frequent updates will be very cool, say in every 20-30 minutes, we can easily scrap the Worldometers page and get relevant information and update data here. I think it will require diving into Github API, I don't have time to do it now, but I will think about it. Seems like a good personal project. JSON output will be easy to generate as well (Though CSV will be a better choice because size is much smaller, hence loading time will be smaller) If you want and you have time to make reliable and regular changes, I'd happily add you as a contributor |
Just a note, I created a small |
I ended up using csvToJson (https://github.com/Keyang/node-csvtojson). Still took a lot of massaging to get the data into the form I wanted to work with, though. Still, glad to have it! Also, neat project there--are you going to be using that app to update your CSV's here? Or what are your thoughts? Yeah, I'd be happy to be a contributor. My site is almost ready to 'beta launch', and its current iteration relies on this data--would be nice if the data was updated more often, already was thinking about what API to jump to next, but if I could just improve this data source that'd be nice too. |
That was my intention, but I spent more time to it than I thought and still without major luck I don't expect it to be automated in a short period of time, so maybe it's better for you if you jump to other API |
Can you explain to me what the whole update sequence is? I'm not familiar with this 'notebook' concept (?). My impression is you have some code on a website (for some reason?) that you run, it goes and scrapes and formats stuff, gives you a csv output, and you upload that here. Is that right? What am I missing, and what are the limitations you're working with? |
Update sequence is the following:
This notebook itself just runs js code automatically, which scrapes yesterdays worldometers data and merges it with data currently exists on Github
For your dashboard, you could use same approach to load live data, this is what I am doing on my dashboard - https://coronavirus.davidb.dev/ . Fetching current worldometers data, merging it with current data existing on Github and then displaying stats |
I see the output here, but I'd love to get more frequent updates, and not be dependent on you doing your updates by hand. How could we update this code so that it scrapes and updates this more frequently, and updates the output here automatically? I'll use it for now, but this is a big hesitation for me.
Also, I'll be converting this to JSON; should be pretty easy to just include that as a file here (or to just output the JSON here directly, either way), if you're open to that.
The text was updated successfully, but these errors were encountered: