Smarter faster data processing (oh and data hot reloading!)
Gatsby v1's data layer is starting to feel quite solid.
For this latest alpha, I did a almost complete refactor of the data layer adding in Redux and ideas from event sourcing and modern build tools like Buck and Bazel from Facebook and Google respectively.
Now all data processing is incremental and completely cached. On booting, Gatsby looks at your data contents to decide if new processing is needed and only processes data that's actually changed. One large site built on 1.0 saw it's bootstrap time drop from nearly a minute to under 5 seconds!
And as part of this work, Gatsby's data system now automatically watches all your data sources for changes and reprocesses data on the fly. Remote or local data, if you change it, the change will be automatically pushed to the development version of your site. This is handy for editing a local markdown file but also a gatsby plugin can be monitoring a remote API for changes and automatically pull down data and update your site when something is updated.
Imagine a small team working on a site with a developer and two people working on content hosted on a CMS. Every time content is updated, the developer sees the changes in near real-time on the development site.
This was the last big piece blocking 1.0.0 from being released. There are a number of smaller tasks to complete but these should be more predictable in scope. There's plenty of smaller tasks to do! Please come help out!