Support versioned HTTP API requests

[peterdemartini]

To support changes to the HTTP API without breaking existing tools, and job, Teraslice needs an API versioning convention. @kstaken and I discussed using a route prefix for all existing machine-readable API endpoints. For example, /cluster/state should be moved to /v1/cluster/state but /txt/workers shouldn't be changed.

Ideally, this can be done without making breaking changes by supporting both the /v1/ prefix and the current non-prefix endpoints.

[jsnoble]

Im not fully sure what this will buy us. A lot of the API's are directly tied to the rest of the services in cluster master like the execution, job and asset service apis. If we make any changes there it's going to change how the outside exposed API's behave. Are you looking to strip out all services and make them versioned outside of teraslice versions? What increased benefits do we see for making another version outside of the version changes that we do with teraslice? The version of teraslice being used is already exposed in cluster state. How are you looking to manage apis? Will we have to load both a v1 of the api for one area, then support another with v2/v3 in the url? Will we have to load multiple different http api's in the system to handle this? This is not even accounting for changes that are made at the top exposed http apis. How are we going to map top level api versions with internal service api changes? Are we versioning those as well?

It seems the real answer would be for the tools to work for certain ranges of teraslice version which we dont do correctly. All we do is just go from 0.0.33 => 0.0.34 with breaking changes. How are other tools supposed to understand whats going on? These changes seem to stick more logic and heaviness into cluster master when I think we need to go the other direction. We should be putting all our breaking changes together and doing a major version update.

If we did go down this route, then not putting a version on the txt apis would not work real well. Most of the /txt/ apis use the exact same internal service api that the versioned api would use. Unless we are just hoping that no one will try to parse a txt endpoint and just solely use it for just views. But this begs the question of whether a view should be versioned?

[peterdemartini]

I believe the goal of this is to leave room for making breaking changes to the API, and it can be done initially with very few changes. You're right, breaking an API has a lot of ramifications and can make the code more complex, but they are inevitable as the developers become dependent on behaviors that can limit the capabilities of the product.

There are many ways of maintaining multiple different versions of an API, the most effective way that I have found is to move the core business logic from the routes and into a shared location, and make the routes purpose to call the core business logic and handle their compatibility requirements. The idea is the routes themselves become very dumb and only handle things like normalising the input and output, integrating with the core system. Doing this also makes it easier to build other interfaces besides HTTP, like WebSockets.

[erik-stephens]

I would like the versioning to be opt-in by clients. In other words, I wouldn't want to have to update clients to point to new routes if there were no compatibility-breaking changes - the default version is latest unless client asks for a specific version.