Test local application on the cluster without impacting any other service

[concaf]

tl;dr -
use telepresence to deploy and test local services to the cluster using selecting routing without impacting any other service in the entire cluster

---

Use case:
Our team of 150 people is working on a huge hosted services platform that consists of at least 20 microservices, all independently maintained by different smaller teams. We have 2 clusters, staging and prod, once everything looks fine on staging, we promote to prod. Now there is chaining among these microservices in all possible ways, and the services are very inter-dependent. Also, since the entire setup is very resource intensive, we cannot deploy everything locally, also it's not feasible to deploy the entire setup for every developer for testing. So everyone has access to a single staging cluster to hack.

The requirement is that developers should be able to develop and deploy their dev version of services to the cluster without breaking anything for the other services and hence, developers.
e.g.

1. microservice_4 and microservice_18 are inter-dependent.
2. Developer of microservice_4 wants to test their changes against the cluster, so they deploy the new shiny hacky microservice_4 to the cluster (which might break microservice_18)
3. microservice_18 should point to the older microservice_4 and not to the newer test version of microservice_4.

Now to avoid this, the microservice_4 developer cannot roll out microservice_4_test, because every service points to microservice_4 and not every reference can be changed in the cluster.

Is there any way this can be solved using telepresence?

Maybe, the fundamental solution will be based on routing selective traffic for different users, i.e. besides the current version, the newer test version of microservice_4 will be deployed, but only the traffic which is generated or requested by the developer is routed through the newer version, but the rest of the users will still hit the older version. So instead of spinning up a new cluster for every user, we have dev and staging running on the same cluster.

On the implementation front, from the first look I guess this can achieved using all the Istio's routing goodness, but from a developer point of view, that's too much work and not feasible.

Is there any way that telepresence could cover this use case such that the developer runs something like -
telepresence --test-deployment microservice_4 --docker-run microservice_4_test

This essentially duplicates the current microservice_4 deployment with all the secrets, configMaps, etc, and routes any traffic generated by this particular user through this newer deployment via the locally running container.

Another challenge is identifying the traffic generated by the given user. This could be using HTTP Headers or using source IP/Host based routing (which is auto detected by telepresence?)

Does this make any sense at all? Or is this use case already documented somewhere? Thoughts?

[ark3]

See #257. In the future, developers will use copy deployment (#363) to work on stuff along with a nuanced "intercept" operation to direct a well-defined subset of traffic to the Telepresence session. We've discussed using special tokens in the request, such as a debug header, to get the API gateway or service mesh to achieve that routing. I know @rhs has ideas about this.

[islamhamdi]

@containscafeine: did you manage to find a way to achieve what you mentioned? A workaround using telepresence?

[donnyyung]

I believe this is no longer an issue in Telepresence 2, see https://www.telepresence.io/docs/latest/reference/intercepts/ for information about selective intercepts.

Here are the docs on how to install Telepresence (https://www.telepresence.io/docs/latest/install/), please re-open if you still see this issue in our latest version!

[StefanLobbenmeierObjego]

FYI this is now called personal intercepts: https://www.getambassador.io/docs/telepresence/latest/concepts/intercepts#personal-intercept

(sorry to comment on this old issue, but the links above are dead)