allow server and scheduler to run on separate nodes on kubernetes

[jrhizor]

Blocked by #3605 and #4206

Once the above tickets are closed we can remove the affinity specifications for both the server and the scheduler and remove their dependence on PVCs.

[cgardens]

@jrhizor I want to make sure I understand the deliverables here, because my interpretation is that there are 2:

1. allow server and scheduled to run on separate nodes (the original intent of this ticket)
2. separate the worker from the scheduler (this issue that got closed as duplicate: Support horizontally scalable job execution #3825). the deliverable here is that all data network traffic would not go through the scheduler anymore and instead the worker could run totally separately

am I understanding right?

[jrhizor]

Oh. I completely misread this ticket.

This ticket basically boils down to removing podAffinity
https://github.com/airbytehq/airbyte/blob/master/kube/resources/server.yaml#L27 as soon as @tuliren adds support for completely deprecating volume-based persistence.

I'm going to mark this as blocked and reopen the other one, which is actually what I was working on (which is blocked from releasing but not blocked by implementation).