Enable pantsd by default

[stuhood]

This is blocked on a bunch of 1.3.0 issues, but primarily by #3941 and #3774: it should likely happen shortly into the 1.4.0.dev cycle, after #3941 is completed.

[kwlzn]

I think #5046 is the last blocker for this, afaict.

[stuhood]

I think #5567 is now the last blocker here.

[blorente]

Coming back to this ticket, I think these are the blockers now:

• Improve --pantsd-invalidation-globs implementation with actual fingerprinting #5567, which remains open.
• Handle signals gracefully #6574, which has had tremendous progress recently.
• [Pantsd] Enable pantsd in Travis #7320, and then fixing any hangs that occur.
• fix pantsd deadlock with ctrl-c while awaiting pid #7301, Ctrl-C deadlock, possibly dependent on [WIP] split NailgunProtocol into PailgunProtocol versions of everything, add PGRP chunk #6579
• Can't ^C when using the daemon, during connection to pailgun #5220, Another Ctrl-C issue

[cosmicexplorer]

I'm going to leave #6574 open for a bit (especially because I need to respond to some review comments), and before trying to merge it I will look at the open issues above and see if it might fix any of them (and add tests if so).

[cosmicexplorer]

I would actually recommend adding #7080 as a blocker as well, as we do not have exception sink/logging testing for the pantsd case right now.

[cosmicexplorer]

#7220 is relevant, not sure if it is a blocker because the failure went unnoticed for a very long time but it is a pretty important function, I just found some difficulty trying to test it then.

[blorente]

#7330 and #7331 are also relevant to strengthening integration testing.

[cosmicexplorer]

I am going to add #5220 and #7301 to the TODO to fix with #6579 if possible. As I just now noted on #7301, I believe that ticket may be very closely related to #5220 / the same cause, although the behavior is different.

[blorente]

After #7596 and some cleanup of very pressing bugs, the tasks for having a solid daemon seem to be:

• The behaviour of exiters in the new multithreaded world is not well-defined. In particular, if we have a thread that wants to exit a pants invocation through the DaemonExiter in DaemonPantsRunner and the main thread is also trying to exit through the PantsDaemon exiter, it's unclear which one should prevail. Some (but not all) context: create nested exiter-contextmanagers #7597
• The precomputing code in DaemonPantsRunner.create() should be moved, as described in [pantsd] Remove dead code around prefork graph warming #8002. This will make exception handling and the exit conditions so much easier.
• Remote exception fetching needs to be better, now that we don't have a PID per pants invocation: Post fork-removal, remote exception fetching is incorrect #7810.

Feel free to add to this list.

[stuhood]

The issues mentioned in #4438 (comment) should be resolved in #9722: a followup change will enable pantsd by default.

[stuhood]

This is done for real in #9935. Thank you everyone!