tracking issue for authz work

[davepacheco]

(moving some notes from a local text file for better visibility)

• initial authn infrastructure with "spoof" authentication (authentication skeleton #314)
• initial authz infastructure with one protected endpoint (authz: first cut #346)
• flesh out authz policy (Polar file) and associated Rust types; prototype by protecting a project-level endpoint (DELETE disk) (flesh out authz policy #405)
• add users to the database (add built-in users to the database #486)
  • database schema
  • prepopulated users, at least for testing (and remove them from in-memory?)
  • API to list and fetch users (not create or update)
• implement predefined roles
  • database schema for predefined roles (authz: add built-in roles to the database #512)
  • prepopulate table of predefined roles (authz: add built-in roles to the database #512)
  • database schema for user <-> role mapping (store role assignments in the database #520)
  • prepopulate user <-> role mapping for predefined users and roles (store role assignments in the database #520)
  • load roles during the request (store role assignments in the database #520)
  • update authz implementation to use the loaded roles (store role assignments in the database #520)
• return 404s rather than 401 or 403 when the user can't read a resource (return 404 rather than 401/403 when user can't even see the object #600)
• get feedback from product/customers about reduced-scope authz model. (2022-01-31 update: we've spoken with two prospects and both were fine with a static set of roles in the MVP. Both felt they were going to need it to be more flexible later.)
• figure out authz from saga actions (e.g., need to construct an OpContext for the saga)
  • authz: add OpContext::for_saga_action #672
• add protection for the remaining endpoints
  • top-level Organization endpoints (authz: first cut #346 (create), flesh out authz policy #405 (list), add authz checks for top-level Organization endpoints #592 (the rest))
  • authz: check project list endpoints #617
  • authz: add checks for project update and delete #646
  • authz: protect disk list, disk create endpoints #661
  • authz: update disk lookup and protect disk get/attach/detach #662
  • authz: protect instance endpoints #673 (blocked on authz: add OpContext::for_saga_action #672)
  • authz: protect VPC endpoints #743
  • authz: protect VPC Subnet endpoints #754
  • /updates/refresh should check authZ #761
  • authz: protect VPC router endpoints #775
  • Move network interface authz to the data store #778
  • authz: protect VPC route endpoints #780
  • authz: protect endpoints for firewall rules #782
  • authz: protect various endpoints #790
  • authz: silo endpoints #936
• all new external endpoints should be authn/authz-protected (see authz: add test to prevent new endpoints not covered by authz #785)
  • stub endpoints should still have working authz + coverage #885
• "unauthorized" test could print summary of requests made #832
• new internal lookup API #798 consistent ways to look up resources by name and return the db::model type and the associated authz type (some discussion of this in 2021-11-22 review for flesh out authz policy #405)
• transition to new lookup API
  • convert route endpoints to new lookup API #839
  • convert router endpoints to new lookup API #840
  • convert all VPC lookups to new API, plus various by-id lookups #842
  • convert Instance-related lookups to new lookup API #843
  • convert disk, project, and organization lookups to new lookup API #855
  • clean up remaining "fetch" interfaces in datastore #845 (depends on authentication operations should use special Nexus context #846, "lookup_resource" macro should support different primary keys #847, generic authz types could be more type-safe #848)
    • remove most remaining hand-written "fetch" interfaces from DataStore #908
  • use pool_authorized() for all lookups #873
• add unit test for authz policy #1123
• authentication operations should use special Nexus context #846
• organization names cannot be reused across Silos #1096
• various cleanup
  • maybe: convert built-in users to service accounts
  • authz: add coverage for more endpoints #882
  • clean up use of public_error_from_diesel* #644
  • authz: make it easier to test authn/authz protection for new endpoints #652
  • all datastore.rs methods should accept OpContext
  • there should be no more uses of Datastore::pool() -- it should all be Datastore::pool_authorized
  • there should probably be no more uses of dropshot's make_request() family of functions in the test suite because everything requires authn except for the unauthorized test.
  • review use of public_error_from_diesel_pool -- that seems a lot less necessary post-authz because there's an authz variant
  • need better testing around minimum privileges #1374
  • related to this: there may be a lot of code paths today that need to assign roles to things that currently don't. e.g., when you create a Project, you should get a role on it.
  • an end-to-end test that every API call bails correctly for every permission that it checks? see abandoned end-to-end role test #1609
  • information leak as a result of the way resource lookups work #665
  • generic authz types could be more type-safe #848
  • *_refetch() functions in datastore.rs could be provided by the lookup API (macro) instead (e.g., top-level functions on LookupPath that take the authz object and work basically the same way as a by-id lookup, but use the lookup_type of the original authz object for a not-found error and preserve the original authz object)
  • review TODO-security
  • review endpoints not tested by unauthorized.rs/unauthorized_coverage.rs. Currently that includes /session/me, /login, /logout, which are all somewhat special cases. For example, "logout" succeeds even if your authn failed. And "/session/me" will work even for unprivileged users. Maybe they should have their own separate tests?
  • lookup_resource! macro could be simplified
    • lookup macro could be simpler #1547
    • harder: figure out what parts of it could be impl'd in common code using a trait instead of a macro
  • "unauthorized" test should support more-public endpoints #1277

[davepacheco]

Closing this because at this point we're not doing more MVP authz work. I don't think there's anything remaining under "cleanup" that doesn't already have its own issue and hasn't been done at least enough for MVP.