Make organization a field on job templates #3903

AlanCoding · 2019-05-15T19:13:29Z

ISSUE TYPE

Feature Idea

SUMMARY

Right now, job templates have an implicit organization, based on the organization of its project (and sometimes inventory).

An organization field will be added directly to the job template. In a data migration, the JT's project's organization will be used for the default value. From then on, the JT organization will be used instead of the project organization for permissions considerations.

This issue should replace:

User Cannot See the Job that was launched by themselves. #458 org admins cannot see jobs after related items are deleted
Allow the use of the same job template name across multiple orgs(multi-tenancy) #3432 job templates are globally unique, whereas they should be unique to each organization

Some more speculative statements about changes which will come with this:

using projects and inventories in JTs from organizations other than the JT organization will be allowed (important question, ping @wenottingham)
since jobs inherit permissions rules from their job templates, it follows that the organization field should also be added to jobs, which is copied on launch. Permission rules will reference this organization
changing the organization of the JT may be allowed (but if it does, the organization of jobs from it will stay the same)

The text was updated successfully, but these errors were encountered:

AlanCoding · 2019-05-16T02:07:40Z

Also, I believe that this special case can be removed after this is done:

awx/awx/main/signals.py

Lines 227 to 239 in 20e5d82

    
           def save_related_job_templates(sender, instance, **kwargs): 
        
               '''save_related_job_templates loops through all of the 
        
               job templates that use an Inventory or Project that have had their 
        
               Organization updated. This triggers the rebuilding of the RBAC hierarchy 
        
               and ensures the proper access restrictions. 
        
               ''' 
        
               if sender not in (Project, Inventory): 
        
                   raise ValueError('This signal callback is only intended for use with Project or Inventory') 
        
               if instance._prior_values_store.get('organization_id') != instance.organization_id: 
        
                   jtq = JobTemplate.objects.filter(**{sender.__name__.lower(): instance}) 
        
                   for jt in jtq: 
        
                       update_role_parentage_for_instance(jt)

We need to very carefully consider whether some other similar signals needs to be implemented after this model change is made. I don't think so, but there are dragons here.

AlanCoding · 2019-05-16T12:59:27Z

@mabashian @jlmitch5 The major UI item will be to add the organization selector to the job template here. I'm advocating handling this as its own vertical slice work item, which will involved API & UI.

AlanCoding · 2019-05-16T13:02:52Z

Another RBAC detail that I want triaged before starting on this:

Job templates have a list of "sensitive fields" which require permission to related resources in order to change. If a user modifies the JT limit field, they will probably require use_role for the project & inventory. Will any organization-level permissions be needed to perform this action?

AlanCoding · 2019-07-11T13:42:10Z

@wenottingham @matburt Users inherit admin, execute, and auditing permissions via the inventory of a JT.

current:

awx/awx/main/models/jobs.py

Lines 244 to 252 in ca0e810

    
           admin_role = ImplicitRoleField( 
        
               parent_role=['project.organization.job_template_admin_role', 'inventory.organization.job_template_admin_role'] 
        
           ) 
        
           execute_role = ImplicitRoleField( 
        
               parent_role=['admin_role', 'project.organization.execute_role', 'inventory.organization.execute_role'], 
        
           ) 
        
           read_role = ImplicitRoleField( 
        
               parent_role=['project.organization.auditor_role', 'inventory.organization.auditor_role', 'execute_role', 'admin_role'], 
        
           )

With this change, will we revoke these permissions obtained via inventory? If we keep any of these, that will significantly limit the simplifications that can be obtained here.

proposed:

diff --git a/awx/main/models/jobs.py b/awx/main/models/jobs.py
index 59b26c66e5..f1ec15c17a 100644
--- a/awx/main/models/jobs.py
+++ b/awx/main/models/jobs.py
@@ -242,13 +249,13 @@ class JobTemplate(UnifiedJobTemplate, JobOptions, SurveyJobTemplateMixin, Resour
     )
 
     admin_role = ImplicitRoleField(
-        parent_role=['project.organization.job_template_admin_role', 'inventory.organization.job_template_admin_role']
+        parent_role=['organization.job_template_admin_role']
     )
     execute_role = ImplicitRoleField(
-        parent_role=['admin_role', 'project.organization.execute_role', 'inventory.organization.execute_role'],
+        parent_role=['admin_role', 'organization.execute_role'],
     )
     read_role = ImplicitRoleField(
-        parent_role=['project.organization.auditor_role', 'inventory.organization.auditor_role', 'execute_role', 'admin_role'],
+        parent_role=['organization.auditor_role', 'execute_role', 'admin_role'],
     )

wenottingham · 2019-07-11T15:06:13Z

Users inherit admin, execute, and auditing permissions via the inventory of a JT.

Sorry, just for clarity: inherit admin, execute, and auditing of ???, via the inventory of a JT they have ??? role on?

AlanCoding · 2019-07-11T15:25:03Z

Sorry, just for clarity: inherit admin, execute, and auditing of ???, via the inventory of a JT they have ??? role on?

Say I have a JT with a project from orgA and inventory from orgB

There are users orgB_leader and orgB_button_pusher. orgB_leader has admin_role for orgB, and orgB_button_pusher has execute_role for orgB. Right now, these users can admin the JT and run the JT respectively. Those permissions will be taken away with the proposal.

Hypothetical users orgA_leader and orgA_button_pusher with the same permissions but to orgA will retain their permissions (same as pre-migration), just via the new relationship.

wenottingham · 2019-07-11T15:41:19Z

There are users orgB_leader and orgB_button_pusher. orgB_leader has admin_role for orgB, and orgB_button_pusher has execute_role for orgB. Right now, these users can admin the JT and run the JT respectively. Those permissions will be taken away with the proposal.

I did not think these permissions would be there, since the implied org on the JT only came from the project.

I think it's fine to remove these going forwards. I could see migrating explicit roles on the JT into existence on upgrade for this case, although I don't know how complex the migration would be.

AlanCoding · 2019-07-11T15:46:42Z

Thanks. The above proposal isn't done yet, but it's workable by writing a data migration.

AlanCoding · 2019-08-09T19:10:49Z

@matburt has suggested providing some read access via project & inventory in order to mitigate the backlash from removing admin-via-inventory permissions.

Here is what that might look like:

diff --git a/awx/main/models/jobs.py b/awx/main/models/jobs.py
index 12c691d195..30730f0417 100644
--- a/awx/main/models/jobs.py
+++ b/awx/main/models/jobs.py
@@ -242,13 +242,17 @@ class JobTemplate(UnifiedJobTemplate, JobOptions, SurveyJobTemplateMixin, Resour
     )
 
     admin_role = ImplicitRoleField(
-        parent_role=['project.organization.job_template_admin_role', 'inventory.organization.job_template_admin_role']
+        parent_role=['organization.job_template_admin_role']
     )
     execute_role = ImplicitRoleField(
-        parent_role=['admin_role', 'project.organization.execute_role', 'inventory.organization.execute_role'],
+        parent_role=['admin_role', 'organization.execute_role'],
     )
     read_role = ImplicitRoleField(
-        parent_role=['project.organization.auditor_role', 'inventory.organization.auditor_role', 'execute_role', 'admin_role'],
+        parent_role=[
+            'organization.auditor_role',
+            'project.organization.project_admin_role', 'inventory.organization.inventory_admin_role',
+            'execute_role', 'admin_role'
+        ],
     )

There are minor things about this I still don't like. In terms of the general principle and usability, this is much better than the previous proposal.

AlanCoding · 2019-08-19T16:05:35Z

@jlmitch5 I'm trying to poke around and find more places that will need updates for this change. One is the job templates list for organizations.

/api/v2/unified_job_templates/?type=job_template&order_by=name&page_size=20&or__jobtemplate__project__organization=35&or__jobtemplate__inventory__organization=35

You can see, this is listing JTs for which project or inventory are in the organization. That will need to be changed as a part of this work.

I also checked deletion warnings.

It looks like no changes are needed here. Credentials are the only thing that cascade delete with organizations (I have created all types of resources I can think of in the screenshot). That remains the same before / after JT.organization thing is implemented.

AlanCoding · 2019-08-19T17:28:23Z

Next finding

The permissions model proposed in prior comment:

#3903 (comment)

is not currently possible. If you grant a user admin_role to the inventory organization, it will display

Where u below is the user with the long username:

In [14]: u in jt.admin_role
Out[14]: False

In [15]: u in jt.read_role
Out[15]: True

This is a re-emergence of the display issue described in #4108. Like several other features, the implementation of this proposal is blocked by that incompleteness of the permissions list.

wenottingham · 2019-08-19T18:17:42Z

To be clear re: above - that is showing a user who has admin on the org of the inventory of the JT, but no roles on the org of the JT itself or the org of the project for the JT?

AlanCoding · 2019-08-19T18:19:36Z

^ correct

wenottingham · 2019-08-19T18:29:06Z

I don't think it's a display issue. I think it's an API bug that we could work around in the UI....

wenottingham · 2019-08-19T18:35:22Z

To be more clear: the API returns roles in indirect_access that it should not, and are not actually roles that apply to the object in question.

AlanCoding · 2019-08-19T18:53:06Z

The UI permission display is like describing a point on the (x,y) plane with a single value x. That works if x and y are always equal. In the proposal here for inventory permission, symmetry between two values is broken, so the display becomes incorrect.

To answer your question, the API is showing the correct roles in the list, and it gives the correct forms of access obtained by that role. In the example for "has_inventory_admin..." user in my last screenshot, see the API data:

        {
            "id": 8,
            "type": "user",
            "url": "/api/v2/users/8/",
            "related": {
                "teams": "/api/v2/users/8/teams/",
                "organizations": "/api/v2/users/8/organizations/",
                "admin_of_organizations": "/api/v2/users/8/admin_of_organizations/",
                "projects": "/api/v2/users/8/projects/",
                "credentials": "/api/v2/users/8/credentials/",
                "roles": "/api/v2/users/8/roles/",
                "activity_stream": "/api/v2/users/8/activity_stream/",
                "access_list": "/api/v2/users/8/access_list/",
                "tokens": "/api/v2/users/8/tokens/",
                "authorized_tokens": "/api/v2/users/8/authorized_tokens/",
                "personal_tokens": "/api/v2/users/8/personal_tokens/"
            },
            "summary_fields": {
                "direct_access": [],
                "indirect_access": [
                    {
                        "role": {
                            "id": 1635,
                            "name": "Admin",
                            "description": "Can manage all aspects of the organization",
                            "resource_name": "alan",
                            "resource_type": "organization",
                            "related": {
                                "organization": "/api/v2/organizations/84/"
                            },
                            "user_capabilities": {
                                "unattach": true
                            }
                        },
                        "descendant_roles": [
                            "read_role",
                            "use_role"
                        ]
                    }
                ]
            },
            "created": "2019-08-19T16:08:06.211769Z",
            "username": "alan_org_admin",
            "first_name": "alan_org_admin",
            "last_name": "",
            "email": "[email protected]",
            "is_superuser": false,
            "is_system_auditor": false,
            "ldap_dn": "",
            "last_login": null,
            "external_account": null
        }

The UI does not do anything with the descendant_roles information in that response. The reason that the API gives that field is for these symmetry-breaking cases. That's the problem.

To clarify, the logic to add the -> notation in #4514 could be moved entirely to the UI. That would be fine with me.

wenottingham · 2019-08-19T19:15:07Z

I don't think that's correct. Note, I'm working on the case from #4108 here.

Here is that case:

orgs A and B
user 'fred' in org B, in a team C
inventory 'foo' in org A

Default:

nothing in access_list for foo

Make fred an admin of org B:

nothing in access_list for foo

Explicitly grant team C use on foo, when fred is not an org admin of B:

direct use role now in access_list

Explicitly grant team C use on foo, when fred is an org admin of B:

direct use role now in access_list
admin role of B now in indirect_roles in access_list, as an ancestor of the use/read role

That last point is the bug. The org admin status of B does not grant them any use/read role on the object, and should not be in the indirect_roles.

AlanCoding · 2019-08-19T20:16:37Z

@pytest.mark.django_db
def test_team_admin_access():
    from awx.main.models import Inventory, User, Project, Organization, Team
    A = Organization.objects.create(name='A')
    B = Organization.objects.create(name='B')
    c = Team.objects.create(name='c', organization=B)
    # user 'fred' in org B, in a team C
    fred = User.objects.create(username='fred')
    B.member_role.members.add(fred)
    c.member_role.members.add(fred)
    # inventory 'foo' in org A
    foo = Inventory.objects.create(name='foo', organization=A)

    # nothing in access_list for foo
    assert fred not in foo.read_role

    # Make fred an admin of org B:
    B.admin_role.members.add(fred)
    # nothing in access_list for foo
    assert fred not in foo.admin_role

    # Explicitly grant team C use on foo, when fred is not an org admin of B:
    B.admin_role.members.remove(fred)
    c.member_role.children.add(foo.use_role)
    # direct use role now in access_list
    assert fred in foo.use_role

    # Explicitly grant team C use on foo, when fred is an org admin of B:
    B.admin_role.members.add(fred)  # team C already has foo.use_role
    # direct use role now in access_list
    assert fred in foo.use_role

    # The org admin status of B does not grant them any use/read role on the object, and should not be in the indirect_roles.
    c.member_role.members.remove(fred)
    assert fred not in foo.use_role  # yes it does

result:

Traceback (most recent call last):
  File "/awx_devel/awx/main/tests/functional/test_rbac_team.py", line 42, in test_team_admin_access
    assert fred not in foo.use_role  # yes it does
AssertionError: assert <User: fred> not in <Role: Use-31>
 +  where <Role: Use-31> = <Inventory: foo-1>.use_role

The access list is correct. Org admins inherit team permissions from teams inside their organization.

This RBAC rule is defined via this fields:

awx/awx/main/models/organization.py

Lines 126 to 131 in dc1bf3e

    
           admin_role = ImplicitRoleField( 
        
               parent_role='organization.admin_role', 
        
           ) 
        
           member_role = ImplicitRoleField( 
        
               parent_role='admin_role', 
        
           )

If you have organization.admin_role then you have the team's admin_role. If you have this, you have the team's member_role, and if you have this, you have the foo.use_role in the above example.

wenottingham · 2019-09-06T17:56:02Z

Flipping back to needs_devel as #4338 was closed.

spsingh1982 · 2019-10-16T23:40:47Z

When is this coming in the release? Thanks in advance.

pin to runner==1.4.4

kdelee · 2020-01-15T19:11:05Z

Any word on this? @AlanCoding

kdelee · 2020-03-18T15:41:58Z

Notes for @tvo318

In Tower 3.7, Job Templates now have a read-only field that shows what organization they are affiliated with. They inherit this from the Project that provides the playbook for the Job Template. In situations where Job Templates have inventories and projects from different organizations, this changes the RBAC relationship subtly. For Job Templates created before migrating to Ansible Tower 3.7, all previous permissions are preserved through explicit permission assignments to job templates. For newly created objects permissions are inherited from roles on the Project’s Organization, which is now explicitly the Job Template’s organization. The only users implicit admin role to the Job Template will be those with job_template_admin_role in the project’s organization. All other users will have explicit permissions to the job template either set on the job template or on the organization that the job template now explicitly belongs to. The only users from the inventory org that will have any access to the Job Template for newly created Job Templates shall be auditors, who will continue to have audit permissions to the Job Template because it affects hosts in the inventory. Additionally, because now Job Templates belong to the same Organization as the project, job templates are only required to have names unique within their organization. This has the consequence that the “named url” for the job template is now officially /api/v2/job_templates/MyJobTemplate++MyOrganization/ . However, for backwards compatibility the deprecated /api/v2/job_templates/MyJobTemplate will still work for Tower 3.7.

kdelee · 2020-03-18T15:49:32Z

First Y/N means had permission on prior version of awx/tower

Second Y/N means has permission after upgrade to new version of awx/tower on existing objects that were upgraded

Third Y/N means has permission on newly created resources in new version of awx/tower

Scope	Role Type	Can read JT	Can read activity stream for JT	Can Launch JT	Can edit JT	Add automated test for
Project Org	Project Admin	NNN	NNN	NNN	NNN	no
Project Org	Admin	YYY	YYY	YYY	YYY	No
Project	Admin	NNN	NNN	NNN	NNN	No
Project Org	Execute	YYY	YYY	YYY	NNN	no
Project	use	NNN	NNN	NNN	NNN	no
Project Org	Auditor	YYY	YYY	NNN	NNN	no
Project Org	JT Admin	YYY	YYY	YYY	YYY	no
Project	Read	NNN	NNN	NNN	NNN	no
System	System Admin	YYY	YYY	YYY	YYY	no
Inventory Org	Admin	YYN	YYN	YYN	YYN	yes
Inventory Org	Inventory Admin	NNN	NNN	NNN	NNN	no
Inventory org	JT admin	YYN	YYN	YYN	YYN	yes
Inventory org	Auditor	YYY	YYY	NNN	NNN	yes
Inventory Org	Execute	YYN	YYN	YYN	NNN	yes
Inventory	use	NNN	NNN	NNN	NNN	no
Both Org	Org admin	YYY	YYY	YYY	YYY	no
Both Org	Inventory Admin	NNN	NNN	NNN	NNN	no
Both Org	Project Admin	NNN	NNN	NNN	NNN	no
Both Org	Execute	YYY	YYY	YYY	NNN	no
Both Org	Auditor	YYY	YYY	NNN	NNN	no
Both org	Read on inventory	NNN	NNN	NNN	NNN	no
Both org	Read on project	NNN	NNN	NNN	NNN	no
Both org	Admin on project	NNN	NNN	NNN	NNN	no
Both org	Admin on inventory	NNN	NNN	NNN	NNN	no
Both org	JT admin	YYY	YYY	YYY	YYY	no

kdelee · 2020-03-18T15:50:18Z

closing as we are done verifying and regression tests merged

AlanCoding added type:enhancement state:needs_devel component:api labels May 15, 2019

AlanCoding mentioned this issue May 15, 2019

Feature: Allow selecting project branch / tag on JT and WFJT #282

Closed

AlanCoding assigned jbradberry and AlanCoding May 15, 2019

awithrow9 added the priority:medium label Jun 28, 2019

AlanCoding added state:in_progress and removed state:needs_devel labels Jul 11, 2019

This was referenced Jul 11, 2019

Remove unused migration methods #4315

Merged

[WIP] Migrate JT project.organization to direct org field #4338

Closed

AlanCoding mentioned this issue Aug 12, 2019

Add auditor_role for projects and inventory #3668

Closed

AlanCoding mentioned this issue Aug 19, 2019

Show non-obvious permission holders with arrow notation #4514

Closed

wenottingham mentioned this issue Aug 20, 2019

Fix display of indirect access permissions. #4529

Merged

wenottingham added state:needs_devel and removed state:in_progress labels Sep 6, 2019

AlanCoding pushed a commit to AlanCoding/awx that referenced this issue Oct 29, 2019

Merge pull request ansible#3903 from ryanpetrello/runner-1-4-4

2b12e26

pin to runner==1.4.4

AlanCoding mentioned this issue Jan 21, 2020

Add read-only organization field to job templates #5726

Merged

ryanpetrello added state:in_progress and removed state:needs_devel labels Feb 20, 2020

ryanpetrello mentioned this issue Feb 28, 2020

Loosen uniqueness on job templates #4008

Closed

ryanpetrello added state:has_unmerged_pr and removed state:in_progress labels Mar 6, 2020

blomquisg added the state:needs_test label Mar 6, 2020

wenottingham removed the state:has_unmerged_pr label Mar 17, 2020

kdelee assigned appuk and kdelee Mar 18, 2020

kdelee closed this as completed Mar 18, 2020

kdelee added state:needs_docs and removed state:needs_test labels Mar 18, 2020

jbradberry mentioned this issue Mar 23, 2020

Awxkit api endpoints #6386

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make organization a field on job templates #3903

Make organization a field on job templates #3903

AlanCoding commented May 15, 2019

AlanCoding commented May 16, 2019

AlanCoding commented May 16, 2019

AlanCoding commented May 16, 2019

AlanCoding commented Jul 11, 2019

wenottingham commented Jul 11, 2019

AlanCoding commented Jul 11, 2019

wenottingham commented Jul 11, 2019 •

edited

Loading

AlanCoding commented Jul 11, 2019

AlanCoding commented Aug 9, 2019

AlanCoding commented Aug 19, 2019

AlanCoding commented Aug 19, 2019

wenottingham commented Aug 19, 2019

AlanCoding commented Aug 19, 2019

wenottingham commented Aug 19, 2019

wenottingham commented Aug 19, 2019 •

edited

Loading

AlanCoding commented Aug 19, 2019

wenottingham commented Aug 19, 2019

AlanCoding commented Aug 19, 2019

wenottingham commented Sep 6, 2019

spsingh1982 commented Oct 16, 2019

kdelee commented Jan 15, 2020

kdelee commented Mar 18, 2020 •

edited

Loading

kdelee commented Mar 18, 2020

kdelee commented Mar 18, 2020

Make organization a field on job templates #3903

Make organization a field on job templates #3903

Comments

AlanCoding commented May 15, 2019

ISSUE TYPE

SUMMARY

AlanCoding commented May 16, 2019

AlanCoding commented May 16, 2019

AlanCoding commented May 16, 2019

AlanCoding commented Jul 11, 2019

wenottingham commented Jul 11, 2019

AlanCoding commented Jul 11, 2019

wenottingham commented Jul 11, 2019 • edited Loading

AlanCoding commented Jul 11, 2019

AlanCoding commented Aug 9, 2019

AlanCoding commented Aug 19, 2019

AlanCoding commented Aug 19, 2019

wenottingham commented Aug 19, 2019

AlanCoding commented Aug 19, 2019

wenottingham commented Aug 19, 2019

wenottingham commented Aug 19, 2019 • edited Loading

AlanCoding commented Aug 19, 2019

wenottingham commented Aug 19, 2019

AlanCoding commented Aug 19, 2019

wenottingham commented Sep 6, 2019

spsingh1982 commented Oct 16, 2019

kdelee commented Jan 15, 2020

kdelee commented Mar 18, 2020 • edited Loading

kdelee commented Mar 18, 2020

kdelee commented Mar 18, 2020

wenottingham commented Jul 11, 2019 •

edited

Loading

wenottingham commented Aug 19, 2019 •

edited

Loading

kdelee commented Mar 18, 2020 •

edited

Loading