Local API test apiSpaces/changeSpaces.feature:221 fails because there is no cleanup after each test #4390

saw-jan · 2022-08-15T04:01:50Z

Test: apiSpaces/changeSpaces.feature:221
Drone job: https://drone.owncloud.com/owncloud/ocis/14430/40/7

Expected behavior: test apiSpaces/changeSpaces.feature:221 passes in CI each running
The reason why it doesn't work: no cleaning after each test and admin has a lot of Brians

Test Log:

  Scenario Outline: An admin user set an user personal space quota of via the Graph API                                                       # /drone/src/tests/acceptance/features/apiSpaces/changeSpaces.feature:214
    When user "Admin" changes the quota of the "Brian Murphy" space to "<quotaValue>"                                                         # SpacesContext::updateSpaceQuota()
    Then the HTTP status code should be "200"                                                                                                 # FeatureContext::thenTheHTTPStatusCodeShouldBe()
    When user "Brian" uploads a file inside space "Brian Murphy" with content "file is more than 15 bytes" to "file.txt" using the WebDAV API # SpacesContext::theUserUploadsAFileToSpace()
    Then the HTTP status code should be <code>                                                                                                # FeatureContext::thenTheHTTPStatusCodeShouldBe()

    Examples:
      | quotaValue | code                    |
      | 15         | "507"                   |
      | 10000      | between "201" and "204" |
      | 0          | between "201" and "204" |
      | -1         | between "201" and "204" |

Drone log:

runsh: Total unexpected passed scenarios throughout the test run:
apiSpaces/changeSpaces.feature:221

The text was updated successfully, but these errors were encountered:

SagarGi · 2022-08-15T08:01:19Z

#4325 creating quota issue has been fixed and working but seems that the tests is failing locally on When user "Brian" uploads a file inside space "Brian Murphy" with content "file is more than 15 bytes" to "file.txt" using the WebDAV API this step.

The above nightly failure seems to be a flaky pass and cannot be reproduce locally. @SwikritiT since you are working on the quota feature. can you also check the above step and create a new issue for it.

Note: Regarding the flaky pass, lets wait up and see on tomorrows nightly.

SwikritiT · 2022-08-15T10:18:56Z

#4325 creating quota issue has been fixed and working but seems that the tests is failing locally on When user "Brian" uploads a file inside space "Brian Murphy" with content "file is more than 15 bytes" to "file.txt" using the WebDAV API this step.

The above nightly failure seems to be a flaky pass and cannot be reproduce locally. @SwikritiT since you are working on the quota feature. can you also check the above step and create a new issue for it.

Note: Regarding the flaky pass, lets wait up and see on tomorrows nightly.

It's weird after setting the quota and uploading a file bigger than the set quota 507 is the expected status code but sometimes the file gets uploaded and 201 is returned. It's only reproducible through tests as of now. through curl command it gives me 507. I'll try if I can reproduce it again.

SwikritiT · 2022-08-15T11:19:34Z

Upon running the following test it gives 201 instead of 507 while trying to upload to a folder owned by user with insufficient quota and the test fails.

Feature: sharing

  Background:                                                                             # /home/swikriti/www/ocis/tests/acceptance/features/apiSpaces/shareOperations.feature:4
    Given using OCS API version "1"                                                       # FeatureContext::usingOcsApiVersion()
    And these users have been created with default attributes and without skeleton files: # FeatureContext::theseUsersHaveBeenCreatedWithDefaultAttributesAndWithoutSkeletonFiles()
      | username |
      | Alice    |
      | Brian    |
    And using spaces DAV path                                                             # FeatureContext::usingOldOrNewDavPath()

  Scenario: Uploading a file to a group shared folder with upload-only permission when the sharer has insufficient quota does not work        # /home/swikriti/www/ocis/tests/acceptance/features/apiSpaces/shareOperations.feature:360
    Given group "grp1" has been created                                                                                                       # FeatureContext::groupHasBeenCreated()
    And user "Brian" has been added to group "grp1"                                                                                           # FeatureContext::userHasBeenAddedToGroup()
    And user "Alice" has created folder "FOLDER"                                                                                              # FeatureContext::userHasCreatedFolder()
    And user "Alice" has created a share with settings                                                                                        # FeatureContext::userHasCreatedAShareWithSettings()
      | path        | FOLDER |
      | shareType   | group  |
      | permissions | create |
      | shareWith   | grp1   |
    And user "Brian" has accepted share "/FOLDER" offered by user "Alice"                                                                     # FeatureContext::userHasReactedToShareOfferedBy()
    And user "Admin" has changed the quota of the "Alice Hansen" space to "1"                                                                 # 
    When user "Brian" uploads a file inside space "Shares Jail" with content "new description" to "/FOLDER/textfile.txt" using the WebDAV API # SpacesContext::theUserUploadsAFileToSpace()
    Then the HTTP status code should be "507"                                                                                                 # FeatureContext::thenTheHTTPStatusCodeShouldBe()
      HTTP status code 201 is not the expected value 507
      Failed asserting that 201 matches expected '507'.
    And as "Alice" file "/FOLDER/textfile.txt" should not exist

OCIS LOG:

ocis_1             | 2022-08-15T11:15:43Z ERR unary code=Unauthenticated end="15/Aug/2022:11:15:43 +0000" from=tcp://127.0.0.1:45126 pkg=rgrpc service=groups start="15/Aug/2022:11:15:43 +0000" time_ns=1602 traceid=00000000000000000000000000000000 uri=/cs3.identity.group.v1beta1.GroupAPI/GetGroup user-agent=grpc-go/1.48.0
ocis_1             | 2022-08-15T11:15:43Z ERR unary code=Unknown end="15/Aug/2022:11:15:43 +0000" from=tcp://127.0.0.1:58388 pkg=rgrpc service=gateway start="15/Aug/2022:11:15:43 +0000" time_ns=193989 traceid=00000000000000000000000000000000 uri=/cs3.gateway.v1beta1.GatewayAPI/GetGroup user-agent=grpc-go/1.48.0
ocis_1             | 2022-08-15T11:15:43Z ERR failed to send a message error="rpc error: code = Unknown desc = gateway: error calling GetGroup: rpc error: code = Unauthenticated desc = auth: core access token not found" event=ShareCreated service=notifications
ocis_1             | 2022/08/15 11:15:44 not registered: events.ReceivedShareUpdated
ocis_1             | 2022/08/15 11:15:44 not registered: events.ReceivedShareUpdated
ocis_1             | 2022/08/15 11:15:44 not registered: events.FileUploaded
ocis_1             | 2022/08/15 11:15:44 not registered: events.UserDeleted
ocis_1             | 2022/08/15 11:15:44 not registered: events.UserDeleted
ocis_1             | 2022/08/15 11:15:44 not registered: events.UserDeleted
ocis_1             | 2022/08/15 11:15:44 not registered: events.UserDeleted
ocis_1             | 2022-08-15T11:15:44Z ERR error using machine auth error="error: not found: unknown client id" authRes={"status":{"code":6,"message":"unknown client id","trace":"00000000000000000000000000000000"}} owner={"id":{"idp":"https://host.docker.internal:9200","opaque_id":"7a489169-36fa-4fb3-a22d-c1051968da74","type":1}} service=search
ocis_1             | 2022-08-15T11:15:44Z ERR failed to stat the changed resource error="error: not found: unknown client id" service=search
ocis_1             | 2022/08/15 11:15:44 not registered: events.GroupDeleted
ocis_1             | 2022/08/15 11:15:44 not registered: events.GroupDeleted
ocis_1             | 2022-08-15T11:15:45Z ERR error using machine auth error="error: not found: unknown client id" authRes={"status":{"code":6,"message":"unknown client id","trace":"00000000000000000000000000000000"}} owner={"id":{"idp":"https://host.docker.internal:9200","opaque_id":"388d1078-13ab-454b-93cd-0e9fe97a27dc","type":1}} service=search
ocis_1             | 2022-08-15T11:15:45Z ERR failed to stat the changed resource error="error: not found: unknown client id" service=search

When I try to reproduce this through curl, it gives me 507 🤷🏽‍♀️

I think there's some issue while setting the quota. Maybe it's not updated properly.

phil-davis · 2022-08-15T11:26:57Z

Note: PR #4369 removed 3 of 4 examples from expected-failures:

changeSpaces.feature:222
changeSpaces.feature:223
changeSpaces.feature:224

And it left changeSpaces.feature:221 in expected-failures. That seems unusual. So maybe there is some timing/race condition with the handling of the quota changes?

@micbar any thoughts? Maybe something in cs3org/reva#3133 has some flakiness?

SwikritiT · 2022-08-15T11:30:48Z

I'm running the server through docker. When I clean start docker container and run the above test for the first two times the tests passes with 507 status code.

ocis_1             | 2022/08/15 11:25:30 not registered: events.ShareCreated
ocis_1             | 2022-08-15T11:25:30Z ERR unary code=Unauthenticated end="15/Aug/2022:11:25:30 +0000" from=tcp://127.0.0.1:45130 pkg=rgrpc service=groups start="15/Aug/2022:11:25:30 +0000" time_ns=1871 traceid=00000000000000000000000000000000 uri=/cs3.identity.group.v1beta1.GroupAPI/GetGroup user-agent=grpc-go/1.48.0
ocis_1             | 2022-08-15T11:25:30Z ERR unary code=Unknown end="15/Aug/2022:11:25:30 +0000" from=tcp://127.0.0.1:58410 pkg=rgrpc service=gateway start="15/Aug/2022:11:25:30 +0000" time_ns=210780 traceid=00000000000000000000000000000000 uri=/cs3.gateway.v1beta1.GatewayAPI/GetGroup user-agent=grpc-go/1.48.0
ocis_1             | 2022-08-15T11:25:30Z ERR failed to send a message error="rpc error: code = Unknown desc = gateway: error calling GetGroup: rpc error: code = Unauthenticated desc = auth: core access token not found" event=ShareCreated service=notifications
ocis_1             | 2022/08/15 11:25:31 not registered: events.ReceivedShareUpdated
ocis_1             | 2022/08/15 11:25:31 not registered: events.ReceivedShareUpdated
ocis_1             | 2022-08-15T11:25:31Z ERR failed to initiate upload error="error: insufficient storage: quota exceeded" pkg=rgrpc service=storage-users status={"code":19,"message":"insufficient storage","trace":"00000000000000000000000000000000"} traceid=00000000000000000000000000000000
ocis_1             | 2022/08/15 11:25:31 not registered: events.UserDeleted
ocis_1             | 2022/08/15 11:25:31 not registered: events.UserDeleted

But from the 3rd time it starts failing and gives 201 instead of 507

ocis_1             | 2022/08/15 11:25:53 not registered: events.ShareCreated
ocis_1             | 2022-08-15T11:25:53Z ERR unary code=Unauthenticated end="15/Aug/2022:11:25:53 +0000" from=tcp://127.0.0.1:45130 pkg=rgrpc service=groups start="15/Aug/2022:11:25:53 +0000" time_ns=1978 traceid=00000000000000000000000000000000 uri=/cs3.identity.group.v1beta1.GroupAPI/GetGroup user-agent=grpc-go/1.48.0
ocis_1             | 2022-08-15T11:25:53Z ERR unary code=Unknown end="15/Aug/2022:11:25:53 +0000" from=tcp://127.0.0.1:58410 pkg=rgrpc service=gateway start="15/Aug/2022:11:25:53 +0000" time_ns=255520 traceid=00000000000000000000000000000000 uri=/cs3.gateway.v1beta1.GatewayAPI/GetGroup user-agent=grpc-go/1.48.0
ocis_1             | 2022-08-15T11:25:53Z ERR failed to send a message error="rpc error: code = Unknown desc = gateway: error calling GetGroup: rpc error: code = Unauthenticated desc = auth: core access token not found" event=ShareCreated service=notifications
ocis_1             | 2022/08/15 11:25:53 not registered: events.ReceivedShareUpdated
ocis_1             | 2022/08/15 11:25:53 not registered: events.ReceivedShareUpdated
ocis_1             | 2022/08/15 11:25:54 not registered: events.FileUploaded
ocis_1             | 2022/08/15 11:25:54 not registered: events.UserDeleted
ocis_1             | 2022/08/15 11:25:54 not registered: events.UserDeleted

The file gets uploaded

SwikritiT · 2022-08-16T07:21:03Z

The behavior is the same if I serve ocis locally from repo
For the first time, the scenario runs it works fine 507 is returned but from the second time onwards it gives 201. I still can't reproduce through curl req. I'll try to write bash script and see if I can reproduce it.

SwikritiT · 2022-08-17T09:24:49Z

The problem seems to be with the test. As spaces aren't cleared after the user is deleted the test is somehow getting the space id of the previously created user with the same name and setting the quota of that space id so when the user uploads file to a currently created user it passes with 201 as the user's quota has not been updated.

ScharfViktor · 2022-08-22T10:23:57Z

step When user "Admin" changes the quota of the "Brian Murphy" space to "<quotaValue>" changes quota of one of "n" possible users Brian (users don't delete after each test and admin have a lot of users with same name).

so second steps When user "Brian" uploads a file inside space "Brian Murphy" with content "file is more than 15 bytes" to "file.txt" using the WebDAV API fails when if admin did not guess the user "Brian"

will be work correct after fixing: #4196 and uncommenting https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/bootstrap/SpacesContext.php#L430-L431

SwikritiT · 2022-08-22T10:29:34Z

step When user "Admin" changes the quota of the "Brian Murphy" space to "<quotaValue>" changes quota of one of "n" possible users Brian (users don't delete after each test and admin have a lot of users with same name).

so second steps When user "Brian" uploads a file inside space "Brian Murphy" with content "file is more than 15 bytes" to "file.txt" using the WebDAV API fails when if admin did not guess the user "Brian"

will be work correct after fixing: #4196 and uncommenting https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/bootstrap/SpacesContext.php#L430-L431

That's true so what do we do for now, find a workaround or put the tests in expected to fail? The test does pass locally for me on the first run but from the second run there are a lot of brian's so it starts failing

SwikritiT · 2022-08-22T10:34:59Z

The tests in this PR are also failing because of the same reason #4371

ScharfViktor · 2022-08-22T11:17:02Z

step When user "Admin" changes the quota of the "Brian Murphy" space to "<quotaValue>" changes quota of one of "n" possible users Brian (users don't delete after each test and admin have a lot of users with same name).
so second steps When user "Brian" uploads a file inside space "Brian Murphy" with content "file is more than 15 bytes" to "file.txt" using the WebDAV API fails when if admin did not guess the user "Brian"
will be work correct after fixing: #4196 and uncommenting https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/bootstrap/SpacesContext.php#L430-L431

That's true so what do we do for now, find a workaround or put the tests in expected to fail? The test does pass locally for me on the first run but from the second run there are a lot of brian's so it starts failing

I suggest we wait for #4196
I will change the name of this issue and the related problem in the expected failures

phil-davis · 2022-08-22T11:42:02Z

I suggest we wait for #4196

yes, if that is implemented "soon".

Otherwise, the tests could remember each space that has been created, and who is the "manager" of that space (usually the user that created it, but there could be a few tests that "handover" the management of a space to another user during the test scenario. Then, before deleting users, the after-scenario can loop through and delete each space, using the authentication of the user who is the "manager" of that space. IMO that algorithm should work, not be difficult to implement, and will continue to work in the future anyway.

ScharfViktor · 2022-08-22T18:55:43Z

@phil-davis I can suggest this solution: #4433

phil-davis · 2022-08-23T04:47:47Z

PR #4433 cleans up the user personal spaces in AfterScenario. So that is a good start.

Is there some way that we can delete project spaces that were created during a scenario?

SwikritiT · 2022-08-23T05:29:15Z

PR #4433 cleans up the user personal spaces in AfterScenario. So that is a good start.

Is there some way that we can delete project spaces that were created during a scenario?

Someone that's granted with the Space Admin role can first disable and then delete the space.
This disables

curl -u user2:123456 -k -X DELETE 'https://host.docker.internal:9200/graph/v1.0/drives/<spaceId>'

This deletes

curl -u user2:123456 -k -X DELETE 'https://host.docker.internal:9200/graph/v1.0/drives/<spaceId>' -H"Purge:T" -v

We just can find out who the manager is of the created space, and then send a request from that user to delete the space in the after scenario. Could be a workaround until the admin has the permission to do so?

ScharfViktor · 2022-08-23T08:24:35Z

deleting the project space is nice to have, but it does not block any tests. here we can definitely wait #4196 IMHO

or implement deleting project space:

admin gets all project space graph/v1.0/drives
finds user with role "manager" in root->permissions section
user deletes space

should we wait or implement deleting project space? What do you think?

ScharfViktor · 2022-08-23T08:49:34Z

deleting spaces increased run time in CI - 28 min. https://drone.owncloud.com/owncloud/ocis/14804/41/7

we should probably parallelize the tests

phil-davis · 2022-08-23T08:54:53Z

we should probably parallelize the tests

Yes, agree. There are 342 scenarios - so it is good that so many run in less than 30 minutes!

A few other pipelines are taking 20 to 25 minutes. But that is a separate thing to look at.

Is there some sensible way to name 2 apiSpacesZzzzz suites that would have about half the scenarios in each?

Or do we just name them apiSpaces1 and apiSpaces2?

phil-davis · 2022-08-23T08:55:41Z

should we wait or implement deleting project space? What do you think?

The code to delete project spaces should be "easy" - why not just do it now?

ScharfViktor · 2022-08-23T13:20:28Z

The code to delete project spaces should be "easy" - why not just do it now?

part 2 #4441

saw-jan added the QA:team label Aug 15, 2022

SagarGi self-assigned this Aug 15, 2022

SagarGi mentioned this issue Aug 15, 2022

[tests-only][full-ci]Changing personal quota on another user as admin is possible #4391

Closed

SagarGi assigned SwikritiT Aug 15, 2022

ScharfViktor changed the title ~~Local API test apiSpaces/changeSpaces.feature:221 passed on nightly~~ Local API test apiSpaces/changeSpaces.feature:221 fails because there is no cleanup after each test Aug 22, 2022

phil-davis mentioned this issue Aug 23, 2022

[tests-only] [full-ci] ApiTest. after each test users delete their personal space #4433

Merged

ScharfViktor mentioned this issue Aug 23, 2022

[tests-only] [full-ci] ApiTest. after each test users-manager delete their project space #4441

Merged

ScharfViktor closed this as completed Aug 25, 2022

This was referenced Aug 25, 2022

[tests-only] [full-ci] Cleanup spaces in all local API test suites #4456

Merged

User cleanup during API acceptance tests #4457

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Local API test apiSpaces/changeSpaces.feature:221 fails because there is no cleanup after each test #4390

Local API test apiSpaces/changeSpaces.feature:221 fails because there is no cleanup after each test #4390

saw-jan commented Aug 15, 2022 •

edited by ScharfViktor

Loading

SagarGi commented Aug 15, 2022 •

edited

Loading

SwikritiT commented Aug 15, 2022

SwikritiT commented Aug 15, 2022 •

edited

Loading

phil-davis commented Aug 15, 2022

SwikritiT commented Aug 15, 2022 •

edited

Loading

SwikritiT commented Aug 16, 2022

SwikritiT commented Aug 17, 2022

ScharfViktor commented Aug 22, 2022

SwikritiT commented Aug 22, 2022

SwikritiT commented Aug 22, 2022

ScharfViktor commented Aug 22, 2022

phil-davis commented Aug 22, 2022

ScharfViktor commented Aug 22, 2022

phil-davis commented Aug 23, 2022

SwikritiT commented Aug 23, 2022

ScharfViktor commented Aug 23, 2022

ScharfViktor commented Aug 23, 2022

phil-davis commented Aug 23, 2022

phil-davis commented Aug 23, 2022

ScharfViktor commented Aug 23, 2022

Local API test apiSpaces/changeSpaces.feature:221 fails because there is no cleanup after each test #4390

Local API test apiSpaces/changeSpaces.feature:221 fails because there is no cleanup after each test #4390

Comments

saw-jan commented Aug 15, 2022 • edited by ScharfViktor Loading

SagarGi commented Aug 15, 2022 • edited Loading

SwikritiT commented Aug 15, 2022

SwikritiT commented Aug 15, 2022 • edited Loading

phil-davis commented Aug 15, 2022

SwikritiT commented Aug 15, 2022 • edited Loading

SwikritiT commented Aug 16, 2022

SwikritiT commented Aug 17, 2022

ScharfViktor commented Aug 22, 2022

SwikritiT commented Aug 22, 2022

SwikritiT commented Aug 22, 2022

ScharfViktor commented Aug 22, 2022

phil-davis commented Aug 22, 2022

ScharfViktor commented Aug 22, 2022

phil-davis commented Aug 23, 2022

SwikritiT commented Aug 23, 2022

ScharfViktor commented Aug 23, 2022

ScharfViktor commented Aug 23, 2022

phil-davis commented Aug 23, 2022

phil-davis commented Aug 23, 2022

ScharfViktor commented Aug 23, 2022

saw-jan commented Aug 15, 2022 •

edited by ScharfViktor

Loading

SagarGi commented Aug 15, 2022 •

edited

Loading

SwikritiT commented Aug 15, 2022 •

edited

Loading

SwikritiT commented Aug 15, 2022 •

edited

Loading