Skip to content

Commit

Permalink
commit eb0ad41
Browse files Browse the repository at this point in the history 
Author: Viktor Scharf <[email protected]>
Date:   Fri Jan 20 11:54:25 2023 +0100

    [docs-only] Change Testing section in the doc (#5430)

    * change docs

    * fix after review
  • Loading branch information
@ScharfViktor
ScharfViktor committed Jan 20, 2023
1 parent b004329 commit c25519c
Show file tree
Hide file tree
Showing 2 changed files with 59 additions and 66 deletions.
75 changes: 34 additions & 41 deletions ocis/development/testing.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -89,15 +89,12 @@ make -C tests/acceptance/docker clean

## Testing with test suite natively installed

We are using the ownCloud 10 acceptance test suite against oCIS.
We have two sets of tests:
- `test-acceptance-from-core-api` set was transferred from [core](https://github.com/owncloud/core) repository
The suite name of all tests transferred from the core starts with "core"

### Getting the tests
- `test-acceptance-api` set was created for ocis. Mainly for testing spaces features

All you need to do to get the acceptance tests is check out the core repo:

```bash
git clone https://github.com/owncloud/core.git
```

### Run ocis

Expand All @@ -106,47 +103,49 @@ Create an up-to-date ocis binary by [building oCIS]({{< ref "build" >}})
To start ocis:

```bash
IDM_ADMIN_PASSWORD=admin ocis/bin/ocis init --insecure true
OCIS_INSECURE=true PROXY_ENABLE_BASIC_AUTH=true ocis/bin/ocis server
IDM_ADMIN_PASSWORD=admin \
ocis/bin/ocis init --insecure true

OCIS_INSECURE=true PROXY_ENABLE_BASIC_AUTH=true \
ocis/bin/ocis server
```

`PROXY_ENABLE_BASIC_AUTH` will allow the acceptance tests to make requests against the provisioning api (and other endpoints) using basic auth.

### Run the acceptance tests

First we will need to clone the testing app in owncloud which contains the skeleton files required for running the tests.
In the ownCloud 10 core clone the testing app with the following command:
### Run the test-acceptance-from-core-api tests

```bash
git clone https://github.com/owncloud/testing apps/testing
make test-acceptance-from-core-api \
TEST_SERVER_URL=https://localhost:9200 \
TEST_WITH_GRAPH_API=true \
TEST_OCIS=true \
```
Note: This command only works for suites that start with core

Then run the api acceptance tests with the following command from the root of the ownCloud 10 core repository:
### Run the test-acceptance-api tests

```bash
make test-acceptance-api \
TEST_SERVER_URL=https://localhost:9200 \
TEST_WITH_GRAPH_API=true \
PATH_TO_OCIS=/var/www/ocis \
PATH_TO_CORE=. \
TEST_OCIS=true \
STORAGE_DRIVER=OCIS \
SKELETON_DIR=apps/testing/data/apiSkeleton \
BEHAT_FILTER_TAGS='~@notToImplementOnOCIS&&~@toImplementOnOCIS'
```

Make sure to adjust the settings `TEST_SERVER_URL` and `PATH_TO_OCIS` according to your environment.
Make sure to adjust the settings `TEST_SERVER_URL` according to your environment.

This will run all tests that are relevant to oCIS.

To run a single feature add `BEHAT_FEATURE=<feature file>`
To run a single feature add `BEHAT_FEATURE=<feature file>`
example: `BEHAT_SUITE=tests/acceptance/features/apiGraph/createUser.feature`
To run a single test add `BEHAT_FEATURE=<file.feature:(line number)>`
example: `BEHAT_SUITE=tests/acceptance/features/apiGraph/createUser.feature:12`
To run a single suite add `BEHAT_SUITE=<test suite>`
example: `BEHAT_SUITE=apiGraph`

To run tests with a different storage driver set `STORAGE_DRIVER` to the correct value. It can be set to `OCIS` or `OWNCLOUD` and uses `OWNCLOUD` as the default value.

### use existing tests for BDD

As a lot of scenarios are written for oC10, we can use those tests for Behaviour driven development in ocis.
Every scenario that does not work in oCIS with "ocis" storage, is listed in `tests/acceptance/expected-failures-on-OCIS-storage.md` with a link to the related issue.
As a lot of scenarios from `test-acceptance-from-core-api` are written for oC10, we can use those tests for Behaviour driven development in ocis.
Every scenario that does not work in oCIS with "ocis" storage, is listed in `tests/acceptance/expected-failures-API-on-OCIS-storage.md` with a link to the related issue.

Those scenarios are run in the ordinary acceptance test pipeline in CI. The scenarios that fail are checked against the
expected failures. If there are any differences then the CI pipeline fails.
Expand All @@ -155,30 +154,24 @@ The tests are not currently run in CI with the OWNCLOUD or EOS storage drivers,

If you want to work on a specific issue

1. adjust the core commit id to the latest commit in core so that CI will run the latest test code and scenarios from core.
For that change `CORE_COMMITID` in `.drone.env`:

# The test runner source for API tests
CORE_COMMITID=38c91e5cf5fc4ffdc0536ba1d147a2a618ef83b5
CORE_BRANCH=master

2. locally run each of the tests marked with that issue in the expected failures file.
1. locally run each of the tests marked with that issue in the expected failures file.

E.g.:

```bash
make test-acceptance-api \
make test-acceptance-from-core-api \
TEST_SERVER_URL=https://localhost:9200 \
TEST_OCIS=true \
TEST_WITH_GRAPH_API=true \
STORAGE_DRIVER=OCIS \
BEHAT_FEATURE='tests/acceptance/features/apiComments/comments.feature:123'
BEHAT_FEATURE='tests/acceptance/features/coreApiVersions/fileVersions.feature:147'
```

3. the tests will fail, try to understand how and why they are failing
4. fix the code
5. go back to 2. and repeat till the tests are passing.
6. remove those tests from the expected failures file
7. make a PR that has the fixed code, and the relevant lines removed from the expected failures file.
2. the tests will fail, try to understand how and why they are failing
3. fix the code
4. go back to 1. and repeat till the tests are passing.
5. remove those tests from the expected failures file
6. make a PR that has the fixed code, and the relevant lines removed from the expected failures file.

## Running tests for parallel deployment

Expand Down
50 changes: 25 additions & 25 deletions services/_includes/adoc/global_configvars.adoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ a| [subs=-attributes]
++false ++

a| [subs=-attributes]
The default role assignments the demo users should be setup.
Flag to enable or disable the creation of the demo users.

a| `LDAP_BIND_DN`

Expand All @@ -41,7 +41,7 @@ a| [subs=-attributes]
++string ++

a| [subs=-attributes]
++uid=reva,ou=sysusers,o=libregraph-idm ++
++uid=idp,ou=sysusers,o=libregraph-idm ++

a| [subs=-attributes]
LDAP DN to use for simple bind authentication with the target LDAP server.
Expand Down Expand Up @@ -131,7 +131,7 @@ a| [subs=-attributes]
++groupOfNames ++

a| [subs=-attributes]
The object class to use for groups in the default group search filter like 'groupOfNames'.
The object class to use for groups in the default group search filter ('groupOfNames').

a| `LDAP_GROUP_SCHEMA_DISPLAYNAME`

Expand Down Expand Up @@ -181,7 +181,7 @@ a| [subs=-attributes]
++ownclouduuid ++

a| [subs=-attributes]
LDAP Attribute to use as the unique ID for groups. This should be a stable globally unique ID like a UUID.
LDAP Attribute to use as the unique id for groups. This should be a stable globally unique ID like a UUID.

a| `LDAP_GROUP_SCHEMA_ID_IS_OCTETSTRING`

Expand Down Expand Up @@ -246,7 +246,7 @@ a| [subs=-attributes]
++sub ++

a| [subs=-attributes]
LDAP search scope to use when looking up groups. Supported values are 'base', 'one' and 'sub'.
LDAP search scope to use when looking up groups. Supported scopes are 'base', 'one' and 'sub'.

a| `LDAP_INSECURE`

Expand Down Expand Up @@ -282,7 +282,7 @@ a| [subs=-attributes]
++ldaps://localhost:9235 ++

a| [subs=-attributes]
URI of the LDAP Server to connect to. Supported URI schemes are 'ldaps://' and 'ldap://'
Url of the LDAP service to use as IDP.

a| `LDAP_USER_BASE_DN`

Expand Down Expand Up @@ -336,7 +336,7 @@ a| [subs=-attributes]
++inetOrgPerson ++

a| [subs=-attributes]
The object class to use for users in the default user search filter like 'inetOrgPerson'.
LDAP User ObjectClass like 'inetOrgPerson'.

a| `LDAP_USER_SCHEMA_DISPLAYNAME`

Expand Down Expand Up @@ -367,10 +367,10 @@ a| [subs=-attributes]
++string ++

a| [subs=-attributes]
++ownclouduuid ++
++uid ++

a| [subs=-attributes]
LDAP Attribute to use as the unique id for users. This should be a stable globally unique id like a UUID.
LDAP User uuid attribute like 'uid'.

a| `LDAP_USER_SCHEMA_ID_IS_OCTETSTRING`

Expand All @@ -386,7 +386,7 @@ a| [subs=-attributes]
++false ++

a| [subs=-attributes]
Set this to true if the defined 'id' attribute for users is of the 'OCTETSTRING' syntax. This is e.g. required when using the 'objectGUID' attribute of Active Directory for the user ID's.
Set this to true if the defined 'id' attribute for users is of the 'OCTETSTRING' syntax. This is e.g. required when using the 'objectGUID' attribute of Active Directory for the user id's.

a| `LDAP_USER_SCHEMA_MAIL`

Expand All @@ -404,7 +404,7 @@ a| [subs=-attributes]
++mail ++

a| [subs=-attributes]
LDAP Attribute to use for the email address of users.
LDAP User email attribute like 'mail'.

a| `LDAP_USER_SCHEMA_USERNAME`

Expand All @@ -419,10 +419,10 @@ a| [subs=-attributes]
++string ++

a| [subs=-attributes]
++uid ++
++displayName ++

a| [subs=-attributes]
LDAP Attribute to use for username of users.
LDAP User name attribute like 'displayName'.

a| `LDAP_USER_SCOPE`

Expand All @@ -440,7 +440,7 @@ a| [subs=-attributes]
++sub ++

a| [subs=-attributes]
LDAP search scope to use when looking up users. Supported values are 'base', 'one' and 'sub'.
LDAP search scope to use when looking up users. Supported scopes are 'base', 'one' and 'sub'.

a| `OCIS_ADMIN_USER_ID`

Expand All @@ -466,13 +466,13 @@ a| [subs=attributes+]
* xref:{s-path}/storage-users.adoc[storage-users] +

a| [subs=-attributes]
++string ++
++[]string ++

a| [subs=-attributes]
++ ++
++[] ++

a| [subs=-attributes]
A comma-separated list of addresses to connect to. Only valid if the above setting is set to "etcd"
Node addresses to use for the cache store.

a| `OCIS_CACHE_STORE_SIZE`

Expand Down Expand Up @@ -501,10 +501,10 @@ a| [subs=-attributes]
++string ++

a| [subs=-attributes]
++ ++
++memory ++

a| [subs=-attributes]
The type of the cache store. Valid options are "noop", "ocmem", "etcd" and "memory"
Store implementation for the cache. Valid values are "memory" (default), "redis", and "etcd".

a| `OCIS_CORS_ALLOW_CREDENTIALS`

Expand Down Expand Up @@ -853,7 +853,7 @@ a| [subs=-attributes]
++false ++

a| [subs=-attributes]
Whether to verify the server TLS certificates.
Whether the NATS server should skip the client certificate verification during the TLS handshake.

a| `OCIS_JWT_SECRET`

Expand Down Expand Up @@ -1078,7 +1078,7 @@ a| [subs=-attributes]
++ ++

a| [subs=-attributes]
Machine auth API key used to validate internal requests necessary to access resources from other services.
Machine auth API key used to validate internal requests necessary for the access to resources from other services.

a| `OCIS_OIDC_ISSUER`

Expand All @@ -1099,7 +1099,7 @@ a| [subs=-attributes]
++https://localhost:9200 ++

a| [subs=-attributes]
The identity provider value to set in the userids of the CS3 user objects for users returned by this user provider.
The OIDC issuer URL to use.

a| `OCIS_SYSTEM_USER_API_KEY`

Expand Down Expand Up @@ -1338,10 +1338,10 @@ a| [subs=-attributes]
++string ++

a| [subs=-attributes]
++https://localhost:9200 ++
++https://localhost:9200/ ++

a| [subs=-attributes]
The identity provider value to set in the userids of the CS3 user objects for users returned by this user provider.
Base url to navigate back from the app the containing folder in the file list.

a| `REVA_GATEWAY`

Expand Down Expand Up @@ -1392,7 +1392,7 @@ a| [subs=-attributes]
++ ++

a| [subs=-attributes]
Transfer secret for signing file up- and download requests.
The storage transfer secret.

a| `STORAGE_USERS_OCIS_ASYNC_UPLOADS`

Expand Down

0 comments on commit c25519c

Please sign in to comment.