Migration and Update Guide for 5 to 7

antora.yml

@@ -22,6 +22,9 @@ asciidoc:
     # https://github.com/owncloud/ocis-charts/tree/master/charts/ocis/docs
     s-path: 'deployment/services/s-list'
 
+    # the name of the sub directory that defines the sources used in the compose deployment examples
+    ocis_wopi: ocis_full
+
     # note that service_url_component is used for services ONLY
     # service_url_component will be used to assemble the url for services to include content (tables) sourced from the ocis repo.
     service_url_component: 'docs' # docs for master or for stable, like docs-stable-5.0
@@ -48,7 +51,7 @@ asciidoc:
 
     # this is the first part of the name for envvars between major versions that will be added or removed
     # example for full name: 4.0.0-5.0.0-added.adoc or 4.0.0-5.0.0-removed.adoc
-    env_var_delta_name: '4.0.0-5.0.0'
+    env_var_delta_name: '5.0.0-7.0.0'
 
     # helm_tab_x will be used to assemble the url (tag) accessing the raw content for helm charts (tag)
     # note that tab 2 always contains the actual release and tab 3 the former

modules/ROOT/pages/depl-examples/ubuntu-compose/ubuntu-compose-hetzner.adoc

@@ -4,9 +4,6 @@
 :keywords: docker compose, raspberry pi, install, ocis, infinite scale, letsencrypt
 :description: Install Infinite Scale using Docker Compose on the Hetzner Cloud for production use.
 
-// the folder to use for the example
-:ocis_wopi: ocis_full
-
 include::partial$multi-location/compose-version.adoc[]
 
 image:depl-examples/ubuntu-compose/ubuntu-basic-teaser-image.png[Teaser Image, width=650]

modules/ROOT/pages/depl-examples/ubuntu-compose/ubuntu-compose-prod.adoc

@@ -4,9 +4,6 @@
 :keywords: docker compose, raspberry pi, install, ocis, infinite scale, letsencrypt
 :description: Install Infinite Scale using Docker Compose on a server for production use.
 
-// the folder to use for the example
-:ocis_wopi: ocis_full
-
 include::partial$multi-location/compose-version.adoc[]
 
 image:depl-examples/ubuntu-compose/ubuntu-basic-teaser-image.png[Teaser Image, width=650]

modules/ROOT/pages/deployment/general/general-info.adoc

@@ -622,13 +622,15 @@ An admin user will be created when running the `ocis init` command with the foll
 
 Login to the webinterface with this admin user and change relevant data according your needs or create new users. As an example to reach out the webinterface use `\https://localhost:9200`.
 
-=== Password Reset for the Admin User
+=== Password Reset for IDM Users
 
 The admin password can be reset via command line if it has been forgotten and the admin can't enter the webUI anymore. Note that the admin user must already exist which happens if you have xref:deployment/general/ocis-init.adoc[initialized Infinite Scale].
 
-After running the respective command and entering a new password, the admin can relogin using the new password.
+See the command below for details and options.
+
+After running the respective command and entering a new password, the admin or user can relogin using the new password.
 
-Note that when Infinite Scale gets initialized with xref:deployment/general/ocis-init.adoc[The ocis init Command], an admin password is created and stored in the `ocis.yaml` file. The lifespan of this admin password is up to the point when it either gets changed in the webUI or via the resetpassword command. Any admin password changes are NOT written back to the `ocis.yaml` file nor manual changes to the `admin_password` are considered as a new password.
+Note that when Infinite Scale gets initialized with xref:deployment/general/ocis-init.adoc[The ocis init Command], an admin password is created and stored in the `ocis.yaml` file. The lifespan of this admin password is up to the point when it either gets changed in the webUI or via the resetpassword command. Any admin password changes are *NOT* written back to the `ocis.yaml` file nor manual changes to the `admin_password` are considered as a new password.
 
 [source,yaml]
 ----
@@ -657,7 +659,7 @@ When the prerequisite from the note above is fulfilled, you can reset the admin
 
 ==== Binary Setup
 
-Run the following command to reset the admin password:
+Run the following command to reset the admin password, use `--user-name <user>` for ordinary users:
 
 [source,bash]
 ----

modules/ROOT/pages/deployment/services/env-var-changes.adoc

@@ -1,9 +1,9 @@
 # Changed Environment Variables in Versions
 :toc: right
-:description: This page contains tables with added and removed environment variables between Infinite Scale version 4.0.0 and 5.0.0.
+:description: This page contains tables with added and removed environment variables between Infinite Scale version 5.0.x and 7.0.0.
 
 :source_path: {ocis_services_raw_url}{service_url_component}{ocis_services_final_path}adoc/env-var-deltas/
-// https://raw.githubusercontent.com/owncloud/ocis/docs-stable-5.0/services/_includes/adoc/env-var-deltas/
+// https://raw.githubusercontent.com/owncloud/ocis/docs-stable-7.0/services/_includes/adoc/env-var-deltas/
 
 == Introduction
 
@@ -23,4 +23,9 @@ Removed::
 --
 include::{source_path}{env_var_delta_name}-removed.adoc[]
 --
+Deprecated::
++
+--
+include::{source_path}{env_var_delta_name}-deprecated.adoc[]
+--
 ====

modules/ROOT/pages/deployment/services/s-list/frontend.adoc

@@ -103,11 +103,11 @@ Aggregating share information is one of the most time consuming operations in OC
 
 To save network trips the sharing implementation can cache the stat requests with an in memory cache or in redis. It will shorten the response time by the network round-trip overhead at the cost of the API only eventually being updated.
 
-Setting `FRONTEND_OCS_RESOURCE_INFO_CACHE_TTL=60` would cache the stat info for 60 seconds. Increasing this value makes sense for large deployments with thousands of active users that keep the cache up to date. Low frequency usage scenarios should not expect a noticeable improvement.
+Setting `FRONTEND_OCS_RESOURCE_INFO_CACHE_TTL=60` (deprecated) would cache the stat info for 60 seconds. Increasing this value makes sense for large deployments with thousands of active users that keep the cache up to date. Low frequency usage scenarios should not expect a noticeable improvement.
 
 == Scalability
 
-While the frontend service does not persist any data, it does cache information about files and filesystem (`Stat()`) responses and user information. Therefore, multiple instances of this service can be spawned in a bigger deployment like when using container orchestration with Kubernetes, when configuring `FRONTEND_OCS_RESOURCE_INFO_CACHE_STORE` and the related config options.
+While the frontend service does not persist any data, it does cache information about files and filesystem (`Stat()`) responses and user information. Therefore, multiple instances of this service can be spawned in a bigger deployment like when using container orchestration with Kubernetes, when configuring `FRONTEND_OCS_RESOURCE_INFO_CACHE_STORE` (deprecated) and the related config options.
 
 == Define Read-Only Attributes
 

modules/ROOT/pages/deployment/services/s-list/postprocessing.adoc

@@ -63,39 +63,73 @@ The backoff behavior as mentioned in the `retry` outcome can be configured using
 
 === Resume Post-Processing
 
-If post-processing fails in one step due to an unforeseen error, current uploads will not be retried automatically. A system administrator can instead run CLI commands to retry the failed upload which is a two step process. For details on the `storage-users` command see the xref:{s-path}/storage-users.adoc#manage-unfinished-uploads[Manage Unfinished Uploads] documentation.
+[NOTE]
+====
+If not noted otherwise, commands with the `restart` option can also use the `resume` option. This changes behaviour slightly.
 
-* First list ongoing upload sessions:
+* `restart` +
+When restarting an upload, all steps for open items will be restarted, except if otherwise defined.
+* `resume` +
+When resuming an upload, processing will continue unfinished items from their last completed step.
+====
+
+If post-processing fails in one step due to an unforeseen error, current uploads will not be resumed automatically. A system administrator can instead run CLI commands to resume the failed upload manually which is at minimum a two step process.
+
+NOTE: For details on the `storage-users` command which provides many options see the xref:{s-path}/storage-users.adoc#manage-unfinished-uploads[Manage Unfinished Uploads] documentation.
+
+Depending if you want to restart/resume all or defined failed uploads, different commands are used.
+
+* First, list ongoing upload sessions to identify possibly failed ones. +
+Note that there never can be a clear identification of a failed upload session due to various reasons causing them. You need to apply more critera like free space on disk, a failed service like antivirus etc. to declare an upload as failed.
 +
 [source,bash]
 ----
 ocis storage-users uploads sessions
 ----
 
-* If you want to restart *all* uploads, just rerun the command with the `--restart` flag:
+{empty}
+
+* *All failed uploads*
 +
+--
+If you want to restart/resume all failed uploads, just rerun the command with the relevant flag. Note that this is the preferred command to handle failed processing steps:
+
 [source,bash]
 ----
-ocis storage-users uploads sessions --restart
+ocis storage-users uploads sessions --resume
 ----
+{empty}
+--
 
-* If you want to restart *only one* upload, use the postprocessing restart command with the ID selected:
+* *Particular failed uploads* +
+Use the `postprocessing` command to resume defined failed uploads. For postprocessing steps, the default is to resume . Note that at the moment, `resume` is an alias for `restart` to keep old functionality. `restart` is subject to change and will most likely be removed in a later version.
+
+** *Defined by ID*
 +
+--
+If you want to resume only a specific upload, use the postprocessing resume command with the ID selected:
+
 [source,bash]
 ----
-ocis postprocessing restart -u <uploadID>
+ocis postprocessing resume -u <uploadID>
 ----
+{empty}
+--
 
-Alternatively, instead of starting one specific upload, a system admin can also restart all uploads that are currently in a specific step.
+** *Defined by step*
++
+--
+Alternatively, instead of restarting one specific upload, a system admin can also resume all uploads that are currently in a specific step.
 
 Examples:
 
 [source,bash]
 ----
-ocis postprocessing restart                # Restarts all uploads where postprocessing is finished, but upload is not finished.
-ocis postprocessing restart -s "finished"  # Equivalent to the above.
-ocis postprocessing restart -s "virusscan" # Restart all uploads currently in virusscan step.
+ocis postprocessing resume                # Resumes all uploads where postprocessing is finished, but upload is not finished.
+ocis postprocessing resume -s "finished"  # Equivalent to the above.
+ocis postprocessing resume -s "virusscan" # Resume all uploads currently in virusscan step.
 ----
+--
 
 == Storing
 

modules/ROOT/pages/deployment/services/s-list/proxy.adoc

@@ -5,6 +5,8 @@
 
 :service_name: proxy
 
+include::partial$multi-location/compose-version.adoc[]
+
 == Introduction
 
 {description} Every HTTP request goes through this service. Authentication, logging and other preprocessing of requests also happens here. Mechanisms like request rate limiting or intrusion prevention are **not** included in the proxy service and must be set up in front of an external reverse proxy.
@@ -174,6 +176,22 @@ The default `role_claim` (or `PROXY_ROLE_ASSIGNMENT_OIDC_CLAIM`) is `roles`. The
 
 * In a production deployment, you want to have basic authentication (`PROXY_ENABLE_BASIC_AUTH`) _disabled_ which is the default state. You should also set up a firewall to only allow requests to the proxy service or the reverse proxy if you have one. Requests to the other services should be blocked by the firewall.
 
+=== Content Security Policy
+
+What is a Content Security Policy (CSP) and why is it used in Infinite Scale::
++
+--
+A Content Security Policy (CSP) is a feature that helps to prevent or minimize the risk of certain types of security threats. It consists of a series of instructions from a website to a browser, which instruct the browser to place restrictions on the things that the code comprising site is allowed to do. It is mainly used as a defense against cross-site scripting (XSS) attacks, in which an attacker is able to inject malicious code into the victim's site and includes defending against clickjacking, and helping to ensure that a site's pages will be loaded over HTTPS.
+
+For Infinite Scale, external resources like an IDP (e.g. Keycloak) or when using web office documents or web apps, require defining a CSP. If not defined, the referenced services will not work.
+--
+
+To create a Content Security Policy (CSP), you need to create a yaml file containing the CSP definitions. To activate the settings, reference the file as value in the `PROXY_CSP_CONFIG_FILE_LOCATION` environment variable. For each change, a restart of the Infinite Scale deployment or the proxy service is required.
+
+A working example for a CSP can be found in a sub path of the `config` directory of the {compose_url}v{compose_version}{compose_final_path}/{ocis_wopi}[{ocis_wopi},window=_blank] deployment example which is the base for our xref:depl-examples/ubuntu-compose/ubuntu-compose-prod.adoc[Local Production Setup] and the xref:depl-examples/ubuntu-compose/ubuntu-compose-hetzner.adoc[Deployment on Hetzner].
+
+See the https://content-security-policy.com[Content Security Policy (CSP) Quick Reference Guide,window=_blank] for a description of directives.
+
 == Presigned Urls
 
 Important, also see section xref:caching[caching] above.

modules/ROOT/pages/deployment/services/s-list/storage-users.adoc

@@ -23,7 +23,7 @@ IMPORTANT: The graceful shutdown period is only applicable if the `storage-users
 
 When hard-stopping Infinite Scale, for example with the `kill <pid>` command (SIGKILL), it is possible and likely that not all data from the decomposedfs (metadata) has been written to the storage which may result in an inconsistent decomposedfs. When gracefully shutting down Infinite Scale, using a command like SIGTERM, the process will no longer accept any write requests from _other_ services and will try to write the internal open  requests which can take an undefined duration based on many factors. To mitigate that situation, the following things have been implemented:
 
-*   With the value of the environment variable `STORAGE_USERS_GRACEFUL_SHUTDOWN_TIMEOUT`, the `storage-users` service will delay its shutdown giving it time to finalize writing necessary data. This delay can be necessary if there is a lot of data to be saved and/or if storage access/thruput is slow. In such a case you would receive an error log entry informing you that not all data could be saved in time. To prevent such occurrences, you must increase the default value.
+*   With the value of the environment variable `STORAGE_USERS_GRACEFUL_SHUTDOWN_TIMEOUT`, the `storage-users` service will delay its shutdown giving it time to finalize writing necessary data. This delay can be necessary if there is a lot of data to be saved and/or if storage access/throughput is slow. In such a case you would receive an error log entry informing you that not all data could be saved in time. To prevent such occurrences, you must increase the default value.
 
 *   If a shutdown error has been logged, the command-line maintenance tool xref:maintenance/commands/commands.adoc#inspect-and-manipulate-node-metadata[Inspect and Manipulate Node Metadata] can help to fix the issue. Please contact support for details. 
 
@@ -69,22 +69,30 @@ ocis storage-users uploads <command>
 ----
 COMMANDS:
    sessions  Print a list of upload sessions
-   clean     Clean up leftovers from expired uploads (deprecated, do not use it)
-   list      Print a list of all incomplete uploads  (deprecated, do not use it)
 ----
 --
 
 ==== Sessions command
 
-The `sessions` command is the entry point for listing, restarting and cleaning unfinished uploads.
+The `sessions` command is the entry point for listing, restarting/resuming and cleaning unfinished uploads.
+
+NOTE: There can never be a clear identification of a failed upload session due to various reasons causing them. You need to apply more critera like free space on disk, a failed service like antivirus etc. to declare an upload as failed.
+
+[NOTE]
+====
+If not noted otherwise, the `restart` option can also be replaced with the `resume` option. This changes behaviour slightly.
+
+* `restart` +
+When restarting an upload, all steps for open items will be restarted, except if otherwise defined.
+* `resume` +
+When resuming an upload, processing will continue unfinished items from their last completed step.
+====
 
 [source,bash]
 ----
 ocis storage-users uploads sessions
 ----
 
-Note, though possible, do not use the deprecated upload commands listed above. Their function has been embedded in the session command as command options.
-
 [source,plaintext]
 ----
 NAME:
@@ -100,6 +108,7 @@ OPTIONS:
    --has-virus   filter sessions by virus scan result (default: unset)
    --json        output as json                       (default: false)
    --restart     send restart event for all listed sessions (default: false)
+   --resume      send resume event for all listed sessions (default: false)
    --clean       remove uploads                       (default: false)
    --help, -h    show help
 ----
@@ -165,7 +174,7 @@ ocis storage-users uploads sessions --expired=false --processing=false
 |
 |===
 
-The sessions command can also clear and restart uploads. The output is the same as if run without the `--clean` or `--restart` flag. Note that it is recommended to run the command first without the `--clean` (`--processing`) flag to double check which uploads get cleaned (restarted).
+The sessions command can also clear and restart/resume uploads. The output is the same as if run without the `--clean` or `--restart/--resume` option. Note that it is recommended to run the command first without the `--clean` (`--processing`) option to double check which uploads would get cleaned (restarted/resumed).
 
 .Cleans all expired uploads regardless of processing and virus state.
 [source,bash]
@@ -175,13 +184,13 @@ ocis storage-users uploads sessions \
      --clean
 ----
 
-.Restarts all uploads that are processing and are not virus infected
+.Resume all uploads that are currently processing and are not virus infected
 [source,bash]
 ----
 ocis storage-users uploads sessions \
      --processing=false \
      --has-virus=false \
-     --restart
+     --resume
 ----
 
 === Manage Trash-Bin Items