Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs-only] Update service readme's #10459

Merged
merged 1 commit into from
Nov 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion services/antivirus/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ The antivirus service currently supports [ICAP](https://tools.ietf.org/html/rfc3

Several factors can make it necessary to limit the maximum filesize the antivirus service will use for scanning. Use the `ANTIVIRUS_MAX_SCAN_SIZE` environment variable to scan only a given amount of bytes. Obviously, it is recommended to scan the whole file, but several factors like scanner type and version, bandwidth, performance issues, etc. might make a limit necessary.

> [!CAUTION]
**IMPORTANT**
> Streaming of files to the virus scan service still [needs to be implemented](https://github.com/owncloud/ocis/issues/6803). To prevent OOM errors `ANTIVIRUS_MAX_SCAN_SIZE` needs to be set lower than available ram.

### Antivirus Workers
Expand Down
54 changes: 36 additions & 18 deletions services/postprocessing/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,29 +75,47 @@ See the [cs3 org](https://github.com/cs3org/reva/blob/edge/pkg/events/postproces

### Resume Postprocessing

If postprocessing fails in one step due to an unforseen error, current uploads will not be retried automatically. A system admin can instead run a CLI command to retry the failed upload which is a two step process. For details on the `storage-users` command see the **Manage Unfinished Uploads** documentation in the `storage-users` service documentation:
**IMPORTANT**
> If not noted otherwise, commands with the `restart` option can also use the `resume` option. This changes behaviour slightly.
>
> * `restart`\
> When restarting an upload, all steps for open items will be restarted, except if otherwise defined.
> * `resume`\
> When resuming an upload, processing will continue unfinished items from their last completed step.

- First list ongoing upload sessions
```bash
ocis storage-users uploads sessions
```
If post-processing fails in one step due to an unforeseen error, current uploads will not be resumed automatically. A system administrator can instead run CLI commands to resume the failed upload manually which is at minimum a two step process.

For details on the `storage-users` command see the **Manage Unfinished Uploads** documentation in the `storage-users` service documentation.

Depending if you want to restart/resume all or defined failed uploads, different commands are used.

- First, list ongoing upload sessions to identify possibly failed ones.\
Note that there never can be a clear identification of a failed upload session due to various reasons causing them. You need to apply more critera like free space on disk, a failed service like antivirus etc. to declare an upload as failed.

- If you want to restart all uploads just rerun the command with the `--restart` flag
```bash
ocis storage-users uploads sessions --restart
ocis storage-users uploads sessions
```

- If you want to restart only one upload use the postprocessing restart command
- **All failed uploads**\
If you want to restart/resume all failed uploads, just rerun the command with the relevant flag. Note that this is the preferred command to handle failed processing steps:
```bash
ocis postprocessing restart -u <uploadID>
ocis storage-users uploads sessions --resume
```

Instead of starting one specific upload, a system admin can also restart all uploads that are currently in a specific step.
Examples:
```
ocis postprocessing restart # Restarts all uploads where postprocessing is finished, but upload is not finished
ocis postprocessing restart -s "finished" # Equivalent to the above
ocis postprocessing restart -s "virusscan" # Restart all uploads currently in virusscan step
```

Note: All above commands containing the word `restart` can also be replaced by `resume`. This changes behaviour slightly. When `restarting` an upload, the whole postprocessing will be (re)done. If `resuming` an upload, it will only continue from the last step that was completed.
- **Particular failed uploads**\
Use the `postprocessing` command to resume defined failed uploads. For postprocessing steps, the default is to resume . Note that at the moment, `resume` is an alias for `restart` to keep old functionality. `restart` is subject of change and will most likely be removed in a later version.

- **Defined by ID**\
If you want to resume only a specific upload, use the postprocessing resume command with the ID selected:
```bash
ocis postprocessing resume -u <uploadID>
```

- **Defined by step**\
Alternatively, instead of restarting one specific upload, a system admin can also resume all uploads that are currently in a specific step.\
Examples:\
```bash
ocis postprocessing resume # Resumes all uploads where postprocessing is finished, but upload is not finished
ocis postprocessing resume -s "finished" # Equivalent to the above
ocis postprocessing resume -s "virusscan" # Resume all uploads currently in virusscan step
```
30 changes: 14 additions & 16 deletions services/storage-users/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,14 +56,20 @@ ocis storage-users uploads <command>
```plaintext
COMMANDS:
sessions Print a list of upload sessions
clean Clean up leftovers from expired uploads (deprecated)
list Print a list of all incomplete uploads (deprecated)
```

#### Sessions command

The `sessions` command is the entry point for listing, restarting and cleaning unfinished uploads.

**IMPORTANT**
> If not noted otherwise, commands with the `restart` option can also use the `resume` option. This changes behaviour slightly.
>
> * `restart`\
> When restarting an upload, all steps for open items will be restarted, except if otherwise defined.
> * `resume`\
> When resuming an upload, processing will continue unfinished items from their last completed step.

```bash
ocis storage-users uploads sessions <commandoptions>
```
Expand All @@ -82,6 +88,7 @@ OPTIONS:
--has-virus filter sessions by virus scan result (default: unset)
--json output as json (default: false)
--restart send restart event for all listed sessions (default: false)
--resume send resume event for all listed sessions (default: false)
--clean remove uploads (default: false)
--help, -h show help
```
Expand Down Expand Up @@ -122,24 +129,15 @@ ocis storage-users uploads sessions --expired=false --processing=false --json
{"id":"f066244d-97b2-48e7-a30d-b40fcb60cec6","space":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c","filename":"bar.txt","offset":0,"size":4321,"executant":{"idp":"https://cloud.ocis.test","opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"spaceowner":{"opaque_id":"f7fbf8c8-139b-4376-b307-cf0a8c2d0d9c"},"expires":"2024-01-26T13:18:47+01:00","processing":false, "scanDate": "2024-04-24T14:38:29+02:00", "scanResult": ""}
```

The sessions command can also clear and restart uploads. The output is the same as if run without `--clean` or `--restart` flag.
Note: It is recommended to run to command first without the `--clean` (`--processing`) flag to double check which uploads get cleaned (restarted).
The sessions command can also clear and restart/resume uploads. The output is the same as if run without `--clean` or `--restart` flag.
Note: It is recommended to run to command first without the `--clean` (`--processing`) flag to double check which uploads get cleaned (restarted/resumed).

```bash
# cleans all expired uploads regardless of processing and virus state.
ocis storage-users uploads sessions --expired=true --clean

# restarts all uploads that are processing and are not virus infected
ocis storage-users uploads sessions --processing=false --has-virus=false --restart
```

IMPORTANT: `list` and `clean` commands are deprecated. Do not use them.

```bash
# deprecated
ocis storage-users uploads list

# deprecated
ocis storage-users uploads clean
# resumes all uploads that are processing and are not virus infected
ocis storage-users uploads sessions --processing=false --has-virus=false --resume
```

### Manage Trash-Bin Items
Expand Down