Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Storage: Deduplicate logic between PureStorage and PowerFlex storage drivers #14700

Draft
wants to merge 60 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
91d9b74
lxd/storage/drivers/pure: Initial scaffolding for storage driver pure
MusicDin Sep 27, 2024
335c60c
lxd/storage/drivers/driver_types: Add driver option PopulateParentVol…
MusicDin Dec 4, 2024
8f322f0
lxd/storage/backend_lxd: Ensure parent volume UUID is populated when …
MusicDin Dec 4, 2024
32e043d
lxd/storage/drivers: Configure PopulateParentVolumeUUID for all drivers
MusicDin Dec 6, 2024
a8ba009
lxd/storage/drivers/load: Add pure as storage driver
MusicDin Nov 20, 2024
635fdd5
lxd/storage/drivers/pure: Add basic pool and volume configuration val…
MusicDin Sep 30, 2024
1e9b998
lxd/storage/drivers/pure: Add login and request wrapper
MusicDin Sep 30, 2024
ebdeb1e
lxd/storage/drivers/pure: Create and delete storage pool
MusicDin Nov 20, 2024
39143b6
lxd/storage/drivers/pure: Add utils for handling PureStorage hosts
MusicDin Nov 20, 2024
c231262
lxd/storage/drivers/pure: Add util function to get server name
MusicDin Nov 20, 2024
8f5b3b8
lxd/storage/drivers/pure: Function to resolve pure storage volume name
MusicDin Nov 20, 2024
86ab5af
lxd/storage/drivers/pure: Test volume name generation
MusicDin Dec 11, 2024
efc6160
lxd/storage/drivers/pure: Add iSCSI support and volume mapping utils
MusicDin Nov 20, 2024
c8b8ea6
lxd/storage/drivers/pure: Create, mount, and unmount volume
MusicDin Oct 3, 2024
5eb9c42
lxd/storage/drivers/pure: Add NVMe/TCP support
MusicDin Dec 5, 2024
1693ad9
lxd/storage/drivers/pure: Delete volume
MusicDin Nov 20, 2024
72bc46f
lxd/storage/drivers/pure: Create and delete volume snapshots
MusicDin Oct 9, 2024
75dee76
lxd/storage/drivers/pure: Add utils for retrieving storage arrays
MusicDin Nov 20, 2024
d720c42
lxd/storage/drivers/pure: Extract storage pool, volume, and array spa…
MusicDin Nov 20, 2024
d3b2ea7
lxd/storage/drivers/pure: Report resource usage of storage pools and …
MusicDin Oct 10, 2024
e9a4837
lxd/storage/drivers/pure: Restore volume snapshots
MusicDin Oct 10, 2024
57c0848
lxd/storage/drivers/pure: Get volume usage from PureStorage
MusicDin Nov 20, 2024
76896ce
lxd/storage/drivers/pure: Set or update volume quota
MusicDin Oct 10, 2024
ba6441f
lxd/storage/drivers/pure: Allow volume copy with snapshots and optimi…
MusicDin Oct 15, 2024
43d77f5
lxd/storage/drivers/pure: Volume refresh
MusicDin Oct 16, 2024
aedd635
lxd/storage/drivers/pure: Mount/unmount volume snapshot
MusicDin Oct 18, 2024
6204f0d
lxd/storage/drivers/pure: Volume migration
MusicDin Oct 22, 2024
4ff7c6d
lxd/storage/drivers/pure: Handle cluster member volume move
MusicDin Nov 22, 2024
035fde2
lxd/storage/drivers/pure: Allow changing storage pool quota
MusicDin Nov 20, 2024
0036432
lxd/storage/backend: Pre-allocate recovered keys slice (linter)
MusicDin Dec 19, 2024
d1ff520
lxd/storage/utils: Add PureStorage to common volume rules
MusicDin Dec 6, 2024
c7ad077
test/backends: Helper functions for creating Pure Storage pools
MusicDin Nov 12, 2024
fb6af53
test/includes/storage: Include Pure Storage driver if gateway and api…
MusicDin Dec 6, 2024
8abc126
test/storage_driver_pure: Add basic Pure Storage tests
MusicDin Nov 8, 2024
3e0cf83
test/container_move: Use helper function to create Pure Storage pool
MusicDin Nov 12, 2024
82c9fcd
test/storage_local_volume_handling: Test Pure Storage with other avai…
MusicDin Nov 14, 2024
81bf9cb
test/storage_snapshots: Use helper function to create Pure Storage pool
MusicDin Nov 14, 2024
52409e7
test/backup: Skip recovery tests for Pure Storage driver
MusicDin Nov 15, 2024
aea78f1
test/stresstest: Remove duplicate definition of BASE_URL
MusicDin Nov 15, 2024
01cde7e
docs: Add Pure Storage driver docs
MusicDin Oct 22, 2024
b8f0bf5
docs: Add example on how to create Pure Storage storage pool
MusicDin Oct 22, 2024
85b0521
docs: Add explanation of Pure Storage remote storage
MusicDin Oct 22, 2024
356dcc6
docs: Add Pure Storage and its features to table of supported storage…
MusicDin Oct 22, 2024
d2da452
docs: Update wordlist
MusicDin Dec 6, 2024
9d8fbac
docs: Update metadata
MusicDin Dec 13, 2024
703e8e5
lxd/storage/connectors: Introduce storage connector
MusicDin Dec 18, 2024
9546b75
lxd/storage/connectors: Add iSCSI connector
MusicDin Dec 18, 2024
1ffdef0
lxd/storage/connectors: Add NVMe/TCP connector
MusicDin Dec 18, 2024
16c32b2
lxd/storage/connectors: Add SDC connector
MusicDin Dec 19, 2024
09b9a52
lxd/storage/connectors: Helper functions for handling disk device paths
MusicDin Dec 16, 2024
72b583e
lxd/storage/connectors/connector: Get versions for supported connectors
MusicDin Dec 19, 2024
c826ee8
lxd/storage/drivers/utils: Helper function to resolve server name
MusicDin Dec 18, 2024
5d67ed3
lxd/storage/drivers/pure: Use driver name for global lock
MusicDin Dec 19, 2024
aebde1a
lxd/storage/drivers/pure: Prevent changing pure.mode
MusicDin Dec 19, 2024
b5c46db
lxd/storage/drivers/pure: Use storage connector
MusicDin Dec 18, 2024
cc4b6fb
lxd/storage/drivers/powerflex: Pre-allocate slice (linter)
MusicDin Dec 19, 2024
c679ef7
lxd/storage/drivers/powerflex: Use driver name for global lock
MusicDin Dec 19, 2024
119e49a
lxd/storage/drivers/powerflex: Prevent changing powerflex.mode
MusicDin Dec 19, 2024
b2cd85c
lxd/storage/drivers/powerflex: Use shared function to resolve server …
MusicDin Dec 19, 2024
4501af1
lxd/storage/drivers/powerflex: Use connector for handling storage sub…
MusicDin Dec 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions doc/.wordlist.txt
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,15 @@ EBS
EKS
enablement
favicon
FlashArray
Furo
GDB
Git
GitHub
Grafana
IAM
installable
iSCSI
JSON
Juju
Kubeflow
Expand Down
15 changes: 8 additions & 7 deletions doc/explanation/storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ The following storage drivers are supported:
- [CephFS - `cephfs`](storage-cephfs)
- [Ceph Object - `cephobject`](storage-cephobject)
- [Dell PowerFlex - `powerflex`](storage-powerflex)
- [Pure Storage - `pure`](storage-pure)

See the following how-to guides for additional information:

Expand All @@ -36,12 +37,12 @@ See the following how-to guides for additional information:
Where the LXD data is stored depends on the configuration and the selected storage driver.
Depending on the storage driver that is used, LXD can either share the file system with its host or keep its data separate.

Storage location | Directory | Btrfs | LVM | ZFS | Ceph (all) | Dell PowerFlex |
:--- | :-: | :-: | :-: | :-: | :-: | :-: |
Shared with the host | ✓ | ✓ | - | ✓ | - | - |
Dedicated disk/partition | - | ✓ | ✓ | ✓ | - | - |
Loop disk | - | ✓ | ✓ | ✓ | - | - |
Remote storage | - | - | - | - | ✓ | ✓ |
Storage location | Directory | Btrfs | LVM | ZFS | Ceph (all) | Dell PowerFlex | Pure Storage |
:--- | :-: | :-: | :-: | :-: | :-: | :-: | :-: |
Shared with the host | ✓ | ✓ | - | ✓ | - | - | - |
Dedicated disk/partition | - | ✓ | ✓ | ✓ | - | - | - |
Loop disk | - | ✓ | ✓ | ✓ | - | - | - |
Remote storage | - | - | - | - | ✓ | ✓ | ✓ |

#### Shared with the host

Expand Down Expand Up @@ -71,7 +72,7 @@ You can increase their size (quota) though; see {ref}`storage-resize-pool`.
#### Remote storage

The `ceph`, `cephfs` and `cephobject` drivers store the data in a completely independent Ceph storage cluster that must be set up separately.
The same applies to the `powerflex` driver.
The same applies to the `powerflex` and `pure` drivers.

(storage-default-pool)=
### Default storage pool
Expand Down
27 changes: 27 additions & 0 deletions doc/howto/storage_pools.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,20 @@ Create a storage pool named `pool5` that explicitly uses the PowerFlex SDC:

lxc storage create pool5 powerflex powerflex.mode=sdc powerflex.pool=<id of sp1> powerflex.gateway=https://powerflex powerflex.user.name=lxd powerflex.user.password=foo

#### Create a Pure Storage pool

Create a storage pool named `pool1` that uses NVMe/TCP by default:

lxc storage create pool1 pure pure.gateway=https://<pure-storage-address> pure.api.token=<pure-storage-api-token>

Create a storage pool named `pool2` that uses a Pure Storage gateway with a certificate that is not trusted:

lxc storage create pool2 pure pure.gateway=https://<pure-storage-address> pure.gateway.verify=false pure.api.token=<pure-storage-api-token>

Create a storage pool named `pool3` that uses iSCSI to connect to Pure Storage array:

lxc storage create pool3 pure pure.gateway=https://<pure-storage-address> pure.api.token=<pure-storage-api-token> pure.mode=iscsi

(storage-pools-cluster)=
## Create a storage pool in a cluster

Expand Down Expand Up @@ -240,6 +254,19 @@ Storage pool my-remote-pool2 pending on member vm03
Storage pool my-remote-pool2 created
```

Create a third storage pool named `my-remote-pool3` using the Pure Storage driver:

```{terminal}
:input: lxc storage create my-remote-pool3 pure --target=vm01
Storage pool my-remote-pool3 pending on member vm01
:input: lxc storage create my-remote-pool3 pure --target=vm02
Storage pool my-remote-pool3 pending on member vm02
:input: lxc storage create my-remote-pool3 pure --target=vm03
Storage pool my-remote-pool3 pending on member vm03
:input: lxc storage create my-remote-pool3 pure pure.gateway=https://<pure-storage-address> pure.api.token=<pure-storage-api-token>
Storage pool my-remote-pool3 created
```

## Configure storage pool settings

See the {ref}`storage-drivers` documentation for the available configuration options for each storage driver.
Expand Down
108 changes: 108 additions & 0 deletions doc/metadata.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5915,6 +5915,114 @@ Specify either a cron expression (`<minute> <hour> <dom> <month> <dow>`), a comm
```

<!-- config group storage-powerflex-volume-conf end -->
<!-- config group storage-pure-pool-conf start -->
```{config:option} pure.api.token storage-pure-pool-conf
:shortdesc: "API token for Pure Storage gateway authentication"
:type: "string"

```

```{config:option} pure.gateway storage-pure-pool-conf
:shortdesc: "Address of the Pure Storage gateway"
:type: "string"

```

```{config:option} pure.gateway.verify storage-pure-pool-conf
:defaultdesc: "`true`"
:shortdesc: "Whether to verify the Pure Storage gateway's certificate"
:type: "bool"

```

```{config:option} pure.mode storage-pure-pool-conf
:defaultdesc: "the discovered mode"
:shortdesc: "How volumes are mapped to the local server"
:type: "string"
The mode to use to map Pure Storage volumes to the local server.
Supported values are `iscsi` and `nvme`.
```

```{config:option} volume.size storage-pure-pool-conf
:defaultdesc: "`10GiB`"
:shortdesc: "Size/quota of the storage volume"
:type: "string"
Default Pure Storage volume size rounded to 512B. The minimum size is 1MiB.
```

<!-- config group storage-pure-pool-conf end -->
<!-- config group storage-pure-volume-conf start -->
```{config:option} block.filesystem storage-pure-volume-conf
:condition: "block-based volume with content type `filesystem`"
:defaultdesc: "same as `volume.block.filesystem`"
:shortdesc: "File system of the storage volume"
:type: "string"
Valid options are: `btrfs`, `ext4`, `xfs`
If not set, `ext4` is assumed.
```

```{config:option} block.mount_options storage-pure-volume-conf
:condition: "block-based volume with content type `filesystem`"
:defaultdesc: "same as `volume.block.mount_options`"
:shortdesc: "Mount options for block-backed file system volumes"
:type: "string"

```

```{config:option} size storage-pure-volume-conf
:defaultdesc: "same as `volume.size`"
:shortdesc: "Size/quota of the storage volume"
:type: "string"
Default Pure Storage volume size rounded to 512B. The minimum size is 1MiB.
```

```{config:option} snapshots.expiry storage-pure-volume-conf
:condition: "custom volume"
:defaultdesc: "same as `volume.snapshots.expiry`"
:scope: "global"
:shortdesc: "When snapshots are to be deleted"
:type: "string"
Specify an expression like `1M 2H 3d 4w 5m 6y`.
```

```{config:option} snapshots.pattern storage-pure-volume-conf
:condition: "custom volume"
:defaultdesc: "same as `volume.snapshots.pattern` or `snap%d`"
:scope: "global"
:shortdesc: "Template for the snapshot name"
:type: "string"
You can specify a naming template that is used for scheduled snapshots and unnamed snapshots.

The `snapshots.pattern` option takes a Pongo2 template string to format the snapshot name.

To add a time stamp to the snapshot name, use the Pongo2 context variable `creation_date`.
Make sure to format the date in your template string to avoid forbidden characters in the snapshot name.
For example, set `snapshots.pattern` to `{{ creation_date|date:'2006-01-02_15-04-05' }}` to name the snapshots after their time of creation, down to the precision of a second.

Another way to avoid name collisions is to use the placeholder `%d` in the pattern.
For the first snapshot, the placeholder is replaced with `0`.
For subsequent snapshots, the existing snapshot names are taken into account to find the highest number at the placeholder's position.
This number is then incremented by one for the new name.
```

```{config:option} snapshots.schedule storage-pure-volume-conf
:condition: "custom volume"
:defaultdesc: "same as `snapshots.schedule`"
:scope: "global"
:shortdesc: "Schedule for automatic volume snapshots"
:type: "string"
Specify either a cron expression (`<minute> <hour> <dom> <month> <dow>`), a comma-separated list of schedule aliases (`@hourly`, `@daily`, `@midnight`, `@weekly`, `@monthly`, `@annually`, `@yearly`), or leave empty to disable automatic snapshots (the default).
```

```{config:option} volatile.uuid storage-pure-volume-conf
:defaultdesc: "random UUID"
:scope: "global"
:shortdesc: "The volume's UUID"
:type: "string"

```

<!-- config group storage-pure-volume-conf end -->
<!-- config group storage-zfs-bucket-conf start -->
```{config:option} size storage-zfs-bucket-conf
:condition: "appropriate driver"
Expand Down
35 changes: 18 additions & 17 deletions doc/reference/storage_drivers.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ storage_cephfs
storage_cephobject
storage_ceph
storage_powerflex
storage_pure
storage_dir
storage_lvm
storage_zfs
Expand All @@ -27,23 +28,23 @@ See the corresponding pages for driver-specific information and configuration op

Where possible, LXD uses the advanced features of each storage system to optimize operations.

Feature | Directory | Btrfs | LVM | ZFS | Ceph RBD | CephFS | Ceph Object | Dell PowerFlex
:--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :---
{ref}`storage-optimized-image-storage` | ❌ | ✅ | ✅ | ✅ | ✅ | ➖ | ➖ | ❌
Optimized instance creation | ❌ | ✅ | ✅ | ✅ | ✅ | ➖ | ➖ | ❌
Optimized snapshot creation | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ➖ | ✅
Optimized image transfer | ❌ | ✅ | ❌ | ✅ | ✅ | ➖ | ➖ | ❌
Optimized backup (import/export) | ❌ | ✅ | ❌ | ✅ | ❌ | ➖ | ➖ | ❌
{ref}`storage-optimized-volume-transfer` | ❌ | ✅ | ❌ | ✅ | ✅[^1] | ➖ | ➖ | ❌
{ref}`storage-optimized-volume-refresh` | ❌ | ✅ | ✅[^2] | ✅ | ✅[^3] | ➖ | ➖ | ❌
Copy on write | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ➖ | ✅
Block based | ❌ | ❌ | ✅ | ❌ | ✅ | ❌ | ➖ | ✅
Instant cloning | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ➖ | ❌
Storage driver usable inside a container | ✅ | ✅ | ❌ | ✅[^4] | ❌ | ➖ | ➖ | ❌
Restore from older snapshots (not latest) | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ➖ | ✅
Storage quotas | ✅[^5] | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅
Available on `lxd init` | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌
Object storage | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ❌
Feature | Directory | Btrfs | LVM | ZFS | Ceph RBD | CephFS | Ceph Object | Dell PowerFlex | Pure Storage
:--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :---
{ref}`storage-optimized-image-storage` | ❌ | ✅ | ✅ | ✅ | ✅ | ➖ | ➖ | ❌ | ✅
Optimized instance creation | ❌ | ✅ | ✅ | ✅ | ✅ | ➖ | ➖ | ❌ | ✅
Optimized snapshot creation | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ➖ | ✅ | ✅
Optimized image transfer | ❌ | ✅ | ❌ | ✅ | ✅ | ➖ | ➖ | ❌ | ✅
Optimized backup (import/export) | ❌ | ✅ | ❌ | ✅ | ❌ | ➖ | ➖ | ❌ | ❌
{ref}`storage-optimized-volume-transfer` | ❌ | ✅ | ❌ | ✅ | ✅[^1] | ➖ | ➖ | ❌ | ❌
{ref}`storage-optimized-volume-refresh` | ❌ | ✅ | ✅[^2] | ✅ | ✅[^3] | ➖ | ➖ | ❌ | ❌
Copy on write | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ➖ | ✅ | ✅
Block based | ❌ | ❌ | ✅ | ❌ | ✅ | ❌ | ➖ | ✅ | ✅
Instant cloning | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ➖ | ❌ | ✅
Storage driver usable inside a container | ✅ | ✅ | ❌ | ✅[^4] | ❌ | ➖ | ➖ | ❌ | ❌
Restore from older snapshots (not latest) | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ➖ | ✅ | ✅
Storage quotas | ✅[^5] | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅
Available on `lxd init` | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌
Object storage | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌

[^1]: Volumes of type `block` will fall back to non-optimized transfer when migrating to an older LXD server that doesn't yet support the `RBD_AND_RSYNC` migration type.
[^2]: Requires {config:option}`storage-lvm-pool-conf:lvm.use_thinpool` to be enabled. Only when refreshing local volumes.
Expand Down
Loading
Loading