Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: LVM as a pool backend #1587

Merged
merged 3 commits into from
Apr 18, 2024
Merged

Feat: LVM as a pool backend #1587

merged 3 commits into from
Apr 18, 2024

Conversation

tiagolobocastro
Copy link
Contributor

@tiagolobocastro tiagolobocastro commented Feb 14, 2024

Add LVM as an experimental alternative backend to Mayastor:

  1. it allows us to use a low latency single replica
    local volume (app pinned to the same node)
  2. robust backend for existing LVM users which may prefer it
  3. combine local engines into mayastor

This differs from the existing lvm localpv as rather than importing
the existing VGs we create the LVM PV's and the LVM VG itself.
On destruction, if the VG has no non-mayastor LVs then we destroy it,
otherwise we leave it behind, untagged.

The existing pool/replica services are also refactored to support
adding other backends in the future as well.
This is achieved by implementing the service interface separately
and binding them at a higher level. This way it avoids mixing
the backend code, at least as much as possible.

io-engine/src/lvm/error.rs Outdated Show resolved Hide resolved
io-engine/src/lvm/lv_replica.rs Outdated Show resolved Hide resolved
io-engine/src/lvm/vg_pool.rs Outdated Show resolved Hide resolved
io-engine/src/pool_backend.rs Outdated Show resolved Hide resolved
io-engine/src/bin/io-engine.rs Outdated Show resolved Hide resolved
io-engine/src/lvm/cli.rs Show resolved Hide resolved
io-engine/src/lvm/cli.rs Outdated Show resolved Hide resolved
io-engine/src/lvm/vg_pool.rs Show resolved Hide resolved
io-engine/src/lvm/vg_pool.rs Show resolved Hide resolved
io-engine/src/lvm/vg_pool.rs Show resolved Hide resolved
io-engine/src/lvm/lv_replica.rs Outdated Show resolved Hide resolved
io-engine/src/lvm/lv_replica.rs Show resolved Hide resolved
@tiagolobocastro tiagolobocastro force-pushed the lvm branch 2 times, most recently from 95ee994 to 30fc472 Compare February 23, 2024 12:11
io-engine/src/grpc/v1/lvm/replica.rs Outdated Show resolved Hide resolved
io-engine/src/lvm/vg_pool.rs Show resolved Hide resolved
io-engine/src/core/env.rs Show resolved Hide resolved
io-engine/src/grpc/v1/lvm/pool.rs Show resolved Hide resolved
io-engine/src/grpc/v1/pool.rs Show resolved Hide resolved
io-engine/src/lvm/lv_replica.rs Show resolved Hide resolved
io-engine/src/lvm/lv_replica.rs Outdated Show resolved Hide resolved
io-engine/src/lvm/vg_pool.rs Outdated Show resolved Hide resolved
@tiagolobocastro tiagolobocastro force-pushed the lvm branch 3 times, most recently from 735bad9 to 63d3d76 Compare February 27, 2024 01:33
io-engine/src/lvm/vg_pool.rs Show resolved Hide resolved
io-engine/src/lvm/vg_pool.rs Show resolved Hide resolved
io-engine/src/core/env.rs Show resolved Hide resolved
io-engine/src/bin/io-engine.rs Outdated Show resolved Hide resolved
io-engine/src/bin/io-engine.rs Outdated Show resolved Hide resolved
ci.nix Show resolved Hide resolved
io-engine/src/core/share.rs Outdated Show resolved Hide resolved
@dsavitskiy
Copy link
Contributor

This PR seems to be quite big and intrusive, and there are concerns about the implementation decisions, e.g. like communication with LVM with CLI tools, etc.
Shouldn't we go through our full feature design process as we do for other features? HLD, LLD, BDDs, and so on.

@tiagolobocastro tiagolobocastro force-pushed the lvm branch 3 times, most recently from 64912ef to 702afdf Compare April 15, 2024 19:05
io-engine/src/lvm/property.rs Show resolved Hide resolved
@tiagolobocastro
Copy link
Contributor Author

bors merge

bors-openebs-mayastor bot pushed a commit that referenced this pull request Apr 18, 2024
1587: Feat: LVM as a pool backend r=tiagolobocastro a=tiagolobocastro

Add LVM as an experimental alternative backend to Mayastor:
1. it allows us to use a low latency single replica
local volume (app pinned to the same node)
2. robust backend for existing LVM users which may prefer it
3. combine local engines into mayastor

This differs from the existing lvm localpv as rather than importing
the existing VGs we create the LVM PV's and the LVM VG itself.
On destruction, if the VG has no non-mayastor LVs then we destroy it,
otherwise we leave it behind, untagged.

The existing pool/replica services are also refactored to support
adding other backends in the future as well.
This is achieved by implementing the service interface separately
and binding them at a higher level. This way it avoids mixing
the backend code, at least as much as possible.

Co-authored-by: Tiago Castro <[email protected]>
@bors-openebs-mayastor
Copy link

Build failed:

@tiagolobocastro
Copy link
Contributor Author

bors merge

@bors-openebs-mayastor
Copy link

🕐 Waiting for PR status (Github check) to be set, probably by CI. Bors will automatically try to run when all required PR statuses are set.

bors-openebs-mayastor bot pushed a commit that referenced this pull request Apr 18, 2024
1587: Feat: LVM as a pool backend r=tiagolobocastro a=tiagolobocastro

Add LVM as an experimental alternative backend to Mayastor:
1. it allows us to use a low latency single replica
local volume (app pinned to the same node)
2. robust backend for existing LVM users which may prefer it
3. combine local engines into mayastor

This differs from the existing lvm localpv as rather than importing
the existing VGs we create the LVM PV's and the LVM VG itself.
On destruction, if the VG has no non-mayastor LVs then we destroy it,
otherwise we leave it behind, untagged.

The existing pool/replica services are also refactored to support
adding other backends in the future as well.
This is achieved by implementing the service interface separately
and binding them at a higher level. This way it avoids mixing
the backend code, at least as much as possible.

Co-authored-by: Tiago Castro <[email protected]>
@tiagolobocastro
Copy link
Contributor Author

bors cancel

@bors-openebs-mayastor
Copy link

Canceled.

@tiagolobocastro
Copy link
Contributor Author

bors merge

bors-openebs-mayastor bot pushed a commit that referenced this pull request Apr 18, 2024
1587: Feat: LVM as a pool backend r=tiagolobocastro a=tiagolobocastro

Add LVM as an experimental alternative backend to Mayastor:
1. it allows us to use a low latency single replica
local volume (app pinned to the same node)
2. robust backend for existing LVM users which may prefer it
3. combine local engines into mayastor

This differs from the existing lvm localpv as rather than importing
the existing VGs we create the LVM PV's and the LVM VG itself.
On destruction, if the VG has no non-mayastor LVs then we destroy it,
otherwise we leave it behind, untagged.

The existing pool/replica services are also refactored to support
adding other backends in the future as well.
This is achieved by implementing the service interface separately
and binding them at a higher level. This way it avoids mixing
the backend code, at least as much as possible.

Co-authored-by: Tiago Castro <[email protected]>
@bors-openebs-mayastor
Copy link

Build failed:

@tiagolobocastro
Copy link
Contributor Author

bors merge

bors-openebs-mayastor bot pushed a commit that referenced this pull request Apr 18, 2024
1587: Feat: LVM as a pool backend r=tiagolobocastro a=tiagolobocastro

Add LVM as an experimental alternative backend to Mayastor:
1. it allows us to use a low latency single replica
local volume (app pinned to the same node)
2. robust backend for existing LVM users which may prefer it
3. combine local engines into mayastor

This differs from the existing lvm localpv as rather than importing
the existing VGs we create the LVM PV's and the LVM VG itself.
On destruction, if the VG has no non-mayastor LVs then we destroy it,
otherwise we leave it behind, untagged.

The existing pool/replica services are also refactored to support
adding other backends in the future as well.
This is achieved by implementing the service interface separately
and binding them at a higher level. This way it avoids mixing
the backend code, at least as much as possible.

Co-authored-by: Tiago Castro <[email protected]>
@bors-openebs-mayastor
Copy link

Build failed:

tiagolobocastro and others added 3 commits April 18, 2024 22:01
Add LVM as an experimental alternative backend to Mayastor:
1. it allows us to use a low latency single replica
local volume (app pinned to the same node)
2. robust backend for existing LVM users which may prefer it
3. combine local engines into mayastor

This differs from the existing lvm localpv as rather than importing
the existing VGs we create the LVM PV's and the LVM VG itself.
On destruction, if the VG has no non-mayastor LVs then we destroy it,
otherwise we leave it behind, untagged.

The existing pool/replica services are also refactored to support
adding other backends in the future as well.
This is achieved by implementing the service interface separately
and binding them at a higher level. This way it avoids mixing
the backend code, at least as much as possible.

Co-authored-by: shubham <[email protected]>
Co-authored-by: Akhil Mohan <[email protected]>
Co-authored-by: Tiago Castro <[email protected]>
Signed-off-by: Tiago Castro <[email protected]>
Supports using CARGO_VENDOR_DIR=/tmp/bob/.. in case bob doesn't exist.
Probably won't ever happen but just in case :)

Signed-off-by: Tiago Castro <[email protected]>
@tiagolobocastro
Copy link
Contributor Author

bors merge

@bors-openebs-mayastor
Copy link

Build succeeded:

@bors-openebs-mayastor bors-openebs-mayastor bot merged commit 3f0aa9c into develop Apr 18, 2024
4 checks passed
@bors-openebs-mayastor bors-openebs-mayastor bot deleted the lvm branch April 18, 2024 22:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants