Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Perform Vault operations concurrently #78

Open
mattlqx opened this issue Feb 12, 2021 · 1 comment · May be fixed by #80
Open

Perform Vault operations concurrently #78

mattlqx opened this issue Feb 12, 2021 · 1 comment · May be fixed by #80
Assignees
@mattlqx
Labels
enhancement New feature or request

Comments

@mattlqx
Copy link
Collaborator

mattlqx commented Feb 12, 2021

Presently all Vault operations are serial, even for reads/writes that operate recursively on paths. These paths are determined by list api calls to find path targets, but after that a for loop iterates over those paths to get or put the data. We could utilize Go channels for this portion to run the read/write api calls simultaneously.
@mattlqx mattlqx added the enhancement New feature or request label Feb 12, 2021
@mattlqx mattlqx self-assigned this Feb 12, 2021
@fishi0x01
Copy link
Owner

You are right. Parallelizing the calls could save quite some time when recursively operating on a decent amount of secrets in a path.

I like your idea 👍

I can take a look at this if you want - but since you self-assigned this I assume you already gave it a shot? 😎

mattlqx pushed a commit that referenced this issue Feb 15, 2021
use concurrency for grep and replace commands
cf627b1 
theres a minor improvement in read/write times when performing a grep or replace across a large number of secrets in a path (by large, i mean around 90). the real killer on runtime though is the traversal to determine which paths are nodes vs. leaves. this is still early in my tinkering with it. this will be a draft until i see some decent gains in reducing runtime on my workloads. will fix #78.
@mattlqx mattlqx linked a pull request Feb 15, 2021 that will close this issue
Draft
mattlqx pushed a commit that referenced this issue Feb 15, 2021
use concurrency for grep and replace commands
eec03db 
theres a minor improvement in read/write times when performing a grep or replace across a large number of secrets in a path (by large, i mean around 90). the real killer on runtime though is the traversal to determine which paths are nodes vs. leaves. this is still early in my tinkering with it. this will be a draft until i see some decent gains in reducing runtime on my workloads. will fix #78.
@fishi0x01 fishi0x01 mentioned this issue Feb 21, 2021
Draft
mattlqx pushed a commit that referenced this issue Sep 10, 2021
use concurrency for grep and replace commands
0ccdd01 
theres a minor improvement in read/write times when performing a grep or replace across a large number of secrets in a path (by large, i mean around 90). the real killer on runtime though is the traversal to determine which paths are nodes vs. leaves. this is still early in my tinkering with it. this will be a draft until i see some decent gains in reducing runtime on my workloads. will fix #78.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mattlqx mattlqx

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

use concurrency for grep and replace commands fishi0x01/vsh
2 participants
@mattlqx @fishi0x01