[SM-394] Secrets Manager #2164
Merged
[SM-394] Secrets Manager #2164
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Objective The purpose of this PR is to create a database table, entity, and repository for the new Secret database table. The new Secret table will use entity framework for all database providers.
Add a controller to fetch secrets associated with an organization ID. To note, the [SecretsManager] attribute makes this controller only available for local development.
…2201) The purpose of this PR is to add API endpoints for getting, creating, and editing secrets for the Secrets Manager project.
…n date (#2206) * Read UTC DateTimes from db and order by revision * Move orderby to repo layer
…feature # Conflicts: # bitwarden-server.sln # bitwarden_license/src/Scim/packages.lock.json # bitwarden_license/src/Sso/packages.lock.json # bitwarden_license/test/Commercial.Core.Test/packages.lock.json # src/Admin/packages.lock.json # src/Api/packages.lock.json # src/Billing/packages.lock.json # src/Events/packages.lock.json # src/EventsProcessor/packages.lock.json # src/Icons/packages.lock.json # src/Identity/packages.lock.json # src/Infrastructure.EntityFramework/packages.lock.json # src/Notifications/packages.lock.json # src/SharedWeb/packages.lock.json # test/Api.Test/packages.lock.json # test/Billing.Test/packages.lock.json # test/Common/packages.lock.json # test/Core.Test/packages.lock.json # test/Icons.Test/packages.lock.json # test/Identity.IntegrationTest/packages.lock.json # test/Identity.Test/packages.lock.json # test/IntegrationTestCommon/packages.lock.json # util/MySqlMigrations/packages.lock.json # util/PostgresMigrations/packages.lock.json
* Project DB addition and sprocs * Adding spaces to the end of each file, fixing minor issues * removing useless comments * Adding soft delete proc to migration * Project EF Scaffold * Additional changes to use EF instead of procedures * Adding dependency injection * Fixing lint errors * Bug fixes * Adding migration scripts, removing sproc files, and setting up Entity framework code * Adding back accidentally deleted sproc * Removing files that shouldn't have been created * Lint * Small changes based on Oscar's rec (#2215) * Migrations for making CreateDate not null * adding space to end of file * Making Revision date not null * dotnet format * Adding nonclustered indexes to SQL * SM-104: Update PR with changes Thomas proposed Co-authored-by: CarleyDiaz-Bitwarden <[email protected]> Co-authored-by: Thomas Avery <[email protected]> Co-authored-by: Colton Hurst <[email protected]>
* SM-114: Initial commit with create project endpoint (for SM) * SM-114: Add Update Project route (for SM) * SM-114: Fix file encodings * Fix DI issue for SM Project Create/Update commands * Fix import ordering for linter * SM-114: Remove unneeded lines setting DeletedDate, as it should already be null * SM-114: Only have OrgId in route for CreateProject * Remove unneeded using * SM-114: Initial commit with create project endpoint (for SM) * SM-114: Add Update Project route (for SM) * SM-114: Fix file encodings * Fix DI issue for SM Project Create/Update commands * Fix import ordering for linter * SM-114: Remove unneeded lines setting DeletedDate, as it should already be null * SM-114: Only have OrgId in route for CreateProject * Remove unneeded using * Fully remove OrgId from ProjectCreateRequestModel
* Bulk delete secrets with command unit tests * Controller unit tests * Optimize conditionals * SM-64 bulk delete integration test * fix test * SM-64 code review updated
* Secrets return empty list
* Adding integration tests for the SecretsController Co-authored-by: Hinton <[email protected]>
* SM-114: Initial commit with create project endpoint (for SM) * SM-114: Add Update Project route (for SM) * SM-114: Fix file encodings * Fix DI issue for SM Project Create/Update commands * Adding GetProjectsByOrg * fixing merge conflicts * fix * Updating to return empty list * removing null check Co-authored-by: Colton Hurst <[email protected]> Co-authored-by: CarleyDiaz-Bitwarden <[email protected]>
* SM-191 Create ServiceAccount Table
…feature # Conflicts: # bitwarden_license/src/Commercial.Core/Utilities/ServiceCollectionExtensions.cs # src/Api/Startup.cs # src/Api/Utilities/SecretsManagerAttribute.cs # src/Infrastructure.EntityFramework/EntityFrameworkServiceCollectionExtensions.cs # src/Infrastructure.EntityFramework/Repositories/DatabaseContext.cs # src/SharedWeb/Utilities/ServiceCollectionExtensions.cs # test/Api.IntegrationTest/Factories/ApiApplicationFactory.cs # test/IntegrationTestCommon/Factories/WebApplicationFactoryBase.cs # util/PostgresMigrations/Factories.cs
* SM-207 list service accounts by org
* SM-96: Small change to allow getting project by id * Fix whitespace issue * Add first integration test and fix date bug * Ensure tests are consistent * Add more project controller integration tests * Remove commented delete for now
* Beggining of changes for Project Secrets mapping * Beggining of changes for project and secrets mapping * Inital changes to add Mapping table for Project Secrets * Resolve migration not working properly * Indent sql * Changes to try and return projects in the GetManyByOrganizaationIDAsync on SecretRepository. * Changes made with Oscar * Add reversemap * running lint and removing comments * Lint fixes * fixing merge issues * Trying to fix the DB issue * DB fixes * fixes * removing unused space * fixing lint issue * final lint fix I hope * removing manually added sql.sqlproj * Lint changes and fixing the sql proj issues * adding ServiceAccount to sql proj * Removing ON DELETE CASCADE * remove On delete cascade * changes for deleting project and secret inside of the Organization_DeleteById procedure. * changes for deleting project and secret inside of the Organization_DeleteById procedure. * migration changes * Updating constraints * removing void * remove spaces * updating cipherRepo tests to be task instead of void * fixing * fixing * test * fix * fix * changes to remove circular dependency * fixes * sending guid and string name of the project over * Update src/Sql/dbo/Tables/Secret.sql Co-authored-by: Oscar Hinton <[email protected]> * Update src/Sql/dbo/Tables/Project.sql Co-authored-by: Oscar Hinton <[email protected]> * removing unused code * Potential refactor (#2340) * migrations * Postgres migraiton * Update src/Api/SecretManagerFeatures/Models/Response/SecretResponseModel.cs Co-authored-by: Oscar Hinton <[email protected]> * rename file * Update util/Migrator/DbScripts/2022-09-19_00_ProjectSecret.sql Co-authored-by: Oscar Hinton <[email protected]> * Lint fixes * removing extra semi colon * removing circular references with projects and secrets * adding back projects * Add ProjectFixture * Update util/Migrator/DbScripts/2022-09-19_00_ProjectSecret.sql Co-authored-by: Oscar Hinton <[email protected]> * Update util/Migrator/DbScripts/2022-09-19_00_ProjectSecret.sql Co-authored-by: Oscar Hinton <[email protected]> Co-authored-by: CarleyDiaz-Bitwarden <[email protected]> Co-authored-by: Hinton <[email protected]>
* Add create access token for service accounts
* Fix namespace for ServiceAccount command tests * Remove "this" from SecretsManager requests * Fix have scope be assigned a JSON list
Co-authored-by: Oscar Hinton <[email protected]>
* Update database to support never expiring ApiKey * Update Api to support never expiring ApiKeys * Fix unit test variable naming * Remove required from model * Fix spacing * Add EF migrations * Run dotnet format * Update util/Migrator/DbScripts/2022-11-29_00_ApiKey_Never_Expire.sql Co-authored-by: Oscar Hinton <[email protected]> Co-authored-by: Oscar Hinton <[email protected]>
* Fix and re-enable parallel integration tests * Fix package lock files * Move fix to ApiApplicationFactory * Run dotnet restore --force * Run dotnet format * Reset packages.lock.json files
* Add GetSecretsByProjectAsync endpoint * Add GetManyByProjectIdAsync endpoint * Update response model for GetSecretsByProjectAsync * Include projects when returning secrets by project id * SM-99: Add ability to specify projectId when creating a secret * SM-99: Update tests to accomodate for new create secret parameter * Fix failing test * SM-99: Handle optional projectId for new secret in ToSecret() * SM-99: Filter out deleted secrets on GetManyByProjectIdAsync() and small refactorings * SM-99: make CreateAsync for secret more clear * Add CreateSecretWithProject integration test * Fix CreateSecretWithProject integration test for SM-99 * Run dotnet format * Undo added space * Refactor test * Refactor CreateSecretWithProject API Integration test again
* testing * test2 * testing * trying to save the projects associated with the secret * changes * more changes * Fix EF error * Second attempt * Replace AddIfNotExists with Add. * changes * fixing await issue * lint * lint fixes * suggested changes * suggested changes * updating tests * fixing tests 2 * fixing tests * fixing test * fixing test * fixing tests * test * testing * fixing tests for the millionth time * fixing tests * allowing nulls for projectIds, fixing lint * fixing tests Co-authored-by: Hinton <[email protected]>
…feature # Conflicts: # bitwarden_license/src/Scim/packages.lock.json # bitwarden_license/src/Sso/packages.lock.json # bitwarden_license/test/Scim.IntegrationTest/packages.lock.json # src/Admin/Views/Organizations/Edit.cshtml # src/Admin/packages.lock.json # src/Api/packages.lock.json # src/Billing/packages.lock.json # src/Events/packages.lock.json # src/EventsProcessor/packages.lock.json # src/Icons/packages.lock.json # src/Identity/IdentityServer/CustomTokenRequestValidator.cs # src/Identity/packages.lock.json # src/Infrastructure.EntityFramework/Infrastructure.EntityFramework.csproj # src/Infrastructure.EntityFramework/Repositories/DatabaseContext.cs # src/Infrastructure.EntityFramework/packages.lock.json # src/Notifications/packages.lock.json # src/SharedWeb/Utilities/ServiceCollectionExtensions.cs # src/SharedWeb/packages.lock.json # src/Sql/dbo/Stored Procedures/Organization_Create.sql # src/Sql/dbo/Stored Procedures/Organization_Update.sql # src/Sql/dbo/Tables/Organization.sql # test/Api.IntegrationTest/packages.lock.json # test/Api.Test/packages.lock.json # test/Billing.Test/packages.lock.json # test/Icons.Test/packages.lock.json # test/Identity.IntegrationTest/packages.lock.json # test/Identity.Test/packages.lock.json # test/Infrastructure.EFIntegration.Test/Repositories/CipherRepositoryTests.cs # test/Infrastructure.EFIntegration.Test/packages.lock.json # test/IntegrationTestCommon/packages.lock.json # util/MySqlMigrations/packages.lock.json # util/PostgresMigrations/Migrations/DatabaseContextModelSnapshot.cs # util/PostgresMigrations/packages.lock.json
* Fix tables not being cleaned up * Fix migration * Squash secrets manager migrations * Reset EF to pre SM state * Add EF migrations * Fix unified docker * Add missed copy
…feature # Conflicts: # util/MySqlMigrations/HelperScripts/2023-01-06_00_SecretsManager.sql # util/PostgresMigrations/HelperScripts/2023-01-06_00_SecretsManager.psql # util/SqliteMigrations/Migrations/DatabaseContextModelSnapshot.cs
* Bump EF.SqlServer version to match EF * Rename AddCommercialSecretsServices to AddCommercialSecretsManagerServices * Fix format of ServiceAccountRepository * Delete sql migration files * Fix build warnings * Remove duplicate LinqToDBForEfTools.Initialize
coltonhurst
reviewed
Jan 12, 2023
There was a problem hiding this comment.
@Hinton thanks for putting this together. I've gone through each commit & followed the review guidelines based on our long-lived feature branch documentation. Everything looks good, I just want to verify if the following have been tested or if we need to wait for testing to complete:
- SM-185
- SM-246
- SM-218
- SM-282
- SM-259
- SM-361
- SM-394
- SM-300
- SM-73
You can find links to the commit, PR, and ticket in the internal excel doc I put together. Please let me know what we might need to wait for on the above, and if we are good if some of these haven't had testing, etc.
Finally, how can we confirm & ensure everything is behind the proper feature flag? I believe everything should be, but we should double check.
|
All the endpoints should be behind the |
coltonhurst
approved these changes
Jan 13, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Objective
Secrets Manager
Type of change
Before you submit