Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: CLI purge command #2998

Merged
merged 9 commits into from
Sep 18, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions cli/cli.go
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,7 @@ func NewDefraCommand() *cobra.Command {
MakeStartCommand(),
MakeServerDumpCmd(),
MakeVersionCommand(),
MakePurgeCommand(),
)

return root
Expand Down
1 change: 1 addition & 0 deletions cli/errors.go
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ var (
ErrSchemaVersionNotOfSchema = errors.New(errSchemaVersionNotOfSchema)
ErrViewAddMissingArgs = errors.New("please provide a base query and output SDL for this view")
ErrPolicyFileArgCanNotBeEmpty = errors.New("policy file argument can not be empty")
ErrPurgeForceFlagRequired = errors.New("run this command again with --force if you really want to purge all data")
)

func NewErrRequiredFlagEmpty(longName string, shortName string) error {
Expand Down
42 changes: 42 additions & 0 deletions cli/purge.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
// Copyright 2024 Democratized Data Foundation
//
// Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt.
//
// As of the Change Date specified in that file, in accordance with
// the Business Source License, use of this software will be governed
// by the Apache License, Version 2.0, included in the file
// licenses/APL.txt.

package cli

import (
"os"

"github.com/spf13/cobra"
)

func MakePurgeCommand() *cobra.Command {
var force bool
var cmd = &cobra.Command{
Use: "purge",
Short: "Delete all persisted DefraDB data",
Long: `Delete all persisted DefraDB data.
WARNING this operation will delete all data and cannot be reversed.`,
PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
if err := setContextRootDir(cmd); err != nil {
return err

Check warning on line 28 in cli/purge.go

View check run for this annotation

Codecov / codecov/patch

cli/purge.go#L28

Added line #L28 was not covered by tests
}
return setContextConfig(cmd)
},
RunE: func(cmd *cobra.Command, args []string) error {
if !force {
return ErrPurgeForceFlagRequired
}
cfg := mustGetContextConfig(cmd)
return os.RemoveAll(cfg.GetString("datastore.badger.path"))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thought: This will only clear the badger directory, if the user is using another datastore this command will not work (unclear if will panic, or quietly do nothing), and the added tests do not automatically cover other dbs. I cannot remember how hard-coded other CLI commands are to badger, this might be a drop in the ocean when it comes to adding support for other datastores in the CLI.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we add more datastores we can have them all use the same datastore.path as their root path and have sub directories for each datastore type. Then we just need to delete the root path to remove them all.

},
}
cmd.Flags().BoolVarP(&force, "force", "f", false, "Must be set for the operation to run")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

todo: I don't think requiring -f is sufficient protection against users nuking a production database, I can't remember the details but I thought we agreed on something slightly safer when we discussed this as a team?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I only remember the force flag from the discussion. As long as its non interactive I'm open to changing it to anything else.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we had discussed the possibility of having a dev flag on startup with only dev instances capable of calling purge. If we do this, I think we don't even need force. Also, having a dev flag will allow for other commands to be added to dev only in the future. dev could by default require no password for the keyring for example. This could all be part of a strategy to get new devs to the "fun part" asap.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we had discussed the possibility of having a dev flag on startup with only dev instances capable of calling purge. If we do this, I think we don't even need force. Also, having a dev flag will allow for other commands to be added to dev only in the future. dev could by default require no password for the keyring for example. This could all be part of a strategy to get new devs to the "fun part" asap.

I don't remember this, but I like it - is kind of like a build flag without the hassle of multiple builds, and like you said it scales to other situations very nicely too

return cmd
}
67 changes: 67 additions & 0 deletions cli/purge_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
// Copyright 2024 Democratized Data Foundation
//
// Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt.
//
// As of the Change Date specified in that file, in accordance with
// the Business Source License, use of this software will be governed
// by the Apache License, Version 2.0, included in the file
// licenses/APL.txt.

package cli

import (
"os"
"testing"

"github.com/spf13/pflag"
"github.com/spf13/viper"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)

func TestPurgeCommandWithoutForceFlag_ReturnsError(t *testing.T) {
rootDir := t.TempDir()

cmd := NewDefraCommand()
cmd.SetArgs([]string{"purge", "--rootdir", rootDir})

err := cmd.Execute()
require.ErrorIs(t, err, ErrPurgeForceFlagRequired)
}

func TestPurgeCommandWithEmptyDirectory_DoesNothing(t *testing.T) {
rootDir := t.TempDir()

cmd := NewDefraCommand()
cmd.SetArgs([]string{"purge", "--force", "--rootdir", rootDir})

err := cmd.Execute()
assert.NoError(t, err)
}

func TestPurgeCommandWithForceFlag_Succeeds(t *testing.T) {
rootDir := t.TempDir()

var dataDir string
// load the config and create the data directory
configLoader = func(rootdir string, flags *pflag.FlagSet) (*viper.Viper, error) {
cfg, err := loadConfig(rootdir, flags)
if err != nil {
return nil, err
}
dataDir = cfg.GetString("datastore.badger.path")
if err := os.MkdirAll(dataDir, 0755); err != nil {
return nil, err
}
return cfg, nil
}

cmd := NewDefraCommand()
cmd.SetArgs([]string{"purge", "--force", "--rootdir", rootDir})

err := cmd.Execute()
require.NoError(t, err)

assert.NoDirExists(t, dataDir)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

todo: This test does not test that the badger data directory is cleared, but the hardcoded path set in the test (that currently corresponds with the default badger path). Please either change this test (or add another one) so that it will test that the correct directory is cleared, no matter what value is set within datastore.badger.path.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you need to change the prod code to do this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was reverted after the team discussion

}
9 changes: 8 additions & 1 deletion cli/utils.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ import (
"github.com/decred/dcrd/dcrec/secp256k1/v4"
"github.com/sourcenetwork/immutable"
"github.com/spf13/cobra"
"github.com/spf13/pflag"
"github.com/spf13/viper"
"golang.org/x/term"

Expand Down Expand Up @@ -66,6 +67,12 @@ var readPassword = func(cmd *cobra.Command, msg string) ([]byte, error) {
return pass, err
}

// configLoader loads a config for the current command. This can be swapped
// during tests to force a common config for a command.
var configLoader = func(rootdir string, flags *pflag.FlagSet) (*viper.Viper, error) {
return loadConfig(rootdir, flags)
}

// mustGetContextDB returns the db for the current command context.
//
// If a db is not set in the current context this function panics.
Expand Down Expand Up @@ -123,7 +130,7 @@ func setContextDB(cmd *cobra.Command) error {
// setContextConfig sets teh config for the current command context.
func setContextConfig(cmd *cobra.Command) error {
rootdir := mustGetContextRootDir(cmd)
cfg, err := loadConfig(rootdir, cmd.Flags())
cfg, err := configLoader(rootdir, cmd.Flags())
if err != nil {
return err
}
Expand Down
1 change: 1 addition & 0 deletions docs/website/references/cli/defradb.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ Start a DefraDB node, interact with a local or remote node, and much more.
* [defradb client](defradb_client.md) - Interact with a DefraDB node
* [defradb identity](defradb_identity.md) - Interact with identity features of DefraDB instance
* [defradb keyring](defradb_keyring.md) - Manage DefraDB private keys
* [defradb purge](defradb_purge.md) - Delete all persisted DefraDB data
* [defradb server-dump](defradb_server-dump.md) - Dumps the state of the entire database
* [defradb start](defradb_start.md) - Start a DefraDB node
* [defradb version](defradb_version.md) - Display the version information of DefraDB and its components
Expand Down
43 changes: 43 additions & 0 deletions docs/website/references/cli/defradb_purge.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
## defradb purge

Delete all persisted DefraDB data

### Synopsis

Delete all persisted DefraDB data.
WARNING this operation will delete all data and cannot be reversed.

```
defradb purge [flags]
```

### Options

```
-f, --force Must be set for the operation to run
-h, --help help for purge
```

### Options inherited from parent commands

```
--keyring-backend string Keyring backend to use. Options are file or system (default "file")
--keyring-namespace string Service name to use when using the system backend (default "defradb")
--keyring-path string Path to store encrypted keys when using the file backend (default "keys")
--log-format string Log format to use. Options are text or json (default "text")
--log-level string Log level to use. Options are debug, info, error, fatal (default "info")
--log-output string Log output path. Options are stderr or stdout. (default "stderr")
--log-overrides string Logger config overrides. Format <name>,<key>=<val>,...;<name>,...
--log-source Include source location in logs
--log-stacktrace Include stacktrace in error and fatal logs
--no-keyring Disable the keyring and generate ephemeral keys
--no-log-color Disable colored log output
--rootdir string Directory for persistent data (default: $HOME/.defradb)
--source-hub-address string The SourceHub address authorized by the client to make SourceHub transactions on behalf of the actor
--url string URL of HTTP endpoint to listen on or connect to (default "127.0.0.1:9181")
```

### SEE ALSO

* [defradb](defradb.md) - DefraDB Edge Database

Loading