From 379d120017588858d1dbd82618b7ad839ab596af Mon Sep 17 00:00:00 2001 From: Eric Schrock Date: Wed, 2 Oct 2019 13:18:24 -0400 Subject: [PATCH 1/3] some CLI commands --- docs/src/cli/cmd/abort.rst | 26 +++++++++++++++++- docs/src/cli/cmd/cp.rst | 47 ++++++++++++++++++++++++++++++++- docs/src/cli/cmd/push.rst | 36 ++++++++++++++++++++++++- docs/src/cli/cmd/remote_add.rst | 33 ++++++++++++++++++++++- docs/src/cli/cmd/remote_log.rst | 36 ++++++++++++++++++++++++- docs/src/cli/cmd/remote_ls.rst | 25 +++++++++++++++++- docs/src/cli/cmd/remote_rm.rst | 27 ++++++++++++++++++- docs/src/cli/cmd/rm.rst | 26 +++++++++++++++++- docs/src/cli/cmd/run.rst | 24 ++++++++++++++++- docs/src/cli/cmd/start.rst | 24 ++++++++++++++++- docs/src/cli/cmd/status.rst | 5 +++- docs/src/cli/cmd/stop.rst | 24 ++++++++++++++++- 12 files changed, 321 insertions(+), 12 deletions(-) diff --git a/docs/src/cli/cmd/abort.rst b/docs/src/cli/cmd/abort.rst index 398184fa..5c707e74 100644 --- a/docs/src/cli/cmd/abort.rst +++ b/docs/src/cli/cmd/abort.rst @@ -3,4 +3,28 @@ titan abort =========== -Coming soon! +Aborts any in-progress push or pull operation for a repository. Most push and +pull operations finish in the context of when they were invoked, but in rare +cases (such as the CLI exiting unexpectedly), they can continue to run in the +background. This command provides a way to abort such operations. + +Syntax +------ + +:: + + titan abort + +Arguments +--------- + +repository + *Required*. The name of the target repository. + +Example +------- + +:: + + $ titan abort hello-world + aborting operation 13ac9b7a-15f2-41ea-9b61-bc271234d123 diff --git a/docs/src/cli/cmd/cp.rst b/docs/src/cli/cmd/cp.rst index a94f845d..1fc4fd55 100644 --- a/docs/src/cli/cmd/cp.rst +++ b/docs/src/cli/cmd/cp.rst @@ -3,4 +3,49 @@ titan cp ======== -Coming soon! +Copies data from an existing directory on the host into a titan repository. +Each container has data at a known place, such as ``/var/lib/data/postgresql`` +for PostgreSQL. This command allows users to copy data created by another +instance of the database into the titan repository. + +If there is only one volume then the destination is not required. If there +is more than one volume, then you must specify the destination within the +repository container. For more information on copying data into repositories, +including how to determine what destinations are available, see the +:ref:`local_copy` section. + +Syntax +------ + +:: + + titan cp <-s source> [-d destination] + +Arguments +--------- + +repository + *Required*. The name of the target repository. + +Options +------- + +-s, --source dir *Required*. Source directory on the host system. This + must match the layout and contents expected by the + container. + +-d, --destination dir Required if there is more than volume associated with + the repository. Specifies the path where data should + be copied to within the repository. Must correspond + to a volume path in the repository container. + +Example +------- + +:: + + $ titan cp -s /var/postgres-data/ hello-world + hello-world stopped + Copying data to hello-world/v0 + hello-world started + hello-world running with data from /var/postgres-data diff --git a/docs/src/cli/cmd/push.rst b/docs/src/cli/cmd/push.rst index 58ce6d28..fc8182b9 100644 --- a/docs/src/cli/cmd/push.rst +++ b/docs/src/cli/cmd/push.rst @@ -3,4 +3,38 @@ titan push ========== -Coming soon! +Push a commit to a remote repository. For more information on pushing +commits, see the :ref:`remote_pushpull` section. + +Syntax +------ + +:: + + titan push [-c commit] [-r remote] + +Arguments +--------- + +repository + *Required*. The name of the target repository. + +Options +------- + +-c, --commit commit Commit to push. If not specified, then the latest + commit is used. + +-r, --remote remote Name of remote to push to. If not specified, defaults + to "origin". + +Example +------- + +:: + + $ titan push hello-world + PUSH 0f53a6a4-90ff-4f8c-843a-a6cce36f4f4f to ssh RUNNING + Pushing 0f53a6a4-90ff-4f8c-843a-a6cce36f4f4f to 'origin' + Syncing /var/lib/postgresql/data + 21.10MB (17.07MB/s)PUSH 0f53a6a4-90ff-4f8c-843a-a6cce36f4f4f to ssh COMPLETE diff --git a/docs/src/cli/cmd/remote_add.rst b/docs/src/cli/cmd/remote_add.rst index cd102539..207a0f08 100644 --- a/docs/src/cli/cmd/remote_add.rst +++ b/docs/src/cli/cmd/remote_add.rst @@ -3,4 +3,35 @@ titan remote add ================ -Coming soon! +Add a new remote. For more information on managing remotes, see +the :ref:`remote_addremove` section. + +Syntax +------ + +:: + + titan remote add [-r remote] + +Arguments +--------- + +repository + *Required*. The name of the target repository. + +uri + *Required*. The remote URI to use. For more information on URI format, + see the provider sections under the :ref:`remote_remote` section. + +Options +------- + +-r, --remote remote Optional remote name. If not provided, then the name + 'origin' is assumed. + +Example +------- + +:: + + $ titan remote add -r upstream s3://titan-data-demo/hello-world/postgres hello-world diff --git a/docs/src/cli/cmd/remote_log.rst b/docs/src/cli/cmd/remote_log.rst index d471ce2e..aa403702 100644 --- a/docs/src/cli/cmd/remote_log.rst +++ b/docs/src/cli/cmd/remote_log.rst @@ -3,4 +3,38 @@ titan remote log ================ -Coming soon! +List commits in a remote. For more information on managing remotes, see +the :ref:`remote_remote` section. + +Syntax +------ + +:: + + titan remote log [-r remote] + +Arguments +--------- + +repository + *Required*. The name of the target repository. + +Options +------- + +-r, --remote remote Optional remote name. If not provided, then the name + 'origin' is assumed. + +Example +------- + +:: + + $ titan remote log hello-world + Remote: origin + Commit 0f53a6a4-90ff-4f8c-843a-a6cce36f4f4f + User: Eric Schrock + Email: Eric.Schrock@delphix.com + Date: 2019-09-20T13:45:38Z + + demo data diff --git a/docs/src/cli/cmd/remote_ls.rst b/docs/src/cli/cmd/remote_ls.rst index a444449c..c209a7b3 100644 --- a/docs/src/cli/cmd/remote_ls.rst +++ b/docs/src/cli/cmd/remote_ls.rst @@ -3,4 +3,27 @@ titan remote ls =============== -Coming soon! +List remotes for the given repository. For more information on managing remotes, +see the :ref:`remote_addremove` section. + +Syntax +------ + +:: + + titan remote ls + +Arguments +--------- + +repository + *Required*. The name of the target repository. + +Example +------- + +:: + + $ titan remote ls hello-world + REMOTE PROVIDER + origin s3 diff --git a/docs/src/cli/cmd/remote_rm.rst b/docs/src/cli/cmd/remote_rm.rst index fbf4e78b..61de932d 100644 --- a/docs/src/cli/cmd/remote_rm.rst +++ b/docs/src/cli/cmd/remote_rm.rst @@ -3,4 +3,29 @@ titan remote rm =============== -Coming soon! +Remove a remote from a repository. For more information on managing remotes, +see the :ref:`remote_addremove` section. + +Syntax +------ + +:: + + titan remote rm + +Arguments +--------- + +repository + *Required*. The name of the target repository. + +remote + *Required*. The name of the remote to remove. + +Example +------- + +:: + + $ titan remote rm hello-world origin + Removed origin from hello-world diff --git a/docs/src/cli/cmd/rm.rst b/docs/src/cli/cmd/rm.rst index bcdbd08d..0a99d793 100644 --- a/docs/src/cli/cmd/rm.rst +++ b/docs/src/cli/cmd/rm.rst @@ -3,4 +3,28 @@ titan rm ======== -Coming soon! +Removes a repository. This will stop the container and destroy any local data. +This operation cannot be undone. + +Syntax +------ + +:: + + titan rm [-f] + +Arguments +--------- + +repository + *Required*. The name of the target repository. + +Example +------- + +:: + + $ titan rm -f hello-world + Removing container hello-world + Deleting volume hello-world/v0 + hello-world removed diff --git a/docs/src/cli/cmd/run.rst b/docs/src/cli/cmd/run.rst index d20a5ccd..7c541ab4 100644 --- a/docs/src/cli/cmd/run.rst +++ b/docs/src/cli/cmd/run.rst @@ -3,4 +3,26 @@ titan run ========= -Coming soon! +Runs a new repository. This command will process all arguments after ``--``` as +arguments to ``docker run``. The name of the repository is derived from the +``--name`` parameter, and all containers must be given explicit names (vs. +having a name auto-generated by docker). In addition, the ``-d`` (daemonize) +flag must be specified such that the container runs in the background. For more +information on running repositories, see the :ref:`local_run` section. + +Syntax +------ + +:: + + titan run -- -d --name [additional docker arguments...] + +Example +------- + +:: + + $ titan run -- --name newRepo -d -p 5432:5432 postgres:10 + Creating repository newRepo + Creating docker volume newRepo/v0 with path /var/lib/postgresql/data + Running controlled container newRepo diff --git a/docs/src/cli/cmd/start.rst b/docs/src/cli/cmd/start.rst index 1798ac30..b12fb6d3 100644 --- a/docs/src/cli/cmd/start.rst +++ b/docs/src/cli/cmd/start.rst @@ -3,4 +3,26 @@ titan start =========== -Coming soon! +Starts the container associated with the repository. Equivalent to +``docker start``. + +Syntax +------ + +:: + + titan start + +Arguments +--------- + +repository + *Required*. The name of the target repository. + +Example +------- + +:: + + $ titan start hello-world + hello-world started diff --git a/docs/src/cli/cmd/status.rst b/docs/src/cli/cmd/status.rst index 03f46ade..9d107a7f 100644 --- a/docs/src/cli/cmd/status.rst +++ b/docs/src/cli/cmd/status.rst @@ -3,4 +3,7 @@ titan status ============ -Coming soon! +.. note:: + + The status command is currently a stub and does not report real information. + This will be addressed in an upcoming release. diff --git a/docs/src/cli/cmd/stop.rst b/docs/src/cli/cmd/stop.rst index 645df74c..70bb5ab1 100644 --- a/docs/src/cli/cmd/stop.rst +++ b/docs/src/cli/cmd/stop.rst @@ -3,4 +3,26 @@ titan stop ========== -Coming soon! +Stops the container associated with the repository. Equivalent to +``docker stop``. + +Syntax +------ + +:: + + titan stop + +Arguments +--------- + +repository + *Required*. The name of the target repository. + +Example +------- + +:: + + $ titan stop hello-world + hello-world stopped From 77ae2a35071fa481746352b1abddd6fb33454bac Mon Sep 17 00:00:00 2001 From: Eric Schrock Date: Wed, 2 Oct 2019 13:45:46 -0400 Subject: [PATCH 2/3] rest of CLI commands --- docs/src/cli/cmd/migrate.rst | 42 +++++++++++++++++++++++++++++++- docs/src/cli/cmd/pull.rst | 35 +++++++++++++++++++++++++- docs/src/cli/cmd/upgrade.rst | 27 +++++++++++++++++++- docs/src/remote/provider/ssh.rst | 13 +++++++++- 4 files changed, 113 insertions(+), 4 deletions(-) diff --git a/docs/src/cli/cmd/migrate.rst b/docs/src/cli/cmd/migrate.rst index 38616db6..f1ce6186 100644 --- a/docs/src/cli/cmd/migrate.rst +++ b/docs/src/cli/cmd/migrate.rst @@ -3,4 +3,44 @@ titan migrate ============= -Coming soon! +Migrate data from a running container into new Titan repository. Given a +stopped container with volumes mounted from the host system, this command +will create a new container with an identical configuration, and then +copy over the data into the new repository. For more information, see the +:ref:`local_migrate` section. + +.. note:: + + The migrate command will only work if the volumes have been mounted from + host system. Migrating local to the container, or from a different volume + driver, is not supported. + +Syntax +------ + +:: + + titan migrate <-s source> + +Arguments +--------- + +repository + *Required*. The name of the repository to create. + +Options +------- + +-s, --source container Name of source container. + +Example +------- + +:: + + $ titan migrate -s postgres_source hello-world + Creating repository hello-world + Creating docker volume hello-world/v0 with path /var/lib/postgresql/data + Copying data to hello-world/v0 + Commit 9560ffb0-6bbc-4b1c-acb5-1142f86c0354 + postgres_source migrated to controlled environment hello-world diff --git a/docs/src/cli/cmd/pull.rst b/docs/src/cli/cmd/pull.rst index d6eb8380..317faa84 100644 --- a/docs/src/cli/cmd/pull.rst +++ b/docs/src/cli/cmd/pull.rst @@ -3,4 +3,37 @@ titan pull ========== -Coming soon! +Pull a commit from a remote repository. For more information on pushing +commits, see the :ref:`remote_pushpull` section. + +Syntax +------ + +:: + + titan pull [-c commit] [-r remote] + +Arguments +--------- + +repository + *Required*. The name of the target repository. + +Options +------- + +-c, --commit commit Commit to pull. If not specified, then the latest + commit is used. + +-r, --remote remote Name of remote to push to. If not specified, defaults + to "origin". + +Example +------- + +:: + + $ titan pull hello-world + PULL 0f53a6a4-90ff-4f8c-843a-a6cce36f4f4f from origin RUNNING + Pulling 0f53a6a4-90ff-4f8c-843a-a6cce36f4f4f from 'origin' + PULL 0f53a6a4-90ff-4f8c-843a-a6cce36f4f4f from origin COMPLETE diff --git a/docs/src/cli/cmd/upgrade.rst b/docs/src/cli/cmd/upgrade.rst index 0fe67e34..330745ff 100644 --- a/docs/src/cli/cmd/upgrade.rst +++ b/docs/src/cli/cmd/upgrade.rst @@ -3,4 +3,29 @@ titan upgrade ============= -Coming soon! +Upgrade the titan software on the host system. This will automatically fetch +the latest version of Titan, replace the current titan binary, and then +update the titan supporting infrastructure. For more information on upgrade, +see the :ref:`local_upgrade` section. + +.. warning:: + + Upgrade requires all containers to be stopped, or the '-f' option to + forcefully stop all containers. + +Syntax +------ + +:: + + titan upgrade [-f] [-p path] + +Options +------- + +-f, --force Stop all containers. + +-p, --path Specify path to titan binary. By default, will attempt to find + the binary in the path. If you are executing titan as an + alias, or in a wrapper script, you will need to specify the + path to the actual titan binary. diff --git a/docs/src/remote/provider/ssh.rst b/docs/src/remote/provider/ssh.rst index 8c3f7a2c..40aae93d 100644 --- a/docs/src/remote/provider/ssh.rst +++ b/docs/src/remote/provider/ssh.rst @@ -6,7 +6,7 @@ SSH Provider The SSH provider enables commits to be stored on any server where the user has remote access over SSH. The URI syntax is:: - ssh://user[:password]@host/path + ssh://user[:password]@host[:port]/path The ``path`` is interpreted as an absolute path unless it starts with ``~``. The SSH provider uses rsync to copy files to subdirectories within the path, @@ -29,3 +29,14 @@ reading the metadata file for each, and comparing the result. It should only be used for storing relatively small numbers of commits. Improving this will require a new provider that includes a robust metadata layer on top of the base SSH functionality. + +.. note :: + + The SSH connection is made from within the Titan server container, so the host + name must be resolvable from within the container. In general, this should + not be an issue because it runs within the host's network namespace, but + there are exceptions, such as specifying ``localhost`` won't work properly. + If you're trying to diagnose a connectivity problem that works from + the host system but not within titan, you can run + ``docker exec -it titan-server /bin/bash`` and run standard Linux commands + (such as ssh and ping) to debug the issue. From e72931a89916fca19ef49539aef13005757df278 Mon Sep 17 00:00:00 2001 From: Eric Schrock Date: Wed, 2 Oct 2019 13:48:55 -0400 Subject: [PATCH 3/3] fix errors --- docs/src/cli/cmd/remote_add.rst | 2 +- docs/src/cli/cmd/remote_log.rst | 2 +- docs/src/cli/cmd/upgrade.rst | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/src/cli/cmd/remote_add.rst b/docs/src/cli/cmd/remote_add.rst index 207a0f08..c3f2a78c 100644 --- a/docs/src/cli/cmd/remote_add.rst +++ b/docs/src/cli/cmd/remote_add.rst @@ -21,7 +21,7 @@ repository uri *Required*. The remote URI to use. For more information on URI format, - see the provider sections under the :ref:`remote_remote` section. + see the provider sections under the :ref:`remote` section. Options ------- diff --git a/docs/src/cli/cmd/remote_log.rst b/docs/src/cli/cmd/remote_log.rst index aa403702..3baf655f 100644 --- a/docs/src/cli/cmd/remote_log.rst +++ b/docs/src/cli/cmd/remote_log.rst @@ -4,7 +4,7 @@ titan remote log ================ List commits in a remote. For more information on managing remotes, see -the :ref:`remote_remote` section. +the :ref:`remote` section. Syntax ------ diff --git a/docs/src/cli/cmd/upgrade.rst b/docs/src/cli/cmd/upgrade.rst index 330745ff..5de70e97 100644 --- a/docs/src/cli/cmd/upgrade.rst +++ b/docs/src/cli/cmd/upgrade.rst @@ -6,7 +6,7 @@ titan upgrade Upgrade the titan software on the host system. This will automatically fetch the latest version of Titan, replace the current titan binary, and then update the titan supporting infrastructure. For more information on upgrade, -see the :ref:`local_upgrade` section. +see the :ref:`lifecycle_upgrade` section. .. warning::