implemented pubkey via func and updated docs.

nikogura · Dec 26, 2020 · 613ac32 · 613ac32
1 parent 2801bcc
commit 613ac32
Show file tree

Hide file tree

Showing 5 changed files with 196 additions and 33 deletions.
diff --git a/README.md b/README.md
@@ -232,6 +232,7 @@ Output:
 
     Use "catalog [command] --help" for more information about a command.
 
+---
 
 ## Boilerplate
 
@@ -287,21 +288,33 @@ Output:
 
     Use "boilerplate [command] --help" for more information about a command.
 
+--- 
+
 ## Reposerver
 
-An http repository server.  It serves up the various dbt tools and components from a file location on disk. 
+An HTTP repository server.  It serves up the various dbt tools and components from a file location on disk. 
+
+Available Reposerver Auth Methods:
 
-At present, the repo server supports basic auth via _htpasswd_ file and public key auth from an ssh-agent via JWT.  Other auth options will be available as time allows.
+* *basic-htpasswd* Your basic htpasswd file.  Supports using different files for GET requests (dbt users) and PUT requests (dbt tool authors).
 
-Separate IDP (Identity Provider) files can be provided to provide privilege separation between tool use (GET) and tool publishing (PUT).  Likewise separate auth methods can be used for GET and PUT.
+* *ssh-agent-file* Authentication via [JWT](https://en.wikipedia.org/wiki/JSON_Web_Token) signed by an SSH key stored in the `ssh-agent`.    Users are mapped to public keys by a server-side file. IDP files for the _ssh-agent-file_ auth method can contain both GET and PUT users in a single file.
 
-Why did I make it possible to have split auth methods?  Flexibility.  Passwordless ssh-key auth for a user is good UX.  It's secure, and easy for the users.  It's kind of a pain for CI systems and other automated uses.  Sometimes just sticking a password in the environment is the best way for these use cases.  Hey, do what you want.  I'm just trying to help.
+* *ssh-agent-func* Authentication via [JWT](https://en.wikipedia.org/wiki/JSON_Web_Token) signed by an SSH key stored in the `ssh-agent`.    Users are mapped to public keys by a server-side shell function.  This method can, for instance, retrieve the SSH public key for a user from an LDAP directory.
+
+You can even have different auth methods for GET and PUT requests.  Why did I make it possible to have split auth methods?  Flexibility.  Passwordless ssh-key auth for a user is good UX for users.  It's secure, and easy for the users.  It's kind of a pain for CI systems and other automated uses.  Sometimes just sticking a password in the environment is the best way for these use cases.  Hey, do what you want.  I'm just trying to help.
 
 The PublicKey Auth IDP file contains sections for both GET and PUT, so a single file can be used for both.  Obviously if you do use separate files, only the appropriate portion of each file will be read.
 
+You can choose to require authentication on GET requests or not.  Unauthenticated gets are appropriate for running a reposerver inside of a VPN.  If you're exposing your reposerver to the internet, authenticating all requests is highly recommended.
+
 ### Reposerver Config
 
-A JSON file of the form:
+Some examples of reposerver config files:
+
+#### Basic Htpasswd Auth
+
+Create your reposerver config file should be of the form:
 
     {
 	    "address": "my-hostname.com",
@@ -313,14 +326,53 @@ A JSON file of the form:
         "authOptsGet": {
             "idpFile": "/path/to/htpasswd/file/for/gets"
         },
-        "authOptsPutt": {
+        "authOptsPut": {
             "idpFile": "/path/to/htpasswd/file/for/puts"
         },
     }
 
+#### JWT Auth with Public Keys
+
+See [https://github.com/orion-labs/jwt-ssh-agent-go#background](https://github.com/orion-labs/jwt-ssh-agent-go#background) for details.
+
+##### Public Keys Mapped to Users in a File
+
+Your reposerver config should look something like:
+
+    {
+	    "address": "my-hostname.com",
+        "port": 443,
+        "serverRoot": "/path/to/where/you/store/tools",
+        "authTypeGet": "ssh-agent-file",
+        "authTypePut": "ssh-agent-file",
+        "authGets": true,
+        "authOptsGet": {
+            "idpFile": "/path/to/idp/file"
+        },
+        "authOptsPut": {
+            "idpFile": "/path/to/idp/file"
+        },
+    }
+
+##### Public Keys Mapped to Users via Function
+    {
+	    "address": "my-hostname.com",
+        "port": 443,
+        "serverRoot": "/path/to/where/you/store/tools",
+        "authTypeGet": "ssh-agent-file",
+        "authTypePut": "ssh-agent-file",
+        "authGets": true,
+        "authOptsGet": {
+            "idpFunc": "ldapsearch '(&(objectClass=posixAccount)(uid='"$1"'))' 'sshPublicKey' | sed -n '/^ /{H;d};/sshPublicKey:/x;$g;s/\n *//g;s/sshPublicKey: //gp'"
+        },
+        "authOptsPut": {
+            "idpFunc": "ldapsearch '(&(objectClass=posixAccount)(uid='"$1"'))' 'sshPublicKey' | sed -n '/^ /{H;d};/sshPublicKey:/x;$g;s/\n *//g;s/sshPublicKey: //gp'"
+        },
+    }
+
 ### Reposerver IDP File
 
-The reposerver takes an IDP file.  In the case of http basic auth, this is a standard htpasswd file.
+The reposerver takes an IDP (Identity Provider) file.  In the case of http basic auth, this is a standard htpasswd file.
 
 In the case of Public Key JWT Auth, it looks like so:
 
@@ -353,6 +405,34 @@ Checkout the [kubernetes](kubernetes) directory for example manifests for runnin
 
 These examples use the HTTPPRoxy ingress from [projectcontour](https://projectcontour.io/).  Any old ingress will do though.
 
+### Reposerver Config Reference
+
+* *address* The IP or hostname on which your reposerver is running.
+
+* *port* The port on which your reposerver is running
+
+* *serverRoot* The directory who's contents get served up to dbt clients
+
+* *authTypeGet* Auth type to use for GET requests.
+
+* *authTypePut* Auth type to use for PUT requests.
+
+* *authGets* Whether to require authentication of GET requests at all.  Obviously if this is false, your authTypeGet doesn't amount to much.
+
+* *authOptsGet* Auth Options for GET Requests.  Can contain:
+
+    * *idpFile* File path to IDP file.
+
+    * *idpFunc* Shell function that receives the username as $1 and is expected to return a ssh public key for that username.
+
+* *authOptsPut* Auth Options for PUT Requests.  Can contain:
+
+    * *idpFile* File path to IDP file.
+
+    * *idpFunc* Shell function that receives the username as $1 and is expected to return a ssh public key for that username.
+
+---
+
 # Installation
 The easiest way to install `dbt` is via a tool called [gomason](https://github.com/nikogura/gomason). You can build via `go build` and move the files any which way you like, but `gomason` makes it easy.
 

diff --git a/metadata.json b/metadata.json
@@ -1,5 +1,5 @@
 {
-  "version": "3.2.0",
+  "version": "3.3.0",
   "package": "github.com/nikogura/dbt",
   "description": "Dynamic Binary Toolkit - A framework for running self-updating signed binaries from a central, trusted repository.",
   "repository": "http://localhost:8081/artifactory/dbt",

diff --git a/pkg/dbt/reposerver.go b/pkg/dbt/reposerver.go
@@ -236,24 +236,6 @@ func LoadPubkeyIdpFile(filePath string) (pkidp PubkeyIdpFile, err error) {
 	return pkidp, err
 }
 
-/*
-Sample Pubkey IDP File
-{
-	"getUsers": [
-		{
-			"username": "foo",
-			"publickey": ""
-		}
-	],
-	"putUsers": [
-		{
-			"username": "bar",
-			"publickey": ""
-		}
-	]
-}
-*/
-
 // PubkeyFromFilePut takes a subject name, and pulls the corresponding pubkey out of the identity provider file for puts
 func (d *DBTRepoServer) PubkeyFromFilePut(subject string) (pubkeys string, err error) {
 	idpFile, err := LoadPubkeyIdpFile(d.AuthOptsPut.IdpFile)
@@ -296,19 +278,27 @@ func (d *DBTRepoServer) PubkeyFromFileGet(subject string) (pubkeys string, err e
 }
 
 // PubkeyFromFuncPut takes a subject name, and runs the configured function to return the corresponding public key
-func (d *DBTRepoServer) PubkeysFromFuncPut(subject string) (pubkeys string, err error) {
+func (d *DBTRepoServer) PubkeysFromFuncPut(subject string) (pubkey string, err error) {
 
-	// TODO need to implement PubkeyFromFunc.
+	pubkey, err = GetFuncUsername(d.AuthOptsPut.IdpFunc, subject)
+	if err != nil {
+		err = errors.Wrapf(err, "failed to get password from shell function %q", d.AuthOptsPut.IdpFunc)
+		return pubkey, err
+	}
 
-	return pubkeys, err
+	return pubkey, err
 }
 
 // PubkeyFromFuncGet takes a subject name, and runs the configured function to return the corresponding public key
-func (d *DBTRepoServer) PubkeysFromFuncGet(subject string) (pubkeys string, err error) {
+func (d *DBTRepoServer) PubkeysFromFuncGet(subject string) (pubkey string, err error) {
 
-	// TODO need to implement PubkeyFromFunc.
+	pubkey, err = GetFuncUsername(d.AuthOptsGet.IdpFunc, subject)
+	if err != nil {
+		err = errors.Wrapf(err, "failed to get password from shell function %q", d.AuthOptsGet.IdpFunc)
+		return pubkey, err
+	}
 
-	return pubkeys, err
+	return pubkey, err
 }
 
 // AuthenticatedHandlerFunc is like http.HandlerFunc, but takes AuthenticatedRequest instead of http.Request

diff --git a/pkg/dbt/reposerver_auth_test.go b/pkg/dbt/reposerver_auth_test.go
@@ -81,7 +81,7 @@ func TestRepoServerAuth(t *testing.T) {
 			},
 		},
 		{
-			"pubkey",
+			"pubkey-file",
 			`{
 	"address": "127.0.0.1",
  "port": {{.Port}},
@@ -111,6 +111,62 @@ func TestRepoServerAuth(t *testing.T) {
     }
   ]
 }
+`,
+			authinfo{
+				user:       "nik",
+				credential: "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6jJu0QdJfhaa8d1EH33/ee8p1JgS885g8P+s4DWbuCYdYITcuHtRq+DqgEeZGBGtocQcv2CFpzHS2K3JZzB8000tz/SOgZHT1ywqCBkaA0HObBR2cpgkC2qUmQT0WFz6/+yOF22KAqKoIRNucwTKPgQGpYeWD13ALMEvh7q1Z1HmIMKdeMCo6ziBkPiMGAbPpKqzjpUbKXaT+PkE37ouCs3YygZv6UtcTzCEsY4CIpuB45FjLKhAhA26wPVsKBSUiJCMwLhN46jDDhJ8BFSv0nUYVBT/+4nriaMeMtKO9lZ6VzHnIYzGmSWH1OWxWdRA1AixJmk2RSWlAq9yIBRJk9Tdc457j7em0hohdCGEeGyb1VuSoiEiHScnPeWsLYjc/kJIBL40vTQRyiNCT+M+7p6BlT9aTBuXsv9Njw2K60u+ekoAOE4+wlKKYNrEj09yYvdl9hVrI1bNg22JsXTYqOe4TT7Cki47cYF9QwwXPZbTBRmdDX6ftOhwBzas2mAs= [email protected]",
+			},
+			[]testfile{
+				{
+					name:     "foo",
+					contents: "frobnitz ene woo",
+					result:   401,
+					auth:     false,
+				},
+				{
+					name:     "bar",
+					contents: "frobnitz ene woo",
+					result:   201,
+					auth:     true,
+				},
+			},
+			"pubkey",
+			authinfo{
+				user:       "nik",
+				credential: "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6jJu0QdJfhaa8d1EH33/ee8p1JgS885g8P+s4DWbuCYdYITcuHtRq+DqgEeZGBGtocQcv2CFpzHS2K3JZzB8000tz/SOgZHT1ywqCBkaA0HObBR2cpgkC2qUmQT0WFz6/+yOF22KAqKoIRNucwTKPgQGpYeWD13ALMEvh7q1Z1HmIMKdeMCo6ziBkPiMGAbPpKqzjpUbKXaT+PkE37ouCs3YygZv6UtcTzCEsY4CIpuB45FjLKhAhA26wPVsKBSUiJCMwLhN46jDDhJ8BFSv0nUYVBT/+4nriaMeMtKO9lZ6VzHnIYzGmSWH1OWxWdRA1AixJmk2RSWlAq9yIBRJk9Tdc457j7em0hohdCGEeGyb1VuSoiEiHScnPeWsLYjc/kJIBL40vTQRyiNCT+M+7p6BlT9aTBuXsv9Njw2K60u+ekoAOE4+wlKKYNrEj09yYvdl9hVrI1bNg22JsXTYqOe4TT7Cki47cYF9QwwXPZbTBRmdDX6ftOhwBzas2mAs= [email protected]",
+			},
+		},
+		{
+			"pubkey-func",
+			`{
+	"address": "127.0.0.1",
+ "port": {{.Port}},
+ "serverRoot": "{{.ServerRoot}}",
+ "authTypeGet": "ssh-agent-func",
+ "authGets": true,
+ "authOptsGet": {
+   "idpFunc": "echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6jJu0QdJfhaa8d1EH33/ee8p1JgS885g8P+s4DWbuCYdYITcuHtRq+DqgEeZGBGtocQcv2CFpzHS2K3JZzB8000tz/SOgZHT1ywqCBkaA0HObBR2cpgkC2qUmQT0WFz6/+yOF22KAqKoIRNucwTKPgQGpYeWD13ALMEvh7q1Z1HmIMKdeMCo6ziBkPiMGAbPpKqzjpUbKXaT+PkE37ouCs3YygZv6UtcTzCEsY4CIpuB45FjLKhAhA26wPVsKBSUiJCMwLhN46jDDhJ8BFSv0nUYVBT/+4nriaMeMtKO9lZ6VzHnIYzGmSWH1OWxWdRA1AixJmk2RSWlAq9yIBRJk9Tdc457j7em0hohdCGEeGyb1VuSoiEiHScnPeWsLYjc/kJIBL40vTQRyiNCT+M+7p6BlT9aTBuXsv9Njw2K60u+ekoAOE4+wlKKYNrEj09yYvdl9hVrI1bNg22JsXTYqOe4TT7Cki47cYF9QwwXPZbTBRmdDX6ftOhwBzas2mAs= [email protected]'"
+ },
+ "authOptsPut": {
+   "idpFunc": "echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6jJu0QdJfhaa8d1EH33/ee8p1JgS885g8P+s4DWbuCYdYITcuHtRq+DqgEeZGBGtocQcv2CFpzHS2K3JZzB8000tz/SOgZHT1ywqCBkaA0HObBR2cpgkC2qUmQT0WFz6/+yOF22KAqKoIRNucwTKPgQGpYeWD13ALMEvh7q1Z1HmIMKdeMCo6ziBkPiMGAbPpKqzjpUbKXaT+PkE37ouCs3YygZv6UtcTzCEsY4CIpuB45FjLKhAhA26wPVsKBSUiJCMwLhN46jDDhJ8BFSv0nUYVBT/+4nriaMeMtKO9lZ6VzHnIYzGmSWH1OWxWdRA1AixJmk2RSWlAq9yIBRJk9Tdc457j7em0hohdCGEeGyb1VuSoiEiHScnPeWsLYjc/kJIBL40vTQRyiNCT+M+7p6BlT9aTBuXsv9Njw2K60u+ekoAOE4+wlKKYNrEj09yYvdl9hVrI1bNg22JsXTYqOe4TT7Cki47cYF9QwwXPZbTBRmdDX6ftOhwBzas2mAs= [email protected]'"
+ },
+	"authTypePut": "ssh-agent-func"
+}`,
+			"pubkey",
+			`{
+  "getUsers": [
+    {
+      "username": "nik",
+      "publickey": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6jJu0QdJfhaa8d1EH33/ee8p1JgS885g8P+s4DWbuCYdYITcuHtRq+DqgEeZGBGtocQcv2CFpzHS2K3JZzB8000tz/SOgZHT1ywqCBkaA0HObBR2cpgkC2qUmQT0WFz6/+yOF22KAqKoIRNucwTKPgQGpYeWD13ALMEvh7q1Z1HmIMKdeMCo6ziBkPiMGAbPpKqzjpUbKXaT+PkE37ouCs3YygZv6UtcTzCEsY4CIpuB45FjLKhAhA26wPVsKBSUiJCMwLhN46jDDhJ8BFSv0nUYVBT/+4nriaMeMtKO9lZ6VzHnIYzGmSWH1OWxWdRA1AixJmk2RSWlAq9yIBRJk9Tdc457j7em0hohdCGEeGyb1VuSoiEiHScnPeWsLYjc/kJIBL40vTQRyiNCT+M+7p6BlT9aTBuXsv9Njw2K60u+ekoAOE4+wlKKYNrEj09yYvdl9hVrI1bNg22JsXTYqOe4TT7Cki47cYF9QwwXPZbTBRmdDX6ftOhwBzas2mAs= [email protected]"
+    }
+  ],
+  "putUsers": [
+    {
+      "username": "nik",
+      "publickey": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6jJu0QdJfhaa8d1EH33/ee8p1JgS885g8P+s4DWbuCYdYITcuHtRq+DqgEeZGBGtocQcv2CFpzHS2K3JZzB8000tz/SOgZHT1ywqCBkaA0HObBR2cpgkC2qUmQT0WFz6/+yOF22KAqKoIRNucwTKPgQGpYeWD13ALMEvh7q1Z1HmIMKdeMCo6ziBkPiMGAbPpKqzjpUbKXaT+PkE37ouCs3YygZv6UtcTzCEsY4CIpuB45FjLKhAhA26wPVsKBSUiJCMwLhN46jDDhJ8BFSv0nUYVBT/+4nriaMeMtKO9lZ6VzHnIYzGmSWH1OWxWdRA1AixJmk2RSWlAq9yIBRJk9Tdc457j7em0hohdCGEeGyb1VuSoiEiHScnPeWsLYjc/kJIBL40vTQRyiNCT+M+7p6BlT9aTBuXsv9Njw2K60u+ekoAOE4+wlKKYNrEj09yYvdl9hVrI1bNg22JsXTYqOe4TT7Cki47cYF9QwwXPZbTBRmdDX6ftOhwBzas2mAs= [email protected]"
+    }
+  ]
+}
 `,
 			authinfo{
 				user:       "nik",

diff --git a/pkg/dbt/util.go b/pkg/dbt/util.go
@@ -263,6 +263,43 @@ func GetFunc(shellCommand string) (result string, err error) {
 	return result, err
 }
 
+// GetFuncUsername runs a shell command that is a getter function for the username.  This could certainly be dangerous, so be careful how you use it.
+func GetFuncUsername(shellCommand string, username string) (result string, err error) {
+	// add the username as the first arg of the shell command
+	shellCommand = fmt.Sprintf("%s %s", shellCommand, username)
+
+	cmd := exec.Command("sh", "-c", shellCommand)
+
+	stdout, err := cmd.StdoutPipe()
+
+	cmd.Stdin = os.Stdin
+	cmd.Stderr = os.Stderr
+
+	cmd.Env = os.Environ()
+
+	err = cmd.Start()
+	if err != nil {
+		err = errors.Wrapf(err, "failed to run %q", shellCommand)
+		return result, err
+	}
+
+	stdoutBytes, err := ioutil.ReadAll(stdout)
+	if err != nil {
+		err = errors.Wrapf(err, "error reading stdout from func")
+		return result, err
+	}
+
+	err = cmd.Wait()
+	if err != nil {
+		err = errors.Wrapf(err, "error waiting for %q to exit", shellCommand)
+		return result, err
+	}
+
+	result = strings.TrimSuffix(string(stdoutBytes), "\n")
+
+	return result, err
+}
+
 // AuthHeaders Convenience function to add auth headers - basic or token for non-s3 requests.  Depending on how client is configured, could result in both Basic Auth and Token headers.  Reposerver will, however only pay attention to one or the other.
 func (dbt *DBT) AuthHeaders(r *http.Request) (err error) {
 	// Basic Auth