Support hcp vault secrets

[xrow]

SUMMARY

https://developer.hashicorp.com/vault/tutorials/hcp-vault-secrets-get-started

I did some testing. I looks like the ansible modules are not designed to work with hcp-vault-secrets out of the box.

ISSUE TYPE

• Feature Idea

COMPONENT NAME

community.hashi_vault.vault_login <- seems to not able to use the oauth endpoint https://auth.hashicorp.com/oauth/token
community.hashi_vault.vault_read <- seems to not able to read secrets endpoint https://api.hashicorp.cloud

[briantist]

Hi @xrow , I don't have access to HCP Vault to run any tests myself, and you haven't included the actual Ansible tasks you're running so it's a bit hard to tell, but my first guess would be that you're missing the namespace.

HCP Vault uses namespaces, and I think the default namespace is admin. Otherwise, HCP Vault, under the hood, is still Vault, and thus uses the same API, and so it should work with this collection, as long as the library we use (hvac) works with it.

All of the content in this collection that interacts with Vault takes a namespace parameter which you can set to the namespace name.

If you have some further information to go on, I can try to help from there.

[xrow]

@briantist

Hi,

you might be right the issue might be a documentation issue. The3 good thing is HCP Vault is free for testing and a small about of serects. You can try it on your own without issues. Would you mind diving it a try? The setup of the cloud hosted hcp vault secrets takes about 5 minutes.

[briantist]

Unfortunately I can't do that at the moment, I've got a lot going on and very limited time in the next several weeks. I wouldn't mind giving it a try once I have some more time.

[xrow]

Do not worry. It was worth trying for me. I just wanted to leave you a note.

[xrow]

About the namespace parameter:

This is the url to get a serect. it looks like i would need to set export ANSIBLE_HASHI_VAULT_NAMESPACE="/secrets/2023-06-13/organizations/147d4839-53ce-41bc-910e-4a4c3cce9ec6/projects/8eda565f-d207-45f6-adb3-a8b7f08ee338"
, but i do not know.

curl \
--location "https://api.cloud.hashicorp.com/secrets/2023-06-13/organizations/147d4839-53ce-41bc-910e-4a4c3cce9ec6/projects/8eda565f-d207-45f6-adb3-a8b7f08ee338/apps/infrastructure/open" \
--request GET \
--header "Authorization: Bearer $HCP_API_TOKEN" | jq

[briantist]

Interesting... that URL looks very foreign to me. Offhand, that URL looks nothing like the standard Vault API, notice the lack of /v1/ anywhere.

Unfortunately HashiCorp's documentation is kind of a mess; I spent a little while before replying above trying to figure it out but was not able to get a definitive answer.

It may be best to reach out to HashiCorp and ask them how you can use the standard Vault API with HCP Vault. As long as you can do that, you should be able to use any of the various tools and libraries out there, including this one.

If this is some new API they've wrapped around Vault, then I'm not sure where to go from there, that would be a baffling decision on their part from my POV but perhaps that's the case.

[xrow]

HCP Vault is some API around vault I guess. At the new API least it is documented ;-) https://developer.hashicorp.com/hcp/api-docs/vault-secrets

[briantist]

I am now seeing that HCP Vault and HCP Vault Secrets are two completely different things 🤦 this naming is pretty terrible: https://www.hashicorp.com/products/vault/pricing

It seems to indeed be a completely different set of APIs unfortunately.

[xrow]

@briantist
Would you maybe recommend this is out of scope for this project? That one shall create a new ansible module?

[briantist]

Support for HCP Vault Secrets is something I would consider in-scope for this collection, but because it's a completely new API, that does add a considerable amount of work for properly adding support. By contrast, adding additional Vault features in the standard Vault API is relatively lightweight since we have so much supporting code around it.

That being said, this collection is essentially a wrapper around the hvac python package, so that is where the support should actually be added first. I also maintain that package, and much of what I said above about there being a fair bit of initial work to support it also holds true in adding the support to hvac, but that would be the way to go. Feel free to open an issue there to request the support; it's something we'll have to give some thought to in any case.

---

You're always free to create your own Ansible content of course, and having something you write locally that's scoped to your own use case will probably be the quickest way forward for your project.

[xrow]

hvac/hvac#1106

[xrow]

Ok to let you know for my current case I will swtich back to the normal vault for now.

[burner1024]

Spent the better part of the day trying to lookup secrets from HCP in Ansible...
Maybe add a line in readme, note it's not supported yet?

[briantist]

I would accept a PR adding the information to the README and/or documentation.

[xrow]

maybe this helps

#!/usr/bin/python
# -*- coding: utf-8 -*-

# GNU General Public License v3.0+ (see LICENSES/GPL-3.0-or-later.txt or https://www.gnu.org/licenses/gpl-3.0.txt)
# SPDX-License-Identifier: GPL-3.0-or-later

from __future__ import absolute_import, division, print_function
__metaclass__ = type

ANSIBLE_METADATA = {
    'metadata_version': '1.0',
    'status': ['preview'],
    'supported_by': 'community'
}

DOCUMENTATION = r'''
---
module: hashicorp_secrets

short_description: This is my hashicorp_secrets module

# If this is part of a collection, you need to use semantic versioning,
# i.e. the version is of the form "2.5.0" and not "2.4".
version_added: "1.0.0"

description: This is my longer description explaining my hashicorp_secrets module.

options:
    name:
        description: This is the message to send to the test module.
        required: true
        type: str
    new:
        description:
            - Control to demo if the result of this module is changed or not.
            - Parameter description can be a list as well.
        required: false
        type: bool
# Specify this value according to your collection
# in format of namespace.collection.doc_fragment_name
# extends_documentation_fragment:
#     - my_namespace.my_collection.my_doc_fragment_name

author:
    - Björn Dieding (@xrow)
'''

EXAMPLES = r'''
# Pass in a message
- name: Test with a message
  imos.infratructure.hashicorp_secrets:
    client_id:
    client_secret:
    organization:
    project:
    application:
    key:
'''

RETURN = r'''
# These are examples of possible return values, and in general should use other names for return values.
original_message:
    description: The original name param that was passed in.
    type: str
    returned: always
    sample: 'hello world'
message:
    description: The output message that the test module generates.
    type: str
    returned: always
    sample: 'goodbye'
'''
from ansible.errors import AnsibleActionFail
from ansible.module_utils.basic import AnsibleModule
import requests
import json
import jmespath
import base64
import os

def run_module():

    # define available arguments/parameters a user can pass to the module
    module_args = dict(
        client_id=dict(type='str', required=True),
        client_secret=dict(type='str', required=True),
        organization=dict(type='str', required=True),
        project=dict(type='str', required=True),
        application=dict(type='str', required=True),
        key=dict(type='str', required=True)
    )

    # seed the result dict in the object
    # we primarily care about changed and state
    # changed is if this module effectively modified the target
    # state will include any data that you want your module to pass back
    # for consumption, for example, in a subsequent task
    result = dict(
        changed=False,
        original_message='',
        message=''
    )

    # the AnsibleModule object will be our abstraction working with Ansible
    # this includes instantiation, a couple of common attr would be the
    # args/params passed to the execution, as well as if the module
    # supports check mode
    module = AnsibleModule(
        argument_spec=module_args,
        supports_check_mode=True
    )

    # if the user is working with this module in only check mode we do not
    # want to make any changes to the environment, just return the current
    # state with no modifications
    if module.check_mode:
        module.exit_json(**result)

    # manipulate or modify the state as needed (this is going to be the
    # part where your module will do what it needs to do)
    result['original_message'] = None
    
    facts = {}

    secrets = setEnv( module.params['client_id'], 
            module.params['client_secret'], 
            module.params['organization'], 
            module.params['project'], 
            module.params['application'], 
            module.params['key']
    )

    if secrets:
        result['message'] = 'Loaded'
        result['ansible_facts'] = {}
        # just as _facts actions, we don't set changed=true as we are not modifying the actual host
        result['ansible_facts']['secrets'] = secrets
    else:
        # this should not happen, but JIC we get here
        raise AnsibleActionFail('Unable to create any variables with provided arguments')

    # in the event of a successful module execution, you will want to
    # simple AnsibleModule.exit_json(), passing the key/value results
    module.exit_json(**result)

def setEnv(client_id,client_secret,organization,project,application,key):
    data = {
        "audience": "https://api.hashicorp.cloud",
        "grant_type": "client_credentials",
        "client_id": client_id,
        "client_secret": client_secret
    }
    response = requests.post('https://auth.hashicorp.com/oauth/token', data=data)
    if response.status_code != 200:
        raise Exception('Status code was not 200: ' + str(response.status_code) + ' ' + response.text)
    token = response.json()['access_token']

    headers = {
            "Authorization": "Bearer " + token
    }
    response = requests.get('https://api.cloud.hashicorp.com/secrets/2023-06-13/organizations/' + organization + '/projects/' + project + '/apps/' + application + '/open', data=data,headers=headers)
    if response.status_code != 200:
        raise Exception('Status code was not 200: ' + str(response.status_code) + ' ' + response.text)
    jsonenv = jmespath.search("secrets[?name=='"+ key +"']", response.json() )
    data = json.loads(str(base64.b64decode(jsonenv.pop()['version']['value'],validate=True),'utf-8'))

    # raise Exception(data)

    for key in data:
        print(key+"="+data[key])
        os.environ[ key ] = data[key]

    return data

def main():
    run_module()

if __name__ == '__main__':
    main()