Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Secret encryption fails with 128+ characters under Python 2 #2473

Closed
flabatut opened this issue Oct 1, 2018 · 1 comment
Closed

Secret encryption fails with 128+ characters under Python 2 #2473

flabatut opened this issue Oct 1, 2018 · 1 comment
Labels
status: accepted This issue has been accepted for implementation type: bug A confirmed report of unexpected behavior in the application

Comments

@flabatut
Copy link

flabatut commented Oct 1, 2018 •
edited
Loading

I can't succeed to store ssh keypair within netbox secrets resource.
I can import the content successfully but I can't retrieve it.
When I try to read/unlock the content, I receive a 500 error code with the following exception:

<pre><strong>&lt;class &#39;UnicodeDecodeError&#39;&gt;</strong><br />
&#39;utf-8&#39; codec can&#39;t decode byte 0xac in position 0: invalid start byte</pre>

Environment

Prerequisites

Steps to Reproduce

generate a ssh keypair

  • write a private key "ssh_host_ecdsa_key" with the following content:
-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAaAAAABNlY2RzYS
1zaGEyLW5pc3RwMjU2AAAACG5pc3RwMjU2AAAAQQRQ6X4h21yAfyfRqXbOgHd6pHe/p0Vh
YyM0/547xNy2bM/mSDLTEtW2Xg9eoK6wz7xFT3Y906SAB2gTFLHQquIPAAAAqGiEqL5ohK
i+AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFDpfiHbXIB/J9Gp
ds6Ad3qkd7+nRWFjIzT/njvE3LZsz+ZIMtMS1bZeD16grrDPvEVPdj3TpIAHaBMUsdCq4g
8AAAAgVbomnr+wervZTRBuK/GcsRs2DXHo0SaiGxG2r6VaUUgAAAAJcm9vdEB0ZXN0AQID
BAUGBw==
-----END OPENSSH PRIVATE KEY-----
  • write a public key "ssh_host_ecdsa_key.pub" with the following content:
ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFDpfiHbXIB/J9Gpds6Ad3qkd7+nRWFjIzT/njvE3LZsz+ZIMtMS1bZeD16grrDPvEVPdj3TpIAHaBMUsdCq4g8= root@test

import content (using web UI)

Assuming the following:

  • your device name is "test"
  • your secret-role name is "ssh_host_ecdsa"
for the private key, copy/paste the following:
device,role,name,plaintext
test,ssh_host_ecdsa,private,"-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAaAAAABNlY2RzYS
1zaGEyLW5pc3RwMjU2AAAACG5pc3RwMjU2AAAAQQRQ6X4h21yAfyfRqXbOgHd6pHe/p0Vh
YyM0/547xNy2bM/mSDLTEtW2Xg9eoK6wz7xFT3Y906SAB2gTFLHQquIPAAAAqGiEqL5ohK
i+AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFDpfiHbXIB/J9Gp
ds6Ad3qkd7+nRWFjIzT/njvE3LZsz+ZIMtMS1bZeD16grrDPvEVPdj3TpIAHaBMUsdCq4g
8AAAAgVbomnr+wervZTRBuK/GcsRs2DXHo0SaiGxG2r6VaUUgAAAAJcm9vdEB0ZXN0AQID
BAUGBw==
-----END OPENSSH PRIVATE KEY-----"
for the public key, copy/paste the following:
device,role,name,plaintext
test,ssh_host_ecdsa,public,ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFDpfiHbXIB/J9Gpds6Ad3qkd7+nRWFjIzT/njvE3LZsz+ZIMtMS1bZeD16grrDPvEVPdj3TpIAHaBMUsdCq4g8= root@test

Expected Behavior

  • we should be able to "unlock" (when using web UI) both public and private secret data.

Observed Behavior

  • Unlocking private key secret data works and display content
  • Unlocking public key secret data pops up a window with the following content:
<!DOCTYPE html>
<html lang="en">

<head>
    <title>Server Error</title>
    <link rel="stylesheet" href="/static/bootstrap-3.3.7-dist/css/bootstrap.min.css">
    <link rel="stylesheet" href="/static/font-awesome-4.7.0/css/font-awesome.min.css">
    <meta charset="UTF-8">
</head>

<body>
    <div class="container-fluid">
        <div class="row">
            <div class="col-md-6 col-md-offset-3">
                <div class="panel panel-danger" style="margin-top: 200px">
                    <div class="panel-heading">
                        <strong>
                            <i class="fa fa-warning"></i>
                            Server Error
                        </strong>
                    </div>
                    <div class="panel-body">
                        
                            <p>
                                There was a problem with your request. Please contact an administrator.
                            </p>
                        
                        <hr />
                        <p>
                            The complete exception is provided below:
                        </p>
<pre><strong>&lt;class &#39;UnicodeDecodeError&#39;&gt;</strong><br />
&#39;utf-8&#39; codec can&#39;t decode byte 0xac in position 0: invalid start byte</pre>
                        <p>
                            If further assistance is required, please post to the <a href="https://groups.google.com/forum/#!forum/netbox-discuss">NetBox mailing list</a>.
                        </p>
                        <div class="text-right">
                            <a href="/" class="btn btn-primary">Home Page</a>
                        </div>
                    </div>
                </div>
            </div>
        </div>
    </div>
</body>

</html>

On server side, nginx logs return the following HTTP error code:

netbox_1    | 172.18.0.1 - - [01/Oct/2018:13:49:49 +0000] "GET /api/secrets/secrets/78/ HTTP/1.0" 500 1798 "http://10.5.2.254:32769/secrets/secrets/78/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:62.0) Gecko/20100101 Firefox/62.0"

Additional comments

  • import is always successful. Unlock/read secret content genrerates the error mentionned above
  • The problem is random and not specific to the tested and mentionned content sample below
  • problem is reproducible either using web UI or via CLI/API
  • problem still persists even if you intend to encode data (using either b64encode/urllib.quote)
  • no matter what key type is, problem occurs (rsa,dsa,...)
@flabatut flabatut changed the title unable to safely import data into secrets resources Unable to safely retrieve stored data from secret resources Oct 2, 2018
mwheckmann pushed a commit to mwheckmann/netbox that referenced this issue Oct 19, 2018
Fix encoding of plaintext length indicator for secrets (fixes netbox-…
11b85e5 
…community#2473)

The encoding of the secret length indicator has likely been broken
since commit b21833f which introduced
Py3 support for secrets: the indicator was switched to using UTF-8 byte
strings and this caused the length indicator to consume 3 bytes instead
of just 2 under certain circumstances. For example when the plaintext
secret length is between 128 and 256 bytes long. This is because contrary
to latin-1 encoding, UTF-8 byte sting encoding consumes 2 bytes for
code points > 80. See the table at https://en.wikipedia.org/wiki/UTF-8#Description

The fix is to explicitely use 'latin-1' encoding for the length indicator.
This makes the code behave exactly as it did with the original Python2
implemenation while remaining compatible w/ Py3.

A test case which uses a 171 byte long plaintext string has also been added.
This test triggers the bug when the fix is not present.

This fixes netbox-community#2473
@mwheckmann mwheckmann mentioned this issue Oct 19, 2018
Closed
mwheckmann pushed a commit to mwheckmann/netbox that referenced this issue Oct 19, 2018
Fix netbox-community#2473: Switch to using a separate code path for p…
21b75fe 
…y2 or py3

for the plaintext length indicator field.

Py2.7 cannot encode code points > 128 so fall back to chr(code_point)
without the "encode()" attribute when Python 2.x is detected.

This does not change the existing decrypt/unpad code paths. So whatever
is currently broken in the DB due to bug netbox-community#2473 will remain broken.
@jeremystretch
Copy link
Member

Wow, this is all over the place. I can confirm that NetBox generates an exception when attempting to create a secret larger than 127 bytes under Python 2, but beyond that much of what you're saying doesn't make sense.

import is always successful.

I receive the same exception no matter how the secret is created. The exception originates from within the _pad method, which is always called on save().

The problem is random

Again, this is consistently reproducible.

problem still persists even if you intend to encode data

I'm not sure what this is in reference too. NetBox doesn't care what you do with the stored data.

no matter what key type is, problem occurs (rsa,dsa,...)

This has nothing to do with SSH key types; it is solely an issue regarding the encoding of data size. The same thing happens with any string of 128+ characters.

@jeremystretch jeremystretch changed the title Unable to safely retrieve stored data from secret resources Secret encryption fails with 128+ characters under Python 2 Nov 5, 2018
@jeremystretch jeremystretch added type: bug A confirmed report of unexpected behavior in the application status: accepted This issue has been accepted for implementation labels Nov 5, 2018
@jeremystretch jeremystretch mentioned this issue Nov 20, 2018
Merged
@lock lock bot locked as resolved and limited conversation to collaborators Jan 16, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
status: accepted This issue has been accepted for implementation type: bug A confirmed report of unexpected behavior in the application
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix encoding of plaintext length indicator for secrets (fixes #2473) mwheckmann/netbox
2 participants
@jeremystretch @flabatut