Issues flashing Jetson Orin nano devkit with TEGRA_SIGNING_ARGS set and fused keys

[wavesid]

Hello

Describe the bug
I have the exact same setup as this issue: #1639
i already opened an issue and the outcome was to fuse the keys and re-test #1674.

I am using Jetson Orin Nano 8GB devkit, with this option: TEGRA_SIGNING_ARGS set to the following

TEGRA_SIGNING_ARGS = "-u pkc.key -v sbk.key"

The keys are fused.
The official script l4t_initrd_flash.sh from Jetson Linux works with my keys so I suppose the fusing happened without issues and the keys are correct.

This is the logs:

using sudo ./doflash.sh

Welcome to Tegra Flash
version 1.0.0
Type ? or help for help and q or quit to exit
Use ! to execute system commands
 
[   0.0100 ] Parsing partition layout
[   0.0105 ] tegraparser_v2 --pt secureflash.xml.tmp
[   0.0119 ] Parsing partition layout
[   0.0122 ] tegraparser_v2 --pt secureflash.xml.tmp
[   0.0134 ] mb1_t234_prod_aligned_sigheader_encrypt.bin.signed filename is from --mb1_bin
[   0.0134 ] psc_bl1_t234_prod_aligned_sigheader_encrypt.bin.signed filename is from --psc_bl1_bin
[   0.0134 ] Boot Rom communication
[   0.0137 ] tegrarcm_v2 --new_session --chip 0x23 0 --uid --download bct_br br_bct_BR.bct --download mb1 mb1_t234_prod_aligned_sigheader_encrypt.bin.signed --download psc_bl1 psc_bl1_t234_prod_aligned_sigheader_encrypt.bin.signed --download bct_mb1 mb1_bct_MB1_sigheader_encrypt.bct.signed
[   0.0141 ] BR_CID: 0x80012344705DF11F2400000013028100
[   0.0405 ] Sending bct_br
[   0.0800 ] Sending mb1
[   0.0807 ] ERROR: might be timeout in USB write.
Error: Return value 3
Command tegrarcm_v2 --new_session --chip 0x23 0 --uid --download bct_br br_bct_BR.bct --download mb1 mb1_t234_prod_aligned_sigheader_encrypt.bin.signed --download psc_bl1 psc_bl1_t234_prod_aligned_sigheader_encrypt.bin.signed --download bct_mb1 mb1_bct_MB1_sigheader_encrypt.bct.signed

using sudo ./initrd-flash

Starting at 2024-08-23T18:03:47+02:00
Machine:       jetson-orin-nano-devkit-nvme
Rootfs device: nvme0n1p1
Found Jetson device in recovery mode at USB 1-1
== Step 1: Signing binaries at 2024-08-23T18:03:47+02:00 ==
== Step 2: Boot Jetson via RCM at 2024-08-23T18:03:48+02:00 ==
Found Jetson device in recovery mode at USB 1-1
./initrd-flash: line 191: ./rcm-boot.sh: No such file or directory
ERR: RCM boot failed at 2024-08-23T18:03:48+02:00

To Reproduce
Steps to reproduce the behavior:

1. Build meta-tegra branch 'scarthgap' (latest commit) with MACHINE set to 'jetson-orin-nano-devkit-nvme'
2. Set TEGRA_SIGNING_ARGS with -u pkc.key -v sbk.key
3. Build with bitbake image
4. Deploy to hardware with method tegraflash using sudo ./doflash.sh (or using sudo ./initrd-flash)
5. See logs above

Additional context
I checked USB connection, using PKC + SBK keys, I do not have logs in UART

The keys are fused.
The official script l4t_initrd_flash.sh from Jetson Linux works with my keys so I suppose the fusing happened without issues and the keys are correct.

Let me know if there is any way to debug

[ichergui]

Hi @wavesid

Did you try to build without TEGRA_SIGNING_ARGS and flash your device with this command line $ sudo ./doflash.sh -u PKC.pem -v SBK.key ?

[wavesid]

Yes if I build an image without TEGRA_SIGNING_ARGS set, both commands works without issues:

sudo ./initrd-flash -u PKC.pem -v SBK.key and sudo ./doflash.sh -u PKC.pem -v SBK.key

[ichergui]

@wavesid thanks for your quick feedback. I need to investigate that.I tried the Orin AGX devkit and it does work perfectly. I mean signing at build time and post build. Both methods works fine.
@madisongh Could you please share your thoughts ?
@wavesid is able to flash his device when signing/encrypting images/binaries post build. with the following command

$ sudo ./doflash.sh -u PKC.pem -v SBK.key

However, the signing at build time doesn't work when using jetson-orin-nano-devkit-nvme.

[madisongh]

If it works when directly flashing, but not when signing during the build, then my bet would be on some difference between the default FAB/BOARDSKU/etc. settings we have for the machine vs. the actual ones being reported during the flashing process.

[ichergui]

@wavesid Any update about this issue ?
Were you able to check the variables mentioned by @madisongh ?

[madisongh]

Could be the issue I mentioned in #1668

[tgagneret-embedded]

Hello,

[   0.0800 ] Sending mb1
[   0.0807 ] ERROR: might be timeout in USB write.
Error: Return value 3

I had the same issue because of USB autosuspend. Run the following command to disable usb autosuspend.
echo -1 > /sys/module/usbcore/parameters/autosuspend

I'm not sure it will solve your issue, but I hope it helps :)

[wavesid]

I will perform some more tests this week and keep you updated

Thanks

[akfaro]

I also observe this symptom with an Orin NX board, when trying to flash signed binaries with .initrd-flash

I work with meta-tegra at scarthgap, commit 5c5de2f from Nov 1 2024, so without the fix #1668.
However, I don't think that this issue is related. The mentioned fix addresses wrong binaries being sent to device, this issue is about device not even starting in RCM when working with initrd-flash.

I see why the script fails where it does, but not how to fix this:

As expected, providing a key via TEGRA_SIGNING_ARGS created a .presigning-vars file in the tegraflash package, thus setting PRESIGNED=yes within initrd-flash, which in turn leads to

• "Step 1: Signing binaries" being skipped,
• subfolder rcmboot_blob/ with a rcmbootcmd.txt file in it not being created, and
• the file ./rcm-boot.sh not being generated while unconditionally required by the function wait_for_rcm.

Or did I miss something, and non-existent ./rcm-boot.sh should not be an issue with a more recent meta-tegra?

[madisongh]

The problem is that NVIDIA changed the way its scripts handle signing the RCM boot blob in L4T R36.4, and during the upgrade to that version. I had updated our scripts to account for that change for normal, non-secured devices, but I don't have a secured Orin myself, so I missed out on the needed changes to handle that case.

Sorry about that. I was counting on others who have secured devices to help find issues like these, but this use case got missed.

I've got PR #1800 pending that should take care of this. I can't fully test it, unfortunately, but it doesn't break the non-secured use case, and it looks like it generates usable binaries when I do signing during the build.

[madisongh]

I've ported the changes from #1800 to the other R36-based branches. Let me know if they work for you.

[akfaro]

Thanks @madisongh!
I have pulled the Scarthgap branch with your fix and rebuilt the tegraflash package.

The missing rcm-boot.sh is now fixed, but I face two other issues now:

1. ERR: did not get device serial number at... - because the variable $serial_number would be written to ./boardvars.sh by the flash helper within sign_binaries(), but the call is skipped with $PRESIGNED set.
2. If I manually hard-code serial_number to the value of my board before the step "Sending flash sequence commands", the script fails at "Step 4: Writing partitions on external storage device" with these errors:

Traceback (most recent call last):
  File "....tegraflash/nvflashxmlparse", line 483, in <module>
    ret = main()
          ^^^^^^
  File "....tegraflash/nvflashxmlparse", line 408, in main
    rewrite_layout(args.filename, args.rewrite_contents_from.split(','), outf)
  File "....tegraflash/nvflashxmlparse", line 266, in rewrite_layout
    maptree = ET.parse(mapfile)
              ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/xml/etree/ElementTree.py", line 1204, in parse
    tree.parse(source, parser)
  File "/usr/lib/python3.12/xml/etree/ElementTree.py", line 558, in parse
    source = open(source, "rb")
             ^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'internal-secureflash.xml'

and

Traceback (most recent call last):
  File "....tegraflash/nvflashxmlparse", line 483, in <module>
    ret = main()
          ^^^^^^
  File "....tegraflash/nvflashxmlparse", line 426, in main
    layout = PartitionLayout(args.filename)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "....tegraflash/nvflashxmlparse", line 142, in __init__
    tree = ET.parse(configfile)
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/xml/etree/ElementTree.py", line 1204, in parse
    tree.parse(source, parser)
  File "/usr/lib/python3.12/xml/etree/ElementTree.py", line 569, in parse
    self._root = parser._parse_whole(source)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
xml.etree.ElementTree.ParseError: no element found: line 1, column 0
No partition definitions found in initrd-flash.xml
ERR: write failure to external storage at 2025-01-13T17:50:41+01:00

Maybe also because the command mv secureflash.xml internal-secureflash.xml is not executed due to early-exit from sign_binaries()?

[madisongh]

See also #1815

[otiv-arne-janssens]

I have exactly the same issue on an orin nx, but unfortunately, I don't have the luxury to update to R36.4.0/JetPack 6.1
Attempting to cherry-pick the solution in #1800 has failed so far. Any ideas on where to go from here?