Scratch org not listed in Scratch section when authenticated via JWT

[jclark-dot-org]

Summary

We use a server-based automation (bitbucket pipelines) to create new scratch orgs, then authenticate from a dev's local machine to the new scratch using auth:jwt:token. When we do this, the scratch org is listed in the first (non-scratch) section of force:org:list, and expiration date info is not available.

Steps To Reproduce:

NOTE you will need two systems with sfdx installed to reproduce this issue. It may be possible to fake this by doing all steps on one system, but deleting the org file from ~/.sfdx after step 3, but I have not tested that scenario.

1. Ensure that your dev hub is configured for JWT auth and that you have your server key file and client key.
2. On machine 1, authenticate to your dev hub via JWT.
3. Using any repo, On machine 1, create a scratch org. Make note of the username created.
4. On machine 2, authenticate to your dev hub via JWT.
5. On machine 2, authenticate to the new scratch org using JWT. For example, sfdx auth:jwt:grant --clientid ${!CONSUMER_KEY} --jwtkeyfile ./server.key -r https://test.salesforce.com --username [email protected] -a bugreport
6. On machine 2, run sfdx force:org:list.

Expected result

The new scratch org (alias bugreport) should be listed in the second section of output, as a scratch org, with an expiration date.

Actual result

The new scratch org is listed in the first section. No expiration date information is available.

Additional information

This is NOT a duplicate of #845; creating a scratch and using it on the same system does not exhibit this issue.

This is actually behavior that has existed for quite a while, however, prior to a recent release (I think it changed in v54.4), running force:org:display for the new org would fix the issue, by updating the org file in ~/.sfdx. This is no longer the case.

If you compare org files in ~/.sfdx, comparing a scratch created on the local system with a JWT-auth'd scratch created elsewhere, you can see the the JWT version is missing the keys createdOrgInstance, created, and expirationDate. Manually adding the expirationDate key to the file with a reasonable value is sufficient to make force:org:list display the org as a scratch org.

SFDX CLI Version(to find the version of the CLI engine run sfdx --version):

sfdx-cli/7.93.1-762bca056d darwin-x64 node-v14.15.4

SFDX plugin Version(to find the version of the CLI plugin run sfdx plugins --core)

@oclif/plugin-autocomplete 0.3.0 (core)
@oclif/plugin-commands 1.3.0 (core)
@oclif/plugin-help 3.2.2 (core)
@oclif/plugin-not-found 1.2.4 (core)
@oclif/plugin-plugins 1.10.0 (core)
@oclif/plugin-update 1.4.0-3 (core)
@oclif/plugin-warn-if-update-available 1.7.0 (core)
@oclif/plugin-which 1.0.3 (core)
@salesforce/sfdx-trust 3.6.0 (core)
alias 1.1.8 (core)
auth 1.5.1 (core)
config 1.2.7 (core)
generator 1.1.5 (core)
salesforcedx 51.5.0 (core)
├─ limits 1.0.4 (core)
├─ schema 1.0.4 (core)
├─ custom-metadata 1.0.11 (core)
├─ apex 0.1.4 (core)
├─ org 1.5.0 (core)
├─ user 1.2.0 (core)
├─ @salesforce/sfdx-plugin-lwc-test 0.1.7 (core)
├─ salesforce-alm 51.6.3 (core)
└─ templates 51.3.0 (core)
sfdx-cli 7.93.1 (core)
sfpowerkit 2.8.6
telemetry 1.1.1 (core)

OS and version:

macOS version 10.14.6

[github-actions]

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

[jclark-dot-org]

I originally stated

Manually adding the expirationDate key to the file with a reasonable value is sufficient to make force:org:list display the org as a scratch org.

However, that was the old behavior; now it appears that manually adding the "devHubUsername" key is the correct workaround.

[mshanemc]

TLDR; it doesn't work the way that you are trying to use it.

Scratch orgs stored locally are meant to be the result of the org:create command (we track them as scratch orgs associated with the dev hub that created them).

If you auth into a scratch org any other way (web via username/password, auth_url, jwt) it's going to be treated as a non-scratch org. There's no good way to ask a scratch org for its devhub info, so we don't, by default, know where to go for that information.

The workaround is as you described--manually setting the "stuff that makes it look like a scratch org" in the auth files. The most important is the dev hub--that's where org:display and org:list go to get the expiration/status information (they always ask the hub for an update).

[uip-robot-zz]

This issue has been linked to a new work item: W-9073698

[github-actions]

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

[github-actions]

Thank you for filing this feature request. We appreciate your feedback and will review the feature at our next grooming or sprint planning session. We prioritize feature requests with more upvotes and comments.

[mshanemc]

I logged this as a feature request, not a bug.

[jclark-dot-org]

@mshanemc Thanks for the explanation, that makes sense. I can say that in the past, force:org:display would fix the problem; perhaps the old version assumed the currently active devhub? An option to check the current devhub, or specify a hub user, during auth would be very helpful. For now, we'll probably look for a simple way to automate updating the local org file after auth.

[jayree]

@jclark-dot-org can you please try to run force:org:shape:list and check if the scratch orgs are listed in the correct section again.

[jclark-dot-org]

@jayree excellent! Workaround confirmed. Running the command did output as expected (I'm not currently using orgshape):

=== Org Shapes 
No org shapes found.

But the next run of force:org:list correctly identified several scratch orgs which were not previously displayed as scratch orgs.

[azlam-abdulsalam]

Thanks @jayree and @jclark-dot-org , this is useful when you fetch scratch org from a pool!

[aheber]

Agreed, this is useful and a topic of discussion among my team this morning. Much appreciated @jayree.

[mshanemc]

https://github.com/forcedotcom/cli/blob/main/releasenotes/README.md#51140-may-27-2021---cli-71030