Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support for queryAll in regular and bulk #602

Merged
merged 4 commits into from
Jun 27, 2023
Merged

Conversation

mshanemc
Copy link
Contributor

requires jsforce/jsforce#1347

What does this PR do?

new --all-rows flag for soql query command that can query deleted stuff

What issues does this PR fix or reference?

forcedotcom/cli#1959
@W-12590109@

@cristiand391
Copy link
Member

@mshanemc bumped jsforce and sfdx-core to get NUTs running. Also updated the oclif snapshot (new flag)

@cristiand391
Copy link
Member

cristiand391 commented Jun 27, 2023

QA notes:

setup using dreamhouse project, after importing data:

➜  dreamhouse-lwc git:(main) sf data query --query 'SELECT Name FROM Broker__c'
Querying Data... ⣾
 NAME
 ─────────────────
 Caroline Kingsley
 Michael Jones
 Jonathan Bradley
 Jennifer Wu
 Olivia Green
 Miriam Aupont
 Michelle Lambert
 Victor Ochoa
Querying Data... done

then deleted:

Miriam Aupont
Michelle Lambert
Victor Ochoa

--all-rows include deleted records
deleted records only

➜  dreamhouse-lwc git:(main) ../sf/plugin-data/bin/dev data:query --query 'SELECT Name from Broker__c WHERE isDeleted = TRUE' --all-rows
 NAME
 ────────────────
 Miriam Aupont
 Michelle Lambert
 Victor Ochoa
Total number of records retrieved: 3.
Querying Data... done

all records

➜  dreamhouse-lwc git:(main) ../sf/plugin-data/bin/dev data:query --query 'SELECT Name from Broker__c' --all-rows
 NAME
 ─────────────────
 Caroline Kingsley
 Michael Jones
 Jonathan Bradley
 Jennifer Wu
 Olivia Green
 Miriam Aupont
 Michelle Lambert
 Victor Ochoa
Total number of records retrieved: 8.
Querying Data... done

without --all-rows only active records are returned

➜  dreamhouse-lwc git:(main) ../sf/plugin-data/bin/dev data:query --query 'SELECT Name from Broker__c'
 NAME
 ─────────────────
 Caroline Kingsley
 Michael Jones
 Jonathan Bradley
 Jennifer Wu
 Olivia Green
Total number of records retrieved: 5.
Querying Data... done

✅ respects max org query limit

➜  dreamhouse-lwc git:(main) SF_ORG_MAX_QUERY_LIMIT=2 ../sf/plugin-data/bin/dev data:query --query 'SELECT Name from Broker__c WHERE isDeleted = TRUE' --all-rows
Warning: The query result is missing 1 records due to a 2 record limit. Increase the number of records returned by setting the config value "org-max-query-limit" or the e
nvironment variable "SF_ORG_MAX_QUERY_LIMIT" to 3 or greater than 2.
 NAME
 ────────────────
 Miriam Aupont
 Michelle Lambert
Total number of records retrieved: 2.
Querying Data... done

✅ pagination still works

API docs says we can use either query or queryAll for pagination. I wasn't able to find an org with > 2000 deleted records to test but the example below shows pagination still works even if --all-rows is passed.

pagination logic in jsforce:
https://github.com/jsforce/jsforce/blob/796d828fb4f6905f332bf0752fd66143f220e570/src/query.ts#L640
https://github.com/jsforce/jsforce/blob/796d828fb4f6905f332bf0752fd66143f220e570/src/query.ts#L595-L598
https://github.com/jsforce/jsforce/blob/796d828fb4f6905f332bf0752fd66143f220e570/src/query.ts#L647

➜  dreamhouse-lwc git:(main) ../sf/plugin-data/bin/dev data:query -q 'SELECT COUNT() FROM ScratchOrgInfo WHERE isDeleted = TRUE' -o na40

Total number of records retrieved: 0.
➜  dreamhouse-lwc git:(main) ../sf/plugin-data/bin/dev data:query -q 'SELECT ScratchOrg FROM ScratchOrgInfo LIMIT 3450' -o na40 --json --all-rows | jq '.result.records | length'
3450
➜  dreamhouse-lwc git:(main) ../sf/plugin-data/bin/dev data:query -q 'SELECT ScratchOrg FROM ScratchOrgInfo LIMIT 3450' -o na40 --json --all-rows | jq '.result.totalSize'

3450

@cristiand391 cristiand391 merged commit a493292 into main Jun 27, 2023
@cristiand391 cristiand391 deleted the sm/queryAll branch June 27, 2023 19:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants