LG-13963: 👤 Socure shadow mode background job #11139

matthinz · 2024-08-22T19:35:08Z

(Previously: #11093)

🎫 Ticket

Link to the relevant ticket:
LG-13963

🛠 Summary of changes

Adds a new disabled background job, SocureShadowModeProofingJob, to do proofing with Socure KYC and log the result.

To enable the job, you need to set idv_socure_shadow_mode: true in your config. Then it will be scheduled automatically by the ResolutionProofingJob.

📜 Testing Plan

To test this locally, you will need a Socure KYC API key. To get one:

Log into the Socure dashboard
Make sure you're in the "Sandbox" environment
Navigate to the "ID+ Keys" tab in the "Developers" section and copy a key

Then update your application.yml:

  idv_socure_shadow_mode_enabled: true
  socure_idplus_api_key: 'YOUR_API_KEY_HERE'
  socure_idplus_base_url: 'https://sandbox.socure.us'
  socure_idplus_timeout_in_seconds: 5

Then, complete identity verification. DO NOT USE REAL PII. You should see a new event logged in your events.log called idv_socure_shadow_mode_proofing_result:

Here is an example

{
  "name": "idv_socure_shadow_mode_proofing_result",
  "properties": {
    "event_properties": {
      "resolution_result": {
        "success": true,
        "errors": {},
        "exception": null,
        "timed_out": false,
        "threatmetrix_review_status": "pass",
        "context": {
          "device_profiling_adjudication_reason": "device_profiling_result_pass",
          "resolution_adjudication_reason": "pass_resolution_and_state_id",
          "should_proof_state_id": true,
          "stages": {
            "resolution": {
              "success": true,
              "errors": {},
              "exception": null,
              "timed_out": false,
              "transaction_id": "resolution-mock-transaction-id-123",
              "reference": "aaa-bbb-ccc",
              "can_pass_with_additional_verification": false,
              "attributes_requiring_additional_verification": [],
              "vendor_name": "ResolutionMock",
              "vendor_workflow": null,
              "verified_attributes": null
            },
            "residential_address": {
              "success": true,
              "errors": {},
              "exception": null,
              "timed_out": false,
              "transaction_id": "",
              "reference": "",
              "can_pass_with_additional_verification": false,
              "attributes_requiring_additional_verification": [],
              "vendor_name": "ResidentialAddressNotRequired",
              "vendor_workflow": null,
              "verified_attributes": null
            },
            "state_id": {
              "success": true,
              "errors": {},
              "exception": null,
              "mva_exception": null,
              "requested_attributes": {},
              "timed_out": false,
              "transaction_id": "state-id-mock-transaction-id-456",
              "vendor_name": "StateIdMock",
              "verified_attributes": []
            },
            "threatmetrix": {
              "client": null,
              "success": true,
              "errors": {},
              "exception": null,
              "timed_out": false,
              "transaction_id": "ddp-mock-transaction-id-123",
              "review_status": "pass"
            }
          }
        },
        "ssn_is_unique": true
      },
      "socure_result": {
        "success": true,
        "errors": {
          "reason_codes": [
            "I919"
          ]
        },
        "exception": null,
        "timed_out": false,
        "transaction_id": "4fc4b744-79d0-4df7-8751-e087609fb1fa",
        "reference": "",
        "can_pass_with_additional_verification": false,
        "attributes_requiring_additional_verification": [],
        "vendor_name": "socure_kyc",
        "vendor_workflow": null,
        "verified_attributes": [
          "address",
          "first_name",
          "last_name",
          "ssn",
          "dob"
        ]
      }
    },
    "new_event": true,
    "path": null,
    "session_duration": null,
    "user_id": "96d74912-6af8-4106-8d93-7485f04a3137",
    "locale": "en"
  },
  "time": "2024-08-22T19:25:45.678Z",
  "id": "d367752a-707e-4708-8ac1-781167163b0c",
  "visitor_id": "0b2d3739-287f-47ef-90c4-952b2fedb72e",
  "visit_id": "8a3f78d2-406e-4481-b839-ac18c03c9fda",
  "log_filename": "events.log"
}

You should also verify that this job does not run when idv_socure_shadow_mode_enabled is not set or is disabled.

matthinz · 2024-08-22T19:36:21Z

app/jobs/resolution_proofing_job.rb

+      SocureShadowModeProofingJob.perform_later(
+        document_capture_session_result_id: document_capture_session.result_id,
+        encrypted_arguments:,
+        service_provider_issuer:,
+        user_email: user_email_for_proofing(user),
+        user_uuid: user.uuid,
+      )


More to do here around including phone number (which we don't currently pass to ResolutionProofingJob). Captured as LG-14282

did we consider enqueuing this job in parallel from inside IDV agent?

This job depends on the resolution proofing job completing before it can run (since it reads results from the proofing doc capture session). So parallel wouldn't work here

Is that so we can compare the results easily in Cloudwatch? I feel like we would be able to accomplish that with a coalesce() | stats ... by properties.user_id or maybe we come up with a job id to join them by that's shared?

Ooh, yeah, I like the job id idea, lemme noodle on that

There is maybe an argument for using trace_id as a value that can connect the two jobs... It gets passed into ResolutionProofingJob, but then the IdV: doc auth verify proofing results event gets logged by the verify info concern as part of a different request, and so will have a different trace id.

So then I thought maybe create a new socure_shadow_mode_job_id, but that gets a little awkward because we have to store it on the DocumentCaptureSession so that the verify info concern can include it in the payload sent to verify proofing results so that we can then link those events with the socure events.

I don't think it's good enough to use user_id as a key here, since a single user can have multiple submissions.

Which brings me back to the solution I'm using here: schedule the job after the resolution proofing job has run, log payloads for existing + socure in a new event.

The intention here is to run this for a short period of time to gather data, so one advantage to this approach is that the change is mostly contained to the SocureShadowModeProofingJob class, with a little tweak to ResolutionProofingJob.

(I did just create LG-14328 to try addressing some of the awkwardness caused by logging our event from a controller rather than a background job)

ok! I was thinking this, but for all the reasons you stated this seems fine as-is too

job_id = SecureRandom.uuid ResolutionProofingJob.perform_later(job_id:, ...) SocureShadowModeProofingJob.perform_later(job_id:, ...)

matthinz · 2024-08-22T19:37:35Z

app/jobs/socure_shadow_mode_proofing_job.rb

+      pii_like_keypaths: [
+        [:errors, :ssn],
+        [:resolution_result, :context, :stages, :resolution, :errors, :ssn],
+        [:resolution_result, :context, :stages, :residential_address, :errors, :ssn],
+        [:resolution_result, :context, :stages, :threatmetrix, :response_body, :first_name],
+        [:resolution_result, :context, :stages, :state_id, :state_id_jurisdiction],
+      ],


These are copied from where we log the Verify proofing results event. Socure results don't include PII so we should be good.

matthinz · 2024-08-22T19:40:26Z

spec/jobs/socure_shadow_mode_proofing_job_spec.rb

+
+      it 'does not log an idv_socure_shadow_mode_proofing_result_missing event' do
+        perform
+        expect(analytics).not_to have_logged_event(:idv_socure_shadow_mode_proofing_result_missing)


We did not previously support not_to have_logged_event -- that's what the update to HaveLoggedEventMatcher (below) is about.

Thanks for adding it!

matthinz · 2024-08-22T19:42:23Z

app/jobs/socure_shadow_mode_proofing_job.rb

+  def load_proofing_result(document_capture_session_result_id:)
+    DocumentCaptureSession.new(
+      result_id: document_capture_session_result_id,
+    ).load_proofing_result&.result
+  end


I realized that the proofing result should still be available assuming the SocureShadowModeProofingJob runs soon after the ResolutionProofingJob, so it makes more sense to load the result from there rather than pass it in (which I'd been thinking when I wrote the ticket).

If jobs get delayed and this result is occasionally not available that's not a big deal.

app/jobs/socure_shadow_mode_proofing_job.rb

n1zyy

I have a few nits and questions, and see that Zach added some as well, but overall this is looking good. Only commenting for the moment as I haven't tested this yet, but no real issues.

n1zyy · 2024-08-26T20:49:36Z

app/jobs/resolution_proofing_job.rb

@@ -109,6 +119,10 @@ def make_vendor_proofing_requests(
    )
  end

+  def user_email_for_proofing(user)
+    user.confirmed_email_addresses.first.email


I got distracted wondering if we needed to worry about getting a nil in here somewhere. It looks like there's a lot of prior art for not needing it, though.

That, in turn, led to me wondering if this should actually be implemented on User as, e.g., user.first_email_address or something.

yeah user.email is a deprecated method but having user.first_email_address would be very clear

app/jobs/socure_shadow_mode_proofing_job.rb

spec/features/idv/analytics_spec.rb

spec/jobs/socure_shadow_mode_proofing_job_spec.rb

Add job to make requests to Socure's KYC API and log the results alongside the original resolution proofing result. changelog: Upcoming Features, Identity verification, Add background job for Socure KYC proofing

- When flag is enabled, invoke Socure KYC as well

These are real big and mess with Cloudwatch's ability to parse fields out oflogs.

Co-authored-by: Zach Margolis <[email protected]>

…-shadow-mode

This reverts commit fd4dcab.

…-shadow-mode

eileen-nava

Thanks for giving Joy a heads-up about this PR. Looks great. 👏🏻

eileen-nava · 2024-08-28T17:48:04Z

app/jobs/socure_shadow_mode_proofing_job.rb

+    socure_result = proofer.proof(applicant)
+
+    analytics.idv_socure_shadow_mode_proofing_result(
+      resolution_result: format_proofing_result_for_logs(proofing_result),


Non-blocking nit: My read is that this method removes the threatmetrix response body. Would it be worth indicating that in the method name?

spec/jobs/socure_shadow_mode_proofing_job_spec.rb

eileen-nava · 2024-08-28T18:02:51Z

spec/jobs/socure_shadow_mode_proofing_job_spec.rb

+
+      it 'does not log an idv_socure_shadow_mode_proofing_result_missing event' do
+        perform
+        expect(analytics).not_to have_logged_event(:idv_socure_shadow_mode_proofing_result_missing)


Thanks for adding it!

matthinz commented Aug 22, 2024

View reviewed changes

matthinz requested a review from a team August 22, 2024 19:42

zachmargolis reviewed Aug 23, 2024

View reviewed changes

app/jobs/socure_shadow_mode_proofing_job.rb Outdated Show resolved Hide resolved

zachmargolis reviewed Aug 23, 2024

View reviewed changes

app/jobs/socure_shadow_mode_proofing_job.rb Outdated Show resolved Hide resolved

n1zyy reviewed Aug 26, 2024

View reviewed changes

matthinz and others added 10 commits August 26, 2024 14:36

Add Socure configs to IdentityConfig

206a468

Add failure_message_when_negated to HaveLoggedEventMatcher

dc29a66

Add SocureShadowModeProofingJob

8bf0ec4

Add job to make requests to Socure's KYC API and log the results alongside the original resolution proofing result. changelog: Upcoming Features, Identity verification, Add background job for Socure KYC proofing

Make ResolutionProofingJob schedule Socure KYC call

8ebc93a

- When flag is enabled, invoke Socure KYC as well

Add verified_attributes to resolution result to_h

05697ad

Add more detail to resolution proofer logging test

5755551

Don't log TMX response body ong idv_socure_shadow_mode_proofing_result

46d1c97

These are real big and mess with Cloudwatch's ability to parse fields out oflogs.

Tweak socure default base URL for dev

fab11b8

Remove pointless user_id arg to analytics event

dbaf5a8

Update app/jobs/socure_shadow_mode_proofing_job.rb

181ca80

Co-authored-by: Zach Margolis <[email protected]>

matthinz force-pushed the matthinz/13963-socure-shadow-mode branch from 5274082 to 181ca80 Compare August 27, 2024 03:10

matthinz added 4 commits August 26, 2024 20:13

Clarify comment in spec

c4fbd40

Clarify spec name

cb276e6

Clarify spec name

91b1736

Merge remote-tracking branch 'origin/main' into matthinz/13963-socure…

3130112

…-shadow-mode

jmhooper approved these changes Aug 28, 2024

View reviewed changes

matthinz added 4 commits August 28, 2024 09:08

Merge branch 'main' into matthinz/13963-socure-shadow-mode

4ebb7cd

Use user.first_email

fd4dcab

Revert "Use user.first_email"

87a34bf

This reverts commit fd4dcab.

Merge remote-tracking branch 'origin/main' into matthinz/13963-socure…

ca499e4

…-shadow-mode

eileen-nava reviewed Aug 28, 2024

View reviewed changes

Merge branch 'main' into matthinz/13963-socure-shadow-mode

e829c13

Remove pointless service_provider_issuer let

6af08a8

matthinz merged commit a52223a into main Aug 30, 2024
2 checks passed

matthinz deleted the matthinz/13963-socure-shadow-mode branch August 30, 2024 23:02

jmdembe mentioned this pull request Sep 3, 2024

Deploy RC 411 to Production #11188

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

LG-13963: 👤 Socure shadow mode background job #11139

LG-13963: 👤 Socure shadow mode background job #11139

matthinz commented Aug 22, 2024 •

edited

Loading

matthinz Aug 22, 2024

zachmargolis Aug 23, 2024

matthinz Aug 26, 2024

zachmargolis Aug 27, 2024

matthinz Aug 27, 2024

matthinz Aug 27, 2024

zachmargolis Aug 27, 2024

matthinz Aug 22, 2024

matthinz Aug 22, 2024

eileen-nava Aug 28, 2024

matthinz Aug 22, 2024

n1zyy left a comment

n1zyy Aug 26, 2024

zachmargolis Aug 27, 2024

eileen-nava left a comment

eileen-nava Aug 28, 2024

eileen-nava Aug 28, 2024

LG-13963: 👤 Socure shadow mode background job #11139

LG-13963: 👤 Socure shadow mode background job #11139

Conversation

matthinz commented Aug 22, 2024 • edited Loading

🎫 Ticket

🛠 Summary of changes

📜 Testing Plan

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

n1zyy left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

eileen-nava left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

matthinz commented Aug 22, 2024 •

edited

Loading