K8s feb 28 (#15715)

* Resolve bundle-audit error (#15704) * Change rack-cors version to fix bundle-audit error * Add ignore flag to bundle audit * Restore rack cors version 2.0.1 * Update bundle-audit * Fix Gemfile.lock (#15700) * remove x86_64-darwin-22 using bundle lock --remove-platform command * add ruby using bundle lock --add-platform command * Dbex user identifiers status check (#14858) * Include Form526 user identifier presence mapping in user profile Includes the mapping of required Form526 user identifier data generated by the Users::Form526UserIdentifiersStatusService in the user profile returned from the main user endpoint. The front end will use this information to display to the user any identifiers we don't have in our system, so they can pass that information on to the Contact Center. Adds a feature flag, form_526_required_identifiers_in_user_object, and gates the presence of this data in the profile. * Update User LOA JSON Schema validations for 526 identifiers Updates four JSON schema validation files to relfect the addition of the form526_required_identifier_presence key to the users profile metadata * Add user arg to feature flags Forgot to pass the user arg to this feature flag. We want to allow this level of control so we could release this change progressively if we wanted to * Satisfy outstanding Rubocop warning to use anonymous positional arguments forwarding * Fix Flipper disable in profile test The user argument doesn't seem to work properly when enabling and disabling Flippers in specs. Confident this is working properly in the actual codebase though. * Address style suggestion in metadata conditional Prefer unless statement and early return to if statment in this case --------- Co-authored-by: Eric Boehs <[email protected]> * Adds count of records written to log message (#15705) * Use LOA2 instead of LOA3 for datebox timestamp logic (#15655) * revert(vaos): removed clinic stop code logging (#15693) va.gov-team#71272 * Filter Unwanted Errors from Retrying (Dependents Forms) (#15545) * Add filters from exception cause * Use sidekiq skip mechanism * Use common method to submit backup * Move filter handling to method * Params for filter method * Lint issues * Fix icn as instance var * Missed 686c param * Move salvage form to top of rescue block * Same for other job * Remove code checks * updated cve file in build workflow --------- Co-authored-by: Holden Hinkle <[email protected]> Co-authored-by: Nathan Burgess <[email protected]> Co-authored-by: Eric Boehs <[email protected]> Co-authored-by: gia-lexa <[email protected]> Co-authored-by: Eric Tillberg <[email protected]> Co-authored-by: AJ Magdub <[email protected]> Co-authored-by: Tyler Fink <[email protected]> Co-authored-by: RachalCassity <[email protected]>
department-of-veterans-affairs · Feb 28, 2024 · 96e2381 · 96e2381
1 parent c21bf42
commit 96e2381
Show file tree

Hide file tree

Showing 20 changed files with 346 additions and 75 deletions.
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
@@ -30,7 +30,7 @@ jobs:
           bundler-cache: true
 
       - name: Run bundle-audit (checks gems for CVE issues)
-        run:  bundle exec bundle-audit check --update --ignore CVE-2023-26141 CVE-2023-49090
+        run:  bundle exec bundle-audit check --update --ignore CVE-2024-27456
 
       - name: Run Rubocop
         run: bundle exec rubocop --parallel --format github

diff --git a/app/services/users/form_526_user_identifiers_status_service.rb b/app/services/users/form_526_user_identifiers_status_service.rb
@@ -0,0 +1,27 @@
+# frozen_string_literal: true
+
+# For a given user, checks for the presence of certain identifiers required by Form526
+# Returns a mapping of the identifier name to a boolean indicating whether we have that information for a user or not
+module Users
+  class Form526UserIdentifiersStatusService
+    FORM526_REQUIRED_IDENTIFIERS = %w[participant_id birls_id ssn birth_date edipi].freeze
+
+    def self.call(*)
+      new(*).call
+    end
+
+    def initialize(user)
+      @user = user
+    end
+
+    def call
+      identifer_mapping
+    end
+
+    private
+
+    def identifer_mapping
+      FORM526_REQUIRED_IDENTIFIERS.index_with { |identifier| @user[identifier].present? }
+    end
+  end
+end
diff --git a/app/services/users/profile.rb b/app/services/users/profile.rb
@@ -96,11 +96,18 @@ def claims
           military_history: Vet360Policy.new(user).military_access?,
           payment_history: BGSPolicy.new(user).access?(log_stats: false),
           personal_information: MPIPolicy.new(user).queryable?,
-          rating_info: LighthousePolicy.new(user).rating_info_access?
+          rating_info: LighthousePolicy.new(user).rating_info_access?,
+          **form_526_required_identifiers
         }
       end
     end
 
+    def form_526_required_identifiers
+      return {} unless Flipper.enabled?(:form_526_required_identifiers_in_user_object, user)
+
+      { form526_required_identifier_presence: Users::Form526UserIdentifiersStatusService.call(user) }
+    end
+
     def vet360_contact_information
       person = user.vet360_contact_info
       return {} if person.blank?

diff --git a/app/sidekiq/bgs/job.rb b/app/sidekiq/bgs/job.rb
@@ -5,6 +5,14 @@
 module BGS
   class Job
     include BGS::Utilities::Helpers
+    FILTERED_ERRORS = [
+      'insertBenefitClaim: Invalid zipcode.',
+      'Maximum number of EPs reached for this bnftClaimTypeCd',
+      'This update is being elevated for additional review due to an Incident Flash associated with this Beneficiary',
+      'ORA-20099: Error - File Number and Social Security number are different',
+      'ORA-00001: unique constraint',
+      'The length of the EXTERNAL_UID or EXTERNAL_KEY exceeds the maximum'
+    ].freeze
 
     def in_progress_form_copy(in_progress_form)
       return nil if in_progress_form.blank?

diff --git a/app/sidekiq/bgs/submit_form674_job.rb b/app/sidekiq/bgs/submit_form674_job.rb
@@ -9,7 +9,7 @@ class Invalid674Claim < StandardError; end
     include Sidekiq::Job
     include SentryLogging
 
-    attr_reader :claim, :user, :in_progress_copy, :user_uuid, :saved_claim_id, :vet_info
+    attr_reader :claim, :user, :in_progress_copy, :user_uuid, :saved_claim_id, :vet_info, :icn
 
     sidekiq_options retry: 14
 
@@ -18,42 +18,52 @@ class Invalid674Claim < StandardError; end
       vet_info = JSON.parse(KmsEncrypted::Box.new.decrypt(encrypted_vet_info))
       Rails.logger.error('BGS::SubmitForm674Job failed, retries exhausted...',
                          { user_uuid:, saved_claim_id:, icn:, error: })
-      if Flipper.enabled?(:dependents_central_submission)
-        user ||= BGS::SubmitForm674Job.generate_user_struct(encrypted_user_struct_hash, vet_info)
-        CentralMail::SubmitCentralForm686cJob.perform_async(saved_claim_id,
-                                                            KmsEncrypted::Box.new.encrypt(vet_info.to_json),
-                                                            KmsEncrypted::Box.new.encrypt(user.to_h.to_json))
-      else
-        DependentsApplicationFailureMailer.build(user).deliver_now if user&.email.present? # rubocop:disable Style/IfInsideElse # Temporary for flipper
-      end
+
+      BGS::SubmitForm674Job.send_backup_submission(encrypted_user_struct_hash, vet_info, saved_claim_id)
     end
 
     def perform(user_uuid, icn, saved_claim_id, encrypted_vet_info, encrypted_user_struct_hash = nil)
-      @vet_info = JSON.parse(KmsEncrypted::Box.new.decrypt(encrypted_vet_info))
       Rails.logger.info('BGS::SubmitForm674Job running!', { user_uuid:, saved_claim_id:, icn: })
-
-      @user = BGS::SubmitForm674Job.generate_user_struct(encrypted_user_struct_hash, @vet_info)
-      @user_uuid = user_uuid
-      @saved_claim_id = saved_claim_id
+      instance_params(encrypted_vet_info, icn, encrypted_user_struct_hash, user_uuid, saved_claim_id)
 
       in_progress_form = InProgressForm.find_by(form_id: FORM_ID, user_uuid:)
       @in_progress_copy = in_progress_form_copy(in_progress_form)
 
-      claim_data = normalize_names_and_addresses!(valid_claim_data)
-
-      BGS::Form674.new(user, claim).submit(claim_data)
+      submit_form
 
       send_confirmation_email
       in_progress_form&.destroy
       Rails.logger.info('BGS::SubmitForm674Job succeeded!', { user_uuid:, saved_claim_id:, icn: })
     rescue => e
+      salvage_save_in_progress_form(FORM_ID, user_uuid, @in_progress_copy) if @in_progress_copy.present?
+      handle_filtered_errors!(e:, encrypted_user_struct_hash:, encrypted_vet_info:)
+
       Rails.logger.warn('BGS::SubmitForm674Job received error, retrying...',
-                        { user_uuid:, saved_claim_id:, icn:, error: e.message })
+                        { user_uuid:, saved_claim_id:, icn:, error: e.message, nested_error: e.cause&.message })
       log_message_to_sentry(e, :warning, {}, { team: 'vfs-ebenefits' })
-      salvage_save_in_progress_form(FORM_ID, user_uuid, @in_progress_copy) if @in_progress_copy.present?
       raise
     end
 
+    def handle_filtered_errors!(e:, encrypted_user_struct_hash:, encrypted_vet_info:)
+      filter = FILTERED_ERRORS.any? { |filtered| e.message.include?(filtered) || e.cause&.message&.include?(filtered) }
+      return unless filter
+
+      Rails.logger.warn('BGS::SubmitForm674Job received error, skipping retries...',
+                        { user_uuid:, saved_claim_id:, icn:, error: e.message, nested_error: e.cause&.message })
+
+      vet_info = JSON.parse(KmsEncrypted::Box.new.decrypt(encrypted_vet_info))
+      self.class.send_backup_submission(encrypted_user_struct_hash, vet_info, saved_claim_id)
+      raise Sidekiq::JobRetry::Skip
+    end
+
+    def instance_params(encrypted_vet_info, icn, encrypted_user_struct_hash, user_uuid, saved_claim_id)
+      @vet_info = JSON.parse(KmsEncrypted::Box.new.decrypt(encrypted_vet_info))
+      @user = BGS::SubmitForm674Job.generate_user_struct(encrypted_user_struct_hash, @vet_info)
+      @icn = icn
+      @user_uuid = user_uuid
+      @saved_claim_id = saved_claim_id
+    end
+
     def self.generate_user_struct(encrypted_user_struct, vet_info)
       if encrypted_user_struct.present?
         return OpenStruct.new(JSON.parse(KmsEncrypted::Box.new.decrypt(encrypted_user_struct)))
@@ -75,16 +85,29 @@ def self.generate_user_struct(encrypted_user_struct, vet_info)
       )
     end
 
+    def self.send_backup_submission(encrypted_user_struct_hash, vet_info, saved_claim_id)
+      if Flipper.enabled?(:dependents_central_submission)
+        user = generate_user_struct(encrypted_user_struct_hash, vet_info)
+        CentralMail::SubmitCentralForm686cJob.perform_async(saved_claim_id,
+                                                            KmsEncrypted::Box.new.encrypt(vet_info.to_json),
+                                                            KmsEncrypted::Box.new.encrypt(user.to_h.to_json))
+      else
+        DependentsApplicationFailureMailer.build(user).deliver_now if user&.email.present? # rubocop:disable Style/IfInsideElse # Temporary for flipper
+      end
+    end
+
     private
 
-    def valid_claim_data
+    def submit_form
       @claim = SavedClaim::DependencyClaim.find(saved_claim_id)
 
       claim.add_veteran_info(vet_info)
 
       raise Invalid674Claim unless claim.valid?(:run_686_form_jobs)
 
-      claim.formatted_674_data(vet_info)
+      claim_data = normalize_names_and_addresses!(claim.formatted_674_data(vet_info))
+
+      BGS::Form674.new(user, claim).submit(claim_data)
     end
 
     def send_confirmation_email

diff --git a/app/sidekiq/bgs/submit_form686c_job.rb b/app/sidekiq/bgs/submit_form686c_job.rb
@@ -9,7 +9,7 @@ class Invalid686cClaim < StandardError; end
     include Sidekiq::Job
     include SentryLogging
 
-    attr_reader :claim, :user, :in_progress_copy, :user_uuid, :saved_claim_id, :vet_info
+    attr_reader :claim, :user, :in_progress_copy, :user_uuid, :saved_claim_id, :vet_info, :icn
 
     sidekiq_options retry: 14
 
@@ -18,45 +18,52 @@ class Invalid686cClaim < StandardError; end
       vet_info = JSON.parse(KmsEncrypted::Box.new.decrypt(encrypted_vet_info))
       Rails.logger.error('BGS::SubmitForm686cJob failed, retries exhausted!',
                          { user_uuid:, saved_claim_id:, icn:, error: })
-      if Flipper.enabled?(:dependents_central_submission)
-        user ||= BGS::SubmitForm686cJob.generate_user_struct(vet_info)
-        CentralMail::SubmitCentralForm686cJob.perform_async(saved_claim_id,
-                                                            KmsEncrypted::Box.new.encrypt(vet_info.to_json),
-                                                            KmsEncrypted::Box.new.encrypt(user.to_h.to_json))
-      else
-        DependentsApplicationFailureMailer.build(user).deliver_now if user&.email.present? # rubocop:disable Style/IfInsideElse
-      end
+
+      BGS::SubmitForm686cJob.send_backup_submission(vet_info, saved_claim_id)
     end
 
     def perform(user_uuid, icn, saved_claim_id, encrypted_vet_info)
-      @vet_info = JSON.parse(KmsEncrypted::Box.new.decrypt(encrypted_vet_info))
       Rails.logger.info('BGS::SubmitForm686cJob running!', { user_uuid:, saved_claim_id:, icn: })
-
-      @user = BGS::SubmitForm686cJob.generate_user_struct(@vet_info)
-      @user_uuid = user_uuid
-      @saved_claim_id = saved_claim_id
+      instance_params(encrypted_vet_info, icn, user_uuid, saved_claim_id)
 
       in_progress_form = InProgressForm.find_by(form_id: FORM_ID, user_uuid:)
       @in_progress_copy = in_progress_form_copy(in_progress_form)
 
-      claim_data = normalize_names_and_addresses!(valid_claim_data)
-
-      BGS::Form686c.new(user, claim).submit(claim_data)
-
-      # If Form 686c job succeeds, then enqueue 674 job.
-      BGS::SubmitForm674Job.perform_async(user_uuid, icn, saved_claim_id, encrypted_vet_info, KmsEncrypted::Box.new.encrypt(user.to_h.to_json)) if claim.submittable_674? # rubocop:disable Layout/LineLength
+      submit_forms(encrypted_vet_info)
 
       send_confirmation_email
       in_progress_form&.destroy
       Rails.logger.info('BGS::SubmitForm686cJob succeeded!', { user_uuid:, saved_claim_id:, icn: })
     rescue => e
+      salvage_save_in_progress_form(FORM_ID, user_uuid, @in_progress_copy) if @in_progress_copy.present?
+      handle_filtered_errors!(e:, encrypted_vet_info:)
+
       Rails.logger.warn('BGS::SubmitForm686cJob received error, retrying...',
-                        { user_uuid:, saved_claim_id:, icn:, error: e.message })
+                        { user_uuid:, saved_claim_id:, icn:, error: e.message, nested_error: e.cause&.message })
       log_message_to_sentry(e, :warning, {}, { team: 'vfs-ebenefits' })
-      salvage_save_in_progress_form(FORM_ID, user_uuid, @in_progress_copy) if @in_progress_copy.present?
       raise
     end
 
+    def handle_filtered_errors!(e:, encrypted_vet_info:)
+      filter = FILTERED_ERRORS.any? { |filtered| e.message.include?(filtered) || e.cause&.message&.include?(filtered) }
+      return unless filter
+
+      Rails.logger.warn('BGS::SubmitForm686cJob received error, skipping retries...',
+                        { user_uuid:, saved_claim_id:, icn:, error: e.message, nested_error: e.cause&.message })
+
+      vet_info = JSON.parse(KmsEncrypted::Box.new.decrypt(encrypted_vet_info))
+      self.class.send_backup_submission(vet_info, saved_claim_id)
+      raise Sidekiq::JobRetry::Skip
+    end
+
+    def instance_params(encrypted_vet_info, icn, user_uuid, saved_claim_id)
+      @vet_info = JSON.parse(KmsEncrypted::Box.new.decrypt(encrypted_vet_info))
+      @user = BGS::SubmitForm686cJob.generate_user_struct(@vet_info)
+      @icn = icn
+      @user_uuid = user_uuid
+      @saved_claim_id = saved_claim_id
+    end
+
     def self.generate_user_struct(vet_info)
       info = vet_info['veteran_information']
       full_name = info['full_name']
@@ -74,16 +81,32 @@ def self.generate_user_struct(vet_info)
       )
     end
 
+    def self.send_backup_submission(vet_info, saved_claim_id)
+      if Flipper.enabled?(:dependents_central_submission)
+        user = generate_user_struct(vet_info)
+        CentralMail::SubmitCentralForm686cJob.perform_async(saved_claim_id,
+                                                            KmsEncrypted::Box.new.encrypt(vet_info.to_json),
+                                                            KmsEncrypted::Box.new.encrypt(user.to_h.to_json))
+      else
+        DependentsApplicationFailureMailer.build(user).deliver_now if user&.email.present? # rubocop:disable Style/IfInsideElse
+      end
+    end
+
     private
 
-    def valid_claim_data
+    def submit_forms(encrypted_vet_info)
       @claim = SavedClaim::DependencyClaim.find(saved_claim_id)
 
       claim.add_veteran_info(vet_info)
 
       raise Invalid686cClaim unless claim.valid?(:run_686_form_jobs)
 
-      claim.formatted_686_data(vet_info)
+      claim_data = normalize_names_and_addresses!(claim.formatted_686_data(vet_info))
+
+      BGS::Form686c.new(user, claim).submit(claim_data)
+
+      # If Form 686c job succeeds, then enqueue 674 job.
+      BGS::SubmitForm674Job.perform_async(user_uuid, icn, saved_claim_id, encrypted_vet_info, KmsEncrypted::Box.new.encrypt(user.to_h.to_json)) if claim.submittable_674? # rubocop:disable Layout/LineLength
     end
 
     def send_confirmation_email

diff --git a/app/sidekiq/education_form/create_daily_spool_files.rb b/app/sidekiq/education_form/create_daily_spool_files.rb
@@ -109,7 +109,7 @@ def write_files(writer, structured_data:)
 
             ## Testing to see if writer is the cause for retry attempt failures
             ## If we get to this message, it's not the writer object
-            log_info("Successfully wrote to filename: #{filename} for region: #{region}")
+            log_info("Successfully wrote #{records.count} applications to filename: #{filename} for region: #{region}")
 
             # send copy of staging spool files to testers
             # This mailer is intended to only work for development, staging and NOT production

diff --git a/config/features.yml b/config/features.yml
@@ -507,6 +507,9 @@ features:
     actor_type: user
     description: Enables form 21-4142 email submission confirmation (VaNotify)
     enable_in_development: true
+  form_526_required_identifiers_in_user_object:
+    actor_type: user
+    description: includes a mapping of booleans in the profile section of a serialized user indicating which ids are nil for the user
   form5490_confirmation_email:
     actor_type: user
     description: Enables form 5490 email submission confirmation (VaNotify)

diff --git a/modules/simple_forms_api/app/services/simple_forms_api/pdf_stamper.rb b/modules/simple_forms_api/app/services/simple_forms_api/pdf_stamper.rb
@@ -19,7 +19,7 @@ def self.stamp_pdf(stamped_template_path, form, current_loa)
       auth_text = case current_loa
                   when 3
                     'Signee signed with an identity-verified account.'
-                  when 1
+                  when 2
                     'Signee signed in but hasn’t verified their identity.'
                   else
                     'Signee not signed in.'

diff --git a/modules/vaos/app/controllers/vaos/v2/clinics_controller.rb b/modules/vaos/app/controllers/vaos/v2/clinics_controller.rb
@@ -11,7 +11,6 @@ def index
                                                         clinical_service: params[:clinical_service],
                                                         page_size: params[:page_size],
                                                         page_number: params[:page_number])
-        log_clinic_info(response)
         render json: VAOS::V2::ClinicsSerializer.new(response)
       end
 
@@ -71,24 +70,6 @@ def unable_to_lookup_clinic?(appt)
         appt.nil? || appt.location_id.nil? || appt.clinic.nil?
       end
 
-      def log_clinic_info(clinic_data)
-        clinic_data.each do |clinic|
-          clinic_info = get_clinic_info(clinic)
-          Rails.logger.info('VAOS Clinic info: ', clinic_info.to_json) unless clinic_info.values.all?(&:nil?)
-        end
-      end
-
-      def get_clinic_info(clinic)
-        {
-          stationId: clinic.station_id,
-          serviceName: clinic.service_name,
-          primaryStopCode: clinic.primary_stop_code,
-          primaryStopCodeName: clinic.primary_stop_code_name,
-          secondaryStopCode: clinic.secondary_stop_code,
-          secondaryStopCodeName: clinic.secondary_stop_code_name
-        }
-      end
-
       def systems_service
         VAOS::V2::SystemsService.new(current_user)
       end

diff --git a/modules/vaos/app/services/vaos/v2/appointments_service.rb b/modules/vaos/app/services/vaos/v2/appointments_service.rb
@@ -12,7 +12,6 @@ class AppointmentsService < VAOS::SessionService
 
       DIRECT_SCHEDULE_ERROR_KEY = 'DirectScheduleError'
       VAOS_SERVICE_DATA_KEY = 'VAOSServiceTypesAndCategory'
-      VAOS_TELEHEALTH_DATA_KEY = 'VAOSTelehealthData'
       FACILITY_ERROR_MSG = 'Error fetching facility details'
       AVS_ERROR_MESSAGE = 'Error retrieving AVS link'
       AVS_APPT_TEST_ID = '192308'

diff --git a/modules/vaos/spec/request/v2/clinics_request_spec.rb b/modules/vaos/spec/request/v2/clinics_request_spec.rb
@@ -20,9 +20,7 @@
       context 'on successful query for clinics given service type' do
         it 'returns a list of clinics' do
           VCR.use_cassette('vaos/v2/systems/get_facility_clinics_200', match_requests_on: %i[method path query]) do
-            allow(Rails.logger).to receive(:info).at_least(:once)
             get '/vaos/v2/locations/983/clinics?clinical_service=audiology', headers: inflection_header
-            expect(Rails.logger).to have_received(:info).with('VAOS Clinic info: ', anything).at_least(:once)
             expect(response).to have_http_status(:ok)
             expect(response.body).to match_camelized_schema('vaos/v2/clinics', { strict: false })
             x = JSON.parse(response.body)