Struggling to toggle form fields with no tag (/T)

[AaronWGoh]

pdf.pdf

Having difficulties allowing checkbox / radiobox toggling with the above file - specifically:
On page 9, there are 4 boxes that are unlabeled. In the above PDF, that is represented as

• Question 4: Have you ever filed a claim with the VA?
• Question 13a: Type of address change
  (There are more per page but this is the simple example)

As noted in a pdf editor, you'll see the 2 checkboxes under these questions are all interactive and toggleable.

and when running

field_name = "RadioButtonList6[0]"
full_field_name = "F[0].Page_10[0].RadioButtonList6[0]"
field = reader.get_fields()[full_field_name]

The field does return as

{'/T': 'RadioButtonList6[0]', '/FT': '/Btn', '/Parent': {'/Kids': [IndirectObject(1039, 0, 4372390928), IndirectObject(1040, 0, 4372390928), IndirectObject(1041, 0, 4372390928), IndirectObject(1042, 0, 4372390928), IndirectObject(1043, 0, 4372390928), IndirectObject(1044, 0, 4372390928), IndirectObject(1045, 0, 4372390928), IndirectObject(1046, 0, 4372390928), IndirectObject(1047, 0, 4372390928), IndirectObject(1048, 0, 4372390928), IndirectObject(1049, 0, 4372390928), IndirectObject(1050, 0, 4372390928), IndirectObject(1051, 0, 4372390928), IndirectObject(1052, 0, 4372390928), IndirectObject(1053, 0, 4372390928), IndirectObject(1054, 0, 4372390928), IndirectObject(1055, 0, 4372390928), IndirectObject(1056, 0, 4372390928), IndirectObject(1057, 0, 4372390928), IndirectObject(1058, 0, 4372390928), IndirectObject(1059, 0, 4372390928), IndirectObject(1060, 0, 4372390928), IndirectObject(1061, 0, 4372390928), IndirectObject(1062, 0, 4372390928), IndirectObject(1063, 0, 4372390928), IndirectObject(1064, 0, 4372390928), IndirectObject(1065, 0, 4372390928), IndirectObject(1066, 0, 4372390928), IndirectObject(1067, 0, 4372390928), IndirectObject(1068, 0, 4372390928), IndirectObject(1069, 0, 4372390928), IndirectObject(1070, 0, 4372390928), IndirectObject(1071, 0, 4372390928), IndirectObject(1072, 0, 4372390928), IndirectObject(1073, 0, 4372390928), IndirectObject(1074, 0, 4372390928), IndirectObject(1075, 0, 4372390928), IndirectObject(1076, 0, 4372390928), IndirectObject(1077, 0, 4372390928), IndirectObject(1078, 0, 4372390928), IndirectObject(1079, 0, 4372390928), IndirectObject(1080, 0, 4372390928), IndirectObject(1081, 0, 4372390928), IndirectObject(1082, 0, 4372390928), IndirectObject(1083, 0, 4372390928), IndirectObject(1084, 0, 4372390928), IndirectObject(1085, 0, 4372390928), IndirectObject(1086, 0, 4372390928)], '/Parent': IndirectObject(752, 0, 4372390928), '/T': 'Page_10[0]'}, '/TU': 'RadioButtonList6', '/Ff': 49152, '/V': '/Off', '/Kids': [IndirectObject(1087, 0, 4372390928), IndirectObject(1088, 0, 4372390928)], '/Opt': ['1', '2'], '/_States_': ['/0', '/1']}

However in,

for page in writer.pages:
    if "/Annots" in page:
        for annot in page["/Annots"]:
            if annot.get("/T") == field_name:
                annot.update(
                    {NameObject("/AS"): new_value, NameObject("/V"): new_value}
                )

The conditional will never be met but

for page in writer.pages:
    if "/Annots" in page:
        for annot in page["/Annots"]:
            if field_name in annot.get("/NM", ""):
                annot.update(
                    {NameObject("/AS"): new_value, NameObject("/V"): new_value}
                )

will hit the condition. The issue with above though is it doesn't reflect the radio button selection in the PDF at all.

new_value = (
    TextStringObject("/0") if field.get("/V") == "/Off" else TextStringObject("/Off")
)

Further, you will find that reader.get_form_text_fields() does not contain the above field key at all (subset or full-qualified name)

I'm starting to get lost as

1. I'm not quite sure the top down structure of the pdf metadata or how the data structure strings together ->
   a) is this a reference issue?
   b) is there something programmatic in the background that is not purely driven by these datastructures?

2. When I check what the field content looks like when I manually select the values in the PDF - the content looks like

{'/Border': [0, 0, 0], '/Rect': [247.524, 465.146, 255.524, 473.146], '/F': 4, '/Subtype': '/Widget', '/TU': 'RadioButtonList2', '/DA': '/Helvetica 12 Tf 0 g', '/MK': IndirectObject(4149, 0, 4334489488), '/AP': IndirectObject(4150, 0, 4334489488), '/H': '/P', '/StructParent': 70, '/AS': '/0', '/Parent': IndirectObject(4138, 0, 4334489488), '/Type': '/Annot', '/Ff': 49152}

whereas the original would be

{'/AP': {'/D': {'/0': IndirectObject(454, 0, 4301853136), '/Off': IndirectObject(453, 0, 4301853136)}, '/N': {'/0': IndirectObject(332, 0, 4301853136)}}, '/AS': '/Off', '/F': 4, '/MK': {'/CA': '8'}, '/NM': 'F[0].Page_10[0].RadioButtonList2[0].Yes1[0]', '/P': IndirectObject(20, 0, 4301853136), '/Parent': IndirectObject(1084, 0, 4301853136), '/Rect': [247.524, 465.146, 255.524, 473.146], '/StructParent': 70, '/Subtype': '/Widget', '/Type': '/Annot'}

Note: the pdf ids in Indirect objects are different but when I did a .get_object() on those instances, it does return the same values underlyingly - it's just a new id per run when doing fixes.

Another point of context is that these radioboxes / checkboxes are unique in this structure where they have no /T during the annotation iteration. Additionally, the output of get_form_text_fields() is which shows most of the fields function as expected.

{'VA_File_Number[0]': None, 'Email_Address_Optional[0]': None, 'Veteran_Service_Member_First_Name[0]': None, 'Veteran_Service_Member_Last_Name[0]': None, 'Veteran_Service_Member_Middle_Initial[0]': None, 'SocialSecurityNumber_FirstThreeNumbers[0]': None, 'SocialSecurityNumber_SecondTwoNumbers[0]': None, 'SocialSecurityNumber_LastFourNumbers[0]': None, 'Date_Of_Birth_Month[0]': None, 'Date_Of_Birth_Day[0]': None, 'Date_Of_Birth_Year[0]': None, 'CurrentMailingAddress_NumberAndStreet[0]': None, 'CurrentMailingAddress_ApartmentOrUnitNumber[0]': None, 'CurrentMailingAddress_City[0]': None, 'CurrentMailingAddress_StateOrProvince[0]': None, 'CurrentMailingAddress_Country[0]': None, 'CurrentMailingAddress_ZIPOrPostalCode_FirstFiveNumbers[0]': None, 'CurrentMailingAddress_ZIPOrPostalCode_LastFourNumbers[0]': None, 'New_Address_NumberAndStreet[0]': None, 'New_Address_ApartmentOrUnitNumber[0]': None, 'New_Address_City[0]': None, 'New_Address_StateOrProvince[0]': None, 'New_Address_Country[0]': None, 'New_Address_ZIPOrPostalCode_FirstFiveNumbers[0]': None, 'New_Address_ZIPOrPostalCode_LastFourNumbers[0]': None, 'Veterans_Service_Number_If_Applicable[0]': None, 'BDD_Claims_Only__Provide_The_Date_Or_Anticipated_Date_Of_Release_From_Active_Duty_Month[0]': None, 'BDD_Claims_Only_Provide_The_Date_Or_Anticipated_Date_Of_Release_From_Active_Duty_Year[0]': None, 'Beginning_Date_Month[0]': None, 'Beginning_Date_Day[0]': None, 'Beginning_Date_Year[0]': None, 'Ending_Date_Month[0]': None, 'Ending_Date_Day[0]': None, 'Ending_Date_Year[0]': None, 'Daytime_Phone_Number_Area_Code[0]': None, 'Telephone_Middle_Three_Numbers[0]': None, 'Telephone_Last_Four_Numbers[0]': None, 'BDD_Claims_Only__Provide_The_Date_Or_Anticipated_Date_Of_Release_From_Active_Duty_Day[0]': None, 'International_Telephone_Number_If_Applicable[0]': None, 'Email_Address_Optional[1]': None, 'SocialSecurityNumber_FirstThreeNumbers[0].2': None, 'SocialSecurityNumber_SecondTwoNumbers[0].2': None, 'SocialSecurityNumber_LastFourNumbers[0].2': None, 'Telephone_Number_Area_Code[0]': None, 'Telephone_Middle_Three_Numbers[0].2': None, 'Telephone_Last_Four_Numbers[0].2': None, 'International_Telephone_Number_If_Applicable[0].2': None, 'From_Date_Month[0]': None, 'From_Date_Year[0]': None, 'List_Other_Locations_Where_You_Served[0]': None, 'Provide_Additional_Dates_And_Locations_Of_Potential_Exposure[0]': None, 'POINT_OF_CONTACT[0]': None, 'To_Date_Year[0]': None, 'To_Date_Month[0]': None, 'From_Date_Month[1]': None, 'From_Date_Year[1]': None, 'To_Date_Year[1]': None, 'To_Date_Month[1]': None, 'From_Date_Month[2]': None, 'From_Date_Year[2]': None, 'To_Date_Year[2]': None, 'To_Date_Month[2]': None, 'OTHER_SPECIFY[0]': None, 'OTHER_SPECIFY[1]': None, 'Other_Specify[0]': None, 'SocialSecurityNumber_FirstThreeNumbers[1]': None, 'SocialSecurityNumber_SecondTwoNumbers[1]': None, 'SocialSecurityNumber_LastFourNumbers[1]': None, 'CURRENTDISABILITY[0]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[0]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[0]': None, 'Date_Disability_Began_Or_Worsened[0]': None, 'Date12[0]': None, 'Date12[1]': None, 'Date12[2]': None, 'Date12[3]': None, 'Date12[4]': None, 'Date12[5]': None, 'Date12[6]': None, 'Date12[7]': None, 'Date12[8]': None, 'Date12[9]': None, 'Date12[10]': None, 'Date12[11]': None, 'Date12[12]': None, 'Date12[13]': None, 'CURRENTDISABILITY[1]': None, 'CURRENTDISABILITY[2]': None, 'CURRENTDISABILITY[3]': None, 'CURRENTDISABILITY[4]': None, 'CURRENTDISABILITY[5]': None, 'CURRENTDISABILITY[6]': None, 'CURRENTDISABILITY[7]': None, 'CURRENTDISABILITY[8]': None, 'CURRENTDISABILITY[9]': None, 'CURRENTDISABILITY[10]': None, 'CURRENTDISABILITY[11]': None, 'CURRENTDISABILITY[12]': None, 'CURRENTDISABILITY[13]': None, 'CURRENTDISABILITY[14]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[1]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[2]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[3]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[4]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[5]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[6]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[7]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[8]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[9]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[10]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[11]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[12]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[13]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[14]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[1]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[2]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[3]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[4]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[5]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[6]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[7]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[8]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[9]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[10]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[11]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[12]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[13]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[14]': None, 'Date_Of_Treatment_Year[0]': None, 'Date_Of_Treatment_Month[0]': None, 'Date_Of_Treatment_Year[1]': None, 'Date_Of_Treatment_Month[1]': None, 'Date_Of_Treatment_Year[2]': None, 'Date_Of_Treatment_Month[2]': None, 'Enter_Disability_Treated_And_Name_And_Location_Of_Treatment_Facility[0]': None, 'Enter_Disability_Treated_And_Name_And_Location_Of_Treatment_Facility[1]': None, 'Enter_Disability_Treated_And_Name_And_Location_Of_Treatment_Facility[2]': None, 'SocialSecurityNumber_FirstThreeNumbers[2]': None, 'SocialSecurityNumber_SecondTwoNumbers[2]': None, 'SocialSecurityNumber_LastFourNumbers[2]': None, 'List_Other_Name_You_Served_Under[0]': None, 'EntryDate_Month[0]': None, 'MostRecentActiveServiceEntryDate_Day[0]': None, 'EntryDate_Year[0]': None, 'ObligationTermOfService_Month[0]': None, 'ObligationTermOfService_Day[0]': None, 'ObligationTermOfService_Year[0]': None, 'ObligationTermOfService_Month[1]': None, 'ObligationTermOfService_Day[1]': None, 'ObligationTermOfService_Year[1]': None, 'DateOfActivation_Month[0]': None, 'DateOfActivation_Day[0]': None, 'DateOfActivation_Year[0]': None, 'AnticipatedSeparationDate_Month[0]': None, 'AnticipatedSeparationDate_Day[0]': None, 'AnticipatedSeparationDate_Year[0]': None, 'DatesOfConfinement_Month[0]': None, 'DatesOfConfinement_Day[0]': None, 'DatesOfConfinement_Year[0]': None, 'DatesOfConfinement_Month[1]': None, 'DatesOfConfinement_Day[1]': None, 'DatesOfConfinement_Year[1]': None, 'DatesOfConfinement_Month[2]': None, 'DatesOfConfinement_Day[2]': None, 'DatesOfConfinement_Year[2]': None, 'DatesOfConfinement_Month[3]': None, 'DatesOfConfinement_Day[3]': None, 'DatesOfConfinement_Year[3]': None, 'PlaceOfLastOrAnticipatedSeparation[0]': None, 'PlaceOfLastOrAnticipatedSeparation[1]': None, 'ExitDate_Month[0]': None, 'ExitDate_Day[0]': None, 'ExitDate_Year[0]': None, 'AdditionalPeriodsOfService_Month[0]': None, 'AdditionalPeriodsOfService_Day[0]': None, 'AdditionalPeriodsOfService_Year[0]': None, 'AdditionalPeriodsOfService_Month[1]': None, 'AdditionalPeriodsOfService_Day[1]': None, 'AdditionalPeriodsOfService_Year[1]': None, 'Monthly_Amount[0]': None, 'Monthly_Amount[1]': None, 'CURRENT_OR_LAST_ASSIGNED_NAME_AND_ADDRESS_OF_UNIT[0]': None, 'Current_Or_Assigned_Phone_Number_Of_Unit_Include_Area_Code[0]': None, 'EXPLAIN[0]': None, 'Routing_Or_Transit_Number[0]': None, 'Account_Number[0]': None, 'DatePaymentReceived_Month[0]': None, 'DatePaymentReceived_Day[0]': None, 'DatePaymentReceived_Year[0]': None, 'Amount_Received_Last_Three_Digits[0]': None, 'Amount_Received_First_Three_Digits[0]': None, 'SocialSecurityNumber_FirstThreeNumbers[3]': None, 'SocialSecurityNumber_SecondTwoNumbers[3]': None, 'SocialSecurityNumber_LastFourNumbers[3]': None, 'Date_Signed_Month[0]': None, 'Date_Signed_Day[0]': None, 'Date_Signed_Year[0]': None, 'Print_Name_And_Address_Of_Witness[0]': None, 'Print_Name_And_Address_Of_Witness[1]': None, 'NAME_OF_FINANCIAL_INSTITUTION[0]': None, 'Date_Signed_Year[1]': None, 'Date_Signed_Day[1]': None, 'Date_Signed_Month[1]': None, 'Date_Signed_Month[2]': None, 'Date_Signed_Day[2]': None, 'Date_Signed_Year[2]': None, 'SocialSecurityNumber_FirstThreeNumbers[4]': None, 'SocialSecurityNumber_SecondTwoNumbers[4]': None, 'SocialSecurityNumber_LastFourNumbers[4]': None, 'SocialSecurityNumber_FirstThreeNumbers[5]': None, 'SocialSecurityNumber_SecondTwoNumbers[5]': None, 'SocialSecurityNumber_LastFourNumbers[5]': None, 'CURRENTDISABILITY[15]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[15]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[15]': None, 'Date_Disability_Began_Or_Worsened[1]': None, 'Date12[14]': None, 'Date12[15]': None, 'Date12[16]': None, 'Date12[17]': None, 'Date12[18]': None, 'Date12[19]': None, 'Date12[20]': None, 'Date12[21]': None, 'Date12[22]': None, 'Date12[23]': None, 'Date12[24]': None, 'Date12[25]': None, 'Date12[26]': None, 'Date12[27]': None, 'CURRENTDISABILITY[16]': None, 'CURRENTDISABILITY[17]': None, 'CURRENTDISABILITY[18]': None, 'CURRENTDISABILITY[19]': None, 'CURRENTDISABILITY[20]': None, 'CURRENTDISABILITY[21]': None, 'CURRENTDISABILITY[22]': None, 'CURRENTDISABILITY[23]': None, 'CURRENTDISABILITY[24]': None, 'CURRENTDISABILITY[25]': None, 'CURRENTDISABILITY[26]': None, 'CURRENTDISABILITY[27]': None, 'CURRENTDISABILITY[28]': None, 'CURRENTDISABILITY[29]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[16]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[17]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[18]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[19]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[20]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[21]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[22]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[23]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[24]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[25]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[26]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[27]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[28]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[29]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[16]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[17]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[18]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[19]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[20]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[21]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[22]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[23]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[24]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[25]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[26]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[27]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[28]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[29]': None, 'Date12[28]': None, 'Date12[29]': None, 'Date12[30]': None, 'Date12[31]': None, 'Date12[32]': None, 'CURRENTDISABILITY[30]': None, 'CURRENTDISABILITY[31]': None, 'CURRENTDISABILITY[32]': None, 'CURRENTDISABILITY[33]': None, 'CURRENTDISABILITY[34]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[30]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[31]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[32]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[33]': None, 'Specify_Type_Of_Exposure_Event_Or_Injury[34]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[30]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[31]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[32]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[33]': None, 'ExplainHowDisabilityRelatesToEvent_Exposure_Injury[34]': None}

Some of the above are checkboxes where 1 has been confirmed to work without issue for validating truth. It seems like this is likely some phenomena with the grouping structure.

Would greatly appreciate any help!

[pubpub-zz]

NM stores an id which is a key to look for objects. It is most of time (but not here) filled with a GUID
when you get the fields with get_fields() you are actually parsing the objects in the tree ["/Root"]["/AcroForm"]["/Fields"] with all branches
The actual name is build joining the names of all the fields to reach the final object, howver the final object is not the ones you are getting in the ["/Annots"] of the page(s). In here you are finding the annotations that are providing the display of the entry. for Convenience Pdf Standard offers to possibility to use the same object as the actual fields (which are actually containing the entries for the field as described in §8.6.2 of pdf reference 1.7) and the entries of the (display) annotation (described in §8.4). as an example of such field/annot look at the first annot in page 9. Note that its qualified name is 'F[0].Page_10[0].VA_File_Number[0]' which is not directly visible in any entry.
For ["/Annots"][48] and ["/Annots"][49] which have no "/T" they are presenting 2 "displays" of the same entry and the actual field object is their parent : [48] shows a checked box if the field is set to '0' and [49] for '1'.
The associated field 'F[0].Page_10[0].RadioButtonList6[0]' being a choice field, it is not considered as text and not returned by get_form_text_fields() but is available in get_fields()

[AaronWGoh]

gotcha, appreciate the reference to the standards manual - will take a closer look to better understand the underlying mechanics for pdf data storage

Though on a practical level, how would I update the values in the get_fields() appropriately? It seems like the update_form_field_values method works not on the fully qualified name but based on the values from the form_text_fields unless I'm making an error somewhere else

[pubpub-zz]

From the top of my head update_form_fields_values() works with both qualified (full) or 'local' names(stored on '/T'). I recommend to use qualified names as keyed in get_fields()