Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Publisher] Add "Other" textbox to race breakdown #1557

Merged
merged 9 commits into from
Jan 16, 2025

Conversation

nasaownsky
Copy link
Collaborator

Description of the change

Added "Other" textbox to race breakdown

Type of change

All pull requests must have at least one of the following labels applied (otherwise the PR will fail):

Label Description
Type: Bug non-breaking change that fixes an issue
Type: Feature non-breaking change that adds functionality
Type: Breaking Change fix or feature that would cause existing functionality to not work as expected
Type: Non-breaking refactor change addresses some tech debt item or prepares for a later change, but does not change functionality
Type: Configuration Change adjusts configuration to achieve some end related to functionality, development, performance, or security
Type: Dependency Upgrade upgrades a project dependency - these changes are not included in release notes

Related issues

closes #1491

Checklists

Development

This box MUST be checked by the submitter prior to merging:

  • Double- and triple-checked that there is no Personally Identifiable Information (PII) being mistakenly added in this pull request

These boxes should be checked by the submitter prior to merging:

  • Tests have been written to cover the code changed/added as part of this pull request

Code review

These boxes should be checked by reviewers prior to merging:

  • This pull request has a descriptive title and information useful to a reviewer
  • This pull request has been moved out of a Draft state, has no "Work In Progress" label, and has assigned reviewers
  • Potential security implications or infrastructural changes have been considered, if relevant

@nasaownsky
Copy link
Collaborator Author

Hi @mxosman! So after playing with this around, I think maybe injecting component into CheckboxOptions not the best idea? In this case we can place textbox either before checkbox or after it, and I personally think the before solution is better, because it will always be visible to the user. What do you think?

Also, how do you like my implementation of other_description into updatedDimensions? Let me know if it suits our needs for this!

And last one: I was thinking about displaying existing other_description in textbox, and noticed that if we selecting "Yes" in Are you able to record an individual’s ethnicity... section -- all of the Other Ethnicities are selected, and if selecting "No" -- only Other Unknown Ethnicity is selected. Am I understanding correctly that their other_description should be the same after all for all ethnicities?

@mxosman
Copy link
Contributor

mxosman commented Oct 18, 2024

Hi @mxosman! So after playing with this around, I think maybe injecting component into CheckboxOptions not the best idea?

Ah, do you mind clarifying your thoughts on this? Is it moreso challenging to refactor the component to inject a text input? I could see us doing this more often - but just wanted to understand the challenges you ran into!

In this case we can place textbox either before checkbox or after it, and I personally think the before solution is better, because it will always be visible to the user. What do you think?

Hmm... the biggest thing I don't like about the textbox before is the big distance between 'Other' and the textbox - it makes it feel less intuitive what that input is connected to, despite the description in the box. Ideally, it makes the most sense from a UX perspective that it happens right after the 'Other' checkbox, but if that's too technically clunky, the next best option IMO is below the checkboxes because it keeps it within the closest proximity to the 'Other' checkbox. To help with it always being visible and to enhance its connection to the 'Other' checkbox, we could add a scrollTo reaction that scrolls to the bottom of the container when 'Other' is checked.

E.g. -

  • Create a ref on the ScrollableInnerWrapper
       <Styled.ScrollableInnerWrapper ref={containerRef}>
  • Move the isOtherChecked && (...input component) below the checkboxes

  • Within the if (key === "Other") { conditional we can add the scroll behavior (using a setTimeout or implementing it via useEffect)

                setTimeout(() => {
                    if (containerRef.current) {
                      containerRef.current.scrollTo({
                        top: containerRef.current.scrollHeight,
                        behavior: "smooth",
                      });
                    }
                  }, 0);

Also, how do you like my implementation of other_description into updatedDimensions? Let me know if it suits our needs for this!

I think this works! It's hard to tell without the backend implemented b/c this is a complex component and the Other applies to 3 different sets of ethnicities, but reading the payload I think this works perfectly fine!

And last one: I was thinking about displaying existing other_description in textbox, and noticed that if we selecting "Yes" in Are you able to record an individual’s ethnicity... section -- all of the Other Ethnicities are selected, and if selecting "No" -- only Other Unknown Ethnicity is selected. Am I understanding correctly that their other_description should be the same after all for all ethnicities?

Ah, hmm... I might not be fully understanding - selecting an answer for Are you able to record an individual’s ethnicity... should not trigger any automatic selections I don't think, is that what's happening on your end? The only automatic selection that I know of is when you select No for that question and then check Hispanic or Latino - this auto selects the Unknown field.

<NewInput
type="text"
label=""
placeholder={`If the listed categories do not adequately describe this breakdown, please describe additional definition/clarification of the "Other" selection.`}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's adjust this to just be Please describe additional definition/clarification of the "Other" selection.

publisher/src/stores/MetricConfigStore.tsx Show resolved Hide resolved
@mxosman
Copy link
Contributor

mxosman commented Oct 21, 2024

Oh, one thing I realized I forgot to include in the ticket - can we make filling out the text field a requirement if a user is going to select "Other"? Meaning, if they enter nothing, we should keep "Other" unchecked. Sorry for the last minute complexity!

@nasaownsky
Copy link
Collaborator Author

Hi @mxosman! I've updated textbox logic according to your comments, so please check out my changes and if it's working correctly!

In the meantime I only have to find a way for this to work properly:

can we make filling out the text field a requirement if a user is going to select "Other"? Meaning, if they enter nothing, we should keep "Other" unchecked

Thank you!

@mxosman
Copy link
Contributor

mxosman commented Dec 10, 2024

Thanks, @nasaownsky - will dive into this tomorrow morning!

@mxosman
Copy link
Contributor

mxosman commented Dec 11, 2024

Nice @nasaownsky! It looks like it's working exactly as intended in all 3 cases I tested. Thank you for updating the logic + positioning of it and working through the last piece. Almost there! I'll review once everything is in place.

@nasaownsky
Copy link
Collaborator Author

Hi @mxosman! Please, play with my latest changes and let me know what you think about my implementation! Does it fit your vision right? I've tried several different approaches and this one seemed to be the most suitable for our needs.

@nasaownsky
Copy link
Collaborator Author

Hi @mxosman! I think I actually need help with this.

So, after playing with BE update @nichelle-hall provided I assume I still need to work with dimensions itself, because RaceEthnicitiesModalForm operate on dimension level and because of the structure of ethnicitiesByRace. But after updating my logic I still loose my changes after page restart even on this test branch, where it should be working. I have feeling that I am missing something but I can't determine what exactly. Maybe you have some ideas? Any thoughts would be much appreciated, thank you in advance!

@mxosman
Copy link
Contributor

mxosman commented Dec 23, 2024

#1491

Hi @nasaownsky! I think I see the confusion here - you'll be writing to the context field in the layer above dimensions - the disaggregations. So, whatever the user enters in the "Other" Race & Ethnicities field, will be saved to and read from the disaggregations object w/ the key "global/race_and_ethnicity". Within this disaggregation object, there will be a contexts field which is an array that includes the OTHER_RACE_DESCRIPTION context that all input for "Other" will be saved to/read from. [Not to be confused with the contexts field within the dimensions] Does that make sense? Sorry for the confusion here!

Instead of this: this.dimensions[systemMetricKey][disaggregationKey][dimensionKey].contexts
You might be able to access it via (this is rough and just to illustrate - don't use this verbatim): this.disaggregations[systemMetricKey][disaggregationKey].contexts.find(...) (where disaggregationKey === "global/race_and_ethnicity" and you're finding the OTHER_RACE_DESCRIPTION object)

Screenshot 2024-12-23 at 4 49 31 PM

@nasaownsky
Copy link
Collaborator Author

Within this disaggregation object, there will be a contexts field which is an array that includes the OTHER_RACE_DESCRIPTION context that all input for "Other" will be saved to/read from.

Right, but then this disaggregation context has only spot for one "Other" description, and how should we write three different versions of it there? For Other / Unknown Ethnicity, Other / Not Hispanic or Latino and Other / Hispanic or Latino

@mxosman
Copy link
Contributor

mxosman commented Dec 26, 2024

Within this disaggregation object, there will be a contexts field which is an array that includes the OTHER_RACE_DESCRIPTION context that all input for "Other" will be saved to/read from.

Right, but then this disaggregation context has only spot for one "Other" description, and how should we write three different versions of it there? For Other / Unknown Ethnicity, Other / Not Hispanic or Latino and Other / Hispanic or Latino

We've simplified it from 3 separate fields to just that one field. So any input in the "Other" just goes in that one field - you no longer have to worry about 3 separate fields as this one will cover all 3.

@nasaownsky
Copy link
Collaborator Author

Hey @nichelle-hall! I've updated logic to save other description in disaggregations contexts, and it sends payload to /api/agencies/${agencyId}/metrics:

Снимок экрана 2024-12-30 в 16 20 51

but it seems like it is not saved in BE, because if I reload the page there is no value in the context:

Снимок экрана 2024-12-30 в 16 26 58

Could you please check this for me? I'm in your test env. Thank you so much in advance!

@mxosman
Copy link
Contributor

mxosman commented Jan 2, 2025

Hey @nichelle-hall! I've updated logic to save other description in disaggregations contexts, and it sends payload to /api/agencies/${agencyId}/metrics:

Снимок экрана 2024-12-30 в 16 20 51 but it seems like it is not saved in BE, because if I reload the page there is no value in the context: Снимок экрана 2024-12-30 в 16 26 58 Could you please check this for me? I'm in your [test](https://github.com//issues/1491#issuecomment-2546345451) env. Thank you so much in advance!

Hi @nasaownsky - Nichelle is out until next Monday - but, I pulled in your latest changes and tried myself, and am not seeing the contexts in the payload - just the key and dimensions. Did you push up all of your changes? The BE saves the JSON blob and returns it too - so it should just return whatever you sent, not sure what's going on - but first, I'm not able to repro your payload.

Screen.Recording.2025-01-02.at.3.44.16.PM.mov

@nasaownsky
Copy link
Collaborator Author

Hello @mxosman! Happy new year, hope you had great holidays!

I'm also out til Wednesday, but decide to respond as I saw your message and I think I know what is going wrong for you specifically -- I think you still using default env url, try change that to REACT_APP_PROXY_HOST=https://nichelletest---publisher-web-b47yvyxs3q-uc.a.run.app/ -- this is Nichelle's test env with changes to BE that we need for this PR. I double checked right now -- it sends payload to BE as expected.

@mxosman
Copy link
Contributor

mxosman commented Jan 3, 2025

Hello @mxosman! Happy new year, hope you had great holidays!

I'm also out til Wednesday, but decide to respond as I saw your message and I think I know what is going wrong for you specifically -- I think you still using default env url, try change that to REACT_APP_PROXY_HOST=https://nichelletest---publisher-web-b47yvyxs3q-uc.a.run.app/ -- this is Nichelle's test env with changes to BE that we need for this PR. I double checked right now -- it sends payload to BE as expected.

Happy New Year to you, Ilya! Hope you had a lovely holiday as well - and are enjoying your time off (please don't feel obligated to respond on your days off). My environment was pointed to Nichelle's test URL - but the payload is coming from the frontend, and I'm not sure why I don't see it included in the PUT request. When you do git log - do you show the same thing as my video? Any active changes you haven't committed?

@mxosman
Copy link
Contributor

mxosman commented Jan 3, 2025

Oh strange, I tried it again this morning and now I see the payload. Maybe I was in the wrong env after all, but could've sworn I adjusted it and re-ran before playing with it and recording. Sorry about that! Totally see what you're seeing - let's see what Nichelle thinks when she returns and go from there. In the meantime, I'll do a review at some point today since this should hopefully be mostly there.

const [otherDescription, setOtherDescription] = useState(
currentOtherDescription
);
const [isOtherChecked, setOtherChecked] = useState(Boolean(otherDescription));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should check for the presence of currentOtherDescription instead of the otherDescription state value for the initializer.

].contexts?.find((context) => context.key === "OTHER_RACE_DESCRIPTION");

if (otherDescription !== undefined) {
if (otherDescriptionContext) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: could we simplify and just do something like if (otherDescription && otherDescriptionContext)

@@ -463,6 +471,8 @@ class MetricConfigStore {
this.dimensions[systemMetricKey][disaggregationKey][
dimensionKey
].description = dimensionData.description;
this.dimensions[systemMetricKey][disaggregationKey][dimensionKey].contexts =
dimensionData.contexts;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to reference the dimension context anymore, right?


useEffect(() => {
setOtherDescription(currentOtherDescription);
}, [currentOtherDescription]);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if we need this useEffect? If currentOtherDescription is defined inline, and used to initialize currentOtherDescription, when do we expect the value of currentOtherDescription to change for this to be useful? I worry we're going to run into some unexpected race conditions with these useEffects.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, that was necessary when we maintained three different other descriptions, now it could be removed. Thank you for pointing this out!

setOtherChecked(!checked);

if (checked) {
// Clear otherDescription if "Other" is unchecked
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the comment!

@nichelle-hall
Copy link
Contributor

Hey Ilya,

I was testing our code end-to-end and found an issue on my side, but also noticed an error on your end. It looks like the contexts aren’t being sent over the first time—only on the second attempt when I redo the request to add another description.

I’ve uploaded a screen recording to illustrate the issue. Let me know your thoughts!

Screen.Recording.2025-01-06.at.4.44.12.PM.mov

I've pushed up my bug fix to same link I sent earlier.

@mxosman
Copy link
Contributor

mxosman commented Jan 7, 2025

Thank you so much for your feedback & fix, @nichelle-hall! I wonder if this is due to this line in the MetricConfigStore (L1222):

  return {
     ...
     contexts: otherDescriptionContext && [otherDescriptionContext],
     ...
  }

Specifically the check for otherDescriptionContext -- it seems the way it's written now, if there is no OTHER_RACE_DESCRIPTION in the backend response, otherDescriptionContext will evaluate to undefined and it won't send this context property.

@nichelle-hall - do you know whether or not the backend sends an empty OTHER_RACE_DESCRIPTION context field if a user has never filled it out?

@nichelle-hall
Copy link
Contributor

@nichelle-hall - do you know whether or not the backend sends an empty OTHER_RACE_DESCRIPTION context field if a user has never filled it out?

It didn't! But I just pushed up a fix so that now it does. That said, I'm still seeing the error.

@mxosman
Copy link
Contributor

mxosman commented Jan 7, 2025

@nichelle-hall - do you know whether or not the backend sends an empty OTHER_RACE_DESCRIPTION context field if a user has never filled it out?

It didn't! But I just pushed up a fix so that now it does. That said, I'm still seeing the error.

Gotcha - thank you, Nichelle! I just tried it again w/ a few agencies and it's working for me on the first go. Would you mind trying again and triple checking your .env is pointed to https://nichelletest---publisher-web-b47yvyxs3q-uc.a.run.app/? Hopefully it'll work - maybe it was in a funky state before.

@nichelle-hall
Copy link
Contributor

I'm now seeing the data saved on the first edit! I must have been on the wrong BE branch.

@nasaownsky
Copy link
Collaborator Author

@mxosman @nichelle-hall Hi all! This is utterly strange but now this is not working for me at all. When I check this.disaggregations[systemMetricKey]?.[RACE_ETHNICITY_DISAGGREGATION_KEY] in MetricConfigStore.tsx it shows me that contexts is undefined. I triple checked .env (it's https://nichelletest---publisher-web-b47yvyxs3q-uc.a.run.app/), reinstalled dependencies, cleared all the caches -- but nothing had effect. It seems like I don't receive contexts from BE. It's especially strange since it's working for you, that I have this issue. Maybe you have any ideas?

@nichelle-hall
Copy link
Contributor

@mxosman @nichelle-hall Hi all! This is utterly strange but now this is not working for me at all. When I check this.disaggregations[systemMetricKey]?.[RACE_ETHNICITY_DISAGGREGATION_KEY] in MetricConfigStore.tsx it shows me that contexts is undefined. I triple checked .env (it's https://nichelletest---publisher-web-b47yvyxs3q-uc.a.run.app/), reinstalled dependencies, cleared all the caches -- but nothing had effect. It seems like I don't receive contexts from BE. It's especially strange since it's working for you, that I have this issue. Maybe you have any ideas?

You know what! I think I made a mistake and wrote over some changes on my playtesting branch. I'l redeploy now!

@nasaownsky
Copy link
Collaborator Author

@nichelle-hall It's working now, thank you!

@nasaownsky
Copy link
Collaborator Author

@mxosman thank you for great comments, now it should be ready!

@mxosman
Copy link
Contributor

mxosman commented Jan 10, 2025

Whew - thank you for the changes! I was sooo close to approving, but ran into a strange issue specifically with Jails > Total Population Race & Ethnicity metric - for some reason this isn't sending a context object - can you take another look? [Super confirmed that I'm using the right - which is Nichelle's playtest link]

Screen.Recording.2025-01-10.at.4.41.04.PM-Issue.mov

@nichelle-hall - wondering if you can take a look too to see if you can repro and if there might be something in the BE related to this particular metric?

@nasaownsky
Copy link
Collaborator Author

nasaownsky commented Jan 13, 2025

Oh, that's unfortunate! I think that I can't test this issue, because the only way for me to test this in my environment is Jails (AR) agency (id: 151), but when I selecting it -- it is stuck in infinite loading with 500 error. Maybe the reason of this issue somewhere in BE?

cc @nichelle-hall

@mxosman
Copy link
Contributor

mxosman commented Jan 13, 2025

Hi @nasaownsky/@nichelle-hall - oh strange - there is something weird going on with that agency... I'm getting a server error too.

You should still be able to test this issue with a different agency or by creating a new agency... Can you try that, please? You can point the environment to regular publisher-staging URL and not the nichelletest since the backend changes were merged in.

@nasaownsky
Copy link
Collaborator Author

Oh, yes, I forgot that I could create one! Sure, I'll give it a shot.

@mxosman
Copy link
Contributor

mxosman commented Jan 13, 2025

OK, hello again @nasaownsky & @nichelle-hall. I did some digging and it looks like for the metrics that are affected (metrics in Jails and Prisons, and probably some others I haven't tested yet) that the backend is not sending over the OTHER_RACE_DESCRIPTION context field.

@nichelle-hall - I saw that you added these in to the TIGs in your #36052 PR - any hunches as to why it's not sending them over? You can try this agency: http://localhost:3000/agency/1693/metric-config?system=jails&metric=jails_total_population - if you look at the network response when you load the Total Daily Populations metric, it doesn't show anything with a OTHER_RACE_DESCRIPTION key.

@nasaownsky - is it necessary for the FE to have this field before it's able to send it in the PUT request? If not, is there a way that we can just always send this context field regardless of whether or not the backend sends it initially?

@nasaownsky
Copy link
Collaborator Author

@mxosman Thank you for your detailed investigation! I played around with implementation and I think we could eliminate this issue on FE side by introducing defaultOtherDescriptionContext object. Tested with newly created Jails agencies and it seems like the issue you describing is gone!

@mxosman
Copy link
Contributor

mxosman commented Jan 14, 2025

@mxosman Thank you for your detailed investigation! I played around with implementation and I think we could eliminate this issue on FE side by introducing defaultOtherDescriptionContext object. Tested with newly created Jails agencies and it seems like the issue you describing is gone!

Great call! I was hoping we could do exactly this - just send it anyway and not depend on the BE giving it to us. Thanks for the adjustment. One small issue I noticed - the "Other" doesn't check after you save:

Screen.Recording.2025-01-14.at.10.11.51.AM.mov

Could you take another look and spend a bit more time manually testing different agencies/metrics to make sure things are working as intended? I recommend creating a new agency with Law Enforcement, Jails, Prisons, etc. sector to get a decent variety of cases.

@nichelle-hall
Copy link
Contributor

nichelle-hall commented Jan 14, 2025

@mxosman @nasaownsky

I know you have a plan of attack, but I'm also looking into a fix. The BE should be sending the contexts over!

I merged a PR yesterday that made some changes regarding sending report contexts and I wonder if I introduced a bug.

@nasaownsky
Copy link
Collaborator Author

One small issue I noticed - the "Other" doesn't check after you save

@mxosman Hmm, this one actually could be tricky without the context. Let me think about it some more.

@nichelle-hall Oh, great to hear! I think it should fix that problem for good, with the context sending from BE.

@nichelle-hall
Copy link
Contributor

@nasaownsky @mxosman

I've found the bug and pushed up the changes to https://nichelletest---publisher-web-b47yvyxs3q-uc.a.run.app/.

@mxosman
Copy link
Contributor

mxosman commented Jan 14, 2025

@nasaownsky @mxosman

I've found the bug and pushed up the changes to https://nichelletest---publisher-web-b47yvyxs3q-uc.a.run.app/.

Ah, amazing - that fixes everything! @nasaownsky & @nichelle-hall - thank you both SO MUCH for all your work on this - what a journey! I did another round of tests and I couldn't break it. I'm glad we're finally able to ship this! 👏 👏 👏

We'll wait for the backend fix to go through review and get merged in before I give this one the green light to make sure everything is in sync - but consider this approved. Thank you both again so much!

@nasaownsky
Copy link
Collaborator Author

Yaaaay! 🚀

@nichelle-hall
Copy link
Contributor

The BE fix has been merged!

Copy link
Contributor

@mxosman mxosman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @nichelle-hall! Good to go!

…to nasaownsky/1491-race-breakdown-other-textbox
@nasaownsky nasaownsky merged commit fa7313a into main Jan 16, 2025
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Publisher] Add "Other" free text box field for Race/Ethnicity breakdowns
3 participants