Flag checkbox

[deasterdaywfp]

option to flag a convo they may need to reassign or need help on. This could be a checkbox in the texter reply screen. This should categorize this conversation as "flagged for support" and ideally should move that conversation into a queue that is accessible by a select group or all that are at supervolunteer permission level or above.

cc @joemcl

[joemcl]

totally needed and a use case is, for example, to flag a convo that needs a response from a Spanish speaker/writer, IMHO also related to need to be able to add users to team(s) like Spanish speakers/writers, see #639

[lperson]

@schuyler1d what do you think about this as an application of the tagging feature that I have an old branch for. The tagging would be a prerequisite for other things, such as the profanity filter.

[schuyler1d]

thinking.... It seems like there are some things that are worth pulling apart. Some of the use-cases bridge both sides of what I'll call 'pure contact data' and 'operational'.

So the general case for tagging is to store data about a contact. So, e.g. their language is something worth tagging or whether they're 'hostile' (or moved, or 'wrong number') -- these seem like good 'tags'. The challenge with the 'operational' connection is let's say someone gets escalatated to be delegated to a Spanish speaking person (or any other issue) -- Once it's reviewed, it might be assigned to someone else, and then it's no longer 'escalated' -- So my temptation is that we want both.
I think 'flagged for support/escalated' should probably be in the campaign_contact table -- and that status column can get updated, etc. On the UI side, we might want to think how we can do 'both at once' -- i.e. maybe the texter has an 'escalate' button, and some choices would be {'Spanish', , 'hostile', [optional additional languages here] 'other'} and then we tag them with the Spanish, etc., but then also update the campaign_contact.assign_status to be searchable/flaggable from the Message Console. (Maybe an intersecting, but different, list would be part of the 'OptOut' interface/screen, and yet another ?super-set might be directly available)

I think my intuition on the Tags side is that they should generally be things that are written/created once, rather than updated/changed (except for rare 'misunderstandings') and more for data-out/reporting than for operational parts of spoke (assignment/delegation/campaign organization/etc)

Thoughts?

[harpojaeger]

Are we envisioning these as tags on a contact entity, or a campaign contact entity? I.e. do they persist between campaigns, or not?

For the "is a Spanish speaker" tag, that data is of much greater value to a campaign if it's surfaced in an exportable way and can be used for creating future campaigns (i.e. one campaign for Spanish speakers and one for English speakers). "Hostile" might be the same way, although it seems more likely to me that an org would like to just opt that person out, rather than have them proactively flagged as "hostile" in the future.

[lperson]

@harpojaeger there is no concept of a contact independent of a campaign in the Spoke app (as opposed to a data warehouse or external system such as VAN or ActionNetwork). Tags would apply to the entity campaign_contact whether we implement it as a column or a more flexible tagging design, or some combination. I agree that it's extremely useful to capture attributes discovered about a contact to apply to future interactions with them.

The opt_out entity enables opt outs to be applied to a phone number, and Spoke does check the opt_out table when importing new contacts, opting them out accordingly.

What I take away from this is maybe we should think about a similar mechanism for information learned about a contact -- aka a phone number -- and that information should be considered immutable.

[harpojaeger]

Right, I know we currently don't have a persistent contact object, and I'm wondering whether we'd need to create one for this kind of thing to work. I'm hesitant about increasing the kinds/quantity of data Spoke stores without having thought about the use case pretty carefully and whether there's another way of doing it. Storing, managing and deduplicating contacts is a big job.

[lperson]

If we use the approach in the opt-out table, the data is keyed on phone number, which is absolutely unique. There are edge cases around a number being reassigned but if we have a way of knowing about that, we can purge the number from the table.

There is a proposed use case in this issue. There's value to building to the first use case and refactoring later if necessary, and reconsidering when there's a second use case.

[harpojaeger]

The other flag (as it were) I have on this is that we're opening Spoke up to persistent storage of personally identifying information.

[lperson]

Huh. Such as "speaks Spanish" ?

[harpojaeger]

For the record, I'm not heavily opposed to this feature, I just want to play out this discussion all the way.

Are we planning to allow only a select set of tags, or would this be user-editable? If the latter, we have no idea what kind of data might be stored.

I also think there's a conceptual difference between persisting an opt-out record (which consists only of a phone number and no other PII) and persisting data about phone numbers. I think we also might want to consider a security audit if we're going to set up Spoke to store arbitrary contact information permanently. Since one of our goals is to make Spoke easy to use by people without deep technical knowledge, we can't necessarily count on them to know everyting about security. Just throwing out all the concerns that come to mind for me on this.

[schuyler1d]

I think my architectural vote is anything that will export and connect to a campaign-contact (specific to a campaign) can/should use tags -- customizable markers (at least in the future) that we learn about the contact. Yes, these will be extremely useful to export and also to feedback into future contacts with the people -- they may tag people's political leanings, language, interest in events, etc.

Anything that is going to cross operational lines or be across campaigns, we should make a column for and not be generic about it. i.e. if you want a phone_owner_memory table that remembers language across campaigns then there should be a lang column and es can be filled in there. similar to hostility -- though the architectural plan with the existing schema was to put that kind of thing in the reason column in the opt_out table -- i haven't heard any use-cases where hostile responses are usefully contacted again (even if they don't explicitly optout).

[harpojaeger]

One issue I foresee with user-editable tags that are related to a campaign contact is confusion on the part of campaign admins with regard to what should be a script question and what should be a tag. You see this happening in VAN a lot: campaigns are undisciplined about what attributes should be stored as a survey question response (equivalent to an interaction step in Spoke) and what is stored as an activist code (equivalent to the contact-based tag we're talking about here). It makes for messy datasets and decreases overall usability of the product by anybody who's not a trained expert, which is the opposite of what we want for Spoke.

I don't want to come across as quarreling with the legitimacy of the use case here: if Spoke can help campaigns more effectively contact people in their native language, that's a good thing and we should make it possible. I just think that persisting that data between campaigns may open us up to more problems in the future. Do we create an interface for editing tags that have already been applied to a phone number? How useful is that interface if it just shows phone numbers and doesn't have contact names associated with it? Are we explicitly making it possible to "slice" campaigns in some way according to contact tags (i.e. creating some kind of teams functionality)? Each of these things would be a nontrivial amount of coding work, and I'm concerned it could lead to feature bloat.

I'm totally open to being convinced on this, but based on what we're talking about so far, I'm inclined to say any tags we create should be associated with the campaign contact for the moment, and we should think very carefully about how to avoid diverting people away from script questions. I can easily imagine a scenario where someone decides it's easier to just give people a Google Doc describing the conversation they should have with a contact and a bunch of tags to describe the interaction for later analysis and import into a CRM (I know some campaigns already find our script creation process prohibitively complex and just write their scripts elsewhere, without using the question or canned response flows). If that's what works for them, fine, but at that point we're essentially building a glorified IM app.

[lperson]

@schuyler1d is there a third category? operational data within a campaign? For example, a flag on a campaign_contact indicating needs_organizer_followup -- that's a column on campaign contact, not a tag?

[lperson]

@harpojaeger your last comment makes a great point (IMHO) about tags vs. scripts. As you said, to address this concern we could restrict tags to a small number of predefined tags.

As an example, the specific use case of flagging non-english speakers has come up repeatedly with the WFP folks (via @joemcl).

[harpojaeger]

Yeah, so I think what we're converging on is (and correct me if this isn't an accurate summary of the discussion so far):

1. tags should be on campaign contacts, exported with a campaign results dump, and not persisted between campaigns, and
2. we should only add tags as we figure out they're useful, and not (at this stage) set up some kind of UI for creating arbitrary tags on the fly.

This feels much more manageable to me and much less likely to lead to bloat/user confusion.

[lperson]

That's about right.

The question remains about operational or workflow information, such as "this contact needs attention." It sounds like that's a column on campaign-contact.

[joemcl]

@harpojaeger re this -

"* tags should be on campaign contacts, exported with a campaign results dump, and not persisted between campaigns, and"

I think we do want persistence between campaigns but can you explain what you mean by 'persisted' pls

"* we should only add tags as we figure out they're useful, and not (at this stage) set up some kind of UI for creating arbitrary tags on the fly."

Who is we? Devs via a PR you mean? In the admin UI we'd def want the ability to create new tags
on the fly, right @deasterdaywfp ?

[harpojaeger]

I've been using "persistence" to mean "attached to something other than a campaign contact." I.e. a separate table in the DB that associates tags with phone numbers. Based on the discussion here, I think the tags we're talking about shouldn't be persistent.

As to the second point, for the reasons I was talking about above, I'm specifically advocating that we don't create a UI that allows admins to make arbitrary new tags, and instead hard-code them.

[deasterdaywfp]

I can live with that as hardcoded until the community comes to consensus on the most commonly needed. That said, I think that moving beyond the ability to "escalate" the conversation and take in specific contextual notes to add to the conversation would relate more to my issue posted here: https://github.com/MoveOnOrg/Spoke/issues/1037 *Dylan Easterday* *National Volunteer Manager, Digital - Working Families Party* *Working Families <http://workingfamilies.org>* *Text *(347) 460-9451 or Schedule https://calendly.com/deasterday Email [email protected] Done with email? me too - join me here <http://bit.ly/WFP4theMany>

…

On Tue, Feb 5, 2019 at 2:42 PM Harpo Jaeger ***@***.***> wrote: I've been using "persistence" to mean "attached to something other than a campaign contact." I.e. a separate table in the DB that associates tags with phone numbers. Based on the discussion here, I think the tags we're talking about *shouldn't* be persistent. As to the second point, for the reasons I was talking about above, I'm specifically advocating that we *don't* create a UI that allows admins to make arbitrary new tags, and instead hard-code them. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/MoveOnOrg/Spoke/issues/1044#issuecomment-460775997>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AiFnyt3iAw89188thkdJyBnKy0k4XNdpks5vKd6hgaJpZM4abJHa> .

[joemcl]

@harpojaeger @deasterdaywfp thanks and I'll actually scan the prior discussion above :) . I still come down in favor of persistence but I'll follow the convo above and perhaps be convinced that I'm wrong.

[harpojaeger]

Thanks for weighing in, @joemcl. In the future, can you try to read the discussion before you comment? Larry & Sky and I had a pretty in-depth conversation about how to address this use case while balancing other development concerns, and it doesn't feel great to have you disagree with our conclusions without bothering to understand how we came to them. Of course, if after reading the comments you think we missed something or you still disagree, you should absolutely say so.

[lperson]

@harpojaeger

There's no disagreement.

My summary understanding is based on these two consecutive comments:

https://github.com/MoveOnOrg/Spoke/issues/1044#issuecomment-460733574
https://github.com/MoveOnOrg/Spoke/issues/1044#issuecomment-460734954

Getting specific, in this use case, that would mean:

• add admin_followup to campaign-contact
• add table campaign_contact_tags with tag (varchar) and campaign_contact_id

For now the only possible value of tag is spanish_speaker

Are we on the same page?

[harpojaeger]

Yes, I think we are on the same page!

[joemcl]

@harpojaeger so I've now read all the discussion.
I agree that tags should be hard-coded to start.
I still think that tags should persist across campaigns if I'm understanding the above discussion correctly.
You asked above "[a]re we explicitly making it possible to "slice" campaigns in some way according to contact tags (i.e. creating some kind of teams functionality)?" The answer is Yes, I think so.
A use case is this - we run say, five, separate text campaigns to support one candidate/campaign. Across those five campaigns, we ID a group of Spanish writers/speakers who are supporters of the candidate, and we want to create a new campaign for GOTV in Spanish.
We could just export out the data from those five campaigns, filter out the tagged Spanish speakers/writers, create a .csv just of those for the new Spanish-language GOTV texting campaign, and run that. Which is basically what we do now right @deasterdaywfp for ID'd Yes votes/supporters, for example, regardless of language.
But, if by persisting data across multiple campaigns, and attaching a tag to a contact, we could filter for say all the Spanish speakers/writers in the Spoke admin UI from all five of the completed texting campaigns about that one campaign/candidate, and create a new contact group to send a new Spanish-language GOTV texting campaign, that would be preferable, yes. That would also be easier, quicker IMHO for the average admin user than exporting out, filtering/querying, importing back in.
I'm thinking of how Relay, for example, functions to allow new campaign contact list creation based on filtering/querying responses to prior texting campaigns directly in the admin UI.

[joemcl]

oh and @harpojaeger @lperson @schuyler1d @deasterdaywfp re teams, see #639 .

[harpojaeger]

So I want to be careful of feature bloat here, where we start creating very complex UIs and entirely new data structures that address only one highly specific use case. This can lead to an application with lots of parts that feel "bolted on," rather than something that feels like a cohesive whole and is intuitive to use.

I understand the connection to #639. I'm very hesitant about the idea of teams as a "first-class" feature, because it feels like a really specific solution to a really specific problem, rather than a more generalizeable, if slightly less precise, way of accomplishing the same thing (per-campaign contact tags).

If we were going to include a "split this upload into campaigns by teams" feature, here are some of the things we'd need to create:

• Texter teams. A new data structure that tracks which teams a texter is part of, and a UI for managing them.
• Campaign splitting. A UI for selecting the criteria upon which a given upload is to be split into campaigns.
• Persistent storage of PII. This remains one of my strongest objections. If we continue developing this feature, we will be storing information about contacts and texters that we don't currently store. When a tech tool is created that allows the storage of information about people, a safe assumption is that, over time, it will store more and more such information. I continue to believe that we haven't fully explored the privacy and security implications of starting down that path. Doing this on a campaign contact level has some of the same concerns, but doesn't have the same effect of creating a single table that maps attributes to people. It's also easier to imagine garbage collection on old campaign contacts (i.e. dropping records from old campaigns to save space and deal with data retention issues) than it is on a contact table that's kept up to date.
• Confusing our users. Right now, Spoke simply receives data from an external tool, processes it and makes it available for export. This change makes us into a CRM of sorts. I have heard from campaigns that use Spoke that it's hard to understand which parts of the application do what. I think we should be prioritizing refining and streamlining what we already have, not adding significant new features without asking our users whether they're needed and how to implement them effectively.

Altogether, it's still not clear to me that the added value from this change (not having to export data and re-import it) justifies the added complexity in the codebase and the UI, especially given that dev time is a finite resource.

[joemcl]

@harpojaeger I wrote about only one use case as an example - language. There are certainly other tags - issue responses, has questions about an issue we should address but have not yet, geography, use of foul or abusive language, etc etc.

Allowing persistence of data allows us to create and use texter teams down the road to send to campaigns of ID'd Spanish speakers writers, etc (see list above) based on tags.

If we only allow export out of info based on tags in a single campaign to an external CRM, as we will do anyway, that will be stored, just elsewhere, where a Spoke admin user may or may not be able to get to it quickly, or easily, if they even have the right permissions on that external CRM, much less the desire or time to learn that additional external CRM system.

Frankly, Spoke is nowhere near having too many features :) that we have to worry about feature bloat.

And Spoke admin time is also a finite resource :) . "[N]ot having to export data and re-import it" conserves some of that finite resource.

[harpojaeger]

Adding more campaign contact tags would make my objections to this even stronger. We'd be storing more PII, and we'd need even more logic to create teams and divide uploads into campaigns. This will add a lot of complexity to our code and to our UI.

Several of the things you described here (issue responses, geography, questions about an issue we haven't addressed yet) also sound to me like they maybe belong as script question responses, not tags (or qualitative notes, a la #1037), which brings me back to the point I made above about introducing data structures that are only ambiguously different from ones that already exist. I can imagine that having the effect of confusing users and making Spoke less usable overall.

Again, I understand what additional things this would enable and the ways it would save time, I just don't see the cost-benefit analysis being positive right now, based on the discussion here. If this is something WFP really needs, maybe WFP can develop it internally and open a PR if it works well. I'm a volunteer contributor, sharing my opinion on what I think will benefit the whole Spoke user and dev community, and I think there are other things we can spend our time on that will make Spoke more useful for more people.

I also disagree that we don't have to worry about feature bloat. It's not a question of how many features we have, it's a question of how the features are implemented and how they work together. For the reasons I've described above, I think the implementation of this feature as it's being discussed here and the ways it butts up against existing UI elements and underlying data structures doesn't pass the test.

[deasterdaywfp]

Just chiming in to ground this in the issue really at hand that, at the moment in the app, there is now easy way to ID conversations that need special handling -- and this is something that is VERY common in organizing and electoral work. I understand there are competing priorities and also considerations (i.e. bloat), but this is really something that I think Erik from MoveOn would agree is in the category of BASIC functionality. I'm not a coder, so don't have that view on the logic or labor needed, but these are my two cents as an organizer with almost a year of using this app daily w/ almost 6 million contacts through it... *Dylan Easterday* *National Volunteer Manager, Digital - Working Families Party* *Working Families <http://workingfamilies.org>* *Text *(347) 460-9451 or Schedule https://calendly.com/deasterday Email [email protected] Done with email? me too - join me here <http://bit.ly/WFP4theMany>

…

On Wed, Feb 6, 2019 at 2:23 PM Harpo Jaeger ***@***.***> wrote: Adding more campaign contact tags would make my objections to this even stronger. We'd be storing more PII, and we'd need even more logic to create teams and divide uploads into campaigns. This will add a lot of complexity to our code and to our UI. Several of the things you described here (issue responses, geography, questions about an issue we haven't addressed yet) also sound to me like they maybe belong as script question responses, not tags (or qualitative notes, a la #1037 <https://github.com/MoveOnOrg/Spoke/issues/1037>), which brings me back to the point I made above <https://github.com/MoveOnOrg/Spoke/issues/1044#issuecomment-460721867> about introducing data structures that are only ambiguously different from ones that already exist. I can imagine that having the effect of confusing users and making Spoke less usable overall. Again, I understand what additional things this would enable and the ways it would save time, I just don't see the cost-benefit analysis being positive right now, based on the discussion here. If this is something WFP really needs, maybe WFP can develop it internally and open a PR if it works well. I'm a volunteer contributor, sharing my opinion on what I think will benefit the whole Spoke user and dev community, and I think there are other things we can spend our time on that will make Spoke more useful for more people. I also disagree that we don't have to worry about feature bloat. It's not a question of how many features we have, it's a question of how the features are implemented and how they work together. For the reasons I've described above, I think the implementation of this feature as it's being discussed here and the ways it butts up against existing UI elements and underlying data structures doesn't pass the test. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/MoveOnOrg/Spoke/issues/1044#issuecomment-461152953>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AiFnyo-x5Kh6MXCZJm3urJ4h9qFa_moTks5vKyuhgaJpZM4abJHa> .

[harpojaeger]

Yeah, we've strayed from the original point of this issue. @lperson's comment above still holds on the "needs admin attention" question, and I think everyone here is in agreement about how that should be implemented.

[joemcl]

@harpojaeger yes "needs admin attention" is the minimum. But not apparent agreement on the persistence question. Teams and uploads are down the road. Having persistence just helps enable that.

[lperson]

it's so easy to go meta

[deasterdaywfp]

Hey folks -- this has been quiet a while and wondering if all the energy left the room and we think this is still a good feature add, or we need to reconsider overall? @lperson @harpojaeger @schuyler1d @joemcl

[deasterdaywfp]

Highlighting for you @codygordon since the rest of the party left 😑

[suegibson]

Moderators also need to be able to flag with a date stamp that we have reached out to the campaign for a response.

[deasterdaywfp]

@suegibson -- an idea. Maybe the way to accomplish this is a button or check to hit when "resolved" so that it is taken out of the flag queue?

…

On Thu, Jul 25, 2019, 1:44 PM suegibson ***@***.***> wrote: Moderators also need to be able to flag *with a date stamp* that we have reached out to the campaign for a response. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/MoveOnOrg/Spoke/issues/1044?email_source=notifications&email_token=AIQWPSUWJ6BESXITSL64IATQBHRANA5CNFSM4GTMSHNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD22HPXQ#issuecomment-515143646>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AIQWPSQ5FIDNQ7ZARWUPX4LQBHRANANCNFSM4GTMSHNA> .

[suegibson]

Seems like closure should be automatic once a response is sent. My concern is the period of time prior to resolution. A message needing texter response that no one knows how to answer can get passed around among several reply specialists, and conceivably could be reported to the campaign by several people not realizing it has already been done. With a date stamp, we would know when to give up on hearing from the campaign and just send Specific Q.

[lperson]

@deasterdaywfp and @suegibson and @joemcl in this situation we probably don't want to mark the flagged message as "resolved" because it really isn't resolved. Do you want to let the texter know somebody reached out to the campaign, and then when a response is received from the campaign, be able to quickly find the conversation and let the texter know when to proceed? This feels almost like a ticketing or task management application. The texter "opens a case" by flagging the message, actions are taken to resolve the case (which you probably want to keep around so the next person to touch the conversation can know what already happened), and eventually the case is resolved somehow. It would be cool to integrate with an issue tracking system (like Jira, Asana, Trello, Monday, baseCamp or whatever) if you already use one. Let's discuss WFP's specific workflow offline.

[bchrobot]

For what it's worth, @ben-pr-p and I have begun working on implementing tags in our fork (see politics-rewired/Spoke#219 for initial work).

The design decisions we have made for current work are:

• Tags are Organization-scoped -- in keeping with our commitment to strong multi-tenancy support.
• Tags are Campaign Contact-scoped -- We agree with @harpojaeger that Spoke is not a CRM and should not become one. Tags may be applied to a Campaign Contact in Spoke to aid in handling that specific conversation but should ultimately be synced back to the CRM. We acknowledge that programmatic syncing is a perennial difficulty and to make this easier with Tags, we store an optional webhook_url with each tag. When the Tag is applied to a Campaign Contact, the Spoke hits the webhook with appropriate details.
• Tags may be managed by Admins -- Our experience and intuition say that Tags are too variable to be hardcoded. A new deployment should not be necessary to support tagging language:arabic in addition to language:spanish.
• Tags may be managed by Spoke -- These are created as part of Organization creation. They are managed by Spoke, however, Admins are able to edit certain properties (e.g. an optional script to send to the Campaign Contact when the Tag is applied). An example system tag would be escalated to indicate that it requires elevated attention from an Admin.

Design decisions we have made for future work are:

• Texters may be tagged -- We want to support reassignment of conversations based on the tag. In the case of a language tag, we would want Admins to be able to reassign to Texters that the Admins know speak the language. This is also important for the logic behind the "request assignment" workflow in our fork.
• Campaigns may be tagged -- This is also to support matching Texters with conversations. With Tags being synced to the CRM, the lists being uploaded to Spoke should already be filtered and the Campaign being created should be specific to that list. In the case of a first language Tag, the script would likely be written in that language. To match Texters to Conversations, both of which are Spoke-specific models, we use Tags.
• Texters, Campaigns, and Campaign Contacts all use the same set of Tags -- This is for simplicity of implementation and consistency/ease of us for Admins.

[arena]

I see this as "forward" rather than flag.
or "request moderator support"

(sorry for not reading every single comment here.)

I'd like this to be a single button or menu item in the Texter responding or reviewing UI (perhaps one step away in a menu) ..but not a tag.

[ibrand]

We've talked about including a way to flag or escalate convos in the sidebar mentioned in RFC #1533

[schuyler1d]

Tags are used, and can also be 'resolved' so this is possible now