Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create bulk moderation actions #3840

Closed
stacimc opened this issue Feb 28, 2024 · 1 comment · Fixed by #4654
Closed

Create bulk moderation actions #3840

stacimc opened this issue Feb 28, 2024 · 1 comment · Fixed by #4654
Assignees
Labels
💻 aspect: code Concerns the software code in the repository 🌟 goal: addition Addition of new feature 🟨 priority: medium Not blocking but should be addressed soon 🧱 stack: api Related to the Django API

Comments

@stacimc
Copy link
Collaborator

stacimc commented Feb 28, 2024

Description

Create the custom Django admin actions for bulk moderation decisions. All bulk actions should go through an intermediate confirmation page which:

  • Indicates the number of records that will be affected by the action

  • Collects the required explanation text for the ModerationDecision (all other columns, such as moderator_id and created_on, are filled in automatically)

  • bulk deindex sensitive

    • This action takes a queryset of AbstractMedia records. It creates a ModerationDecision with type deindexed_sensitive and adds all records from the queryset. It then creates and saves DeletedMedia records for each of these, deleting the record from the API and deindexing it in Elasticsearch. The confirmation page for this action should include a highly visible warning that once records are deindexed, they cannot be restored immediately.
    • Note that it is not necessary to check that a DeletedMedia record does not already exist, because deindexing a record deletes it from the database (so it is not possible to call this action on already deleted records).**
  • bulk deindex copyright

    • Identical to the other bulk deindex, except the action type is deindexed_copyright.
  • bulk mark sensitive

    • This action takes a queryset of AbstractMedia records. It filters the queryset for those records which do not already have a related SensitiveMedia record, then adds them to a new ModerationDecision with type marked_sensitive. It then creates and saves SensitiveMedia records for each of the added records.
    • The confirmation page for this action should indicate the number of records that will actually be affected. If some records are already marked sensitive, that should be indicated on the confirmation page as well.
  • bulk reverse mark sensitive

    • This action takes a queryset of SensitiveMedia records, rather than AbstractMedia. Each SensitiveMedia record has a related media_obj, which are what will be added to the ModerationDecision. It creates a ModerationDecision with type reversed_mark_sensitive, and adds the related media. Then it first calls _update_es on the SensitiveMedia records to update the Elasticsearch index with the “sensitive” field for each media record, and finally deletes the actual SensitiveMedia records.
    • Although the SensitiveMedia records are deleted, the ModerationDecisions preserve the history of these actions, and Django’s LogEntry table will also automatically record the creation and deletion of the SensitiveMedia records.
  • bulk reverse deindex

    • This action takes a queryset of DeletedMedia records, rather than AbstractMedia. Like SensitiveMedia, each DeletedMedia record has a related media_obj, which will be added to the ModerationDecision. The action creates a ModerationDecision with type reversed_deindex and adds the related records. Then it deletes the DeletedMedia records.
    • It is extremely important to note that this action does not immediately result in deindexed records being added back to the DB and Elasticsearch indices. When the records were originally deindexed, they were deleted from both, and there is no quick way to restore them without running a data refresh. The confirmation page should contain a highly visible info block explaining this to the user.
    • Although the DeletedMedia records are deleted, the ModerationDecisions preserve the history of these actions, and Django’s LogEntry table will also automatically record the creation and deletion of the DeletedMedia records.

More details can be found in the IP here.

Actions will be added to the appropriate views in subsequent issues.

Blocks #3841, #3842, and #3843

@stacimc stacimc added 🟨 priority: medium Not blocking but should be addressed soon 🌟 goal: addition Addition of new feature 💻 aspect: code Concerns the software code in the repository ⛔ status: blocked Blocked & therefore, not ready for work 🧱 stack: api Related to the Django API labels Feb 28, 2024
@stacimc stacimc added this to the Bulk moderation actions milestone Feb 28, 2024
@openverse-bot openverse-bot moved this to ⛔ Blocked in Openverse Backlog Feb 28, 2024
@stacimc stacimc removed the ⛔ status: blocked Blocked & therefore, not ready for work label Apr 3, 2024
@stacimc stacimc moved this from ⛔ Blocked to 📋 Backlog in Openverse Backlog Apr 3, 2024
@sarayourfriend
Copy link
Collaborator

Adding a note here that we should make sure not to rely on the existing methods for marking sensitive or deleting media, as those do not do bulk operations and are wildly inefficient if used for a bulk operation (e.g., if used in a for loop over a set of sensitive or deleted media instances).

@dhruvkb dhruvkb moved this from 📋 Backlog to 📅 To Do in Openverse Backlog Jul 1, 2024
@openverse-bot openverse-bot moved this from 📅 To Do to 🏗 In Progress in Openverse Backlog Jul 24, 2024
@openverse-bot openverse-bot moved this from 🏗 In Progress to ✅ Done in Openverse Backlog Jul 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
💻 aspect: code Concerns the software code in the repository 🌟 goal: addition Addition of new feature 🟨 priority: medium Not blocking but should be addressed soon 🧱 stack: api Related to the Django API
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

3 participants