Skip to content

Commit

Permalink
[NEW][BREAK] Message retention policy and pruning (#11236)
Browse files Browse the repository at this point in the history 
Closes #6749
Closes #8321
Closes #9374
Closes #2700
Closes #2639
Closes #2355 
Closes #1861
Closes #8757
Closes #7228
Closes #10870
Closes #6193 
Closes #11299
Closes #11468
Closes #9317
Closes #11300 (will incorporate a fix to this PR's issue)
Closes #11046 (will incorporate a fix to this PR's issue)
Contributes to #5944 
Contributes to #11475
_...and possibly more!_

This PR makes deleting messages (automatically and manually) a lot easier on Rocket.Chat.

- [X] Implement a bulk message deletion notification, to quickly push large message deletions to users without reload
  - [X] Use it in `rooms.cleanHistory`
  - [X] Use it in user deletions
- [X] Completely remove cleanChannelHistory as required by v0.67
  - [X] Remove server method `cleanChannelHistory`
  - [X] Remove REST API `channels.cleanHistory`
- [x] Implement a sidebar option to clean history
  - [x] Basic implementation
  - [x] Allow excluding pinned messages
  - [x] Allow attachment-only mode
  - [x] Allow specifying user(s) to narrow down to
    - [x] Also update REST API
    - [x] Also update docs
  - [x] Break the deletion into multiple different requests, so the client can keep track of progress
  - [x] Clear animation / progress bar for deleting
- [x] Retention policy
  - [X] Global, set by admin
    - [X] Global timer that runs every second and deletes messages over the set limit
      - [X] Can change its timer's resolution to prevent insane CPU overhead
    - [X] Admin can decide what room types to target (channels, groups and/or DMs)
    - [X] Allow excluding pinned messages
    - [X] Allow attachment-only mode
  - [x] Per-channel, set by those with a new permission
    - [x] Disabled when master switch off
    - [x] Set in channel info
    - [x] Can override global policy with a switch that requires `edit-privileged-setting`
    - [x] Allow excluding pinned messages
    - [x] Allow attachment-only mode
    - [x] Uses same global timer for cleanup
  - [X] Message at start of channel history / in channel info if there is a retention policy set
  - [x] Message in channel info if there is a retention policy set on that channel specifically
- [X] Make cleaning history also delete files (completely!)
  - [X] Manual purging
  - [X] Automatic purging
- [x] Make other deletions also delete files
  - [x] User deletion
    - [X] Own messages
    - [x] DMs with them's partner messages
  - [x] Room deletion
- [x] Cleanup
- [x] Finish related [docs](https://github.com/RocketChat/docs/pull/815)
- [x] Link to the docs in the settings

Please suggest any cool changes/additions! Any support is greatly appreciated.

**Breaking change:** This PR removes REST API endpoint `channels.cleanHistory` and Meteor callable `cleanChannelHistory` as per the protocol specified for them.

![bzzzzzzzz](https://user-images.githubusercontent.com/39674991/41799087-56d1dea0-7670-11e8-94c0-bc534b1f832d.png)
  • Loading branch information
@vynmera @ggazzo
vynmera authored and ggazzo committed Jul 20, 2018
1 parent 5244bd5 commit 829c5d1
Show file tree
Hide file tree
Showing 36 changed files with 1,769 additions and 131 deletions.
2 changes: 2 additions & 0 deletions .meteor/packages
View file
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,7 @@ rocketchat:otr
rocketchat:postcss
rocketchat:push-notifications
rocketchat:reactions
rocketchat:retention-policy
rocketchat:apps
rocketchat:sandstorm
rocketchat:setup-wizard
Expand Down Expand Up @@ -142,6 +143,7 @@ rocketchat:tutum
rocketchat:ui
rocketchat:ui-account
rocketchat:ui-admin
rocketchat:ui-clean-history
rocketchat:ui-flextab
rocketchat:ui-login
rocketchat:ui-master
Expand Down
2 changes: 2 additions & 0 deletions .meteor/versions
View file
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,7 @@ rocketchat:[email protected]
rocketchat:[email protected]
rocketchat:[email protected]
rocketchat:[email protected]
rocketchat:[email protected]
rocketchat:[email protected]
rocketchat:[email protected]
rocketchat:[email protected]
Expand Down Expand Up @@ -234,6 +235,7 @@ rocketchat:[email protected]
rocketchat:[email protected]
rocketchat:[email protected]
rocketchat:[email protected]
rocketchat:[email protected]
rocketchat:[email protected]
rocketchat:[email protected]
rocketchat:[email protected]
Expand Down
35 changes: 0 additions & 35 deletions packages/rocketchat-api/server/v1/channels.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -80,41 +80,6 @@ RocketChat.API.v1.addRoute('channels.archive', { authRequired: true }, {
}
});

/**
DEPRECATED
// TODO: Remove this after three versions have been released. That means at 0.67 this should be gone.
**/
RocketChat.API.v1.addRoute('channels.cleanHistory', { authRequired: true }, {
post() {
const findResult = findChannelByIdOrName({ params: this.requestParams() });

if (!this.bodyParams.latest) {
return RocketChat.API.v1.failure('Body parameter "latest" is required.');
}

if (!this.bodyParams.oldest) {
return RocketChat.API.v1.failure('Body parameter "oldest" is required.');
}

const latest = new Date(this.bodyParams.latest);
const oldest = new Date(this.bodyParams.oldest);

let inclusive = false;
if (typeof this.bodyParams.inclusive !== 'undefined') {
inclusive = this.bodyParams.inclusive;
}

Meteor.runAsUser(this.userId, () => {
Meteor.call('cleanChannelHistory', { roomId: findResult._id, latest, oldest, inclusive });
});

return RocketChat.API.v1.success(this.deprecationWarning({
endpoint: 'channels.cleanHistory',
versionWillBeRemove: 'v0.67'
}));
}
});

RocketChat.API.v1.addRoute('channels.close', { authRequired: true }, {
post() {
const findResult = findChannelByIdOrName({ params: this.requestParams(), checkedArchived: false });
Expand Down
2 changes: 1 addition & 1 deletion packages/rocketchat-api/server/v1/rooms.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@ RocketChat.API.v1.addRoute('rooms.cleanHistory', { authRequired: true }, {
}

Meteor.runAsUser(this.userId, () => {
Meteor.call('cleanRoomHistory', { roomId: findResult._id, latest, oldest, inclusive });
Meteor.call('cleanRoomHistory', { roomId: findResult._id, latest, oldest, inclusive, limit: this.bodyParams.limit, excludePinned: this.bodyParams.excludePinned, filesOnly: this.bodyParams.filesOnly, fromUsers: this.bodyParams.users });
});

return RocketChat.API.v1.success();
Expand Down
3 changes: 2 additions & 1 deletion packages/rocketchat-authorization/server/startup.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Meteor.startup(function() {
{ _id: 'create-d', roles : ['admin', 'user', 'bot'] },
{ _id: 'create-p', roles : ['admin', 'user', 'bot'] },
{ _id: 'create-user', roles : ['admin'] },
{ _id: 'clean-channel-history', roles : ['admin'] }, // special permission to bulk delete a channel's mesages
{ _id: 'clean-channel-history', roles : ['admin'] },
{ _id: 'delete-c', roles : ['admin', 'owner'] },
{ _id: 'delete-d', roles : ['admin'] },
{ _id: 'delete-message', roles : ['admin', 'owner', 'moderator'] },
Expand All @@ -32,6 +32,7 @@ Meteor.startup(function() {
{ _id: 'edit-other-user-password', roles : ['admin'] },
{ _id: 'edit-privileged-setting', roles : ['admin'] },
{ _id: 'edit-room', roles : ['admin', 'owner', 'moderator'] },
{ _id: 'edit-room-retention-policy', roles : ['admin'] },
{ _id: 'force-delete-message', roles : ['admin', 'owner'] },
{ _id: 'join-without-join-code', roles : ['admin', 'bot'] },
{ _id: 'leave-c', roles : ['admin', 'user', 'bot', 'anonymous'] },
Expand Down
118 changes: 113 additions & 5 deletions packages/rocketchat-channel-settings/client/views/channelSettings.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -117,12 +117,11 @@
</div>
{{/with}}


{{#with settings.reactWhenReadOnly}}
{{#if canView}}
<div class="rc-user-info__row rc-user-info__row--separator">
<div class="rc-switch-double">
<div class="rc-switch-double__label {{equal true value 'disabled'}}">
<div class="rc-switch-double__label {{equal false value 'disabled'}}">
{{_ "React_when_read_only"}}
<div class="rc-switch-double__description">
{{_ "React_when_read_only"}}
Expand All @@ -136,7 +135,7 @@
</span>
</label>
</div>
<div class="rc-switch-double__label {{equal false value 'disabled'}}">
<div class="rc-switch-double__label {{equal true value 'disabled'}}">
{{_ "Disallow_reacting"}}
<div class="rc-switch-double__description">
{{_ "Disallow_reacting_Description"}}
Expand Down Expand Up @@ -164,7 +163,7 @@
{{/if}}
{{/with}}

{{#with settings.sysMes}}
{{#with settings.sysMes}}
{{#if canView}}
<div class="rc-user-info__row">
<div class="rc-switch rc-switch--blue">
Expand Down Expand Up @@ -200,7 +199,7 @@
{{/if}}
{{/with}}
{{#with settings.joinCode}}
<div class="rc-user-info__row">
<div class="rc-user-info__row rc-user-info__row--separator">
<div class="rc-input">
<label class="rc-input__label">
<div class="rc-input__title">{{_ label}}{{equal default value '*'}}</div>
Expand All @@ -211,6 +210,93 @@
</div>
</div>
{{/with}}

{{#if hasRetentionPermission}}
<div class="rc-user-info__config">
<div class="rc-user-info__config-header">
{{> icon block="rc-user-info__config-icon" icon="trash"}}
<span class="rc-user-info__config-label">{{_ "Prune"}}</span>
</div>
{{#with settings.retentionEnabled}}
<div class="rc-user-info__config-content">
<div class="rc-user-info__config-name">{{_ label}}:</div>
<div class="rc-user-info__config-value">
{{subValue value}} {{> icon block="rc-user-info__config-content-icon" icon="arrow-down"}}
</div>
</div>
{{/with}}
</div>
{{# if retentionEnabled settings.retentionEnabled.value.get }}
{{#with settings.retentionOverrideGlobal}}
<div class="rc-user-info__row">
<div class="rc-switch rc-switch--blue">
<label class="rc-switch__label">
<span class="rc-switch__text">
{{_ label}}{{equal default value '*'}}
</span>
<input type="checkbox" class="rc-switch__input js-input-check" name="retentionOverrideGlobal" checked="{{checked}}" disabled="{{./disabled}}">
<span class="rc-switch__button">
<span class="rc-switch__button-inside"></span>
</span>
</label>
</div>
</div>
{{/with}}
{{/if}}
{{# if settings.retentionOverrideGlobal.value.get }}
<div class="mail-messages__instructions mail-messages__instructions--warning" style="margin-bottom: 0;">
<div class="mail-messages__instructions-wrapper">
<div class="mail-messages__instructions-text">
<span>
{{{_ "RetentionPolicyRoom_ReadTheDocs"}}}
</span>
</div>
</div>
</div>
{{#with settings.retentionMaxAge}}
<div class="rc-user-info__row">
<div class="rc-input">
<label class="rc-input__label">
<div class="rc-input__title">{{retentionMaxAgeLabel label}}{{equal default value '*'}}</div>
<div class="rc-input__wrapper">
<input type="number" name="retentionMaxAge" value="{{value}}" class="rc-input__element js-input" disabled="{{./disabled}}"/>
</div>
</label>
</div>
</div>
{{/with}}
{{#with settings.retentionExcludePinned}}
<div class="rc-user-info__row">
<div class="rc-switch rc-switch--blue">
<label class="rc-switch__label">
<span class="rc-switch__text">
{{_ label}}{{equal default value '*'}}
</span>
<input type="checkbox" class="rc-switch__input js-input-check" name="retentionExcludePinned" checked="{{checked}}" disabled="{{./disabled}}">
<span class="rc-switch__button">
<span class="rc-switch__button-inside"></span>
</span>
</label>
</div>
</div>
{{/with}}
{{#with settings.retentionFilesOnly}}
<div class="rc-user-info__row">
<div class="rc-switch rc-switch--blue">
<label class="rc-switch__label">
<span class="rc-switch__text">
{{_ label}}{{equal default value '*'}}
</span>
<input type="checkbox" class="rc-switch__input js-input-check" name="retentionFilesOnly" checked="{{checked}}" disabled="{{./disabled}}">
<span class="rc-switch__button">
<span class="rc-switch__button-inside"></span>
</span>
</label>
</div>
</div>
{{/with}}
{{/if}}
{{/if}}
</div>
<div class="rc-user-info__row">
<div class="rc-user-info__flex rc-user-info__row rc-user-info__row--separator">
Expand Down Expand Up @@ -269,6 +355,28 @@ <h3 title="{{name}}" class="rc-user-info__name">{{> icon block="rc-header__icon"
<b>{{_ "Broadcast_channel"}}:</b> {{_ "Broadcast_channel_Description"}}
</label>
{{/if}}
{{#if hasPurge}}
<div class="mail-messages__instructions mail-messages__instructions--warning">
<div class="mail-messages__instructions-wrapper">
{{> icon block="mail-messages__instructions-icon" icon="warning-empty"}}
<div class="mail-messages__instructions-text">
{{#unless filesOnly}}
{{#unless excludePinned}}
{{_ "RetentionPolicy_RoomWarning" purgeTimeout}}
{{else}}
{{_ "RetentionPolicy_RoomWarning_Unpinned" purgeTimeout}}
{{/unless}}
{{else}}
{{#unless excludePinned}}
{{_ "RetentionPolicy_RoomWarning_FilesOnly" purgeTimeout}}
{{else}}
{{_ "RetentionPolicy_RoomWarning_UnpinnedFilesOnly" purgeTimeout}}
{{/unless}}
{{/unless}}
</div>
</div>
</div>
{{/if}}
{{/with}}
{{#each channelSettings}}
<div class="rc-user-info__row">
Expand Down
Loading

0 comments on commit 829c5d1

Please sign in to comment.