Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Deleting old messages? #6749

Closed
WebSavvyDude opened this issue Apr 21, 2017 · 5 comments · Fixed by #11236
Closed

Deleting old messages? #6749

WebSavvyDude opened this issue Apr 21, 2017 · 5 comments · Fixed by #11236
Labels
feat: message type: support

Comments

@WebSavvyDude
Copy link

WebSavvyDude commented Apr 21, 2017 •
edited
Loading

Rocket.Chat Version: 0.55.0 rc-6
Running Instances: 1
DB Replicaset OpLog:
Node Version: 4.5.0

Hi

Is there a way to delete (prune) old messages automatically? Ideally i would only want something like the last 50 messages posted to the room to be saved and viewable for any particular user.

I am rather new to Rocket.chat and did not find this function. It seems to store the entire history since the beginning and that would take up a lot of disk space if left on long enough.

Thanks
@WebSavvyDude
Copy link
Author

Just found a similar request #795 but no progress for a couple years.

@JSzaszvari
Copy link
Contributor

@WebSavvyGuy - You could easily script it with the API to remove all messages older than lets say 30 days using the cleanHistory method. That would at least limit the amount of messages stored. It's not 50 messages but better than nothing :)

https://rocket.chat/docs/developer-guides/rest-api/channels/cleanhistory

Just set 'latest' as 30 days ago and 'oldest' as a date from 1 year ago, then run that every evening so only 30 days worth of messages are kept/displayed.

I guess you could use the API to look up all rooms then loop through them with the cleanHistory method with a bit of scripting magic.

@JSzaszvari
Copy link
Contributor

JSzaszvari commented Apr 21, 2017 •
edited
Loading

Although the disk space usage should be minimal, we just hit 80,500 messages total and are using 750mb disk space in Mongo - So I'm not sure I'd worry about the last 50 messages....

screen shot 2017-04-22 at 3 54 23 am

screen shot 2017-04-22 at 3 52 55 am

If I do the math thats roughly 100 messages per 1MB.

@WebSavvyDude
Copy link
Author

@JSzaszvari Thanks for the reply. We will look into something like that. We run a popular website that we might eventually migrate over into Rocket.chat. We probably would get well over those numbers you have each DAY, so that is why were looking for something that will prune messages down. The last 50 number is just kind of like what our current chat application does. It only shows (caches) the last 20 messages posted in the main room. Private messages are totally deleted if the user leaves the room. A system like that would better work for us.

@vynmera vynmera mentioned this issue Jun 22, 2018
Merged
42 tasks
@vynmera
Copy link
Contributor

vynmera commented Jun 25, 2018

I'm making it happen! ---> #11236

ggazzo pushed a commit that referenced this issue Jul 20, 2018
@vynmera @ggazzo
[NEW][BREAK] Message retention policy and pruning (#11236)
829c5d1 
Closes #6749
Closes #8321
Closes #9374
Closes #2700
Closes #2639
Closes #2355 
Closes #1861
Closes #8757
Closes #7228
Closes #10870
Closes #6193 
Closes #11299
Closes #11468
Closes #9317
Closes #11300 (will incorporate a fix to this PR's issue)
Closes #11046 (will incorporate a fix to this PR's issue)
Contributes to #5944 
Contributes to #11475
_...and possibly more!_

This PR makes deleting messages (automatically and manually) a lot easier on Rocket.Chat.

- [X] Implement a bulk message deletion notification, to quickly push large message deletions to users without reload
  - [X] Use it in `rooms.cleanHistory`
  - [X] Use it in user deletions
- [X] Completely remove cleanChannelHistory as required by v0.67
  - [X] Remove server method `cleanChannelHistory`
  - [X] Remove REST API `channels.cleanHistory`
- [x] Implement a sidebar option to clean history
  - [x] Basic implementation
  - [x] Allow excluding pinned messages
  - [x] Allow attachment-only mode
  - [x] Allow specifying user(s) to narrow down to
    - [x] Also update REST API
    - [x] Also update docs
  - [x] Break the deletion into multiple different requests, so the client can keep track of progress
  - [x] Clear animation / progress bar for deleting
- [x] Retention policy
  - [X] Global, set by admin
    - [X] Global timer that runs every second and deletes messages over the set limit
      - [X] Can change its timer's resolution to prevent insane CPU overhead
    - [X] Admin can decide what room types to target (channels, groups and/or DMs)
    - [X] Allow excluding pinned messages
    - [X] Allow attachment-only mode
  - [x] Per-channel, set by those with a new permission
    - [x] Disabled when master switch off
    - [x] Set in channel info
    - [x] Can override global policy with a switch that requires `edit-privileged-setting`
    - [x] Allow excluding pinned messages
    - [x] Allow attachment-only mode
    - [x] Uses same global timer for cleanup
  - [X] Message at start of channel history / in channel info if there is a retention policy set
  - [x] Message in channel info if there is a retention policy set on that channel specifically
- [X] Make cleaning history also delete files (completely!)
  - [X] Manual purging
  - [X] Automatic purging
- [x] Make other deletions also delete files
  - [x] User deletion
    - [X] Own messages
    - [x] DMs with them's partner messages
  - [x] Room deletion
- [x] Cleanup
- [x] Finish related [docs](https://github.com/RocketChat/docs/pull/815)
- [x] Link to the docs in the settings

Please suggest any cool changes/additions! Any support is greatly appreciated.

**Breaking change:** This PR removes REST API endpoint `channels.cleanHistory` and Meteor callable `cleanChannelHistory` as per the protocol specified for them.

![bzzzzzzzz](https://user-images.githubusercontent.com/39674991/41799087-56d1dea0-7670-11e8-94c0-bc534b1f832d.png)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feat: message type: support
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[NEW] Message retention policy and pruning vynmera/Rocket.Chat
4 participants
@JSzaszvari @MartinSchoeler @WebSavvyDude @vynmera