Deleting Chat Room Logs +30 day crontab

[waveyuk]

I've looked to see if anyone has come up with a solution for this.
I would like to be able to run a crontab job which deletes any messages from any room which are older than 30 days. Is this at all possible? My concern is that the Mongo database will quickly fill with messages and cause the server to run out of disk space on active channels. I've seen posts about using the API but it isn't clearly explained how to achieve this. Our Rocket Chat is running in a Snap environment.
Thanks

Rocket.Chat Version: 0.56.0
Running Instances: 1
DB Replicaset OpLog: Enabled
Node Version: 4.8.1

[graywolf336]

You could possibly use this api: https://rocket.chat/docs/developer-guides/rest-api/channels/cleanhistory

[waveyuk]

@graywolf336 that was the API I was referencing in the post. However, what is not clear is on the parameters. For example, where do you get the X-Auth-Token from (or how do you generate it). I assume the X-User-Id would be the system admin ID, but again, how do you find that out?

[graywolf336]

@waveyuk Take a look at how to authenticate: https://rocket.chat/docs/developer-guides/rest-api/authentication

[waveyuk]

@graywolf336 ok- seen that... unfortunately, I am not a whizz with linux and curl, so don't know how to build a script that could pass the authentication results into the command line to accomplish this. If anyone can help, would appreciate it.

[mostlycoolmark]

Here's a sample script to get you started. You probably want to iterate through ALL rooms, though, right?
rocketchat_delete_old_messages.txt
@waveyuk

[waveyuk]

Hi @geesper
Thanks for that. You'll have to excuse me, as I am not very tech savvy when it comes to Linux (I am more of a Windows man!!).
Firstly... How would I run that script? What do I need to be aware of if I am running Rocket.Chat in a Snap environment?
Secondly, is there a way to iterate through ALL rooms, including Direct Messages, programatically? Obviously I could use your script to clean an individual room by specifying it, but not so easy for cleaning Direct Messages. Is this even possible?
Really appreciate your help @geesper
Dave

[mostlycoolmark]

powershell_remove_old_messages.txt
Unfortunately, it doesn't look like there's a way through the api to query for direct messages. It's doable directly in the database, but I'm not inclined to go down that route.

We can definitely get a list of the public rooms and then go through them all, though. I could spend a few minutes and put this into powershell, if that makes more sense to you than a shell script.

edit: I've added a powershell script that iterates through all channels and deletes messages older than X days. Please test this before using in production!

[mostlycoolmark]

Actually, looks like there may be a way to get a list of all direct messages? https://rocket.chat/docs/developer-guides/rest-api/im/list-everyone

I'll take a closer look at that.

[graywolf336]

@geesper I meant to respond earlier and say there was, but it is disabled by default due to security and privacy purposes.

[waveyuk]

@geesper ... Thanks for the powershell script. I'll give that a go as soon as my internet connection returns to being stable (having issues with Virgin Media broadband at the moment).

@graywolf336 is there any way to get around the disabled function to delete direct messages? Obviously as server admin I would like to keep my server as clean as possible and dispose of old, out of date direct messages that consume database space but are no longer being used.

Thanks guys
Dave

[waveyuk]

@geesper
Is there a way to make the bin/bash script you created for me iterate through all the public chat rooms on the server? I've been doing some research and think I've found a way to run this. It would be better than running it via PowerShell as I wouldn't have to open up port 3000 on the firewall, as I could run it directly on the server as a cronjob.
Thanks for all your help
Dave

[mostlycoolmark]

rocketchat_delete_old_messages.txt

OK, here's an updated bash script that will iterate through all channels. Unfortunately, while the api allows you to grab a list of the direct message rooms, the api doesn't allow you to clean their history.

The script should be easy to update later if/when that functionality is added.

IMPORTANT NOTE: You will need to install jq on the linux machine you're running this from. This is used to parse the json response that the api gives back. Let me know if you run into any problems! Please be sure to run this on a test server before using it in production!

[graywolf336]

@geesper if you really wanted to, you could get a list of the messages in a direct room and call chat.delete https://rocket.chat/docs/developer-guides/rest-api/chat/delete I know it's not the best, but it can make do right now 🤔

[mostlycoolmark]

Yeah, I don't really want to do that because of how inefficient it would be.

We'd have to:

• get a list of the direct message rooms from im.list.everyone https://rocket.chat/docs/developer-guides/rest-api/im/list-everyone
• get a list of all the messages from each room im.messages.others https://rocket.chat/docs/developer-guides/rest-api/im/messages-others
• iterate through every message and compare the date in order to chat.delete https://rocket.chat/docs/developer-guides/rest-api/chat/delete

Considering there may be thousands or tens of thousands of messages, that could take a long long time.

My hope is that the channels.cleanHistory allows DM's, or that maybe the ims api gets its own cleanHistory call.

[mnlbox]

Guys any news about this?