MM-23110 CSV Exporter

[agarciamontoro]

Summary

This PR implements the first version of the CSV exporter.

The structure of the columns is adapted from the existing bulk export feature, removing the unneeded fields in the export of a specific channel. The columns are:

• Post Creation Time
• User Id
• User Email
• User Type ("user", "bot" or "system")
• User Name
• Post Id
• Parent Post Id
• Post Message
• Post Type ("mesage" for normal messages or the specific type of system message, in which case, the User Type is set to "system")

Implementation Details

The Exporter interface is designed so we can add more exporters in the future. It has a couple of functions:

• FileName, that returns the name of the file given the name of the channel (the CSV implements this by just adding the .csv extension to the channel name)
• Export, that receives a function that returns a batch of new posts every time it is called (until there are no more posts in the channel), and a io.Writer, used by the Exporter to write the exported posts.

The architecture of the plugin is as follows:

• The plugin function channelPostsIterator builds an iterator on the posts of the channel (implemented with a simple closure).
• When the /export command is executed, the plugin creates a io.Pipe to concurrently export the channel and upload the file:
  • The pipeWriter is passed to the implementation of the Exporter, along with the channel iterator built with channelPostsIterator.
  • The pipeReader is passed to the UploadFile API function.
• Both the exporting and the uploading are processed in separate go routines, so the user instantly receives an ephemeral post informing them that the export is in process.

Screenshots

After executing the /export command

DM with the bot, once the file is sent

CSV example

Dummy example to see the CSV columns. There is only one user posting, apart from the system, and the first exported post is a reply to the second one.

Post Creation Time,User Id,User Email,User Type,User Name,Post Id,Parent Post Id,Post Message,Post Type
1585568890473,4yqjmq39pbbz3cgq8yzbhfager,[email protected],user,,pzbuuuroyjribmfynre737kdhy,78oczhundty5zrjbohpfhhkjpa,respuesta a hola test hi test hej test,message
1585568853677,4yqjmq39pbbz3cgq8yzbhfager,[email protected],user,,78oczhundty5zrjbohpfhhkjpa,,hola test hi test hej test,message
1585568806234,4yqjmq39pbbz3cgq8yzbhfager,[email protected],user,,tcjjnophdtfhj8gzaaqp5s4rww,,10,message
1585568801307,4yqjmq39pbbz3cgq8yzbhfager,[email protected],user,,mwe31dniwty79j3f3tzyk65z1y,,9,message
1585568800863,4yqjmq39pbbz3cgq8yzbhfager,[email protected],user,,he4chmdhkpdatco83s7p8ujtge,,8,message
1585568800157,4yqjmq39pbbz3cgq8yzbhfager,[email protected],user,,comwmpxff7yedkbw3tpgmj9zke,,7,message
1585568799685,4yqjmq39pbbz3cgq8yzbhfager,[email protected],user,,1sfmde6fnbdpfj9ijyd8d3q66w,,6,message
1585568799182,4yqjmq39pbbz3cgq8yzbhfager,[email protected],user,,4ksad7o1rp8wtjhzn33x848sjr,,5,message
1585568798734,4yqjmq39pbbz3cgq8yzbhfager,[email protected],user,,iqdsdghmfbbg8fhikcyitcomzh,,4,message
1585568798228,4yqjmq39pbbz3cgq8yzbhfager,[email protected],user,,b34sutt6c3gh3fctyk1s7od17w,,3,message
1585568796722,4yqjmq39pbbz3cgq8yzbhfager,[email protected],user,,rmcsdrygff8qidmroia5763kmo,,2,message
1585568794920,4yqjmq39pbbz3cgq8yzbhfager,[email protected],user,,d89whc8uctd4jeeaxcrmac6qdh,,1,message
1585568791028,4yqjmq39pbbz3cgq8yzbhfager,[email protected],system,,hu3s5j5gtf8kirmk4iij79t7me,,sysadmin joined the channel.,system_join_channel

Ticket Link

https://mattermost.atlassian.net/browse/MM-23110

[lieut-data]

Nice! Just a few comments below.

[agarciamontoro]

After a meaningful conversation with @cpoile, I'm reimplementing the users cache to something much simpler, rebuilt in every command and not shared at all. I'll ping you when this is ready, @lieut-data!

[lieut-data]

Looks good!

[marianunez]

Great work @agarciamontoro! 🎉 Non-blocking comment below:

[lieut-data]

@agarciamontoro, propose we merge as-is, and schedule a PM review alongside cutting the "first release". I think that would make this ready to merge if you're ok with that approach.

[agarciamontoro]

Sounds good, yup. Merging!