ACK Queue: clear_acked_data() behavior

[employee-1234567]

Playing around with the clear_acked_data() function, it seems to hang on to the last 1000 acked queue items. Why is that? I've already acked the data, yet disk space continued to be used.

Looking at the code in question:

    @sqlbase.with_conditional_transaction
    def clear_acked_data(self):
        sql = """DELETE FROM {table_name}
            WHERE {key_column} IN (
                SELECT _id FROM {table_name} WHERE status = ?
                ORDER BY {key_column} DESC
                LIMIT 1000 OFFSET {max_acked_length}
            )""".format(table_name=self._table_name,
                        key_column=self._key_column,
                        max_acked_length=self._MAX_ACKED_LENGTH)
        return sql, AckStatus.acked

It seems that self._MAX_ACKED_LENGTH is a private member constant. Can this not be made tunable by the user (e.g.. a kwarg in init for the class)?

I opened my resulting sqlite data files and manually ran:

DELETE FROM ack_queue_default WHERE status = 5;
VACUUM;

Which reduced the file size by several GB. Unless there is some edge case, surely you'd want to do something more like this?

    @sqlbase.with_conditional_transaction
    def clear_acked_data(self):
        sql = """DELETE FROM {table_name} WHERE status = ?""".format(table_name=self._table_name)
        return sql, AckStatus.acked

    @sqlbase.with_conditional_transaction
    def shrink_disk_usage(self):
        sql = """VACUUM"""
        return sql, None

[peter-wangxu]

@employee-1234567 do you mind providing a PR based on you mentioned above? I am pleased if you can do that.

Thanks
Peter

[employee-1234567]

Sure, I'd be happy to. Before I make the PR, can you confirm if the propsed SQL is correct? I still feel like I'm missing something or I didn't understand the original intent. Is there a reason the existing code is keeping 1000 ACK'd items around?

[peter-wangxu]

@employee-1234567 please go ahead with your GH-82, the comment #82 (comment) had confirmed it's fine to proceed.

Thanks
Peter

[dcj]

Doesn't seem like @employee-1234567 is going to revise the PR.
Nevertheless, a PR was submitted which is only stylistically incompatible.
Any chance this fix could revised/committed?