Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Server: Automatic Database Trimming #5787

Closed
ChrisM-S opened this issue Oct 8, 2020 · 3 comments
Closed

Server: Automatic Database Trimming #5787

ChrisM-S opened this issue Oct 8, 2020 · 3 comments

Comments

@ChrisM-S
Copy link

ChrisM-S commented Oct 8, 2020

As an IBEX developer or systems maintainer, I can see that about 14 points were spent recently manually trimming space and copying database files. I would like to keep this time for development and other more fruitful maintenance.

Acceptance Criteria

  1. The maximum IBEX database size is controlled by a PV. When the PV in GB is set lower than the size of the database, the database selects and removes the oldest records to ensure it does not exceed the maximum figure.
  2. The process is completed with no significant increase in used disk space (if control is applied in emergency, it is important that it does not become part of the problem itself).
  3. records should be removed at the end of a run or on an hourly basis.

Notes

  1. Choice of records for removal may be something other than first in last out (possibly even controlled by another PV), for example specifying.
@kjwoodsISIS
Copy link
Contributor

The acceptance criteria are defining the solution - that's wrong! The acceptance criteria should define the outcome - the what, not the how.

@FreddieAkeroyd
Copy link
Member

I think deleting rows does not reclaim the space until an optimize tables command is run, but that makes a new copy of the table and then drops the previous table, so temporarily uses the (old+new) space and may take a while. Also during the optimise it may not be able to take any new data, so logging would be disrupted, but i don;t knwo for sure - it may be able to add stuff to the transaction log that can then be played in the new table.

@DominicOram
Copy link
Contributor

I believe this superseded by #5818

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants