Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Also cleanup version metadata if expiring #39786

Merged
merged 1 commit into from
Aug 14, 2023

Conversation

juliusknorr
Copy link
Member

@juliusknorr juliusknorr commented Aug 10, 2023

Found while writing e2e tests for #39140

Steps to reproduce:

  • Upload some file multiple times with close timestamps
  • Run cron which expires versions (ref
    private static $max_versions_per_interval = [
    //first 10sec, one version every 2sec
    1 => ['intervalEndsAfter' => 10, 'step' => 2],
    //next minute, one version every 10sec
    2 => ['intervalEndsAfter' => 60, 'step' => 10],
    //next hour, one version every minute
    3 => ['intervalEndsAfter' => 3600, 'step' => 60],
    //next 24h, one version every hour
    4 => ['intervalEndsAfter' => 86400, 'step' => 3600],
    //next 30days, one version per day
    5 => ['intervalEndsAfter' => 2592000, 'step' => 86400],
    //until the end one version per week
    6 => ['intervalEndsAfter' => -1, 'step' => 604800],
    ];
    )

This leads to the version files being deleted while the oc_files_versions table still contains references, meaning that they show up as broken files in the versions sidebar.

Ideally expiration should also be handled by the version manager, but this requires some larger efforts and is not backportable.

Quick script to reproduce:

#!/bin/bash
NC_URL=https://admin:[email protected]/remote.php/webdav/

filename=$(date | md5sum | cut -d " " -f 1)
echo $filename

echo a1 > 1/$filename.md
sleep 2
echo a2 > 2/$filename.md
sleep 2
echo a3 > 3/$filename.md
sleep 2
echo a4 > 4/$filename.md
sleep 2
echo a5 > 5/$filename.md

curl -T "1/$filename.md" "$NC_URL/$filename.md"
curl -T "2/$filename.md" "$NC_URL/$filename.md"
curl -T "3/$filename.md" "$NC_URL/$filename.md"
curl -T "4/$filename.md" "$NC_URL/$filename.md"
curl -T "5/$filename.md" "$NC_URL/$filename.md"

Afterwards you can compare the following sql queries before and after cron for validation:

# with the echoed filename
select * from oc_filecache where name like '3882259fffd1ef36e300511488e561be%';
# with the file id
select * from oc_files_versions where file_id = 8740;

Checklist

@juliusknorr juliusknorr requested review from a team, ArtificialOwl, icewind1991, nfebe and artonge and removed request for a team August 10, 2023 07:29
@juliusknorr juliusknorr force-pushed the bugfix/version-expire-cleanup branch from 26c7cda to e3bdb72 Compare August 10, 2023 07:31
@juliusknorr juliusknorr added bug 3. to review Waiting for reviews labels Aug 10, 2023
@juliusknorr juliusknorr added this to the Nextcloud 28 milestone Aug 10, 2023
@juliusknorr
Copy link
Member Author

/backport to stable27

@juliusknorr
Copy link
Member Author

/backport to stable26

@juliusknorr juliusknorr force-pushed the bugfix/version-expire-cleanup branch from e3bdb72 to bbb7172 Compare August 14, 2023 17:31
@solracsf
Copy link
Member

@juliushaertl no 25 ?

@juliusknorr
Copy link
Member Author

Only 26+ is affected where the new table for metadata was introduced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3. to review Waiting for reviews bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants