Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mimetype list integrity check should not fail if it's changed #15810

Merged
merged 9 commits into from
Jul 7, 2019

Conversation

xh3n1
Copy link
Member

@xh3n1 xh3n1 commented May 30, 2019

Issue
Integrity check failed if mimetype list was changed. The expected behaviour would be that the integrity check would still be passing.

Fixes #10411

Fix
The fix is aiming to still perform some kind of integrity check over the mimetype list. It would still fail for manually changed mimetype lists but will pass for any valid ones.

This is done by:

Regenerating the mimetype list with all mimetype aliases
If the genereated file is same as the one in the disk, then generate the sha512 of the mimetype list file with only default aliases since they are the only ones that are signed.
I have also added "application/internet-shortcut": "link" in resources/config/mimetypealiases.dist.json
Because it's had been manually added in the generated list https://github.com/nextcloud/server/pull/6297/files#diff-6bcc9a99f4c75054294c154bc8e2f490 but not when you generated it via maintenance:mimetype:update-js .

Test plan
Create a mimetypealiases.json file with custom mimetypes and save it in config dir.
Run ./occ maintenance:mimetype:update-js
And then ./occ integrity:check-core
With these changes it should work, without it the integrity check would fail.

PS. I have written the tests for this locally, but I having issues with running PHPUnit locally.

xh3n1 added 4 commits May 29, 2019 22:43
Signed-off-by: Xheni Myrtaj <[email protected]>
Signed-off-by: Xheni Myrtaj <[email protected]>
@xh3n1 xh3n1 requested review from MorrisJobke and rullzer May 30, 2019 09:41
Signed-off-by: Xheni Myrtaj <[email protected]>
@xh3n1 xh3n1 force-pushed the mimetype-integrity-check branch from af34cca to 5fc0477 Compare May 30, 2019 09:50
Signed-off-by: Xheni Myrtaj <[email protected]>
*/
public function generateFile(array $aliases): string {
// Remove comments
$keys = array_filter(array_keys($aliases), function($k) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can improve this a bit. It's possible to pass a flag ARRAY_FILTER_USE_KEY to array_filter. We can omit the foreach-unset below and call the array_filter on $aliases.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As this is code copied over from the old location it's fine to leave it as it is now in this PR and change the code in a separate PR to separate concerns properly. But still a nice finding 👍

@MorrisJobke
Copy link
Member

PS. I have written the tests for this locally, but I having issues with running PHPUnit locally.

Do you mind to add them here as well?

Regarding executing unit tests locally typically ./autotest.sh sqlite tests/lib/IntegrityCheck/CheckerTest.php should work. If not then we can have a look what is not working.

@MorrisJobke MorrisJobke added this to the Nextcloud 17 milestone May 31, 2019
@xh3n1
Copy link
Member Author

xh3n1 commented May 31, 2019

@MorrisJobke Thanks, I had to downgrade my PHPUnit version to make it work and I will push my test later.

Copy link
Member

@MorrisJobke MorrisJobke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beside that the code looks good and makes sense 👍

@MorrisJobke
Copy link
Member

@MorrisJobke Thanks, I had to downgrade my PHPUnit version to make it work and I will push my test later.

Ah ... right. We only run on version 5 or 6 and not on the latest version.

@xh3n1 xh3n1 force-pushed the mimetype-integrity-check branch from e5608d9 to 1d434cb Compare May 31, 2019 21:30
@MorrisJobke
Copy link
Member

Let me rebase to trigger CI again here.

@MorrisJobke MorrisJobke force-pushed the mimetype-integrity-check branch from 1d434cb to 46ffdc4 Compare July 3, 2019 16:29
@MorrisJobke
Copy link
Member

@MorrisJobke Thanks, I had to downgrade my PHPUnit version to make it work and I will push my test later.

I just noticed that the tests are not yet included. Mind to push them? Feel free to overwrite my push, because I just rebased on master and it went through without conflicts.

@xh3n1 xh3n1 force-pushed the mimetype-integrity-check branch from 46ffdc4 to 72bb035 Compare July 3, 2019 21:20
@MorrisJobke MorrisJobke added 3. to review Waiting for reviews and removed 2. developing Work in progress labels Jul 3, 2019
@xh3n1 xh3n1 force-pushed the mimetype-integrity-check branch from f73050d to 1d434cb Compare July 3, 2019 22:49
@skjnldsv
Copy link
Member

skjnldsv commented Jul 4, 2019

Welcome back @xh3n1 👋 :D

Tests failures seems unrelated.

@skjnldsv skjnldsv added 4. to release Ready to be released and/or waiting for tests to finish and removed 3. to review Waiting for reviews labels Jul 4, 2019
xh3n1 added 2 commits July 4, 2019 09:35
Signed-off-by: Xheni Myrtaj <[email protected]>
@xh3n1
Copy link
Member Author

xh3n1 commented Jul 4, 2019

Thanks @skjnldsv 😄
@MorrisJobke Sorry, I have pushed them now. The test was tricky and I asked for help from @LukasReschke 😉

@skjnldsv
Copy link
Member

skjnldsv commented Jul 5, 2019

Failures still seems unrelated :)

@welcome
Copy link

welcome bot commented Jul 7, 2019

Thanks for your first pull request and welcome to the community! Feel free to keep them coming! If you are looking for issues to tackle then have a look at this selection: https://github.com/nextcloud/server/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22
Most developers hang out on IRC. So join #nextcloud-dev on Freenode for a chat!

@delete-merged-branch delete-merged-branch bot deleted the mimetype-integrity-check branch July 7, 2019 18:02
@MorrisJobke
Copy link
Member

@xh3n1 Nice! Thanks for your help on this 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
4. to release Ready to be released and/or waiting for tests to finish enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Integrity fails if mime type list is extended
4 participants