collectstatic can overwrite files it skipped, resulting in different behaviour in development and production

[jaap3]

Given the following settings:

STATICFILES_FINDERS = (
    'staticfiles.finders.FileSystemFinder',
    'staticfiles.finders.AppDirectoriesFinder',
)

If you have two files called css/body.css one in STATIC_DIR/css/body.css and an app with (a different) static/css/body.css.

I would expect that staticfiles.views.serve always serves the same file as after running collectstatic.

In some rare conditions however this does not work. I've been able to reproduce it by doing the following:

First creat css/body.css in the project, no external apps yet, and run collectstatic.
Then I add an external app (to INSTALLED_APPS), that has a newer css/body.css and run collectstatic again.

The file is now copied from the external app, overwriting the one from the project.

It seems to me that copy_file/link_file should also check self.unmodified_files when deciding to skip.

[jaap3]

OK, this is not true, something else is happening resulting it the wrong file being copied. Need to check it out further, will do so on Friday.

[jaap3]

I've checked again and was able to reproduce the issue. I've updated the description to include the steps to reproduce this issue.

[jezdez]

Ah, interesting, that may indeed be a problem of collectstatic. Would you mind writing a small test case for the problem?

[jaap3]

This shows the issue... it's a nasty testcase because it uses sleep to make sure there's a time difference.

class TestCollectionIgnoresSkipped(CollectionTestCase):
    """
    Test consecutive runs of collectstatic where an ignorable file changes.

    For issue Github #19.
    """
    def setUp(self):
        super(CollectionTestCase, self).setUp()
        _stdout = sys.stdout
        sys.stdout = StringIO()
        try:
            call_command('findstatic', 'test/file.txt', verbosity='0')
            sys.stdout.seek(0)
            lines = [l.strip() for l in sys.stdout.readlines()]
        finally:
            sys.stdout = _stdout
        time.sleep(1)
        os.utime(lines[2], None) # 'touch' the app file
        self.run_collectstatic()

    def test_staticfiles_dirs_priority(self):
        """
        File in STATICFILES_DIRS has priority over file in app.
        """
        self.assertFileContains('test/file.txt', 'STATICFILES_DIRS')

[jezdez]

I'm not sure I can follow how this findstatic test case is connected to an alleged issue in collectstatic to be honest.

[jaap3]

Because it shows the issue? It's a subclass of CollectionTestCase that checks if the file that's actually collected is the expected file. If this test fails then the wrong file has been collected.

I'm not able to run tests at this time, but when this test was written it cleary showed the issue. Have you ran it?

The only use of findstatic is to find the location of the lower priority file, so it can be touched. After that it runs collectstatic. The actual test checks the content of the collected file, if it's not the expected content the assertion fails.

Please let me know why you think this is not doing what I think it does.

[zyegfryed]

Hi,
I think i've got a patch to this issue - which is (sadly) embedded to pull-request #17.

The error is triggered when files are post-processed by collectstatic command - which is the case when using CachedStaticFilesStorage - because the last file found is used to the post-processing phase instead of the first one.

@jezdez i tried to make two different pull-requests, but haven't succeeded so far. If you have a solution to do so, don't hesitate to contact me.