django - collectstatic incorrectly creates multiple CSS files in S3

Question

I have uploading files to S3 working fine with my Wagtail/django application (both static and uploads). Now I'm trying to use ManifestStaticFilesStorage to enable cache busting. The urls are correctly being generated by the application and files are being copied with hashes to S3.

But each time I run collectstatic some files get copied twice to S3 - each with a different hash. So far the issue is ocurring for all CSS files.

file.a.css is loaded by the application and is the file referenced in staticfiles.json - however it is a 20.0B file in S3 (should be 6.3KB).

file.b.css has the correct contents in S3 - however it does NOT appear in the output generated by collectstatic.

# custom_storages.py
from django.conf import settings
from django.contrib.staticfiles.storage import ManifestFilesMixin
from storages.backends.s3boto import S3BotoStorage


class CachedS3Storage(ManifestFilesMixin, S3BotoStorage):
    pass


class StaticStorage(CachedS3Storage):
    location = settings.STATICFILES_LOCATION


class MediaStorage(S3BotoStorage):
    location = settings.MEDIAFILES_LOCATION
    file_overwrite = False

Deps:

"boto==2.47.0",
"boto3==1.4.4",
"django-storages==1.5.2"
"Django==2.0.8"

Any pointers on where to look to track down this issue would be appreciated! :)

Edit:

Looking more carefully at all the files copied to S3 the issue is ONLY occurring for CSS files.

Disabling pushing assets to S3 and writing them to the local filesystem works as expected.

Edit 2:

Updated all the deps to the latest version - same behavior as above.

Answer 1

I eventually stumbled across this issue in django-storages issue tracker which then lead me to a very similar question on SO.

Between these two pages I managed to resolve the issue. I did the following to get django-storages + ManifestStaticFilesStorage + S3 to work together:

# custom_storages.py
from django.conf import settings
from django.contrib.staticfiles.storage import ManifestFilesMixin
from storages.backends.s3boto3 import S3Boto3Storage  # note boto3!!


class PatchedS3StaticStorage(S3Boto3Storage):
    def _save(self, name, content):
        if hasattr(content, 'seek') and hasattr(content, 'seekable') and content.seekable():
            content.seek(0)
        return super()._save(name, content)


class CachedS3Storage(ManifestFilesMixin, PatchedS3StaticStorage):
    pass


class StaticStorage(CachedS3Storage):
    location = settings.STATICFILES_LOCATION


class MediaStorage(S3Boto3Storage):
    location = settings.MEDIAFILES_LOCATION
    file_overwrite = False

Note that I had to use boto3 to get this to work django-storages must be >= 1.5 to use boto3. I removed boto as a dep. My final deps were:

"boto3==1.4.4",
"django-storages==1.7.1"
"Django==2.0.8"