Understanding Django’s system of handling static and media files can be a complex topic. Add a connection to S3 and django-compressor and things become downright difficult.
I’ll share with you a working configuration that uses multiple buckets to serve files; a private bucket for media files and a public bucket for static files.
Setting up your Requirements
In requirements.txt add:
django==3.0.4
django-storages[boto3]==1.9.1 # https://github.com/jschneier/django-storages
django-compressor==2.4 # https://github.com/django-compressor/django-compressorYou’ll notice that the Django version is 3.0.4. I’m sure you can get this to work in other versions of Django, but this is the version I’m working with to get this running.
Configuring Settings
In your settings.py file:
Make sure you include "django.contrib.staticfiles" in your INSTALLED_APPS.
# django-compressor
# ------------------------------------------------------------------------------
# https://django-compressor.readthedocs.io/en/latest/quickstart/#installation
INSTALLED_APPS += ["compressor"]
STATICFILES_FINDERS += ["compressor.finders.CompressorFinder"]
# ######### COMPRESSOR CONFIG
COMPRESS_CSS_FILTERS = [
'compressor.filters.css_default.CssAbsoluteFilter',
'compressor.filters.cssmin.rCSSMinFilter'
]
COMPRESS_JS_FILTERS = [
'compressor.filters.jsmin.JSMinFilter'
]
# ######### END COMPRESSOR CONFIG
# STORAGES
# ------------------------------------------------------------------------------
# https://django-storages.readthedocs.io/en/latest/#installation
INSTALLED_APPS += ["storages"] # noqa F405
# https://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html#settings
AWS_ACCESS_KEY_ID = env("DJANGO_AWS_ACCESS_KEY_ID", default="")
# https://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html#settings
AWS_SECRET_ACCESS_KEY = env("DJANGO_AWS_SECRET_ACCESS_KEY", default="")
# https://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html#settings
AWS_STORAGE_BUCKET_NAME = env("DJANGO_AWS_STORAGE_BUCKET_NAME", default='')
AWS_MEDIA_STORAGE_BUCKET_NAME = env("DJANGO_AWS_STORAGE_BUCKET_NAME", default='')
# https://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html#settings
AWS_QUERYSTRING_AUTH = False
# DO NOT change these unless you know what you're doing.
_AWS_EXPIRY = 60 * 60 * 24 * 7
# https://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html#settings
AWS_S3_OBJECT_PARAMETERS = {
"CacheControl": f"max-age={_AWS_EXPIRY}, s-maxage={_AWS_EXPIRY}, must-revalidate"
}
# https://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html#settings
AWS_DEFAULT_ACL = None
# https://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html#settings
AWS_S3_REGION_NAME = env("DJANGO_AWS_S3_REGION_NAME", default=None)
# STATIC
# ------------------------
STATICFILES_STORAGE = 'config.storages.CachedS3Boto3Storage'
COLLECTFAST_STRATEGY = "collectfast.strategies.boto3.Boto3Strategy"
STATIC_URL = f"https://{AWS_STORAGE_BUCKET_NAME}.s3.amazonaws.com/static/"
DEFAULT_FILE_STORAGE = "config.storages.MediaCachedS3Boto3Storage"
# django-compressor
# ------------------------------------------------------------------------------
# https://django-compressor.readthedocs.io/en/latest/settings/#django.conf.settings.COMPRESS_ENABLED
COMPRESS_ENABLED = env.bool("COMPRESS_ENABLED", default=True)
COMPRESS_OFFLINE = env.bool("COMPRESS_OFFLINE", default=True)
# https://django-compressor.readthedocs.io/en/latest/settings/#django.conf.settings.COMPRESS_STORAGE
COMPRESS_STORAGE = 'config.storages.CachedS3Boto3Storage'
# https://django-compressor.readthedocs.io/en/latest/settings/#django.conf.settings.COMPRESS_URL
COMPRESS_URL = STATIC_URL
COMPRESS_ROOT = STATIC_ROOT
# MEDIA
# ------------------------------------------------------------------------------
MEDIA_URL = f"https://{AWS_MEDIA_STORAGE_BUCKET_NAME}.s3.amazonaws.com/media/"
You’ll notice a bunch of calls to env(SOME_SETTING, default=''). These settings must be set in your local environment or your application will throw and IMPROPERLY CONFIGURED error. You can either set them in a .env file and install something like dotenv, or you can add them directly in the os. This follows the 12 Factor App methodology and keeps your private settings out of your settings files.
Project Structure
You’ll also notice a couple references to config.storages.xxx. My structure for this project is as follows:project_folder
-config
-settings
__init__.py
storages.py
-requirements
-base.txt
-production.txt
-apps
Configuring Storages
Here’s what needs to be in storages.py:
from django.conf import settings
from django.core.files.storage import get_storage_class
from storages.backends.s3boto3 import S3Boto3Storage
class CachedS3Boto3Storage(S3Boto3Storage):
"""
S3 storage backend that saves the files locally, too.
"""
location = "static"
default_acl = "public-read"
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.local_storage = get_storage_class("compressor.storage.CompressorFileStorage")()
def save(self, name, content, max_length=100):
self.local_storage._save(name, content)
super().save(name, self.local_storage._open(name))
return name
class MediaCachedS3Boto3Storage(CachedS3Boto3Storage):
location = "media"
file_overwrite = False
bucket_name = settings.AWS_MEDIA_STORAGE_BUCKET_NAME
default_acl = 'private'
bucket_acl = 'private'
Amazon S3 Bucket Configuration
At this point, most of the configuration is done. There’s only a few pieces left, and they need to be done inside your S3 bucket.
Public Bucket
Create a new bucket (or go to the bucket you’re going to use for your public static files). You’ll want to make sure that you’ve got “Block all public access” turned OFF.
Next you’ll want to navigate to the “Bucket Policy” tab and insert the following:
{
"Version": "2008-10-17",
"Id": "http better policy",
"Statement": [
{
"Sid": "readonly policy",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::your_bucket_name/*"
}
]
}
Make sure to replace “your_bucket_name” with the name of your public bucket. This will give full read access to every object in your bucket. Make sure this is what you want to do before continuing. If this is really only for your static files (css, js, images on the site), and nothing else, then you should be fine.
We have one more configuration to complete for this bucket. Click over to the CORS Configuration tab and enter the following:
<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedOrigin>https://yourdomainname.com</AllowedOrigin>
<AllowedMethod>GET</AllowedMethod>
<MaxAgeSeconds>3000</MaxAgeSeconds>
<AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>
Make sure you replace “yourdomainname.com” with the public domain name you plan to use to serve your fonts from. The main purpose of this bit of configuration is to allow S3 to serve your fonts from a different domain than your primary domain (e.g. s3.amazonaws.com)
Private Bucket
Next we need to configure your private bucket. Create a new bucket (or go to the bucket you plan to use for your private media files). The “Block public access” section will look a little different for this:
Basically, we are blocking access to buckets and objects granted through new or any public bucket or access point policy. We are NOT blocking access through ACLs.
Finally, we need to configure is the Access Control List.
Your ACL should have NO ACCESS for anyone except the bucket owner. Full rights should be given to the owner. This allows your server to read/write/delete files from this bucket.
Your bucket policy tab should be empty, and your CORS configuration should be the same as your public bucket (or you could leave this blank since you likely won’t be serving any files from this location publicly).
Final Steps
Make sure you run a collectstatic on your server to push all your static files up to S3. You should be good to go now.
