Recently, working with s3cmd has been difficult because the command suffers from a lack of documentation regarding the AWS policy requirements the software needs to perform its duties. The following is a policy configuration for S3 and CloudFront that enumerates what I hope are the minimum requirements that s3cmd needs to deploy a static site to S3 while also invalidating CloudFront paths at the same time. I established these policies through trial and error.

A key component is to notice that when assigning the S3 user’s policy, you must ensure that you both allow access to the bucket objects with "arn:aws:s3:::bucket/*", as well as "arn:aws:s3:::bucket" (note the first example has a trailing ‘/*’ and the following doesn’t) - for some reason this is necessary for the --cf-invalidate option to work with s3cmd, and was the most diffuclt part in getting this to work correctly.

The resultant command to publish the website with Pelican’s Makefile is make s3_upload and now looks as follows:

s3_upload: publish
        s3cmd sync $(OUTPUTDIR)/ s3://$(S3_BUCKET) --acl-public --delete-removed \
        --guess-mime-type --no-mime-magic --no-preserve --cf-invalidate

S3 Tangent: --no-mime-magic is not a default option either, it is required in order to avoid a broken libmagic library setting the wrong MIME header in s3. If you don’t use this, you’ll be serving CSS content as text and your website will be broken. Additionally, --no-preserve is important because s3cmd was built with file preservation in mind, and will set a custom header for each object that stores sensitive information, such as UID and GID of the uploader. This is beneficial when archiving files (because it acts more like tar), but not so useful when trying to serve a website.


IAM Policies:

CloudFront policy configuration:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1446510379000",
            "Effect": "Allow",
            "Action": [
                "cloudfront:CreateInvalidation",
                "cloudfront:GetCloudFrontOriginAccessIdentity",
                "cloudfront:GetCloudFrontOriginAccessIdentityConfig",
                "cloudfront:GetDistribution",
                "cloudfront:GetDistributionConfig",
                "cloudfront:GetInvalidation",
                "cloudfront:ListDistributions",
                "cloudfront:ListInvalidations"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}
Reminder: There are no ARNs for CloudFront, simply use ‘*’

S3 Bucket policy configuration:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1446509955000",
            "Effect": "Allow",
            "Action": [
                "s3:DeleteObject",
                "s3:GetBucketLocation",
                "s3:GetBucketWebsite",
                "s3:GetObject",
                "s3:GetObjectAcl",
                "s3:ListBucket",
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::bucket/*",
                "arn:aws:s3:::bucket"
            ]
        }
    ]
}

- Mike