AWS S3 skill for Codex

Done well means: no bucket is public unless it serves public content through CloudFront, every object is encrypted and every request uses TLS, clients upload and download directly with short-lived presigned URLs, lifecycle rules keep storage cost bounded, and event consumers survive duplicates and out-of-order delivery.

Before changing anything

1. Read AGENTS.md and find where buckets are defined (CDK, Terraform, SAM). Change IaC, not the console.
2. Inspect the current state (needs network and credentials):

B=acme-uploads-prod
aws s3api get-public-access-block --bucket $B
aws s3api get-bucket-ownership-controls --bucket $B
aws s3api get-bucket-encryption --bucket $B
aws s3api get-bucket-versioning --bucket $B
aws s3api get-bucket-policy --bucket $B --query Policy --output text | python3 -m json.tool
aws s3api get-bucket-lifecycle-configuration --bucket $B
aws s3api get-bucket-notification-configuration --bucket $B

NoSuchPublicAccessBlockConfiguration or ServerSideEncryptionConfigurationNotFoundError are findings, not errors to ignore.

Baseline for every bucket

Setting	Value	Why
Block Public Access	All four flags on, at bucket and account level	Default for new buckets; keep it
Object Ownership	BucketOwnerEnforced (ACLs disabled)	Access only through policies
Encryption	SSE-S3 default; SSE-KMS with BucketKeyEnabled: true when you need key-level access control or audit	Bucket keys cut KMS request cost sharply
TLS	Bucket policy denying aws:SecureTransport = false	Blocks plaintext requests
Versioning	On for anything you cannot regenerate	Recovery from overwrite and delete
Lifecycle	Abort incomplete multipart uploads after 7 days; expire noncurrent versions	Otherwise invisible storage grows forever
Access logging	Server access logs or CloudTrail data events for sensitive buckets	Forensics

{
  "Version": "2012-10-17",
  "Statement": [{
    "Sid": "DenyInsecureTransport",
    "Effect": "Deny",
    "Principal": "*",
    "Action": "s3:*",
    "Resource": ["arn:aws:s3:::acme-uploads-prod", "arn:aws:s3:::acme-uploads-prod/*"],
    "Condition": { "Bool": { "aws:SecureTransport": "false" } }
  }]
}

Static websites: keep the bucket private and serve it through CloudFront with Origin Access Control (see $aws-cloudfront). Do not enable S3 website hosting with a public bucket policy unless the user explicitly wants that and accepts it.

Presigned URLs

Use presigned URLs so file bytes never pass through your API servers.

Need	Use
Download a private object	Presigned GET, 5-15 min expiry
Browser upload, size limit required	Presigned POST with content-length-range
Browser or mobile upload, no size enforcement needed	Presigned PUT with fixed ContentType
Files over ~100 MB	Multipart upload with presigned part URLs

import boto3
from botocore.config import Config

s3 = boto3.client("s3", region_name="eu-west-1",
                  config=Config(signature_version="s3v4"))

url = s3.generate_presigned_url(
    "get_object",
    Params={"Bucket": "acme-uploads-prod", "Key": key,
            "ResponseContentDisposition": 'attachment; filename="report.pdf"'},
    ExpiresIn=900,
)

post = s3.generate_presigned_post(
    Bucket="acme-uploads-prod",
    Key=f"incoming/{user_id}/{uuid4()}.jpg",
    Fields={"Content-Type": "image/jpeg"},
    Conditions=[{"Content-Type": "image/jpeg"},
                ["content-length-range", 1, 10 * 1024 * 1024]],
    ExpiresIn=300,
)
# client sends multipart/form-data to post["url"] with post["fields"] then the file

import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3';
import { getSignedUrl } from '@aws-sdk/s3-request-presigner';

const s3 = new S3Client({ region: 'eu-west-1' });
const url = await getSignedUrl(
  s3,
  new PutObjectCommand({ Bucket: 'acme-uploads-prod', Key: key, ContentType: 'image/png' }),
  { expiresIn: 300 },
);
// the client must send exactly this Content-Type header or the signature fails

Rules:

• Generate the key server-side. Never let the client choose the full key; it can overwrite other users’ objects.
• The URL carries the signer’s permissions. Sign with a role that can only touch the upload prefix.
• A URL signed with temporary credentials (Lambda, ECS task role) dies when those credentials expire, even if ExpiresIn is longer. SigV4 maximum is 7 days.
• Treat uploads as untrusted: validate type and size after upload (event-driven scan or check) before using the object.
• Use the regional endpoint for the bucket’s region; signing for the wrong region gives SignatureDoesNotMatch or a redirect.

Browser uploads need CORS on the bucket:

[{
  "AllowedOrigins": ["https://app.acme.com"],
  "AllowedMethods": ["PUT", "POST", "GET"],
  "AllowedHeaders": ["content-type"],
  "ExposeHeaders": ["ETag"],
  "MaxAgeSeconds": 3000
}]

Expose ETag for multipart uploads; the client needs each part’s ETag to complete.

Lifecycle and storage classes

Access pattern	Class
Unknown or changing	S3 Intelligent-Tiering (no retrieval fees; small monitoring fee per object)
Read rarely, needed in milliseconds	Standard-IA after 30 days
Archives read a few times a year	Glacier Instant Retrieval
Compliance archives, hours to restore is fine	Glacier Flexible Retrieval or Deep Archive

{
  "Rules": [
    { "ID": "abort-mpu", "Status": "Enabled", "Filter": {},
      "AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 } },
    { "ID": "logs-tiering", "Status": "Enabled", "Filter": { "Prefix": "logs/" },
      "Transitions": [{ "Days": 30, "StorageClass": "STANDARD_IA" },
                      { "Days": 180, "StorageClass": "GLACIER_IR" }],
      "Expiration": { "Days": 730 } },
    { "ID": "old-versions", "Status": "Enabled", "Filter": {},
      "NoncurrentVersionExpiration": { "NoncurrentDays": 30, "NewerNoncurrentVersions": 3 } }
  ]
}

Gotchas: IA and Glacier classes have minimum storage durations and minimum billable object sizes, so tiering millions of tiny objects can cost more than it saves. By default lifecycle does not transition objects smaller than 128 KB. Expiration rules on a production bucket delete data: confirm the prefix and day count with the user first.

Event notifications

Need	Use
One consumer, simple	S3 notification to SQS (then Lambda), not direct to Lambda, so you get buffering and a DLQ
Several consumers or content-based filtering	Enable EventBridge on the bucket and use rules
Fan-out to many queues	S3 to SNS to SQS, or EventBridge

• Delivery is at-least-once and not ordered. Deduplicate on bucket + key + versionId or the sequencer field.
• Object keys in events are URL-encoded (spaces arrive as +). Decode with urllib.parse.unquote_plus before calling GetObject.
• A consumer that writes back into the same bucket and prefix triggers itself. Write output to a different prefix excluded by the filter, or a different bucket.
• Prefix/suffix filters on one bucket cannot overlap for the same event type.

Gotchas

• S3 is strongly consistent for reads after writes and for lists. Old advice about eventual consistency is obsolete.
• Prevent overwrites with conditional writes: PutObject with If-None-Match: * fails with 412 if the key exists. Use If-Match: <etag> for optimistic updates.
• aws s3 sync --delete and aws s3 rm --recursive are destructive. Run with --dryrun first and get explicit confirmation before the real run. Deleting a bucket or disabling versioning on a production bucket also needs confirmation.
• Request rates are per prefix (about 3,500 writes and 5,500 reads per second); spread hot keys across prefixes for very high throughput.
• Cross-region transfer and internet egress cost money; NAT gateway processing for S3 traffic is avoidable with a free S3 gateway VPC endpoint.

Verification

• Re-run the inspection commands; all four Block Public Access flags true, encryption present, policy denies insecure transport.
• curl -I "https://acme-uploads-prod.s3.eu-west-1.amazonaws.com/some-key" without a signature returns 403.
• Presigned flow: upload with the URL, check an oversized file is rejected, check the URL fails after expiry.
• IAM Access Analyzer for S3 lists no unintended external or public access.