AWS SDK for Java 2.x - Amazon S3 Skill

AWS SDK for Java 2.x - Amazon S3

Overview

Provides patterns for S3 operations: bucket management, object upload/download with multipart support, presigned URLs, S3 Transfer Manager, and S3-specific configurations using AWS SDK for Java 2.x.

When to Use

Creating, listing, or deleting S3 buckets with proper configuration
Uploading or downloading objects from S3 with metadata and encryption
Working with multipart uploads for large files (>100MB) with error handling
Generating presigned URLs for temporary access to S3 objects
Copying or moving objects between S3 buckets with metadata preservation
Setting object metadata, storage classes, and access controls
Implementing S3 Transfer Manager for optimized file transfers
Integrating S3 with Spring Boot applications for cloud storage

Quick Reference

| Operation | Method | Notes | |-----------|--------|-------| | Create bucket | createBucket() | Wait with waiter().waitUntilBucketExists() | | Upload object | putObject() | Use RequestBody.fromFile() | | Download object | getObject() | Streams to file or memory | | Delete objects | deleteObjects() | Batch up to 1000 keys | | Presigned URL | presigner.presignGetObject() | Max 7 days expiration |

Storage Classes

| Class | Use Case | |-------|----------| | STANDARD | Frequently accessed data | | STANDARD_IA | Infrequently accessed data | | GLACIER | Long-term archive | | INTELLIGENT_TIERING | Automatic cost optimization |

Instructions

1. Add Dependencies

<dependency>
    <groupId>software.amazon.awssdk</groupId>
    <artifactId>s3</artifactId>
    <version>2.20.0</version>
</dependency>

<dependency>
    <groupId>software.amazon.awssdk</groupId>
    <artifactId>s3-transfer-manager</artifactId>
    <version>2.20.0</version>
</dependency>

2. Create S3 Client

S3Client s3Client = S3Client.builder()
    .region(Region.US_EAST_1)
    .build();

// With retry logic
S3Client s3Client = S3Client.builder()
    .region(Region.US_EAST_1)
    .overrideConfiguration(b -> b
        .retryPolicy(RetryPolicy.builder()
            .numRetries(3)
            .build()))
    .build();

3. Create Bucket

CreateBucketRequest request = CreateBucketRequest.builder()
    .bucket(bucketName)
    .build();

s3Client.createBucket(request);

// Wait until ready
s3Client.waiter().waitUntilBucketExists(
    HeadBucketRequest.builder().bucket(bucketName).build()
);

4. Upload Object

PutObjectRequest request = PutObjectRequest.builder()
    .bucket(bucketName)
    .key(key)
    .contentType("application/pdf")
    .serverSideEncryption(ServerSideEncryption.AES256)
    .storageClass(StorageClass.STANDARD_IA)
    .build();

s3Client.putObject(request, RequestBody.fromFile(Paths.get(filePath)));

// Validate upload completion
HeadObjectResponse headResp = s3Client.headObject(HeadObjectRequest.builder()
    .bucket(bucketName)
    .key(key)
    .build());

5. Download Object

GetObjectRequest request = GetObjectRequest.builder()
    .bucket(bucketName)
    .key(key)
    .build();

s3Client.getObject(request, Paths.get(destPath));

6. Generate Presigned URL

try (S3Presigner presigner = S3Presigner.create()) {
    GetObjectRequest getRequest = GetObjectRequest.builder()
        .bucket(bucketName)
        .key(key)
        .build();

    GetObjectPresignRequest presignRequest = GetObjectPresignRequest.builder()
        .signatureDuration(Duration.ofMinutes(10))
        .getObjectRequest(getRequest)
        .build();

    String url = presigner.presignGetObject(presignRequest).url().toString();
}

7. Use Transfer Manager (Large Files)

try (S3TransferManager tm = S3TransferManager.create()) {
    UploadFileRequest request = UploadFileRequest.builder()
        .putObjectRequest(req -> req.bucket(bucketName).key(key))
        .source(Paths.get(filePath))
        .build();

    FileUpload upload = tm.uploadFile(request);
    CompletedFileUpload result = upload.completionFuture().join();
}

Best Practices

Performance

Use S3 Transfer Manager: Automatic multipart uploads for files >100MB
Reuse S3 Client: Clients are thread-safe; reuse throughout application
Enable async operations: Use S3AsyncClient for I/O-bound operations
Configure timeouts: Set appropriate timeouts for large file operations

Security

Use temporary credentials: IAM roles or AWS STS for short-lived tokens
Enable encryption: Use AES-256 or AWS KMS for sensitive data
Use presigned URLs: Avoid exposing credentials with temporary access
Validate metadata: Sanitize user-provided metadata

Error Handling

Implement retry logic: Exponential backoff for network operations
Handle throttling: Proper handling of 429 responses
Clean up failures: Abort failed multipart uploads

Cost Optimization

Use appropriate storage classes: STANDARD, STANDARD_IA, INTELLIGENT_TIERING
Implement lifecycle policies: Automatic transition/expiration
Minimize API calls: Use batch operations when possible

Constraints and Warnings

Object Size: Single PUT limited to 5GB; use multipart for larger files
Bucket Names: Must be globally unique across all AWS accounts
Object Immutability: Objects cannot be modified; must be replaced entirely
Eventual Consistency: List operations may have slight delays after uploads
Presigned URLs: Maximum expiration time is 7 days
Multipart Uploads: Parts must be at least 5MB except last part

Examples

Complete Upload Workflow with Validation

// 1. Upload with validation
PutObjectRequest putRequest = PutObjectRequest.builder()
    .bucket(bucketName)
    .key(key)
    .contentType(contentType)
    .build();

s3Client.putObject(putRequest, RequestBody.fromFile(Paths.get(filePath)));

// 2. Validate with headObject
HeadObjectResponse headResp = s3Client.headObject(HeadObjectRequest.builder()
    .bucket(bucketName)
    .key(key)
    .build());

// 3. Verify metadata
long fileSize = Files.size(Paths.get(filePath));
if (headResp.contentLength() != fileSize) {
    throw new IllegalStateException("Upload size mismatch");
}

Multipart Upload with Abort-on-Failure

// 1. Initiate multipart upload
CreateMultipartUploadRequest createRequest = CreateMultipartUploadRequest.builder()
    .bucket(bucketName)
    .key(key)
    .build();

CreateMultipartUploadResponse multipartUpload = s3Client.createMultipartUpload(createRequest);
String uploadId = multipartUpload.uploadId();

try {
    // 2. Upload parts
    List<CompletedPart> parts = new ArrayList<>();
    int partNumber = 1;
    byte[] fileBytes = Files.readAllBytes(Paths.get(filePath));
    int chunkSize = 5 * 1024 * 1024; // 5MB minimum

    for (int offset = 0; offset < fileBytes.length; offset += chunkSize) {
        int length = Math.min(chunkSize, fileBytes.length - offset);
        UploadPartRequest uploadPartRequest = UploadPartRequest.builder()
            .bucket(bucketName)
            .key(key)
            .uploadId(uploadId)
            .partNumber(partNumber)
            .build();

        UploadPartResponse partResponse = s3Client.uploadPart(uploadPartRequest,
            RequestBody.fromBytes(Arrays.copyOfRange(fileBytes, offset, offset + length)));

        parts.add(CompletedPart.builder()
            .partNumber(partNumber)
            .eTag(partResponse.eTag())
            .build());
        partNumber++;
    }

    // 3. Complete multipart upload
    CompleteMultipartUploadRequest completeRequest = CompleteMultipartUploadRequest.builder()
        .bucket(bucketName)
        .key(key)
        .uploadId(uploadId)
        .multipartUpload(CompletedMultipartUpload.builder().parts(parts).build())
        .build();
    s3Client.completeMultipartUpload(completeRequest);

} catch (Exception e) {
    // 4. Abort on failure
    AbortMultipartUploadRequest abortRequest = AbortMultipartUploadRequest.builder()
        .bucket(bucketName)
        .key(key)
        .uploadId(uploadId)
        .build();
    s3Client.abortMultipartUpload(abortRequest);
    throw new RuntimeException("Upload failed, cleanup performed", e);
}

References

references/s3-client-setup.md — Client configuration and basic operations
references/s3-object-operations.md — Advanced object operations
references/s3-transfer-patterns.md — Transfer Manager and multipart uploads
references/s3-spring-boot-integration.md — Spring Boot integration patterns
AWS S3 Developer Guide
AWS SDK for Java 2.x S3 API

Related Skills

aws-sdk-java-v2-core - Core AWS SDK patterns and configuration
spring-boot-dependency-injection - Spring dependency injection patterns

Agent Skills: AWS SDK for Java 2.x - Amazon S3

Install this agent skill to your local

Skill Files