ADR 018: Database Patterns

Status: Proposed | Date: 2025-07-28

Context

Applications need managed persistent storage (databases, datalakes, files, objects) with automatic scaling and jurisdiction-compliant backup strategies.

• AWS Aurora Serverless v2 Documentation
• Percona Everest Documentation and Pigsty Documentation for development/non-AWS environments
• s3proxy and rclone serve s3 for development/non-AWS object storage

Decision

Use Aurora Serverless v2 outside EKS clusters with automated scaling, multi-AZ deployment, and dual backup strategy.

Datalakes: Use SQL engines over object storage:

• DuckLake over AWS S3 for simpler analytical workloads
• Trino over S3 Tables for larger-scale data processing

See Reference Architecture: Data Pipelines for full datalake patterns.

Implementation

• Database: Aurora Serverless v2 (PostgreSQL/MySQL) with built-in connection pooling and automatic scaling
• Object Storage: Amazon S3 and Amazon S3 Tables for datalakes, files and objects
• Deployment: Outside EKS cluster (handles complexity automatically)
• Credentials: Follow ADR 005: Secrets Management for endpoint and credential management
• Backup: Follow ADR 014: Object Storage Backups plus AWS automated snapshots
• Security: Follow ADR 007: Centralised Security Logging and ADR 012: Privileged Remote Access

Consequences

Benefits:

• Serverless scaling reducing operational costs during low usage periods
• Automated high availability with managed backup strategies per ADR 014: Object Backup
• Compliance with jurisdiction requirements through dual backup approach

Risks if not implemented:

• High operational overhead managing database infrastructure
• Inconsistent backup strategies across database systems
• Cost inefficiency from overprovisioned database resources