Compare commits

10 Commits

182 changed files with 18578 additions and 10709 deletions

2
.gitignore vendored
View File

@@ -2,3 +2,5 @@ node_modules
dist
data
.env
*.sqlite
.DS_Store

112
CODING_STANDARDS.md Normal file
View File

@@ -0,0 +1,112 @@
# Coding Standards
## Code Structure
### No Else Blocks - Use Early Returns
Always prefer early returns and guard clauses over else blocks. This reduces nesting and improves readability.
**❌ Bad:**
```typescript
if (condition) {
// do something
return result;
} else {
// do something else
return otherResult;
}
```
**✅ Good:**
```typescript
if (condition) {
return result;
}
return otherResult;
```
**❌ Bad:**
```typescript
if (error) {
throw new Error('Failed');
} else {
return processData();
}
```
**✅ Good:**
```typescript
if (error) {
throw new Error('Failed');
}
return processData();
```
### Multiple Conditions
For multiple conditions, stack guard clauses:
**❌ Bad:**
```typescript
if (cas !== undefined) {
if (cas === 0 && existing) {
return false;
} else {
if (cas > 0 && (!existing || existing.modifyIndex !== cas)) {
return false;
}
}
}
```
**✅ Good:**
```typescript
if (cas === 0 && existing) {
return false;
}
if (cas > 0 && (!existing || existing.modifyIndex !== cas)) {
return false;
}
```
### No Unnecessary Comments
Avoid conversational comments like "let's", "now we", etc. Code should be self-documenting through good naming.
**❌ Bad:**
```typescript
// Let's check if the user exists
const user = await findUser(id);
// Now let's validate the permissions
if (!user.hasPermission()) {
throw new Error();
}
```
**✅ Good:**
```typescript
const user = await findUser(id);
if (!user.hasPermission()) {
throw new UnauthorizedException();
}
```
Comments should explain **why**, not **what**.
**✅ Acceptable:**
```typescript
// Consul API requires plain text "true"/"false" responses, not JSON
return success.toString();
```
## Benefits
1. **Reduced Nesting**: Flatter code structure
2. **Better Readability**: Main logic flow is clear
3. **Easier Maintenance**: Less cognitive load
4. **Clearer Intent**: Guard clauses make preconditions explicit

257
CONSUL_BLOCKING_QUERIES.md Normal file
View File

@@ -0,0 +1,257 @@
# Consul KV Blocking Queries
## Overview
The Consul KV service now supports blocking queries (also known as long polling), which allows clients to efficiently wait for changes to keys without continuous polling.
## How It Works
When a client makes a GET request with blocking query parameters, the request will:
1. **Return immediately** if the key's ModifyIndex is greater than the provided index
2. **Block and wait** if the index matches the current value, until either:
- The key is updated (new ModifyIndex)
- The wait timeout expires
3. **Return the current state** when either condition is met
## Usage
### Query Parameters
- `index` - The ModifyIndex from a previous request. The query will block until the key's index changes
- `wait` - Duration to wait for changes (e.g., `5s`, `10m`, `1h`). Defaults to 5 minutes if not specified
### Example: Basic Blocking Query
```bash
# Get the current state
curl http://localhost:8500/v1/kv/my-key
# Response includes ModifyIndex
# [{"CreateIndex":1,"ModifyIndex":5,"Key":"my-key","Value":"..."}]
# Block until the key changes, up to 30 seconds
curl "http://localhost:8500/v1/kv/my-key?index=5&wait=30s"
# This request will:
# - Return immediately if ModifyIndex > 5
# - Wait up to 30s if ModifyIndex == 5
# - Return with new value when key is updated
```
### Example: Watch Loop in JavaScript
```javascript
async function watchKey(key) {
let currentIndex = 0;
while (true) {
try {
// Make blocking query
const response = await fetch(`http://localhost:8500/v1/kv/${key}?index=${currentIndex}&wait=60s`);
if (response.ok) {
const data = await response.json();
const newIndex = data[0].ModifyIndex;
// Check if the value actually changed
if (newIndex > currentIndex) {
console.log('Key changed:', data[0]);
currentIndex = newIndex;
// Process the change
handleChange(data[0]);
}
}
} catch (error) {
console.error('Watch error:', error);
await new Promise(r => setTimeout(r, 1000)); // Backoff
}
}
}
```
### Example: Terraform State Watching
```javascript
// Watch for Terraform state changes
async function watchTerraformState(statePath) {
let index = 0;
while (true) {
const response = await fetch(`http://localhost:8500/v1/kv/${statePath}?index=${index}&wait=5m`);
if (response.ok) {
const [entry] = await response.json();
if (entry.ModifyIndex > index) {
const stateData = JSON.parse(Buffer.from(entry.Value, 'base64').toString('utf-8'));
console.log('Terraform state updated:', {
version: stateData.version,
serial: stateData.serial,
modifyIndex: entry.ModifyIndex,
});
index = entry.ModifyIndex;
}
}
}
}
watchTerraformState('terraform/myproject/state');
```
## Implementation Details
### Concurrent Watchers
Multiple clients can simultaneously watch the same key. When the key is updated, all waiting requests will be notified and return with the new value.
### Performance
- Minimal server overhead - no continuous polling
- Automatic cleanup of watchers on timeout
- Notifications are delivered immediately when changes occur
- No database polling - uses in-memory event system
### Wait Durations
Supported time units:
- `s` - seconds (e.g., `30s`)
- `m` - minutes (e.g., `5m`)
- `h` - hours (e.g., `1h`)
Default: 5 minutes if not specified
### Edge Cases
#### Non-existent Keys
Blocking queries on non-existent keys with `index=0` will return 404 immediately. To wait for a key to be created, you need to handle 404 responses and retry.
#### Key Deletion
If a watched key is deleted, waiting queries will be notified and return 404.
#### Recursive Queries
Blocking queries do not currently support `recurse=true`. Use blocking queries only on specific keys.
## Testing
The test suite includes comprehensive blocking query tests:
```bash
npm test -- consul-kv.spec.ts
```
Tests verify:
- ✅ Immediate return when index is outdated
- ✅ Blocking until key changes
- ✅ Timeout behavior when no changes occur
- ✅ Multiple concurrent watchers on same key
- ✅ Non-existent key handling
## Comparison with Real Consul
This implementation matches Consul's blocking query behavior for:
- Standard wait/index semantics
- Multiple concurrent watchers
- Timeout handling
- Immediate return on index mismatch
Differences from real Consul:
- No support for `recurse` with blocking queries
- Simplified implementation (in-memory events vs raft log)
- Wait times capped at request timeout (no separate max wait limit)
## Use Cases
### Configuration Management
Watch for configuration changes and reload application settings:
```javascript
async function watchConfig(configKey) {
let index = 0;
while (true) {
const response = await fetch(`http://localhost:8500/v1/kv/${configKey}?index=${index}&wait=5m`);
if (response.ok) {
const [entry] = await response.json();
if (entry.ModifyIndex > index) {
const config = JSON.parse(Buffer.from(entry.Value, 'base64').toString('utf-8'));
reloadConfig(config);
index = entry.ModifyIndex;
}
}
}
}
```
### Service Coordination
Wait for another service to signal readiness:
```javascript
async function waitForService(serviceKey) {
let index = 0;
while (true) {
const response = await fetch(`http://localhost:8500/v1/kv/${serviceKey}/ready?index=${index}&wait=1m`);
if (response.ok) {
const [entry] = await response.json();
const value = Buffer.from(entry.Value, 'base64').toString('utf-8');
if (value === 'true') {
console.log('Service is ready!');
return;
}
index = entry.ModifyIndex;
}
}
}
```
### State Change Notifications
Monitor Terraform state for external changes:
```javascript
async function monitorTerraformState(statePath) {
let lastSerial = 0;
let index = 0;
while (true) {
const response = await fetch(`http://localhost:8500/v1/kv/${statePath}?index=${index}&wait=10m`);
if (response.ok) {
const [entry] = await response.json();
if (entry.ModifyIndex > index) {
const state = JSON.parse(Buffer.from(entry.Value, 'base64').toString('utf-8'));
if (state.serial > lastSerial) {
console.log(`State changed: serial ${lastSerial}${state.serial}`);
lastSerial = state.serial;
// Notify about drift
if (await detectDrift(state)) {
notifyTeam('Terraform drift detected!');
}
}
index = entry.ModifyIndex;
}
}
}
}
```

812
PROJECT_STRUCTURE.md Normal file
View File

@@ -0,0 +1,812 @@
# Local AWS - Project Structure Documentation
## Overview
Local AWS is a NestJS-based emulator for AWS services. It implements AWS APIs locally for development and testing purposes, using SQLite for persistence and following AWS API specifications.
**Key Architecture Points:**
- **Handler Pattern**: Each AWS action is implemented as a separate handler class
- **Modular Structure**: One module per AWS service (IAM, KMS, S3, SNS, SQS, etc.)
- **Multi-Server Architecture**: Main app + separate S3 microservice + Consul KV microservice
- S3 uses REST-style routing
- Consul KV implements HashiCorp Consul API
- **Database**: SQLite with Prisma ORM
- **Test Structure**: Each module has its own test suite using AWS SDK or service-specific clients
---
## Application Entry Points
### Main Entry (`src/main.ts`)
Bootstraps two NestJS applications:
1. **Main Application** (default port 8081)
- Handles IAM, KMS, SecretsManager, SNS, SQS, STS services
- Uses query-based routing (Action parameter in body/query)
- Body parsers: JSON (for SNS/SQS), XML, raw binary
2. **S3 Microservice** (default port 4567)
- Separate app due to S3's REST/path-style routing
- Routes based on HTTP method + path + query parameters
- Body parsers: raw binary, URL-encoded
3. **Consul KV Microservice** (default port 8500)
- Implements HashiCorp Consul Key-Value store API
- REST-style routing at `/v1/kv/*` endpoints
- Supports recursive queries, CAS operations, locks, flags
- Body parsers: JSON, raw binary, text/plain
**Key Global Features:**
```typescript
Date.prototype.getAwsTime(); // Extension for AWS timestamp format
```
---
## Module Architecture
### Module Structure Pattern
Each AWS service follows this structure:
```
<service>/
├── __tests__/
│ └── <service>.spec.ts # Integration tests using AWS SDK
├── <action>-handler.ts # One handler per AWS action
├── <service>.constants.ts # Action enum list & injection tokens
├── <service>.module.ts # NestJS module definition
├── <service>.service.ts # Business logic layer
├── <service>-<entity>.entity.ts # Prisma model wrapper classes
└── (optional) <service>.controller.ts # Custom controller if needed
```
### Current Modules
#### Main Application Modules
1. **IAM** (`src/iam/`)
- Handlers: CreateRole, GetRole, DeleteRole, CreatePolicy, GetPolicy, AttachRolePolicy, PutRolePolicy, GetRolePolicy, etc.
- Entities: IamRole, IamPolicy, IamRoleInlinePolicy
- Database: Roles, policies, role-policy attachments, inline policies
2. **KMS** (`src/kms/`)
- Handlers: CreateKey, DescribeKey, CreateAlias, Sign, GetPublicKey, EnableKeyRotation, etc.
- Entities: KmsKey, KmsAlias
- Database: KMS keys, aliases
3. **Secrets Manager** (`src/secrets-manager/`)
- Handlers: CreateSecret, GetSecretValue, PutSecretValue, DeleteSecret, etc.
- Database: Secrets with versioning
4. **SNS** (`src/sns/`)
- Handlers: CreateTopic, Subscribe, Publish, etc.
- Database: Topics, subscriptions
5. **SQS** (`src/sqs/`)
- Handlers: CreateQueue, SendMessage, ReceiveMessage, etc.
- Database: Queues, messages
6. **STS** (`src/sts/`)
- Handlers: AssumeRole, GetCallerIdentity, etc.
- Temporary credential generation
#### S3 Microservice Module
**S3** (`src/s3/`)
- Special module with its own app (`S3AppModule`) and controller
- Handlers: CreateBucket, ListBuckets, PutObject, GetObject, DeleteObject, PutBucketAcl, GetBucketAcl, etc.
- Entities: S3Bucket, S3Object
- Custom routing logic in `S3Controller` based on HTTP method + path + query params
- Database: Buckets with tags/ACL/policy, objects with metadata
#### Consul KV Microservice Module
**Consul KV** (`src/consul-kv/`)
- Special module with its own app (`ConsulKVAppModule`) and controller
- Implements HashiCorp Consul KV API spec: https://developer.hashicorp.com/consul/api-docs/kv
- Service: ConsulKVService with getKey, putKey, deleteKey, listKeys, recursive operations
- Custom routing in `ConsulKVController` for `/v1/kv/*` paths
- Features:
- Key-value storage with base64 encoding
- Recursive queries with prefix matching
- Keys-only listing with separator support
- Check-And-Set (CAS) operations
- Distributed lock acquisition/release
- Flags and index tracking (createIndex, modifyIndex, lockIndex)
- Multi-tenancy support (datacenter, namespace)
- Database: ConsulKVEntry model
- Tests: 19 integration tests using `consul` npm package
---
## Handler Pattern Deep Dive
### AbstractActionHandler (`src/abstract-action.handler.ts`)
Base class for all action handlers. All handlers must extend this class.
**Required Properties:**
```typescript
abstract class AbstractActionHandler<T> {
format: Format; // Format.Xml or Format.Json
action: Action | Action[]; // AWS action enum(s)
validator: Joi.ObjectSchema; // Request validation schema
protected abstract handle(
queryParams: T, // Validated query parameters
context: RequestContext, // AWS properties + request metadata
): Record<string, any> | void;
}
```
**Response Handling:**
- XML format: Wraps result in `{Action}Result` node with RequestId metadata
- JSON format: Returns result directly
- Empty response: Returns ResponseMetadata only (for XML) or nothing (for JSON)
### Handler Implementation Example
```typescript
@Injectable()
export class CreateRoleHandler extends AbstractActionHandler<QueryParams> {
constructor(private readonly iamService: IamService) {
super();
}
format = Format.Xml;
action = Action.IamCreateRole;
validator = Joi.object<QueryParams>({
RoleName: Joi.string().required(),
Path: Joi.string().required(),
AssumeRolePolicyDocument: Joi.string().required(),
// ... other parameters
});
protected async handle(params: QueryParams, { awsProperties }: RequestContext) {
const role = await this.iamService.createRole({
accountId: awsProperties.accountId,
name: params.RoleName,
// ...
});
return { Role: role.metadata };
}
}
```
### Handler Registration Flow
1. **Define handler class** extending `AbstractActionHandler`
2. **Add to module's handlers array** in `<service>.module.ts`
3. **ExistingActionHandlersProvider** collects all handlers into a map keyed by Action enum
4. **DefaultActionHandlerProvider** fills gaps with stub handlers for unimplemented actions
5. **Module provides injection token** (e.g., `IAMHandlers`, `S3Handlers`) containing the handler map
6. **Controller receives handler map** and routes requests to appropriate handler
```typescript
// In module
const handlers = [CreateRoleHandler, GetRoleHandler, /* ... */];
@Module({
providers: [
...handlers,
ExistingActionHandlersProvider(handlers),
DefaultActionHandlerProvider(IAMHandlers, Format.Xml, allIamActions),
]
})
```
---
## Controllers
### Main Controller (`src/app.controller.ts`)
Routes all non-S3 AWS service requests.
**Request Flow:**
1. Extracts action from `x-amz-target` header or `Action` body parameter
2. Validates action exists in Action enum
3. Looks up handler from injected ActionHandlers map
4. Validates request params using handler's Joi validator
5. Calls `handler.getResponse(params, context)`
6. Returns XML or JSON based on handler's format
### S3 Controller (`src/s3/s3.controller.ts`)
Custom controller for S3 REST-style API (separate microservice).
**Request Flow:**
1. Parses path into bucket/key components
2. Determines action from HTTP method + path + query parameters
- `PUT /{bucket}` → CreateBucket
- `GET /{bucket}` → ListObjects (or other sub-resource if query param present)
- `PUT /{bucket}?acl` → PutBucketAcl
- `GET /{bucket}/{key}` → GetObject
3. Normalizes query parameters (e.g., `max-keys``MaxKeys`)
4. Extracts metadata from `x-amz-meta-*` headers
5. Converts binary body to base64 for handlers
6. Routes to handler and returns response with appropriate headers
**Special S3 Response Handling:**
- Creates Location header for bucket creation
- Returns ETag, Last-Modified, Content-Type headers
- Returns raw binary data for GetObject (not XML/JSON)
- Returns empty 200 for PutBucketAcl, PutBucketTagging
- Returns 204 for DeleteBucket, DeleteObject
### Consul KV Controller (`src/consul-kv/consul-kv.controller.ts`)
Custom controller for Consul KV REST API (separate microservice).
**Request Flow:**
1. Catches all `/v1/kv/*` paths
2. Extracts and URL-decodes the key from the path
3. Routes based on HTTP method (GET/PUT/DELETE)
4. Parses query parameters:
- `dc`: datacenter (default: dc1)
- `ns`: namespace (default: default)
- `recurse`: recursive key retrieval
- `keys`: return only key names
- `raw`: return raw value (not JSON)
- `separator`: group keys by separator
- `flags`: numeric flags for application use
- `cas`: Check-And-Set with modify index
- `acquire`: session ID for lock acquisition
- `release`: session ID for lock release
5. Processes request through ConsulKVService
6. Returns appropriate response format
**Special Consul Response Handling:**
- PUT/DELETE return plain text "true" or "false"
- GET returns JSON array of key metadata or raw value
- Keys-only returns JSON array of key strings
- 404 status for non-existent keys
- Base64 encoding for values in JSON responses
- Supports keys with slashes (proper URL decoding)
---
## Service Layer
Each module has a service class that:
- Encapsulates business logic
- Interacts with Prisma for database operations
- Validates entity constraints
- Throws AWS-compatible exceptions
**Example: IAM Service** (`src/iam/iam.service.ts`)
```typescript
@Injectable()
export class IamService {
constructor(private readonly prisma: PrismaService) {}
async createRole(data: CreateRoleInput): Promise<IamRole> {
// Check for duplicates
// Create in database
// Return entity
}
async putRoleInlinePolicy(...): Promise<void> {
// Verify role exists
// Upsert policy
}
}
```
---
## Entity Layer
Entity classes wrap Prisma models and provide:
- Type safety
- Computed properties (e.g., ARNs, metadata formatters)
- Serialization logic
**Example: IAM Role Entity** (`src/iam/iam-role.entity.ts`)
```typescript
export class IamRole implements PrismaIamRole {
id: string;
name: string;
path: string;
accountId: string;
assumeRolePolicy: string;
// ...
get arn(): string {
return `arn:aws:iam::${this.accountId}:role${this.path}${this.name}`;
}
get metadata() {
// Returns AWS API response format
}
}
```
---
## Database Layer
### Prisma Setup (`src/_prisma/`)
- **prisma.service.ts**: PrismaClient wrapper with connection lifecycle
- **prisma.module.ts**: Exports PrismaService globally
### Schema Location: `prisma/schema.prisma`
**Key Models:**
- `IamRole`, `IamPolicy`, `IamRoleIamPolicyAttachment`, `IamRoleInlinePolicy`
- `KmsKey`, `KmsAlias`
- `S3Bucket`, `S3Object`
- `SecretsManagerSecret`
- `SnsTopic`, `SnsSubscription`
- `SqsQueue`
- `ConsulKVEntry`
**Migrations:** `prisma/migrations/`
---
## Shared Infrastructure
### AWS Shared Entities (`src/aws-shared-entities/`)
- **aws-exceptions.ts**: All AWS exception classes
- Base `AwsException` with `toXml()` and `toJson()` methods
- Specific exceptions: `NoSuchEntity`, `ValidationError`, `AccessDeniedException`, etc.
- **attributes.service.ts**: Common attribute parsing/validation
- **tags.service.ts**: Resource tagging utilities
### Request Context (`src/_context/`)
- **request.context.ts**: TypeScript interfaces for request context
```typescript
interface RequestContext {
action?: Action;
format?: Format;
awsProperties: AwsProperties; // accountId, region, host
requestId: string;
}
```
- **exception.filter.ts**: Global exception filter for AWS exceptions
### Audit System (`src/audit/`)
- **audit.interceptor.ts**: Logs requests/responses (main app)
- **s3-audit.interceptor.ts**: Logs S3 requests/responses
- **audit.service.ts**: Audit persistence service (currently minimal)
### Configuration (`src/config/`)
- **local.config.ts**: Loads environment variables
- **config.validator.ts**: Validates config on startup
- **common-config.interface.ts**: TypeScript interface for config
**Environment Variables:**
- `AWS_ACCOUNT_ID` (default: 000000000000)
- `AWS_REGION` (default: us-east-1)
- `PORT` (default: 8081) - Main app port
- `S3_PORT` (default: 4567) - S3 microservice port
- `CONSUL_PORT` (default: 8500) - Consul KV microservice port
- `DATABASE_URL` (default: :memory:) - SQLite connection string
- `HOST`, `PROTO` - Used for URL generation
### Default Action Handler (`src/default-action-handler/`)
Provides stub implementations for unimplemented actions.
- **default-action-handler.provider.ts**: Creates default handlers that throw `UnsupportedOperationException`
- **existing-action-handlers.provider.ts**: Collects implemented handlers into map
- **default-action-handler.constants.ts**: Shared constants
**Purpose:** Allows modules to declare all AWS actions in their enum list, then provides automatic "not implemented" responses for actions without handlers.
---
## Action Enum (`src/action.enum.ts`)
Central enum containing all AWS actions across all services:
```typescript
export enum Action {
// IAM
IamCreateRole = 'CreateRole',
IamGetRole = 'GetRole',
IamPutRolePolicy = 'PutRolePolicy',
// KMS
KmsCreateKey = 'CreateKey',
KmsSign = 'Sign',
// S3
S3CreateBucket = 'CreateBucket',
S3GetObject = 'GetObject',
S3PutBucketAcl = 'PutBucketAcl',
// ... all other actions
}
```
---
## Testing Strategy
### Test Structure
Each module has a dedicated test suite in `src/<module>/__tests__/<module>.spec.ts`
**Test Pattern:**
1. Create isolated test app with in-memory SQLite database
2. Use actual AWS SDK clients pointing to local endpoints
3. Test against real AWS API contracts
4. Verify both API responses and database state
**Example Test Setup:**
```typescript
describe('IAM Integration Tests', () => {
let app: INestApplication;
let iamClient: IAMClient;
let prismaService: PrismaService;
beforeAll(async () => {
// Unique in-memory database per test suite
process.env.DATABASE_URL = `file::memory:?cache=shared&unique=${Date.now()}`;
const moduleFixture = await Test.createTestingModule({
imports: [AppModule],
}).compile();
app = moduleFixture.createNestApplication();
await app.init();
// Real AWS SDK client pointing to local endpoint
iamClient = new IAMClient({
endpoint: `http://localhost:${testPort}`,
credentials: { accessKeyId: 'test', secretAccessKey: 'test' },
});
});
it('should create a role', async () => {
const response = await iamClient.send(new CreateRoleCommand({ ... }));
expect(response.Role.RoleName).toBe('TestRole');
});
});
```
**Test Organization:**
- `src/iam/__tests__/iam.spec.ts` - IAM tests
- `src/s3/__tests__/s3.spec.ts` - S3 tests
- `src/kms/__tests__/kms.spec.ts` - KMS tests
- etc.
**Current Test Count:** 222 tests across 8 test suites
---
## Key Patterns & Conventions
### 1. Handler Naming
- Handlers named after AWS action: `<Action>Handler` (e.g., `CreateRoleHandler`)
- One handler per file: `<action>-handler.ts` (kebab-case)
- Handler must be Injectable and registered in module
### 2. Service Layer
- One service per module: `<service>.service.ts`
- Service injected into handlers via constructor
- Services interact with Prisma, not handlers directly
### 3. Entity Layer
- Entities wrap Prisma models: `<service>-<entity>.entity.ts`
- Implement Prisma type: `implements Prisma<Entity>`
- Provide computed properties and formatting methods
### 4. Module Constants
- `<service>.constants.ts` contains:
- Injection token: `export const ServiceHandlers = Symbol('ServiceHandlers');`
- Action list: `export const serviceActions: Action[] = [...]`
### 5. Error Handling
- Throw AWS exception classes from `aws-shared-entities/aws-exceptions.ts`
- Exception filter converts to XML or JSON automatically
- Include descriptive messages matching AWS API
### 6. Database Conventions
- Use Prisma for all database operations
- Models use camelCase, match entity class names
- Migrations created via `npx prisma migrate dev --name <description>`
- Always include `accountId` for multi-tenancy support
### 7. Request Validation
- Use Joi schemas in handler's `validator` property
- Validate early, fail fast with `ValidationError`
- Allow unknown properties for forward compatibility
---
## Development Workflow
### Adding a New Handler
1. **Create handler file** `src/<service>/<action>-handler.ts`
```typescript
@Injectable()
export class MyActionHandler extends AbstractActionHandler<QueryParams> {
format = Format.Xml;
action = Action.ServiceMyAction;
validator = Joi.object({
/* ... */
});
protected async handle(params, context) {
/* ... */
}
}
```
2. **Add to module** in `src/<service>/<service>.module.ts`
```typescript
const handlers = [
// ... existing handlers
MyActionHandler,
];
```
3. **Implement service method** if needed in `src/<service>/<service>.service.ts`
4. **Add tests** in `src/<service>/__tests__/<service>.spec.ts`
```typescript
it('should perform my action', async () => {
const response = await client.send(new MyActionCommand({ ... }));
expect(response).toBeDefined();
});
```
5. **Run tests** `npm test`
### Adding a New AWS Service Module
1. **Create directory** `src/<service>/`
2. **Create files:**
- `<service>.module.ts` - NestJS module
- `<service>.service.ts` - Business logic
- `<service>.constants.ts` - Injection tokens and action list
- `<handler>-handler.ts` - Handler implementations
- `<service>-<entity>.entity.ts` - Entity classes
- `__tests__/<service>.spec.ts` - Tests
3. **Update** `src/action.enum.ts` with service actions
4. **Update** `src/app.module.ts` to import new module
5. **Create Prisma models** in `prisma/schema.prisma`
6. **Run migration** `npx prisma migrate dev --name add_<service>`
7. **Implement handlers** following handler pattern
8. **Write tests** using AWS SDK client
---
## Common Troubleshooting
### Handler not found
- Check handler is in module's `handlers` array
- Verify action enum matches exactly
- Ensure handler is Injectable
### Database errors
- Run `npx prisma generate` after schema changes
- Check migration applied: `npx prisma migrate dev`
- Verify Prisma model matches entity class
### Validation errors
- Check Joi schema matches AWS API requirements
- Verify parameter casing (AWS uses PascalCase)
- Test with actual AWS SDK client
### S3 routing issues
- S3 uses different routing logic (HTTP method + path)
- Check `determineS3Action()` in S3Controller
- Verify query parameter handling (e.g., `?acl`)
### Test failures
- Ensure unique database: `file::memory:?cache=shared&unique=...`
- Check port conflicts (8086-8090 commonly used)
- Clean up resources in afterEach hooks
---
## Architecture Decisions
### Why Separate S3 and Consul KV Microservices?
**S3 Microservice:**
S3 uses REST-style routing (path-based) rather than action-based routing. The main app routes based on `Action` parameter, while S3 needs to parse URLs like `/{bucket}/{key}` and route based on HTTP method + query parameters.
**Consul KV Microservice:**
Consul KV implements the HashiCorp Consul API specification, which uses REST-style routing at `/v1/kv/*` endpoints. It requires different routing logic, query parameter handling, and response formats than AWS services. Running it as a separate microservice allows:
- Clean separation of concerns between AWS and Consul APIs
- Different port (8500) matching standard Consul deployment
- Independent body parsing and response formatting
- Compatibility with existing Consul client libraries
### Why Handler Pattern?
- **Modularity**: Each AWS action is isolated
- **Testability**: Easy to test individual handlers
- **Scalability**: Easy to add new actions
- **Type Safety**: Each handler has its own request type
- **Validation**: Built-in Joi validation per action
### Why Prisma + SQLite?
- **Simplicity**: No external database required
- **Type Safety**: Generated TypeScript types
- **Migrations**: Schema versioning out of the box
- **Performance**: Fast for local development/testing
- **Portability**: Single file database or in-memory
### Why Dual Format Support (XML/JSON)?
Different AWS services use different protocols:
- IAM, KMS: XML (AWS Query protocol)
- SNS, SQS: JSON (AWS JSON protocol)
- S3: XML with custom headers
---
## File Organization Summary
```
src/
├── main.ts # Application bootstrap
├── app.module.ts # Main app module
├── app.controller.ts # Main API controller
├── abstract-action.handler.ts # Base handler class
├── action.enum.ts # All AWS actions
├── app.constants.ts # App-level constants
├── _context/ # Request context types
├── _prisma/ # Prisma client wrapper
├── audit/ # Request/response logging
├── aws-shared-entities/ # AWS exceptions & utilities
├── config/ # Environment configuration
├── default-action-handler/ # Stub handler generation
├── iam/ # IAM service module
│ ├── __tests__/
│ ├── *-handler.ts # Action handlers
│ ├── iam.service.ts # Business logic
│ ├── iam.module.ts # Module definition
│ ├── iam.constants.ts # Constants
│ └── *.entity.ts # Entity classes
├── s3/ # S3 service module (microservice)
│ ├── __tests__/
│ ├── s3-app.module.ts # Separate app module
│ ├── s3.controller.ts # Custom controller
│ ├── *-handler.ts # Action handlers
│ └── ...
├── consul-kv/ # Consul KV module (microservice)
│ ├── __tests__/
│ ├── consul-kv-app.module.ts # Separate app module
│ ├── consul-kv.controller.ts # Custom controller
│ ├── consul-kv.service.ts # Business logic
│ └── ...
├── kms/ # KMS service module
├── secrets-manager/ # Secrets Manager module
├── sns/ # SNS service module
├── sqs/ # SQS service module
└── sts/ # STS service module
prisma/
├── schema.prisma # Database schema
└── migrations/ # Schema migrations
```
---
## Quick Reference
**Start Development:**
```bash
PORT=8081 S3_PORT=4567 CONSUL_PORT=8500 yarn start:dev
```
**Run Tests:**
```bash
npm test # All tests
npm test -- iam.spec.ts # Specific suite
```
**Database Commands:**
```bash
npx prisma migrate dev --name <description>
npx prisma generate
npx prisma studio # GUI database browser
```
**Test AWS CLI:**
```bash
# IAM
aws iam create-role --role-name TestRole --assume-role-policy-document '{}' \
--endpoint-url http://localhost:8081
# S3
aws s3 mb s3://my-bucket --endpoint-url http://localhost:4567
aws s3 ls s3://my-bucket --endpoint-url http://localhost:4567
```
**Test Consul API:**
```bash
# Using curl
curl -X PUT http://localhost:8500/v1/kv/my-key -d 'my-value'
curl http://localhost:8500/v1/kv/my-key
curl -X DELETE http://localhost:8500/v1/kv/my-key
# Using consul CLI (if installed)
export CONSUL_HTTP_ADDR=http://localhost:8500
consul kv put my-key my-value
consul kv get my-key
consul kv delete my-key
```
---
This documentation should be your starting point for understanding the project. Read this first before making changes to understand the patterns and conventions in use.

View File

@@ -33,5 +33,5 @@ abstract-action.handler.ts
* format: the format for output (XML or JSON)
* action: the action the handler is implementing (will be use to key by)
* validator: the Joi validator to be executed to check for required params
* handle(queryParams: T, awsProperties: AwsProperties): Record<string, any> | void
* handle(queryParams: T, { awsProperties} : RequestContext): Record<string, any> | void
* the method that implements the AWS action

View File

@@ -1,4 +0,0 @@
version: 3.7
services:
s3_provider:
image: minio

16
jest.config.js Normal file
View File

@@ -0,0 +1,16 @@
module.exports = {
preset: 'ts-jest',
testEnvironment: 'node',
roots: ['<rootDir>/src'],
testMatch: ['**/__tests__/**/*.spec.ts', '**/?(*.)+(spec|test).ts'],
transform: {
'^.+\\.ts$': 'ts-jest',
},
collectCoverageFrom: ['src/**/*.ts', '!src/**/*.d.ts', '!src/**/__tests__/**'],
moduleFileExtensions: ['ts', 'js', 'json'],
coverageDirectory: 'coverage',
verbose: true,
testTimeout: 10000,
maxConcurrency: 1,
maxWorkers: 1,
};

6066
package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -10,49 +10,40 @@
"test": "jest"
},
"dependencies": {
"@nestjs/common": "^9.3.10",
"@nestjs/config": "^2.3.1",
"@nestjs/core": "^9.3.10",
"@nestjs/platform-express": "^9.3.10",
"@nestjs/typeorm": "^9.0.1",
"class-transformer": "^0.5.1",
"@aws-sdk/client-kms": "^3.968.0",
"@nestjs/common": "^10.4.15",
"@nestjs/config": "^3.3.0",
"@nestjs/core": "^10.4.15",
"@nestjs/platform-express": "^10.4.15",
"@prisma/client": "^6.1.0",
"consul": "^2.0.1",
"joi": "^17.9.0",
"js2xmlparser": "^5.0.0",
"morgan": "^1.10.0",
"reflect-metadata": "^0.2.2",
"rxjs": "^7.8.0",
"sqlite3": "^5.1.6",
"typeorm": "^0.3.12",
"uuidv4": "^6.2.13"
"sqlite3": "^5.1.6"
},
"devDependencies": {
"@aws-sdk/client-sns": "^3.321.1",
"@nestjs/cli": "^9.3.0",
"@nestjs/testing": "^9.4.0",
"@aws-sdk/client-iam": "^3.969.0",
"@aws-sdk/client-s3": "^3.969.0",
"@aws-sdk/client-secrets-manager": "^3.968.0",
"@aws-sdk/client-sns": "^3.968.0",
"@aws-sdk/client-sqs": "^3.968.0",
"@aws-sdk/client-sts": "^3.969.0",
"@nestjs/cli": "^10.4.9",
"@nestjs/testing": "10.4.15",
"@types/consul": "^2.0.0",
"@types/express": "^4.17.17",
"@types/jest": "^29.5.1",
"@types/supertest": "^2.0.12",
"@types/jest": "^30.0.0",
"@types/joi": "^17.2.2",
"@types/node": "^22.10.2",
"eslint": "^8.36.0",
"jest": "^29.5.0",
"supertest": "^6.3.3",
"ts-jest": "^29.1.0"
"jest": "^30.2.0",
"prisma": "^6.1.0",
"ts-jest": "^29.4.6"
},
"jest": {
"globalSetup": "./_jest_/setup.ts",
"globalTeardown": "./_jest_/teardown.ts",
"moduleFileExtensions": [
"js",
"json",
"ts"
],
"rootDir": "src",
"testRegex": ".*\\.*spec\\.ts$",
"transform": {
"^.+\\.(t|j)s$": "ts-jest"
},
"collectCoverageFrom": [
"**/*.(t|j)s"
],
"coverageDirectory": "../coverage",
"testEnvironment": "node"
"engines": {
"node": ">=22.11.0",
"npm": ">=10.9.0"
}
}

View File

@@ -0,0 +1,170 @@
-- CreateTable
CREATE TABLE "Attribute" (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"arn" TEXT NOT NULL,
"name" TEXT NOT NULL,
"value" TEXT NOT NULL
);
-- CreateTable
CREATE TABLE "Audit" (
"id" TEXT NOT NULL PRIMARY KEY,
"createdAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
"action" TEXT,
"request" TEXT,
"response" TEXT
);
-- CreateTable
CREATE TABLE "IamRole" (
"id" TEXT NOT NULL PRIMARY KEY,
"path" TEXT,
"name" TEXT NOT NULL,
"assumeRolePolicy" TEXT,
"description" TEXT,
"maxSessionDuration" INTEGER,
"permissionBoundaryArn" TEXT,
"lastUsedDate" DATETIME,
"lastUsedRegion" TEXT,
"accountId" TEXT NOT NULL,
"createdAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
);
-- CreateTable
CREATE TABLE "IamPolicy" (
"id" TEXT NOT NULL,
"version" INTEGER NOT NULL DEFAULT 1,
"isDefault" BOOLEAN NOT NULL,
"path" TEXT,
"name" TEXT NOT NULL,
"description" TEXT,
"policy" TEXT NOT NULL,
"isAttachable" BOOLEAN NOT NULL DEFAULT false,
"accountId" TEXT NOT NULL,
"createdAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
"updatedAt" DATETIME NOT NULL,
PRIMARY KEY ("id", "version")
);
-- CreateTable
CREATE TABLE "IamRoleIamPolicyAttachment" (
"iamRoleId" TEXT NOT NULL,
"iamPolicyId" TEXT NOT NULL,
PRIMARY KEY ("iamRoleId", "iamPolicyId"),
CONSTRAINT "IamRoleIamPolicyAttachment_iamRoleId_fkey" FOREIGN KEY ("iamRoleId") REFERENCES "IamRole" ("id") ON DELETE RESTRICT ON UPDATE CASCADE
);
-- CreateTable
CREATE TABLE "KmsAlias" (
"name" TEXT NOT NULL,
"accountId" TEXT NOT NULL,
"region" TEXT NOT NULL,
"kmsKeyId" TEXT NOT NULL,
"createdAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
"updatedAt" DATETIME NOT NULL,
PRIMARY KEY ("accountId", "region", "name"),
CONSTRAINT "KmsAlias_kmsKeyId_fkey" FOREIGN KEY ("kmsKeyId") REFERENCES "KmsKey" ("id") ON DELETE RESTRICT ON UPDATE CASCADE
);
-- CreateTable
CREATE TABLE "KmsKey" (
"id" TEXT NOT NULL PRIMARY KEY,
"enabled" BOOLEAN NOT NULL,
"usage" TEXT NOT NULL,
"description" TEXT NOT NULL,
"keySpec" TEXT NOT NULL,
"keyState" TEXT NOT NULL,
"origin" TEXT NOT NULL,
"multiRegion" BOOLEAN NOT NULL,
"policy" TEXT NOT NULL,
"key" BLOB NOT NULL,
"rotationPeriod" INTEGER,
"nextRotation" DATETIME,
"accountId" TEXT NOT NULL,
"region" TEXT NOT NULL,
"createdAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
"updatedAt" DATETIME NOT NULL
);
-- CreateTable
CREATE TABLE "Secret" (
"versionId" TEXT NOT NULL PRIMARY KEY,
"name" TEXT NOT NULL,
"description" TEXT,
"secretString" TEXT NOT NULL,
"accountId" TEXT NOT NULL,
"region" TEXT NOT NULL,
"createdAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
"deletionDate" DATETIME
);
-- CreateTable
CREATE TABLE "SnsTopic" (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"name" TEXT NOT NULL,
"accountId" TEXT NOT NULL,
"region" TEXT NOT NULL
);
-- CreateTable
CREATE TABLE "SnsTopicSubscription" (
"id" TEXT NOT NULL PRIMARY KEY,
"topicArn" TEXT NOT NULL,
"endpoint" TEXT,
"protocol" TEXT NOT NULL,
"accountId" TEXT NOT NULL,
"region" TEXT NOT NULL
);
-- CreateTable
CREATE TABLE "SqsQueue" (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"name" TEXT NOT NULL,
"accountId" TEXT NOT NULL,
"region" TEXT NOT NULL,
"createdAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
"updatedAt" DATETIME NOT NULL
);
-- CreateTable
CREATE TABLE "SqsQueueMessage" (
"id" TEXT NOT NULL PRIMARY KEY,
"queueId" INTEGER NOT NULL,
"senderId" TEXT NOT NULL,
"message" TEXT NOT NULL,
"inFlightRelease" DATETIME NOT NULL,
"createdAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
CONSTRAINT "SqsQueueMessage_queueId_fkey" FOREIGN KEY ("queueId") REFERENCES "SqsQueue" ("id") ON DELETE RESTRICT ON UPDATE CASCADE
);
-- CreateTable
CREATE TABLE "Tag" (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"arn" TEXT NOT NULL,
"name" TEXT NOT NULL,
"value" TEXT NOT NULL
);
-- CreateIndex
CREATE UNIQUE INDEX "Attribute_arn_name_key" ON "Attribute"("arn", "name");
-- CreateIndex
CREATE UNIQUE INDEX "IamRole_accountId_name_key" ON "IamRole"("accountId", "name");
-- CreateIndex
CREATE INDEX "Secret_name_idx" ON "Secret"("name");
-- CreateIndex
CREATE UNIQUE INDEX "SnsTopic_accountId_region_name_key" ON "SnsTopic"("accountId", "region", "name");
-- CreateIndex
CREATE UNIQUE INDEX "SqsQueue_accountId_region_name_key" ON "SqsQueue"("accountId", "region", "name");
-- CreateIndex
CREATE INDEX "SqsQueueMessage_queueId_idx" ON "SqsQueueMessage"("queueId");
-- CreateIndex
CREATE UNIQUE INDEX "Tag_arn_name_key" ON "Tag"("arn", "name");

View File

@@ -0,0 +1,34 @@
-- CreateTable
CREATE TABLE "S3Bucket" (
"id" TEXT NOT NULL PRIMARY KEY,
"name" TEXT NOT NULL,
"accountId" TEXT NOT NULL,
"region" TEXT NOT NULL,
"createdAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
);
-- CreateTable
CREATE TABLE "S3Object" (
"id" TEXT NOT NULL PRIMARY KEY,
"bucketId" TEXT NOT NULL,
"key" TEXT NOT NULL,
"versionId" TEXT,
"content" BLOB NOT NULL,
"contentType" TEXT NOT NULL DEFAULT 'application/octet-stream',
"size" INTEGER NOT NULL,
"etag" TEXT NOT NULL,
"metadata" TEXT NOT NULL DEFAULT '{}',
"storageClass" TEXT NOT NULL DEFAULT 'STANDARD',
"createdAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
"updatedAt" DATETIME NOT NULL,
CONSTRAINT "S3Object_bucketId_fkey" FOREIGN KEY ("bucketId") REFERENCES "S3Bucket" ("id") ON DELETE CASCADE ON UPDATE CASCADE
);
-- CreateIndex
CREATE UNIQUE INDEX "S3Bucket_accountId_region_name_key" ON "S3Bucket"("accountId", "region", "name");
-- CreateIndex
CREATE INDEX "S3Object_bucketId_idx" ON "S3Object"("bucketId");
-- CreateIndex
CREATE UNIQUE INDEX "S3Object_bucketId_key_key" ON "S3Object"("bucketId", "key");

View File

@@ -0,0 +1,21 @@
/*
Warnings:
- You are about to drop the column `accountId` on the `S3Bucket` table. All the data in the column will be lost.
- You are about to drop the column `region` on the `S3Bucket` table. All the data in the column will be lost.
*/
-- RedefineTables
PRAGMA defer_foreign_keys=ON;
PRAGMA foreign_keys=OFF;
CREATE TABLE "new_S3Bucket" (
"id" TEXT NOT NULL PRIMARY KEY,
"name" TEXT NOT NULL,
"createdAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
);
INSERT INTO "new_S3Bucket" ("createdAt", "id", "name") SELECT "createdAt", "id", "name" FROM "S3Bucket";
DROP TABLE "S3Bucket";
ALTER TABLE "new_S3Bucket" RENAME TO "S3Bucket";
CREATE UNIQUE INDEX "S3Bucket_name_key" ON "S3Bucket"("name");
PRAGMA foreign_keys=ON;
PRAGMA defer_foreign_keys=OFF;

View File

@@ -0,0 +1,15 @@
-- RedefineTables
PRAGMA defer_foreign_keys=ON;
PRAGMA foreign_keys=OFF;
CREATE TABLE "new_S3Bucket" (
"id" TEXT NOT NULL PRIMARY KEY,
"name" TEXT NOT NULL,
"tags" TEXT NOT NULL DEFAULT '{}',
"createdAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
);
INSERT INTO "new_S3Bucket" ("createdAt", "id", "name") SELECT "createdAt", "id", "name" FROM "S3Bucket";
DROP TABLE "S3Bucket";
ALTER TABLE "new_S3Bucket" RENAME TO "S3Bucket";
CREATE UNIQUE INDEX "S3Bucket_name_key" ON "S3Bucket"("name");
PRAGMA foreign_keys=ON;
PRAGMA defer_foreign_keys=OFF;

View File

@@ -0,0 +1,2 @@
-- AlterTable
ALTER TABLE "S3Bucket" ADD COLUMN "policy" TEXT;

View File

@@ -0,0 +1,32 @@
-- CreateTable
CREATE TABLE "IamRoleInlinePolicy" (
"id" TEXT NOT NULL PRIMARY KEY,
"roleName" TEXT NOT NULL,
"policyName" TEXT NOT NULL,
"policyDocument" TEXT NOT NULL,
"accountId" TEXT NOT NULL,
"createdAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
"updatedAt" DATETIME NOT NULL,
CONSTRAINT "IamRoleInlinePolicy_accountId_roleName_fkey" FOREIGN KEY ("accountId", "roleName") REFERENCES "IamRole" ("accountId", "name") ON DELETE CASCADE ON UPDATE CASCADE
);
-- RedefineTables
PRAGMA defer_foreign_keys=ON;
PRAGMA foreign_keys=OFF;
CREATE TABLE "new_S3Bucket" (
"id" TEXT NOT NULL PRIMARY KEY,
"name" TEXT NOT NULL,
"tags" TEXT NOT NULL DEFAULT '{}',
"policy" TEXT,
"acl" TEXT NOT NULL DEFAULT '{"Owner":{"ID":"local-user","DisplayName":"local-user"},"Grants":[]}',
"createdAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
);
INSERT INTO "new_S3Bucket" ("createdAt", "id", "name", "policy", "tags") SELECT "createdAt", "id", "name", "policy", "tags" FROM "S3Bucket";
DROP TABLE "S3Bucket";
ALTER TABLE "new_S3Bucket" RENAME TO "S3Bucket";
CREATE UNIQUE INDEX "S3Bucket_name_key" ON "S3Bucket"("name");
PRAGMA foreign_keys=ON;
PRAGMA defer_foreign_keys=OFF;
-- CreateIndex
CREATE UNIQUE INDEX "IamRoleInlinePolicy_accountId_roleName_policyName_key" ON "IamRoleInlinePolicy"("accountId", "roleName", "policyName");

View File

@@ -0,0 +1,20 @@
-- CreateTable
CREATE TABLE "ConsulKVEntry" (
"key" TEXT NOT NULL PRIMARY KEY,
"value" TEXT NOT NULL,
"flags" INTEGER NOT NULL DEFAULT 0,
"createIndex" INTEGER NOT NULL,
"modifyIndex" INTEGER NOT NULL,
"lockIndex" INTEGER NOT NULL DEFAULT 0,
"session" TEXT,
"datacenter" TEXT NOT NULL DEFAULT 'dc1',
"namespace" TEXT NOT NULL DEFAULT 'default',
"createdAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
"updatedAt" DATETIME NOT NULL
);
-- CreateIndex
CREATE INDEX "ConsulKVEntry_datacenter_idx" ON "ConsulKVEntry"("datacenter");
-- CreateIndex
CREATE INDEX "ConsulKVEntry_namespace_idx" ON "ConsulKVEntry"("namespace");

View File

@@ -0,0 +1,29 @@
/*
Warnings:
- You are about to alter the column `flags` on the `ConsulKVEntry` table. The data in that column could be lost. The data in that column will be cast from `Int` to `BigInt`.
*/
-- RedefineTables
PRAGMA defer_foreign_keys=ON;
PRAGMA foreign_keys=OFF;
CREATE TABLE "new_ConsulKVEntry" (
"key" TEXT NOT NULL PRIMARY KEY,
"value" TEXT NOT NULL,
"flags" BIGINT NOT NULL DEFAULT 0,
"createIndex" INTEGER NOT NULL,
"modifyIndex" INTEGER NOT NULL,
"lockIndex" INTEGER NOT NULL DEFAULT 0,
"session" TEXT,
"datacenter" TEXT NOT NULL DEFAULT 'dc1',
"namespace" TEXT NOT NULL DEFAULT 'default',
"createdAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
"updatedAt" DATETIME NOT NULL
);
INSERT INTO "new_ConsulKVEntry" ("createIndex", "createdAt", "datacenter", "flags", "key", "lockIndex", "modifyIndex", "namespace", "session", "updatedAt", "value") SELECT "createIndex", "createdAt", "datacenter", "flags", "key", "lockIndex", "modifyIndex", "namespace", "session", "updatedAt", "value" FROM "ConsulKVEntry";
DROP TABLE "ConsulKVEntry";
ALTER TABLE "new_ConsulKVEntry" RENAME TO "ConsulKVEntry";
CREATE INDEX "ConsulKVEntry_datacenter_idx" ON "ConsulKVEntry"("datacenter");
CREATE INDEX "ConsulKVEntry_namespace_idx" ON "ConsulKVEntry"("namespace");
PRAGMA foreign_keys=ON;
PRAGMA defer_foreign_keys=OFF;

View File

@@ -0,0 +1,2 @@
-- AlterTable
ALTER TABLE "ConsulKVEntry" ADD COLUMN "lockInfo" TEXT;

View File

@@ -0,0 +1,29 @@
/*
Warnings:
- You are about to drop the column `lockInfo` on the `ConsulKVEntry` table. All the data in the column will be lost.
*/
-- RedefineTables
PRAGMA defer_foreign_keys=ON;
PRAGMA foreign_keys=OFF;
CREATE TABLE "new_ConsulKVEntry" (
"key" TEXT NOT NULL PRIMARY KEY,
"value" TEXT NOT NULL,
"flags" BIGINT NOT NULL DEFAULT 0,
"createIndex" INTEGER NOT NULL,
"modifyIndex" INTEGER NOT NULL,
"lockIndex" INTEGER NOT NULL DEFAULT 0,
"session" TEXT,
"datacenter" TEXT NOT NULL DEFAULT 'dc1',
"namespace" TEXT NOT NULL DEFAULT 'default',
"createdAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
"updatedAt" DATETIME NOT NULL
);
INSERT INTO "new_ConsulKVEntry" ("createIndex", "createdAt", "datacenter", "flags", "key", "lockIndex", "modifyIndex", "namespace", "session", "updatedAt", "value") SELECT "createIndex", "createdAt", "datacenter", "flags", "key", "lockIndex", "modifyIndex", "namespace", "session", "updatedAt", "value" FROM "ConsulKVEntry";
DROP TABLE "ConsulKVEntry";
ALTER TABLE "new_ConsulKVEntry" RENAME TO "ConsulKVEntry";
CREATE INDEX "ConsulKVEntry_datacenter_idx" ON "ConsulKVEntry"("datacenter");
CREATE INDEX "ConsulKVEntry_namespace_idx" ON "ConsulKVEntry"("namespace");
PRAGMA foreign_keys=ON;
PRAGMA defer_foreign_keys=OFF;

View File

@@ -0,0 +1,3 @@
# Please do not edit this file manually
# It should be added in your version-control system (e.g., Git)
provider = "sqlite"

231
prisma/schema.prisma Normal file
View File

@@ -0,0 +1,231 @@
generator client {
provider = "prisma-client-js"
}
datasource db {
provider = "sqlite"
url = "file:local-aws-state.sqlite"
}
model Attribute {
id Int @id @default(autoincrement())
arn String
name String
value String
@@unique([arn, name])
}
model Audit {
id String @id
createdAt DateTime @default(now())
action String?
request String?
response String?
}
model IamRole {
id String @id
path String?
name String
assumeRolePolicy String?
description String?
maxSessionDuration Int?
permissionBoundaryArn String?
lastUsedDate DateTime?
lastUsedRegion String?
accountId String
createdAt DateTime @default(now())
policies IamRoleIamPolicyAttachment[]
inlinePolicies IamRoleInlinePolicy[]
@@unique([accountId, name])
}
model IamRoleInlinePolicy {
id String @id @default(uuid())
roleName String
policyName String
policyDocument String
accountId String
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
role IamRole @relation(fields: [accountId, roleName], references: [accountId, name], onDelete: Cascade)
@@unique([accountId, roleName, policyName])
}
model IamPolicy {
id String
version Int @default(1)
isDefault Boolean
path String?
name String
description String?
policy String
isAttachable Boolean @default(false)
accountId String
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
@@id([id, version])
}
model IamRoleIamPolicyAttachment {
iamRoleId String
iamPolicyId String
role IamRole @relation(fields: [iamRoleId], references: [id])
@@id([iamRoleId, iamPolicyId])
}
model KmsAlias {
name String
accountId String
region String
kmsKeyId String
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
kmsKey KmsKey @relation(fields: [kmsKeyId], references: [id])
@@id([accountId, region, name])
}
model KmsKey {
id String @id
enabled Boolean
usage String
description String
keySpec String
keyState String
origin String
multiRegion Boolean
policy String
key Bytes
rotationPeriod Int?
nextRotation DateTime?
accountId String
region String
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
aliases KmsAlias[]
}
model Secret {
versionId String @id
name String
description String?
secretString String
accountId String
region String
createdAt DateTime @default(now())
deletionDate DateTime?
@@index([name])
}
model SnsTopic {
id Int @id @default(autoincrement())
name String
accountId String
region String
@@unique([accountId, region, name])
}
model SnsTopicSubscription {
id String @id
topicArn String
endpoint String?
protocol String
accountId String
region String
}
model SqsQueue {
id Int @id @default(autoincrement())
name String
accountId String
region String
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
messages SqsQueueMessage[]
@@unique([accountId, region, name])
}
model SqsQueueMessage {
id String @id
queueId Int
senderId String
message String
inFlightRelease DateTime
createdAt DateTime @default(now())
queue SqsQueue @relation(fields: [queueId], references: [id])
@@index([queueId])
}
model Tag {
id Int @id @default(autoincrement())
arn String
name String
value String
@@unique([arn, name])
}
model S3Bucket {
id String @id
name String @unique
tags String @default("{}")
policy String?
acl String @default("{\"Owner\":{\"ID\":\"local-user\",\"DisplayName\":\"local-user\"},\"Grants\":[]}")
createdAt DateTime @default(now())
objects S3Object[]
}
model S3Object {
id String @id
bucketId String
key String
versionId String?
content Bytes
contentType String @default("application/octet-stream")
size Int
etag String
metadata String @default("{}")
storageClass String @default("STANDARD")
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
bucket S3Bucket @relation(fields: [bucketId], references: [id], onDelete: Cascade)
@@unique([bucketId, key])
@@index([bucketId])
}
model ConsulKVEntry {
key String @id
value String // Base64 encoded
flags BigInt @default(0)
createIndex Int
modifyIndex Int
lockIndex Int @default(0)
session String?
datacenter String @default("dc1")
namespace String @default("default")
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
@@index([datacenter])
@@index([namespace])
}

View File

@@ -0,0 +1,25 @@
import { ArgumentsHost, Catch, ExceptionFilter } from "@nestjs/common";
import { Response } from 'express';
import { AwsException } from "../aws-shared-entities/aws-exceptions";
import { IRequest } from "./request.context";
import { Format } from "../abstract-action.handler";
@Catch(AwsException)
export class AwsExceptionFilter implements ExceptionFilter {
catch(exception: AwsException, host: ArgumentsHost) {
const ctx = host.switchToHttp();
const request = ctx.getRequest<IRequest>();
const response = ctx.getResponse<Response>();
exception.requestId = request.context.requestId;
if (request.context.format === Format.Xml) {
const xml = exception.toXml();
return response.status(exception.statusCode).send(xml);
}
const [newError, newHeaders] = exception.toJson();
response.setHeaders(new Map(Object.entries(newHeaders)));
return response.status(exception.statusCode).json(newError.getResponse());
}
}

View File

@@ -0,0 +1,21 @@
import { Request } from "express";
import { Action } from "../action.enum";
import { AwsProperties, Format } from "../abstract-action.handler";
export interface RequestContext {
action?: Action;
format?: Format;
awsProperties: AwsProperties;
readonly requestId: string;
}
export interface IRequest extends Request {
context: RequestContext;
headers: {
'x-amz-target'?: string;
},
body: {
'Action'?: string;
}
}

View File

@@ -1,19 +0,0 @@
import { INestApplication } from '@nestjs/common';
import { Test, TestingModule } from '@nestjs/testing';
import { AppModule } from '../app.module';
const globalSetup = async (_globalConfig, _projectConfig) => {
const module: TestingModule = await Test.createTestingModule({
imports: [AppModule],
}).compile();
const app: INestApplication = module.createNestApplication();
await app.listen(4566);
globalThis.__TESTMODULE__ = module;
globalThis.__NESTAPP__ = app;
globalThis.__ENDPOINT__ = 'http://127.0.0.1:4566';
}
export default globalSetup;

View File

@@ -1,8 +0,0 @@
import { INestApplication } from '@nestjs/common';
const globalTeardown = async (_globalConfig, _projectConfig) => {
await (globalThis.__NESTAPP__ as INestApplication).close();
}
export default globalTeardown;

View File

@@ -0,0 +1,9 @@
import { Module } from "@nestjs/common";
import { PrismaService } from "./prisma.service";
@Module({
imports: [],
providers: [PrismaService],
exports: [PrismaService],
})
export class PrismaModule {}

View File

@@ -0,0 +1,8 @@
import { OnModuleInit } from "@nestjs/common";
import { PrismaClient } from "@prisma/client";
export class PrismaService extends PrismaClient implements OnModuleInit {
async onModuleInit() {
await this.$connect();
}
}

View File

@@ -1,6 +1,7 @@
import { randomUUID } from 'crypto';
import { Action } from './action.enum';
import * as uuid from 'uuid';
import * as Joi from 'joi';
import { RequestContext } from './_context/request.context';
export type AwsProperties = {
accountId: string;
@@ -17,42 +18,43 @@ export abstract class AbstractActionHandler<T = Record<string, string | number |
audit = true;
abstract format: Format;
abstract action: Action;
abstract action: Action | Action[];
abstract validator: Joi.ObjectSchema<T>;
protected abstract handle(queryParams: T, awsProperties: AwsProperties): Record<string, any> | void;
protected abstract handle(queryParams: T, context: RequestContext): Record<string, any> | void;
async getResponse(queryParams: T, awsProperties: AwsProperties) {
async getResponse(queryParams: T, context: RequestContext) {
if (this.format === Format.Xml) {
return await this.getXmlResponse(queryParams, awsProperties);
return await this.getXmlResponse(queryParams, context);
}
return await this.getJsonResponse(queryParams, awsProperties);
return await this.getJsonResponse(queryParams, context);
}
private async getXmlResponse(queryParams: T, awsProperties: AwsProperties) {
private async getXmlResponse(queryParams: T, context: RequestContext) {
const response = {
'@': {
xmlns: "https://sns.amazonaws.com/doc/2010-03-31/"
},
ResponseMetadata: {
RequestId: uuid.v4(),
RequestId: randomUUID(),
}
}
const result = await this.handle(queryParams, awsProperties);
const result = await this.handle(queryParams, context);
if (!result) {
return response;
}
const action = Array.isArray(this.action) ? this.action[0] : this.action;
return {
[`${this.action}Result`]: {
[`${action}Result`]: {
...result,
}
}
}
private async getJsonResponse(queryParams: T, awsProperties: AwsProperties) {
const result = await this.handle(queryParams, awsProperties);
private async getJsonResponse(queryParams: T, context: RequestContext) {
const result = await this.handle(queryParams, context);
if (result) {
return result;
}

View File

@@ -1,5 +1,4 @@
export enum Action {
// IAM
IamAddClientIDToOpenIDConnectProvider = 'AddClientIDToOpenIDConnectProvider',
IamAddRoleToInstanceProfile = 'AddRoleToInstanceProfile',
@@ -301,4 +300,58 @@ export enum Action {
SqsSetQueueAttributes = 'SetQueueAttributes',
SqsTagQueue = 'TagQueue',
SqsUntagQueue = 'UntagQueue',
// V2 SQS
V2_SqsAddPermisson = 'AmazonSQS.AddPermission',
V2_SqsChangeMessageVisibility = 'AmazonSQS.ChangeMessageVisibility',
V2_SqsChangeMessageVisibilityBatch = 'AmazonSQS.ChangeMessageVisibilityBatch',
V2_SqsCreateQueue = 'AmazonSQS.CreateQueue',
V2_SqsDeleteMessage = 'AmazonSQS.DeleteMessage',
V2_SqsDeleteMessageBatch = 'AmazonSQS.DeleteMessageBatch',
V2_SqsDeleteQueue = 'AmazonSQS.DeleteQueue',
V2_SqsGetQueueAttributes = 'AmazonSQS.GetQueueAttributes',
V2_SqsGetQueueUrl = 'AmazonSQS.GetQueueUrl',
V2_SqsListDeadLetterSourceQueues = 'AmazonSQS.ListDeadLetterSourceQueues',
V2_SqsListQueues = 'AmazonSQS.ListQueues',
V2_SqsListQueueTags = 'AmazonSQS.ListQueueTags',
V2_SqsPurgeQueue = 'AmazonSQS.PurgeQueue',
V2_SqsReceiveMessage = 'AmazonSQS.ReceiveMessage',
V2_SqsRemovePermission = 'AmazonSQS.RemovePermission',
V2_SqsSendMessage = 'AmazonSQS.SendMessage',
V2_SqsSendMessageBatch = 'AmazonSQS.SendMessageBatch',
V2_SqsSetQueueAttributes = 'AmazonSQS.SetQueueAttributes',
V2_SqsTagQueue = 'AmazonSQS.TagQueue',
V2_SqsUntagQueue = 'AmazonSQS.UntagQueue',
// S3
S3AbortMultipartUpload = 'AbortMultipartUpload',
S3CompleteMultipartUpload = 'CompleteMultipartUpload',
S3CreateBucket = 'CreateBucket',
S3CreateMultipartUpload = 'CreateMultipartUpload',
S3DeleteBucket = 'DeleteBucket',
S3DeleteObject = 'DeleteObject',
S3GetBucketAcl = 'GetBucketAcl',
S3GetBucketPolicy = 'GetBucketPolicy',
S3GetBucketTagging = 'GetBucketTagging',
S3GetObject = 'GetObject',
S3HeadBucket = 'HeadBucket',
S3HeadObject = 'HeadObject',
S3ListBuckets = 'ListBuckets',
S3ListObjects = 'ListObjects',
S3ListObjectsV2 = 'ListObjectsV2',
S3PutBucketAcl = 'PutBucketAcl',
S3PutBucketTagging = 'PutBucketTagging',
S3PutObject = 'PutObject',
S3UploadPart = 'UploadPart',
// STS
StsAssumeRole = 'AssumeRole',
StsAssumeRoleWithSaml = 'AssumeRoleWithSaml',
StsAssumeRoleWithWebIdentity = 'AssumeRoleWithWebIdentity',
StsAssumeRoot = 'AssumeRoot',
StsDecodeAuthorizationMessage = 'DecodeAuthorizationMessage',
StsGetAccessKeyInfo = 'GetAccessKeyInfo',
StsGetCallerIdentity = 'GetCallerIdentity',
StsGetFederationToken = 'GetFederationToken',
StsGetSessionToken = 'GetSessionToken',
}

View File

@@ -1,17 +1,23 @@
import { BadRequestException, Body, Controller, Inject, Post, Headers, Req, HttpCode, UseInterceptors } from '@nestjs/common';
import { ActionHandlers } from './app.constants';
import * as Joi from 'joi';
import { Action } from './action.enum';
import { AbstractActionHandler, Format } from './abstract-action.handler';
import * as js2xmlparser from 'js2xmlparser';
import { All, Body, Controller, Delete, Get, Headers, HttpCode, Inject, Post, Put, Query, Req, Res, UseInterceptors } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import { CommonConfig } from './config/common-config.interface';
import { Request } from 'express';
import { Response } from 'express';
import * as Joi from 'joi';
import * as js2xmlparser from 'js2xmlparser';
import { AbstractActionHandler, Format } from './abstract-action.handler';
import { Action } from './action.enum';
import { ActionHandlers } from './app.constants';
import { AuditInterceptor } from './audit/audit.interceptor';
import { CommonConfig } from './config/common-config.interface';
import { InvalidAction, ValidationError } from './aws-shared-entities/aws-exceptions';
import { IRequest } from './_context/request.context';
type QueryParams = {
__path: string;
} & Record<string, string>;
@Controller()
export class AppController {
constructor(
@Inject(ActionHandlers)
private readonly actionHandlers: ActionHandlers,
@@ -21,44 +27,40 @@ export class AppController {
@Post()
@HttpCode(200)
@UseInterceptors(AuditInterceptor)
async post(
@Req() request: Request,
@Body() body: Record<string, any>,
@Headers() headers: Record<string, any>,
) {
async post(@Req() request: IRequest, @Body() body: Record<string, any>, @Headers() headers: Record<string, any>) {
const lowerCasedHeaders = Object.keys(headers).reduce((o, k) => {
o[k.toLocaleLowerCase()] = headers[k];
return o;
}, {})
}, {} as Record<string, string>);
const queryParams = { __path: request.path, ...body, ...lowerCasedHeaders };
const queryParams: QueryParams = { __path: request.path, ...body, ...lowerCasedHeaders };
const actionKey = queryParams['x-amz-target'] ? 'x-amz-target' : 'Action';
const { error: actionError } = Joi.object({
[actionKey]: Joi.string().valid(...Object.values(Action)).required(),
[actionKey]: Joi.string()
.valid(...Object.values(Action))
.required(),
}).validate(queryParams, { allowUnknown: true });
if (actionError) {
throw new BadRequestException(actionError.message, { cause: actionError });
throw new InvalidAction(actionError.message);
}
const action = queryParams[actionKey];
const action = queryParams[actionKey] as Action;
const handler: AbstractActionHandler = this.actionHandlers[action];
const { error: validatorError, value: validQueryParams } = handler.validator.validate(queryParams, { allowUnknown: true, abortEarly: false });
const { error: validatorError, value: validQueryParams } = handler.validator.validate(queryParams, {
allowUnknown: true,
abortEarly: false,
});
if (validatorError) {
throw new BadRequestException(validatorError.message, { cause: validatorError });
throw new ValidationError(validatorError.message);
}
const awsProperties = {
accountId: this.configService.get('AWS_ACCOUNT_ID'),
region: this.configService.get('AWS_REGION'),
host: `${this.configService.get('PROTO')}://${this.configService.get('HOST')}:${this.configService.get('PORT')}`,
};
const jsonResponse = await handler.getResponse(validQueryParams, request.context);
const jsonResponse = await handler.getResponse(validQueryParams, awsProperties);
if (handler.format === Format.Xml) {
return js2xmlparser.parse(`${handler.action}Response`, jsonResponse);
const action = Array.isArray(handler.action) ? handler.action[0] : handler.action;
return js2xmlparser.parse(`${action}Response`, jsonResponse);
}
return jsonResponse;
}

View File

@@ -1,22 +1,22 @@
import { Module } from '@nestjs/common';
import { ConfigModule, ConfigService } from '@nestjs/config';
import { TypeOrmModule } from '@nestjs/typeorm';
import { ConfigModule } from '@nestjs/config';
import { ActionHandlers } from './app.constants';
import { CommonConfig } from './config/common-config.interface';
import { AppController } from './app.controller';
import { AuditInterceptor } from './audit/audit.interceptor';
import { AwsSharedEntitiesModule } from './aws-shared-entities/aws-shared-entities.module';
import localConfig from './config/local.config';
import { KMSHandlers } from './kms/kms.constants';
import { KmsModule } from './kms/kms.module';
import { SecretsManagerHandlers } from './secrets-manager/secrets-manager.constants';
import { SecretsManagerModule } from './secrets-manager/secrets-manager.module';
import { SnsHandlers } from './sns/sns.constants';
import { SnsModule } from './sns/sns.module';
import { AppController } from './app.controller';
import { AwsSharedEntitiesModule } from './aws-shared-entities/aws-shared-entities.module';
import { SecretsManagerModule } from './secrets-manager/secrets-manager.module';
import { SecretsManagerHandlers } from './secrets-manager/secrets-manager.constants';
import { SqsModule } from './sqs/sqs.module';
import { SqsHandlers } from './sqs/sqs.constants';
import { Audit } from './audit/audit.entity';
import { AuditInterceptor } from './audit/audit.interceptor';
import { KmsModule } from './kms/kms.module';
import { KMSHandlers } from './kms/kms.constants';
import { configValidator } from './config/config.validator';
import { SqsModule } from './sqs/sqs.module';
import { PrismaModule } from './_prisma/prisma.module';
import { StsModule } from './sts/sts.module';
import { StsHandlers } from './sts/sts.constants';
import { IamModule } from './iam/iam.module';
import { IAMHandlers } from './iam/iam.constants';
@@ -26,39 +26,22 @@ import { IAMHandlers } from './iam/iam.constants';
load: [localConfig],
isGlobal: true,
}),
TypeOrmModule.forRootAsync({
inject: [ConfigService],
useFactory: (configService: ConfigService<CommonConfig>) => ({
type: 'sqlite',
database: configService.get('DB_DATABASE') === ':memory:' ? configService.get('DB_DATABASE') : `${__dirname}/../data/${configService.get('DB_DATABASE')}`,
logging: configService.get('DB_LOGGING'),
synchronize: configService.get('DB_SYNCHRONIZE'),
entities: [__dirname + '/**/*.entity{.ts,.js}'],
}),
}),
TypeOrmModule.forFeature([Audit]),
PrismaModule,
AwsSharedEntitiesModule,
IamModule,
KmsModule,
SecretsManagerModule,
SnsModule,
SqsModule,
AwsSharedEntitiesModule,
],
controllers: [
AppController,
StsModule,
],
controllers: [AppController],
providers: [
AuditInterceptor,
{
provide: ActionHandlers,
useFactory: (...args) => args.reduce((m, hs) => ({ ...m, ...hs }), {}),
inject: [
SnsHandlers,
SqsHandlers,
SecretsManagerHandlers,
KMSHandlers,
IAMHandlers,
],
inject: [IAMHandlers, KMSHandlers, SecretsManagerHandlers, SnsHandlers, SqsHandlers, StsHandlers],
},
],
})

View File

@@ -0,0 +1,11 @@
import { Controller } from "@nestjs/common";
import { AuditService } from "./audit.service";
@Controller('_audit')
export class AuditController {
constructor(
private readonly auditService: AuditService,
) {}
}

View File

@@ -1,20 +0,0 @@
import { BaseEntity, Column, CreateDateColumn, Entity, PrimaryColumn } from 'typeorm';
@Entity('audit')
export class Audit extends BaseEntity {
@PrimaryColumn()
id: string;
@CreateDateColumn()
createdAt: string;
@Column({ nullable: true })
action: string;
@Column({ nullable: true })
request: string;
@Column({ nullable: true })
response: string;
}

View File

@@ -1,56 +1,149 @@
import { CallHandler, ExecutionContext, Inject, Injectable, NestInterceptor } from '@nestjs/common';
import { InjectRepository } from '@nestjs/typeorm';
import { Observable, tap } from 'rxjs';
import { Repository } from 'typeorm';
import { Audit } from './audit.entity';
import * as uuid from 'uuid';
import {
CallHandler,
ExecutionContext,
HttpException,
Inject,
Injectable,
Logger,
NestInterceptor,
RequestTimeoutException,
} from '@nestjs/common';
import { randomUUID } from 'crypto';
import { catchError, Observable, tap, throwError } from 'rxjs';
import { Request as ExpressRequest, Response } from 'express';
import * as Joi from 'joi';
import { PrismaService } from '../_prisma/prisma.service';
import { ActionHandlers } from '../app.constants';
import { Action } from '../action.enum';
import { Format } from '../abstract-action.handler';
import { AwsException, InternalFailure } from '../aws-shared-entities/aws-exceptions';
import { IRequest, RequestContext } from '../_context/request.context';
import { ConfigService } from '@nestjs/config';
@Injectable()
export class AuditInterceptor<T> implements NestInterceptor<T, Response> {
private readonly logger = new Logger(AuditInterceptor.name);
constructor(
@InjectRepository(Audit)
private readonly auditRepo: Repository<Audit>,
@Inject(ActionHandlers)
private readonly handlers: ActionHandlers,
private readonly prismaService: PrismaService,
private readonly configService: ConfigService,
) {}
intercept(context: ExecutionContext, next: CallHandler<T>): Observable<any> {
const awsProperties = {
accountId: this.configService.get('AWS_ACCOUNT_ID'),
region: this.configService.get('AWS_REGION'),
host: `${this.configService.get('PROTO')}://${this.configService.get('HOST')}:${this.configService.get('PORT')}`,
};
const requestContext: RequestContext = {
requestId: randomUUID(),
awsProperties,
};
const requestId = uuid.v4();
const httpContext = context.switchToHttp();
const request = httpContext.getRequest<IRequest>();
request.context = requestContext;
const request = httpContext.getRequest();
const targetHeaderKey = Object.keys(request.headers).find( k => k.toLocaleLowerCase() === 'x-amz-target');
const action = request.headers[targetHeaderKey] ? request.headers[targetHeaderKey] : request.body.Action;
const hasTargetHeader = Object.keys(request.headers).some(k => k.toLocaleLowerCase() === 'x-amz-target');
const action = hasTargetHeader ? request.headers['x-amz-target'] : request.body.Action;
const { value: resolvedAction } = Joi.string()
.required()
.valid(...Object.values(Action))
.validate(action) as { value: Action | undefined };
requestContext.action = resolvedAction;
const response = context.switchToHttp().getResponse();
const response = context.switchToHttp().getResponse<Response>();
response.header('x-amzn-RequestId', requestContext.requestId);
response.header('x-amzn-RequestId', requestId);
const requestStartTime = Date.now();
if (!this.handlers[action]?.audit) {
return next.handle();
if (!resolvedAction || !this.handlers[resolvedAction]?.audit) {
return next.handle().pipe(
catchError(async (error: Error) => {
const duration = Date.now() - requestStartTime;
this.logger.error(`${action} - ${duration}ms - Error: ${error.message}`);
await this.prismaService.audit.create({
data: {
id: requestContext.requestId,
action,
request: JSON.stringify({
__path: request.path,
__accountId: requestContext.awsProperties.accountId,
__region: requestContext.awsProperties.region,
...request.headers,
...request.body,
}),
response: JSON.stringify(error),
},
});
return error;
}),
);
}
const handler = this.handlers[resolvedAction];
requestContext.format = handler.format;
return next.handle().pipe(
catchError((error: Error) => {
return throwError(() => {
if (error instanceof AwsException) {
return error;
}
const defaultError = new InternalFailure('Unexpected local AWS exception...');
this.logger.error(error.message);
defaultError.requestId = requestContext.requestId;
return defaultError;
});
}),
tap({
next: async data => {
const duration = Date.now() - requestStartTime;
this.logger.log(`${action} - ${duration}ms`);
next: async (data) => await this.auditRepo.create({
id: requestId,
action,
request: JSON.stringify({ __path: request.path, ...request.headers, ...request.body }),
response: JSON.stringify(data),
}).save(),
await this.prismaService.audit.create({
data: {
id: requestContext.requestId,
action,
request: JSON.stringify({
__path: request.path,
__accountId: requestContext.awsProperties.accountId,
__region: requestContext.awsProperties.region,
...request.headers,
...request.body,
}),
response: JSON.stringify(data),
},
});
},
error: async (error) => await this.auditRepo.create({
id: requestId,
action,
request: JSON.stringify({ __path: request.path, ...request.headers, ...request.body }),
response: JSON.stringify(error),
}).save(),
})
error: async error => {
const duration = Date.now() - requestStartTime;
this.logger.error(`${action} - ${duration}ms - Error: ${error.message}`);
await this.prismaService.audit.create({
data: {
id: requestContext.requestId,
action,
request: JSON.stringify({
__path: request.path,
__accountId: requestContext.awsProperties.accountId,
__region: requestContext.awsProperties.region,
...request.headers,
...request.body,
}),
response: JSON.stringify(error),
},
});
},
}),
);
}
}

12
src/audit/audit.module.ts Normal file
View File

@@ -0,0 +1,12 @@
import { Module } from "@nestjs/common";
import { PrismaModule } from "../_prisma/prisma.module";
import { AuditController } from "./audit.controller";
import { AuditInterceptor } from "./audit.interceptor";
@Module({
imports: [PrismaModule],
controllers: [AuditController],
providers: [AuditInterceptor],
})
export class AuditModule {}

View File

@@ -0,0 +1,14 @@
import { Injectable } from "@nestjs/common";
import { PrismaService } from "../_prisma/prisma.service";
@Injectable()
export class AuditService {
constructor(
private readonly prismaService: PrismaService,
) {}
}

View File

@@ -1,18 +0,0 @@
import { BaseEntity, Column, Entity, Index, PrimaryColumn, PrimaryGeneratedColumn } from 'typeorm';
@Entity('attributes')
export class Attribute extends BaseEntity {
@PrimaryGeneratedColumn({ name: 'id' })
id: string;
@Column({ name: 'arn', nullable: false })
@Index()
arn: string;
@Column({ name: 'name', nullable: false })
name: string;
@Column({ name: 'value', nullable: false })
value: string;
}

View File

@@ -1,75 +1,114 @@
import { Injectable } from '@nestjs/common';
import { InjectRepository } from '@nestjs/typeorm';
import { In, Repository } from 'typeorm';
import { Attribute } from './attributes.entity';
import { CreateAttributeDto } from './create-attribute.dto';
import { Attribute, Prisma } from '@prisma/client';
import { PrismaService } from '../_prisma/prisma.service';
import { breakdownAwsQueryParam } from '../util/breakdown-aws-query-param';
const ResourcePolicyName = 'ResourcePolicy';
@Injectable()
export class AttributesService {
constructor(
@InjectRepository(Attribute)
private readonly repo: Repository<Attribute>,
) {}
constructor(private readonly prismaService: PrismaService) {}
async getByArn(arn: string): Promise<Attribute[]> {
return await this.repo.find({ where: { arn }});
return await this.prismaService.attribute.findMany({ where: { arn } });
}
async getResourcePolicyByArn(arn: string): Promise<Attribute> {
return await this.repo.findOne({ where: { arn, name: ResourcePolicyName }});
async getResourcePolicyByArn(arn: string): Promise<Attribute | null> {
return await this.prismaService.attribute.findFirst({ where: { arn, name: ResourcePolicyName } });
}
async getByArnAndName(arn: string, name: string): Promise<Attribute> {
return await this.repo.findOne({ where: { arn, name }});
async getByArnAndName(arn: string, name: string): Promise<Attribute | null> {
return await this.prismaService.attribute.findFirst({ where: { arn, name } });
}
async getByArnAndNames(arn: string, names: string[]): Promise<Attribute[]> {
return await this.repo.find({ where: { arn, name: In(names) }});
return await this.prismaService.attribute.findMany({
where: {
arn,
name: {
in: names,
},
},
});
}
async createResourcePolicy(arn: string, value: string): Promise<Attribute> {
return await this.create({arn, value, name: ResourcePolicyName });
return await this.create({ arn, value, name: ResourcePolicyName });
}
async create(dto: CreateAttributeDto): Promise<Attribute> {
return await this.repo.save(dto);
async create(data: Prisma.AttributeCreateArgs['data']): Promise<Attribute> {
return await this.prismaService.attribute.upsert({
where: {
arn_name: {
arn: data.arn,
name: data.name,
},
},
update: {
value: data.value,
},
create: data,
});
}
async deleteByArn(arn: string) {
await this.repo.delete({ arn });
async deleteByArn(arn: string): Promise<void> {
await this.prismaService.attribute.deleteMany({ where: { arn } });
}
async deleteByArnAndName(arn: string, name: string) {
await this.repo.delete({ arn, name });
async deleteByArnAndName(arn: string, name: string): Promise<void> {
await this.prismaService.attribute.deleteMany({ where: { arn, name } });
}
async createMany(arn: string, records: { key: string, value: string }[]): Promise<void> {
for (const record of records) {
await this.create({ arn, name: record.key, value: record.value });
}
async createMany(arn: string, records: { key: string; value: string }[]): Promise<void> {
// Use upsert to handle both create and update cases
await Promise.all(
records
.filter(r => !!r)
.map(r =>
this.prismaService.attribute.upsert({
where: {
arn_name: {
arn,
name: r.key,
},
},
update: {
value: r.value,
},
create: {
name: r.key,
value: r.value,
arn,
},
}),
),
);
}
static attributePairs(queryParams: Record<string, string>): { key: string, value: string }[] {
const pairs = [null];
static attributePairs(queryParams: Record<string, string>): { key: string; value: string }[] {
const pairs: { key: string; value: string }[] = [];
for (const param of Object.keys(queryParams)) {
const [type, _, idx, slot] = param.split('.');
const components = breakdownAwsQueryParam(param);
if (!components) {
continue;
}
const [type, _, idx, slot] = components;
if (type === 'Attributes') {
if (!pairs[+idx]) {
pairs[+idx] = { key: '', value: ''};
if (!pairs[idx]) {
pairs[idx] = { key: '', value: '' };
}
pairs[+idx][slot] = queryParams[param];
pairs[idx][slot] = queryParams[param];
}
}
pairs.shift();
return pairs;
}
static getXmlSafeAttributesMap(attributes: Record<string, string>) {
return { Attributes: { entry: Object.keys(attributes).map(key => ({ key, value: attributes[key] })) } }
return { Attributes: { entry: Object.keys(attributes).map(key => ({ key, value: attributes[key] })) } };
}
}

View File

@@ -0,0 +1,204 @@
import { HttpException, HttpStatus } from "@nestjs/common";
import { randomUUID } from "crypto";
import * as js2xmlparser from 'js2xmlparser';
export abstract class AwsException {
requestId: string = randomUUID();
constructor(
readonly message: string,
readonly errorType: string,
readonly statusCode: HttpStatus,
) {}
toXml(): string {
return js2xmlparser.parse(`ErrorResponse`, {
RequestId: this.requestId,
Error: {
Code: this.errorType,
Message: this.message,
}
});
}
toJson(): [HttpException, Record<string, string>] {
return [
new HttpException({
message: this.message,
__type: this.errorType,
}, this.statusCode),
{
'Server': 'NestJS/local-aws',
'X-Amzn-Errortype': this.errorType,
'x-amzn-requestid': this.requestId,
}
];
}
}
export class AccessDeniedException extends AwsException {
constructor(message: string) {
super(
message,
AccessDeniedException.name,
HttpStatus.BAD_REQUEST,
)
}
}
export class IncompleteSignature extends AwsException {
constructor(message: string) {
super(
message,
IncompleteSignature.name,
HttpStatus.BAD_REQUEST,
)
}
}
export class InternalFailure extends AwsException {
constructor(message: string) {
super(
message,
InternalFailure.name,
HttpStatus.INTERNAL_SERVER_ERROR,
)
}
}
export class InvalidAction extends AwsException {
constructor(message: string) {
super(
message,
InvalidAction.name,
HttpStatus.BAD_REQUEST,
)
}
}
export class InvalidClientTokenId extends AwsException {
constructor(message: string) {
super(
message,
InvalidClientTokenId.name,
HttpStatus.FORBIDDEN,
)
}
}
export class NotAuthorized extends AwsException {
constructor(message: string) {
super(
message,
NotAuthorized.name,
HttpStatus.BAD_REQUEST,
)
}
}
export class OptInRequired extends AwsException {
constructor(message: string) {
super(
message,
OptInRequired.name,
HttpStatus.FORBIDDEN,
)
}
}
export class RequestExpired extends AwsException {
constructor(message: string) {
super(
message,
RequestExpired.name,
HttpStatus.BAD_REQUEST,
)
}
}
export class ServiceUnavailable extends AwsException {
constructor(message: string) {
super(
message,
ServiceUnavailable.name,
HttpStatus.SERVICE_UNAVAILABLE,
)
}
}
export class ThrottlingException extends AwsException {
constructor(message: string) {
super(
message,
ThrottlingException.name,
HttpStatus.BAD_REQUEST,
)
}
}
export class ValidationError extends AwsException {
constructor(message: string) {
super(
message,
ValidationError.name,
HttpStatus.BAD_REQUEST,
)
}
}
export class NotFoundException extends AwsException {
constructor() {
super(
'The request was rejected because the specified entity or resource could not be found.',
NotFoundException.name,
HttpStatus.BAD_REQUEST,
)
}
}
export class InvalidArnException extends AwsException {
constructor(message: string) {
super(
message,
InvalidArnException.name,
HttpStatus.BAD_REQUEST,
)
}
}
export class UnsupportedOperationException extends AwsException {
constructor(message: string) {
super(
message,
UnsupportedOperationException.name,
HttpStatus.BAD_REQUEST,
)
}
}
export class EntityAlreadyExists extends AwsException {
constructor(message: string) {
super(
message,
EntityAlreadyExists.name,
HttpStatus.CONFLICT,
)
}
}
export class NoSuchEntity extends AwsException {
constructor() {
super(
'The request was rejected because it referenced a resource entity that does not exist. The error message describes the resource.',
NoSuchEntity.name,
HttpStatus.NOT_FOUND,
)
}
}
export class NotFound extends AwsException {
constructor() {
super(
'Indicates that the requested resource does not exist.',
NotFound.name,
HttpStatus.NOT_FOUND,
)
}
}
export class QueueNameExists extends AwsException {
constructor() {
super(
'A queue with this name already exists. Amazon SQS returns this error only if the request includes attributes whose values differ from those of the existing queue.',
QueueNameExists.name,
HttpStatus.BAD_REQUEST,
)
}
}

View File

@@ -1,12 +1,11 @@
import { Module } from '@nestjs/common';
import { TypeOrmModule } from '@nestjs/typeorm';
import { Attribute } from './attributes.entity';
import { AttributesService } from './attributes.service';
import { Tag } from './tags.entity';
import { TagsService } from './tags.service';
import { PrismaModule } from '../_prisma/prisma.module';
@Module({
imports: [TypeOrmModule.forFeature([Attribute, Tag])],
imports: [PrismaModule],
providers: [AttributesService, TagsService],
exports: [AttributesService, TagsService],
})

View File

@@ -1,5 +0,0 @@
export interface CreateAttributeDto {
arn: string;
name: string;
value: string;
}

View File

@@ -1,5 +0,0 @@
export interface CreateTagDto {
arn: string;
name: string;
value: string;
}

View File

@@ -1,18 +0,0 @@
import { BaseEntity, Column, Entity, Index, PrimaryColumn, PrimaryGeneratedColumn } from 'typeorm';
@Entity('tags')
export class Tag extends BaseEntity {
@PrimaryGeneratedColumn({ name: 'id' })
id: string;
@Column({ name: 'arn', nullable: false})
@Index()
arn: string;
@Column({ name: 'name', nullable: false })
name: string;
@Column({ name: 'value', nullable: false })
value: string;
}

View File

@@ -1,54 +1,77 @@
import { Injectable } from '@nestjs/common';
import { InjectRepository } from '@nestjs/typeorm';
import { Repository } from 'typeorm';
import { Tag } from './tags.entity';
import { CreateTagDto } from './create-tag.dto';
import { Prisma, Tag } from '@prisma/client';
import { PrismaService } from '../_prisma/prisma.service';
import { breakdownAwsQueryParam } from '../util/breakdown-aws-query-param';
@Injectable()
export class TagsService {
constructor(
@InjectRepository(Tag)
private readonly repo: Repository<Tag>,
) {}
constructor(private readonly prismaService: PrismaService) {}
async getByArn(arn: string): Promise<Tag[]> {
return await this.repo.find({ where: { arn }});
return await this.prismaService.tag.findMany({ where: { arn } });
}
async create(dto: CreateTagDto): Promise<Tag> {
return await this.repo.save(dto);
async create(data: Prisma.TagCreateArgs['data']): Promise<Tag> {
return await this.prismaService.tag.create({ data });
}
async createMany(arn: string, records: { Key: string, Value: string }[]): Promise<void> {
async createMany(arn: string, records: { key: string; value: string }[]): Promise<void> {
if (records.length === 0) {
return;
}
// Upsert each tag individually to handle duplicates
for (const record of records) {
await this.create({ arn, name: record.Key, value: record.Value });
await this.prismaService.tag.upsert({
where: {
arn_name: {
arn,
name: record.key,
},
},
update: {
value: record.value,
},
create: {
arn,
name: record.key,
value: record.value,
},
});
}
}
async deleteByArn(arn: string) {
await this.repo.delete({ arn });
async deleteByArn(arn: string): Promise<void> {
await this.prismaService.tag.deleteMany({ where: { arn } });
}
async deleteByArnAndName(arn: string, name: string) {
await this.repo.delete({ arn, name });
async deleteByArnAndName(arn: string, name: string): Promise<void> {
await this.prismaService.tag.deleteMany({ where: { arn, name } });
}
static tagPairs(queryParams: Record<string, string>): { Key: string, Value: string }[] {
const pairs = [null];
static tagPairs(queryParams: Record<string, any>): { key: string; value: string }[] {
const pairs: { key: string; value: string }[] = [];
for (const param of Object.keys(queryParams)) {
const [type, _, idx, slot] = param.split('.');
const components = breakdownAwsQueryParam(param);
if (!components) {
continue; // Skip params that don't match the pattern
}
const [type, _, idx, slot] = components;
if (type === 'Tags') {
if (!pairs[+idx]) {
pairs[+idx] = { Key: '', Value: ''};
pairs[+idx] = { key: '', value: '' };
}
pairs[+idx][slot] = queryParams[param];
// Normalize slot to lowercase (AWS sends 'Key' and 'Value', we need 'key' and 'value')
const normalizedSlot = slot.toLowerCase() as 'key' | 'value';
pairs[+idx][normalizedSlot] = queryParams[param];
}
}
pairs.shift();
return pairs;
return pairs.filter(p => p); // Filter out empty slots
}
static getXmlSafeTagsMap(tags: Tag[]) {

View File

@@ -6,5 +6,7 @@ export interface CommonConfig {
DB_SYNCHRONIZE?: boolean;
HOST: string;
PORT: number;
S3_PORT: number;
CONSUL_PORT: number;
PROTO: string;
}

View File

@@ -9,5 +9,7 @@ export const configValidator = Joi.object<CommonConfig, true>({
DB_SYNCHRONIZE: Joi.boolean().valid(true).required(),
HOST: Joi.string().required(),
PORT: Joi.number().required(),
S3_PORT: Joi.number().required(),
CONSUL_PORT: Joi.number().required(),
PROTO: Joi.string().valid('http', 'https').required(),
});

View File

@@ -1,22 +1,26 @@
import { CommonConfig } from "./common-config.interface";
import { CommonConfig } from './common-config.interface';
import { configValidator } from './config.validator';
export default (): CommonConfig => {
const { error, value } = configValidator.validate({
AWS_ACCOUNT_ID: process.env.AWS_ACCOUNT_ID ?? '000000000000',
AWS_REGION: process.env.AWS_REGION ?? 'us-east-1',
DB_DATABASE: process.env.PERSISTANCE ?? ':memory:',
DB_LOGGING: process.env.DEBUG ? true : false,
DB_SYNCHRONIZE: true,
HOST: process.env.HOST ?? 'localhost',
PROTO: process.env.PROTOCOL ?? 'http',
PORT: process.env.PORT as any ?? 8081,
}, { abortEarly: false });
const { error, value } = configValidator.validate(
{
AWS_ACCOUNT_ID: process.env.AWS_ACCOUNT_ID ?? '000000000000',
AWS_REGION: process.env.AWS_REGION ?? 'us-east-1',
DB_DATABASE: process.env.PERSISTANCE ?? ':memory:',
DB_LOGGING: process.env.DEBUG ? true : false,
DB_SYNCHRONIZE: true,
HOST: process.env.HOST ?? 'localhost',
PROTO: process.env.PROTOCOL ?? 'http',
PORT: (process.env.PORT as any) ?? 4566,
S3_PORT: (process.env.S3_PORT as any) ?? 9000,
CONSUL_PORT: (process.env.CONSUL_PORT as any) ?? 8500,
},
{ abortEarly: false },
);
if (error) {
throw error;
}
return value;
}
};

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,27 @@
import { Module } from '@nestjs/common';
import { ConfigModule } from '@nestjs/config';
import { APP_INTERCEPTOR } from '@nestjs/core';
import { PrismaModule } from '../_prisma/prisma.module';
import localConfig from '../config/local.config';
import { ConsulKVController } from './consul-kv.controller';
import { ConsulKVService } from './consul-kv.service';
import { ConsulKVAuditInterceptor } from './consul-kv-audit.interceptor';
@Module({
imports: [
ConfigModule.forRoot({
isGlobal: true,
load: [localConfig],
}),
PrismaModule,
],
controllers: [ConsulKVController],
providers: [
ConsulKVService,
{
provide: APP_INTERCEPTOR,
useClass: ConsulKVAuditInterceptor,
},
],
})
export class ConsulKVAppModule {}

View File

@@ -0,0 +1,114 @@
import { CallHandler, ExecutionContext, Injectable, Logger, NestInterceptor } from '@nestjs/common';
import { randomUUID } from 'crypto';
import { catchError, Observable, tap, throwError } from 'rxjs';
import { Request, Response } from 'express';
import { PrismaService } from '../_prisma/prisma.service';
@Injectable()
export class ConsulKVAuditInterceptor implements NestInterceptor {
private readonly logger = new Logger(ConsulKVAuditInterceptor.name);
constructor(private readonly prismaService: PrismaService) {}
intercept(context: ExecutionContext, next: CallHandler): Observable<any> {
const httpContext = context.switchToHttp();
const request = httpContext.getRequest<Request>();
const response = httpContext.getResponse<Response>();
const requestId = randomUUID();
response.header('x-consul-request-id', requestId);
const requestStartTime = Date.now();
const method = request.method;
const path = request.path;
const query = request.query;
const headers = request.headers;
return next.handle().pipe(
tap({
next: async data => {
const duration = Date.now() - requestStartTime;
this.logger.log(`${method} ${path} - ${duration}ms - ${response.statusCode}`);
await this.prismaService.audit.create({
data: {
id: requestId,
action: `consul:${method}:${path}`,
request: JSON.stringify({
method,
path,
query,
headers: {
'content-type': headers['content-type'],
'x-consul-namespace': headers['x-consul-namespace'],
},
}),
response: JSON.stringify({
statusCode: response.statusCode,
data: this.sanitizeResponse(data),
}),
},
});
},
error: async error => {
const duration = Date.now() - requestStartTime;
this.logger.error(`${method} ${path} - ${duration}ms - Error: ${error.message}`);
await this.prismaService.audit.create({
data: {
id: requestId,
action: `consul:${method}:${path}`,
request: JSON.stringify({
method,
path,
query,
headers: {
'content-type': headers['content-type'],
'x-consul-namespace': headers['x-consul-namespace'],
},
}),
response: JSON.stringify({
error: error.message,
statusCode: error.status || 500,
}),
},
});
},
}),
catchError((error: Error) => {
return throwError(() => error);
}),
);
}
private sanitizeResponse(data: any): any {
if (!data) {
return null;
}
if (typeof data === 'string') {
return data.length > 1000 ? `${data.substring(0, 1000)}...` : data;
}
if (Array.isArray(data)) {
return data.map(item => this.sanitizeResponseItem(item));
}
return this.sanitizeResponseItem(data);
}
private sanitizeResponseItem(item: any): any {
if (!item || typeof item !== 'object') {
return item;
}
const sanitized: any = { ...item };
if (sanitized.Value && typeof sanitized.Value === 'string' && sanitized.Value.length > 100) {
sanitized.Value = `${sanitized.Value.substring(0, 100)}...`;
}
return sanitized;
}
}

View File

@@ -0,0 +1,167 @@
import { Controller, Get, Put, Delete, Headers, Query, Req, Body, NotFoundException, HttpCode, Header, Param } from '@nestjs/common';
import { Request } from 'express';
import { ConsulKVService } from './consul-kv.service';
import { ConsulKVEntry } from '@prisma/client';
@Controller()
export class ConsulKVController {
constructor(private readonly consulKVService: ConsulKVService) {}
@Get('v1/kv/*')
async getKey(@Req() request: Request, @Headers() headers: Record<string, any>, @Query() query: Record<string, any>) {
const key = this.extractKey(request.path);
const { datacenter, namespace } = this.extractTenancy(query, headers);
const recurse = query.recurse === 'true' || query.recurse === '';
const raw = query.raw === 'true' || query.raw === '';
const keys = query.keys === 'true' || query.keys === '';
const separator = query.separator;
// Blocking query parameters
const waitIndex = query.index !== undefined ? parseInt(query.index, 10) : undefined;
const waitDuration = query.wait;
if (keys) {
// Return only key names
return this.consulKVService.listKeys(key, separator, datacenter, namespace);
}
if (recurse) {
// Return all keys with prefix
const entries = await this.consulKVService.getKeysWithPrefix(key, datacenter, namespace);
if (entries.length === 0) {
throw new NotFoundException();
}
if (raw) {
// For recursive + raw, return the first entry's raw value
return Buffer.from(entries[0].value, 'base64').toString('utf-8');
}
return entries.map(entry => this.formatEntry(entry));
}
// Return single key - use blocking query if wait parameters provided
let entry: ConsulKVEntry | null;
if (waitIndex !== undefined && waitDuration) {
entry = await this.consulKVService.getKeyBlocking(key, datacenter, namespace, waitIndex, waitDuration);
} else {
entry = await this.consulKVService.getKey(key, datacenter, namespace);
}
if (!entry) {
throw new NotFoundException('Key not found');
}
if (raw) {
return Buffer.from(entry.value, 'base64').toString('utf-8');
}
return [this.formatEntry(entry)];
}
@Put('v1/kv/*')
@HttpCode(200)
@Header('Content-Type', 'text/plain')
async putKey(
@Req() request: Request,
@Body() body: any,
@Headers() headers: Record<string, any>,
@Query() query: Record<string, any>,
): Promise<string> {
const key = this.extractKey(request.path);
const { datacenter, namespace } = this.extractTenancy(query, headers);
const flags = parseInt(query.flags) || 0;
const cas = query.cas !== undefined ? parseInt(query.cas) : undefined;
const acquire = query.acquire;
const release = query.release;
// Body is the raw value
const rawValue = Buffer.isBuffer(body) ? body.toString('utf-8') : typeof body === 'string' ? body : JSON.stringify(body);
const value = Buffer.from(rawValue).toString('base64');
const result = await this.consulKVService.putKey(key, value, flags, datacenter, namespace, cas, acquire, release);
// If this was a lock acquisition and there's lock info in headers, store it
// if (result.success && acquire && headers['x-consul-lock-info']) {
// try {
// const lockInfo = JSON.parse(headers['x-consul-lock-info']);
// await this.consulKVService.putLockInfo(key, lockInfo, datacenter, namespace);
// } catch {
// // Ignore invalid lock info
// }
// }
// Consul API returns plain text "true" or "false"
return result.success.toString();
}
@Delete('v1/kv/*')
@HttpCode(200)
@Header('Content-Type', 'text/plain')
async deleteKey(@Req() request: Request, @Headers() headers: Record<string, any>, @Query() query: Record<string, any>): Promise<string> {
const key = this.extractKey(request.path);
const { datacenter, namespace } = this.extractTenancy(query, headers);
const recurse = query.recurse === 'true' || query.recurse === '';
const cas = query.cas !== undefined ? parseInt(query.cas) : undefined;
const success = await this.consulKVService.deleteKey(key, datacenter, namespace, recurse, cas);
// Consul API returns plain text "true" or "false"
return success.toString();
}
@Put('v1/session/create')
async createSession(@Body() body: any, @Query() query: Record<string, any>) {
const { datacenter } = this.extractTenancy(query, {});
const sessionId = await this.consulKVService.createSession(body?.Name, body?.TTL, datacenter);
return { ID: sessionId };
}
@Put('v1/session/renew/:sessionId')
async renewSession(@Param('sessionId') sessionId: string, @Query() query: Record<string, any>) {
const success = await this.consulKVService.renewSession(sessionId);
if (!success) {
throw new NotFoundException('Session not found');
}
// Consul returns the session with TTL info on renewal
const session = await this.consulKVService.getSession(sessionId);
return [{ ID: sessionId, TTL: session?.ttl || '15s' }];
}
@Put('v1/session/destroy/:sessionId')
async destroySession(@Param('sessionId') sessionId: string, @Query() query: Record<string, any>) {
const { datacenter } = this.extractTenancy(query, {});
const success = await this.consulKVService.destroySession(sessionId, datacenter);
return success;
}
private extractKey(path: string): string {
// Extract key from path: /v1/kv/:key and decode URL encoding
const encodedKey = path.replace(/^\/v1\/kv\/?/, '') || '';
return decodeURIComponent(encodedKey);
}
private extractTenancy(query: Record<string, any>, headers: Record<string, any>): { datacenter: string; namespace: string } {
return {
datacenter: query.dc || 'dc1',
namespace: query.ns || headers['x-consul-namespace'] || 'default',
};
}
private formatEntry(entry: ConsulKVEntry) {
return {
CreateIndex: entry.createIndex,
ModifyIndex: entry.modifyIndex,
LockIndex: entry.lockIndex,
Key: entry.key,
Flags: Number(entry.flags),
Value: entry.value,
Session: entry.session || undefined,
};
}
}

View File

@@ -0,0 +1,404 @@
import { Injectable, OnModuleInit } from '@nestjs/common';
import { PrismaService } from '../_prisma/prisma.service';
import { ConsulKVEntry } from '@prisma/client';
import { randomUUID } from 'crypto';
@Injectable()
export class ConsulKVService implements OnModuleInit {
private indexCounter = 1;
private sessions = new Map<string, { name?: string; ttl?: string; datacenter: string; createdAt: Date; lastRenewal: Date }>();
private sessionCleanupInterval?: NodeJS.Timeout;
private changeNotifiers = new Map<string, Set<(entry: ConsulKVEntry | null) => void>>();
constructor(private readonly prismaService: PrismaService) {}
onModuleDestroy() {
if (this.sessionCleanupInterval) {
clearInterval(this.sessionCleanupInterval);
}
}
async onModuleInit() {
const maxEntry = await this.prismaService.consulKVEntry.findFirst({
orderBy: { modifyIndex: 'desc' },
select: { modifyIndex: true },
});
if (maxEntry) {
this.indexCounter = maxEntry.modifyIndex + 1;
}
this.sessionCleanupInterval = setInterval(() => {
this.cleanupExpiredSessions();
}, 5000);
}
private getNextIndex(): number {
return this.indexCounter++;
}
async getKey(key: string, datacenter: string = 'dc1', namespace: string = 'default'): Promise<ConsulKVEntry | null> {
return await this.prismaService.consulKVEntry.findFirst({
where: { key, datacenter, namespace },
});
}
async getKeyBlocking(
key: string,
datacenter: string = 'dc1',
namespace: string = 'default',
waitIndex?: number,
waitDuration?: string,
): Promise<ConsulKVEntry | null> {
const entry = await this.getKey(key, datacenter, namespace);
if (waitIndex === undefined || !waitDuration) {
return entry;
}
if (entry && entry.modifyIndex > waitIndex) {
return entry;
}
if (!entry && waitIndex === 0) {
return null;
}
const waitMs = this.parseDuration(waitDuration);
const notifierKey = `${datacenter}:${namespace}:${key}`;
return new Promise<ConsulKVEntry | null>(resolve => {
const timeout = setTimeout(() => {
this.removeChangeNotifier(notifierKey, notifier);
resolve(entry);
}, waitMs);
const notifier = (updatedEntry: ConsulKVEntry | null) => {
clearTimeout(timeout);
this.removeChangeNotifier(notifierKey, notifier);
if (updatedEntry && updatedEntry.modifyIndex > waitIndex) {
resolve(updatedEntry);
return;
}
if (!updatedEntry && entry !== null) {
resolve(null);
}
};
this.addChangeNotifier(notifierKey, notifier);
});
}
private parseDuration(duration: string): number {
const match = duration.match(/^(\d+)([smh])$/);
if (!match) {
return 5 * 60 * 1000;
}
const value = parseInt(match[1], 10);
const unit = match[2];
if (unit === 's') {
return value * 1000;
}
if (unit === 'm') {
return value * 60 * 1000;
}
return value * 60 * 60 * 1000;
}
private addChangeNotifier(key: string, callback: (entry: ConsulKVEntry | null) => void): void {
if (!this.changeNotifiers.has(key)) {
this.changeNotifiers.set(key, new Set());
}
this.changeNotifiers.get(key)!.add(callback);
}
private removeChangeNotifier(key: string, callback: (entry: ConsulKVEntry | null) => void): void {
const notifiers = this.changeNotifiers.get(key);
if (notifiers) {
notifiers.delete(callback);
if (notifiers.size === 0) {
this.changeNotifiers.delete(key);
}
}
}
private notifyChange(key: string, datacenter: string, namespace: string, entry: ConsulKVEntry | null): void {
const notifierKey = `${datacenter}:${namespace}:${key}`;
const notifiers = this.changeNotifiers.get(notifierKey);
if (notifiers) {
notifiers.forEach(callback => callback(entry));
}
}
async getKeysWithPrefix(prefix: string, datacenter: string = 'dc1', namespace: string = 'default'): Promise<ConsulKVEntry[]> {
return await this.prismaService.consulKVEntry.findMany({
where: {
key: { startsWith: prefix },
datacenter,
namespace,
},
orderBy: { key: 'asc' },
});
}
async listKeys(
prefix: string,
separator: string | undefined,
datacenter: string = 'dc1',
namespace: string = 'default',
): Promise<string[]> {
const entries = await this.getKeysWithPrefix(prefix, datacenter, namespace);
if (!separator) {
return entries.map(e => e.key);
}
const keys = new Set<string>();
for (const entry of entries) {
const keyAfterPrefix = entry.key.substring(prefix.length);
const sepIndex = keyAfterPrefix.indexOf(separator);
if (sepIndex === -1) {
keys.add(entry.key);
continue;
}
keys.add(prefix + keyAfterPrefix.substring(0, sepIndex + separator.length));
}
return Array.from(keys).sort();
}
async putKey(
key: string,
value: string,
flags: number = 0,
datacenter: string = 'dc1',
namespace: string = 'default',
cas?: number,
acquire?: string,
release?: string,
): Promise<{ success: boolean; entry?: ConsulKVEntry }> {
const existing = await this.getKey(key, datacenter, namespace);
// Validate session exists if trying to acquire
if (acquire && !this.isSessionValid(acquire)) {
return { success: false };
}
// Handle lock acquisition - must check before CAS
if (acquire && existing?.session && existing.session !== acquire) {
return { success: false };
}
if (cas === 0 && existing) {
const isUnlocked = !existing.session;
const isSameSession = existing.session === acquire;
const allowedOperation = acquire && (isUnlocked || isSameSession);
if (!allowedOperation) {
return { success: false };
}
}
if (cas !== undefined && cas > 0 && (!existing || existing.modifyIndex !== cas)) {
return { success: false };
}
// Handle lock release
if (release && (!existing || existing.session !== release)) {
return { success: false };
}
const now = new Date();
const modifyIndex = this.getNextIndex();
if (existing) {
const lockIndex = acquire && existing.session !== acquire ? existing.lockIndex + 1 : existing.lockIndex;
const session = release ? null : acquire || existing.session;
const updated = await this.prismaService.consulKVEntry.update({
where: { key },
data: {
value,
flags,
modifyIndex,
lockIndex,
session,
updatedAt: now,
},
});
this.notifyChange(key, datacenter, namespace, updated);
return { success: true, entry: updated };
}
const createIndex = this.getNextIndex();
const entry = await this.prismaService.consulKVEntry.create({
data: {
key,
value,
flags,
createIndex,
modifyIndex: createIndex,
lockIndex: acquire ? 1 : 0,
session: acquire || null,
datacenter,
namespace,
},
});
this.notifyChange(key, datacenter, namespace, entry);
return { success: true, entry };
}
async deleteKey(
key: string,
datacenter: string = 'dc1',
namespace: string = 'default',
recurse: boolean = false,
cas?: number,
): Promise<boolean> {
if (recurse) {
const entries = await this.getKeysWithPrefix(key, datacenter, namespace);
await this.prismaService.consulKVEntry.deleteMany({
where: {
key: { in: entries.map(e => e.key) },
},
});
entries.forEach(entry => {
this.notifyChange(entry.key, datacenter, namespace, null);
});
return true;
}
const existing = await this.getKey(key, datacenter, namespace);
if (!existing) {
return false;
}
if (cas !== undefined && cas > 0 && existing.modifyIndex !== cas) {
return false;
}
await this.prismaService.consulKVEntry.delete({
where: { key },
});
this.notifyChange(key, datacenter, namespace, null);
return true;
}
async createSession(name?: string, ttl?: string, datacenter: string = 'dc1'): Promise<string> {
const sessionId = randomUUID();
const now = new Date();
this.sessions.set(sessionId, {
name,
ttl: ttl || '15s',
datacenter,
createdAt: now,
lastRenewal: now,
});
return sessionId;
}
async renewSession(sessionId: string): Promise<boolean> {
const session = this.sessions.get(sessionId);
if (!session) {
return false;
}
session.lastRenewal = new Date();
return true;
}
private isSessionValid(sessionId: string): boolean {
const session = this.sessions.get(sessionId);
if (!session) {
return false;
}
const ttlMs = this.parseTTL(session.ttl);
const now = Date.now();
const lastRenewalTime = session.lastRenewal.getTime();
return now - lastRenewalTime < ttlMs;
}
private parseTTL(ttl?: string): number {
if (!ttl) {
return 15000;
}
const match = ttl.match(/^(\d+)(s|m|h)?$/);
if (!match) {
return 15000;
}
const value = parseInt(match[1]);
const unit = match[2] || 's';
if (unit === 's') {
return value * 1000;
}
if (unit === 'm') {
return value * 60 * 1000;
}
return value * 60 * 60 * 1000;
}
private async cleanupExpiredSessions(): Promise<void> {
const expiredSessions: string[] = [];
for (const [sessionId, session] of this.sessions.entries()) {
if (!this.isSessionValid(sessionId)) {
expiredSessions.push(sessionId);
}
}
for (const sessionId of expiredSessions) {
await this.destroySession(sessionId, this.sessions.get(sessionId)!.datacenter);
}
}
async destroySession(sessionId: string, datacenter: string = 'dc1'): Promise<boolean> {
const sessionExists = this.sessions.has(sessionId);
if (!sessionExists) {
return false;
}
await this.prismaService.consulKVEntry.updateMany({
where: { session: sessionId },
data: { session: null },
});
this.sessions.delete(sessionId);
return true;
}
async getSession(sessionId: string): Promise<{ name?: string; ttl?: string; datacenter: string } | undefined> {
const session = this.sessions.get(sessionId);
if (!session) return undefined;
return {
name: session.name,
ttl: session.ttl,
datacenter: session.datacenter,
};
}
}

View File

@@ -1,19 +1,27 @@
import { Provider } from '@nestjs/common';
import { InjectionToken, Provider } from '@nestjs/common';
import { Action } from '../action.enum';
import { ExistingActionHandlers } from './default-action-handler.constants';
import * as Joi from 'joi';
import { AbstractActionHandler, Format } from '../abstract-action.handler';
import { UnsupportedOperationException } from '../aws-shared-entities/aws-exceptions';
export const DefaultActionHandlerProvider = (symbol, format: Format, actions: Action[]): Provider => ({
export const DefaultActionHandlerProvider = (symbol: InjectionToken, format: Format, actions: Action[]): Provider => ({
provide: symbol,
useFactory: (existingActionHandlers: ExistingActionHandlers) => {
const cloned = { ...existingActionHandlers };
for (const action of actions) {
if (!cloned[action]) {
cloned[action] = new (class Default extends AbstractActionHandler { action = action; format = format; validator = Joi.object(); handle = () => {} });
cloned[action] = new (class Default extends AbstractActionHandler {
action = action;
format = format;
validator = Joi.object();
handle = () => {
throw new UnsupportedOperationException(`Action ${action} is not yet implemented`);
};
})();
}
}
return cloned;
},
inject: [ExistingActionHandlers]
inject: [ExistingActionHandlers],
});

View File

@@ -1,12 +1,22 @@
import { Provider } from '@nestjs/common';
import { InjectionToken, OptionalFactoryDependency, Provider } from '@nestjs/common';
import { AbstractActionHandler } from '../abstract-action.handler';
import { Action } from '../action.enum';
import { ExistingActionHandlers } from './default-action-handler.constants';
export const ExistingActionHandlersProvider = (inject): Provider => ({
export const ExistingActionHandlersProvider = (inject: Array<InjectionToken | OptionalFactoryDependency>): Provider => ({
provide: ExistingActionHandlers,
useFactory: (...args: AbstractActionHandler[]) => args.reduce((m, h) => {
if (Array.isArray(h.action)) {
for (const action of h.action) {
m[action] = h;
}
return m;
}
m[h.action] = h;
return m;
}, {}),
}, {} as Record<Action, AbstractActionHandler>),
inject,
});

File diff suppressed because it is too large Load Diff

View File

@@ -2,12 +2,8 @@ import { Injectable } from '@nestjs/common';
import { AbstractActionHandler, AwsProperties, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import * as Joi from 'joi';
import { InjectRepository } from '@nestjs/typeorm';
import { Repository } from 'typeorm';
import * as uuid from 'uuid';
import { IamPolicy } from './iam-policy.entity';
import { IamRolePolicyAttachment } from './iam-role-policy-attachment.entity';
import { IamRole } from './iam-role.entity';
import { IamService } from './iam.service';
import { RequestContext } from '../_context/request.context';
type QueryParams = {
PolicyArn: string;
@@ -18,10 +14,7 @@ type QueryParams = {
export class AttachRolePolicyHandler extends AbstractActionHandler<QueryParams> {
constructor(
@InjectRepository(IamRole)
private readonly roleRepo: Repository<IamRole>,
@InjectRepository(IamRolePolicyAttachment)
private readonly attachRepo: Repository<IamRolePolicyAttachment>,
private readonly iamService: IamService,
) {
super();
}
@@ -33,15 +26,12 @@ export class AttachRolePolicyHandler extends AbstractActionHandler<QueryParams>
RoleName: Joi.string().required(),
});
protected async handle({ PolicyArn, RoleName }: QueryParams, awsProperties: AwsProperties) {
protected async handle({ PolicyArn, RoleName }: QueryParams, context: RequestContext) {
const role = await this.roleRepo.findOne({ where: { roleName: RoleName, accountId: awsProperties.accountId} });
await this.attachRepo.create({
id: uuid.v4(),
policyArn: PolicyArn,
roleId: role.id,
accountId: awsProperties.accountId,
}).save();
await this.iamService.attachPolicyToRoleName(
context.awsProperties.accountId,
PolicyArn,
RoleName
);
}
}

View File

@@ -2,61 +2,57 @@ import { Injectable } from '@nestjs/common';
import { AbstractActionHandler, AwsProperties, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import * as Joi from 'joi';
import { InjectRepository } from '@nestjs/typeorm';
import { Repository } from 'typeorm';
import * as uuid from 'uuid';
import { IamPolicy } from './iam-policy.entity';
import { breakdownArn } from '../util/breakdown-arn';
import { IamService } from './iam.service';
import { RequestContext } from '../_context/request.context';
type QueryParams = {
PolicyArn: string;
PolicyDocument: string;
SetAsDefault: boolean;
}
};
@Injectable()
export class CreatePolicyVersionHandler extends AbstractActionHandler<QueryParams> {
constructor(
@InjectRepository(IamPolicy)
private readonly policyRepo: Repository<IamPolicy>,
) {
constructor(private readonly iamService: IamService) {
super();
}
format = Format.Xml;
action = Action.IamCreatePolicyVersion;
validator = Joi.object<QueryParams, true>({
validator = Joi.object<QueryParams, true>({
PolicyArn: Joi.string().required(),
PolicyDocument: Joi.string().required(),
SetAsDefault: Joi.boolean().required(),
SetAsDefault: Joi.boolean().default(false),
});
protected async handle({ PolicyArn, PolicyDocument, SetAsDefault }: QueryParams, awsProperties: AwsProperties) {
const { identifier, accountId } = breakdownArn(PolicyArn);
const [_policy, name] = identifier.split('/');
const currentPolicy = await this.policyRepo.findOne({ where: { accountId, name, isDefault: true } });
protected async handle({ PolicyArn, PolicyDocument, SetAsDefault }: QueryParams, { awsProperties }: RequestContext) {
// Get the current policy to find the latest version
const currentPolicy = await this.iamService.getPolicyByArn(PolicyArn);
const newVersion = currentPolicy.version + 1;
// If setting as default, mark all existing versions as non-default
if (SetAsDefault) {
await this.policyRepo.update({ accountId, name }, { isDefault: false })
await this.iamService.updateAllPolicyVersionsDefaultStatus(currentPolicy.id, false);
}
const policy = await this.policyRepo.create({
id: uuid.v4(),
name: name,
// Create new policy version
const newPolicy = await this.iamService.createPolicyVersion({
id: currentPolicy.id,
version: newVersion,
isDefault: SetAsDefault,
version: currentPolicy.version + 1,
document: PolicyDocument,
name: currentPolicy.name,
path: currentPolicy.path,
description: currentPolicy.description,
policy: PolicyDocument,
accountId: awsProperties.accountId,
}).save();
});
return {
PolicyVersion: {
IsDefaultVersion: policy.isDefault,
VersionId: `v${policy.version}`,
CreateDate: new Date(policy.createdAt).toISOString(),
}
}
VersionId: `v${newVersion}`,
IsDefaultVersion: SetAsDefault,
CreateDate: newPolicy.createdAt.toISOString(),
},
};
}
}

View File

@@ -1,23 +1,25 @@
import { Injectable } from '@nestjs/common';
import { AbstractActionHandler, AwsProperties, Format } from '../abstract-action.handler';
import { AbstractActionHandler, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import * as Joi from 'joi';
import { InjectRepository } from '@nestjs/typeorm';
import { Repository } from 'typeorm';
import * as uuid from 'uuid';
import { IamPolicy } from './iam-policy.entity';
import { IamService } from './iam.service';
import { TagsService } from '../aws-shared-entities/tags.service';
import { randomUUID } from 'crypto';
import { RequestContext } from '../_context/request.context';
type QueryParams = {
PolicyName: string;
Description: string;
Path: string;
PolicyDocument: string;
}
PolicyName: string;
} & Record<string, string>;
@Injectable()
export class CreatePolicyHandler extends AbstractActionHandler<QueryParams> {
constructor(
@InjectRepository(IamPolicy)
private readonly policyRepo: Repository<IamPolicy>,
private readonly iamService: IamService,
private readonly tagsService: TagsService,
) {
super();
}
@@ -25,30 +27,29 @@ export class CreatePolicyHandler extends AbstractActionHandler<QueryParams> {
format = Format.Xml;
action = Action.IamCreatePolicy;
validator = Joi.object<QueryParams, true>({
PolicyName: Joi.string().required(),
PolicyDocument: Joi.string().required(),
Description: Joi.string().max(1000).allow(null, '').default(null),
Path: Joi.string().min(1).max(512).default(null).regex(new RegExp(`((/[A-Za-z0-9\.,\+@=_-]+)*)/`)),
PolicyDocument: Joi.string().min(1).max(131072).required(),
PolicyName: Joi.string().min(1).max(128).required(),
});
protected async handle({ PolicyName, PolicyDocument }: QueryParams, awsProperties: AwsProperties) {
protected async handle(params: QueryParams, context: RequestContext) {
const policy = await this.policyRepo.create({
id: uuid.v4(),
const { Description, Path, PolicyName, PolicyDocument } = params;
const policy = await this.iamService.createPolicy({
id: randomUUID(),
version: 1,
isDefault: true,
name: PolicyName,
document: PolicyDocument,
accountId: awsProperties.accountId,
}).save();
path: Path,
description: Description,
policy: PolicyDocument,
accountId: context.awsProperties.accountId,
});
return {
Policy: {
PolicyName: policy.name,
DefaultVersionId: policy.version,
PolicyId: policy.id,
Path: '/',
Arn: policy.arn,
AttachmentCount: 0,
CreateDate: new Date(policy.createdAt).toISOString(),
UpdateDate: new Date(policy.updatedAt).toISOString(),
}
}
Policy: policy.metadata
};
}
}

View File

@@ -2,61 +2,48 @@ import { Injectable } from '@nestjs/common';
import { AbstractActionHandler, AwsProperties, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import * as Joi from 'joi';
import { InjectRepository } from '@nestjs/typeorm';
import { Repository } from 'typeorm';
import { IamRole } from './iam-role.entity';
import * as uuid from 'uuid';
import { IamPolicy } from './iam-policy.entity';
import { IamService } from './iam.service';
import { randomUUID } from 'crypto';
import { RequestContext } from '../_context/request.context';
type QueryParams = {
RoleName: string;
Path: string;
AssumeRolePolicyDocument: string;
MaxSessionDuration: number;
Description: string;
}
@Injectable()
export class CreateRoleHandler extends AbstractActionHandler<QueryParams> {
constructor(
@InjectRepository(IamRole)
private readonly roleRepo: Repository<IamRole>,
@InjectRepository(IamPolicy)
private readonly policyRepo: Repository<IamPolicy>,
private readonly iamService: IamService,
) {
super();
}
format = Format.Xml;
action = Action.IamCreateRole;
validator = Joi.object<QueryParams, true>({
RoleName: Joi.string().required(),
Path: Joi.string().required(),
AssumeRolePolicyDocument: Joi.string().required(),
MaxSessionDuration: Joi.number().default(3600),
validator = Joi.object<QueryParams, true>({
AssumeRolePolicyDocument: Joi.string().min(1).max(131072).required(),
Description: Joi.string().max(1000).allow(null, '').default(null),
MaxSessionDuration: Joi.number().min(3600).max(43200).default(3600),
Path: Joi.string().min(1).max(512).required(),
RoleName: Joi.string().min(1).max(64).required(),
});
protected async handle({ RoleName, Path, AssumeRolePolicyDocument, MaxSessionDuration }: QueryParams, awsProperties: AwsProperties) {
protected async handle({ RoleName, Path, AssumeRolePolicyDocument, MaxSessionDuration, Description }: QueryParams, { awsProperties} : RequestContext) {
const policy = await this.policyRepo.create({
id: uuid.v4(),
name: `${RoleName}-AssumeRolePolicyDocument`,
document: AssumeRolePolicyDocument,
const role = await this.iamService.createRole({
id: randomUUID(),
accountId: awsProperties.accountId,
}).save();
const id = uuid.v4();
await this.roleRepo.create({
id,
roleName: RoleName,
name: RoleName,
path: Path,
accountId: awsProperties.accountId,
assumeRolePolicyDocumentId: policy.id,
assumeRolePolicy: AssumeRolePolicyDocument,
maxSessionDuration: MaxSessionDuration,
}).save();
const role = await this.roleRepo.findOne({ where: { id }});
description: Description,
});
return {
Role: role.metadata,

View File

@@ -0,0 +1,31 @@
import { Injectable } from '@nestjs/common';
import { AbstractActionHandler, AwsProperties, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import * as Joi from 'joi';
import { IamService } from './iam.service';
import { NotFoundException } from '../aws-shared-entities/aws-exceptions';
import { RequestContext } from '../_context/request.context';
type QueryParams = {
RoleName: string;
}
@Injectable()
export class DeleteRoleHandler extends AbstractActionHandler<QueryParams> {
constructor(
private readonly iamService: IamService,
) {
super();
}
format = Format.Xml;
action = Action.IamDeleteRole;
validator = Joi.object<QueryParams, true>({
RoleName: Joi.string().min(1).max(64).required(),
});
protected async handle({ RoleName }: QueryParams, { awsProperties} : RequestContext) {
await this.iamService.deleteRoleByName(awsProperties.accountId, RoleName);
}
}

View File

@@ -1,54 +1,39 @@
import { Injectable, NotFoundException, Version } from '@nestjs/common';
import { Injectable } from '@nestjs/common';
import { AbstractActionHandler, AwsProperties, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import * as Joi from 'joi';
import { InjectRepository } from '@nestjs/typeorm';
import { Repository } from 'typeorm';
import { IamPolicy } from './iam-policy.entity';
import { breakdownArn } from '../util/breakdown-arn';
import { IamRolePolicyAttachment } from './iam-role-policy-attachment.entity';
import { RequestContext } from '../_context/request.context';
import { IamService } from './iam.service';
type QueryParams = {
PolicyArn: string;
VersionId: string;
}
};
@Injectable()
export class GetPolicyVersionHandler extends AbstractActionHandler<QueryParams> {
constructor(
@InjectRepository(IamPolicy)
private readonly policyRepo: Repository<IamPolicy>,
@InjectRepository(IamRolePolicyAttachment)
private readonly attachmentRepo: Repository<IamRolePolicyAttachment>,
) {
constructor(private readonly iamService: IamService) {
super();
}
format = Format.Xml;
action = Action.IamGetPolicyVersion;
validator = Joi.object<QueryParams, true>({
validator = Joi.object<QueryParams, true>({
PolicyArn: Joi.string().required(),
VersionId: Joi.string().required(),
});
protected async handle({ PolicyArn, VersionId }: QueryParams, awsProperties: AwsProperties) {
const { identifier, accountId } = breakdownArn(PolicyArn);
const [_policy, name] = identifier.split('/');
const policy = await this.policyRepo.findOne({ where: { name, accountId, version: +VersionId }});
if (!policy) {
throw new NotFoundException('NoSuchEntity', 'The request was rejected because it referenced a resource entity that does not exist. The error message describes the resource.');
}
protected async handle({ PolicyArn, VersionId }: QueryParams, { awsProperties }: RequestContext) {
const maybeVersion = Number(VersionId);
const version = Number.isNaN(maybeVersion) ? Number(VersionId.toLowerCase().split('v')[1]) : Number(maybeVersion);
const policy = await this.iamService.getPolicyByArnAndVersion(PolicyArn, version);
return {
PolicyVersion: {
Document: policy.document,
Document: policy.policy,
IsDefaultVersion: policy.isDefault,
VersionId: `${policy.version}`,
CreateDate: new Date(policy.createdAt).toISOString(),
}
}
VersionId: `v${policy.version}`,
CreateDate: policy.createdAt.toISOString(),
},
};
}
}

View File

@@ -1,12 +1,9 @@
import { Injectable, NotFoundException } from '@nestjs/common';
import { Injectable } from '@nestjs/common';
import { AbstractActionHandler, AwsProperties, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import * as Joi from 'joi';
import { InjectRepository } from '@nestjs/typeorm';
import { Repository } from 'typeorm';
import { IamPolicy } from './iam-policy.entity';
import { breakdownArn } from '../util/breakdown-arn';
import { IamRolePolicyAttachment } from './iam-role-policy-attachment.entity';
import { RequestContext } from '../_context/request.context';
import { IamService } from './iam.service';
type QueryParams = {
PolicyArn: string;
@@ -16,10 +13,7 @@ type QueryParams = {
export class GetPolicyHandler extends AbstractActionHandler<QueryParams> {
constructor(
@InjectRepository(IamPolicy)
private readonly policyRepo: Repository<IamPolicy>,
@InjectRepository(IamRolePolicyAttachment)
private readonly attachmentRepo: Repository<IamRolePolicyAttachment>,
private readonly iamService: IamService,
) {
super();
}
@@ -30,29 +24,10 @@ export class GetPolicyHandler extends AbstractActionHandler<QueryParams> {
PolicyArn: Joi.string().required(),
});
protected async handle({ PolicyArn }: QueryParams, awsProperties: AwsProperties) {
const { identifier, accountId } = breakdownArn(PolicyArn);
const [_policy, name] = identifier.split('/');
const policy = await this.policyRepo.findOne({ where: { name, accountId, isDefault: true }});
if (!policy) {
throw new NotFoundException('NoSuchEntity', 'The request was rejected because it referenced a resource entity that does not exist. The error message describes the resource.');
}
const attachmentCount = await this.attachmentRepo.count({ where: { policyArn: policy.arn } });
protected async handle({ PolicyArn }: QueryParams, { awsProperties} : RequestContext) {
const policy = await this.iamService.getPolicyByArn(PolicyArn);
return {
Policy: {
PolicyName: policy.name,
DefaultVersionId: policy.version,
PolicyId: policy.id,
Path: '/',
Arn: policy.arn,
AttachmentCount: attachmentCount,
CreateDate: new Date(policy.createdAt).toISOString(),
UpdateDate: new Date(policy.updatedAt).toISOString(),
}
Policy: policy.metadata,
}
}
}

View File

@@ -0,0 +1,40 @@
import { Injectable } from '@nestjs/common';
import { AbstractActionHandler, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import * as Joi from 'joi';
import { RequestContext } from '../_context/request.context';
import { IamService } from './iam.service';
import { NoSuchEntity } from '../aws-shared-entities/aws-exceptions';
type QueryParams = {
PolicyName: string;
RoleName: string;
};
@Injectable()
export class GetRolePolicyHandler extends AbstractActionHandler<QueryParams> {
constructor(private readonly iamService: IamService) {
super();
}
format = Format.Xml;
action = Action.IamGetRolePolicy;
validator = Joi.object<QueryParams, true>({
PolicyName: Joi.string().required(),
RoleName: Joi.string().required(),
});
protected async handle({ PolicyName, RoleName }: QueryParams, { awsProperties }: RequestContext) {
// Verify role exists
await this.iamService.findOneRoleByName(awsProperties.accountId, RoleName);
// Get inline policy
const inlinePolicy = await this.iamService.getRoleInlinePolicy(awsProperties.accountId, RoleName, PolicyName);
return {
RoleName,
PolicyName,
PolicyDocument: inlinePolicy.policyDocument,
};
}
}

View File

@@ -1,10 +1,10 @@
import { Injectable, NotFoundException } from '@nestjs/common';
import { Injectable } from '@nestjs/common';
import { AbstractActionHandler, AwsProperties, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import * as Joi from 'joi';
import { InjectRepository } from '@nestjs/typeorm';
import { Repository } from 'typeorm';
import { IamRole } from './iam-role.entity';
import { IamService } from './iam.service';
import { NotFoundException } from '../aws-shared-entities/aws-exceptions';
import { RequestContext } from '../_context/request.context';
type QueryParams = {
RoleName: string;
@@ -14,8 +14,7 @@ type QueryParams = {
export class GetRoleHandler extends AbstractActionHandler<QueryParams> {
constructor(
@InjectRepository(IamRole)
private readonly roleRepo: Repository<IamRole>,
private readonly iamService: IamService,
) {
super();
}
@@ -23,17 +22,11 @@ export class GetRoleHandler extends AbstractActionHandler<QueryParams> {
format = Format.Xml;
action = Action.IamGetRole;
validator = Joi.object<QueryParams, true>({
RoleName: Joi.string().required(),
RoleName: Joi.string().min(1).max(64).required(),
});
protected async handle({ RoleName }: QueryParams, awsProperties: AwsProperties) {
const role = await this.roleRepo.findOne({ where: { roleName: RoleName, accountId: awsProperties.accountId } });
if (!role) {
throw new NotFoundException('NoSuchEntity', 'The request was rejected because it referenced a resource entity that does not exist. The error message describes the resource.');
}
protected async handle({ RoleName }: QueryParams, { awsProperties} : RequestContext) {
const role = await this.iamService.findOneRoleByName(awsProperties.accountId, RoleName);
return {
Role: role.metadata,
}

View File

@@ -1,38 +1,54 @@
import { BaseEntity, Column, CreateDateColumn, Entity, JoinColumn, OneToMany, OneToOne, PrimaryColumn, UpdateDateColumn } from 'typeorm';
import { IamRolePolicyAttachment } from './iam-role-policy-attachment.entity';
import { IamRole } from './iam-role.entity';
import { IamPolicy as PrismaIamPolicy } from "@prisma/client";
@Entity({ name: 'iam_policy' })
export class IamPolicy extends BaseEntity {
export class IamPolicy implements PrismaIamPolicy {
@PrimaryColumn()
id: string;
@Column({ default: 1 })
version: number;
@Column({ name: 'is_default', default: true })
isDefault: boolean;
@Column()
name: string;
@Column()
document: string;
@Column({ name: 'account_id', nullable: false })
accountId: string;
name: string;
version: number;
isDefault: boolean;
policy: string;
path: string | null;
description: string | null;
isAttachable: boolean;
createdAt: Date;
updatedAt: Date;
@CreateDateColumn()
createdAt: string;
@UpdateDateColumn()
updatedAt: string;
@OneToOne(() => IamRole, role => role.assumeRolePolicyDocument)
iamRole: IamRole;
constructor(p: PrismaIamPolicy) {
this.id = p.id;
this.accountId = p.accountId;
this.name = p.name;
this.version = p.version;
this.isDefault = p.isDefault;
this.policy = p.policy;
this.path = p.path;
this.description = p.description;
this.isAttachable = p.isAttachable;
this.createdAt = p.createdAt;
this.updatedAt = p.updatedAt;
}
get arn() {
return `arn:aws:iam::${this.accountId}:policy/${this.name}`;
const parts = ['policy'];
if (this.path && this.path !== '/') {
parts.push(this.path);
}
parts.push(this.name);
return `arn:aws:iam::${this.accountId}:${parts.join('/')}`;
}
get metadata() {
return {
Arn: this.arn,
AttachmentCount: 0,
CreateDate: this.createdAt.toISOString(),
DefaultVersionId: `v${this.version}`,
Description: this.description,
IsAttachable: this.isAttachable,
Path: this.path,
PolicyId: this.id,
PolicyName: this.name,
UpdateDate: this.updatedAt.toISOString(),
}
}
}

View File

@@ -0,0 +1,21 @@
import { IamRoleInlinePolicy as PrismaIamRoleInlinePolicy } from '@prisma/client';
export class IamRoleInlinePolicy implements PrismaIamRoleInlinePolicy {
id: string;
roleName: string;
policyName: string;
policyDocument: string;
accountId: string;
createdAt: Date;
updatedAt: Date;
constructor(policy: PrismaIamRoleInlinePolicy) {
this.id = policy.id;
this.roleName = policy.roleName;
this.policyName = policy.policyName;
this.policyDocument = policy.policyDocument;
this.accountId = policy.accountId;
this.createdAt = policy.createdAt;
this.updatedAt = policy.updatedAt;
}
}

View File

@@ -1,18 +0,0 @@
import { BaseEntity, Column, Entity, JoinColumn, ManyToOne, PrimaryColumn } from 'typeorm';
import { IamPolicy } from './iam-policy.entity';
@Entity({ name: 'iam_role_policy_attachment' })
export class IamRolePolicyAttachment extends BaseEntity {
@PrimaryColumn()
id: string;
@Column({ name: 'policy_arn' })
policyArn: string;
@Column({ name: 'role_name' })
roleId: string;
@Column({ name: 'account_id'})
accountId: string;
}

View File

@@ -1,52 +1,51 @@
import { BaseEntity, Column, CreateDateColumn, Entity, JoinColumn, OneToOne, PrimaryColumn, UpdateDateColumn } from 'typeorm';
import { IamPolicy } from './iam-policy.entity';
import { IamRole as PrismaIamRole } from '@prisma/client';
@Entity({ name: 'iam_role' })
export class IamRole extends BaseEntity {
@PrimaryColumn()
id: string
@Column({ name: 'role_name' })
roleName: string;
@Column()
path: string;
@Column({ name: 'assume_role_policy_document_id', nullable: false })
assumeRolePolicyDocumentId: string;
@Column({ name: 'account_id', nullable: false })
export class IamRole implements PrismaIamRole {
accountId: string;
path: string | null;
name: string;
createdAt: Date;
id: string;
maxSessionDuration: number | null;
assumeRolePolicy: string | null;
description: string | null;
permissionBoundaryArn: string | null;
lastUsedDate: Date | null;
lastUsedRegion: string | null;
@Column({ name: 'max_session_duration', nullable: false, default: 0 })
maxSessionDuration: number;
@CreateDateColumn()
createdAt: string;
@UpdateDateColumn()
updatedAt: string;
@OneToOne(() => IamPolicy, (policy) => policy.id, { eager: true })
@JoinColumn({ name: 'assume_role_policy_document_id' })
assumeRolePolicyDocument: IamPolicy;
constructor(p: PrismaIamRole) {
this.accountId = p.accountId;
this.path = p.path;
this.name = p.name;
this.createdAt = p.createdAt;
this.id = p.id;
this.maxSessionDuration = p.maxSessionDuration;
this.assumeRolePolicy = p.assumeRolePolicy;
this.description = p.description;
this.permissionBoundaryArn = p.permissionBoundaryArn;
this.lastUsedDate = p.lastUsedDate;
this.lastUsedRegion = p.lastUsedRegion;
}
get arn() {
const identifier = this.path.split('/');
identifier.push(this.roleName);
return `arn:aws:iam::${this.accountId}:role/${identifier.join('/')}`;
const parts = ['role'];
if (this.path && this.path !== '/') {
parts.push(this.path);
}
parts.push(this.name);
return `arn:aws:iam::${this.accountId}:${parts.join('/')}`;
}
get metadata() {
return {
Path: this.path,
Arn: this.arn,
RoleName: this.roleName,
AssumeRolePolicyDocument: this.assumeRolePolicyDocument.document,
CreateDate: new Date(this.createdAt).toISOString(),
RoleName: this.name,
AssumeRolePolicyDocument: this.assumeRolePolicy,
CreateDate: this.createdAt.toISOString(),
RoleId: this.id,
MaxSessionDuration: this.maxSessionDuration,
}
Description: this.description,
};
}
}

View File

@@ -1,35 +1,41 @@
import { Module } from '@nestjs/common';
import { TypeOrmModule } from '@nestjs/typeorm';
import { Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import { AwsSharedEntitiesModule } from '../aws-shared-entities/aws-shared-entities.module';
import { DefaultActionHandlerProvider } from '../default-action-handler/default-action-handler.provider';
import { ExistingActionHandlersProvider } from '../default-action-handler/existing-action-handlers.provider';
import { AttachRolePolicyHandler } from './attach-role-policy.handler';
import { CreatePolicyVersionHandler } from './create-policy-version.handler';
import { CreatePolicyHandler } from './create-policy.handler';
import { CreatePolicyVersionHandler } from './create-policy-version.handler';
import { CreateRoleHandler } from './create-role.handler';
import { GetPolicyVersionHandler } from './get-policy-version.handler';
import { GetPolicyHandler } from './get-policy.handler';
import { GetRoleHandler } from './get-role.handler';
import { IamPolicy } from './iam-policy.entity';
import { IamRolePolicyAttachment } from './iam-role-policy-attachment.entity';
import { IamRole } from './iam-role.entity';
import { DeleteRoleHandler } from './delete-role.handler';
import { IAMHandlers } from './iam.constants';
import { PrismaModule } from '../_prisma/prisma.module';
import { IamService } from './iam.service';
import { GetRoleHandler } from './get-role.handler';
import { GetPolicyHandler } from './get-policy.handler';
import { GetPolicyVersionHandler } from './get-policy-version.handler';
import { AttachRolePolicyHandler } from './attach-role-policy.handler';
import { ListAttachedRolePoliciesHandler } from './list-attached-role-policies';
import { ListRolePoliciesHandler } from './list-role-policies.handler';
import { GetRolePolicyHandler } from './get-role-policy.handler';
import { PutRolePolicyHandler } from './put-role-policy.handler';
import { UpdateRoleDescriptionHandler } from './update-role-description.handler';
const handlers = [
AttachRolePolicyHandler,
CreatePolicyHandler,
CreatePolicyVersionHandler,
CreateRoleHandler,
DeleteRoleHandler,
GetPolicyVersionHandler,
GetPolicyHandler,
GetRoleHandler,
GetPolicyVersionHandler,
GetRolePolicyHandler,
ListAttachedRolePoliciesHandler,
ListRolePoliciesHandler,
]
PutRolePolicyHandler,
UpdateRoleDescriptionHandler,
];
const actions = [
Action.IamAddClientIDToOpenIDConnectProvider,
@@ -190,15 +196,13 @@ const actions = [
Action.IamUploadServerCertificate,
Action.IamUploadSigningCertificate,
Action.IamUploadSSHPublicKey,
]
];
@Module({
imports: [
TypeOrmModule.forFeature([IamPolicy, IamRole, IamRolePolicyAttachment]),
AwsSharedEntitiesModule,
],
imports: [AwsSharedEntitiesModule, PrismaModule],
providers: [
...handlers,
IamService,
ExistingActionHandlersProvider(handlers),
DefaultActionHandlerProvider(IAMHandlers, Format.Xml, actions),
],

263
src/iam/iam.service.ts Normal file
View File

@@ -0,0 +1,263 @@
import { Injectable } from '@nestjs/common';
import { PrismaService } from '../_prisma/prisma.service';
import { Prisma } from '@prisma/client';
import { IamPolicy } from './iam-policy.entity';
import { IamRole } from './iam-role.entity';
import { EntityAlreadyExists, NoSuchEntity, NotFoundException } from '../aws-shared-entities/aws-exceptions';
import { ArnUtil } from '../util/arn-util.static';
@Injectable()
export class IamService {
constructor(private readonly prismaService: PrismaService) {}
async createRole(data: Prisma.IamRoleCreateInput): Promise<IamRole> {
try {
const record = await this.prismaService.iamRole.create({ data });
return new IamRole(record);
} catch (err) {
throw new EntityAlreadyExists(`RoleName ${data.name} already exists`);
}
}
async findOneRoleByName(accountId: string, name: string): Promise<IamRole> {
try {
const record = await this.prismaService.iamRole.findFirstOrThrow({
where: {
name,
accountId,
},
});
return new IamRole(record);
} catch (error) {
throw new NotFoundException();
}
}
async deleteRoleByName(accountId: string, name: string) {
// First find the role
const role = await this.findOneRoleByName(accountId, name);
// Delete all policy attachments first
await this.prismaService.iamRoleIamPolicyAttachment.deleteMany({
where: {
iamRoleId: role.id,
},
});
// Then delete the role
await this.prismaService.iamRole.delete({
where: {
id: role.id,
},
});
}
async updateRoleDescription(accountId: string, name: string, description: string): Promise<IamRole> {
// First verify the role exists
const role = await this.findOneRoleByName(accountId, name);
// Update the description
const updated = await this.prismaService.iamRole.update({
where: {
id: role.id,
},
data: {
description,
},
});
return new IamRole(updated);
}
async listRolePolicies(): Promise<IamPolicy[]> {
const records = await this.prismaService.iamPolicy.findMany();
return records.map(r => new IamPolicy(r));
}
async getPolicyByArn(arn: string): Promise<IamPolicy> {
try {
const name = arn.split('/')[1];
const record = await this.prismaService.iamPolicy.findFirstOrThrow({
where: {
name,
},
orderBy: {
version: 'desc',
},
});
return new IamPolicy(record);
} catch (err) {
throw new NoSuchEntity();
}
}
async getPolicyByArnAndVersion(arn: string, version: number): Promise<IamPolicy> {
try {
const name = arn.split('/')[1];
const record = await this.prismaService.iamPolicy.findFirstOrThrow({
where: {
name,
version,
},
});
return new IamPolicy(record);
} catch (err) {
throw new NoSuchEntity();
}
}
async createPolicy(data: Prisma.IamPolicyCreateInput): Promise<IamPolicy> {
// Check if policy with same name already exists
const existing = await this.prismaService.iamPolicy.findFirst({
where: {
accountId: data.accountId,
name: data.name,
path: data.path,
},
});
if (existing) {
throw new EntityAlreadyExists(`PolicyName ${data.name} already exists`);
}
const record = await this.prismaService.iamPolicy.create({ data });
return new IamPolicy(record);
}
async createPolicyVersion(data: Prisma.IamPolicyCreateInput): Promise<IamPolicy> {
const record = await this.prismaService.iamPolicy.create({ data });
return new IamPolicy(record);
}
async updatePolicyDefaultStatus(id: string, version: number, isDefault: boolean): Promise<void> {
await this.prismaService.iamPolicy.update({
where: {
id_version: {
id,
version,
},
},
data: {
isDefault,
},
});
}
async updateAllPolicyVersionsDefaultStatus(policyId: string, isDefault: boolean): Promise<void> {
await this.prismaService.iamPolicy.updateMany({
where: { id: policyId },
data: { isDefault },
});
}
async attachPolicyToRoleName(accountId: string, arn: string, roleName: string) {
const policy = await this.getPolicyByArn(arn);
const role = await this.findOneRoleByName(accountId, roleName);
// Check if already attached
const existing = await this.prismaService.iamRoleIamPolicyAttachment.findFirst({
where: {
iamRoleId: role.id,
iamPolicyId: policy.id,
},
});
if (!existing) {
await this.prismaService.iamRoleIamPolicyAttachment.create({
data: {
iamPolicyId: policy.id,
iamRoleId: role.id,
},
});
}
}
async findAttachedRolePoliciesByRoleName(accountId: string, roleName: string): Promise<IamPolicy[]> {
try {
const record = await this.prismaService.iamRole.findFirstOrThrow({
where: {
name: roleName,
accountId,
},
include: {
policies: true,
},
});
const policyIds = record.policies.map(p => p.iamPolicyId);
const policies = await this.prismaService.iamPolicy.findMany({
where: {
id: {
in: policyIds,
},
isDefault: true,
},
});
return policies.map(p => new IamPolicy(p));
} catch (error) {
throw new NotFoundException();
}
}
async putRoleInlinePolicy(accountId: string, roleName: string, policyName: string, policyDocument: string): Promise<void> {
// Verify role exists
await this.findOneRoleByName(accountId, roleName);
await this.prismaService.iamRoleInlinePolicy.upsert({
where: {
accountId_roleName_policyName: {
accountId,
roleName,
policyName,
},
},
create: {
accountId,
roleName,
policyName,
policyDocument,
},
update: {
policyDocument,
},
});
}
async getRoleInlinePolicy(accountId: string, roleName: string, policyName: string): Promise<{ policyDocument: string }> {
// Verify role exists
await this.findOneRoleByName(accountId, roleName);
try {
const policy = await this.prismaService.iamRoleInlinePolicy.findUniqueOrThrow({
where: {
accountId_roleName_policyName: {
accountId,
roleName,
policyName,
},
},
});
return { policyDocument: policy.policyDocument };
} catch (err) {
throw new NoSuchEntity();
}
}
async listRoleInlinePolicies(accountId: string, roleName: string): Promise<string[]> {
// Verify role exists
await this.findOneRoleByName(accountId, roleName);
const policies = await this.prismaService.iamRoleInlinePolicy.findMany({
where: {
accountId,
roleName,
},
select: {
policyName: true,
},
});
return policies.map(p => p.policyName);
}
}

View File

@@ -1,57 +1,36 @@
import { Injectable, NotFoundException } from '@nestjs/common';
import { Injectable } from '@nestjs/common';
import { AbstractActionHandler, AwsProperties, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import * as Joi from 'joi';
import { InjectRepository } from '@nestjs/typeorm';
import { In, Repository } from 'typeorm';
import { IamRole } from './iam-role.entity';
import { IamRolePolicyAttachment } from './iam-role-policy-attachment.entity';
import { IamPolicy } from './iam-policy.entity';
import { breakdownArn } from '../util/breakdown-arn';
import { RequestContext } from '../_context/request.context';
import { IamService } from './iam.service';
type QueryParams = {
RoleName: string;
}
};
@Injectable()
export class ListAttachedRolePoliciesHandler extends AbstractActionHandler<QueryParams> {
constructor(
@InjectRepository(IamRole)
private readonly roleRepo: Repository<IamRole>,
@InjectRepository(IamPolicy)
private readonly policyRepo: Repository<IamPolicy>,
@InjectRepository(IamRolePolicyAttachment)
private readonly attachmentRepo: Repository<IamRolePolicyAttachment>,
) {
constructor(private readonly iamService: IamService) {
super();
}
format = Format.Xml;
action = Action.IamListAttachedRolePolicies;
validator = Joi.object<QueryParams, true>({
validator = Joi.object<QueryParams, true>({
RoleName: Joi.string().required(),
});
protected async handle({ RoleName }: QueryParams, awsProperties: AwsProperties) {
const role = await this.roleRepo.findOne({ where: { roleName: RoleName, accountId: awsProperties.accountId } });
if (!role) {
throw new NotFoundException('NoSuchEntity', 'The request was rejected because it referenced a resource entity that does not exist. The error message describes the resource.');
}
const attachments = await this.attachmentRepo.find({ where: { roleId: role.id } })
const policyIds = attachments.map(({ policyArn }) => breakdownArn(policyArn)).map(({ identifier }) => identifier.split('/')[1]);
const policies = await this.policyRepo.find({ where: { name: In(policyIds), isDefault: true } });
protected async handle({ RoleName }: QueryParams, { awsProperties }: RequestContext) {
const policies = await this.iamService.findAttachedRolePoliciesByRoleName(awsProperties.accountId, RoleName);
return {
AttachedPolicies: {
member: [role.assumeRolePolicyDocument, ...policies].map(p => ({
member: policies.map(p => ({
PolicyName: p.name,
PolicyArn: p.arn,
})),
}
}
},
IsTruncated: false,
};
}
}

View File

@@ -1,44 +1,38 @@
import { Injectable, NotFoundException } from '@nestjs/common';
import { Injectable } from '@nestjs/common';
import { AbstractActionHandler, AwsProperties, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import * as Joi from 'joi';
import { InjectRepository } from '@nestjs/typeorm';
import { Repository } from 'typeorm';
import { IamRole } from './iam-role.entity';
import { IamRolePolicyAttachment } from './iam-role-policy-attachment.entity';
import { RequestContext } from '../_context/request.context';
import { IamService } from './iam.service';
type QueryParams = {
Marker: string;
MaxItems: number;
RoleName: string;
}
};
@Injectable()
export class ListRolePoliciesHandler extends AbstractActionHandler<QueryParams> {
constructor(
@InjectRepository(IamRole)
private readonly roleRepo: Repository<IamRole>,
@InjectRepository(IamRolePolicyAttachment)
private readonly attachmentRepo: Repository<IamRolePolicyAttachment>,
) {
constructor(private readonly iamService: IamService) {
super();
}
format = Format.Xml;
action = Action.IamListRolePolicies;
validator = Joi.object<QueryParams, true>({
validator = Joi.object<QueryParams, true>({
Marker: Joi.string().allow(null),
MaxItems: Joi.number().min(1).max(1000).default(100),
RoleName: Joi.string().required(),
});
protected async handle({ RoleName }: QueryParams, awsProperties: AwsProperties) {
const role = await this.roleRepo.findOne({ where: { roleName: RoleName, accountId: awsProperties.accountId } });
if (!role) {
throw new NotFoundException('NoSuchEntity', 'The request was rejected because it referenced a resource entity that does not exist. The error message describes the resource.');
}
protected async handle({ RoleName }: QueryParams, { awsProperties }: RequestContext) {
const policyNames = await this.iamService.listRoleInlinePolicies(awsProperties.accountId, RoleName);
return {
PolicyNames: [],
}
IsTruncated: false,
PolicyNames: {
member: policyNames,
},
};
}
}

View File

@@ -0,0 +1,32 @@
import { Injectable } from '@nestjs/common';
import { AbstractActionHandler, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import * as Joi from 'joi';
import { RequestContext } from '../_context/request.context';
import { IamService } from './iam.service';
type QueryParams = {
PolicyDocument: string;
PolicyName: string;
RoleName: string;
};
@Injectable()
export class PutRolePolicyHandler extends AbstractActionHandler<QueryParams> {
constructor(private readonly iamService: IamService) {
super();
}
format = Format.Xml;
action = Action.IamPutRolePolicy;
validator = Joi.object<QueryParams, true>({
PolicyDocument: Joi.string().required(),
PolicyName: Joi.string().required(),
RoleName: Joi.string().required(),
});
protected async handle({ PolicyDocument, PolicyName, RoleName }: QueryParams, { awsProperties }: RequestContext) {
await this.iamService.putRoleInlinePolicy(awsProperties.accountId, RoleName, PolicyName, PolicyDocument);
return {};
}
}

View File

@@ -0,0 +1,33 @@
import { Injectable } from '@nestjs/common';
import { AbstractActionHandler, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import * as Joi from 'joi';
import { IamService } from './iam.service';
import { RequestContext } from '../_context/request.context';
type QueryParams = {
RoleName: string;
Description: string;
};
@Injectable()
export class UpdateRoleDescriptionHandler extends AbstractActionHandler<QueryParams> {
constructor(private readonly iamService: IamService) {
super();
}
format = Format.Xml;
action = Action.IamUpdateRoleDescription;
validator = Joi.object<QueryParams, true>({
RoleName: Joi.string().min(1).max(64).required(),
Description: Joi.string().max(1000).allow('').required(),
});
protected async handle({ RoleName, Description }: QueryParams, { awsProperties }: RequestContext) {
const role = await this.iamService.updateRoleDescription(awsProperties.accountId, RoleName, Description);
return {
Role: role.metadata,
};
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -2,9 +2,9 @@ import { Injectable } from '@nestjs/common';
import { AbstractActionHandler, AwsProperties, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import * as Joi from 'joi';
import { KmsKeyAlias } from './kms-key-alias.entity';
import { InjectRepository } from '@nestjs/typeorm';
import { Repository } from 'typeorm';
import { KmsService } from './kms.service';
import { NotFoundException } from '../aws-shared-entities/aws-exceptions';
import { RequestContext } from '../_context/request.context';
type QueryParams = {
AliasName: string;
@@ -15,8 +15,7 @@ type QueryParams = {
export class CreateAliasHandler extends AbstractActionHandler<QueryParams> {
constructor(
@InjectRepository(KmsKeyAlias)
private readonly aliasRepo: Repository<KmsKeyAlias>,
private readonly kmsService: KmsService,
) {
super();
}
@@ -24,17 +23,27 @@ export class CreateAliasHandler extends AbstractActionHandler<QueryParams> {
format = Format.Json;
action = Action.KmsCreateAlias;
validator = Joi.object<QueryParams, true>({
AliasName: Joi.string().required(),
TargetKeyId: Joi.string().required(),
AliasName: Joi.string().min(1).max(256).regex(new RegExp(`^alias/[a-zA-Z0-9/_-]+$`)).required(),
});
protected async handle({ AliasName, TargetKeyId }: QueryParams, awsProperties: AwsProperties) {
protected async handle({ TargetKeyId, AliasName }: QueryParams, { awsProperties} : RequestContext) {
await this.aliasRepo.save({
name: AliasName.split('/')[1],
targetKeyId: TargetKeyId,
const keyRecord = await this.kmsService.findOneByRef(TargetKeyId, awsProperties);
if (!keyRecord) {
throw new NotFoundException();
}
await this.kmsService.createAlias({
accountId: awsProperties.accountId,
region: awsProperties.region,
name: AliasName,
kmsKey: {
connect: {
id: keyRecord.id,
},
},
});
}
}

View File

@@ -0,0 +1,208 @@
import { CustomerMasterKeySpec, KeySpec, KeyState, KeyUsageType, OriginType, Tag } from '@aws-sdk/client-kms';
import { Injectable } from '@nestjs/common';
import * as Joi from 'joi';
import { AbstractActionHandler, AwsProperties, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import { KmsService } from './kms.service';
import * as crypto from 'crypto';
import { keySpecToUsageType } from './kms-key.entity';
import { UnsupportedOperationException } from '../aws-shared-entities/aws-exceptions';
import { TagsService } from '../aws-shared-entities/tags.service';
import { RequestContext } from '../_context/request.context';
type NoUndefinedField<T> = { [P in keyof T]-?: NoUndefinedField<NonNullable<T[P]>> };
type QueryParams = {
BypassPolicyLockoutSafetyCheck: boolean;
CustomerMasterKeySpec: CustomerMasterKeySpec;
CustomKeyStoreId: string;
Description: string;
KeySpec: KeySpec;
KeyUsage: KeyUsageType;
MultiRegion: boolean;
Origin: OriginType;
Policy: string;
Tags: NoUndefinedField<Tag>[];
XksKeyId: string;
};
const generateDefaultPolicy = (accountId: string) =>
JSON.stringify({
Sid: 'Enable IAM User Permissions',
Effect: 'Allow',
Principal: {
AWS: `arn:aws:iam::${accountId}:root`,
},
Action: 'kms:*',
Resource: '*',
});
@Injectable()
export class CreateKeyHandler extends AbstractActionHandler<QueryParams> {
constructor(private readonly kmsService: KmsService, private readonly tagsService: TagsService) {
super();
}
format = Format.Json;
action = Action.KmsCreateKey;
validator = Joi.object<QueryParams, true>({
BypassPolicyLockoutSafetyCheck: Joi.boolean().default(false),
CustomerMasterKeySpec: Joi.string().allow(...Object.values(CustomerMasterKeySpec)),
CustomKeyStoreId: Joi.string().min(1).max(64),
Description: Joi.string().min(0).max(8192).default(''),
KeySpec: Joi.string()
.allow(...Object.values(KeySpec))
.default(KeySpec.SYMMETRIC_DEFAULT),
KeyUsage: Joi.string()
.allow(...Object.values(KeyUsageType))
.default(KeyUsageType.ENCRYPT_DECRYPT),
MultiRegion: Joi.boolean().default(false),
Origin: Joi.string()
.allow(...Object.values(OriginType))
.default(OriginType.AWS_KMS),
Policy: Joi.string().min(1).max(32768),
Tags: Joi.array().items(
Joi.object<Tag, true>({
TagKey: Joi.string().min(1).max(128).required(),
TagValue: Joi.string().min(0).max(256).required(),
}),
),
XksKeyId: Joi.when('Origin', {
is: OriginType.EXTERNAL_KEY_STORE,
then: Joi.string().min(1).max(128),
otherwise: Joi.forbidden(),
}) as unknown as Joi.StringSchema,
});
protected async handle(
{ KeyUsage, Description, KeySpec, Origin, MultiRegion, Policy, Tags, CustomerMasterKeySpec }: QueryParams,
{ awsProperties }: RequestContext,
) {
const keySpec = CustomerMasterKeySpec ?? KeySpec;
if (!keySpecToUsageType[keySpec].includes(KeyUsage)) {
throw new UnsupportedOperationException(`KeySpec ${KeySpec} is not valid for KeyUsage ${KeyUsage}`);
}
const key = this.keyGeneratorMap[keySpec]();
const createdKey = await this.kmsService.createKmsKey({
id: crypto.randomUUID(),
enabled: true,
usage: KeyUsage,
description: Description,
keySpec: keySpec,
keyState: KeyState.Enabled,
origin: Origin,
multiRegion: MultiRegion,
policy: Policy ?? generateDefaultPolicy(awsProperties.accountId),
key: new Uint8Array(key),
accountId: awsProperties.accountId,
region: awsProperties.region,
});
if (Tags && Tags.length > 0) {
await this.tagsService.createMany(
createdKey.arn,
Tags.map(({ TagKey, TagValue }) => ({ key: TagKey, value: TagValue })),
);
}
return {
KeyMetadata: createdKey.metadata,
};
}
private keyGeneratorMap: Record<KeySpec, () => Buffer> = {
ECC_NIST_P256: function (): Buffer {
const { privateKey, publicKey } = crypto.generateKeyPairSync('ec', { namedCurve: 'prime256v1' });
return Buffer.from(JSON.stringify({ privateKey, publicKey }));
},
ECC_NIST_P384: function (): Buffer {
const { privateKey, publicKey } = crypto.generateKeyPairSync('ec', { namedCurve: 'secp384r1' });
return Buffer.from(JSON.stringify({ privateKey, publicKey }));
},
ECC_NIST_P521: function (): Buffer {
const { privateKey, publicKey } = crypto.generateKeyPairSync('ec', { namedCurve: 'secp521r1' });
return Buffer.from(JSON.stringify({ privateKey, publicKey }));
},
ECC_SECG_P256K1: function (): Buffer {
const { privateKey, publicKey } = crypto.generateKeyPairSync('ec', { namedCurve: 'secp256k1' });
return Buffer.from(JSON.stringify({ privateKey, publicKey }));
},
ECC_NIST_EDWARDS25519: function (): Buffer {
const { privateKey, publicKey } = crypto.generateKeyPairSync('ed25519');
return Buffer.from(JSON.stringify({ privateKey, publicKey }));
},
ML_DSA_44: function (): Buffer {
return crypto.randomBytes(2528);
},
ML_DSA_65: function (): Buffer {
return crypto.randomBytes(4000);
},
ML_DSA_87: function (): Buffer {
return crypto.randomBytes(4896);
},
HMAC_224: function (): Buffer {
return crypto.randomBytes(32);
},
HMAC_256: function (): Buffer {
return crypto.randomBytes(32);
},
HMAC_384: function (): Buffer {
return crypto.randomBytes(32);
},
HMAC_512: function (): Buffer {
return crypto.randomBytes(32);
},
RSA_2048: function (): Buffer {
const { privateKey, publicKey } = crypto.generateKeyPairSync('rsa', {
modulusLength: 2048,
publicKeyEncoding: {
type: 'pkcs1',
format: 'pem',
},
privateKeyEncoding: {
type: 'pkcs8',
format: 'pem',
},
});
return Buffer.from(JSON.stringify({ privateKey, publicKey }));
},
RSA_3072: function (): Buffer {
const { privateKey, publicKey } = crypto.generateKeyPairSync('rsa', {
modulusLength: 3072,
publicKeyEncoding: {
type: 'pkcs1',
format: 'pem',
},
privateKeyEncoding: {
type: 'pkcs8',
format: 'pem',
},
});
return Buffer.from(JSON.stringify({ privateKey, publicKey }));
},
RSA_4096: function (): Buffer {
const { privateKey, publicKey } = crypto.generateKeyPairSync('rsa', {
modulusLength: 4096,
publicKeyEncoding: {
type: 'pkcs1',
format: 'pem',
},
privateKeyEncoding: {
type: 'pkcs8',
format: 'pem',
},
});
return Buffer.from(JSON.stringify({ privateKey, publicKey }));
},
SM2: function (): Buffer {
throw new Error('Function not implemented.');
},
SYMMETRIC_DEFAULT: function (): Buffer {
return crypto.randomBytes(32);
},
};
}

View File

@@ -0,0 +1,38 @@
import { Injectable } from '@nestjs/common';
import { AbstractActionHandler, AwsProperties, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import * as Joi from 'joi';
import { KmsService } from './kms.service';
import { NotFoundException } from '../aws-shared-entities/aws-exceptions';
import { RequestContext } from '../_context/request.context';
type QueryParams = {
AliasName: string;
};
@Injectable()
export class DeleteAliasHandler extends AbstractActionHandler<QueryParams> {
constructor(private readonly kmsService: KmsService) {
super();
}
format = Format.Json;
action = Action.KmsDeleteAlias;
validator = Joi.object<QueryParams, true>({
AliasName: Joi.string().min(1).max(256).regex(new RegExp(`^alias/[a-zA-Z0-9/_-]+$`)).required(),
});
protected async handle({ AliasName }: QueryParams, { awsProperties }: RequestContext) {
// Verify the alias exists before deleting
const alias = await this.kmsService.findAliasByName(awsProperties.accountId, awsProperties.region, AliasName);
if (!alias) {
throw new NotFoundException();
}
await this.kmsService.deleteAlias(awsProperties.accountId, awsProperties.region, AliasName);
// DeleteAlias returns an empty response on success
return {};
}
}

View File

@@ -2,23 +2,24 @@ import { Injectable } from '@nestjs/common';
import { AbstractActionHandler, AwsProperties, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import * as Joi from 'joi';
import { InjectRepository } from '@nestjs/typeorm';
import { Repository } from 'typeorm';
import { KmsKey } from './kms-key.entity';
import { breakdownArn } from '../util/breakdown-arn';
import { KmsService } from './kms.service';
import { NotFoundException } from '../aws-shared-entities/aws-exceptions';
import { RequestContext } from '../_context/request.context';
type QueryParams = {
GrantTokens?: string[];
KeyId: string;
}
/**
* Known Issues:
* - Terraform apply with lookup loops on describe-key
*/
@Injectable()
export class DescribeKeyHandler extends AbstractActionHandler<QueryParams> {
constructor(
private readonly kmsService: KmsService,
@InjectRepository(KmsKey)
private readonly keyRepo: Repository<KmsKey>,
) {
super();
}
@@ -27,27 +28,16 @@ export class DescribeKeyHandler extends AbstractActionHandler<QueryParams> {
action = Action.KmsDescribeKey;
validator = Joi.object<QueryParams, true>({
KeyId: Joi.string().required(),
GrantTokens: Joi.array().items(Joi.string()),
});
protected async handle({ KeyId }: QueryParams, awsProperties: AwsProperties) {
protected async handle({ KeyId }: QueryParams, { awsProperties} : RequestContext) {
const searchable = KeyId.startsWith('arn') ? breakdownArn(KeyId) : {
service: 'kms',
region: awsProperties.region,
accountId: awsProperties.accountId,
identifier: KeyId,
};
const [ type, pk ] = searchable.identifier.split('/');
const keyId: Promise<string> = type === 'key' ?
Promise.resolve(pk) :
this.kmsService.findKeyIdFromAlias(pk, searchable);
const keyRecord = await this.kmsService.findOneByRef(KeyId, awsProperties);
const keyRecord = await this.keyRepo.findOne({ where: {
id: await keyId,
region: searchable.region,
accountId: searchable.accountId,
}});
if (!keyRecord) {
throw new NotFoundException();
}
return {
KeyMetadata: keyRecord.metadata,

View File

@@ -0,0 +1,46 @@
import { Injectable } from '@nestjs/common';
import { AbstractActionHandler, AwsProperties, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import * as Joi from 'joi';
import { KmsService } from './kms.service';
import { NotFoundException } from '../aws-shared-entities/aws-exceptions';
import { RequestContext } from '../_context/request.context';
type QueryParams = {
KeyId: string;
RotationPeriodInDays: number;
}
@Injectable()
export class EnableKeyRotationHandler extends AbstractActionHandler<QueryParams> {
constructor(
private readonly kmsService: KmsService,
) {
super();
}
format = Format.Json;
action = Action.KmsEnableKeyRotation;
validator = Joi.object<QueryParams, true>({
KeyId: Joi.string().required(),
RotationPeriodInDays: Joi.number().min(90).max(2560).default(365),
});
protected async handle({ KeyId, RotationPeriodInDays }: QueryParams, context: RequestContext) {
const keyRecord = await this.kmsService.findOneByRef(KeyId, context.awsProperties);
if (!keyRecord) {
throw new NotFoundException();
}
const next = new Date();
next.setDate(next.getDate() + RotationPeriodInDays);
await this.kmsService.updateKmsKey(keyRecord.id, {
rotationPeriod: RotationPeriodInDays,
nextRotation: next,
});
}
}

View File

@@ -0,0 +1,43 @@
import { Injectable } from '@nestjs/common';
import { AbstractActionHandler, AwsProperties, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import * as Joi from 'joi';
import { KmsService } from './kms.service';
import { NotFoundException } from '../aws-shared-entities/aws-exceptions';
import { RequestContext } from '../_context/request.context';
type QueryParams = {
PolicyName: string;
KeyId: string;
}
@Injectable()
export class GetKeyPolicyHandler extends AbstractActionHandler<QueryParams> {
constructor(
private readonly kmsService: KmsService,
) {
super();
}
format = Format.Json;
action = Action.KmsGetKeyPolicy;
validator = Joi.object<QueryParams, true>({
KeyId: Joi.string().required(),
PolicyName: Joi.string().min(1).max(128).default('default'),
});
protected async handle({ KeyId, PolicyName }: QueryParams, context: RequestContext) {
const keyRecord = await this.kmsService.findOneByRef(KeyId, context.awsProperties);
if (!keyRecord) {
throw new NotFoundException();
}
return {
PolicyName,
Policy: keyRecord.policy,
}
}
}

View File

@@ -0,0 +1,43 @@
import { Injectable } from '@nestjs/common';
import { AbstractActionHandler, AwsProperties, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import * as Joi from 'joi';
import { KmsService } from './kms.service';
import { NotFoundException } from '../aws-shared-entities/aws-exceptions';
import { RequestContext } from '../_context/request.context';
type QueryParams = {
KeyId: string;
}
@Injectable()
export class GetKeyRotationStatusHandler extends AbstractActionHandler<QueryParams> {
constructor(
private readonly kmsService: KmsService,
) {
super();
}
format = Format.Json;
action = Action.KmsGetKeyRotationStatus;
validator = Joi.object<QueryParams, true>({
KeyId: Joi.string().required(),
});
protected async handle({ KeyId }: QueryParams, { awsProperties} : RequestContext) {
const keyRecord = await this.kmsService.findOneByRef(KeyId, awsProperties);
if (!keyRecord) {
throw new NotFoundException();
}
return {
KeyId: keyRecord.id,
KeyRotationEnabled: !!keyRecord.rotationPeriod,
NextRotationDate: keyRecord.nextRotation?.getAwsTime(),
RotationPeriodInDays: keyRecord.rotationPeriod,
}
}
}

View File

@@ -2,45 +2,18 @@ import { Injectable } from '@nestjs/common';
import { AbstractActionHandler, AwsProperties, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import * as Joi from 'joi';
import { InjectRepository } from '@nestjs/typeorm';
import { Repository } from 'typeorm';
import { KeySpec, KeyUsage, KmsKey } from './kms-key.entity';
import { breakdownArn } from '../util/breakdown-arn';
import { KmsService } from './kms.service';
import * as crypto from 'crypto';
import { NotFoundException } from '../aws-shared-entities/aws-exceptions';
import { RequestContext } from '../_context/request.context';
type QueryParams = {
GrantTokens: string[];
KeyId: string;
}
interface StandardOutput {
KeyId: string;
KeySpec: KeySpec;
KeyUsage: KeyUsage;
PublicKey: string;
CustomerMasterKeySpec: KeySpec;
}
interface EncryptDecrypt extends StandardOutput {
KeyUsage: 'ENCRYPT_DECRYPT';
EncryptionAlgorithms: ('SYMMETRIC_DEFAULT' | 'RSAES_OAEP_SHA_1' | 'RSAES_OAEP_SHA_256' | 'SM2PKE')[];
}
interface SignVerify extends StandardOutput {
KeyUsage: 'SIGN_VERIFY';
SigningAlgorithms: ('RSASSA_PSS_SHA_256' | 'RSASSA_PSS_SHA_384' | 'RSASSA_PSS_SHA_512' | 'RSASSA_PKCS1_V1_5_SHA_256' | 'RSASSA_PKCS1_V1_5_SHA_384' | 'RSASSA_PKCS1_V1_5_SHA_512' | 'ECDSA_SHA_256' | 'ECDSA_SHA_384' | 'ECDSA_SHA_512' | 'SM2DSA')[];
}
type Output = EncryptDecrypt | SignVerify | StandardOutput;
@Injectable()
export class GetPublicKeyHandler extends AbstractActionHandler<QueryParams> {
constructor(
@InjectRepository(KmsKey)
private readonly keyRepo: Repository<KmsKey>,
private readonly kmsService: KmsService,
) {
super();
@@ -50,74 +23,19 @@ export class GetPublicKeyHandler extends AbstractActionHandler<QueryParams> {
action = Action.KmsGetPublicKey;
validator = Joi.object<QueryParams, true>({
KeyId: Joi.string().required(),
GrantTokens: Joi.array().items(Joi.string()),
});
protected async handle({ KeyId }: QueryParams, awsProperties: AwsProperties): Promise<Output> {
protected async handle({ KeyId }: QueryParams, { awsProperties} : RequestContext) {
const searchable = KeyId.startsWith('arn') ? breakdownArn(KeyId) : {
service: 'kms',
region: awsProperties.region,
accountId: awsProperties.accountId,
identifier: KeyId,
};
const [ type, pk ] = searchable.identifier.split('/');
const keyId: Promise<string> = type === 'key' ?
Promise.resolve(pk) :
this.kmsService.findKeyIdFromAlias(pk, searchable);
const keyRecord = await this.kmsService.findOneByRef(KeyId, awsProperties);
const keyRecord = await this.keyRepo.findOne({ where: {
id: await keyId,
region: searchable.region,
accountId: searchable.accountId,
}});
const pubKeyObject = crypto.createPublicKey({
key: keyRecord.key,//.split(String.raw`\n`).join('\n'),
format: 'pem',
});
if (keyRecord.usage === 'ENCRYPT_DECRYPT') {
return {
CustomerMasterKeySpec: keyRecord.keySpec,
EncryptionAlgorithms: [ "SYMMETRIC_DEFAULT" ],
KeyId: keyRecord.arn,
KeySpec: keyRecord.keySpec,
KeyUsage: keyRecord.usage,
PublicKey: Buffer.from(pubKeyObject.export({
format: 'der',
type: 'spki',
})).toString('base64'),
}
}
if (keyRecord.usage === 'SIGN_VERIFY') {
const PublicKey = Buffer.from(pubKeyObject.export({
format: 'der',
type: 'spki',
})).toString('base64')
console.log({PublicKey})
return {
CustomerMasterKeySpec: keyRecord.keySpec,
KeyId: keyRecord.arn,
KeySpec: keyRecord.keySpec,
KeyUsage: keyRecord.usage,
PublicKey,
SigningAlgorithms: [ 'RSASSA_PKCS1_V1_5_SHA_256' ]
}
if (!keyRecord) {
throw new NotFoundException();
}
return {
CustomerMasterKeySpec: keyRecord.keySpec,
KeyId: keyRecord.arn,
KeySpec: keyRecord.keySpec,
KeyUsage: keyRecord.usage,
PublicKey: Buffer.from(pubKeyObject.export({
format: 'pem',
type: 'spki',
})).toString('utf-8'),
...keyRecord.metadata,
PublicKey: Buffer.from(keyRecord.keyPair.publicKey).toString('base64'),
}
}
}

View File

@@ -0,0 +1,34 @@
import { KmsAlias as PrismaKeyAlias } from "@prisma/client"
export class KmsAlias implements PrismaKeyAlias {
name: string
accountId: string
region: string
kmsKeyId: string
createdAt: Date;
updatedAt: Date;
constructor(p: PrismaKeyAlias) {
this.name = p.name;
this.accountId = p.accountId;
this.region = p.region;
this.kmsKeyId = p.kmsKeyId;
this.createdAt = p.createdAt;
this.updatedAt = p.updatedAt;
}
get arn() {
return `arn:aws:kms:${this.region}:${this.accountId}:${this.name}`;
}
toAws() {
return {
AliasArn: this.arn,
AliasName: this.name,
CreationDate: this.createdAt.getAwsTime(),
LastUpdatedDate: this.updatedAt.getAwsTime(),
TargetKeyId: this.kmsKeyId,
}
}
}

View File

@@ -1,21 +0,0 @@
import { BaseEntity, Column, Entity, PrimaryColumn } from 'typeorm';
@Entity({ name: 'kms_key_alias' })
export class KmsKeyAlias extends BaseEntity {
@PrimaryColumn()
name: string;
@Column({ name: 'target_key_id' })
targetKeyId: string;
@Column({ name: 'account_id', nullable: false })
accountId: string;
@Column({ name: 'region', nullable: false })
region: string;
get arn() {
return `arn:aws:kms:${this.region}:${this.accountId}:alias/${this.name}`;
}
}

View File

@@ -1,61 +1,135 @@
import { BaseEntity, Column, CreateDateColumn, Entity, PrimaryColumn } from 'typeorm';
import {
KeySpec,
KeyUsageType,
KeyState,
AlgorithmSpec,
OriginType,
ExpirationModelType,
KeyAgreementAlgorithmSpec,
MacAlgorithmSpec,
MultiRegionKeyType,
SigningAlgorithmSpec,
} from '@aws-sdk/client-kms';
import { KmsKey as PrismaKmsKey } from '@prisma/client';
export type KeySpec = 'RSA_2048' | 'RSA_3072' | 'RSA_4096' | 'ECC_NIST_P256' | 'ECC_NIST_P384' | 'ECC_NIST_P521' | 'ECC_SECG_P256K1' | 'SYMMETRIC_DEFAULT' | 'HMAC_224' | 'HMAC_256' | 'HMAC_384' | 'HMAC_512' | 'SM2';
export type KeyUsage = 'SIGN_VERIFY' | 'ENCRYPT_DECRYPT' | 'GENERATE_VERIFY_MAC';
export const keySpecToUsageType: Record<KeySpec, KeyUsageType[]> = {
ECC_NIST_P256: [KeyUsageType.SIGN_VERIFY, KeyUsageType.KEY_AGREEMENT],
ECC_NIST_P384: [KeyUsageType.SIGN_VERIFY, KeyUsageType.KEY_AGREEMENT],
ECC_NIST_P521: [KeyUsageType.SIGN_VERIFY, KeyUsageType.KEY_AGREEMENT],
ECC_SECG_P256K1: [KeyUsageType.SIGN_VERIFY],
ECC_NIST_EDWARDS25519: [KeyUsageType.SIGN_VERIFY],
HMAC_224: [KeyUsageType.GENERATE_VERIFY_MAC],
HMAC_256: [KeyUsageType.GENERATE_VERIFY_MAC],
HMAC_384: [KeyUsageType.GENERATE_VERIFY_MAC],
HMAC_512: [KeyUsageType.GENERATE_VERIFY_MAC],
RSA_2048: [KeyUsageType.ENCRYPT_DECRYPT, KeyUsageType.SIGN_VERIFY],
RSA_3072: [KeyUsageType.ENCRYPT_DECRYPT, KeyUsageType.SIGN_VERIFY],
RSA_4096: [KeyUsageType.ENCRYPT_DECRYPT, KeyUsageType.SIGN_VERIFY],
SM2: [KeyUsageType.ENCRYPT_DECRYPT, KeyUsageType.SIGN_VERIFY, KeyUsageType.KEY_AGREEMENT],
ML_DSA_44: [KeyUsageType.SIGN_VERIFY],
ML_DSA_65: [KeyUsageType.SIGN_VERIFY],
ML_DSA_87: [KeyUsageType.SIGN_VERIFY],
SYMMETRIC_DEFAULT: [KeyUsageType.ENCRYPT_DECRYPT],
};
@Entity({ name: 'kms_key'})
export class KmsKey extends BaseEntity {
@PrimaryColumn()
export class KmsKey implements PrismaKmsKey {
id: string;
@Column({ name: 'usage' })
usage: KeyUsage;
@Column({ name: 'description' })
enabled: boolean;
usage: KeyUsageType;
description: string;
@Column({ name: 'key_spec' })
keySpec: KeySpec;
@Column({ name: 'key' })
key: string;
@Column({ name: 'account_id', nullable: false })
keyState: KeyState;
origin: OriginType;
multiRegion: boolean;
policy: string;
key: Uint8Array<ArrayBuffer>;
nextRotation: Date | null;
rotationPeriod: number | null;
accountId: string;
@Column({ name: 'region', nullable: false })
region: string;
createdAt: Date;
updatedAt: Date;
@CreateDateColumn()
createdAt: string;
constructor(p: PrismaKmsKey) {
this.id = p.id;
this.enabled = p.enabled;
this.usage = p.usage as KeyUsageType;
this.description = p.description;
this.keySpec = p.keySpec as KeySpec;
this.keyState = p.keyState as KeyState;
this.origin = p.origin as OriginType;
this.multiRegion = p.multiRegion;
this.policy = p.policy;
this.key = Buffer.from(p.key);
this.nextRotation = p.nextRotation;
this.rotationPeriod = p.rotationPeriod;
this.accountId = p.accountId;
this.region = p.region;
this.createdAt = p.createdAt;
this.updatedAt = p.updatedAt;
}
get arn() {
return `arn:aws:kms:${this.region}:${this.accountId}:key/${this.id}`;
}
get keyPair(): { publicKey: string; privateKey: string } {
return JSON.parse(Buffer.from(this.key).toString('utf-8'));
}
get metadata() {
const dynamicContent: Record<string, any> = {};
if (keySpecToUsageType[this.keySpec].includes(KeyUsageType.ENCRYPT_DECRYPT)) {
// Symmetric keys don't include EncryptionAlgorithms in the response
// Only asymmetric encryption keys (RSA, SM2) include this field
if (this.keySpec !== KeySpec.SYMMETRIC_DEFAULT) {
dynamicContent.EncryptionAlgorithms = Object.values(AlgorithmSpec);
}
}
if (this.origin === OriginType.EXTERNAL) {
dynamicContent.ExpirationModel = ExpirationModelType.KEY_MATERIAL_DOES_NOT_EXPIRE;
}
if (keySpecToUsageType[this.keySpec].includes(KeyUsageType.KEY_AGREEMENT)) {
dynamicContent.KeyAgreementAlgorithms = Object.values(KeyAgreementAlgorithmSpec);
}
if (keySpecToUsageType[this.keySpec].includes(KeyUsageType.GENERATE_VERIFY_MAC)) {
dynamicContent.MacAlgorithms = Object.values(MacAlgorithmSpec);
}
if (this.multiRegion) {
dynamicContent.MultiRegionConfiguration = {
MultiRegionKeyType: MultiRegionKeyType.PRIMARY,
PrimaryKey: {
Arn: this.arn,
Region: this.region,
},
ReplicaKeys: [],
};
}
if (keySpecToUsageType[this.keySpec].includes(KeyUsageType.SIGN_VERIFY)) {
dynamicContent.SigningAlgorithms = Object.values(SigningAlgorithmSpec);
}
return {
AWSAccountId: this.accountId,
KeyId: this.id,
Arn: this.arn,
CreationDate: new Date(this.createdAt).toISOString(),
Enabled: true,
CreationDate: this.createdAt.getAwsTime(),
CustomerMasterKeySpec: this.keySpec, // Deprecated but still returned by AWS API for backwards compatibility
Description: this.description,
KeyUsage: this.usage,
KeyState: 'Enabled',
KeyManager: "CUSTOMER",
CustomerMasterKeySpec: this.keySpec,
Enabled: this.enabled,
KeyId: this.id,
KeyManager: 'CUSTOMER',
KeySpec: this.keySpec,
DeletionDate: null,
SigningAlgorithms: [
"RSASSA_PSS_SHA_256",
"RSASSA_PSS_SHA_384",
"RSASSA_PSS_SHA_512",
"RSASSA_PKCS1_V1_5_SHA_256",
"RSASSA_PKCS1_V1_5_SHA_384",
"RSASSA_PKCS1_V1_5_SHA_512"
]
}
KeyState: this.keyState,
KeyUsage: this.usage,
MultiRegion: this.multiRegion,
Origin: this.origin,
...dynamicContent,
};
}
}

View File

@@ -1,23 +1,40 @@
import { Module } from '@nestjs/common';
import { TypeOrmModule } from '@nestjs/typeorm';
import { Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import { AwsSharedEntitiesModule } from '../aws-shared-entities/aws-shared-entities.module';
import { DefaultActionHandlerProvider } from '../default-action-handler/default-action-handler.provider';
import { ExistingActionHandlersProvider } from '../default-action-handler/existing-action-handlers.provider';
import { CreateAliasHandler } from './create-alias.handler';
import { DescribeKeyHandler } from './describe-key.handler';
import { KmsKeyAlias } from './kms-key-alias.entity';
import { KmsKey } from './kms-key.entity';
import { KMSHandlers } from './kms.constants';
import { KmsService } from './kms.service';
import { KMSHandlers } from './kms.constants';
import { DescribeKeyHandler } from './describe-key.handler';
import { PrismaModule } from '../_prisma/prisma.module';
import { ListAliasesHandler } from './list-aliases.handler';
import { CreateKeyHandler } from './create-key.handler';
import { EnableKeyRotationHandler } from './enable-key-rotation.handler';
import { GetKeyRotationStatusHandler } from './get-key-rotation-status.handler';
import { GetKeyPolicyHandler } from './get-key-policy.handler';
import { ListResourceTagsHandler } from './list-resource-tags.handler';
import { CreateAliasHandler } from './create-alias.handler';
import { GetPublicKeyHandler } from './get-public-key.handler';
import { SignHandler } from './sign.handler';
import { DeleteAliasHandler } from './delete-alias.handler';
import { ScheduleKeyDeletionHandler } from './schedule-key-deletion.handler';
const handlers = [
CreateAliasHandler,
CreateKeyHandler,
DeleteAliasHandler,
DescribeKeyHandler,
EnableKeyRotationHandler,
GetKeyPolicyHandler,
GetKeyRotationStatusHandler,
GetPublicKeyHandler,
]
ListAliasesHandler,
ListResourceTagsHandler,
ScheduleKeyDeletionHandler,
SignHandler,
];
const actions = [
Action.KmsCancelKeyDeletion,
@@ -70,13 +87,10 @@ const actions = [
Action.KmsUpdatePrimaryRegion,
Action.KmsVerify,
Action.KmsVerifyMac,
]
];
@Module({
imports: [
TypeOrmModule.forFeature([KmsKey, KmsKeyAlias]),
AwsSharedEntitiesModule,
],
imports: [AwsSharedEntitiesModule, PrismaModule],
providers: [
...handlers,
KmsService,

View File

@@ -1,22 +1,140 @@
import { Injectable } from '@nestjs/common';
import { ArnParts } from '../util/breakdown-arn';
import { InjectRepository } from '@nestjs/typeorm';
import { KmsKeyAlias } from './kms-key-alias.entity';
import { Repository } from 'typeorm';
import { Prisma } from '@prisma/client';
import { PrismaService } from '../_prisma/prisma.service';
import { breakdownArn } from '../util/breakdown-arn';
import { KmsKey } from './kms-key.entity';
import { KmsAlias } from './kms-alias.entity';
import { AwsProperties } from '../abstract-action.handler';
import { NotFoundException } from '../aws-shared-entities/aws-exceptions';
import { RequestContext } from '../_context/request.context';
@Injectable()
export class KmsService {
constructor(
@InjectRepository(KmsKeyAlias)
private readonly aliasRepo: Repository<KmsKeyAlias>,
) {}
constructor(private readonly prismaService: PrismaService) {}
async findKeyIdFromAlias(alias: string, arn: ArnParts): Promise<string> {
const record = await this.aliasRepo.findOne({ where: {
name: alias,
accountId: arn.accountId,
region: arn.region,
}});
return record.targetKeyId;
async findOneByRef(ref: string, awsProperties: AwsProperties): Promise<KmsKey> {
if (ref.startsWith('arn')) {
return await this.findOneByArn(ref);
}
return await this.findOneById(awsProperties.accountId, awsProperties.region, ref);
}
async findOneByArn(arn: string): Promise<KmsKey> {
const parts = breakdownArn(arn);
return await this.findOneById(parts.accountId, parts.region, parts.identifier.split('/')[1]);
}
async findOneById(accountId: string, region: string, ref: string): Promise<KmsKey> {
const [alias, record] = await Promise.all([
this.prismaService.kmsAlias.findFirst({
include: {
kmsKey: true,
},
where: {
accountId,
region,
name: ref,
},
}),
this.prismaService.kmsKey.findFirst({
where: {
accountId,
region,
id: ref,
},
}),
]);
if (!alias?.kmsKey && !record) {
throw new NotFoundException();
}
return record ? new KmsKey(record) : new KmsKey(alias!.kmsKey);
}
async findAndCountAliasesByKeyId(accountId: string, region: string, limit: number, kmsKeyId: string, marker = ''): Promise<KmsAlias[]> {
const take = limit + 1;
const records = await this.prismaService.kmsAlias.findMany({
where: {
accountId,
region,
kmsKeyId,
name: {
gte: marker,
},
},
take,
orderBy: {
name: 'desc',
},
});
return records.map(r => new KmsAlias(r));
}
async findAndCountAliases(accountId: string, region: string, limit: number, marker = ''): Promise<KmsAlias[]> {
const take = limit + 1;
const records = await this.prismaService.kmsAlias.findMany({
where: {
accountId,
region,
name: {
gte: marker,
},
},
take,
orderBy: {
name: 'desc',
},
});
return records.map(r => new KmsAlias(r));
}
async createKmsKey(data: Prisma.KmsKeyCreateInput): Promise<KmsKey> {
const record = await this.prismaService.kmsKey.create({
data,
});
return new KmsKey(record);
}
async updateKmsKey(id: string, data: Prisma.KmsKeyUpdateInput): Promise<void> {
await this.prismaService.kmsKey.update({
where: { id },
data,
});
}
async createAlias(data: Prisma.KmsAliasCreateInput) {
await this.prismaService.kmsAlias.create({
data,
});
}
async findAliasByName(accountId: string, region: string, name: string): Promise<KmsAlias | null> {
const record = await this.prismaService.kmsAlias.findUnique({
where: {
accountId_region_name: {
accountId,
region,
name,
},
},
});
return record ? new KmsAlias(record) : null;
}
async deleteAlias(accountId: string, region: string, name: string): Promise<void> {
await this.prismaService.kmsAlias.delete({
where: {
accountId_region_name: {
accountId,
region,
name,
},
},
});
}
}

View File

@@ -0,0 +1,47 @@
import { Injectable } from '@nestjs/common';
import * as Joi from 'joi';
import { AbstractActionHandler, AwsProperties, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import { KmsService } from './kms.service';
import { RequestContext } from '../_context/request.context';
type QueryParams = {
KeyId?: string;
Limit: number;
Marker?: string;
}
@Injectable()
export class ListAliasesHandler extends AbstractActionHandler<QueryParams> {
constructor(
private readonly kmsService: KmsService,
) {
super();
}
format = Format.Json;
action = Action.KmsListAliases;
validator = Joi.object<QueryParams, true>({
KeyId: Joi.string(),
Limit: Joi.number().min(1).max(100).default(50),
Marker: Joi.string(),
});
protected async handle({ KeyId, Limit, Marker }: QueryParams, { awsProperties} : RequestContext) {
const records = await (KeyId
? this.kmsService.findAndCountAliasesByKeyId(awsProperties.accountId, awsProperties.region, Limit, KeyId, Marker)
: this.kmsService.findAndCountAliases(awsProperties.accountId, awsProperties.region, Limit, Marker)
)
const nextMarker = records.length > Limit ? records.pop() : null;
return {
Aliases: records.map(r => r.toAws()),
NextMarker: nextMarker?.name,
Truncated: !!nextMarker,
}
}
}

View File

@@ -0,0 +1,49 @@
import { Injectable } from '@nestjs/common';
import { AbstractActionHandler, AwsProperties, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import * as Joi from 'joi';
import { KmsService } from './kms.service';
import { NotFoundException } from '../aws-shared-entities/aws-exceptions';
import { TagsService } from '../aws-shared-entities/tags.service';
import { RequestContext } from '../_context/request.context';
type QueryParams = {
KeyId: string;
Limit: number;
Marker: string;
}
@Injectable()
export class ListResourceTagsHandler extends AbstractActionHandler<QueryParams> {
constructor(
private readonly kmsService: KmsService,
private readonly tagsService: TagsService,
) {
super();
}
format = Format.Json;
action = Action.KmsListResourceTags;
validator = Joi.object<QueryParams, true>({
KeyId: Joi.string().required(),
Limit: Joi.number().min(1).max(100).default(50),
Marker: Joi.string(),
});
protected async handle({ KeyId }: QueryParams, context: RequestContext) {
const keyRecord = await this.kmsService.findOneByRef(KeyId, context.awsProperties);
if (!keyRecord) {
throw new NotFoundException();
}
const tags = await this.tagsService.getByArn(keyRecord.arn);
return {
Tags: tags.map(({ name, value }) => ({ TagKey: name, TagValue: value })),
Truncated: false,
}
}
}

View File

@@ -0,0 +1,53 @@
import { Injectable } from '@nestjs/common';
import { AbstractActionHandler, AwsProperties, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import * as Joi from 'joi';
import { KmsService } from './kms.service';
import { NotFoundException } from '../aws-shared-entities/aws-exceptions';
import { RequestContext } from '../_context/request.context';
import { KeyState } from '@aws-sdk/client-kms';
type QueryParams = {
KeyId: string;
PendingWindowInDays?: number;
};
@Injectable()
export class ScheduleKeyDeletionHandler extends AbstractActionHandler<QueryParams> {
constructor(private readonly kmsService: KmsService) {
super();
}
format = Format.Json;
action = Action.KmsScheduleKeyDeletion;
validator = Joi.object<QueryParams, true>({
KeyId: Joi.string().required(),
PendingWindowInDays: Joi.number().integer().min(7).max(30).optional(),
});
protected async handle({ KeyId, PendingWindowInDays = 30 }: QueryParams, { awsProperties }: RequestContext) {
const keyRecord = await this.kmsService.findOneByRef(KeyId, awsProperties);
if (!keyRecord) {
throw new NotFoundException();
}
// Calculate the deletion date based on the pending window
const deletionDate = new Date();
deletionDate.setDate(deletionDate.getDate() + PendingWindowInDays);
// Update the key state to PendingDeletion
await this.kmsService.updateKmsKey(keyRecord.id, {
keyState: KeyState.PendingDeletion,
// Note: In a full implementation, you'd store the deletion date
// and have a background job that actually deletes keys after the window
});
return {
KeyId: keyRecord.id,
DeletionDate: deletionDate.getAwsTime(),
KeyState: KeyState.PendingDeletion,
PendingWindowInDays,
};
}
}

96
src/kms/sign.handler.ts Normal file
View File

@@ -0,0 +1,96 @@
import { Injectable } from '@nestjs/common';
import { AbstractActionHandler, AwsProperties, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import * as Joi from 'joi';
import { KmsService } from './kms.service';
import { NotFoundException, UnsupportedOperationException } from '../aws-shared-entities/aws-exceptions';
import * as crypto from 'crypto';
import { KeySpec, SigningAlgorithmSpec } from '@aws-sdk/client-kms';
import { KmsKey } from './kms-key.entity';
import { RequestContext } from '../_context/request.context';
type QueryParams = {
KeyId: string;
Message: string;
MessageType: string;
SigningAlgorithm: string;
};
const signingAlgorithmToSigningFn: Record<SigningAlgorithmSpec, (base64: string, key: KmsKey) => string> = {
ECDSA_SHA_256: function (base64: string): string {
throw new Error('Function not implemented.');
},
ECDSA_SHA_384: function (base64: string): string {
throw new Error('Function not implemented.');
},
ECDSA_SHA_512: function (base64: string): string {
throw new Error('Function not implemented.');
},
ED25519_PH_SHA_512: function (base64: string): string {
throw new Error('Function not implemented.');
},
ED25519_SHA_512: function (base64: string): string {
throw new Error('Function not implemented.');
},
ML_DSA_SHAKE_256: function (base64: string): string {
throw new Error('Function not implemented.');
},
RSASSA_PKCS1_V1_5_SHA_256: function (base64: string, key: KmsKey): string {
const buffer = Buffer.from(base64);
return crypto.sign('sha256WithRSAEncryption', buffer, key.keyPair.privateKey).toString('base64');
},
RSASSA_PKCS1_V1_5_SHA_384: function (base64: string): string {
throw new Error('Function not implemented.');
},
RSASSA_PKCS1_V1_5_SHA_512: function (base64: string): string {
throw new Error('Function not implemented.');
},
RSASSA_PSS_SHA_256: function (base64: string): string {
throw new Error('Function not implemented.');
},
RSASSA_PSS_SHA_384: function (base64: string): string {
throw new Error('Function not implemented.');
},
RSASSA_PSS_SHA_512: function (base64: string): string {
throw new Error('Function not implemented.');
},
SM2DSA: function (base64: string): string {
throw new Error('Function not implemented.');
},
};
@Injectable()
export class SignHandler extends AbstractActionHandler<QueryParams> {
constructor(private readonly kmsService: KmsService) {
super();
}
format = Format.Json;
action = Action.KmsSign;
validator = Joi.object<QueryParams, true>({
KeyId: Joi.string().required(),
Message: Joi.string().required(),
MessageType: Joi.string().default('RAW'),
SigningAlgorithm: Joi.string().required(),
});
protected async handle({ KeyId, Message, SigningAlgorithm }: QueryParams, { awsProperties }: RequestContext) {
const keyRecord = await this.kmsService.findOneByRef(KeyId, awsProperties);
if (!keyRecord) {
throw new NotFoundException();
}
if (!(keyRecord.metadata as any).SigningAlgorithms.includes(SigningAlgorithm)) {
throw new UnsupportedOperationException('Invalid signing algorithm');
}
const signature = signingAlgorithmToSigningFn[SigningAlgorithm as SigningAlgorithmSpec](Message, keyRecord);
return {
KeyId: keyRecord.arn,
Signature: signature,
SigningAlgorithm,
};
}
}

View File

@@ -1,19 +1,75 @@
import { ClassSerializerInterceptor } from '@nestjs/common';
import { NestFactory, Reflector } from '@nestjs/core';
import { AppModule } from './app.module';
import * as morgan from 'morgan';
import { CommonConfig } from './config/common-config.interface';
import { ConfigService } from '@nestjs/config';
import { NestFactory, Reflector } from '@nestjs/core';
import { AppModule } from './app.module';
import { S3AppModule } from './s3/s3-app.module';
import { ConsulKVAppModule } from './consul-kv/consul-kv-app.module';
import { CommonConfig } from './config/common-config.interface';
import { AwsExceptionFilter } from './_context/exception.filter';
const bodyParser = require('body-parser');
declare global {
interface Date {
getAwsTime(): number;
}
}
Date.prototype.getAwsTime = function (this: Date) {
return Math.floor(this.getTime() / 1000);
};
(async () => {
// Start main application
const app = await NestFactory.create(AppModule);
app.use(morgan('dev'));
app.useGlobalInterceptors(new ClassSerializerInterceptor(app.get(Reflector)));
app.use(bodyParser.json({ type: 'application/x-amz-json-1.1'}));
app.useGlobalFilters(new AwsExceptionFilter());
const configService: ConfigService<CommonConfig> = app.get(ConfigService)
// Parse JSON for SNS/SQS
app.use(bodyParser.json({ type: 'application/x-amz-json-1.0' }));
app.use(bodyParser.json({ type: 'application/x-amz-json-1.1' }));
await app.listen(configService.get('PORT'), () => console.log(`Listening on port ${configService.get('PORT')}`));
// Parse raw body for S3 binary data
app.use(bodyParser.raw({ type: 'application/octet-stream', limit: '50mb' }));
app.use(bodyParser.raw({ type: 'binary/octet-stream', limit: '50mb' }));
// Parse XML for S3
app.use(bodyParser.text({ type: 'application/xml' }));
app.use(bodyParser.text({ type: 'text/xml' }));
const configService: ConfigService<CommonConfig, true> = app.get(ConfigService);
const mainPort = configService.get('PORT');
await app.listen(mainPort, () => console.log(`Main service listening on port ${mainPort}`));
// Start S3 microservice
const s3App = await NestFactory.create(S3AppModule);
s3App.useGlobalFilters(new AwsExceptionFilter());
// Parse raw body for S3 binary data
s3App.use(bodyParser.raw({ type: '*/*', limit: '50mb' }));
// Parse URL encoded for S3 operations
s3App.use(bodyParser.urlencoded({ extended: true }));
const s3ConfigService: ConfigService<CommonConfig, true> = s3App.get(ConfigService);
const s3Port = s3ConfigService.get('S3_PORT');
await s3App.listen(s3Port, () => console.log(`S3 service listening on port ${s3Port}`));
// Start Consul KV microservice
const consulApp = await NestFactory.create(ConsulKVAppModule);
// Parse JSON for Consul KV
consulApp.use(bodyParser.json());
// Parse raw body for Consul KV binary data
consulApp.use(bodyParser.raw({ type: '*/*', limit: '50mb' }));
// Parse text for Consul KV
consulApp.use(bodyParser.text({ type: 'text/plain' }));
const consulConfigService: ConfigService<CommonConfig, true> = consulApp.get(ConfigService);
const consulPort = consulConfigService.get('CONSUL_PORT');
await consulApp.listen(consulPort, () => console.log(`Consul KV service listening on port ${consulPort}`));
})();

967
src/s3/__tests__/s3.spec.ts Normal file
View File

@@ -0,0 +1,967 @@
import { Test, TestingModule } from '@nestjs/testing';
import { INestApplication } from '@nestjs/common';
import {
S3Client,
CreateBucketCommand,
ListBucketsCommand,
HeadBucketCommand,
DeleteBucketCommand,
PutObjectCommand,
GetObjectCommand,
HeadObjectCommand,
DeleteObjectCommand,
ListObjectsCommand,
ListObjectsV2Command,
PutBucketTaggingCommand,
GetBucketTaggingCommand,
GetBucketPolicyCommand,
PutBucketAclCommand,
GetBucketAclCommand,
} from '@aws-sdk/client-s3';
import { S3AppModule } from '../s3-app.module';
import { PrismaService } from '../../_prisma/prisma.service';
import { AwsExceptionFilter } from '../../_context/exception.filter';
const bodyParser = require('body-parser');
describe('S3 Integration Tests', () => {
let app: INestApplication;
let s3Client: S3Client;
let prismaService: PrismaService;
const testPort = 8088;
beforeAll(async () => {
// Set test environment variables
process.env.PORT = testPort.toString();
process.env.S3_PORT = testPort.toString();
process.env.AWS_ACCOUNT_ID = '123456789012';
process.env.AWS_REGION = 'us-east-1';
process.env.DATABASE_URL = `file::memory:?cache=shared&unique=${Date.now()}-${Math.random()}`;
process.env.PERSISTANCE = ':memory:';
// Create NestJS testing module
const moduleFixture: TestingModule = await Test.createTestingModule({
imports: [S3AppModule],
}).compile();
app = moduleFixture.createNestApplication();
app.useGlobalFilters(new AwsExceptionFilter());
app.use(bodyParser.raw({ type: '*/*', limit: '10mb' }));
app.use(bodyParser.urlencoded({ extended: true }));
await app.init();
await app.listen(testPort);
// Configure S3 client to point to local endpoint
s3Client = new S3Client({
region: 'us-east-1',
endpoint: `http://localhost:${testPort}`,
credentials: {
accessKeyId: 'test',
secretAccessKey: 'test',
},
forcePathStyle: true, // Use path-style URLs (localhost:8088/bucket/key)
});
prismaService = moduleFixture.get<PrismaService>(PrismaService);
});
afterAll(async () => {
await app.close();
s3Client.destroy();
});
beforeEach(async () => {
// Clean up database before each test
await prismaService.s3Object.deleteMany({});
await prismaService.s3Bucket.deleteMany({});
});
describe('CreateBucket', () => {
it('should create a bucket successfully', async () => {
const command = new CreateBucketCommand({
Bucket: 'test-bucket',
});
const response = await s3Client.send(command);
// Verify response structure per AWS S3 API
expect(response.Location).toBeDefined();
expect(response.Location).toBe('/test-bucket');
expect(response.$metadata).toBeDefined();
expect(response.$metadata.httpStatusCode).toBe(200);
// Verify in database
const bucket = await prismaService.s3Bucket.findFirst({
where: { name: 'test-bucket' },
});
expect(bucket).toBeDefined();
expect(bucket?.name).toBe('test-bucket');
});
it('should handle duplicate bucket creation', async () => {
await s3Client.send(new CreateBucketCommand({ Bucket: 'duplicate-bucket' }));
const duplicateCommand = new CreateBucketCommand({ Bucket: 'duplicate-bucket' });
try {
await s3Client.send(duplicateCommand);
fail('Expected BucketAlreadyExists error');
} catch (error: any) {
expect(error.name).toBe('BucketAlreadyExists');
expect(error.$metadata?.httpStatusCode).toBe(409);
}
});
it('should create multiple buckets', async () => {
await s3Client.send(new CreateBucketCommand({ Bucket: 'bucket1' }));
await s3Client.send(new CreateBucketCommand({ Bucket: 'bucket2' }));
await s3Client.send(new CreateBucketCommand({ Bucket: 'bucket3' }));
const buckets = await prismaService.s3Bucket.findMany({});
expect(buckets.length).toBe(3);
});
});
describe('ListBuckets', () => {
it('should list all buckets', async () => {
await s3Client.send(new CreateBucketCommand({ Bucket: 'bucket-a' }));
await s3Client.send(new CreateBucketCommand({ Bucket: 'bucket-b' }));
const command = new ListBucketsCommand({});
const response = await s3Client.send(command);
// Verify response structure per AWS S3 API
expect(response.Buckets).toBeDefined();
expect(response.Buckets!.length).toBe(2);
expect(response.Buckets!.map(b => b.Name)).toContain('bucket-a');
expect(response.Buckets!.map(b => b.Name)).toContain('bucket-b');
expect(response.Owner).toBeDefined();
expect(response.Owner?.ID).toBeDefined();
expect(response.$metadata.httpStatusCode).toBe(200);
});
it('should return empty list when no buckets exist', async () => {
const command = new ListBucketsCommand({});
const response = await s3Client.send(command);
expect(response.Buckets).toBeDefined();
expect(response.Buckets!.length).toBe(0);
});
it('should include creation dates', async () => {
await s3Client.send(new CreateBucketCommand({ Bucket: 'dated-bucket' }));
const command = new ListBucketsCommand({});
const response = await s3Client.send(command);
expect(response.Buckets![0].CreationDate).toBeDefined();
expect(response.Buckets![0].CreationDate).toBeInstanceOf(Date);
});
});
describe('HeadBucket', () => {
it('should check if bucket exists', async () => {
await s3Client.send(new CreateBucketCommand({ Bucket: 'existing-bucket' }));
const command = new HeadBucketCommand({ Bucket: 'existing-bucket' });
const response = await s3Client.send(command);
// HeadBucket returns 200 with no body on success
expect(response.$metadata.httpStatusCode).toBe(200);
});
it('should return 404 for non-existent bucket', async () => {
const command = new HeadBucketCommand({ Bucket: 'non-existent-bucket' });
try {
await s3Client.send(command);
fail('Expected NoSuchBucket error');
} catch (error: any) {
// SDK may parse error.name as NotFound but Code should be NoSuchBucket
expect(error.Code || error.name).toMatch(/NoSuchBucket|NotFound/);
expect(error.$metadata?.httpStatusCode).toBe(404);
}
});
});
describe('DeleteBucket', () => {
it('should delete an empty bucket', async () => {
await s3Client.send(new CreateBucketCommand({ Bucket: 'delete-me' }));
const command = new DeleteBucketCommand({ Bucket: 'delete-me' });
await s3Client.send(command);
// Verify deleted
const bucket = await prismaService.s3Bucket.findFirst({
where: { name: 'delete-me' },
});
expect(bucket).toBeNull();
});
it('should fail to delete non-empty bucket', async () => {
await s3Client.send(new CreateBucketCommand({ Bucket: 'non-empty-bucket' }));
await s3Client.send(
new PutObjectCommand({
Bucket: 'non-empty-bucket',
Key: 'test.txt',
Body: Buffer.from('data'),
}),
);
const command = new DeleteBucketCommand({ Bucket: 'non-empty-bucket' });
await expect(s3Client.send(command)).rejects.toThrow();
});
it('should fail to delete non-existent bucket', async () => {
const command = new DeleteBucketCommand({ Bucket: 'does-not-exist' });
await expect(s3Client.send(command)).rejects.toThrow();
});
});
describe('PutObject', () => {
beforeEach(async () => {
await s3Client.send(new CreateBucketCommand({ Bucket: 'test-bucket' }));
});
it('should put an object successfully', async () => {
const content = Buffer.from('Hello, S3!');
const command = new PutObjectCommand({
Bucket: 'test-bucket',
Key: 'hello.txt',
Body: content,
ContentType: 'text/plain',
});
const response = await s3Client.send(command);
expect(response.ETag).toBeDefined();
expect(response.ETag).toMatch(/^"[a-f0-9]{32}"$/);
// Verify in database
const object = await prismaService.s3Object.findFirst({
where: { key: 'hello.txt' },
});
expect(object).toBeDefined();
expect(Buffer.from(object!.content).toString()).toBe('Hello, S3!');
});
it('should put object with metadata', async () => {
const command = new PutObjectCommand({
Bucket: 'test-bucket',
Key: 'metadata.txt',
Body: Buffer.from('test'),
Metadata: {
author: 'test-user',
version: '1.0',
},
});
await s3Client.send(command);
const object = await prismaService.s3Object.findFirst({
where: { key: 'metadata.txt' },
});
const metadata = JSON.parse(object!.metadata);
expect(metadata.author).toBe('test-user');
expect(metadata.version).toBe('1.0');
});
it('should overwrite existing object', async () => {
await s3Client.send(
new PutObjectCommand({
Bucket: 'test-bucket',
Key: 'overwrite.txt',
Body: Buffer.from('version 1'),
}),
);
await s3Client.send(
new PutObjectCommand({
Bucket: 'test-bucket',
Key: 'overwrite.txt',
Body: Buffer.from('version 2'),
}),
);
const objects = await prismaService.s3Object.findMany({
where: { key: 'overwrite.txt' },
});
expect(objects.length).toBe(1);
expect(Buffer.from(objects[0].content).toString()).toBe('version 2');
});
it('should handle nested keys', async () => {
const command = new PutObjectCommand({
Bucket: 'test-bucket',
Key: 'folder/subfolder/file.txt',
Body: Buffer.from('nested'),
});
await s3Client.send(command);
const object = await prismaService.s3Object.findFirst({
where: { key: 'folder/subfolder/file.txt' },
});
expect(object).toBeDefined();
});
it('should store binary content', async () => {
const binaryData = Buffer.from([0x00, 0x01, 0x02, 0xff]);
const command = new PutObjectCommand({
Bucket: 'test-bucket',
Key: 'binary.dat',
Body: binaryData,
});
await s3Client.send(command);
const object = await prismaService.s3Object.findFirst({
where: { key: 'binary.dat' },
});
expect(Buffer.from(object!.content)).toEqual(binaryData);
});
});
describe('GetObject', () => {
beforeEach(async () => {
await s3Client.send(new CreateBucketCommand({ Bucket: 'test-bucket' }));
});
it('should get an object successfully', async () => {
const content = Buffer.from('Hello, World!');
await s3Client.send(
new PutObjectCommand({
Bucket: 'test-bucket',
Key: 'test.txt',
Body: content,
}),
);
const command = new GetObjectCommand({
Bucket: 'test-bucket',
Key: 'test.txt',
});
const response = await s3Client.send(command);
expect(response.Body).toBeDefined();
expect(response.ETag).toBeDefined();
expect(response.ContentLength).toBe(content.length);
const bodyString = await response.Body!.transformToString();
expect(bodyString).toBe('Hello, World!');
});
it('should return metadata headers', async () => {
await s3Client.send(
new PutObjectCommand({
Bucket: 'test-bucket',
Key: 'meta.txt',
Body: Buffer.from('test'),
Metadata: {
custom: 'value',
},
}),
);
const command = new GetObjectCommand({
Bucket: 'test-bucket',
Key: 'meta.txt',
});
const response = await s3Client.send(command);
expect(response.Metadata).toBeDefined();
expect(response.Metadata!.custom).toBe('value');
});
it('should fail for non-existent object', async () => {
const command = new GetObjectCommand({
Bucket: 'test-bucket',
Key: 'does-not-exist.txt',
});
try {
await s3Client.send(command);
fail('Expected error to be thrown');
} catch (error: any) {
expect(error.name).toBe('NoSuchKey');
expect(error.Code).toBe('NoSuchKey');
expect(error.Key).toBe('does-not-exist.txt');
expect(error.$metadata?.httpStatusCode).toBe(404);
}
});
it('should fail for non-existent bucket', async () => {
const command = new GetObjectCommand({
Bucket: 'non-existent-bucket',
Key: 'test.txt',
});
await expect(s3Client.send(command)).rejects.toThrow();
});
});
describe('HeadObject', () => {
beforeEach(async () => {
await s3Client.send(new CreateBucketCommand({ Bucket: 'test-bucket' }));
});
it('should get object metadata without body', async () => {
await s3Client.send(
new PutObjectCommand({
Bucket: 'test-bucket',
Key: 'head.txt',
Body: Buffer.from('content'),
ContentType: 'text/plain',
}),
);
const command = new HeadObjectCommand({
Bucket: 'test-bucket',
Key: 'head.txt',
});
const response = await s3Client.send(command);
expect(response.ContentLength).toBe(7);
expect(response.ContentType).toBe('text/plain');
expect(response.ETag).toBeDefined();
expect(response.LastModified).toBeDefined();
});
it('should fail for non-existent object', async () => {
const command = new HeadObjectCommand({
Bucket: 'test-bucket',
Key: 'missing.txt',
});
await expect(s3Client.send(command)).rejects.toThrow();
});
});
describe('DeleteObject', () => {
beforeEach(async () => {
await s3Client.send(new CreateBucketCommand({ Bucket: 'test-bucket' }));
});
it('should delete an object', async () => {
await s3Client.send(
new PutObjectCommand({
Bucket: 'test-bucket',
Key: 'delete.txt',
Body: Buffer.from('delete me'),
}),
);
const command = new DeleteObjectCommand({
Bucket: 'test-bucket',
Key: 'delete.txt',
});
await s3Client.send(command);
// Verify deleted
const object = await prismaService.s3Object.findFirst({
where: { key: 'delete.txt' },
});
expect(object).toBeNull();
});
it('should succeed even if object does not exist', async () => {
const command = new DeleteObjectCommand({
Bucket: 'test-bucket',
Key: 'never-existed.txt',
});
await expect(s3Client.send(command)).resolves.toBeDefined();
});
it('should fail for non-existent bucket', async () => {
const command = new DeleteObjectCommand({
Bucket: 'non-existent-bucket',
Key: 'test.txt',
});
await expect(s3Client.send(command)).rejects.toThrow();
});
});
describe('ListObjects', () => {
beforeEach(async () => {
await s3Client.send(new CreateBucketCommand({ Bucket: 'test-bucket' }));
});
it('should list objects in a bucket using ListObjects v1', async () => {
await s3Client.send(
new PutObjectCommand({
Bucket: 'test-bucket',
Key: 'file1.txt',
Body: Buffer.from('data1'),
}),
);
await s3Client.send(
new PutObjectCommand({
Bucket: 'test-bucket',
Key: 'file2.txt',
Body: Buffer.from('data2'),
}),
);
const command = new ListObjectsCommand({
Bucket: 'test-bucket',
});
const response = await s3Client.send(command);
expect(response.Contents).toBeDefined();
expect(response.Contents!.length).toBe(2);
expect(response.Contents!.map(o => o.Key)).toContain('file1.txt');
expect(response.Contents!.map(o => o.Key)).toContain('file2.txt');
});
it('should filter by prefix using ListObjects v1', async () => {
await s3Client.send(
new PutObjectCommand({
Bucket: 'test-bucket',
Key: 'folder1/file1.txt',
Body: Buffer.from('data'),
}),
);
await s3Client.send(
new PutObjectCommand({
Bucket: 'test-bucket',
Key: 'folder2/file2.txt',
Body: Buffer.from('data'),
}),
);
const command = new ListObjectsCommand({
Bucket: 'test-bucket',
Prefix: 'folder1/',
});
const response = await s3Client.send(command);
expect(response.Contents!.length).toBe(1);
expect(response.Contents![0].Key).toBe('folder1/file1.txt');
});
});
describe('ListObjectsV2', () => {
beforeEach(async () => {
await s3Client.send(new CreateBucketCommand({ Bucket: 'test-bucket' }));
});
it('should list objects in a bucket', async () => {
await s3Client.send(
new PutObjectCommand({
Bucket: 'test-bucket',
Key: 'file1.txt',
Body: Buffer.from('data1'),
}),
);
await s3Client.send(
new PutObjectCommand({
Bucket: 'test-bucket',
Key: 'file2.txt',
Body: Buffer.from('data2'),
}),
);
const command = new ListObjectsV2Command({
Bucket: 'test-bucket',
});
const response = await s3Client.send(command);
expect(response.Contents).toBeDefined();
expect(response.Contents!.length).toBe(2);
expect(response.KeyCount).toBe(2);
expect(response.Contents!.map(o => o.Key)).toContain('file1.txt');
expect(response.Contents!.map(o => o.Key)).toContain('file2.txt');
});
it('should filter by prefix', async () => {
await s3Client.send(
new PutObjectCommand({
Bucket: 'test-bucket',
Key: 'folder1/file1.txt',
Body: Buffer.from('data'),
}),
);
await s3Client.send(
new PutObjectCommand({
Bucket: 'test-bucket',
Key: 'folder2/file2.txt',
Body: Buffer.from('data'),
}),
);
const command = new ListObjectsV2Command({
Bucket: 'test-bucket',
Prefix: 'folder1/',
});
const response = await s3Client.send(command);
expect(response.Contents!.length).toBe(1);
expect(response.Contents![0].Key).toBe('folder1/file1.txt');
});
it('should respect MaxKeys parameter', async () => {
for (let i = 0; i < 5; i++) {
await s3Client.send(
new PutObjectCommand({
Bucket: 'test-bucket',
Key: `file${i}.txt`,
Body: Buffer.from(`data${i}`),
}),
);
}
const command = new ListObjectsV2Command({
Bucket: 'test-bucket',
MaxKeys: 3,
});
const response = await s3Client.send(command);
expect(response.Contents!.length).toBe(3);
expect(response.IsTruncated).toBe(true);
});
it('should return empty list for empty bucket', async () => {
const command = new ListObjectsV2Command({
Bucket: 'test-bucket',
});
const response = await s3Client.send(command);
// Contents can be undefined or empty array when bucket is empty
expect(response.Contents || []).toEqual([]);
expect(response.KeyCount).toBe(0);
});
});
describe('End-to-End Workflow', () => {
it('should complete full S3 workflow', async () => {
// 1. Create bucket
await s3Client.send(new CreateBucketCommand({ Bucket: 'workflow-bucket' }));
// 2. List buckets
const listBucketsResponse = await s3Client.send(new ListBucketsCommand({}));
expect(listBucketsResponse.Buckets!.some(b => b.Name === 'workflow-bucket')).toBe(true);
// 3. Put objects
await s3Client.send(
new PutObjectCommand({
Bucket: 'workflow-bucket',
Key: 'doc1.txt',
Body: Buffer.from('Document 1'),
}),
);
await s3Client.send(
new PutObjectCommand({
Bucket: 'workflow-bucket',
Key: 'doc2.txt',
Body: Buffer.from('Document 2'),
}),
);
// 4. List objects
const listObjectsResponse = await s3Client.send(new ListObjectsV2Command({ Bucket: 'workflow-bucket' }));
expect(listObjectsResponse.KeyCount).toBe(2);
// 5. Get object
const getResponse = await s3Client.send(new GetObjectCommand({ Bucket: 'workflow-bucket', Key: 'doc1.txt' }));
const content = await getResponse.Body!.transformToString();
expect(content).toBe('Document 1');
// 6. Delete object
await s3Client.send(new DeleteObjectCommand({ Bucket: 'workflow-bucket', Key: 'doc1.txt' }));
// 7. Verify deletion
const listAfterDelete = await s3Client.send(new ListObjectsV2Command({ Bucket: 'workflow-bucket' }));
expect(listAfterDelete.KeyCount).toBe(1);
// 8. Delete remaining object
await s3Client.send(new DeleteObjectCommand({ Bucket: 'workflow-bucket', Key: 'doc2.txt' }));
// 9. Delete bucket
await s3Client.send(new DeleteBucketCommand({ Bucket: 'workflow-bucket' }));
// 10. Verify bucket deletion
await expect(s3Client.send(new HeadBucketCommand({ Bucket: 'workflow-bucket' }))).rejects.toThrow();
});
});
describe('PutBucketTagging', () => {
let bucketName: string;
beforeEach(async () => {
bucketName = `tagging-test-${Date.now()}`;
await s3Client.send(new CreateBucketCommand({ Bucket: bucketName }));
});
afterEach(async () => {
await s3Client.send(new DeleteBucketCommand({ Bucket: bucketName }));
});
it('should put bucket tags successfully', async () => {
const command = new PutBucketTaggingCommand({
Bucket: bucketName,
Tagging: {
TagSet: [
{ Key: 'Environment', Value: 'Test' },
{ Key: 'Owner', Value: 'TestUser' },
],
},
});
const response = await s3Client.send(command);
expect(response.$metadata.httpStatusCode).toBe(200);
});
it('should overwrite existing tags', async () => {
// Set initial tags
await s3Client.send(
new PutBucketTaggingCommand({
Bucket: bucketName,
Tagging: {
TagSet: [{ Key: 'Environment', Value: 'Dev' }],
},
}),
);
// Overwrite with new tags
await s3Client.send(
new PutBucketTaggingCommand({
Bucket: bucketName,
Tagging: {
TagSet: [
{ Key: 'Environment', Value: 'Prod' },
{ Key: 'Team', Value: 'Engineering' },
],
},
}),
);
// Verify new tags
const getResponse = await s3Client.send(new GetBucketTaggingCommand({ Bucket: bucketName }));
expect(getResponse.TagSet).toHaveLength(2);
expect(getResponse.TagSet).toContainEqual({ Key: 'Environment', Value: 'Prod' });
expect(getResponse.TagSet).toContainEqual({ Key: 'Team', Value: 'Engineering' });
});
it('should handle empty tag set', async () => {
const command = new PutBucketTaggingCommand({
Bucket: bucketName,
Tagging: {
TagSet: [],
},
});
const response = await s3Client.send(command);
expect(response.$metadata.httpStatusCode).toBe(200);
});
it('should fail for non-existent bucket', async () => {
const command = new PutBucketTaggingCommand({
Bucket: 'non-existent-bucket',
Tagging: {
TagSet: [{ Key: 'Test', Value: 'Value' }],
},
});
await expect(s3Client.send(command)).rejects.toThrow();
});
it('should handle tags with special characters', async () => {
await s3Client.send(
new PutBucketTaggingCommand({
Bucket: bucketName,
Tagging: {
TagSet: [
{ Key: 'app:name', Value: 'my-app' },
{ Key: 'cost-center', Value: 'eng_123' },
],
},
}),
);
const getResponse = await s3Client.send(new GetBucketTaggingCommand({ Bucket: bucketName }));
expect(getResponse.TagSet).toContainEqual({ Key: 'app:name', Value: 'my-app' });
expect(getResponse.TagSet).toContainEqual({ Key: 'cost-center', Value: 'eng_123' });
});
});
describe('GetBucketTagging', () => {
let bucketName: string;
beforeEach(async () => {
bucketName = `tagging-get-test-${Date.now()}`;
await s3Client.send(new CreateBucketCommand({ Bucket: bucketName }));
});
afterEach(async () => {
await s3Client.send(new DeleteBucketCommand({ Bucket: bucketName }));
});
it('should get bucket tags', async () => {
// Set tags
await s3Client.send(
new PutBucketTaggingCommand({
Bucket: bucketName,
Tagging: {
TagSet: [
{ Key: 'Project', Value: 'TestProject' },
{ Key: 'Stage', Value: 'Development' },
],
},
}),
);
// Get tags
const response = await s3Client.send(new GetBucketTaggingCommand({ Bucket: bucketName }));
expect(response.TagSet).toBeDefined();
expect(response.TagSet).toHaveLength(2);
expect(response.TagSet).toContainEqual({ Key: 'Project', Value: 'TestProject' });
expect(response.TagSet).toContainEqual({ Key: 'Stage', Value: 'Development' });
});
it('should return empty tags for bucket without tags', async () => {
const response = await s3Client.send(new GetBucketTaggingCommand({ Bucket: bucketName }));
expect(response.TagSet).toBeDefined();
expect(response.TagSet).toHaveLength(0);
});
it('should fail for non-existent bucket', async () => {
const command = new GetBucketTaggingCommand({
Bucket: 'non-existent-bucket',
});
await expect(s3Client.send(command)).rejects.toThrow();
});
it('should retrieve tags after multiple updates', async () => {
// First set of tags
await s3Client.send(
new PutBucketTaggingCommand({
Bucket: bucketName,
Tagging: {
TagSet: [{ Key: 'Version', Value: '1.0' }],
},
}),
);
// Second set of tags
await s3Client.send(
new PutBucketTaggingCommand({
Bucket: bucketName,
Tagging: {
TagSet: [
{ Key: 'Version', Value: '2.0' },
{ Key: 'Status', Value: 'Active' },
],
},
}),
);
// Get final tags
const response = await s3Client.send(new GetBucketTaggingCommand({ Bucket: bucketName }));
expect(response.TagSet).toHaveLength(2);
expect(response.TagSet).toContainEqual({ Key: 'Version', Value: '2.0' });
expect(response.TagSet).toContainEqual({ Key: 'Status', Value: 'Active' });
});
});
describe('GetBucketPolicy', () => {
let bucketName: string;
beforeEach(async () => {
bucketName = `policy-bucket-${Date.now()}`;
await s3Client.send(new CreateBucketCommand({ Bucket: bucketName }));
});
it('should throw NoSuchBucketPolicy for bucket without policy', async () => {
const command = new GetBucketPolicyCommand({ Bucket: bucketName });
await expect(s3Client.send(command)).rejects.toThrow();
});
it('should fail for non-existent bucket', async () => {
const command = new GetBucketPolicyCommand({
Bucket: 'non-existent-bucket',
});
await expect(s3Client.send(command)).rejects.toThrow();
});
});
describe('Bucket ACL', () => {
const bucketName = 'acl-test-bucket';
beforeEach(async () => {
await s3Client.send(new CreateBucketCommand({ Bucket: bucketName }));
});
afterEach(async () => {
try {
await s3Client.send(new DeleteBucketCommand({ Bucket: bucketName }));
} catch (e) {
// Ignore if bucket doesn't exist
}
});
it('should return default ACL for newly created bucket', async () => {
const response = await s3Client.send(new GetBucketAclCommand({ Bucket: bucketName }));
expect(response.Owner).toBeDefined();
expect(response.Owner?.ID).toBe('local-user');
expect(response.Owner?.DisplayName).toBe('local-user');
expect(response.Grants).toBeDefined();
expect(Array.isArray(response.Grants)).toBe(true);
});
it('should set and retrieve bucket ACL using canned ACL (private)', async () => {
await s3Client.send(
new PutBucketAclCommand({
Bucket: bucketName,
ACL: 'private',
}),
);
const response = await s3Client.send(new GetBucketAclCommand({ Bucket: bucketName }));
expect(response.Owner).toBeDefined();
expect(response.Owner?.ID).toBe('local-user');
expect(response.Grants).toBeDefined();
});
it('should set and retrieve bucket ACL using canned ACL (public-read)', async () => {
await s3Client.send(
new PutBucketAclCommand({
Bucket: bucketName,
ACL: 'public-read',
}),
);
const response = await s3Client.send(new GetBucketAclCommand({ Bucket: bucketName }));
expect(response.Owner).toBeDefined();
expect(response.Grants).toBeDefined();
expect(response.Grants!.length).toBeGreaterThan(0);
// Check for public read grant
const publicReadGrant = response.Grants!.find(
grant => grant.Grantee?.URI === 'http://acs.amazonaws.com/groups/global/AllUsers' && grant.Permission === 'READ',
);
expect(publicReadGrant).toBeDefined();
});
});
});

View File

@@ -0,0 +1,31 @@
import { Injectable } from '@nestjs/common';
import * as Joi from 'joi';
import { AbstractActionHandler, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import { RequestContext } from '../_context/request.context';
import { S3Service } from './s3.service';
type QueryParams = {
Bucket: string;
};
@Injectable()
export class CreateBucketHandler extends AbstractActionHandler<QueryParams> {
constructor(private readonly s3Service: S3Service) {
super();
}
format = Format.Xml;
action = Action.S3CreateBucket;
validator = Joi.object<QueryParams, true>({
Bucket: Joi.string().required(),
});
protected async handle({ Bucket }: QueryParams, { awsProperties }: RequestContext) {
const bucket = await this.s3Service.createBucket(Bucket);
return {
Location: bucket.location,
};
}
}

View File

@@ -0,0 +1,27 @@
import { Injectable } from '@nestjs/common';
import * as Joi from 'joi';
import { AbstractActionHandler, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import { RequestContext } from '../_context/request.context';
import { S3Service } from './s3.service';
type QueryParams = {
Bucket: string;
};
@Injectable()
export class DeleteBucketHandler extends AbstractActionHandler<QueryParams> {
constructor(private readonly s3Service: S3Service) {
super();
}
format = Format.Xml;
action = Action.S3DeleteBucket;
validator = Joi.object<QueryParams, true>({
Bucket: Joi.string().required(),
});
protected async handle({ Bucket }: QueryParams, { awsProperties }: RequestContext) {
await this.s3Service.deleteBucket(Bucket);
}
}

View File

@@ -0,0 +1,29 @@
import { Injectable } from '@nestjs/common';
import * as Joi from 'joi';
import { AbstractActionHandler, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import { RequestContext } from '../_context/request.context';
import { S3Service } from './s3.service';
type QueryParams = {
Bucket: string;
Key: string;
};
@Injectable()
export class DeleteObjectHandler extends AbstractActionHandler<QueryParams> {
constructor(private readonly s3Service: S3Service) {
super();
}
format = Format.Xml;
action = Action.S3DeleteObject;
validator = Joi.object<QueryParams, true>({
Bucket: Joi.string().required(),
Key: Joi.string().required(),
});
protected async handle({ Bucket, Key }: QueryParams, { awsProperties }: RequestContext) {
await this.s3Service.deleteObject(Bucket, Key);
}
}

View File

@@ -0,0 +1,34 @@
import { Injectable } from '@nestjs/common';
import * as Joi from 'joi';
import { AbstractActionHandler, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import { RequestContext } from '../_context/request.context';
import { S3Service } from './s3.service';
type QueryParams = {
Bucket: string;
};
@Injectable()
export class GetBucketAclHandler extends AbstractActionHandler<QueryParams> {
constructor(private readonly s3Service: S3Service) {
super();
}
format = Format.Xml;
action = Action.S3GetBucketAcl;
validator = Joi.object<QueryParams, true>({
Bucket: Joi.string().required(),
});
protected async handle({ Bucket }: QueryParams, { awsProperties }: RequestContext) {
const acl = await this.s3Service.getBucketAcl(Bucket);
return {
Owner: acl.Owner,
AccessControlList: {
Grant: acl.Grants.length > 0 ? acl.Grants : undefined,
},
};
}
}

View File

@@ -0,0 +1,31 @@
import { Injectable } from '@nestjs/common';
import * as Joi from 'joi';
import { AbstractActionHandler, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import { RequestContext } from '../_context/request.context';
import { S3Service } from './s3.service';
type QueryParams = {
Bucket: string;
};
@Injectable()
export class GetBucketPolicyHandler extends AbstractActionHandler<QueryParams> {
constructor(private readonly s3Service: S3Service) {
super();
}
format = Format.Json;
action = Action.S3GetBucketPolicy;
validator = Joi.object<QueryParams, true>({
Bucket: Joi.string().required(),
});
protected async handle({ Bucket }: QueryParams, { awsProperties }: RequestContext) {
const policy = await this.s3Service.getBucketPolicy(Bucket);
// AWS S3 GetBucketPolicy returns {Policy: policyDocument}
// where policyDocument is either a JSON string or null
return { Policy: policy };
}
}

View File

@@ -0,0 +1,37 @@
import { Injectable } from '@nestjs/common';
import * as Joi from 'joi';
import { AbstractActionHandler, Format } from '../abstract-action.handler';
import { Action } from '../action.enum';
import { RequestContext } from '../_context/request.context';
import { S3Service } from './s3.service';
type QueryParams = {
Bucket: string;
};
@Injectable()
export class GetBucketTaggingHandler extends AbstractActionHandler<QueryParams> {
constructor(private readonly s3Service: S3Service) {
super();
}
format = Format.Xml;
action = Action.S3GetBucketTagging;
validator = Joi.object<QueryParams, true>({
Bucket: Joi.string().required(),
});
protected async handle({ Bucket }: QueryParams, { awsProperties }: RequestContext) {
const tags = await this.s3Service.getBucketTagging(Bucket);
// Convert tags object to AWS S3 TagSet format
const tagArray = Object.entries(tags).map(([Key, Value]) => ({
Key,
Value,
}));
return {
TagSet: tagArray.length > 0 ? { Tag: tagArray } : {},
};
}
}

Some files were not shown because too many files have changed in this diff Show More