cloud-server/CACHE_KV_IMPLEMENTATION.md

# KV-Based Cache Index Implementation

## Overview

Implemented KV-based cache index for `CacheService` to enable pattern-based cache invalidation and enumeration. This resolves the TODO at line 274 in `src/services/cache.ts`.

## Changes Summary

### 1. Core Implementation (`src/services/cache.ts`)

**Constructor Update:**
- Added optional `kvNamespace: KVNamespace | null` parameter
- Defaults to `null` for backward compatibility
- Logs KV index availability on initialization

**New Features:**
- `invalidatePattern(pattern: string): Promise<number>` - Pattern-based cache invalidation with wildcard support
- `clearAll(prefix?: string): Promise<number>` - Actually clears and counts entries when KV is available
- `getStats(): Promise<{ supported: boolean; indexed_keys?: number }>` - Returns actual key count when KV is available

**Private Helper Methods:**
- `_registerCacheKey(key: string, ttlSeconds: number): Promise<void>` - Registers cache keys in KV index
- `_unregisterCacheKey(key: string): Promise<void>` - Unregisters cache keys from KV index
- `_listCacheKeys(prefix?: string): Promise<string[]>` - Lists all cache keys from KV index

**Key Design Decisions:**
- **KV Index Prefix**: `cache_index:` to separate from rate limiting data
- **Auto-Expiration**: KV entries use same TTL as cache entries (automatic cleanup)
- **Graceful Degradation**: All KV operations are non-blocking, failures are logged but don't break cache operations
- **Backward Compatibility**: Works with or without KV namespace, existing code unchanged

### 2. Route Updates

**`src/routes/instances.ts`:**
- Updated `getCacheService()` to accept `env: Env` parameter
- Passes `env.RATE_LIMIT_KV` to CacheService constructor
- Enables KV index for all instance cache operations

**`src/routes/recommend.ts`:**
- Updated `getCacheService()` to accept `env: Env` parameter
- Passes `env.RATE_LIMIT_KV` to CacheService constructor
- Enables KV index for all recommendation cache operations

### 3. Test Coverage (`src/services/cache-kv.test.ts`)

**24 comprehensive tests:**
- Cache key registration/unregistration
- Pattern invalidation with wildcards
- clearAll() with and without KV
- Error handling and graceful degradation
- Backward compatibility (no KV)
- Large-scale operations (150 keys)

**Test Results:**
```
✓ 24 tests passed (24)
Duration: 26ms
```

## API Changes

### Pattern Invalidation

**Before:**
```typescript
async invalidatePattern(pattern: string): Promise<void> {
  logger.warn(`Pattern invalidation not supported: ${pattern}`);
  // TODO: Implement with KV-based cache index if needed
}
```

**After:**
```typescript
async invalidatePattern(pattern: string): Promise<number> {
  // Returns count of invalidated entries
  // Supports wildcards: *, case-insensitive
  // Examples:
  //   '*instances*' - All instance caches
  //   '*provider=linode*' - All linode caches
  //   '*pricing*' - All pricing caches
}
```

### clearAll() Enhancement

**Before:**
```typescript
async clearAll(prefix?: string): Promise<number> {
  logger.info('Cache clearAll requested', { prefix });
  return 0; // Always returns 0
}
```

**After:**
```typescript
async clearAll(prefix?: string): Promise<number> {
  // Returns actual count of cleared entries when KV is available
  // Returns 0 when KV is not available (backward compatible)
}
```

### getStats() Enhancement

**Before:**
```typescript
async getStats(): Promise<{ supported: boolean }> {
  return { supported: false };
}
```

**After:**
```typescript
async getStats(): Promise<{ supported: boolean; indexed_keys?: number }> {
  // Returns indexed_keys count when KV is available
  // Returns { supported: false } when KV is not available
}
```

## Usage Examples

### With KV Index (Production)

```typescript
import { Env } from './types';
import { CacheService } from './services/cache';
import { CACHE_TTL } from './constants';

// Initialize with KV namespace for full functionality
const cache = new CacheService(CACHE_TTL.INSTANCES, env.RATE_LIMIT_KV);

// Set cache entries
await cache.set('https://cache.internal/instances?provider=linode', instancesData);
await cache.set('https://cache.internal/instances?provider=vultr', vultrData);
await cache.set('https://cache.internal/pricing?provider=linode', pricingData);

// Pattern invalidation (wildcard support)
const count = await cache.invalidatePattern('*instances*');
console.log(`Invalidated ${count} instance cache entries`);

// Clear all caches
const cleared = await cache.clearAll();
console.log(`Cleared ${cleared} total cache entries`);

// Clear with prefix filter
const clearedInstances = await cache.clearAll('https://cache.internal/instances');
console.log(`Cleared ${clearedInstances} instance cache entries`);

// Get statistics
const stats = await cache.getStats();
console.log(`Cache has ${stats.indexed_keys} indexed keys`);
```

### Without KV Index (Development/Backward Compatible)

```typescript
// Initialize without KV namespace (backward compatible)
const cache = new CacheService(CACHE_TTL.INSTANCES);

// Basic cache operations work normally
await cache.set('key', data);
const result = await cache.get<MyType>('key');

// Pattern invalidation returns 0 (logs warning)
const invalidated = await cache.invalidatePattern('*pattern*');
// → Returns 0, logs warning

// clearAll returns 0 (logs info)
const cleared = await cache.clearAll();
// → Returns 0, logs info

// Stats indicate not supported
const stats = await cache.getStats();
// → { supported: false }
```

## KV Index Structure

**Index Key Format:**
```
cache_index:{original_cache_key}
```

**Example:**
```
cache_index:https://cache.internal/instances?provider=linode
```

**KV Entry Metadata:**
```typescript
{
  cached_at: "2026-01-25T15:00:00.000Z",
  ttl: 300
}
```

**Auto-Cleanup:**
- KV entries use same TTL as cache entries
- KV automatically deletes expired entries
- No manual cleanup required

## Pattern Matching

**Wildcard Support:**
- `*` matches any characters
- Case-insensitive matching
- Regex special characters are properly escaped

**Examples:**
```typescript
// Match all instance caches
invalidatePattern('*instances*');

// Match specific provider
invalidatePattern('*provider=linode*');

// Match specific endpoint
invalidatePattern('https://cache.internal/pricing*');

// Match exact pattern
invalidatePattern('*instances?provider=vultr*');
```

## Performance Characteristics

**With KV Index:**
- `set()`: +1 KV write (async, non-blocking)
- `delete()`: +1 KV delete (async, non-blocking)
- `clearAll()`: 1 KV list + N cache deletes + N KV deletes
- `invalidatePattern()`: 1 KV list + N cache deletes + N KV deletes
- `getStats()`: 1 KV list operation

**Without KV Index:**
- All operations identical to original implementation
- No performance overhead

**Graceful Degradation:**
- KV failures don't break cache operations
- Errors are logged but not thrown
- Cache operations continue normally

## Migration Guide

### For Existing Code (No Changes Required)

Existing code continues to work without changes:

```typescript
// This still works (no KV index)
const cache = new CacheService(CACHE_TTL.INSTANCES);
```

### For New Features (Enable KV Index)

To enable pattern invalidation and enumeration:

```typescript
// Pass KV namespace as second parameter
const cache = new CacheService(CACHE_TTL.INSTANCES, env.RATE_LIMIT_KV);
```

### Production Deployment

**Option 1: Reuse RATE_LIMIT_KV (Current Implementation)**
- No configuration changes required
- Cache index data stored in same KV as rate limiting
- Prefix separation prevents conflicts (`cache_index:` vs `ratelimit:`)

**Option 2: Dedicated CACHE_INDEX_KV (Future Enhancement)**
- Add new KV namespace in wrangler.toml:
  ```toml
  [[kv_namespaces]]
  binding = "CACHE_INDEX_KV"
  id = "your-kv-id-here"
  ```
- Update routes to use `env.CACHE_INDEX_KV` instead of `env.RATE_LIMIT_KV`

## Testing

**Run all cache tests:**
```bash
npm run test -- cache-kv.test.ts
```

**Coverage:**
- 24 unit tests
- All edge cases covered
- Error handling validated
- Backward compatibility verified

## Security Considerations

- KV index only stores cache keys (not data)
- Same security as rate limiting KV
- No sensitive data exposure
- Automatic cleanup via TTL expiration

## Benefits

1. **Pattern Invalidation**: Invalidate multiple cache entries at once
2. **Accurate Counts**: `clearAll()` now returns actual count
3. **Cache Statistics**: Monitor cache usage and size
4. **Backward Compatible**: Works with or without KV
5. **Automatic Cleanup**: KV TTL keeps index clean
6. **Graceful Degradation**: KV failures don't break cache operations

## Known Limitations

- KV eventual consistency: Recent writes may not appear immediately in list operations
- Pattern matching is done in-memory after KV list (not server-side)
- Large cache sizes (>1000 keys) may require pagination handling
- KV list operations have ~100ms latency