ComplyAI Service Architecture
Technical documentation for all microservices
Architecture Overview
ComplyAI is built on a microservices architecture with 12 independent services communicating via REST APIs and message queues.
┌─────────────────────────────────────┐
│ LOAD BALANCER │
│ (AWS ALB/NLB) │
└───────────────┬─────────────────────┘
│
┌───────────────────────────────┼───────────────────────────────┐
│ │ │
▼ ▼ ▼
┌───────────────┐ ┌───────────────┐ ┌───────────────┐
│ Frontend │ │ Core API │ │ Main API │
│ (React) │ │ (Flask) │ │ (Flask) │
│ │◀───────────▶│ │◀───────────▶│ │
│ User Portal │ │ Auth/Users │ │ Ad Data │
│ Dashboard │ │ Orgs/BMs │ │ Webhooks │
└─────────── ────┘ └───────────────┘ └───────────────┘
│ │
│ │
▼ ▼
┌───────────────┐ ┌───────────────┐
│ Maestro │ │ Violin │
│ (Flask) │ │ (Flask) │
│ │ │ │
│ ML Pipeline │◀───────────▶│ AI Services │
│ Webhooks │ │ Scoring │
└───────────────┘ └───────────────┘
│
▼
┌───────────────┐
│ Celery │
│ Workers │
│ │
│ Background │
│ Tasks │
└───────────────┘
Service Inventory
| Service | Technology | Port | Purpose | Status |
|---|---|---|---|---|
| complyai-frontend | React/Vite | 3000 | User interface | Production |
| complyai-core | Flask | 5000 | User/org management | Production |
| complyai-api | Flask | 5001 | Ad data processing | Production |
| complyai-maestro | Flask | 5002 | ML pipeline, webhooks | Production |
| complyai-violin | Flask | 5003 | AI scoring services | Production |
| complyai-triangle | Flask | 5004 | Notifications, rules | Production |
| complyai-gong | Flask | 5005 | External integrations | Production |
| complyai-ipu | Flask | 5006 | Image processing | Production |
| api-async | FastAPI | 8000 | Async operations | Production |
| complyai-cms | Django/Wagtail | 8001 | Content management | Production |
| www | React | 3001 | Marketing website | Production |
| complyai-adhoc | Python Scripts | - | Utilities/reports | Maintenance |
Detailed Service Documentation
1. complyai-core
Purpose: Central service for authentication, user management, organizations, and business manager connections.
Technology Stack:
- Python 3.11
- Flask 2.x
- Flask-Security-Too
- SQLAlchemy
- PostgreSQL
Key Modules:
complyai-core/
├── app/
│ ├── __init__.py # App factory
│ ├── models/
│ │ ├── core_models.py # User, Organization, etc.
│ │ ├── api_models.py # API-specific models
│ │ └── prompts.py # AI prompt templates
│ ├── views/
│ │ ├── auth.py # Authentication endpoints
│ │ ├── orgs.py # Organization management
│ │ ├── businessmanagers.py
│ │ ├── adaccounts.py
│ │ ├── ads.py
│ │ ├── user.py
│ │ └── notifications.py
│ ├── helpers/
│ │ ├── general.py # Utility functions
│ │ ├── db.py # Database helpers
│ │ └── validation.py # Input validation
│ └── extensions/
│ └── encryption.py # Token encryption
└── config.py # Configuration
Environment Variables:
| Variable | Description |
|---|---|
DATABASE_URL | PostgreSQL connection string |
SECRET_KEY | Flask secret key |
FERNET_KEY | Encryption key for tokens |
AUTH0_DOMAIN | Auth0 domain |
AUTH0_CLIENT_ID | Auth0 client ID |
API Routes:
| Route | Method | Description |
|---|---|---|
/auth/login | POST | User authentication |
/orgs/ | GET/POST | Organization CRUD |
/businessmanagers/ | GET/POST | BM management |
/adaccounts/ | GET | Ad account listing |
/ads/ | GET | Ad listing |
/user/ | GET/PUT | User profile |
2. complyai-api
Purpose: Ad data processing, Meta API synchronization, and internal service communication.
Technology Stack:
- Python 3.11
- Flask 2.x
- Celery
- Redis (broker)
- PostgreSQL
Key Modules:
complyai-api/
├── app/
│ ├── models/
│ │ ├── ad_account_models.py
│ │ ├── business_manager_models.py
│ │ ├── core_models.py
│ │ ├── precheck_models.py
│ │ └── prompts_models.py
│ ├── views/
│ │ ├── api.py # Internal API endpoints
│ │ ├── adaccounts.py
│ │ ├── insights.py
│ │ └── webhooks.py
│ ├── tasks/
│ │ ├── fetch_ad_account_data.py
│ │ ├── fetch_ad_creative_data.py
│ │ ├── generate_ai_score.py
│ │ └── webhook.py
│ └── helpers/
│ ├── fb_request.py # Meta API wrapper
│ └── ai.py # AI integration
└── worker.py # Celery worker entry
Celery Tasks:
| Task | Schedule | Description |
|---|---|---|
fetch_ad_accounts | Every 15 min | Sync ad accounts from Meta |
fetch_ad_data | Every 30 min | Sync ad creative data |
generate_scores | On webhook | Generate compliance scores |
sync_db | Daily | Database cleanup/sync |
3. complyai-maestro
Purpose: ML pipeline orchestration, webhook processing, and media analysis coordination.
Technology Stack:
- Python 3.11
- Flask 2.x
- Celery
- Redis
- GCP AI Platform (ML inference)
Key Modules:
complyai-maestro/
├── app/
│ ├── models/
│ │ └── webhook_data.py
│ ├── views/
│ │ ├── api.py
│ │ ├── webhook.py # Meta webhook receiver
│ │ └── results.py # ML results endpoint
│ └── tasks/
│ ├── ingest.py # Data ingestion
│ ├── image.py # Image processing
│ ├── text.py # Text analysis
│ ├── media.py # Media handling
│ └── webhook.py # Webhook processing
└── celery_app.py
Webhook Flow:
Meta Platform ──webhook──▶ /webhook/meta
│
▼
┌───────────────────┐
│ Verify signature │
│ Parse payload │
└───────────────────┘
│
▼
┌───────────────────┐
│ Queue Celery task │
│ (async processing)│
└───────────────────┘
│
▼
┌───────────────────┐
│ Process webhook │
│ Update status │
│ Send notification │
└───────────────────┘
4. complyai-frontend
Purpose: User-facing web application for ad management and compliance monitoring.
Technology Stack:
- React 18
- TypeScript
- Vite
- TailwindCSS
- React Query
Key Components:
complyai-frontend/
├── src/
│ ├── components/
│ │ ├── Dashboard/
│ │ ├── AdList/
│ │ ├── AdDetail/
│ │ ├── Settings/
│ │ └── common/
│ ├── hooks/
│ │ ├── useAuth.ts
│ │ ├── useAds.ts
│ │ └── useOrganization.ts
│ ├── services/
│ │ └── api.ts # API client
│ ├── store/
│ │ └── authStore.ts
│ └── pages/
│ ├── Login.tsx
│ ├── Dashboard.tsx
│ ├── Ads.tsx
│ └── Settings.tsx
└── vite.config.ts
Inter-Service Communication
API-to-API Calls
Services communicate via HTTP REST APIs with bearer token authentication.
# Example: Core API calling Main API
import requests
def get_ad_account_tokens(ad_account_ids):
response = requests.post(
f"{API_BASE_URL}/bm_tokens_from_adaccounts",
headers={
"Authorization": f"Bearer {API_TOKEN}",
"Content-Type": "application/json"
},
json={"ad_account_ids": ad_account_ids}
)
return response.json()
Service Dependencies
┌─────────────────────────────────────────────────────────────┐
│ SERVICE DEPENDENCIES │
├─────────────────────────────────────────────────────────────┤
│ │
│ Frontend ──▶ Core API ──▶ PostgreSQL │
│ │ │
│ └──▶ Main API ──▶ Core API │
│ │ │
│ └──▶ Meta Graph API │
│ │
│ Maestro ──▶ Core API │
│ │ │
│ ├──▶ Main API │
│ │ │
│ ├──▶ Violin (AI Services) │
│ │ │
│ └──▶ Redis (Celery broker) │
│ │
│ All Services ──▶ PostgreSQL │
│ │
└───────────────────────── ────────────────────────────────────┘
Database Architecture
Primary Database (PostgreSQL)
Connection:
- Host: AWS RDS
- Port: 5432
- SSL: Required
Connection Pooling:
- PgBouncer for connection management
- Pool size: 20 per service
- Overflow: 10
Schema Organization:
-- Core tables (complyai-core)
users
organizations
user_organizations
roles
roles_users
org_business_accounts
org_ad_accounts
org_ads
org_ads_score
activity_events
notifications
issues
invite_codes
-- API tables (complyai-api)
ad_accounts
ad_accounts_total
ad_account_webhook_subscriptions
business_manager_ad_account
merged_ad_accounts
-- Status tables
facebook_ad_status
facebook_total_ad_data_feedback
ad_account_case
Deployment Architecture
Container Orchestration
All services run as Docker containers on AWS ECS Fargate.
# Example task definition
{
"family": "complyai-core",
"containerDefinitions": [
{
"name": "core-api",
"image": "complyai/core:latest",
"memory": 512,
"cpu": 256,
"portMappings": [
{"containerPort": 5000}
],
"environment": [
{"name": "FLASK_ENV", "value": "production"}
],
"secrets": [
{"name": "DATABASE_URL", "valueFrom": "arn:aws:ssm:..."}
]
}
]
}
CI/CD Pipeline
┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ GitHub │───▶│ GitHub │───▶│ AWS │───▶│ AWS │
│ Push │ │ Actions │ │ ECR │ │ ECS │
│ │ │ │ │ │ │ │
│ - PR │ │ - Test │ │ - Push │ │ - Deploy │
│ - Merge │ │ - Build │ │ Image │ │ - Scale │
└─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘
Monitoring & Observability
Logging
All services use structured JSON logging:
{
"timestamp": "2024-12-25T10:00:00Z",
"level": "INFO",
"service": "complyai-core",
"message": "User logged in",
"user_id": 42,
"ip_address": "192.168.1.1",
"request_id": "req-abc123"
}
Metrics
Key metrics tracked:
- Request latency (p50, p95, p99)
- Error rate
- Webhook processing time
- ML inference latency
- Database query time
Alerting
| Alert | Threshold | Channel |
|---|---|---|
| High error rate | >1% | Slack #alerts |
| Slow response | p95 > 2s | Slack #alerts |
| Database connections | >80% pool | Slack #alerts |
| Celery queue depth | >100 | Slack #alerts |
Security
Authentication Flow
User ──▶ Auth0 ──▶ JWT Token ──▶ Core API ──▶ Session
│
▼
Internal API Token
│
▼
Other Services
Secrets Management
- All secrets stored in AWS Secrets Manager or SSM Parameter Store
- Rotated every 90 days
- Access logged and audited
Network Security
- All services in private VPC subnets
- Public access only through ALB
- Security groups restrict inter-service traffic
- TLS 1.2+ required for all connections
📝 Changelog
| Date | Change |
|---|---|
| 2024-12 | Initial architecture documentation |