ComplyAI Master Data Project
Your single source of truth for all data across the ComplyAI organization
🎯 Purpose
This Master Data Project provides complete transparency and understanding of all data assets across ComplyAI. It serves both technical and non-technical stakeholders with comprehensive documentation of our data ecosystem.
📚 Documentation Index
| Section | Description | Audience |
|---|---|---|
| Data Dictionary | Complete definitions of all data entities, fields, and relationships | All Teams |
| Data Lineage | Visual flows showing how data moves through our systems | Engineering, Analytics |
| API Documentation | OpenAPI/Swagger specs for all internal and external APIs | Engineering, Partners |
| Entity Relationships | ERD diagrams showing data model connections | Engineering, Product |
| Service Architecture | Microservice documentation and inter-service communication | Engineering |
| Third-Party Integrations | Meta, Stripe, and other platform integrations | All Teams |
| Glossary | Business and technical terminology definitions | All Teams |
| Data Governance | Policies, ownership, and compliance frameworks | Leadership, Compliance |
| Quick Reference | Cheat sheets and common lookup tables | All Teams |
🏗️ System Overview
ComplyAI Platform Services
┌─────────────────────────────────────────────────────────────────────────┐
│ COMPLYAI PLATFORM │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Frontend │ │ Core API │ │ Main API │ │ Maestro │ │
│ │ (React) │ │ (Flask) │ │ (Flask) │ │ (Flask) │ │
│ │ │ │ │ │ │ │ │ │
│ │ User Portal │ │ Auth/Users │ │ Ad Processing│ │ ML Inference │ │
│ │ Dashboard │ │ Organizations│ │ Webhooks │ │ Media Process│ │
│ │ Ad Mgmt │ │ Business Mgmt│ │ Ad Accounts │ │ Results │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ └──────────────┘ │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Violin │ │ Triangle │ │ Gong │ │ IPU │ │
│ │ (Flask) │ │ (Flask) │ │ (Flask) │ │ (Flask) │ │
│ │ │ │ │ │ │ │ │ │
│ │ AI Services │ │ Notifications│ │ Integrations │ │ Image Process│ │
│ │ Scoring │ │ Ad Rules │ │ External API │ │ Unit Testing │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ └──────────────┘ │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Async API │ │ CMS │ │ WWW │ │ Adhoc │ │
│ │ (FastAPI) │ │ (Django) │ │ (React) │ │ (Scripts) │ │
│ │ │ │ │ │ │ │ │ │
│ │ Async Tasks │ │ Content Mgmt │ │ Public Site │ │ Utilities │ │
│ │ Background │ │ Wagtail │ │ Marketing │ │ Reports │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ └──────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────── ──────┘
│
┌───────────────┴───────────────┐
▼ ▼
┌──────────────┐ ┌──────────────┐
│ PostgreSQL │ │ External │
│ Database │ │ APIs │
│ │ │ │
│ Users │ │ Meta Graph │
│ Organizations│ │ Stripe │
│ Ad Accounts │ │ Slack │
│ Ads/Reviews │ │ AWS/GCP │
└──────────────┘ └──────────── ──┘
🗄️ Data Domains
1. Customer Domain
- Organizations - Client companies using ComplyAI
- Users - Individual users within organizations
- Subscriptions - Billing and plan information
- Roles/Permissions - Access control
2. Ad Content Domain
- Ad Accounts - Facebook/Meta ad accounts
- Business Managers - Meta business manager connections
- Ads - Individual ad content and metadata
- Media Assets - Images, videos, creative content
3. Compliance Domain
- Reviews - AI compliance review results
- Violations - Policy violation detections
- Policies - Platform policy rules
- Recommendations - AI-generated fix suggestions
4. Operational Domain
- Activity Events - User action audit logs
- Notifications - System alerts and messages
- Webhook Subscriptions - Event subscription tracking
- Issues - Support tickets and feedback
🔑 Key Concepts
For Non-Technical Stakeholders
| Concept | What It Means |
|---|---|
| Ad Account | A Facebook advertising account where ads are created and run |
| Business Manager | A Meta tool that organizes ad accounts, pages, and team members |
| Organization | A ComplyAI customer (company) with one or more users |
| Review | The AI analysis of an ad for policy compliance |
| Webhook | Real-time notifications when something changes (like ad status) |
For Technical Stakeholders
| Concept | Technical Definition |
|---|---|
| System User Token | Long-lived OAuth token for server-to-server Meta API calls |
| OBO (On-Behalf-Of) | Token that allows acting on behalf of a business manager |
| Celery Task | Background job for async processing (ad fetching, scoring) |
| RSID | Run-specific ID for Word document tracked changes |
📊 Quick Stats
| Metric | Count |
|---|---|
| Microservices | 12 |
| Database Tables | 40+ |
| API Endpoints | 150+ |
| External Integrations | 5 |
🚀 Getting Started
For Product/Business Teams
- Start with the Glossary to understand terminology
- Review Data Dictionary for entity definitions
- Check Quick Reference for common lookups
For Engineering Teams
- Review Service Architecture for system design
- Explore API Documentation for endpoint specs
- Study Data Lineage for data flow understanding
- Reference Entity Relationships for data modeling
For Analytics/BI Teams
- Start with Data Dictionary for field definitions
- Review Data Lineage for data pipeline understanding
- Check Data Governance for data quality standards
📝 Contributing
To update this documentation:
- Create a branch from
main - Make your changes in the appropriate section
- Update the changelog in that section's README
- Submit a PR for review
📅 Last Updated
December 2024 | Version 1.0.0
👥 Maintainers
- Data Team - Overall documentation ownership
- Engineering - Technical accuracy review
- Product - Business context validation