# Plan: Zero-Touch WhatsApp Integration & Self-Service Onboarding

## Context

**Problem**: Onboarding a new business on Mawidi requires significant manual work from Asim - WhatsApp connection, KB setup, workflow configuration. Two native conversation flows already exist (real estate viewings + generic appointments) but they're separate implementations, not unified into a single engine that works for all sectors.

**Goal**: Business signs up → selects industry → connects WhatsApp → system is fully operational immediately. Handles customer messages, books appointments, sends quotations, collects reviews, answers from KB, and learns from every conversation. Zero manual intervention.

**Key Decision**: The real estate flow at `/api/webhooks/twilio-whatsapp-re/route.ts` (621 lines) and the generic appointment engine at `whatsapp-ai-engine.service.ts` (1156 lines) are the two proven native workflows. We unify them into a **single ConversationEngine** with sector-specific configs so every sector works out of the box.

**Key Finding**: The webhook logic between RE and generic flows is **95% identical**. The only differences are: which AI engine is called, what system prompt is used, and what domain services (search/create/cancel) are invoked. This means we need a thin sector registry, not 12 separate implementations.

---

## Architecture

```
                    ┌─────────────────────────────────┐
                    │  SIGNUP FLOW (existing)          │
                    │  1. Clerk auth                   │
                    │  2. Industry selection            │
                    │  3. Stripe payment               │
                    └─────────┬───────────────────────┘
                              ↓
                    ┌─────────────────────────────────┐
                    │  AUTO-PROVISIONING (new)         │
                    │  1. Create organization           │
                    │  2. Seed sector KB (FAQs)        │
                    │  3. Seed appointment types        │
                    │  4. Configure AI agent defaults   │
                    └─────────┬───────────────────────┘
                              ↓
                    ┌─────────────────────────────────┐
                    │  WHATSAPP CONNECT                │
                    │  Meta Embedded Signup (OAuth)    │
                    │  → Auto webhook registration     │
                    │  → Credentials encrypted & stored│
                    └─────────┬───────────────────────┘
                              ↓
                    ┌─────────────────────────────────┐
                    │  UNIFIED CONVERSATION ENGINE     │
                    │                                   │
                    │  Webhook → Org Lookup → Sector   │
                    │  Config → AI Engine → Actions    │
                    │  → Store → Learn → Respond       │
                    └─────────────────────────────────┘
```

---

## Phase 1: Unified Conversation Engine (Core)

**Goal**: Unify the existing RE + appointment workflows into a single engine that handles all 12 sectors via a sector config registry.

### 1.1 Sector Registry
**New file**: `lib/config/sector-registry.ts`

Define a `SectorConfig` interface that each sector implements:
```typescript
interface SectorConfig {
  key: string;                    // 'healthcare', 'beauty_wellness', etc.
  transactionNoun: { en: string; ar: string };  // 'appointment', 'reservation'
  systemPromptTemplate: string;   // Sector-specific AI instructions
  requiredFields: FieldDef[];     // What to collect (name, date, time, service, etc.)
  optionalFields: FieldDef[];     // Sector extras (doctor, stylist, party_size, etc.)
  catalogService: CatalogAdapter; // How to search/list services or products
  bookingService: BookingAdapter; // How to create/amend/cancel bookings
  paymentConfig: PaymentConfig;   // Deposit required? Amount? Auto-send link?
  postBookingActions: Action[];   // Send confirmation, request review, etc.
}
```

Source content from existing `sector-ai-context.ts` (12 sectors already defined with vocabulary, hints, example dialogues) and `industry-appointment-types.ts`.

### 1.2 Unified Webhook Handler
**Modify**: `/app/api/integrations/whatsapp/webhook/route.ts`

Unify the patterns from `twilio-whatsapp-re/route.ts` and the main webhook into one handler:
1. Receive message (Meta Cloud API format - already handled)
2. Look up org by `phone_number_id` (new: `whatsapp-router.service.ts`)
3. Load sector config from registry based on org's industry
4. Rate limit, signature verify, sanitize (already exists)
5. Language detect, conversation upsert (already exists)
6. Call `ConversationEngine.processMessage(context, message, sectorConfig)` — routes to the right sector logic (RE uses property search + viewings, appointments use service catalog + bookings, etc.)
7. Store messages, send response (already exists)

### 1.3 Generalized ConversationEngine
**Modify**: `lib/services/whatsapp-ai-engine.service.ts`

The existing engine already has sector context injection via `getSectorAIContext()`. Enhance it:
- Accept `SectorConfig` to dynamically load catalog (services/properties/slots)
- Use sector's `requiredFields` to drive the collection state machine
- Use sector's `bookingService` adapter for create/amend/cancel
- Use sector's `paymentConfig` to determine when to send payment links
- Use sector's `postBookingActions` for review requests, confirmations

The state machine stays generic:
```
selecting_language → idle → browsing_catalog → collecting_info 
→ awaiting_payment → completed
```

### 1.4 Multi-Tenant Message Router
**New file**: `lib/services/whatsapp-router.service.ts`

- Map incoming `phone_number_id` → organization
- Redis cache for fast lookup
- Handles the case where multiple orgs are on the system

### 1.5 Sector Service Adapters
For each sector that has unique domain services, create thin adapters:

- **Healthcare**: Wraps `AppointmentTypeService` - doctor selection, department routing
- **Beauty**: Wraps `AppointmentTypeService` - stylist preference, service duration
- **Food**: Party size, dietary needs, table preference
- **Real Estate**: Already exists - `ViewingService`, `PropertyService`, `BrokerAssignment`
- **Default**: Generic `AppointmentTypeService` (works for most sectors)

Most sectors can use the generic `AppointmentTypeService` + `BookingService` directly. Only RE and a few others need custom adapters.

**Files to modify:**
- `lib/services/whatsapp-ai-engine.service.ts` - generalize sector handling, accept SectorConfig
- `app/api/integrations/whatsapp/webhook/route.ts` - add sector-aware routing via registry

**New files:**
- `lib/config/sector-registry.ts` - sector config registry
- `lib/services/whatsapp-router.service.ts` - multi-tenant routing

**Existing files to reuse:**
- `lib/config/sector-ai-context.ts` - 12 sectors already configured
- `lib/config/industry-sector-map.ts` - industry→sector mapping
- `lib/config/industry-appointment-types.ts` - appointment type definitions
- `lib/services/appointment-type.service.ts` - generic catalog service
- `lib/services/reviews.service.ts` - review collection (already built)
- `lib/services/quotation-notifier.service.ts` - quotation sending (already built)
- `lib/stripe/checkout.ts` - payment link generation (already built)

---

## Phase 2: Auto-Provisioning Pipeline

**Goal**: When business completes signup + payment, everything is ready automatically.

### 2.1 Auto-Create Organization on Signup Success
**Modify**: `app/api/signup/success/route.ts`

After Stripe subscription is created:
1. Create org via Convex `createOrganization` with industry + sector
2. Link user as OWNER
3. Trigger seeding pipeline (KB + appointment types + AI config)

Currently org creation only happens for tier3+ when they hit `/api/organization` - change to happen for ALL tiers on signup success.

### 2.2 Seed Industry-Specific Knowledge Base
**New file**: `lib/config/sector-default-faqs.ts`

Per-sector FAQ content (bilingual EN/AR), 10-15 entries each. Source from:
- `sector-ai-context.ts` example dialogues → convert to Q&A format
- Common questions every business in that sector gets asked

**New file**: `lib/services/kb-seeder.service.ts`

Called after org creation:
1. Load sector FAQs from `sector-default-faqs.ts`
2. Insert into `company_knowledge_base` table for the new org
3. Mark as `isActive: true`, `category: 'default'`

### 2.3 Seed Appointment Types
**Existing**: `lib/services/appointment-type-seeder.ts` has `seedIndustryDefaults()`
- Wire it into org creation flow (currently exists but NOT called)

### 2.4 Configure AI Agent Defaults
Store in org settings:
- Default greeting message per sector (bilingual)
- AI model preference
- Business hours (placeholder - owner can update)
- Deposit requirements per sector default

**Files to modify:**
- `app/api/signup/success/route.ts` - trigger auto-provisioning
- `convex/organizations/mutations.ts` - add seeding call

**New files:**
- `lib/config/sector-default-faqs.ts`
- `lib/services/kb-seeder.service.ts`

---

## Phase 3: WhatsApp Connection (Non-Technical UX)

**Goal**: Non-technical business owners can connect WhatsApp without understanding Meta APIs.

### 3.1 Meta Embedded Signup Flow
**Modify**: `components/onboarding/steps/WhatsAppSetupStep.tsx`

Use Meta's **Embedded Signup** (Facebook Login for Business):
1. Business owner clicks "Connect WhatsApp"
2. Opens Meta OAuth popup (Facebook Login)
3. They select/create their WhatsApp Business account
4. Meta returns `phone_number_id`, `waba_id`, `access_token` via callback
5. We store credentials encrypted, register webhook automatically

This requires:
- Meta App configured for Embedded Signup
- Facebook SDK integration on frontend
- OAuth callback handler on backend

### 3.2 Webhook Auto-Registration
**Modify**: `app/api/integrations/whatsapp/connect/route.ts`

After receiving OAuth credentials:
1. Call Meta API to register our webhook URL for this WABA
2. Subscribe to `messages` and `message_status` events
3. Verify webhook with challenge-response (already exists)
4. Send test message to verify connectivity
5. Mark WhatsApp as connected + webhook verified

### 3.3 Fallback: Manual Credentials
Keep existing manual input as fallback for technical users who already have Meta API credentials.

**Files to modify:**
- `components/onboarding/steps/WhatsAppSetupStep.tsx` - embedded signup UI
- `app/api/integrations/whatsapp/connect/route.ts` - OAuth handling + auto webhook

**New files:**
- `app/api/integrations/whatsapp/oauth/callback/route.ts` - Meta OAuth callback

---

## Phase 4: Full Booking Capabilities

**Goal**: AI handles complete end-to-end: booking + payment + quotations + reviews.

### 4.1 Booking with Payment
Already exists in AI engine (`createCheckoutSessionForOrganization`). Ensure:
- **Stripe key check**: Before sending a payment link, verify the org has configured their Stripe key in the dashboard. If no Stripe key is present, skip the payment step — book without deposit and notify the business owner that payment collection requires Stripe setup
- Payment link sent via WhatsApp after booking info collected (only if Stripe key exists)
- Stripe webhook updates booking status on payment
- Confirmation message sent after payment
- AI gracefully handles the no-Stripe case: "Your booking is confirmed. The business will collect payment directly."

### 4.2 Quotation Sending
**Existing**: `lib/services/quotation-notifier.service.ts`
- Add as action in AI engine: when customer asks for a quote, generate and send via WhatsApp
- AI collects requirements → creates quotation → sends PDF + payment link (payment link only if org has Stripe key configured)

### 4.3 Review Collection
**Existing**: `lib/services/reviews.service.ts`
- Add as `postBookingAction`: after appointment is completed, send review request via WhatsApp
- Customer can rate + leave feedback directly in WhatsApp conversation
- Reviews stored and visible in dashboard

### 4.4 Sector-Specific Actions
Each sector's `postBookingActions` array defines what happens after booking:
- Healthcare: send appointment reminder, pre-visit instructions
- Beauty: send service prep instructions, post-service review request
- Food: send reservation confirmation, dietary info request
- Real Estate: send viewing confirmation, property documents, broker intro

**Files to modify:**
- `lib/services/whatsapp-ai-engine.service.ts` - add quotation + review actions
- Sector configs in `lib/config/sector-registry.ts` - define postBookingActions

---

## Phase 5: Knowledge Base & Learning Loop

**Goal**: System gets smarter over time by learning from conversations.

### 5.1 Enhanced Conversation Storage
**Modify**: Convex `org_whatsapp_messages` mutations

Add metadata per message:
- `kbUsed: boolean` - was KB queried for this response?
- `kbConfidence: number` - match quality (0-1)
- `questionCategory: string` - auto-classified topic
- `escalatedToHuman: boolean`

### 5.2 Unanswered Question Tracker
**New file**: `lib/services/conversation-learning.service.ts`

After each conversation turn:
- If KB confidence < threshold or AI had to make up an answer → log to `unanswered_questions`
- Group similar questions (text similarity)
- Track frequency per org and per sector

**New Convex table**: `unanswered_questions`
- `organizationId`, `question`, `frequency`, `firstAskedAt`, `lastAskedAt`, `sector`

### 5.3 FAQ Auto-Suggestion
**New Convex table**: `suggested_faqs`
- `organizationId`, `question`, `suggestedAnswer`, `frequency`, `status` (pending/approved/rejected)

Periodic process (or triggered after N unanswered questions):
- Use AI to generate suggested FAQ entries from unanswered patterns
- Surface in dashboard for business owner to approve with one click

### 5.4 Cross-Sector Learning
**New Convex table**: `sector_common_questions`
- Aggregate anonymized question patterns across all orgs in a sector
- Use to improve sector default FAQ seeding for new businesses
- Privacy: only question patterns, never customer data or PII

### 5.5 Dashboard UI
**New component**: `components/dashboard/SuggestedFAQs.tsx`
- Shows: "Your AI couldn't answer these questions X times this week"
- One-click approve to add to KB
- KB coverage score (% of questions answered from KB)

---

## Security

### Message & Webhook Security
- HMAC-SHA256 signature verification on webhooks — already enforced, must stay for all sectors
- AES-256-GCM encryption for stored WhatsApp access tokens — already in place
- Rate limiting (100 req/min per IP) on webhook endpoints — already exists
- **Org isolation in multi-tenant router**: `whatsapp-router.service.ts` must guarantee messages can't leak between orgs. Routing keyed strictly by `phone_number_id` with validation that the resolved org owns that number

### Meta Embedded Signup / OAuth
- OAuth tokens: exchange short-lived tokens for long-lived tokens **server-side only**, never expose to client
- CSRF protection on OAuth callback via `state` parameter validation
- Automated token refresh — Meta tokens expire, need rotation logic (cron or on-demand refresh before expiry)
- Scope minimization — only request `whatsapp_business_messaging` and `whatsapp_business_management`, nothing more

### AI Engine Security
- Existing guardrails: prompt injection detection, abuse detection, PII leak prevention — keep enforced
- **KB query org-scoping**: Customer messaging Business A must never get answers from Business B's KB. All KB queries must include `organizationId` filter (verify Pinecone, Gemini RAG, and FAQ fallback all enforce this)
- **System prompt isolation**: Each org's sector config + KB context must be sandboxed per conversation
- **Output sanitization**: AI responses checked before sending via WhatsApp — no markdown injection, no leaking other customers' phone numbers or PII

### Data & Privacy
- Conversation storage in Convex — enforce org-level data isolation at query level (all indexes include `organizationId`)
- **Cross-sector learning anonymization** (Phase 5): Strip phone numbers, names, emails before aggregation into `sector_common_questions`. Only question patterns, never customer data
- **GDPR/data retention**: Conversations should have configurable TTL per org (add `dataRetentionDays` to org settings, default 365)
- KB documents uploaded by businesses contain proprietary info — strict org-scoped access enforced on all document/FAQ queries

### Credential Security
- Meta API credentials stored AES-256-GCM encrypted — already in place
- Stripe Connect keys per org — already handled
- **No plaintext credential logging**: Verify auto-provisioning pipeline doesn't log credentials in Convex function logs, Sentry breadcrumbs, or console output

### Multi-Tenant Trust Boundaries
- `whatsapp-router.service.ts` Redis cache (`phone_number_id` → org) is a critical trust boundary:
  - Cache poisoning would route messages to wrong org — validate against DB on cache miss
  - **Cache invalidation on WhatsApp disconnect** — mandatory
  - Cache TTL of 1 hour max with DB fallback
- **Ownership validation on every action**: Validate org ownership on every booking/payment/quotation action, not just at message routing level. A compromised conversation context must not allow cross-org writes

### Security Tests (per phase)
- **Phase 1**: Test org isolation — send messages to two different orgs, verify no cross-contamination in KB queries, conversation storage, or responses
- **Phase 2**: Verify auto-provisioning doesn't log credentials; verify seeded KB is org-scoped
- **Phase 3**: Test OAuth CSRF protection; verify token storage is server-side only; test token refresh
- **Phase 4**: Verify payment links are org-scoped; verify quotations can't be sent to wrong org's customers
- **Phase 5**: Verify anonymization — no PII in `sector_common_questions` table

---

## Implementation Order

| Order | Phase | What | Why |
|-------|-------|------|-----|
| 1 | Phase 1 | Unified Conversation Engine | Core - everything depends on this working |
| 2 | Phase 2 | Auto-Provisioning | Makes signup → ready automatic |
| 3 | Phase 4 | Full Booking Capabilities | Quotations, reviews, payment - the value prop |
| 4 | Phase 3 | WhatsApp Connection UX | Embedded signup for non-technical users |
| 5 | Phase 5 | Learning Loop | Continuous improvement layer |

---

## Verification Plan

### Per-Phase Verification
1. **Engine**: Send test WhatsApp messages to a connected number, verify AI responds with sector-appropriate language, state machine progresses correctly, booking is created
2. **Auto-Provisioning**: Create test account for each sector, verify org + KB + appointment types are seeded
3. **Booking Flow**: Complete end-to-end booking with payment link for 3 sectors (healthcare, beauty, food)
4. **Connection**: Test Meta Embedded Signup flow end-to-end
5. **Learning**: Send unknown questions, verify they appear in unanswered_questions, verify suggested FAQs generated

### End-to-End Test
1. Fresh signup as healthcare business
2. Verify org auto-created with healthcare KB (10+ FAQs seeded)
3. Connect WhatsApp via Embedded Signup
4. Send customer message: "I want to book an appointment with Dr. Smith"
5. Verify AI collects name, date, time, creates booking, sends payment link
6. Verify booking appears in dashboard
7. Send unknown question → verify it's tracked for learning
