# Mawidi WhatsApp Native Conversation & Booking Flows - Complete Analysis

**Generated:** 2026-04-05  
**Status:** Comprehensive review of NATIVE flows (NOT n8n dependencies)  
**Purpose:** Understand existing native implementation to replicate without n8n dependency

---

## EXECUTIVE SUMMARY

The system has **TWO PARALLEL IMPLEMENTATIONS**:

1. **N8N-DEPENDENT FLOW** (Generic businesses)
   - WhatsApp webhook → forwardMessageToN8N → N8N orchestrates everything
   - Called from `/api/integrations/whatsapp/webhook/route.ts`
   - Falls back to Convex mutations

2. **FULLY NATIVE FLOW** (Real Estate only - production-ready)
   - WhatsApp webhook → Twilio WhatsApp Service → Real Estate AI Engine → Direct DB updates
   - Called from `/api/webhooks/twilio-whatsapp-re/route.ts`
   - **This is what you should replicate for generic bookings**

---

## CRITICAL FLOW DIAGRAMS

### Native Real Estate WhatsApp Flow (COMPLETE):

```
Meta WhatsApp User Message
       ↓
POST /api/webhooks/twilio-whatsapp-re (620 lines)
       ↓
┌─────────────────────────────────────────┐
│ 1. Rate Limit (Upstash Redis)          │
│ 2. Signature Validation (Twilio)        │
│ 3. Parse & Sanitize Message             │
│ 4. Detect Language (Regex: Arabic chars) │
│ 5. Upsert org_whatsapp_conversations   │
└─────────────────────────────────────────┘
       ↓
   [Language Selection?] ──→ Send bilingual greeting, state="selecting_language"
       ↓ (Language chosen)
   [Human Control?] ──→ Skip AI (30 min cooldown)
       ↓
   Load last 20 messages from DB
       ↓
┌─────────────────────────────────────────┐
│ Build ConversationContext object        │
│ - organizationId, orgName               │
│ - customerPhone, customerName           │
│ - language (en/ar)                      │
│ - conversationHistory (reversed)        │
│ - conversationState (BookingFlowState)  │
│ - isReturningCustomer                   │
└─────────────────────────────────────────┘
       ↓
realEstateAIEngine.processMessage(context, body)
       ↓
┌──────────────────────────────────────────────────────┐
│ WhatsApp AI Engine (/lib/services/)                  │
│                                                      │
│ 1. Pre-check guardrails (abuse, injection)          │
│ 2. Query knowledge base (Pinecone → Gemini → FAQ)   │
│ 3. Build system prompt (330+ lines template)        │
│ 4. Call OpenRouter API (Claude Sonnet)              │
│ 5. Post-check guardrails (output safety)            │
│ 6. Execute action (booking, reschedule, cancel)     │
│                                                      │
│ Returns: {                                           │
│   responseText: string                              │
│   intent: AIIntent (booking_request, etc)           │
│   action?: AIActionType                             │
│   newState: BookingFlowState                        │
│   paymentUrl?: string (for Stripe)                  │
│   bookingId?: string                                │
│ }                                                   │
└──────────────────────────────────────────────────────┘
       ↓
[Arabic] ──→ qatariDialectService.translateToQatari()
       ↓
Store messages & state in org_whatsapp_messages
       ↓
Update org_whatsapp_conversations (state, messageCount, timestamp)
       ↓
twilioWhatsAppService.sendTextMessage(credentials, phone, responseText)
       ↓
Return 200 to webhook
```

### N8N-Dependent Flow (Current Generic):

```
Meta WhatsApp User Message
       ↓
POST /api/integrations/whatsapp/webhook (215 lines)
       ↓
Signature Verification
       ↓
whatsappService.forwardMessageToN8N(message, metadata)
       ↓
POST ${N8N_WEBHOOK_URL}
{
  platform: 'whatsapp',
  user_id: userId,
  customer_phone: message.from,
  message: { id, timestamp, type, text },
  metadata: { phone_number_id, display_phone_number }
}
       ↓
N8N Workflow (external orchestration)
       ↓
Calls back to:
  - POST /api/whatsapp/conversations/log (with n8n auth token)
  - POST /api/whatsapp/bookings (with n8n auth token)
       ↓
Conversation & Booking stored
```

---

## DETAILED COMPONENT BREAKDOWN

### 1. MESSAGE ENTRY POINTS

#### File: `/api/integrations/whatsapp/webhook/route.ts`
- **GET Handler** (Lines 36-85): Webhook verification
  - Query params: `hub.mode`, `hub.verify_token`, `hub.challenge`
  - Validates via `whatsappService.verifyWebhook()`
  - Returns challenge for Meta verification
  
- **POST Handler** (Lines 93-215): Incoming messages
  - Rate limit check (100/min per IP)
  - Signature verification: HMAC-SHA256 (Line 109-119)
  - Payload validation (Line 135-143)
  - **Forwards to N8N** (Line 190)
  - Also stores in Convex if configured (Lines 153-186)

#### File: `/api/webhooks/twilio-whatsapp-re/route.ts`
- **GET Handler** (Lines 78-84): Health check
- **POST Handler** (Lines 90-620): Full native processing
  - Rate limiting (Upstash) (Lines 93-125)
  - Signature validation per-org (Lines 168-181)
  - Message parsing & sanitization (Lines 184-201)
  - Language detection (Lines 68-72, 203)
  - Conversation upsert (Lines 206-225)
  - Complete state machine (Lines 267-437)
  - AI processing (Line 527)
  - Message sending (Lines 593-605)

---

### 2. MESSAGE PROCESSING PIPELINE

#### WhatsApp Service (`/lib/services/whatsapp.service.ts`)

**Key Functions:**

1. `sendMessage(userId, to, message)` - Send via Meta API
   - Endpoint: `https://graph.facebook.com/v21.0/{phoneNumberId}/messages`
   - Requires: phoneNumberId, accessToken

2. `verifyWebhook(mode, token, challenge)` - Initial setup
   - Validates `hub.verify_token`

3. `verifyWebhookSignature(payload, signature)` - Per-message
   - HMAC-SHA256 with `WHATSAPP_APP_SECRET`
   - Constant-time comparison (timing attack prevention)

4. `processIncomingMessage(payload)` - Parses Meta webhook
   - Extracts message from nested payload structure
   - Handles different message types (text, interactive, etc)
   - **Calls `forwardMessageToN8N()`** (Lines 317-407)

5. `forwardMessageToN8N(message, metadata)` - N8N routing
   - Resolves user by `phone_number_id`
   - Builds N8N payload with org context
   - Sends to `N8N_WEBHOOK_URL` with Bearer token
   - Logged but non-blocking if fails

6. `connectWhatsApp(userId, phoneNumberId, accessToken)` - Setup
   - Verifies credentials via test API call
   - Stores encrypted access token
   - Updates user record

---

### 3. CONVERSATION STATE MACHINE

#### State Definition (`/lib/types/twilio-whatsapp.types.ts` Lines 140-153)

**Generic Booking States:**
```typescript
selecting_language    → idle (once chosen)
idle                 → browsing_categories (booking intent)
browsing_categories  → browsing_services (category selected)
browsing_services    → collecting_info (service selected)
collecting_info      → awaiting_deposit_choice (all fields complete)
awaiting_deposit_choice → awaiting_payment OR completed
awaiting_payment     → completed (payment webhook)
completed            → idle (conversation can restart)
```

**Real Estate States:**
```typescript
re_searching         → re_viewing_results
re_viewing_results   → re_collecting_viewing_info
re_collecting_viewing_info → re_viewing_booked
```

#### State Transition Logic

**File:** `/lib/services/whatsapp-ai-engine.service.ts` Lines 1067-1155

Function: `updateBookingState(currentState, aiResponse, organizationId)`

- Detects category/service selection by `categorySelection` or `serviceSelection` numbers
- Looks up actual category/service names from AppointmentTypeService
- Tracks collected fields (name, email, service, date, time)
- Compares against `REQUIRED_BOOKING_FIELDS`
- Looks up appointment type for duration & price
- Pre-fills service when transitioning from browsing_services

**Collected Info Structure:**
```typescript
interface BookingInfo {
  name: string;
  email?: string;
  phone: string; // NOT collected by AI - comes from WhatsApp
  service: string;
  date: string;  // YYYY-MM-DD
  time: string;  // HH:MM
  duration: number;
  depositRequired: boolean;
  depositAmount?: number;
  sectorMetadata?: Record<string, unknown>;
}
```

---

### 4. AI ENGINE ARCHITECTURE

#### File: `/lib/services/whatsapp-ai-engine.service.ts`

**Main Function:** `async processMessage(context, incomingMessage)` (Lines 147-255)

**Pipeline:**

1. **PRE-GUARDRAILS** (Lines 160-174)
   ```typescript
   preCheck = aiGuardrailsService.preCheck(incomingMessage)
   if (!preCheck.safe) return redirect message
   ```
   - Detects: prompt_injection, abuse, xss, off_topic

2. **LOAD CONTEXT** (Lines 178-186, parallel)
   ```typescript
   [categoriesList, servicesInCategory, kbContext] = await Promise.all([
     AppointmentTypeService.getFormattedCategoriesList(...),
     AppointmentTypeService.getFormattedServicesInCategory(...),
     this.queryKnowledgeBase(...)
   ])
   ```

3. **BUILD SYSTEM PROMPT** (Lines 189-193)
   - Template: 130+ lines (Lines 26-132)
   - Replaces {{PLACEHOLDERS}} with context values
   - Includes: business name, customer context, categories, services, KB, sector context
   - For Real Estate: Additional "CUSTOMER HISTORY" section (prev searches, properties viewed)

4. **BUILD MESSAGES** (Lines 194-200)
   ```typescript
   messages = [
     { role: 'system', content: systemPrompt },
     ...conversationHistory.slice(-MAX_HISTORY_MESSAGES),
     { role: 'user', content: incomingMessage }
   ]
   ```
   - Guard: Skip corrupted JSON assistant messages (Lines 358-383)

5. **CALL AI** (Lines 203-205)
   ```typescript
   aiResponse = await openRouterService.structuredCompletion(messages)
   ```
   - Returns: `StructuredAIResponse` (structured JSON)

6. **POST-GUARDRAILS** (Lines 207-220)
   ```typescript
   postCheck = aiGuardrailsService.postCheck(aiResponse, context)
   if (!postCheck.safe) return sanitized fallback message
   ```

7. **EXECUTE ACTION** (Lines 227-235)
   ```typescript
   result = await this.executeAction(context, aiResponse, responseText)
   ```
   - Routes by `aiResponse.intent`

---

### 5. KNOWLEDGE BASE QUERY FLOW

#### File: `/lib/services/whatsapp-knowledge-base.service.ts`

**Entry Function:** `queryKnowledgeBase(organizationId, query)` (Lines 110-211)

**Three-tier Fallback Strategy:**

**Tier 1: Pinecone Multimodal RAG** (Lines 567-605 in AI engine)
```typescript
const result = await PineconeMultimodalService.queryAndGenerate(
  ownerId, query, { language: 'en', topK: 5 }
)
// Returns: { success, answer, mediaResults?, citations?, metadata? }
```
- Vector search using Gemini Embedding 2
- Supports documents, images, videos, audio
- Includes media descriptions

**Tier 2: Gemini FileSearch RAG** (Lines 609-642)
- Used only if Tier 1 returned no results
- RAG over uploaded documents
- Includes citations

**Tier 3: FAQ Keyword Match** (Lines 645-704)
- Keyword extraction from query (>= 3 chars, up to 10 keywords)
- Searches `company_knowledge_base` table
- Case-insensitive contains match on question
- Sorted by priority

**Return Format:**
```
[From knowledge base]
<Pinecone answer>

[Related media]
<media descriptions>

---

[From FAQ entries]
Q: <question>
A: <answer>
[<category>]
```

---

### 6. ACTION EXECUTION & BOOKING CREATION

#### File: `/lib/services/whatsapp-ai-engine.service.ts`

**Main Handler:** `executeAction(context, aiResponse, responseText)` (Lines 756-815)

**Routing:**
```typescript
switch(aiResponse.intent) {
  case 'booking_request':
    return handleBookingRequest(...)
  case 'amend_appointment':
    return handleAmendAppointment(...)
  case 'check_status':
    return handleCheckStatus(...)
  default:
    return baseResult
}
```

#### Booking Request Handler (Lines 817-1049)

**State Machine:**

**State: collecting_info**
- Missing fields? → Keep asking for missing ones

**State: awaiting_deposit_choice**
```
if (depositChoice === true)
  → Create booking
  → Generate Stripe session
  → Return payment link
  → New state: awaiting_payment

else if (depositChoice === false)
  → Create booking directly
  → New state: completed

else (depositChoice === null)
  → Stay in awaiting_deposit_choice
  → Ask customer for decision
```

#### Booking Creation (Lines 917-1008)

**Function:** `createBooking(organizationId, info, customerPhone, customerEmail)`

**Process:**
1. Resolve organization owner (for userId field)
2. Create booking record:
   ```typescript
   await prisma.bookings.create({
     id: `wa_${Date.now()}_${random}`,
     userId: org.ownerId,
     organizationId,
     customerName: info.name,
     customerPhone, // from WhatsApp
     customerEmail,
     service: info.service,
     appointmentDate,
     appointmentTime: info.time,
     duration: info.duration,
     depositAmount: info.depositRequired ? info.depositAmount : null,
     depositPaid: false,
     totalAmount: 0,
     source: 'whatsapp',
     status: 'pending',
     notes: 'Booked via WhatsApp AI agent'
   })
   ```

3. Auto-create customer (upsert by phone):
   ```typescript
   // Only if not already exists
   await prisma.customers.create({
     name: info.name,
     phone: customerPhone,
     email: customerEmail || info.email,
     source: 'whatsapp'
   })
   ```

#### Stripe Payment Integration (Lines 945-1006)

**Triggered:** When `depositChoice === true`

**Steps:**
1. Call `createCheckoutSessionForOrganization()`:
   ```typescript
   const { session } = await createCheckoutSessionForOrganization(
     organizationId, {
       mode: 'payment',
       customer_email: customerEmail,
       line_items: [{
         price_data: {
           currency: 'qar',
           unit_amount: depositAmount * 100,
           product_data: {
             name: `Appointment Deposit - ${service}`,
             description: `${formattedDate} at ${time}`
           }
         },
         quantity: 1
       }],
       metadata: {
         type: 'booking_deposit',
         booking_id,
         customer_name,
         customer_phone: context.customerPhone,
         organization_id: organizationId,
         source: 'whatsapp'
       },
       success_url: `${APP_URL}/${lang}/payment/success?booking_id=${bookingId}&session_id={CHECKOUT_SESSION_ID}`,
       cancel_url: `${APP_URL}/${lang}/payment/cancelled?booking_id=${bookingId}`
     }
   )
   ```

2. Update booking with payment session:
   ```typescript
   await prisma.bookings.update({
     where: { id: booking.bookingId },
     data: {
       depositAmount,
       depositPaymentId: session.id
     }
   })
   ```

3. Update state:
   ```typescript
   newState = {
     phase: 'awaiting_payment',
     bookingId: booking.bookingId,
     paymentUrl: session.url
   }
   ```

4. Append payment link to response text

---

### 7. LANGUAGE & LOCALIZATION

#### Language Detection (Lines 66-72)

```typescript
const ARABIC_RANGE = /[\u0600-\u06FF\u0750-\u077F\u08A0-\u08FF]/

function detectLanguage(text: string): 'en' | 'ar' {
  const arabicChars = (text.match(ARABIC_RANGE) || []).length
  return arabicChars > text.length * 0.3 ? 'ar' : 'en'
}
```

**Thresholds:**
- > 30% Arabic chars = Arabic language

#### Language Preference Storage

**Service:** `/lib/services/whatsapp-language.service.ts`

- `getLanguagePreference(organizationId, phone)` - Redis lookup
- `setLanguagePreference(organizationId, phone, language)` - Redis store
- Stored per customer (org + phone key)

**Used in webhook:** Lines 228-245 (override detected language with stored preference)

#### Qatari Dialect Post-Processing

**Service:** `qatariDialectService.translateToQatari(text)`

**Applied:** Only to Arabic responses (Lines 537-566)

**Process:**
1. Extract lines with URLs (preserve during translation)
2. Send text-only to Qatari dialect LLM
3. Reappend URLs after translation

**Purpose:** Convert MSA (Modern Standard Arabic) to natural Qatari dialect

---

### 8. REAL ESTATE SPECIFIC FEATURES

#### File: `/lib/services/real-estate-ai-engine.service.ts`

**Extends Generic AI Engine:**
- Same AI processing pipeline
- Different system prompt (RE-specific)
- Different action handlers
- Access to property search & viewing booking

**System Prompt Additions:**
- "You ARE a real estate agent" (not a bot)
- Natural conversational style
- Customer history from previous viewings
- Available properties context
- Viewing booking instructions

**Action Handlers:**
- `search_properties` → Calls `searchProperties(userId, criteria)`
- `book_viewing` → Calls `ViewingService.createViewing()`
- `reschedule_viewing` → Calls `ViewingService.rescheduleViewing()`
- `cancel_viewing` → Calls `ViewingService.cancelViewing()`

#### Property Search Service

**File:** `/lib/services/real-estate/property.service.ts`

```typescript
searchProperties(userId: string, criteria: {
  pageSize?: number,
  sort?: string,  // 'price' | 'oldest'
  status?: string,
  propertyType?: string,
  listingType?: string,
  city?: string,
  district?: string,
  minPrice?: number,
  maxPrice?: number,
  bedrooms?: number,
  minArea?: number,
  maxArea?: number
})
```

Returns: `{ properties: PropertyResult[], total: number }`

#### Viewing Booking Service

**File:** `/lib/services/real-estate/viewing.service.ts`

```typescript
ViewingService.createViewing(userId, {
  propertyId: string,
  brokerId?: string,
  clientName: string,
  clientPhone: string,
  clientEmail?: string,
  viewingDate: Date,
  viewingTime: string,
  source?: string
})
```

Creates viewing record + sends notifications + assigns broker

#### WhatsApp Templates

**File:** `/lib/services/real-estate/whatsapp-templates.service.ts` (547 lines)

Provides formatted messages for:
- Viewing confirmation
- Viewing reminder (24h, 2h)
- Post-viewing follow-up
- Viewing cancellation
- Price drop alert
- Status change alert
- New matches notification

All templates support English & Arabic (with Qatari dialect)

---

## NATIVE VS N8N DEPENDENCY SUMMARY

### Currently NATIVE (No N8N Required):
✅ WhatsApp/Twilio webhook receipt & validation  
✅ Conversation state machine & persistence  
✅ Message history retrieval & storage  
✅ Language detection & preference management  
✅ AI processing via OpenRouter  
✅ Knowledge base querying (3-tier fallback)  
✅ Guardrails (pre-check + post-check)  
✅ Booking creation (DB + customer auto-creation)  
✅ Stripe payment integration  
✅ Real estate: property search  
✅ Real estate: viewing booking  
✅ Sector-specific customization  
✅ Qatari dialect post-processing  

### Currently N8N-Dependent:
❌ `/api/integrations/whatsapp/webhook/route.ts` forwards to N8N  
❌ N8N handles: AI processing, booking creation, conversation logging  
❌ `/api/whatsapp/conversations/log/route.ts` expects N8N to call  
❌ `/api/whatsapp/bookings/route.ts` expects N8N to call  

---

## CRITICAL FILE REFERENCES

### System Prompt Templates:
- Generic: `/lib/services/whatsapp-ai-engine.service.ts` Lines 26-132
- Real Estate: `/lib/services/real-estate-ai-engine.service.ts` Lines 38-110

### State Machine:
- Types: `/lib/types/twilio-whatsapp.types.ts` Lines 140-153
- Transitions: `/lib/services/whatsapp-ai-engine.service.ts` Lines 1067-1155

### Action Execution:
- Main: `/lib/services/whatsapp-ai-engine.service.ts` Lines 756-815
- Booking: Lines 817-1049
- Amendment: Lines 851-920
- Status Check: Lines 922-940

### Webhook Handlers:
- Real Estate (NATIVE): `/api/webhooks/twilio-whatsapp-re/route.ts` Lines 1-620
- Generic (N8N): `/api/integrations/whatsapp/webhook/route.ts` Lines 1-215

### Message Storage:
- Conversations: `/lib/services/whatsapp-conversations.service.ts` Lines 1-109
- Schema: `org_whatsapp_conversations` + `org_whatsapp_messages` tables

---

## IMPLEMENTATION PATH: MIGRATE GENERIC TO NATIVE

1. **Create webhook handler** (copy from RE handler)
   - Replace Twilio calls with Meta Cloud API calls
   - Use generic AI engine instead of RE engine
   - Update conversation table references (org_whatsapp_*)

2. **Keep existing native services:**
   - `whatsappAIEngine` ✓
   - `openRouterService` ✓
   - Knowledge base querying ✓
   - Guardrails ✓
   - State machine ✓

3. **Remove N8N dependency:**
   - Delete N8N forwarding from `whatsapp.service.ts`
   - Don't call `/api/whatsapp/conversations/log`
   - Don't call `/api/whatsapp/bookings`
   - Handle everything directly in webhook handler

4. **Test with Meta Cloud API or Twilio:**
   - Set NGROK_URL for local development
   - Verify message flow end-to-end
   - Test state transitions
   - Test booking creation with Stripe

---

**This analysis proves the native architecture is complete and production-ready (proven by real estate implementation). Migration is straightforward pattern replication.**
