# WhatsApp via Twilio + ngrok — Local Setup Guide

This guide walks you through connecting a Twilio WhatsApp Sandbox (or production WhatsApp number) to your local Mawidi dev server using **ngrok** to expose `localhost:9001` to the internet so Twilio can deliver webhooks.

## Prerequisites

- Twilio account (free tier works for sandbox)
- ngrok installed (`brew install ngrok` or [download from ngrok.com](https://ngrok.com))
- Mawidi dev server running locally on port 9001 (or 9000)
- An organization in Convex with `industry` set (so the sector router resolves)

---

## Step 1: Start ngrok

Open a terminal and start ngrok pointing at your dev server:

```bash
ngrok http 9001
```

You'll see something like:

```
Forwarding   https://abc-12-34-56-78.ngrok-free.app -> http://localhost:9001
```

Copy the `https://abc-12-34-56-78.ngrok-free.app` URL — you'll use it everywhere below as `${NGROK_URL}`.

> **Important**: Set `NGROK_URL` in `mawidi-site/.env.local` so signature validation passes:
> ```
> NGROK_URL=https://abc-12-34-56-78.ngrok-free.app
> ```
> Then restart `npm run dev`. The webhook uses this to reconstruct the public URL Twilio signed against.

---

## Step 2: Create a Twilio WhatsApp Sandbox

1. Log in to [Twilio Console](https://console.twilio.com/)
2. Go to **Messaging → Try it out → Send a WhatsApp message**
3. You'll see a sandbox number like `+1 415 523 8886` and a join code (e.g. `join purple-tiger`)
4. From your personal WhatsApp, send the join code to `+1 415 523 8886`
5. Twilio will confirm: *"Twilio Sandbox: Connected! Reply 'stop' to unsubscribe."*

You're now able to send messages to the sandbox number and receive replies.

---

## Step 3: Configure the Twilio Webhook URL

In Twilio Console:

1. Go to **Messaging → Try it out → Send a WhatsApp message → Sandbox settings**
2. Set **"When a message comes in"** to:
   ```
   https://abc-12-34-56-78.ngrok-free.app/api/webhooks/twilio-whatsapp
   ```
   (Use your actual ngrok URL.)
3. Method: **POST**
4. Click **Save**

> **For real estate orgs** — use `/api/webhooks/twilio-whatsapp-re` instead. Both endpoints exist.

---

## Step 4: Connect Your Org to the Sandbox in Convex

The router resolves orgs by their `twilioWhatsappNumber` field. You need to set this on your test organization.

### Option A: Via the Convex dashboard

1. Open the [Convex dashboard](https://dashboard.convex.dev) → your project → `organizations` table
2. Find your test org
3. Set:
   - `twilioWhatsappNumber` = `+14155238886` (the sandbox number, **without** the `whatsapp:` prefix)
   - `twilioAccountSid` = your Twilio Account SID
   - `twilioAuthTokenEncrypted` = the encrypted version of your auth token (see Option B for the easy way)
   - `twilioWhatsappConfigured` = `true`
   - `industry` = e.g. `"healthcare"` (so the sector router returns `health_care`)
4. Save

### Option B: Via a one-off script (recommended)

Run this from `mawidi-site/`:

```bash
npx tsx -e "
import { config } from 'dotenv';
config({ path: '.env.local' });

import { ConvexHttpClient } from 'convex/browser';
import { encrypt } from './lib/encryption';

const client = new ConvexHttpClient(process.env.NEXT_PUBLIC_CONVEX_URL!);

// === Fill these in ===
const ORG_ID                = 'YOUR_ORG_ID_HERE';        // get from Convex dashboard
const TWILIO_ACCOUNT_SID    = 'ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx';
const TWILIO_AUTH_TOKEN     = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx';
const TWILIO_WHATSAPP_NUMBER = '+14155238886';            // sandbox: +14155238886
const INDUSTRY              = 'healthcare';

(async () => {
  await client.mutation('organizations/mutations:updateOrganizationNoAuth' as any, {
    organizationId: ORG_ID,
    industry: INDUSTRY,
    sector: undefined, // will be derived from industry by the AI engine
    twilioAccountSid: TWILIO_ACCOUNT_SID,
    twilioAuthTokenEncrypted: encrypt(TWILIO_AUTH_TOKEN),
    twilioWhatsappNumber: TWILIO_WHATSAPP_NUMBER,
    twilioWhatsappConfigured: true,
    twilioWhatsappConfiguredAt: Date.now(),
    twilioWhatsappConfiguredBy: 'manual-setup',
  });
  console.log('Org configured for Twilio WhatsApp');
})();
"
```

`updateOrganizationNoAuth` accepts all Twilio + greeting fields, so this single mutation is enough to wire up a test org end-to-end.

---

## Step 5: Send a Test Message

From your personal WhatsApp (the one that joined the sandbox):

```
Hi, I'd like to book an appointment
```

You should see:
- ngrok terminal: `POST /api/webhooks/twilio-whatsapp 200 OK`
- Dev server logs:
  ```
  [INFO] Twilio route resolved { organizationId: '...', organizationName: '...' }
  [INFO] Twilio WhatsApp: sent greeting to new user
  ```
- WhatsApp: **the bilingual greeting from your org's custom welcome message** (or the default if not set)

---

## Step 6: Verify the Full Flow

After picking a language, send:

```
2  (or "English")
```

Then ask a sector-relevant question:

```
What are your opening hours?
```

You should see:
- Dev server logs show `intent: "company_query"`, `kbUsed: true/false`
- AI responds in English with sector-appropriate language
- The unified `whatsAppAIEngine` is called (not the RE one)

If you ask:
```
I want to book an appointment with Dr. Smith on Monday at 10am
```
You should see the AI start collecting booking fields per the healthcare sector workflow.

---

## Troubleshooting

| Symptom | Fix |
|---------|-----|
| **403 from webhook** | `NGROK_URL` not set in `.env.local` → restart dev server |
| **`unknown To number`** | Org's `twilioWhatsappNumber` doesn't match the sandbox number — must be `+14155238886` exactly (no `whatsapp:` prefix) |
| **`credentials not found`** | Org has no `twilioAuthTokenEncrypted` — set it in Convex dashboard |
| **Generic fallback greeting** (not your custom one) | Custom greeting fields are blank in Convex, or `autoGreetingEnabled === false` |
| **AI ignores sector context** | `industry` field is `null` on the org — set it via the Convex dashboard or `updateOrganizationNoAuth` |
| **Conversation stuck in `selecting_language`** | Reply with `1` or `2` — the AI won't proceed until language is chosen |

---

## Architecture Notes

- **Endpoint**: `POST /api/webhooks/twilio-whatsapp` (unified, sector-aware)
- **Router**: `whatsAppRouter.resolveOrganizationByTwilioNumber(toNumber)` — Redis-cached, falls back to Convex query `getOrganizationByTwilioNumber`
- **Engine**: `whatsAppAIEngine.processMessage(context, body, sectorConfig)` — picks the right sector workflow, gates payments on Stripe key presence, tracks KB metadata
- **Learning**: Every turn fires `analyzeConversationTurn` (non-blocking) to track unanswered questions for the dashboard
- **Real Estate**: Still uses the dedicated `/api/webhooks/twilio-whatsapp-re` endpoint with `realEstateAIEngine`

---

## Promoting from Sandbox to Production

When ready to use a real WhatsApp number (not the shared sandbox):

1. Apply for a Twilio WhatsApp sender (requires Facebook Business Manager + WhatsApp display name approval)
2. Once approved, Twilio gives you a real number like `+97412345678`
3. Update the org's `twilioWhatsappNumber` to the real number
4. Update Twilio's webhook config to point to your **production** URL (not ngrok)
5. The same code runs — no changes needed