Webhooks

Receive real-time notifications when crawl events occur. Instead of polling the API, Mulberry pushes events directly to your endpoint.

Creating a Webhook Endpoint

Create webhook endpoints from the Settings → Webhooks page in your dashboard. Configure the URL where you want to receive events and select which events to subscribe to.

Available Events

Subscribe to specific events or use wildcards:

Crawl Events

crawl.started Crawl began processing
crawl.completed Crawl finished successfully
crawl.failed Crawl encountered an error
crawl.page_success Individual page crawled successfully
crawl.* All crawl events (wildcard)

Business Listing Events

listing.created New listing discovered
listing.updated Existing listing modified
listing.* All listing events (wildcard)

Live Page Events

live_page.polled Page was polled for changes
live_page.changed Changes detected on monitored page
live_page.error Polling encountered an error
live_page.* All live page events (wildcard)

Webhook Payload

When an event occurs, Mulberry sends a POST request to your endpoint with a JSON payload:

Example Payload

{
  "id": "evt_xyz789",
  "type": "crawl.completed",
  "created_at": "2024-01-15T10:35:00Z",
  "data": {
    "crawl_id": "crawl_abc123",
    "url": "https://docs.example.com",
    "status": "completed",
    "pages_crawled": 120,
    "started_at": "2024-01-15T10:30:00Z",
    "completed_at": "2024-01-15T10:35:00Z"
  }
}

Field	Type	Description
`id`	string	Unique event identifier
`type`	string	Event type (e.g., `crawl.completed`)
`created_at`	string	ISO 8601 timestamp of event creation
`data`	object	Event-specific data payload

Verifying Webhook Signatures

All webhook requests include a signature header for verification. This ensures the request came from Mulberry and wasn't tampered with.

Security: Always verify webhook signatures before processing events in production. Never trust unverified payloads.

Signature Header

The X-Mulberry-Signature header contains the signature:

Signature Header Format

X-Mulberry-Signature: t=1705315200,v1=abc123...

Component	Description
`t`	Unix timestamp when the signature was generated
`v1`	HMAC-SHA256 signature of the payload

Verification Steps

Extract the timestamp and signature from the header
Construct the signed payload: {timestamp}.{request_body}
Compute HMAC-SHA256 using your webhook signing secret
Compare the computed signature with the received signature
Verify the timestamp is within an acceptable window (e.g., 5 minutes)

Example Verification (Node.js)

const crypto = require('crypto');

function verifyWebhook(payload, signature, secret) {
  const [tPart, v1Part] = signature.split(',');
  const timestamp = tPart.split('=')[1];
  const receivedSig = v1Part.split('=')[1];

  const signedPayload = `${timestamp}.${payload}`;
  const expectedSig = crypto
    .createHmac('sha256', secret)
    .update(signedPayload)
    .digest('hex');

  return crypto.timingSafeEqual(
    Buffer.from(receivedSig),
    Buffer.from(expectedSig)
  );
}

Handling Webhook Requests

Your endpoint should follow these best practices:

Respond quickly

Return a 2xx status code within 30 seconds

Verify first

Always verify the signature before processing

Process asynchronously

Handle heavy work in background jobs

Be idempotent

Handle duplicate events gracefully

Tip: Return a 200 response immediately, then process the event in a background job. This prevents timeouts and ensures reliable delivery.

Retry Behavior

If your endpoint fails to respond with a 2xx status, Mulberry retries with exponential backoff:

1st 1 minute

2nd 5 minutes

3rd 30 minutes

4th 2 hours

5th 24 hours

After 5 failed attempts, the webhook delivery is marked as failed and you'll receive a notification.

Webhook Signing Secret

Your webhook signing secret is available in Settings → Webhooks. You can regenerate it at any time, but this will invalidate signatures for any in-flight requests.

Note: Store your signing secret securely using environment variables. Never commit it to version control or expose it in client-side code.

Next Steps

Connect AI Agents via MCP Manage API Keys