Assemble your own assistant

This guide shows one way to put four of the packages around your own agent: a phone line, SMS follow-ups that respect consent, Instagram DMs, and email. The agent is yours. The prompt, model choice, tools and business rules never pass through a package. The packages handle the mechanics at the edges.

What this is, and is not

This is a worked example of wiring, not a product, a template you should deploy as is, or a compliance recipe. It uses the packages’ in-memory fakes for the demo and assumes you understand your own consent, privacy and platform-policy obligations.

The shape

The packages handle the mechanics between your providers and your agent; the agent itself stays yours.

Layer	You provide	A package provides
Phone	Twilio number, Deepgram, ElevenLabs and Anthropic accounts and keys	voice-engine: Media Streams transport, mark-confirmed playback, barge-in history rewrite, speculative turns
SMS	An SMS provider call, a database for consent	sms-consent: keyword handling, consent ledger, sendGuarded
Instagram	A Meta app, App Review where needed, token storage	instagram-dm: signature checks, dedupe, window-aware Send API client
Email	A Resend key and verified domain	email-kit: retry, idempotency, suppression, “email me this”
The assistant	System prompt, tools, model, rules, storage	nothing. This is the part you own.

1. Write the agent: prompt, tools, and guarded side effects

The most useful pattern in this wiring is that tools call the guards, and tell the model the truth. sendGuarded only reports sent: true when the guard allowed the send and the provider resolved, so the tool result can say “not sent” and the model should not claim otherwise.

samples/src/assistant/agent.ts

// YOUR code. The packages never see your prompt, your model choice or your business rules.
import type { ToolDef, ToolExecutor } from '@27colab/voice-engine';
import { ensureOptOutFooter, sendGuarded, type ConsentStore } from '@27colab/sms-consent';
import { createEmailMeThis } from '@27colab/email-kit';
import type { Mailer } from '@27colab/email-kit';

export const system = (lang: 'en' | 'es') =>
  `You are the front-desk assistant for Acme Dental. Reply in ${lang === 'es' ? 'Spanish' : 'English'}, ` +
  'in one or two short sentences. Only say you sent something after the tool result says it was sent.';

export const tools: ToolDef[] = [
  {
    name: 'text_me_details',
    description: 'Text the caller the appointment details. Only works if they have opted in.',
    input_schema: { type: 'object', properties: { body: { type: 'string' } }, required: ['body'] },
  },
  {
    name: 'email_me_this',
    description: 'Email the caller their own details. Read the address back and get a yes first.',
    input_schema: {
      type: 'object',
      properties: { address: { type: 'string' }, confirmed: { type: 'boolean' }, subject: { type: 'string' }, text: { type: 'string' } },
      required: ['address', 'confirmed', 'subject', 'text'],
    },
  },
];

export interface AgentDeps {
  consent: ConsentStore;
  /** Your SMS provider call (Twilio, etc.): (to, body) => Promise<void>. */
  sendSms: (to: string, body: string) => Promise<void>;
  mailer: Mailer;
  accountId: string;
  timeZone: string;
  now?: () => Date;
}

/** One executor per call. `callerPhone` comes from a <Parameter> in your TwiML. */
export function makeExecutor(deps: AgentDeps, callerPhone: string): ToolExecutor {
  const emailMeThis = createEmailMeThis(deps.mailer, { locale: 'en' });
  return async (name, input, ctx) => {
    if (name === 'text_me_details') {
      const { body } = input as { body: string };
      const r = await sendGuarded({
        store: deps.consent,
        accountId: deps.accountId,
        phone: callerPhone,
        timeZone: deps.timeZone,
        now: deps.now?.(),
        body: ensureOptOutFooter(body),
        send: deps.sendSms,
      });
      // The model only learns "sent" when the guard allowed it AND the provider resolved.
      return r.sent
        ? { content: 'Text sent.' }
        : { content: `Text NOT sent (${r.reason}). Do not tell the caller it was sent.`, isError: true };
    }
    if (name === 'email_me_this') {
      const a = input as { address: string; confirmed: boolean; subject: string; text: string };
      const out = await emailMeThis({ ...a, requestId: ctx.callId });
      if (out.status === 'needs-confirmation') return { content: `Read this back and ask for a yes: ${out.readBack}` };
      if (out.status === 'sent') return { content: 'Email sent.' };
      return { content: `Email NOT sent (${out.status}).`, isError: true };
    }
    return { content: `Unknown tool ${name}`, isError: true };
  };
}

Two things to notice:

• The caller’s phone number is passed in by your code (makeExecutor(deps, callerPhone)), not chosen by the model. The “email me this” helper likewise can only send to the address the user gave for themselves.
• The tool result for a refused text says Text NOT sent (opted_out). That is what the model sees on the next turn.

2. Run it with no keys

This script wires voice-engine (fake transport, speech-to-text, text-to-speech and a scripted model), sms-consent, email-kit (fake provider) and instagram-dm (fake Graph API) around the agent above. It:

1. Runs a call for a caller who opted in on a web form; the scripted model calls the text and email tools.
2. Delivers an inbound STOP through handleInbound, as your SMS webhook would.
3. Runs a second call; the same text tool call is now refused, and the model is told so.
4. Sends a signed Instagram webhook through the real handler and shows the reply.

samples/src/assistant/loopback.ts

// Keyless: every provider is one of the packages' fakes. Run: npm run loopback
import { createCallSession } from '@27colab/voice-engine';
import { FakeChat, FakeStt, FakeTransport, FakeTts } from '@27colab/voice-engine/testing';
import { MemoryConsentStore, applyConsentEvent, defaultReplies, handleInbound } from '@27colab/sms-consent';
import { createMailer } from '@27colab/email-kit';
import { FakeProvider } from '@27colab/email-kit/testing';
import { createWebhookHandler, InstagramClient } from '@27colab/instagram-dm';
import { FakeGraphApi, textEvent } from '@27colab/instagram-dm/testing';
import { makeExecutor, system, tools, type AgentDeps } from './agent.js';

const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));

// --- shared state and fakes ------------------------------------------------
const consent = new MemoryConsentStore();
const smsOutbox: Array<{ to: string; body: string }> = [];
const emailProvider = new FakeProvider();
const deps: AgentDeps = {
  consent,
  sendSms: async (to, body) => void smsOutbox.push({ to, body }),
  mailer: createMailer({ provider: emailProvider, from: 'Acme Dental <hello@example.com>' }),
  accountId: 'acct-1',
  timeZone: 'America/New_York',
  now: () => new Date('2026-03-10T17:00:00Z'), // 1 pm local, outside quiet hours
};
const caller = '+15550001111';

// --- a phone call: voice-engine around your tools --------------------------
async function runCall(callId: string, script: (chat: FakeChat) => void, utterance: string) {
  const transport = new FakeTransport({ autoConfirmMarks: true });
  const stt = new FakeStt();
  const chat = new FakeChat();
  script(chat);
  const executor = makeExecutor(deps, caller);

  const session = createCallSession(
    { transport, sttFactory: stt, ttsFactory: new FakeTts({ msPerChar: 5 }), chat },
    { callId, startedAt: new Date() },
    { onEnd: () => {} },
    {
      languages: ['en'],
      greeting: 'Acme Dental, how can I help?',
      system: () => system('en'),
      llm: { model: 'scripted', maxTokens: 200 },
      tools,
      executeTool: executor,
      voice: { provider: 'fake', voices: {}, defaultVoiceId: 'v', model: 'fake', outputFormat: 'ulaw_8000' },
      stt: { model: 'fake' },
    },
  );
  await session.start();
  await sleep(50);
  stt.last.emitEndOfTurn(utterance);
  await sleep(150);
  await session.end('caller_hangup');
  return chat;
}

// 1. The caller opted in on a web form earlier.
await applyConsentEvent(consent, deps.accountId, caller, { type: 'grant', source: 'web-form' });
await runCall(
  'call-1',
  (chat) =>
    chat
      .pushToolCall('text_me_details', { body: 'Acme Dental: you are booked for Tue 3pm.' })
      .pushToolCall('email_me_this', { address: 'jane dot doe at example dot com', confirmed: true, subject: 'Your visit', text: 'Tue 3pm.' })
      .pushText('Done: I texted and emailed you the details.'),
  'Please text and email me the details',
);
console.log('call 1 -> SMS:', smsOutbox.map((m) => m.body));
console.log('call 1 -> email to:', emailProvider.outbox.map((e) => e.message.to));

// 2. The recipient texts STOP. Your Twilio webhook calls handleInbound.
const stop = await handleInbound({ store: consent, accountId: deps.accountId, from: caller, body: 'STOP', replies: defaultReplies('Acme Dental') });
console.log('inbound STOP ->', stop.record?.status);

// 3. Next call: the same tool call is refused, and the tool result says so.
const before = smsOutbox.length;
const call2 = await runCall(
  'call-2',
  (chat) => chat.pushToolCall('text_me_details', { body: 'Reminder: Tue 3pm.' }).pushText('I could not text you, but your visit is Tue 3pm.'),
  'Text me a reminder',
);
console.log('call 2 -> SMS sent:', smsOutbox.length - before);
const lastTurn = JSON.parse(call2.seenMessages.at(-1)!) as Array<{ content: unknown }>;
console.log('call 2 -> what the model was told:', JSON.stringify(lastTurn.at(-1)?.content));

// --- an Instagram DM: instagram-dm around the same kind of agent -----------
const fake = new FakeGraphApi({ appSecret: 'test' });
const igClient = new InstagramClient({ accessToken: fake.issueToken('1000000000001'), fetch: fake.fetch });
const webhook = createWebhookHandler({
  appSecret: 'test',
  verifyToken: 'verify-me',
  resolveClient: () => igClient,
  conversation: { onMessage: async ({ event }) => `Acme Dental: thanks for writing ("${event.text}"). A teammate will confirm your visit.` },
});
fake.recordInbound('1000000000001', '2000000000002');
const hook = fake.buildWebhook([textEvent({ accountId: '1000000000001', userId: '2000000000002', mid: 'm1', text: 'Can I move my visit?' })], '1000000000001');
await webhook.handle({ method: 'POST', headers: hook.headers, rawBody: hook.rawBody });
console.log('instagram reply:', fake.sent.map((m) => m.text));

Recorded output of that run:

output

call 1 -> SMS: [ 'Acme Dental: you are booked for Tue 3pm. Reply STOP to opt out.' ]
call 1 -> email to: [ 'jane.doe@example.com' ]
inbound STOP -> opted_out
call 2 -> SMS sent: 0
call 2 -> what the model was told: [{"type":"tool_result","tool_use_id":"tool_1","content":"Text NOT sent (opted_out). Do not tell the caller it was sent.","is_error":true}]
instagram reply: [
  'Acme Dental: thanks for writing ("Can I move my visit?"). A teammate will confirm your visit.'
]

To run it yourself from a checkout of the docs repository:

npm run samples:install
npm --prefix samples run loopback

Note

The model here is scripted (FakeChat), so this shows the plumbing, not how a real model behaves. The text-to-speech and speech-to-text are fakes too. Nothing was measured: no latency, quality or reliability claims come from this demo.

3. Swap in real providers

Needs keys. Typechecked, not executed. You supply every key: Twilio, Deepgram, ElevenLabs, Anthropic and Resend. The agent file does not change; only the dependencies handed to it do.

samples/src/assistant/live.ts

// Typechecked, not run: the same agent with real providers. You supply every key.
import { WebSocketServer } from 'ws';
import { createCallSession, TwilioMediaTransport } from '@27colab/voice-engine';
import { createAnthropicChat, createDeepgramFluxFactory, createElevenLabsFactory } from '@27colab/voice-engine/providers';
import { SqliteConsentStore } from '@27colab/sms-consent';
import { createMailer, ResendProvider } from '@27colab/email-kit';
import Database from 'better-sqlite3';
import { makeExecutor, system, tools, type AgentDeps } from './agent.js';

const env = process.env;

const deps: AgentDeps = {
  consent: new SqliteConsentStore(new Database('consent.db')),
  sendSms: async (to, body) => {
    // Twilio REST API; no SDK required. Your Account SID, Auth Token, and sender number.
    const res = await fetch(`https://api.twilio.com/2010-04-01/Accounts/${env.TWILIO_ACCOUNT_SID}/Messages.json`, {
      method: 'POST',
      headers: {
        Authorization: 'Basic ' + Buffer.from(`${env.TWILIO_ACCOUNT_SID}:${env.TWILIO_AUTH_TOKEN}`).toString('base64'),
        'Content-Type': 'application/x-www-form-urlencoded',
      },
      body: new URLSearchParams({ To: to, From: env.TWILIO_FROM!, Body: body }),
    });
    if (!res.ok) throw new Error(`Twilio ${res.status}`); // sendGuarded turns this into sent: false
  },
  mailer: createMailer({ provider: new ResendProvider({ apiKey: env.RESEND_API_KEY! }), from: 'Acme Dental <hello@example.com>' }),
  accountId: 'acct-1',
  timeZone: 'America/New_York',
};

const sttFactory = createDeepgramFluxFactory(env.DEEPGRAM_API_KEY!);
const ttsFactory = createElevenLabsFactory(env.ELEVENLABS_API_KEY!);
const chat = createAnthropicChat(env.ANTHROPIC_API_KEY!);

new WebSocketServer({ port: 8080, path: '/media' }).on('connection', async (ws) => {
  const transport = new TwilioMediaTransport(ws);
  const start = await transport.started;
  const callerPhone = start.customParameters.from!; // <Parameter name="from" value="{{From}}"/> in your TwiML

  const session = createCallSession(
    { transport, sttFactory, ttsFactory, chat },
    { callId: start.callSid, startedAt: new Date() },
    { onEnd: (record) => console.log(record.callId, record.stats) },
    {
      languages: ['en', 'es'],
      languagePolicy: 'follow-caller',
      greeting: { en: 'Acme Dental, how can I help?', es: 'Acme Dental, ¿en qué puedo ayudarle?' },
      system: (lang) => system(lang),
      llm: { model: 'claude-haiku-4-5', maxTokens: 300 },
      tools,
      executeTool: makeExecutor(deps, callerPhone),
      voice: {
        provider: 'elevenlabs',
        voices: { en: env.ELEVENLABS_VOICE_EN, es: env.ELEVENLABS_VOICE_ES },
        defaultVoiceId: env.ELEVENLABS_VOICE_EN!,
        model: 'eleven_flash_v2_5',
        outputFormat: 'ulaw_8000',
      },
      stt: { model: 'flux-general-multi', eagerEotThreshold: 0.6 },
    },
  );
  await session.start();
});

In your Twilio TwiML, pass the caller’s number to the stream so the tools know who is calling:

<Response>
  <Connect>
    <Stream url="wss://YOUR-HOST/media">
      <Parameter name="from" value="{{From}}" />
    </Stream>
  </Connect>
</Response>

Then wire the other entry points to the same stores:

• Inbound SMS → handleTwilioInbound (see sms-consent) with the same SqliteConsentStore, so STOP on any channel your ledger sees is honored by every later send.
• Instagram → createWebhookHandler (see instagram-dm) with your conversation handler calling the same agent core.
• Unsubscribe links → an endpoint you host that calls suppression.add(address, 'unsubscribed').

What to decide before you ship anything like this

• Consent is your responsibility. sms-consent records and enforces what you tell it. How consent was collected, what registrations your messaging traffic needs, and what your jurisdiction requires are outside the package, and nothing here is legal advice.
• Disclose automation where the platform or the law requires it, on calls, texts and DMs.
• Persistence. The demo uses in-memory stores. Consent, tokens, idempotency keys and suppression lists need durable storage in anything real.
• Single process. voice-engine keeps call state in memory. There is no reconnect or resumption.
• Provider drift. The live adapters follow vendor docs at the time of writing and are not tested against the live services in CI.
• Business integrations. For accounting and property data, see quickbooks-online-kit and fl-parcel-lookup; they plug in as more tools in the same executor.