Visitor Chat

Visitor Chat enables third-party applications to integrate SecondMe avatar conversations. Your users (logged in or anonymous) can chat with any SecondMe avatar in real time. The other party may be an AI avatar, or the avatar's owner replying in person via the SecondMe app.

Base URL: https://api.mindverse.com/gate/lab

Overview

Visitor Chat works through two HTTP calls plus one long-lived WebSocket connection:

1. Initialize (POST /visitor-chat/init) — Authenticate, create session, return WebSocket credentials
2. Send message (POST /visitor-chat/send) — Send text and trigger a reply
3. Receive messages (WebSocket) — Connect directly to the wsUrl returned from init. All inbound messages (AI streaming frames, human takeover replies, voice notifications) arrive over this socket.

Identity Modes

Authenticated Flow (2 steps)

Use the logged-in user's access token directly:

1. POST /visitor-chat/init (Authorization: Bearer userToken, body: {apiKey})
2. POST /visitor-chat/send (Authorization: Bearer userToken, body: {sessionId, apiKey, message})

Anonymous Flow (3 steps)

Your backend obtains an app-level token first, then calls on behalf of the anonymous user:

1. POST /oauth/token/client  → Get app token (cacheable for 7 days)
2. POST /visitor-chat/init   → Initialize chat (visitorId required)
3. POST /visitor-chat/send   → Send message

Security: client_secret must only be used on your backend — never expose it to the frontend. Proxy anonymous frontend calls through your own backend API.

Get App Token (Anonymous Mode Only)

Use client_credentials grant to obtain an application-level access token. This token represents your application (not a specific user) and is used for anonymous visitor scenarios.

POST /api/oauth/token/client

Request

curl -X POST "https://api.mindverse.com/gate/lab/api/oauth/token/client" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "grant_type=client_credentials&client_id=YOUR_CLIENT_ID&client_secret=YOUR_CLIENT_SECRET&scope=chat.write"

Response

{
  "code": 0,
  "data": {
    "accessToken": "lba_at_xxx...",
    "tokenType": "Bearer",
    "expiresIn": 604800,
    "scope": ["chat.write"]
  }
}

Initialize Chat

Creates a visitor chat session and returns WebSocket credentials.

POST /api/secondme/visitor-chat/init

Authentication

Requires an OAuth2 Token (authorization_code or client_credentials).

Headers

Request Parameters

Request Examples

Authenticated user:

curl -X POST "https://api.mindverse.com/gate/lab/api/secondme/visitor-chat/init" \
  -H "Authorization: Bearer lba_at_user_access_token" \
  -H "Content-Type: application/json" \
  -d '{
    "apiKey": "sk-your-avatar-api-key"
  }'

Anonymous user:

curl -X POST "https://api.mindverse.com/gate/lab/api/secondme/visitor-chat/init" \
  -H "Authorization: Bearer lba_at_app_token" \
  -H "Content-Type: application/json" \
  -d '{
    "apiKey": "sk-your-avatar-api-key",
    "visitorId": "device_abc123",
    "visitorName": "Alice"
  }'

The avatar center will display {visitorName}({appName}) (e.g., "Alice(My App)"), where appName is automatically fetched from your registered application info.

Response

{
  "code": 0,
  "data": {
    "sessionId": "6ff56704-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
    "wsUrl": "wss://ws.mindos.com/os/ws?wsId=ws:xxx&authBody=yyy",
    "avatarName": "My Avatar",
    "opening": "Hello! How can I help you?"
  }
}

Important: wsUrl already contains wsId and authBody as query parameters. Extract wsId via new URL(wsUrl).searchParams.get("wsId") — you'll need it for the heartbeat protocol (see below).

Send Message

Sends a text message to the current session. The reply is pushed asynchronously via WebSocket.

POST /api/secondme/visitor-chat/send

Authentication

Requires an OAuth2 Token (same token as init).

Request Parameters

If the token expires or the cache is invalidated, /send automatically recovers the session using the current OAuth token + apiKey. You don't need to call /init again.

Request Example

curl -X POST "https://api.mindverse.com/gate/lab/api/secondme/visitor-chat/send" \
  -H "Authorization: Bearer lba_at_your_token" \
  -H "Content-Type: application/json" \
  -d '{
    "sessionId": "6ff56704-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
    "apiKey": "sk-your-avatar-api-key",
    "message": "Hello, who are you?"
  }'

Response

{
  "code": 0,
  "data": {
    "sent": true
  }
}

This response only indicates that Labs has accepted your message and forwarded it to SecondMe. The actual reply is pushed asynchronously via WebSocket, not returned here.

WebSocket Message Format

After connecting to wsUrl, all messages are pushed via the WebSocket. Clients must classify messages by the type field.

Message Type Overview

Business Message (type: "msg") Full Fields

{
  "type": "msg",
  "sender": "umm",
  "sendUserId": "dbfj9",
  "messageId": "c70824b9-bc73-48ad-bc8d-d7319e00a2f7",
  "sessionId": "a791bb21...",
  "index": 0,
  "dataType": "text",
  "audioPlayable": true,
  "data": {
    "content": "Hello! I am Wuyuxiang",
    "msgDataType": "text"
  },
  "multipleData": [
    { "singleDataType": "text", "modal": { "answer": "Hello! I am Wuyuxiang" } }
  ],
  "channel": "iOS"
}

Distinguishing AI / Human Takeover / Your Own Echo

Key rule: The combination of sender + sendUserId determines the message source.

Semantic note: The avatar owner can open their SecondMe app at any time, view visitor conversations, and manually reply. In that case the message has sender: "client" (because it is sent from a human client), but the sendUserId is the owner's, not yours — this is the only way to distinguish "human takeover" from "echo bounce-back".

How to identify "your own sendUserId"

The init response does not currently return visitorUserId (it is kept as an internal cache field inside labs). Recommended approach:

Learning-based filter:

1. When your client sends a message, store lastSentContent = message locally.
2. When the first sender === "client" message arrives with data.content === lastSentContent, record its sendUserId as your own.
3. Thereafter, any sender === "client" && sendUserId === selfSendUserId message is an echo and should be skipped.
4. Any sender === "client" && sendUserId !== selfSendUserId message is a human takeover — render it.

let selfSendUserId = null;
let lastSentContent = null;

function send(message) {
  lastSentContent = message;
  // Call /visitor-chat/send ...
}

function handleWsMessage(msg) {
  if (msg.type !== "msg") return;
  if (msg.sender === "client") {
    if (!selfSendUserId && msg.data?.content === lastSentContent) {
      selfSendUserId = msg.sendUserId;  // learn
      return;                            // echo, skip
    }
    if (msg.sendUserId === selfSendUserId) return;  // known echo
    // Otherwise owner takeover, continue rendering
  }
  renderAssistantBubble(msg.data?.content, msg.messageId);
}

Streaming Reply Assembly

An AI reply usually arrives as several streaming frames, followed by an end frame with index: -1:

Frame 1: { sender: "umm", messageId: "abc", index: 0,  data: { content: "Hello" } }
Frame 2: { sender: "umm", messageId: "abc", index: 1,  data: { content: "Hello, I" } }
Frame 3: { sender: "umm", messageId: "abc", index: 2,  data: { content: "Hello, I am Wuyuxiang" } }
Frame 4: { sender: "umm", messageId: "abc", index: -1, data: { content: "" } }   ← end frame

Important: data.content is typically the full accumulated text (not a delta). Clients should overwrite the bubble content instead of concatenating. Use messageId as the bubble key.

Human takeover messages are usually single complete messages with no -1 end frame and index fixed at 0. Clients should finalize the bubble after ~5 seconds of idle time.

Voice Messages (type: "notice" + messageAudioReady)

After an AI reply completes, the backend asynchronously pushes a separate notification frame carrying the voice URL:

{
  "type": "notice",
  "messageId": "c70824b9-...",
  "data": {
    "sourceType": "messageAudioReady",
    "sourceAction": "ready",
    "sourceCustom": {
      "messageId": "c70824b9-...",
      "sendUserId": "dbfj9",
      "audioUrl": "https://object.me.bot/chat_audio/.../1775807567706.mp3",
      "audioDurationMs": 2120
    }
  }
}

Handling recommendation:

1. Locate the existing AI text bubble by data.sourceCustom.messageId
2. Attach audioUrl / audioDurationMs to that bubble
3. Render a play button below the bubble, playing via new Audio(audioUrl).play()
4. Edge case: messageAudioReady may arrive before the msg end frame. Buffer it in a pendingAudio Map and merge when the text arrives.

const pendingAudio = new Map();

function handleNotice(msg) {
  if (msg.data?.sourceType !== "messageAudioReady") return;
  const sc = msg.data.sourceCustom;
  if (!sc?.messageId || !sc.audioUrl) return;

  const existing = findBubbleByMessageId(sc.messageId);
  if (existing) {
    existing.audioUrl = sc.audioUrl;
    existing.audioDurationMs = sc.audioDurationMs;
  } else {
    pendingAudio.set(sc.messageId, { url: sc.audioUrl, durationMs: sc.audioDurationMs });
  }
}

function createBubble(messageId, content) {
  const bubble = { id: messageId, content };
  const pending = pendingAudio.get(messageId);
  if (pending) {
    bubble.audioUrl = pending.url;
    bubble.audioDurationMs = pending.durationMs;
    pendingAudio.delete(messageId);
  }
  return bubble;
}

Heartbeat Protocol

The WebSocket connection requires the client to send periodic heartbeat frames, otherwise the server will close the connection.

Protocol

// Send a ping every 5 seconds
setInterval(() => {
  if (ws.readyState === WebSocket.OPEN) {
    ws.send(JSON.stringify({ type: "ping", wsId: "ws:..." }));
  }
}, 5000);

Reset rule: Every time the client receives any WebSocket message (business frame / notice / pong), reset the "missed response" counter — any message proves the connection is alive.

Session Lifecycle & Reconnection

Recommended Reconnection Strategy

• On ws.onclose with a non-manual code (code !== 3000), schedule a reconnect
• Use exponential backoff: delay 2^N seconds for the Nth attempt, up to 5 attempts
• window.addEventListener("online", reconnect) to reconnect immediately when network recovers
• On document.visibilitychange === "visible", check ws.readyState and reconnect if needed

Complete Example

// 1. Get app token (anonymous mode, backend only)
const tokenRes = await fetch("https://api.mindverse.com/gate/lab/api/oauth/token/client", {
  method: "POST",
  headers: { "Content-Type": "application/x-www-form-urlencoded" },
  body: new URLSearchParams({
    grant_type: "client_credentials",
    client_id: process.env.CLIENT_ID,
    client_secret: process.env.CLIENT_SECRET,
    scope: "chat.write",
  }),
});
const { data: tokenData } = await tokenRes.json();
const appToken = tokenData.accessToken;

// 2. Initialize chat
const initRes = await fetch("https://api.mindverse.com/gate/lab/api/secondme/visitor-chat/init", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${appToken}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    apiKey: "sk-your-avatar-api-key",
    visitorId: "web_" + crypto.randomUUID(),
    visitorName: "Visitor",
  }),
});
const { data } = await initRes.json();

// 3. Connect WebSocket
const wsId = new URL(data.wsUrl).searchParams.get("wsId");
const ws = new WebSocket(data.wsUrl);

let selfSendUserId = null;
let lastSentContent = null;
const pendingAudio = new Map();
const bubbles = new Map();  // messageId -> { content, audioUrl? }

ws.onopen = () => {
  // Heartbeat
  setInterval(() => {
    if (ws.readyState === WebSocket.OPEN) {
      ws.send(JSON.stringify({ type: "ping", wsId }));
    }
  }, 5000);

  // Show opening
  if (data.opening) {
    renderBubble("opening", data.opening);
  }
};

ws.onmessage = (event) => {
  const msg = JSON.parse(event.data);

  // Protocol frames
  if (["ping", "pong", "ack"].includes(msg.type)) return;
  if (msg.type === "hint") return; // AI pre-thinking placeholder

  // Voice notification
  if (msg.type === "notice") {
    if (msg.data?.sourceType === "messageAudioReady") {
      const sc = msg.data.sourceCustom;
      const bubble = bubbles.get(sc.messageId);
      if (bubble) {
        bubble.audioUrl = sc.audioUrl;
        bubble.audioDurationMs = sc.audioDurationMs;
        updateBubble(sc.messageId, bubble);
      } else {
        pendingAudio.set(sc.messageId, {
          url: sc.audioUrl,
          durationMs: sc.audioDurationMs,
        });
      }
    }
    return;
  }

  // Business messages
  if (msg.type !== "msg") return;
  if (msg.index === -1) {
    finalizeBubble(msg.messageId);
    return;
  }

  const content = msg.data?.content || msg.multipleData?.[0]?.modal?.answer;
  if (!content) return;

  // Echo filter
  if (msg.sender === "client") {
    if (!selfSendUserId && content === lastSentContent) {
      selfSendUserId = msg.sendUserId;
      return;
    }
    if (msg.sendUserId === selfSendUserId) return;
    // Otherwise owner takeover, render it
  }

  // Create or update bubble
  const bubble = bubbles.get(msg.messageId) || { content: "" };
  bubble.content = content;
  const pending = pendingAudio.get(msg.messageId);
  if (pending) {
    bubble.audioUrl = pending.url;
    bubble.audioDurationMs = pending.durationMs;
    pendingAudio.delete(msg.messageId);
  }
  bubbles.set(msg.messageId, bubble);
  renderBubble(msg.messageId, bubble.content, bubble.audioUrl);
};

// 4. Send a message
async function sendMessage(message) {
  lastSentContent = message;
  await fetch("https://api.mindverse.com/gate/lab/api/secondme/visitor-chat/send", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${appToken}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      sessionId: data.sessionId,
      apiKey: "sk-your-avatar-api-key",
      message,
    }),
  });
}

Gotchas

1. The other party may be a real human: The counterpart in visitor chat is not necessarily an AI. The avatar owner can take over via the app at any time — in that case sender is "client" (same as echo), and you must use sendUserId to tell them apart.

2. Content field priority: data.content is the primary field; multipleData[0].modal.answer is legacy. Read the former first, fall back to the latter.

3. content is full text, not a delta: For streaming AI replies, each index frame's data.content is the full accumulated text. Overwrite the bubble content; do not concatenate.

4. Filter hint frames: Messages with type: "hint" are AI pre-thinking placeholders (e.g., "Let me check..."), not actual replies. Do not render them.

5. messageId is the bubble unique key: Use messageId as the React component key / database primary key. All frames of one reply (including the later voice notification) share the same messageId.

6. Voice is pushed asynchronously: The AI text arrives first; if audioPlayable: true, a messageAudioReady notification carrying the audioUrl arrives 1–3 seconds later. Handle the edge case where the notification arrives before the text.

7. Heartbeat is required: Without {type:"ping"} pings from the client, the server will close the connection. See the heartbeat protocol section.

8. wsUrl expires in 60 seconds: Call new WebSocket(wsUrl) immediately after init — don't delay.

9. visitorId must be persistent: The same visitorId reuses the session, so clients should store a stable visitorId in localStorage / cookie — don't generate a new one on every refresh.

10. Persist message history locally: Human replies may come hours later. If the user closes the page and comes back, restore the message list from localStorage. Labs does not provide a history-fetch API.

Error Codes

FAQ

What can I use as visitorId?

Any persistent user identifier you have — device fingerprint, user ID from your own database, or a random UUID. Reusing the same visitorId reuses the existing session.

Should I cache the client_credentials token?

Yes. Its lifetime is 7 days — cache it and refresh on expiry.

What to do if the WebSocket disconnects?

Call visitor-chat/init again — you'll get a new wsUrl (the same visitorId reuses the same sessionId). See the Session Lifecycle & Reconnection section.

How do I tell whether the other party is AI or human?

Technically: sender === "umm" is AI, sender === "client" && sendUserId !== self is owner human. From the visitor's UX perspective you usually don't need to surface this — to them, the counterpart is just "the avatar".

Note: Do not use the channel field to determine message origin. channel is a platform tag filled by the sender client itself — not every client fills it. Messages sent via labs visitor-chat/send don't carry channel; messages from the SecondMe iOS App carry channel: "iOS" — but that only means the sender is on iOS, it does not mean "messages from iOS come from the owner". Message origin can only be determined via the sender + sendUserId combination.

Does the voice URL expire?

The audioUrl in the messageAudioReady notification is a directly-accessible CDN URL. Clients should cache or play it promptly — do not rely on it being available long-term.

Can I fetch message history?

Labs visitor-chat does not currently provide a history-fetch API. Clients should persist the message list in localStorage themselves.