Skip to content

fix(voice): create ElevenLabs tools before agent setup#464

Open
Shujakuinkuraudo wants to merge 1 commit into
tiann:mainfrom
Shujakuinkuraudo:fix/elevenlabs-tool-ids-only
Open

fix(voice): create ElevenLabs tools before agent setup#464
Shujakuinkuraudo wants to merge 1 commit into
tiann:mainfrom
Shujakuinkuraudo:fix/elevenlabs-tool-ids-only

Conversation

@Shujakuinkuraudo
Copy link
Copy Markdown
Contributor

Summary

  • replace deprecated ElevenLabs prompt.tools usage with prompt.tool_ids
  • auto-create or reuse the required client tools before creating/updating the Hapi voice agent
  • keep the shared web helper aligned with the new ElevenLabs tool-based agent setup

Testing

  • verified createOrUpdateHapiAgent() succeeds with an ElevenLabs key that has ConvAI write access
  • verified the resulting ElevenLabs agent includes messageCodingAgent and processPermissionRequest
  • built the Linux x64 single-executable binary successfully via bun run build:single-exe

Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings

  • [Major] Auto-discovery now hard-fails without ConvAI write scope — getOrCreateAgentId() now treats tool creation and agent patching as mandatory before it will return an existing agent ID, so installations that rely on ELEVENLABS_API_KEY only can start returning Failed to create ElevenLabs agent automatically even when the agent already exists and token minting would otherwise be possible. Evidence hub/src/web/routes/voice.ts:203, hub/src/web/routes/voice.ts:217.

Summary

  • Review mode: initial
  • 1 finding. The new auto-provisioning path turns token issuance into a write operation for existing agents, which is a regression for accounts without ConvAI write permission.
  • Residual risk: Not found in repo/docs: automated coverage for the write-denied / existing-agent path.

Testing

  • Not run (automation; untrusted PR code)

HAPI Bot

Comment thread hub/src/web/routes/voice.ts
Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings

  • [Major] Existing auto-discovered agents now require tool write access — getOrCreateAgentId() calls ensureHapiToolIds() before it even looks for an existing Hapi Voice Assistant, and then returns null if patching that existing agent fails. That makes /api/voice/token fail for deployments with a usable existing agent but an ElevenLabs key that cannot create/list tools or patch agents, even though token minting only needs the agent id. Evidence hub/src/web/routes/voice.ts:205, hub/src/web/routes/voice.ts:217.
    Suggested fix:
    const existingAgentId = await findHapiAgent(apiKey)
    if (existingAgentId) {
        try {
            const toolIds = await ensureHapiToolIds(apiKey)
            await updateHapiAgent(apiKey, existingAgentId, toolIds)
        } catch (error) {
            console.warn('[Voice] Using existing agent without auto-update:', error)
        }
        agentIdCache.set(cacheKey, existingAgentId)
        return existingAgentId
    }
    
    const toolIds = await ensureHapiToolIds(apiKey)
    const agentId = await createHapiAgent(apiKey, toolIds)

Questions

  • None.

Summary

  • Review mode: follow-up after new commits
  • 1 major issue remains: token issuance still hard-fails on the new tool provisioning/update path before reusing an existing auto-discovered agent.
  • Residual risk: Not found in repo/docs: coverage for an existing-agent path where ElevenLabs tool creation or agent patching is denied.

Testing

  • Not run (automation; untrusted PR code)

HAPI Bot


let toolIds: string[]
try {
toolIds = await ensureHapiToolIds(apiKey)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR] This still makes the auto-discovery path depend on ConvAI tool/agent write access. A deployment can already have a usable Hapi Voice Assistant, but this new ensureHapiToolIds() call runs before findHapiAgent() and returns null on any tool list/create failure; the later update failure also returns null. /api/voice/token then emits Failed to create ElevenLabs agent automatically instead of minting a token for the existing agent.

Suggested fix:

const existingAgentId = await findHapiAgent(apiKey)
if (existingAgentId) {
    try {
        const toolIds = await ensureHapiToolIds(apiKey)
        await updateHapiAgent(apiKey, existingAgentId, toolIds)
    } catch (error) {
        console.warn('[Voice] Using existing agent without auto-update:', error)
    }
    agentIdCache.set(cacheKey, existingAgentId)
    return existingAgentId
}

const toolIds = await ensureHapiToolIds(apiKey)
const agentId = await createHapiAgent(apiKey, toolIds)

@tiann tiann force-pushed the fix/elevenlabs-tool-ids-only branch from c1ead75 to 567bc0b Compare May 21, 2026 02:26
Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings

  • [Major] Existing agents still cannot be reused if tool provisioning or patching fails — /api/voice/token only needs an agent id to mint the conversation token, but getOrCreateAgentId() now runs ensureHapiToolIds() before findHapiAgent() and returns null on any tool list/create failure; it also returns null if patching an already-found agent fails. A deployment with a usable existing Hapi Voice Assistant and a key that can read agents/token but cannot manage ConvAI tools/agent config will now fail token issuance. Evidence hub/src/web/routes/voice.ts:205.
    Suggested fix:
    const existingAgentId = await findHapiAgent(apiKey)
    if (existingAgentId) {
        try {
            const toolIds = await ensureHapiToolIds(apiKey)
            const updated = await updateHapiAgent(apiKey, existingAgentId, toolIds)
            if (!updated) {
                console.warn('[Voice] Using existing agent without auto-update')
            }
        } catch (error) {
            console.warn('[Voice] Using existing agent without auto-update:', error)
        }
        agentIdCache.set(cacheKey, existingAgentId)
        return existingAgentId
    }
    
    const toolIds = await ensureHapiToolIds(apiKey)
    const agentId = await createHapiAgent(apiKey, toolIds)

Questions

  • None.

Summary

  • Review mode: follow-up after new commits
  • 1 major issue remains: the server token path still hard-fails on the new tool provisioning/update work before falling back to an existing auto-discovered agent.
  • Residual risk/testing gap: Not found in repo/docs: coverage for reusing an existing ElevenLabs agent when tool list/create or agent patch permissions are denied.

Testing

  • Not run: bun typecheck could not start because bun is not installed in this runner.

HAPI Bot


let toolIds: string[]
try {
toolIds = await ensureHapiToolIds(apiKey)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR] This still blocks the existing-agent fallback on the new tool-management path. /api/voice/token only needs an agent id, but this ensureHapiToolIds() call runs before findHapiAgent() and returns null on any tool list/create failure; the later updateHapiAgent() failure also returns null. Deployments with a usable existing Hapi Voice Assistant but an ElevenLabs key that cannot manage ConvAI tools/agent config will fail token issuance instead of using the existing agent.

Suggested fix:

const existingAgentId = await findHapiAgent(apiKey)
if (existingAgentId) {
    try {
        const toolIds = await ensureHapiToolIds(apiKey)
        const updated = await updateHapiAgent(apiKey, existingAgentId, toolIds)
        if (!updated) {
            console.warn('[Voice] Using existing agent without auto-update')
        }
    } catch (error) {
        console.warn('[Voice] Using existing agent without auto-update:', error)
    }
    agentIdCache.set(cacheKey, existingAgentId)
    return existingAgentId
}

const toolIds = await ensureHapiToolIds(apiKey)
const agentId = await createHapiAgent(apiKey, toolIds)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant