ElevenLabs used its New York Summit on 11 November to unveil 13 new products and upgrades that push the voice AI company well beyond voice generation. The package turns ElevenLabs into an enterprise AI platform, with a new workplace app, agent upgrades, model fine-tuning and a wider partner network.
The headline act is ElevenAI, a new app that works as an AI teammate for employees. It listens to spoken conversations, including in-person meetings, and turns them into notes, summaries and action items. Free early access opened with the announcement.
“Our mission at ElevenLabs is to solve human-technology interaction, and for the first time, it’s solvable.”
Mati Staniszewski, CEO and co-founder of ElevenLabs
ElevenAI listens so teams can stay in the meeting
ElevenAI is built for the workday rather than the call centre. It takes notes with the consent of everyone in the room, then delivers a summary and action items to each attendee when the call ends. It also handles dictation into any app and offers live assistance during a call.
The app runs on Scribe v3, the speech-to-text model ElevenLabs released on 10 November. ElevenLabs describes Scribe v3 as its most accurate transcription model, covering 102 languages, with a live variant for real-time audio. Consent and notice tools are built into the app rather than added as an afterthought.
What you can use today and what is still coming
Not everything on stage is ready to use. ElevenAI free early access is open now, and so are Scribe v3, Scribe v3 Live and Campaign Manager, the outbound calling tool ElevenLabs released on 9 November. The rest of the package is a preview, with no dates attached.
The previews are the Interaction Engine, Memory and Routines for ElevenAgents, plus model fine-tuning for enterprises and a financial-services model. ElevenLabs says the fine-tuning work will begin in the coming months, with selected enterprises first. No pricing has been disclosed for any of it.
Agents that remember, route and follow through
The Interaction Engine manages the rhythm of a conversation. It handles turn-taking and interruptions, then routes each part of an exchange to the right model. A cheaper model can handle simple triage while a frontier model steps in for complex work such as refunds.
Memory gives ElevenAgents a record of returning customers. It captures details that never had a field in the CRM, such as a preference for Spanish or a move planned for March. Routines handle the work around the conversation, from pre-screening a mortgage applicant to writing back to the CRM.
Your data becomes the moat
The most consequential announcement may be the least visible. Enterprises will be able to bring data from their live deployments and work with ElevenLabs researchers to fine-tune open-source language models and ElevenLabs voice models. Each business picks its own target metrics, such as customer satisfaction or reliability.
Staniszewski compared the setup to each company having its own reinforcement-learning stack. The same thinking sits behind Scribe v3 Medical, a healthcare transcription model that tells apart drug names such as cycloserine and cyclosporine. A financial-services model is in the works.
NVIDIA, McKinsey and a marketplace widen the push
ElevenLabs also widened its enterprise ecosystem on stage. Its partner network now counts more than 550 partners across 44 countries, including Google Cloud, IBM, Deloitte, DXC and Genesys. A new Partner Program and Marketplace will help enterprises deploy agents through consultancies and buy partner-built apps.
The company announced an alliance with McKinsey and QuantumBlack, AI by McKinsey, pairing its agents with enterprise transformation work. It also expanded its NVIDIA partnership, with Nemotron 3.5 Super VL now available in ElevenAgents and 15 MW of new Vera Rubin compute capacity on the way.
The two companies will also build an on-premises voice agent for Poland’s national e-Health platform. It will handle appointment scheduling for patients who only have landlines. The Marketplace starts with curated apps for healthcare, financial services, insurance, education and retail.
From voice tool to infrastructure
ElevenLabs started in 2022 as a voice generation company founded by Mati Staniszewski and Piotr Dabkowski, two childhood friends from Poland. Four years later it is pitching itself as the interaction layer for AI, the connective tissue between businesses and their customers and employees.
“Within a few years, conversational agents will become critical infrastructure for every enterprise, as essential as cloud or payments.”
Mati Staniszewski, CEO and co-founder of ElevenLabs
The ambition is real and the early access is free, so ElevenAI is easy to test this week. The harder question is how much of the Summit package ships on schedule. Interaction Engine, Memory, Routines and fine-tuning are all previews, and the company has given none of them a date.
