100+ calls a day, answered: an AI voice agent for a clothing wholesaler

We built an AI voice agent that answers 100+ phone calls a day for a clothing wholesaler in Hyderabad. The callers run their own clothing shops all over India, and they call about everything: whether a design is in stock, what's new, what it costs, where their parcel is, and above all, when they can get on a video call to order.

The busiest days are when rare pieces or limited editions arrive. Retailers all over India race to claim them and call again and again, and answering every call, following up and keeping track of who wants what had become very hard for the staff. A person can take one call at a time. The agent takes many at once.

The video call is the other reason the phone never stops. For a retailer, coming to Hyderabad in person means losing a full day of business at their own shop. A video call with the wholesaler's staff lets them order from behind their own counter.

So for these retailers the phone line is the shop floor, and it was ringing all day. The agent now answers it, and it does a fair share of the selling. This is how it works, and how it plugs into the staff console we had already built for the shop.

#Why not just sell online?

It's the first question people ask me. This isn't a clothing brand selling single pieces. The wholesaler only sells in bulk, with a minimum order of ₹25,000, and new stock sells out so fast that there isn't time to photograph and list every piece online before it's gone.

So retailers prefer a video call with the staff: they see the stock as it is that day, discuss the deal, negotiate and close it on the call. The video call is the sales channel, and the phone is how retailers get onto it.

#What the agent does on a call

• Takes many calls at once, so a rush for a new arrival doesn't leave retailers waiting for a free line.
• Speaks the caller's language. Callers come from many states: Telugu, Hindi, Tamil, Kannada, Marathi, Gujarati, English and more, with dialects, often switching mid-conversation.
• Checks stock the way retailers ask for it: by the code on a tag, a colour or a design, and says whether it's in stock, sold out or how many pieces are left.
• Pitches the latest arrivals, then sends the photos on WhatsApp after the call.
• Quotes wholesale prices, but only after checking that the caller's shop is verified with its GST registration (India's business tax ID).
• Books the video call with the staff, notes which designs and quantities to show, and moves or cancels it when plans change.
• Handles after-sales: answers "where's my parcel?" from the order's progress, and logs a UPI payment reference (India's instant bank transfer) on the order for staff to check.
• Calls out, too: it rings retailers whose cart went quiet, and buyers after delivery to ask how the order went.
• Hands over: after every call, a formatted note of who called and what they want lands in the staff console.
• The basics: where the shop is and when it's open.

#The system design

The agent runs on Sarvam AI's voice agent platform. Sarvam builds its speech models for Indian languages: in its own words, its speech recognition handles Indian accents, dialects and language switches in the middle of a sentence. That is exactly what these calls sound like.

Under the hood it's a cascaded voice pipeline: speech recognition (ASR), an LLM, then text to speech (TTS), with the LLM running a tool-calling loop in the middle. Every turn of the conversation goes through it:

1. Listen. Sarvam's speech to text turns the caller's words into text, in whatever language they spoke.
2. Understand. A Sarvam LLM works out what the caller wants, and whether it needs to look something up or do something.
3. Look up or book. If it does, the agent makes a tool call to the staff console's API.
4. Reply. Sarvam's Bulbul voices say the answer out loud, in the caller's language.

When the call ends, a structured hand-off runs once: the agent sends the console a formatted note of who called and what they want.

The agent never gets the database. It gets a small set of API calls: check stock and new arrivals, read prices for a verified shop, read an order's progress, find a free video call slot, add, move or cancel a booking, log a payment reference, and send a note to the team. Everything it can do to the business goes through the same console the staff use, so there is one record of every booking, whoever made it.

#Where the stock answer comes from

The agent answers stock questions from the console's copy of the shop's inventory system (their ERP). Today the ERP hands stock over as a file import, because the ERP hasn't given us API credentials yet. Each import is a run in the console with its own errors kept, so the team can always see when the numbers were last true. An API connection comes once the ERP provides credentials.

That's the honest limit of the stock answer today: it is only as fresh as the last import.

#The staff side

Bookings land on the console's call day: every booked video call, in order. Each one opens a call card with everything the person on the call needs: who the buyer is, whether their shop is verified, their cart with photos, the items they want to discuss, and what they ordered before.

The call itself is dialled by hand on WhatsApp, from the shop's own number, because WhatsApp's API can't start a video call. Calls run from 1 pm to 7 pm IST, one at a time. The slot grid lives in the database, so a slot can't be double booked, and every call ends in an outcome or an order.

The console is a TanStack Start app on Vercel in Mumbai, with a Supabase Postgres database, and I built it with Claude Code.

#The engineering principles behind it

In the language of Anthropic's Building Effective Agents, this is an augmented LLM: a model with tools and retrieval around it. The model was the easy part. Most of the care goes into what that essay calls the agent-computer interface: which tools the model gets, and the rules behind them.

• Treat the LLM as an untrusted client. It never touches the database. It gets a few scoped tools on the console's API, the same API the staff work through. Least privilege, applied to a model.
• Invariants live below the model. The slot grid is enforced in the database, so a slot can't be double booked, whatever happens in the conversation.
• Ground every factual answer. Stock, prices and order status come from tool calls into the system of record, never from the model's memory.
• Guardrails before sensitive actions. Wholesale prices are quoted only after the shop's GST registration has been verified.
• One system of record. Bookings, call notes and payment references from the agent land in the same console as everything the staff do, so each shop has one history.
• Human in the loop where the money is. The agent qualifies and books. Staff negotiate and close on the video call, and check every payment reference against the bank.
• Concurrency is a feature. A person takes one call at a time. The agent takes many, which is exactly what a rush for limited editions needs.
• Be honest about freshness. The stock answer is only as fresh as the last ERP import, and the console shows when that was.

#Why a person still takes the call

The agent does a lot of the early selling: it checks stock, pitches what's new, quotes prices and sends photos. But the deal itself is done on the video call, by people. That's where retailers see the stock, negotiate and close, and the staff know the stock, the regulars and the trade.

I think that is the right split for a business like this one: let the agent take the 100+ calls a day, so the team's time goes into the video calls that turn into orders.

#The result

The agent is live on the shop's own number and answers 100+ calls a day, in the caller's own language. I don't have a clean before and after number to share yet, so I'm not going to invent one.

If you're building voice agents for Indian languages, or you run a business whose phone never stops ringing, I'd love to hear from you.