Intent Classification & Entity Extraction | AnveVoice

What is Intent Classification?

Intent classification maps user input to predefined action categories like check_order_status or request_refund, answering what a person wants from their voice AI interaction. Intent classification maps user input to a predefined action or category. Think of it as answering: "What does this person want?" // Example: Customer Service Intents "Where is my order?" → intent: check_order_status "I want a refund" → intent: request_refund "میرا آرڈر کہاں ہے؟" → intent: check_order_status "مجھے ریفنڈ چاہیے" → intent: request_refund "How do I reset my password?" → intent: account_help "Can I change my delivery address?" → intent: modify_order Before annotating anything, design your intent hierarchy carefully. For customer service:

• Designing Your Intent Taxonomy
• • order_status — Track/check order
• • request_refund — Want money back
• • modify_order — Change order details
• • cancel_order — Cancel an order
• • billing_inquiry — Payment questions
• • technical_support — Product issues
• • account_help — Login/password/profile
• • product_info — Product questions
• • complaint — Express dissatisfaction
• • greeting — Hello/hi
• • goodbye — End conversation
• • fallback — Unclear/out of scope
• ✅ Start with 10-15 intents — Don't over-segment. You can always split later.
• ✅ Use verb_noun format — check_status , request_refund , cancel_order
• ✅ Include a fallback intent — Catches everything your model doesn't understand
• ✅ Test for overlap — If annotators frequently disagree between two intents, merge them
• ✅ Balance your dataset — Don't have 500 examples for "greeting" and 20 for "refund"
• ⚠️ Avoid deep hierarchies — 2 levels max (category → sub-intent)

What is Entity Extraction (NER)?

Entity extraction identifies specific data points like names, dates, amounts, and products within user input, providing the details voice AI needs to take action. Entity extraction (Named Entity Recognition) identifies specific data points within user input. While intent tells you what they want, entities tell you the details . // Example: Entity Extraction "I ordered blue shoes on Monday and want a refund " → product: blue shoes | date: Monday | action: refund " میرے نیلے جوتے جو پیر کو آرڈر کیے تھے، واپس کرنا ہے" → product: نیلے جوتے | date: پیر | action: واپس Customer name, agent name #12345, ORD-789 Blue shoes, iPhone 15 Monday, last week, Feb 10 ₹500, $20, 2999 rupees Delhi, Lahore, Sydney Broken, late, wrong item Email, phone number 3 days, 2 weeks ago The standard annotation format for entity extraction is BIO tagging (Beginning, Inside, Outside): // BIO Tagging Example I → O ordered → O blue → B-PRODUCT shoes → I-PRODUCT on → O Monday → B-DATE B = Beginning of entity | I = Inside (continuation) | O = Outside (not an entity) 💡 Pro Tip for Urdu/Hindi: BIO tagging works the same for right-to-left scripts. Most annotation tools handle RTL automatically. The key difference is tokenization — use a language-specific tokenizer (like UrduHack or iNLTK) rather than whitespace splitting, since Urdu has complex word boundaries.

• Common Entity Types for Customer Service
• BIO Annotation Format

Step-by-Step Annotation Process

Follow this six-step annotation process to build high-quality intent classification and entity extraction datasets for low-resource languages.

• Collect Raw Data
• Define Annotation Guidelines
• Annotate with Intent + Entities Simultaneously
• Augment with Synthetic Data (LLMs)
• Validate with Inter-Annotator Agreement
• Balance and Split Your Dataset

Special Considerations for Urdu, Hindi & Low-Resource Languages

Low-resource languages like Urdu and Hindi require specialized tokenization, code-switching handling, and transfer learning strategies for voice AI. Urdu's Nastaliq script has complex ligatures and word boundaries. Use specialized tokenizers like UrduHack , Stanza (supports Urdu), or SentencePiece (subword tokenization). Avoid naive whitespace splitting — it breaks compound words and postpositions. South Asian users frequently mix languages: "Mera order track karna hai" (Hindi + English). Your dataset MUST include a substantial share of code-switched examples. Train a separate language-ID model or use multilingual embeddings (XLM-RoBERTa). Hindi appears in Devanagari AND Roman script. Urdu in Nastaliq AND Roman. Include BOTH script variants in your dataset. For Romanized text, normalize common variations: "kahan" / "kahaan" / "khn" should all map to the same tokens. Start with a multilingual base model: XLM-RoBERTa (best for NER), mBERT , or IndicBERT (optimized for Indian languages). Fine-tune on your annotated dataset. Transfer learning lets a few hundred examples per intent go much further than training from scratch. XLM-RoBERTa-large Best overall for NER + intent classification in 100+ languages IndicBERT / IndicNLP Optimized for Indian languages (Hindi, Marathi, Tamil, etc.) Coqui XTTS v2 Best for TTS fine-tuning on Urdu/Hindi (voice synthesis) Whisper large-v3 Best ASR for Urdu/Hindi speech-to-text MMS (Meta) 1100+ languages for both ASR and TTS

• 🔤 Tokenization Challenges
• 🔀 Code-Switching
• 📝 Script Variations
• 🤖 Transfer Learning Strategy
• Recommended Base Models for Urdu/Hindi

Complete Annotated Example: Urdu Customer Service

This complete Urdu customer service annotation example shows intent classification and entity extraction with BIO tagging in practice.

Tools & Resources

Use these annotation tools, datasets, research papers, and training frameworks to build production-ready intent classification and NER models.

• 📦 Annotation Tools
• • Label Studio — Open source, best for custom NER + intent. Supports RTL.
• • Prodigy — By spaCy team. Active learning cuts down how many examples you label by hand.
• • Doccano — Lightweight, open source. Good for small teams.
• • Argilla — ML-focused. Great for iterative annotation with model-in-the-loop.
• 📚 Datasets & Corpora
• • UrduHack — Urdu NLP toolkit (tokenization, normalization)
• • iNLTK — Indian NLP library (Hindi, Marathi, Bengali, etc.)
• • AI4Bharat IndicNLP — Pre-trained models for 11 Indian languages
• • Common Voice — Mozilla's multilingual speech dataset
• • MASSIVE — Amazon's 51-language intent + slot dataset
• 🔬 Research Papers
• • "Cross-lingual Transfer for NER in Low-Resource Languages" — ACL 2024
• • "MASSIVE: A 1M-Example Multilingual NLU Dataset" — Amazon Science
• • "IndicNLP Suite: Monolingual Corpora for Indian Languages" — AI4Bharat
• • "Zero-Shot Cross-Lingual NER" — using XLM-RoBERTa
• 🎯 Training Frameworks
• • Hugging Face Transformers — Fine-tune XLM-RoBERTa for NER
• • spaCy 3 — Production NER with custom training pipelines
• • Rasa NLU — Open source intent + entity pipeline
• • FastText — Lightweight intent classification baseline

Frequently Asked Questions

Find answers to common questions about intent classification, entity extraction, dataset annotation, and voice AI for low-resource languages.

Putting It All Together

Combine transfer learning, synthetic data generation, and careful annotation to build production-quality voice AI for low-resource languages. Building voice AI for low-resource languages is challenging but increasingly achievable. The combination of transfer learning from multilingual models, synthetic data generation with LLMs, and careful human annotation creates a powerful pipeline that can achieve production-quality results with relatively small datasets. The key is to start small, iterate quickly, and let your model's errors guide your annotation priorities. Don't try to annotate everything at once — build a minimum viable dataset, train a baseline model, analyze its mistakes, and annotate more examples where it struggles. AnveVoice uses advanced NLU to understand user intent and extract entities from voice commands — then takes real action on websites. Try it free and see how voice AI turns natural language into website navigation, form filling, and button clicks. Add voice AI to your website in 10 minutes

• Want to See Intent Classification + Entity Extraction in Action?
• 50+ languages, including Hindi
• Intent classification + entity extraction built-in
• Free tier: 50K tokens, no credit card required
• Try AnveVoice Free
• • 50K tokens free, no card
• • No credit card required
• • One-line embed