Natural Language Understanding - C++

Introduction

VSDK includes built-in support for Natural Language Understanding (NLU) through the VNLU engine, developed by Vivoka. Integrated under the module name vsdk-vnlu, it is designed to work alongside an ASR engine to extract meaning and structure from spoken commands.

With VNLU, your application can go beyond recognizing words—it can understand intents (what the user wants to do) and entities (the relevant pieces of information within a sentence). This makes it ideal for building natural and conversational voice interfaces.

Example: Smart Home

User says: "Turn on the kitchen lights"

Extracted Intent:
TurnOnDevice

Extracted Entities:

• device: "lights"

• location: "kitchen"

Training model

NLU is domain-specific module, meaning it’s trained on the vocabulary and sentence structures relevant to your use case. We have a guide that could help you build you dataset and train your model.

Languages

• 🇫🇷 fra-FR – French (France)

• 🇺🇸 eng-US – English (United States)

• 🇮🇹 ita-IT – Italian (Italy)

• 🇪🇸 spa-ES – Spanish (Spain)

• 🇩🇪 deu-DE – German (Germany)

Getting Started

Before you begin, make sure you’ve completed all the necessary preparation steps.
There are two ways to prepare your project for integration of NLU:

1. Using sample code

2. Starting from scratch

From Sample Code

To download the sample code, you'll need Conan. All the necessary steps are outlined in the general Getting Started guide.

📦 voice-commands-language-understanding

conan search -r vivoka-customer voice-commands-language-understanding  # To get the latest version.
conan inspect -r vivoka-customer -a options voice-commands-language-understanding/<version>@vivoka/customer
conan install -if voice-commands-language-understanding voice-commands-language-understanding/<version>@vivoka/customer

From Scratch

Before proceeding, make sure you’ve completed the following steps:

1. Prepare your VDK Studio project

• Create a new project in VDK Studio

• Add the Natural Language Understanding technology

• Train model.

• Export the project to generate the required assets and configuration

2. Set up your project

• Install the necessary libraries

  • vsdk-audio-portaudio/<version>@vivoka/customer

  • vsdk-vnlu/<version>@vivoka/customer

These steps are better explained in the Get Started guide.

Start Parsing

#include <vsdk/global.hpp> 
#include <vsdk/nlu/ResultSerialization.hpp>
#include <vsdk/nlu/vnlu.hpp>

void onNluResult(Vsdk::Nlu::Parser::Result const & r)
{
    auto const result = Vsdk::Nlu::Result(r.json);
    if (result.intent.name)
    {
        std::unordered_map<size_t, std::reference_wrapper<Vsdk::Nlu::Entity const>> entities;
        std::string text;

        for (auto const & entity : result.entities)
            entities.emplace(entity.startIndex, entity);

        for (size_t pos = 0; pos < result.originalSentence.size(); )
        {
            auto const it = entities.find(pos);
            if (it != entities.cend())
            {
                Vsdk::Nlu::Entity const & entity = it->second;
                text += fmt::format("[{} ({})]", entity.value, entity.name);
                pos  += entity.value.size();
            }
            else
                text += result.originalSentence[pos++];
        }

        fmt::print("NLU Result: '{}' (confidence: {}) => {}\n",
                   result.intent.name.value(), result.intent.confidence, text);
    }
}

auto engine = Vsdk::Nlu::Engine::make<Vsdk::Nlu::Vnlu::Engine>("config/vsdk.json");
auto parser = engine->parser("model-name");

parser->subscribe(&onNluResult);

parser->process("I need a ticket from Paris to London.");

The model returns a confidence score for both the intent and each slot. Confidence values range from 0 to 1, with 1 indicating the highest certainty.