Conversational Plugins

5min
A Plugin enables the connection of a Digital Human to an external API or service. For instance, a plugin can facilitate integration with a conversational AI solution, utilize a specific LLM, access knowledge bases, or conduct internet searches.
A plugin communicates with the Unith Digital Humans Backend system to provide responses to user messages.
Plugin webhook
Unith plugins communicate with the backend following a webhook communication pattern.
The plugin should implement a specific POST endpoint /conversation/{userid}/message. The Unith backend will send events to this endpoint for each user message. The body of the POST request follows this format:
JSON
{

[
    {
        "type": "text",
        "payload": {
            "type": "text",
            "message": "Hello World!"
        }
    }
]

} 
{

[
    {
        "type": "text",
        "payload": {
            "type": "text",
            "message": "Hello World!"
        }
    }
]

} 
﻿
The plugin should respond to the POST request with an equivalent json format:
JSON
{

[
    {
        "type": "text",
        "payload": {
            "type": "text",
            "message": "this is the response from plugin"
        }
    }
]

} 
{

[
    {
        "type": "text",
        "payload": {
            "type": "text",
            "message": "this is the response from plugin"
        }
    }
]

} 
﻿
Note that the POST body and the response are formatted as a List. The aim of this List is to support several parts for each message, potentially of different type, like "images" or other types of data or metadata. At the moment the API only supports text.
Options
The plugin events can include optional parameters in the payload. These parameters can be used to provide extra metadata needed for the plugin. The options is a json object that is added to the payload
JSON
{

[
    {
        "type": "text",
        "payload": {
            "type": "text",
            "message": "Hello World!"
            "options": {
                "example_param": "foo"
            }
        }
    }
]

} 
{

[
    {
        "type": "text",
        "payload": {
            "type": "text",
            "message": "Hello World!"
            "options": {
                "example_param": "foo"
            }
        }
    }
]

} 
﻿
How to register a plugin with a Digital Human
A plugin can be registered with a Digital Human using the operationMode="plugin" through the API. In the Digital Human creation process, use the following parameters in the POST /head/create endpoint.
JSON
{
    ...
    "operationMode": "plugin",
    "pluginOperationalModeConfig": {
        "name": "<plugin name>",
        "url": "<plugin url>",
        "options": {
            "<optional param>": "option value"
            ...
        }
      }
    ...
} 
{
    ...
    "operationMode": "plugin",
    "pluginOperationalModeConfig": {
        "name": "<plugin name>",
        "url": "<plugin url>",
        "options": {
            "<optional param>": "option value"
            ...
        }
      }
    ...
} 
﻿
The parameter pluginOperationalModeConfig.url should be the full base path of the url preceding /conversation/{userid}/message. For instance, if the the plugin endpoint path is
https://myservice.example.com/api/conversation/{userid}/message
you have to set the url field as
JSON
"pluginOperationalModeConfig": {
      "name": "example",
      "url": "https://myservice.example.com/api/"
}
"pluginOperationalModeConfig": {
      "name": "example",
      "url": "https://myservice.example.com/api/"
}
﻿
Please see How to create a custom plugin﻿, where we provide a step by step tutorial that explains how to create a custom plugin using python.
Updated 09 Apr 2024
Did this page help you?
Live Video-Only Embedding
Embed using iframe