Create a custom plugin

7min
In this step by step tutorial we will show how to create a custom Plugin﻿ and how to use with a Digital Human.
In this tutorial we will implement a simple plugin that responds to user messages with a fixed "Hello user". In order to keep the example simple, we'll deploy the server locally and expose it using ngrok. For production you will need to host the server somewhere. Note that we'll use python and FastAPI but you can use any language or framework if that's more convenient.  To achieve that we'll do the following:
Implement a FastAPI server with a POST endpoint that will parse the input message and generate the reponse.
Run the server locally and use ngrok to expose the plugin publicly.
Create a digital human that uses the plugin.
1. The server
To implement the server using FastAPI you need the python packages that you can install with pip:
Text
pip install fastapi
pip install "uvicorn[standard]"
pip install fastapi
pip install "uvicorn[standard]"
﻿
Then create a file main.py with:
Python

from typing import Union, List, Annotated
from fastapi import FastAPI, Path, Body
from pydantic import BaseModel

class Payload(BaseModel):
    type: str
    message: str

class PluginMessage(BaseModel):
    type: str
    payload: Payload

app = FastAPI()

@app.post("/conversation/{userid}/message")
def post_message(userid: Annotated[str, Path()], 
    body: Annotated[List[PluginMessage], Body()]):
    
    user_message = ""
    if len(body):
        body = body[0]
        user_message = body.payload.message
    plugin_response = [
        PluginMessage(
        type="text",
        payload=Payload(type="text", message=f"Hello User, you said {user_message}!"))
        ]
    return plugin_response


from typing import Union, List, Annotated
from fastapi import FastAPI, Path, Body
from pydantic import BaseModel

class Payload(BaseModel):
    type: str
    message: str

class PluginMessage(BaseModel):
    type: str
    payload: Payload

app = FastAPI()

@app.post("/conversation/{userid}/message")
def post_message(userid: Annotated[str, Path()], 
    body: Annotated[List[PluginMessage], Body()]):
    
    user_message = ""
    if len(body):
        body = body[0]
        user_message = body.payload.message
    plugin_response = [
        PluginMessage(
        type="text",
        payload=Payload(type="text", message=f"Hello User, you said {user_message}!"))
        ]
    return plugin_response
﻿
This server implements a POST endpoint that validates the input message and creates and returns the response message that includes the input message.
2. Run locally and expose using ngrok
To run the server locally launch the following command from bash:
Bash
uvicorn main:app --reload
uvicorn main:app --reload
﻿
At this point the server is running locally at http://127.0.0.1:8000
Now we will need a public endpoint for this plugin to register it with the digital human. In a production setting you will need to deploy the server somewhere and host it. But for the sake of the tutorial, we will just use a ngrok, that will be useful for testing.
First create a ngrok account and install it following the instructions here.
Then in a new terminal add the token provided in your ngrok account
Text
ngrok config add-authtoken <token>
ngrok config add-authtoken <token>
﻿
Then run it providing the 8000 port
Text
ngrok http 8000
ngrok http 8000
﻿
This will give us a public URL that should look something like this
Text
https://0246-140-228-56-51.ngrok-free.app
https://0246-140-228-56-51.ngrok-free.app
﻿
3. Create a Digital Human registered with the plugin.
In order to create a digital human an register the plugin you have to use the /head/create endpoint of the Unith API.
Use the following parameters in the POST /head/create endpoint as reference. Replace the <HEAD VISUAL ID> with the head visual of your choice. Replace <PLUGIN URL> with the ngrok url (e.g https://0246-140-228-56-51.ngrok-free.app)
JSON

{
    "headVisualId": <HEAD VISUAL ID>,
    "suggestions": [],
    "name": "my_plugin_example",
    "alias": "my_plugin_example",
    "greetings": "hello, this is a welcome message.",
    "language": "en-US",
    "languageSpeechRecognition": "en-US",
    "phrases": [
    ],
    "ttsProvider": "audiostack",
    "operationMode": "plugin",
    "ocProvider": "playground",
    "ttsVoice": "carmela",
    "pluginOperationalModeConfig":{
            "name": "testPlugin",
            "url": <PLUGIN URL>,
            "options": {}
    }

} 

{
    "headVisualId": <HEAD VISUAL ID>,
    "suggestions": [],
    "name": "my_plugin_example",
    "alias": "my_plugin_example",
    "greetings": "hello, this is a welcome message.",
    "language": "en-US",
    "languageSpeechRecognition": "en-US",
    "phrases": [
    ],
    "ttsProvider": "audiostack",
    "operationMode": "plugin",
    "ocProvider": "playground",
    "ttsVoice": "carmela",
    "pluginOperationalModeConfig":{
            "name": "testPlugin",
            "url": <PLUGIN URL>,
            "options": {}
    }

} 
﻿
Once the head is created you can test it at the   "publicUrl": "https://chat.unith.ai/[org-id]/[head-id]?api_key=[org-api-key]",
Updated 18 Apr 2024
Did this page help you?
Create a doc_qa Digital Human
Update the Voice