Text Splitter

overview the text splitter is a crucial mechanism that breaks down longer messages into smaller, more manageable chunks this process significantly improves the readability, speed, and responsiveness of the digital human by allowing responses to be delivered incrementally by default, the unith platform automatically engages the text splitter for all digital humans in non streaming mode however, when you use a plugin, you take full control of the response processing chain, meaning you are responsible for implementing the text splitting logic on your end plugin implementation when building a plugin, your core function is to respond synchronously to unith's requests this requires you to receive the complete response text from your source (e g , an llm), split it into smaller chunks, and then send those chunks back to the platform in the required format quick start guide here's a quick guide to help you implement text splitting capabilities within your plugin first, you'll need a function to perform the actual splitting this example uses a regular expression to split the text by common sentence ending punctuation next, after you've generated the complete response text, you can use the split response by delimiter function to prepare the message for the digital human remember that the plugin response must be a list of pluginmessage objects def split response by delimeter(response message) delimiters = \[' ', '!', '?', ' '] regex pattern = '|' join('(?<={})' format(re escape(delimiter)) for delimiter in delimiters) chunks = re split(regex pattern, response message) return \[chunk strip() for chunk in chunks if chunk] plugin response = \[] response = response content if response is not none \# valid response response chunks = split response by delimeter(response) for idx, chunk in enumerate(response chunks) plugin response append( pluginmessage( type="text", payload=payload(type="text", message=f"{chunk}")) ) else \# fallback message plugin response append( pluginmessage( type="text", payload=payload(type="text", message="i'm sorry but i can't retrieve the information")) ) return plugin response by adding this logic to your plugin, you ensure that the digital human receives the response in manageable segments, which improves both responsiveness and the user experience the unith api's post body and response are formatted as a list this list is designed to support multiple parts for a single message, which can be of different data types (e g , text, image, or other metadata) in case of errors, you are responsible to return to the platform a fallback message or a valid pluginmessage