Advanced Topics

Deep dive into agent architecture, services, and customization.

Overview

This section covers advanced concepts for developers who need:

• Deep understanding of agent internals

• Custom service implementations

• Advanced event handling

• Performance optimization

Topics

Agent Services

Internal services that power the agent:

• Chat API Service

• Speech API Service

• Transcription API Service

• Image API Service

• File API Service

var services = new AgentServices
{
    ChatApi = customChatService,
    SpeechApi = customSpeechService,
    TranscriptionApi = customTranscriptionService,
    ImageApi = customImageService
};

var agent = new Agent(settings, behaviour, hooks, services);

Controllers Architecture

Agent uses specialized controllers for different concerns:

• ConversationController - Manages conversation state

• AudioController - Handles audio I/O

• ImageController - Manages image generation

• ToolCallController - Executes tool calls

• McpController - MCP integration

• ParametersController - Model parameters

// Access controllers
agent.conversationController.LoadAsync(conversationId);
agent.audioController.StartRecording();
agent.toolCallController.RegisterExecutor(tool);

Event Router

Internal event routing and propagation system.

// Event flow:
Service → Controller → Agent → AgentBehaviour → UI

Custom Chat Services

Implement custom AI providers:

public class CustomChatService : IChatService
{
    public async UniTask<Response> SendAsync(Parameters parameters)
    {
        // Call custom AI API
        var result = await MyCustomApi.GenerateAsync(parameters);
        return ConvertToResponse(result);
    }
}

Architecture Diagram

┌─────────────────────────────────────────────┐
│           AgentBehaviour (Unity)            │
│  ┌───────────────────────────────────────┐  │
│  │         Agent (Core Logic)            │  │
│  │  ┌────────────────────────────────┐   │  │
│  │  │      Controllers               │   │  │
│  │  │  ┌──────────────────────────┐  │   │  │
│  │  │  │  Services                │  │   │  │
│  │  │  │  - Chat API              │  │   │  │
│  │  │  │  - Speech API            │  │   │  │
│  │  │  │  - Transcription API     │  │   │  │
│  │  │  │  - Image API             │  │   │  │
│  │  │  └──────────────────────────┘  │   │  │
│  │  └────────────────────────────────┘   │  │
│  └───────────────────────────────────────┘  │
└─────────────────────────────────────────────┘

Controllers Responsibility

ConversationController

• Load/save conversations

• Manage conversation items

• Assemble context for API calls

• Handle conversation listeners

AudioController

• Record input audio

• Transcribe speech to text

• Generate output audio

• Play audio responses

ImageController

• Generate images from text

• Handle image responses

• Manage image parameters

ToolCallController

• Execute function tools

• Handle local shell tools

• Manage tool choice

• Submit tool outputs

McpController

• Connect to MCP servers

• Handle tool approvals

• Manage OAuth tokens

• Execute MCP tools

ParametersController

• Manage model parameters

• Handle preferences storage

• Dynamic tool registration

• Model API switching

Custom Service Implementation

1. Define Service Interface

public interface ICustomService : IAgentService<Input, Output, Parameters>
{
    UniTask<Output> ProcessAsync(Input input, Parameters parameters);
}

2. Implement Service

public class CustomServiceImpl : ICustomService
{
    public async UniTask<Output> ProcessAsync(Input input, Parameters parameters)
    {
        // Custom implementation
        return await CustomApiCall(input, parameters);
    }
    
    public void Dispose()
    {
        // Cleanup
    }
}

3. Register Service

var services = new AgentServices();
services.RegisterService<ICustomService>(new CustomServiceImpl());

var agent = new Agent(settings, behaviour, hooks, services);

Performance Optimization

1. Minimize Conversation Context

agent.Memory.MaxContextMessages = 20;
agent.Memory.SummarizationThreshold = 50;

2. Use Streaming

agent.Stream = true; // Reduces perceived latency

3. Cache Responses

var cache = new ResponseCache();
agent.onResponseCompleted += response => cache.Add(response);

4. Parallel Tool Execution

settings.ParallelToolCalls = true;

Debugging

Enable Verbose Logging

behaviour.LogLevel = TraceLevel.Verbose;

Monitor Controller State

Debug.Log($"Conversation: {agent.conversationController.IsInitialized}");
Debug.Log($"Audio: {agent.audioController.IsRecording}");
Debug.Log($"Tools: {agent.toolCallController.RegisteredToolCount}");

Trace API Calls

var logger = new CustomLogger();
var agent = new Agent(settings, behaviour, hooks, services, logger: logger);

Next Steps

• Agent Services - Service details

• Controllers Architecture - Controller deep dive

• Event Router - Event system

• Custom Chat Services - Custom providers

• API Reference - Complete API docs