Tool Calling Runtime 设计：工具注册、发现与动态调度

Tool Calling Runtime 是 Agent 系统中连接 LLM 推理和外部操作的桥梁。它不仅要管理工具的注册和发现，还要处理调用路由、权限控制和结果格式化。一个设计良好的 Runtime 可以让 Agent 高效、安全地使用各种工具。

工具注册中心

注册机制

interface ToolDefinition {
  name: string;
  description: string;
  inputSchema: JSONSchema;
  outputSchema?: JSONSchema;
  metadata: {
    category: string;
    tags: string[];
    version: string;
    author: string;
    permissions: string[];
  };
}

class ToolRegistry {
  private tools: Map<string, ToolDefinition> = new Map();
  private handlers: Map<string, ToolHandler> = new Map();

  register(definition: ToolDefinition, handler: ToolHandler): void {
    this.validateDefinition(definition);
    this.tools.set(definition.name, definition);
    this.handlers.set(definition.name, handler);
  }

  unregister(name: string): void {
    this.tools.delete(name);
    this.handlers.delete(name);
  }

  get(name: string): ToolDefinition | undefined {
    return this.tools.get(name);
  }

  list(filter?: ToolFilter): ToolDefinition[] {
    let tools = Array.from(this.tools.values());

    if (filter?.category) {
      tools = tools.filter(t => t.metadata.category === filter.category);
    }

    if (filter?.tags) {
      tools = tools.filter(t =>
        filter.tags!.some(tag => t.metadata.tags.includes(tag))
      );
    }

    return tools;
  }

  private validateDefinition(definition: ToolDefinition): void {
    if (!definition.name || !definition.description) {
      throw new Error('Tool must have name and description');
    }
    if (!definition.inputSchema) {
      throw new Error('Tool must have input schema');
    }
  }
}

动态注册

支持在运行时动态注册和注销工具：

class DynamicToolRegistry extends ToolRegistry {
  private sources: Map<string, ToolSource> = new Map();

  async addSource(source: ToolSource): Promise<void> {
    this.sources.set(source.id, source);

    // 从源加载工具
    const tools = await source.discover();
    for (const tool of tools) {
      this.register(tool.definition, tool.handler);
    }
  }

  async refreshSource(sourceId: string): Promise<void> {
    const source = this.sources.get(sourceId);
    if (!source) return;

    // 注销旧工具
    const oldTools = this.list({ source: sourceId });
    for (const tool of oldTools) {
      this.unregister(tool.name);
    }

    // 重新加载
    const tools = await source.discover();
    for (const tool of tools) {
      this.register(tool.definition, tool.handler);
    }
  }
}

能力发现

语义搜索

当工具数量很多时，使用语义搜索找到最相关的工具：

class ToolDiscovery {
  private embeddings: Map<string, number[]> = new Map();

  async findRelevant(query: string, limit: number = 5): Promise<ToolDefinition[]> {
    const queryEmbedding = await this.embed(query);
    const scored: Array<{ tool: ToolDefinition; score: number }> = [];

    for (const [name, embedding] of this.embeddings) {
      const score = this.cosineSimilarity(queryEmbedding, embedding);
      const tool = this.registry.get(name);
      if (tool) {
        scored.push({ tool, score });
      }
    }

    scored.sort((a, b) => b.score - a.score);
    return scored.slice(0, limit).map(s => s.tool);
  }

  private cosineSimilarity(a: number[], b: number[]): number {
    let dot = 0, normA = 0, normB = 0;
    for (let i = 0; i < a.length; i++) {
      dot += a[i] * b[i];
      normA += a[i] * a[i];
      normB += b[i] * b[i];
    }
    return dot / (Math.sqrt(normA) * Math.sqrt(normB));
  }
}

动态调度

负载均衡

class ToolLoadBalancer {
  private instances: Map<string, ToolInstance[]> = new Map();
  private roundRobinCounters: Map<string, number> = new Map();

  select(toolName: string): ToolInstance {
    const instances = this.instances.get(toolName) || [];
    if (instances.length === 0) {
      throw new Error(`No instances available for tool ${toolName}`);
    }

    // 轮询选择
    const counter = this.roundRobinCounters.get(toolName) || 0;
    const selected = instances[counter % instances.length];
    this.roundRobinCounters.set(toolName, counter + 1);

    return selected;
  }
}

熔断机制

class CircuitBreaker {
  private failures: Map<string, number> = new Map();
  private states: Map<string, 'closed' | 'open' | 'half-open'> = new Map();

  async execute<T>(toolName: string, fn: () => Promise<T>): Promise<T> {
    const state = this.states.get(toolName) || 'closed';

    if (state === 'open') {
      throw new Error(`Circuit breaker open for ${toolName}`);
    }

    try {
      const result = await fn();
      this.onSuccess(toolName);
      return result;
    } catch (error) {
      this.onFailure(toolName);
      throw error;
    }
  }

  private onSuccess(toolName: string): void {
    this.failures.set(toolName, 0);
    this.states.set(toolName, 'closed');
  }

  private onFailure(toolName: string): void {
    const failures = (this.failures.get(toolName) || 0) + 1;
    this.failures.set(toolName, failures);

    if (failures >= 5) {
      this.states.set(toolName, 'open');
      setTimeout(() => this.states.set(toolName, 'half-open'), 60000);
    }
  }
}

结果格式化

class ResultFormatter {
  format(result: any, schema?: JSONSchema): FormattedResult {
    // 根据输出 schema 格式化结果
    if (schema) {
      const validated = this.validate(result, schema);
      return { content: [{ type: 'text', text: JSON.stringify(validated) }] };
    }

    // 默认格式化
    if (typeof result === 'string') {
      return { content: [{ type: 'text', text: result }] };
    }

    return { content: [{ type: 'text', text: JSON.stringify(result, null, 2) }] };
  }
}

常见问题（FAQ）

如何处理工具版本不兼容？

在工具注册时声明版本，Runtime 在调用时检查版本兼容性。使用语义化版本号（SemVer）。

工具的热更新如何实现？

通过动态注册机制，注销旧版本工具、注册新版本。使用中的工具调用不受影响。

如何限制工具的调用频率？

在 Runtime 层实现速率限制器，按工具名或用户 ID 进行限流。

总结

Tool Calling Runtime 是 Agent 系统的核心组件。通过注册中心管理工具定义，通过能力发现帮助 LLM 选择工具，通过动态调度保证调用的可靠性和性能。