feat: 优化运行时判定
This commit is contained in:
richarjiang
26
README.md
26
README.md
@@ -85,8 +85,14 @@ Also delete config, logs, and the virtualenv:
|
||||
| `port` | Default `8765`. |
|
||||
| `device` | `auto`, `mps`, `cuda`, or `cpu`. `auto` falls through if warmup fails. |
|
||||
| `log_prompts` | Write the raw prompt into `data/routes.jsonl`. |
|
||||
| `request_timeout` | Seconds the hook waits for the service. Default `2.5`. |
|
||||
| `request_timeout` | Seconds the prompt hook waits for the service. Default `2.5`. |
|
||||
| `event_timeout` | Seconds a tool hook waits. Default `1.0`. |
|
||||
| `max_prompt_chars` | Max characters sent to the model. Longer prompts keep the head and tail. |
|
||||
| `trajectory` | Inject a short hint after tool calls, and before an edit of unread code. |
|
||||
| `gate` | Tighten risky Bash and edit calls. Never returns allow. |
|
||||
| `stop_check` | Ask the agent to verify after edits before it stops. |
|
||||
| `skills` | Suggest at most one installed skill on each user prompt. |
|
||||
| `skill_inject` | `suggest` names the skill. `content` attaches `SKILL.md` once per session. |
|
||||
|
||||
Restart after config or server changes: `laya-router restart`.
|
||||
|
||||
@@ -103,6 +109,20 @@ Every call is appended to `~/.laya-router/data/routes.jsonl` (timestamp, client,
|
||||
|
||||
If the service is down, the hook prints `{}` and exits 0. Codex and Claude Code continue. The error is appended to `~/.laya-router/logs/hook.log`.
|
||||
|
||||
## During a turn
|
||||
|
||||
The same hook command also listens for `PreToolUse`, `PostToolUse`, and `Stop`. The resident service keeps the session trace in memory.
|
||||
|
||||
- `PreToolUse` on Bash or an edit: a high risk is denied; a medium risk asks on Claude Code and warns on Codex, because Codex rejects `permissionDecision: ask`. An edit of a file that was not read gets a short hint.
|
||||
- `PostToolUse`: every Bash or edit checkpoint gets a short hint. The same hint is not repeated. Two failures in a row, or a high `is_stuck`, tells the agent to stop and replan. `Read`, `Grep`, and `Glob` are recorded and stay quiet.
|
||||
- `Stop`: if code changed and nothing verified it, the hook blocks once so the agent keeps going. `stop_hook_active` is always allowed through.
|
||||
|
||||
On `UserPromptSubmit`, a second pair of questions can name one installed skill. The roster is every `SKILL.md` under `~/.claude/skills`, `~/.codex/skills`, `~/.agents/skills`, the project skill directories, and Claude plugin install paths. A skill with `disable-model-invocation: true` is skipped. The first pass ranks the roster and gates on whether a skill is warranted. The second pass re-reads the top three. Either threshold below 0.30 suggests nothing.
|
||||
|
||||
`laya-mcp` exposes `laya_judge` over stdio. Allow `mcp__laya` in Claude Code if you want the agent to call it. Install registers it with `claude mcp add` and `codex mcp add` when those CLIs are on `PATH`.
|
||||
|
||||
The first Codex session must trust the new hook entries. Run `/hooks` again after upgrading.
|
||||
|
||||
## Develop
|
||||
|
||||
```bash
|
||||
@@ -115,7 +135,7 @@ Unit tests do not load the model. After the service is up, `laya-router route` r
|
||||
| --- | --- | --- |
|
||||
| `src/laya_router/questions.py` | Question text. Bump `QUESTIONS_VERSION` when the wording changes. | `laya-router restart` |
|
||||
| `src/laya_router/render.py` | Text injected into the agent | `laya-router restart`, then the next prompt |
|
||||
| `src/laya_router/hook.py` | Timeout, slash-command skip, fail-open | `laya-router restart`, then the next prompt |
|
||||
| `src/laya_router/hook.py` | Prompt and tool events, timeout, fail-open | `laya-router restart`, then the next prompt |
|
||||
| `src/laya_router/server.py` | Model load, `/route`, dashboard | `laya-router restart` |
|
||||
| `src/laya_router/dashboard.html` | Browser UI | `laya-router restart` |
|
||||
|
||||
@@ -129,7 +149,7 @@ The service has no authentication. It listens on `127.0.0.1` only. `/route` acce
|
||||
|
||||
`routes.jsonl` stores the raw prompt by default, including anything you paste into Codex or Claude Code. Set `log_prompts = false` to keep only a SHA-256. The runtime directory is `~/.laya-router/`, outside this repo. Do not commit it.
|
||||
|
||||
The installer rewrites `~/.codex/hooks.json` and merges a `UserPromptSubmit` entry into `~/.claude/settings.json`. That Claude file often holds API credentials. The tool preserves existing keys, writes a timestamped `*.bak-*` copy before changing a file, and removes only its own hook on uninstall. Read the diff before you publish a dotfile.
|
||||
The installer rewrites `~/.codex/hooks.json` and merges `UserPromptSubmit`, `PreToolUse`, `PostToolUse`, and `Stop` entries into `~/.claude/settings.json`. That Claude file often holds API credentials. The tool preserves existing keys, writes a timestamped `*.bak-*` copy before changing a file, and removes only its own hook on uninstall. Read the diff before you publish a dotfile.
|
||||
|
||||
A web page on another origin cannot read these responses: the server does not send `Access-Control-Allow-Origin`. Keep the default bind address anyway.
|
||||
|
||||
|
||||
@@ -85,8 +85,14 @@ laya-router start
|
||||
| `port` | 默认 `8765`。 |
|
||||
| `device` | `auto`、`mps`、`cuda` 或 `cpu`。`auto` 预热失败会回退。 |
|
||||
| `log_prompts` | 是否把原文写入 `data/routes.jsonl`。 |
|
||||
| `request_timeout` | Hook 等待服务的秒数，默认 `2.5`。 |
|
||||
| `request_timeout` | 用户 prompt 的 hook 等待秒数，默认 `2.5`。 |
|
||||
| `event_timeout` | 工具 hook 的等待秒数，默认 `1.0`。 |
|
||||
| `max_prompt_chars` | 送进模型的最大字符数。超长时保留头尾。 |
|
||||
| `trajectory` | 工具调用之后，以及改未读过的文件之前，注入一句短提示。 |
|
||||
| `gate` | 收紧有风险的 Bash 和编辑。不会返回 allow。 |
|
||||
| `stop_check` | 改完代码却没验证时，不让这一轮停下来。 |
|
||||
| `skills` | 每条用户 prompt 最多推荐一个已安装的 skill。 |
|
||||
| `skill_inject` | `suggest` 只给名字和路径。`content` 在同一会话里附上一次 `SKILL.md` 正文。 |
|
||||
|
||||
改完配置或服务端代码后执行 `laya-router restart`。
|
||||
|
||||
@@ -103,6 +109,20 @@ laya-router start
|
||||
|
||||
服务挂了时，hook 输出 `{}` 并退出 0，Codex 和 Claude Code 照常工作。原因写在 `~/.laya-router/logs/hook.log`。
|
||||
|
||||
## 运行过程中
|
||||
|
||||
同一条 hook 命令也监听 `PreToolUse`、`PostToolUse` 和 `Stop`。会话轨迹放在常驻服务的内存里。
|
||||
|
||||
- `PreToolUse` 遇到 Bash 或编辑：风险高就拒绝；风险中等时，Claude Code 会询问，Codex 只注入警告，因为 Codex 不接受 `permissionDecision: ask`。要改一个还没读过的文件时，会提示先读。
|
||||
- `PostToolUse`：每次 Bash 或编辑都会注入一句短提示。连续两条完全相同的提示会去掉。连续失败两次，或者 `is_stuck` 很高，就要求先停下来重新规划。`Read`、`Grep`、`Glob` 只记账，不插话。
|
||||
- `Stop`：代码改过、之后没有跑测试或构建时，hook 最多拦一次，让 agent 继续做验证。`stop_hook_active` 一律放行，避免死循环。
|
||||
|
||||
`UserPromptSubmit` 上还会再做一轮 skill 推荐，最多一个。名单来自 `~/.claude/skills`、`~/.codex/skills`、`~/.agents/skills`、项目里的 skill 目录，以及 Claude 插件的安装路径。frontmatter 里 `disable-model-invocation: true` 的 skill 会跳过。第一问给全部候选排序，并用三个是否题决定这条 prompt 是否需要 skill。第二问只重读前三名。任一阈值低于 0.30 就不推荐。
|
||||
|
||||
`laya-mcp` 用 stdio 提供 `laya_judge`。想让模型自己调用，在 Claude Code 里允许 `mcp__laya`。安装时如果 `PATH` 上有对应命令，会执行 `claude mcp add` 和 `codex mcp add`。
|
||||
|
||||
升级之后，Codex 要再跑一次 `/hooks`，信任新加的 hook。
|
||||
|
||||
## 开发
|
||||
|
||||
```bash
|
||||
@@ -115,7 +135,7 @@ PYTHONPATH=src uv run --no-project --with pytest python -m pytest
|
||||
| --- | --- | --- |
|
||||
| `src/laya_router/questions.py` | 问题措辞。改措辞时把 `QUESTIONS_VERSION` 加一。 | `laya-router restart` |
|
||||
| `src/laya_router/render.py` | 注入给 Agent 的文本 | `laya-router restart`，然后下一条 prompt |
|
||||
| `src/laya_router/hook.py` | 超时、跳过斜杠命令、失败放行 | `laya-router restart`，然后下一条 prompt |
|
||||
| `src/laya_router/hook.py` | 用户 prompt 和工具事件、超时、失败放行 | `laya-router restart`，然后下一条 prompt |
|
||||
| `src/laya_router/server.py` | 加载模型、`/route`、看板 | `laya-router restart` |
|
||||
| `src/laya_router/dashboard.html` | 浏览器页面 | `laya-router restart` |
|
||||
|
||||
@@ -129,7 +149,7 @@ PYTHONPATH=src uv run --no-project --with pytest python -m pytest
|
||||
|
||||
`routes.jsonl` 默认保存完整 prompt，包括你贴进 Codex 或 Claude Code 的内容。`log_prompts = false` 时只保存 SHA-256。运行目录是仓库外面的 `~/.laya-router/`，不要把它提交进 git。
|
||||
|
||||
安装程序会改写 `~/.codex/hooks.json`，并往 `~/.claude/settings.json` 合并一条 `UserPromptSubmit`。Claude 的这个文件里常常有 API 凭证。程序会保留原有字段，改文件前写一份带时间戳的 `*.bak-*`，卸载时只删自己的 hook。公开任何本机配置之前先看 diff。
|
||||
安装程序会改写 `~/.codex/hooks.json`，并往 `~/.claude/settings.json` 合并 `UserPromptSubmit`、`PreToolUse`、`PostToolUse` 和 `Stop`。Claude 的这个文件里常常有 API 凭证。程序会保留原有字段，改文件前写一份带时间戳的 `*.bak-*`，卸载时只删自己的 hook。公开任何本机配置之前先看 diff。
|
||||
|
||||
其他网站的页面读不到这些接口：服务没有发送 `Access-Control-Allow-Origin`。监听地址仍然应该保持默认。
|
||||
|
||||
|
||||
@@ -17,6 +17,7 @@ dependencies = [
|
||||
[project.scripts]
|
||||
laya-router = "laya_router.cli:main"
|
||||
laya-hook = "laya_router.hook:main"
|
||||
laya-mcp = "laya_router.mcp:main"
|
||||
|
||||
[dependency-groups]
|
||||
dev = ["pytest>=8"]
|
||||
|
||||
@@ -63,6 +63,109 @@ def _settings():
|
||||
return load_settings(home)
|
||||
|
||||
|
||||
def _bin_link_targets(settings) -> list[tuple[str, Path]]:
|
||||
"""Pairs of (link name, source binary in the venv) exposed on the user's PATH."""
|
||||
return [
|
||||
("laya-router", settings.router_bin),
|
||||
("laya-hook", settings.hook_bin),
|
||||
("laya-mcp", settings.mcp_bin),
|
||||
]
|
||||
|
||||
|
||||
def _user_path_bin_dir() -> Path | None:
|
||||
"""Pick the first writable PATH directory that is not a system location.
|
||||
|
||||
Prefers ``~/.local/bin`` per the pip / pipx convention. Falls back to
|
||||
scanning ``$PATH`` left-to-right, so users with a customised PATH still get
|
||||
a sensible target. Returns ``None`` when no writable candidate exists.
|
||||
|
||||
``LAYA_ROUTER_BIN_DIR`` overrides the directory entirely — useful for
|
||||
shared / locked-down hosts where ``~/.local/bin`` is not writable and the
|
||||
caller wants to install elsewhere.
|
||||
"""
|
||||
override = os.environ.get("LAYA_ROUTER_BIN_DIR", "").strip()
|
||||
if override:
|
||||
path = Path(override).expanduser()
|
||||
if path.is_dir() and os.access(path, os.W_OK):
|
||||
return path
|
||||
return None
|
||||
preferred = Path.home() / ".local" / "bin"
|
||||
if preferred.is_dir() and os.access(preferred, os.W_OK):
|
||||
return preferred
|
||||
for entry in os.environ.get("PATH", "").split(":"):
|
||||
if not entry:
|
||||
continue
|
||||
path = Path(entry)
|
||||
if not path.is_dir() or not os.access(path, os.W_OK):
|
||||
continue
|
||||
resolved = path.resolve() if path.exists() else path
|
||||
if str(resolved).startswith(("/usr/", "/bin/", "/sbin/", "/System/")):
|
||||
continue
|
||||
return path
|
||||
return None
|
||||
|
||||
|
||||
def ensure_path_symlinks(settings, *, target_dir: Path | None = None) -> None:
|
||||
"""Expose the venv's CLI binaries on the user's PATH via symlinks.
|
||||
|
||||
The venv lives under ``~/.laya-router/.venv/bin/``, which is normally off
|
||||
PATH. Without a symlink, ``laya-router restart`` and friends only work
|
||||
when invoked through the absolute path.
|
||||
"""
|
||||
if target_dir is None:
|
||||
target_dir = _user_path_bin_dir()
|
||||
if target_dir is None:
|
||||
print(
|
||||
"警告: PATH 上没有可写的目录；'laya-router' 命令不会直接可用。",
|
||||
file=sys.stderr,
|
||||
)
|
||||
print(
|
||||
" 手动设置: export PATH=\"$HOME/.laya-router/.venv/bin:$PATH\"",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return
|
||||
_apply_path_symlinks(settings, target_dir)
|
||||
|
||||
|
||||
def remove_path_symlinks(settings, *, target_dir: Path | None = None) -> None:
|
||||
"""Remove the symlinks installed by ``ensure_path_symlinks``."""
|
||||
if target_dir is None:
|
||||
target_dir = _user_path_bin_dir()
|
||||
if target_dir is None:
|
||||
return
|
||||
_remove_path_symlinks(settings, target_dir)
|
||||
|
||||
|
||||
def _apply_path_symlinks(settings, target_dir: Path) -> None:
|
||||
"""Idempotent: creates / repairs the venv symlinks inside ``target_dir``."""
|
||||
for name, source in _bin_link_targets(settings):
|
||||
if not source.is_file():
|
||||
continue
|
||||
link = target_dir / name
|
||||
try:
|
||||
if link.is_symlink():
|
||||
if link.resolve() == source.resolve():
|
||||
continue
|
||||
link.unlink()
|
||||
elif link.exists():
|
||||
print(f"跳过 {link}: 已被占用（不是符号链接）", file=sys.stderr)
|
||||
continue
|
||||
link.symlink_to(source)
|
||||
print(f"{link} -> {source}")
|
||||
except OSError as exc:
|
||||
print(f"建立 {link} 失败: {exc}", file=sys.stderr)
|
||||
|
||||
|
||||
def _remove_path_symlinks(settings, target_dir: Path) -> None:
|
||||
for name, _source in _bin_link_targets(settings):
|
||||
link = target_dir / name
|
||||
try:
|
||||
if link.is_symlink():
|
||||
link.unlink()
|
||||
except OSError as exc:
|
||||
print(f"移除 {link} 失败: {exc}", file=sys.stderr)
|
||||
|
||||
|
||||
def _clients(only: list[str] | None, *, detected_only: bool):
|
||||
from laya_router.clients import ClaudeClient, CodexClient
|
||||
|
||||
@@ -161,6 +264,22 @@ def _print_log_tail(settings, lines: int = 40) -> None:
|
||||
print("\n".join(content[-lines:]), file=sys.stderr)
|
||||
|
||||
|
||||
def _register_client_mcp(name: str, settings) -> None:
|
||||
from laya_router.clients.base import register_mcp
|
||||
|
||||
if not settings.mcp_bin.is_file():
|
||||
print(f"{name}: 跳过 MCP 注册，找不到 {settings.mcp_bin}", file=sys.stderr)
|
||||
return
|
||||
if name == "claude":
|
||||
register_mcp(
|
||||
"claude",
|
||||
["mcp", "add", "--scope", "user", "--transport", "stdio", "laya", "--", str(settings.mcp_bin)],
|
||||
)
|
||||
elif name == "codex":
|
||||
register_mcp("codex", ["mcp", "add", "laya", "--", str(settings.mcp_bin)])
|
||||
print(f"{name}: 已尝试注册 MCP 工具 laya_judge（失败不影响 hook）")
|
||||
|
||||
|
||||
def cmd_install(args: argparse.Namespace) -> int:
|
||||
settings = _settings()
|
||||
sync_source(settings)
|
||||
@@ -184,6 +303,7 @@ def cmd_install(args: argparse.Namespace) -> int:
|
||||
_print_log_tail(settings)
|
||||
return 1
|
||||
print(f"服务就绪: {settings.base_url} device={health.get('device')}", flush=True)
|
||||
ensure_path_symlinks(settings)
|
||||
|
||||
clients = _clients(args.only, detected_only=True)
|
||||
if not clients:
|
||||
@@ -196,6 +316,7 @@ def cmd_install(args: argparse.Namespace) -> int:
|
||||
return 1
|
||||
state = "已更新" if changed else "已是最新"
|
||||
print(f"{client.name}: hook {state}")
|
||||
_register_client_mcp(client.name, settings)
|
||||
|
||||
print("冒烟测试...", flush=True)
|
||||
smoke = argparse.Namespace(
|
||||
@@ -206,8 +327,8 @@ def cmd_install(args: argparse.Namespace) -> int:
|
||||
return 1
|
||||
print()
|
||||
print("安装完成。")
|
||||
print("- Codex: 启动 codex 后运行 /hooks，首次需要信任这条 hook。")
|
||||
print("- Claude Code: 新会话会自动加载。")
|
||||
print("- Codex: 启动 codex 后运行 /hooks，首次需要信任这条 hook。新增的 PreToolUse、PostToolUse、Stop 也要信任一次。")
|
||||
print("- Claude Code: 新会话会自动加载。若要让模型主动调用判定，在权限里允许 mcp__laya。")
|
||||
print(f"- 路由记录: {settings.routes_path}")
|
||||
print("- 查看服务: laya-router status")
|
||||
return 0
|
||||
@@ -223,6 +344,7 @@ def cmd_uninstall(args: argparse.Namespace) -> int:
|
||||
return 1
|
||||
print(f"{client.name}: 已移除 hook")
|
||||
uninstall_agent()
|
||||
remove_path_symlinks(settings)
|
||||
print("已停止并移除 launchd 服务。")
|
||||
if args.purge:
|
||||
home = settings.home.resolve()
|
||||
|
||||
@@ -21,6 +21,20 @@ def is_our_command(command: str) -> bool:
|
||||
return "laya-hook" in command
|
||||
|
||||
|
||||
def register_mcp(binary: str, args: list[str]) -> None:
|
||||
"""Best-effort MCP registration. A missing CLI or a rejected command is not fatal."""
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
exe = shutil.which(binary)
|
||||
if exe is None:
|
||||
return
|
||||
try:
|
||||
subprocess.run([exe, *args], check=False, timeout=30)
|
||||
except (OSError, subprocess.TimeoutExpired):
|
||||
return
|
||||
|
||||
|
||||
def backup_file(path: Path) -> Path | None:
|
||||
if not path.is_file():
|
||||
return None
|
||||
@@ -64,20 +78,22 @@ def _hook_entry(command: str, *, timeout: int, status_message: str | None) -> di
|
||||
return entry
|
||||
|
||||
|
||||
def upsert_user_prompt_hook(
|
||||
def upsert_hook(
|
||||
config: dict,
|
||||
event_name: str,
|
||||
command: str,
|
||||
*,
|
||||
matcher: str | None = None,
|
||||
timeout: int = 5,
|
||||
status_message: str | None = None,
|
||||
) -> dict:
|
||||
"""Insert or replace this project's UserPromptSubmit hook. Other hooks stay."""
|
||||
"""Insert or replace this project's hook for one event. Other hooks stay."""
|
||||
hooks = config.setdefault("hooks", {})
|
||||
if not isinstance(hooks, dict):
|
||||
raise ValueError("hooks 字段必须是对象")
|
||||
event = hooks.get("UserPromptSubmit", [])
|
||||
event = hooks.get(event_name, [])
|
||||
if not isinstance(event, list):
|
||||
raise ValueError("hooks.UserPromptSubmit 必须是数组")
|
||||
raise ValueError(f"hooks.{event_name} 必须是数组")
|
||||
|
||||
kept: list = []
|
||||
for group in event:
|
||||
@@ -96,19 +112,63 @@ def upsert_user_prompt_hook(
|
||||
if remaining:
|
||||
kept.append({**group, "hooks": remaining})
|
||||
|
||||
kept.append({"hooks": [_hook_entry(command, timeout=timeout, status_message=status_message)]})
|
||||
hooks["UserPromptSubmit"] = kept
|
||||
group: dict = {"hooks": [_hook_entry(command, timeout=timeout, status_message=status_message)]}
|
||||
if matcher:
|
||||
group["matcher"] = matcher
|
||||
kept.append(group)
|
||||
hooks[event_name] = kept
|
||||
return config
|
||||
|
||||
|
||||
def remove_user_prompt_hook(config: dict) -> dict:
|
||||
hooks = config.get("hooks")
|
||||
if not isinstance(hooks, dict):
|
||||
return config
|
||||
event = hooks.get("UserPromptSubmit")
|
||||
if not isinstance(event, list):
|
||||
def upsert_user_prompt_hook(
|
||||
config: dict,
|
||||
command: str,
|
||||
*,
|
||||
timeout: int = 5,
|
||||
status_message: str | None = None,
|
||||
) -> dict:
|
||||
"""Insert or replace this project's UserPromptSubmit hook. Other hooks stay."""
|
||||
return upsert_hook(
|
||||
config,
|
||||
"UserPromptSubmit",
|
||||
command,
|
||||
timeout=timeout,
|
||||
status_message=status_message,
|
||||
)
|
||||
|
||||
|
||||
PRE_TOOL_MATCHER = "Bash|Edit|Write|MultiEdit|apply_patch"
|
||||
POST_TOOL_MATCHER = "Bash|Edit|Write|MultiEdit|apply_patch|Read|Grep|Glob"
|
||||
|
||||
|
||||
def install_router_hooks(
|
||||
config: dict,
|
||||
command: str,
|
||||
*,
|
||||
prompt_status: str | None,
|
||||
) -> dict:
|
||||
upsert_hook(config, "UserPromptSubmit", command, timeout=5, status_message=prompt_status)
|
||||
upsert_hook(
|
||||
config,
|
||||
"PreToolUse",
|
||||
command,
|
||||
matcher=PRE_TOOL_MATCHER,
|
||||
timeout=5,
|
||||
status_message="Laya check",
|
||||
)
|
||||
upsert_hook(
|
||||
config,
|
||||
"PostToolUse",
|
||||
command,
|
||||
matcher=POST_TOOL_MATCHER,
|
||||
timeout=5,
|
||||
status_message="Laya check",
|
||||
)
|
||||
upsert_hook(config, "Stop", command, timeout=5, status_message="Laya check")
|
||||
return config
|
||||
|
||||
|
||||
def _without_our_hooks(event: list) -> list:
|
||||
kept: list = []
|
||||
for group in event:
|
||||
if not isinstance(group, dict) or not isinstance(group.get("hooks"), list):
|
||||
@@ -121,10 +181,26 @@ def remove_user_prompt_hook(config: dict) -> dict:
|
||||
]
|
||||
if remaining:
|
||||
kept.append({**group, "hooks": remaining})
|
||||
return kept
|
||||
|
||||
|
||||
def remove_user_prompt_hook(config: dict) -> dict:
|
||||
return remove_our_hooks(config)
|
||||
|
||||
|
||||
def remove_our_hooks(config: dict) -> dict:
|
||||
hooks = config.get("hooks")
|
||||
if not isinstance(hooks, dict):
|
||||
return config
|
||||
for event_name in list(hooks):
|
||||
event = hooks.get(event_name)
|
||||
if not isinstance(event, list):
|
||||
continue
|
||||
kept = _without_our_hooks(event)
|
||||
if kept:
|
||||
hooks["UserPromptSubmit"] = kept
|
||||
hooks[event_name] = kept
|
||||
else:
|
||||
hooks.pop("UserPromptSubmit", None)
|
||||
hooks.pop(event_name, None)
|
||||
if not hooks:
|
||||
config.pop("hooks", None)
|
||||
return config
|
||||
|
||||
@@ -5,7 +5,7 @@ from __future__ import annotations
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
from laya_router.clients.base import read_json, remove_user_prompt_hook, upsert_user_prompt_hook, write_json
|
||||
from laya_router.clients.base import install_router_hooks, read_json, remove_our_hooks, write_json
|
||||
from laya_router.config import Settings, hook_command
|
||||
|
||||
|
||||
@@ -20,17 +20,12 @@ class ClaudeClient:
|
||||
|
||||
def install(self, settings: Settings) -> bool:
|
||||
config = read_json(self.config_path)
|
||||
upsert_user_prompt_hook(
|
||||
config,
|
||||
hook_command(settings, self.name),
|
||||
timeout=5,
|
||||
status_message=None,
|
||||
)
|
||||
install_router_hooks(config, hook_command(settings, self.name), prompt_status=None)
|
||||
return write_json(self.config_path, config)
|
||||
|
||||
def uninstall(self) -> bool:
|
||||
if not self.config_path.is_file():
|
||||
return False
|
||||
config = read_json(self.config_path)
|
||||
remove_user_prompt_hook(config)
|
||||
remove_our_hooks(config)
|
||||
return write_json(self.config_path, config)
|
||||
|
||||
@@ -11,7 +11,7 @@ import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
from laya_router.clients.base import read_json, remove_user_prompt_hook, upsert_user_prompt_hook, write_json
|
||||
from laya_router.clients.base import install_router_hooks, read_json, remove_our_hooks, write_json
|
||||
from laya_router.config import Settings, hook_command
|
||||
|
||||
|
||||
@@ -40,11 +40,10 @@ class CodexClient:
|
||||
self.ensure_hooks_feature()
|
||||
config = read_json(self.hooks_path)
|
||||
config.setdefault("description", "Laya local System-1 router")
|
||||
upsert_user_prompt_hook(
|
||||
install_router_hooks(
|
||||
config,
|
||||
hook_command(settings, self.name),
|
||||
timeout=5,
|
||||
status_message="Laya routing",
|
||||
prompt_status="Laya routing",
|
||||
)
|
||||
return write_json(self.hooks_path, config)
|
||||
|
||||
@@ -52,7 +51,7 @@ class CodexClient:
|
||||
if not self.hooks_path.is_file():
|
||||
return False
|
||||
config = read_json(self.hooks_path)
|
||||
remove_user_prompt_hook(config)
|
||||
remove_our_hooks(config)
|
||||
if config.get("description") == "Laya local System-1 router" and "hooks" not in config:
|
||||
config.pop("description", None)
|
||||
if not config:
|
||||
|
||||
@@ -33,8 +33,20 @@ log_prompts = true
|
||||
# Hook 调用本服务的超时（秒）。应小于客户端 hook timeout（5 秒）。
|
||||
request_timeout = 2.5
|
||||
|
||||
# 工具事件的超时。PreToolUse 会挡住每一次工具调用。
|
||||
event_timeout = 1.0
|
||||
|
||||
# 送进模型的最大字符数。中英混合下大约落在 1024 token 以内，超长时保留头尾。
|
||||
max_prompt_chars = 3000
|
||||
|
||||
# 运行中判定。关掉某一项后，对应事件只记轨迹、不再注入。
|
||||
trajectory = true
|
||||
gate = true
|
||||
stop_check = true
|
||||
skills = true
|
||||
|
||||
# suggest 只给出 skill 名字和路径；content 把 SKILL.md 正文附在 prompt 上。
|
||||
skill_inject = "suggest"
|
||||
"""
|
||||
|
||||
|
||||
@@ -55,7 +67,13 @@ class Settings:
|
||||
subfolder: str = "multilingual"
|
||||
log_prompts: bool = True
|
||||
request_timeout: float = 2.5
|
||||
event_timeout: float = 1.0
|
||||
max_prompt_chars: int = 3000
|
||||
trajectory: bool = True
|
||||
gate: bool = True
|
||||
stop_check: bool = True
|
||||
skills: bool = True
|
||||
skill_inject: str = "suggest"
|
||||
|
||||
@property
|
||||
def config_path(self) -> Path:
|
||||
@@ -93,6 +111,10 @@ class Settings:
|
||||
def router_bin(self) -> Path:
|
||||
return self.venv_dir / "bin" / "laya-router"
|
||||
|
||||
@property
|
||||
def mcp_bin(self) -> Path:
|
||||
return self.venv_dir / "bin" / "laya-mcp"
|
||||
|
||||
@property
|
||||
def base_url(self) -> str:
|
||||
return f"http://{self.host}:{self.port}"
|
||||
@@ -124,7 +146,15 @@ def load_settings(home: Path | None = None) -> Settings:
|
||||
subfolder = str(data.get("subfolder", "multilingual"))
|
||||
log_prompts = _as_bool(data.get("log_prompts", True), True)
|
||||
request_timeout = float(data.get("request_timeout", 2.5))
|
||||
event_timeout = float(data.get("event_timeout", 1.0))
|
||||
max_prompt_chars = int(data.get("max_prompt_chars", 3000))
|
||||
trajectory = _as_bool(data.get("trajectory", True), True)
|
||||
gate = _as_bool(data.get("gate", True), True)
|
||||
stop_check = _as_bool(data.get("stop_check", True), True)
|
||||
skills = _as_bool(data.get("skills", True), True)
|
||||
skill_inject = str(data.get("skill_inject", "suggest"))
|
||||
if skill_inject not in {"suggest", "content"}:
|
||||
skill_inject = "suggest"
|
||||
|
||||
if os.environ.get("LAYA_ROUTER_HOST"):
|
||||
host = os.environ["LAYA_ROUTER_HOST"]
|
||||
@@ -139,6 +169,8 @@ def load_settings(home: Path | None = None) -> Settings:
|
||||
raise ValueError(f"无效端口: {port}")
|
||||
if request_timeout <= 0:
|
||||
raise ValueError(f"request_timeout 必须大于 0: {request_timeout}")
|
||||
if event_timeout <= 0:
|
||||
raise ValueError(f"event_timeout 必须大于 0: {event_timeout}")
|
||||
if max_prompt_chars < 200:
|
||||
raise ValueError(f"max_prompt_chars 过小: {max_prompt_chars}")
|
||||
|
||||
@@ -151,7 +183,13 @@ def load_settings(home: Path | None = None) -> Settings:
|
||||
subfolder=subfolder,
|
||||
log_prompts=log_prompts,
|
||||
request_timeout=request_timeout,
|
||||
event_timeout=event_timeout,
|
||||
max_prompt_chars=max_prompt_chars,
|
||||
trajectory=trajectory,
|
||||
gate=gate,
|
||||
stop_check=stop_check,
|
||||
skills=skills,
|
||||
skill_inject=skill_inject,
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -147,6 +147,33 @@
|
||||
box-shadow: 0 1px 2px rgba(0, 0, 0, 0.04);
|
||||
}
|
||||
.switch input { accent-color: var(--blue); width: 16px; height: 16px; }
|
||||
.stage-controls {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 10px;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
.segmented {
|
||||
display: inline-flex;
|
||||
background: white;
|
||||
border: 1px solid var(--line);
|
||||
border-radius: 999px;
|
||||
padding: 3px;
|
||||
box-shadow: 0 1px 2px rgba(0, 0, 0, 0.04);
|
||||
}
|
||||
.segmented button {
|
||||
border: 0;
|
||||
background: transparent;
|
||||
padding: 5px 12px;
|
||||
border-radius: 999px;
|
||||
font-size: 13px;
|
||||
color: var(--secondary);
|
||||
transition: background 0.18s, color 0.18s;
|
||||
}
|
||||
.segmented button.active {
|
||||
background: var(--blue);
|
||||
color: white;
|
||||
}
|
||||
.empty {
|
||||
max-width: 36rem;
|
||||
color: var(--secondary);
|
||||
@@ -168,6 +195,15 @@
|
||||
to { opacity: 1; transform: none; }
|
||||
}
|
||||
.when { margin: 0; color: var(--tertiary); font-size: 12px; }
|
||||
.tag {
|
||||
display: inline-block;
|
||||
margin-right: 6px;
|
||||
padding: 0 6px;
|
||||
border-radius: 999px;
|
||||
background: var(--fill);
|
||||
color: var(--secondary);
|
||||
font-size: 12px;
|
||||
}
|
||||
.prompt {
|
||||
margin: 10px 0 16px;
|
||||
font-size: 17px;
|
||||
@@ -214,7 +250,7 @@
|
||||
}
|
||||
@media (prefers-reduced-motion: reduce) {
|
||||
.card.arrive { animation: none; }
|
||||
.split span, .track i { transition: none; }
|
||||
.split span, .track i, .segmented button { transition: none; }
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
@@ -244,7 +280,28 @@ const METERS = [
|
||||
["needs_web", "先查文档"],
|
||||
["needs_deep_reasoning", "先想清楚"],
|
||||
];
|
||||
const state = { records: [], follow: true, selected: null, fresh: new Set() };
|
||||
const EVENTS = {
|
||||
UserPromptSubmit: "用户提问",
|
||||
PreToolUse: "工具前",
|
||||
PostToolUse: "工具后",
|
||||
Stop: "结束检查",
|
||||
judge: "主动判定",
|
||||
};
|
||||
const ACTIONS = {
|
||||
inject: "注入",
|
||||
deny: "拦截",
|
||||
ask: "询问",
|
||||
block: "拦截",
|
||||
silent: "沉默",
|
||||
};
|
||||
const SORT_KEY = "laya-router:turn-sort";
|
||||
const state = {
|
||||
records: [],
|
||||
follow: true,
|
||||
selected: null,
|
||||
fresh: new Set(),
|
||||
sort: localStorage.getItem(SORT_KEY) === "asc" ? "asc" : "desc",
|
||||
};
|
||||
|
||||
function keyOf(record) {
|
||||
if (record.session_id) return record.session_id;
|
||||
@@ -295,7 +352,7 @@ function render() {
|
||||
aside.innerHTML = groups.map(([key, items]) => {
|
||||
const last = items[items.length - 1];
|
||||
const snippet = (last.prompt || "没有记下原文").replace(/\s+/g, " ");
|
||||
const levelName = LEVELS.find(([id]) => id === last.level)?.[1] || last.level;
|
||||
const levelName = LEVELS.find(([id]) => id === last.level)?.[1] || ACTIONS[last.action] || EVENTS[last.event] || last.level || "";
|
||||
return `<button class="session ${key === state.selected ? "active" : ""}" data-key="${encodeURIComponent(key)}" type="button">
|
||||
<div class="rowline"><span>${escapeHtml(titleOf(last))}</span><span class="level quiet">${escapeHtml(levelName)}</span></div>
|
||||
<p class="snippet">${escapeHtml(snippet)}</p>
|
||||
@@ -312,24 +369,32 @@ function render() {
|
||||
const [key, items] = group;
|
||||
const head = items[items.length - 1];
|
||||
const followChecked = state.follow ? "checked" : "";
|
||||
const ordered = state.sort === "desc"
|
||||
? items.map((r, i) => [i, r]).reverse()
|
||||
: items.map((r, i) => [i, r]);
|
||||
stage.innerHTML = `<div class="stage-head">
|
||||
<div>
|
||||
<h2>${escapeHtml(titleOf(head))}</h2>
|
||||
<p class="path">${escapeHtml(head.cwd || "没有记录工作目录")}</p>
|
||||
</div>
|
||||
<div class="stage-controls">
|
||||
<div class="segmented" role="group" aria-label="排序方式">
|
||||
<button type="button" data-sort="desc" class="${state.sort === "desc" ? "active" : ""}" aria-pressed="${state.sort === "desc"}">最新在前</button>
|
||||
<button type="button" data-sort="asc" class="${state.sort === "asc" ? "active" : ""}" aria-pressed="${state.sort === "asc"}">最早在前</button>
|
||||
</div>
|
||||
<label class="switch"><input id="follow" type="checkbox" ${followChecked} /> 跟随最新</label>
|
||||
</div>
|
||||
${items.map((record, index) => {
|
||||
</div>
|
||||
${ordered.map(([index, record]) => {
|
||||
const id = recordId(record);
|
||||
const arrive = state.fresh.has(id) ? "arrive" : "";
|
||||
state.fresh.delete(id);
|
||||
const probs = record.level_probabilities || {};
|
||||
const levelName = LEVELS.find(([name]) => name === record.level)?.[1] || record.level;
|
||||
return `<article class="turn">
|
||||
<div class="card ${arrive}">
|
||||
<p class="when">第 ${index + 1} 次 · ${when(record.ts)} · ${record.elapsed_ms ?? "?"} 毫秒</p>
|
||||
<p class="prompt">${escapeHtml(record.prompt || "原文没有写入日志")}</p>
|
||||
<div class="split" role="img" aria-label="复杂度分布">
|
||||
const eventLabel = EVENTS[record.event] || (record.event ? record.event : "用户提问");
|
||||
const actionLabel = ACTIONS[record.action] || "";
|
||||
const hasLevel = Boolean(record.level);
|
||||
const distribution = hasLevel ? `<div class="split" role="img" aria-label="复杂度分布">
|
||||
${LEVELS.map(([name]) => `<span class="${name}" style="width:${Math.max(0, Math.min(100, Number(probs[name] || 0) * 100))}%"></span>`).join("")}
|
||||
</div>
|
||||
<div class="legend">
|
||||
@@ -339,7 +404,14 @@ function render() {
|
||||
const value = Number(record[field] || 0);
|
||||
return `<div class="meter"><div class="rowline"><span>${label}</span><span class="quiet">${pct(value)}</span></div><div class="track ${value >= 0.7 ? "hot" : ""}"><i style="width:${Math.max(0, Math.min(100, value * 100))}%"></i></div></div>`;
|
||||
}).join("")}
|
||||
<p class="note">${escapeHtml(levelName)}。${escapeHtml(advice(record))}</p>
|
||||
<p class="note">${escapeHtml(levelName)}。${escapeHtml(advice(record))}</p>` : "";
|
||||
const hint = !hasLevel && record.hint ? `<p class="note">${escapeHtml(record.hint)}</p>` : "";
|
||||
return `<article class="turn">
|
||||
<div class="card ${arrive}">
|
||||
<p class="when"><span class="tag">${escapeHtml(eventLabel)}</span>${actionLabel ? `<span class="tag">${escapeHtml(actionLabel)}</span>` : ""}第 ${index + 1} 次 · ${when(record.ts)} · ${record.elapsed_ms ?? "?"} 毫秒</p>
|
||||
<p class="prompt">${escapeHtml(record.prompt || "原文没有写入日志")}</p>
|
||||
${distribution}
|
||||
${hint}
|
||||
</div>
|
||||
</article>`;
|
||||
}).join("")}`;
|
||||
@@ -350,10 +422,22 @@ function render() {
|
||||
render();
|
||||
});
|
||||
}
|
||||
if (state.follow) stage.scrollTop = stage.scrollHeight;
|
||||
const segmented = stage.querySelector(".segmented");
|
||||
if (segmented) {
|
||||
segmented.addEventListener("click", (event) => {
|
||||
const button = event.target.closest("button[data-sort]");
|
||||
if (!button || button.classList.contains("active")) return;
|
||||
state.sort = button.dataset.sort === "asc" ? "asc" : "desc";
|
||||
localStorage.setItem(SORT_KEY, state.sort);
|
||||
render();
|
||||
});
|
||||
}
|
||||
if (state.follow) {
|
||||
stage.scrollTop = state.sort === "desc" ? 0 : stage.scrollHeight;
|
||||
}
|
||||
}
|
||||
function recordId(record) {
|
||||
return `${record.ts}|${record.prompt_sha256}|${record.session_id || ""}`;
|
||||
return `${record.seq || ""}|${record.ts}|${record.prompt_sha256}|${record.session_id || ""}|${record.event || ""}|${record.tool_name || ""}`;
|
||||
}
|
||||
function remember(record) {
|
||||
const id = recordId(record);
|
||||
|
||||
169
src/laya_router/evaluate.py
Normal file
169
src/laya_router/evaluate.py
Normal file
@@ -0,0 +1,169 @@
|
||||
"""Apply a tool-lifecycle event to the session trace and decide what to inject."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from laya_router.config import Settings
|
||||
from laya_router.policy import Decision, decide_post_tool, decide_pre_tool, decide_stop
|
||||
from laya_router.questions import (
|
||||
GATE_QUESTIONS,
|
||||
GATE_VERSION,
|
||||
POST_TOOL_QUESTIONS,
|
||||
POST_TOOL_VERSION,
|
||||
PRE_EDIT_QUESTIONS,
|
||||
PRE_EDIT_VERSION,
|
||||
STOP_QUESTIONS,
|
||||
STOP_VERSION,
|
||||
)
|
||||
from laya_router.trajectory import (
|
||||
BASH_TOOLS,
|
||||
EDIT_TOOLS,
|
||||
READ_TOOLS,
|
||||
SessionStore,
|
||||
SessionTrajectory,
|
||||
consecutive_failures,
|
||||
edited_without_verification,
|
||||
pre_edit_targets,
|
||||
repeated_command,
|
||||
step_from_event,
|
||||
summarize_steps,
|
||||
was_inspected,
|
||||
)
|
||||
|
||||
|
||||
def clip(value: object, limit: int = 800) -> str:
|
||||
if value is None:
|
||||
return ""
|
||||
if not isinstance(value, str):
|
||||
import json
|
||||
|
||||
try:
|
||||
value = json.dumps(value, ensure_ascii=False)
|
||||
except TypeError:
|
||||
value = str(value)
|
||||
text = value.strip()
|
||||
if len(text) <= limit:
|
||||
return text
|
||||
return text[:limit] + "…"
|
||||
|
||||
|
||||
def predict_noul(agent: object, state: dict, questions: dict) -> dict:
|
||||
predicted = agent.predict(state, questions) # type: ignore[attr-defined]
|
||||
answers = predicted.get("answers") if isinstance(predicted, dict) else None
|
||||
if not isinstance(answers, dict):
|
||||
raise ValueError("predict 没有返回 answers")
|
||||
scores: dict[str, float] = {}
|
||||
for name, spec in questions.items():
|
||||
entry = answers.get(name)
|
||||
if spec.get("type") != "noul" or not isinstance(entry, dict) or "noul" not in entry:
|
||||
raise KeyError(f"问题 {name} 缺少 noul")
|
||||
scores[name] = float(entry["noul"])
|
||||
return scores
|
||||
|
||||
|
||||
def _state(traj: SessionTrajectory, body: dict, signals: dict) -> dict:
|
||||
return {
|
||||
"user_request": clip(traj.prompt, 1500),
|
||||
"recent_steps": summarize_steps(traj.steps),
|
||||
"tool_name": body.get("tool_name") or "",
|
||||
"tool_input": clip(body.get("tool_input"), 800),
|
||||
"tool_response": clip(body.get("tool_response"), 800),
|
||||
"last_assistant_message": clip(body.get("last_assistant_message"), 800),
|
||||
"signals": signals,
|
||||
}
|
||||
|
||||
|
||||
def _summary(body: dict) -> str:
|
||||
tool = body.get("tool_name") or ""
|
||||
event = body.get("event") or ""
|
||||
detail = clip(body.get("tool_input"), 180)
|
||||
if event == "Stop":
|
||||
return "Stop"
|
||||
if tool and detail:
|
||||
return f"{event} {tool}: {detail}"
|
||||
return f"{event} {tool}".strip()
|
||||
|
||||
|
||||
def evaluate_event(agent: object, settings: Settings, store: SessionStore, body: dict) -> dict:
|
||||
event = str(body.get("event") or "")
|
||||
traj = store.get(body.get("session_id"))
|
||||
if body.get("prompt"):
|
||||
traj.remember_prompt(str(body["prompt"]))
|
||||
tool_name = str(body.get("tool_name") or "")
|
||||
after = event == "PostToolUse"
|
||||
step = step_from_event(tool_name, body.get("tool_input"), body.get("tool_response"), after=after)
|
||||
if step is not None:
|
||||
traj.add(step)
|
||||
|
||||
decision = Decision("silent")
|
||||
scores: dict = {}
|
||||
version = ""
|
||||
if event == "PostToolUse" and tool_name not in READ_TOOLS:
|
||||
signals = {
|
||||
"consecutive_failures": consecutive_failures(traj.steps),
|
||||
"repeated_command": repeated_command(traj.steps),
|
||||
}
|
||||
if settings.trajectory and tool_name in BASH_TOOLS | EDIT_TOOLS | {"apply_patch"}:
|
||||
scores = predict_noul(agent, _state(traj, body, signals), POST_TOOL_QUESTIONS)
|
||||
version = POST_TOOL_VERSION
|
||||
decision = decide_post_tool(
|
||||
enabled=True,
|
||||
signals=signals,
|
||||
scores=scores,
|
||||
last_hint=traj.last_hint,
|
||||
)
|
||||
elif event == "PreToolUse":
|
||||
targets = pre_edit_targets(tool_name, body.get("tool_input"))
|
||||
unread = bool(targets) and any(not was_inspected(traj.steps, path) for path in targets)
|
||||
signals = {"is_edit": tool_name in EDIT_TOOLS, "target_unread": unread}
|
||||
edit_scores: dict = {}
|
||||
gate_scores: dict = {}
|
||||
if settings.trajectory and signals["is_edit"]:
|
||||
edit_scores = predict_noul(agent, _state(traj, body, signals), PRE_EDIT_QUESTIONS)
|
||||
version = PRE_EDIT_VERSION
|
||||
if settings.gate and tool_name in BASH_TOOLS | EDIT_TOOLS:
|
||||
gate_scores = predict_noul(agent, _state(traj, body, signals), GATE_QUESTIONS)
|
||||
version = GATE_VERSION if not version else f"{version}+{GATE_VERSION}"
|
||||
scores = {**edit_scores, **gate_scores}
|
||||
decision = decide_pre_tool(
|
||||
trajectory_enabled=settings.trajectory,
|
||||
gate_enabled=settings.gate,
|
||||
client=str(body.get("client") or ""),
|
||||
tool_name=tool_name,
|
||||
signals=signals,
|
||||
edit_scores=edit_scores,
|
||||
gate_scores=gate_scores,
|
||||
)
|
||||
elif event == "Stop":
|
||||
signals = {"edited_without_verification": edited_without_verification(traj.steps)}
|
||||
turn_id = str(body.get("turn_id") or "")
|
||||
already = traj.stop_already_blocked(turn_id)
|
||||
if (
|
||||
settings.stop_check
|
||||
and signals["edited_without_verification"]
|
||||
and not body.get("stop_hook_active")
|
||||
and not already
|
||||
):
|
||||
scores = predict_noul(agent, _state(traj, body, signals), STOP_QUESTIONS)
|
||||
version = STOP_VERSION
|
||||
decision = decide_stop(
|
||||
enabled=settings.stop_check,
|
||||
stop_hook_active=bool(body.get("stop_hook_active")),
|
||||
already_blocked=already,
|
||||
signals=signals,
|
||||
scores=scores,
|
||||
)
|
||||
if decision.action == "block":
|
||||
traj.mark_stop_blocked(turn_id or "turn")
|
||||
|
||||
if decision.context:
|
||||
traj.remember_hint(decision.context)
|
||||
return {
|
||||
"action": decision.action,
|
||||
"context": decision.context,
|
||||
"reason": decision.reason,
|
||||
"scores": scores,
|
||||
"questions_version": version,
|
||||
"summary": _summary(body),
|
||||
"event": event,
|
||||
"tool_name": tool_name or None,
|
||||
}
|
||||
@@ -17,7 +17,7 @@ from datetime import datetime
|
||||
|
||||
from laya_router.clients.base import additional_context_output
|
||||
from laya_router.config import Settings, load_settings
|
||||
from laya_router.render import is_slash_command, render_context, truncate_prompt
|
||||
from laya_router.render import is_slash_command, render_context, render_skill, truncate_prompt
|
||||
|
||||
|
||||
def extract_prompt(payload: Mapping) -> str:
|
||||
@@ -52,20 +52,64 @@ def extract_meta(payload: Mapping) -> tuple[str | None, str | None]:
|
||||
|
||||
|
||||
def call_laya(settings: Settings, body: dict) -> dict:
|
||||
parsed = post_json(settings, "/route", body, settings.request_timeout)
|
||||
if "level" not in parsed:
|
||||
raise ValueError("路由服务返回了无法识别的 JSON")
|
||||
return parsed
|
||||
|
||||
|
||||
def post_json(settings: Settings, path: str, body: dict, timeout: float) -> dict:
|
||||
data = json.dumps(body).encode("utf-8")
|
||||
request = urllib.request.Request(
|
||||
f"{settings.base_url}/route",
|
||||
f"{settings.base_url}{path}",
|
||||
data=data,
|
||||
headers={"Content-Type": "application/json"},
|
||||
method="POST",
|
||||
)
|
||||
with urllib.request.urlopen(request, timeout=settings.request_timeout) as response:
|
||||
with urllib.request.urlopen(request, timeout=timeout) as response:
|
||||
parsed = json.loads(response.read().decode("utf-8"))
|
||||
if not isinstance(parsed, dict) or "level" not in parsed:
|
||||
if not isinstance(parsed, dict):
|
||||
raise ValueError("路由服务返回了无法识别的 JSON")
|
||||
return parsed
|
||||
|
||||
|
||||
def event_name(payload: Mapping) -> str:
|
||||
for key in ("hook_event_name", "hookEventName"):
|
||||
value = payload.get(key)
|
||||
if isinstance(value, str) and value:
|
||||
return value
|
||||
return "UserPromptSubmit"
|
||||
|
||||
|
||||
def event_output(event: str, result: Mapping) -> str:
|
||||
action = result.get("action") or "silent"
|
||||
if action == "silent":
|
||||
return "{}"
|
||||
context = result.get("context") if isinstance(result.get("context"), str) else ""
|
||||
reason = result.get("reason") if isinstance(result.get("reason"), str) else ""
|
||||
if event == "PreToolUse":
|
||||
specific: dict = {"hookEventName": "PreToolUse"}
|
||||
if action in {"deny", "ask"}:
|
||||
specific["permissionDecision"] = action
|
||||
specific["permissionDecisionReason"] = reason or context
|
||||
if context:
|
||||
specific["additionalContext"] = context
|
||||
return json.dumps({"hookSpecificOutput": specific}, ensure_ascii=False)
|
||||
if event == "PostToolUse" and context:
|
||||
return json.dumps(
|
||||
{
|
||||
"hookSpecificOutput": {
|
||||
"hookEventName": "PostToolUse",
|
||||
"additionalContext": context,
|
||||
}
|
||||
},
|
||||
ensure_ascii=False,
|
||||
)
|
||||
if event == "Stop" and action == "block" and reason:
|
||||
return json.dumps({"decision": "block", "reason": reason}, ensure_ascii=False)
|
||||
return "{}"
|
||||
|
||||
|
||||
def log_hook_error(settings: Settings, exc: BaseException) -> None:
|
||||
message = f"{datetime.now().isoformat(timespec='seconds')} {type(exc).__name__}: {exc}\n"
|
||||
try:
|
||||
@@ -80,6 +124,13 @@ def handle(payload: Mapping, env: Mapping[str, str], client: str) -> str:
|
||||
if env.get("CURSOR_VERSION"):
|
||||
return "{}"
|
||||
|
||||
event = event_name(payload)
|
||||
if event == "UserPromptSubmit":
|
||||
return handle_prompt(payload, client)
|
||||
return handle_lifecycle(payload, client, event)
|
||||
|
||||
|
||||
def handle_prompt(payload: Mapping, client: str) -> str:
|
||||
prompt = extract_prompt(payload)
|
||||
if not prompt or is_slash_command(prompt):
|
||||
return "{}"
|
||||
@@ -99,13 +150,46 @@ def handle(payload: Mapping, env: Mapping[str, str], client: str) -> str:
|
||||
}
|
||||
try:
|
||||
result = call_laya(settings, body)
|
||||
context = render_context(result)
|
||||
context = render_context(result) + render_skill(result.get("skill") if isinstance(result.get("skill"), dict) else None)
|
||||
except Exception as exc:
|
||||
log_hook_error(settings, exc)
|
||||
return "{}"
|
||||
return json.dumps(additional_context_output(context), ensure_ascii=False)
|
||||
|
||||
|
||||
def handle_lifecycle(payload: Mapping, client: str, event: str) -> str:
|
||||
try:
|
||||
settings = load_settings()
|
||||
except Exception as exc:
|
||||
print(f"Laya router config error: {exc}", file=sys.stderr)
|
||||
return "{}"
|
||||
|
||||
cwd, session_id = extract_meta(payload)
|
||||
turn_id = payload.get("turn_id") or payload.get("turnId")
|
||||
body = {
|
||||
"event": event,
|
||||
"client": client,
|
||||
"cwd": cwd,
|
||||
"session_id": session_id,
|
||||
"turn_id": turn_id if isinstance(turn_id, str) else None,
|
||||
"tool_name": payload.get("tool_name") or payload.get("toolName"),
|
||||
"tool_input": payload.get("tool_input") if "tool_input" in payload else payload.get("toolInput"),
|
||||
"tool_response": payload.get("tool_response") if "tool_response" in payload else payload.get("toolResponse"),
|
||||
"stop_hook_active": bool(payload.get("stop_hook_active") or payload.get("stopHookActive")),
|
||||
"last_assistant_message": payload.get("last_assistant_message") or payload.get("lastAssistantMessage"),
|
||||
}
|
||||
try:
|
||||
result = post_json(settings, "/event", body, settings.event_timeout)
|
||||
except Exception as exc:
|
||||
log_hook_error(settings, exc)
|
||||
return "{}"
|
||||
try:
|
||||
return event_output(event, result)
|
||||
except Exception as exc:
|
||||
log_hook_error(settings, exc)
|
||||
return "{}"
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(description="Inject a local Laya routing hint")
|
||||
parser.add_argument("--client", choices=("codex", "claude"), default="codex")
|
||||
|
||||
173
src/laya_router/mcp.py
Normal file
173
src/laya_router/mcp.py
Normal file
@@ -0,0 +1,173 @@
|
||||
"""Stdio MCP server that lets an agent ask Laya one typed question.
|
||||
|
||||
The process only speaks JSON-RPC. Judgment runs in the resident router, so this
|
||||
entrypoint does not load the model.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sys
|
||||
import urllib.request
|
||||
|
||||
from laya_router import __version__
|
||||
from laya_router.config import load_settings
|
||||
|
||||
_PROTOCOL = "2024-11-05"
|
||||
|
||||
|
||||
def _tool_schema() -> dict:
|
||||
return {
|
||||
"name": "laya_judge",
|
||||
"description": (
|
||||
"Ask the local Laya model a typed yes/no or multiple-choice question. "
|
||||
"It returns a probability, not prose. Use it when you are unsure whether "
|
||||
"to plan, read more, verify, or treat an action as risky."
|
||||
),
|
||||
"inputSchema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"state": {"type": "string", "description": "The situation to judge."},
|
||||
"question": {"type": "string", "description": "One yes/no or choice question."},
|
||||
"options": {
|
||||
"type": "array",
|
||||
"items": {"type": "string"},
|
||||
"description": "Choice labels. Omit for a yes/no probability.",
|
||||
},
|
||||
},
|
||||
"required": ["state", "question"],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def judge(arguments: dict) -> dict:
|
||||
state = arguments.get("state")
|
||||
question = arguments.get("question")
|
||||
if not isinstance(state, str) or not state.strip():
|
||||
raise ValueError("state 必须是非空字符串")
|
||||
if not isinstance(question, str) or not question.strip():
|
||||
raise ValueError("question 必须是非空字符串")
|
||||
options = arguments.get("options")
|
||||
if options is not None and not isinstance(options, list):
|
||||
raise ValueError("options 必须是字符串数组")
|
||||
settings = load_settings()
|
||||
body = {
|
||||
"state": state,
|
||||
"question": question,
|
||||
"options": options,
|
||||
"client": "mcp",
|
||||
}
|
||||
data = json.dumps(body).encode("utf-8")
|
||||
request = urllib.request.Request(
|
||||
f"{settings.base_url}/judge",
|
||||
data=data,
|
||||
headers={"Content-Type": "application/json"},
|
||||
method="POST",
|
||||
)
|
||||
with urllib.request.urlopen(request, timeout=settings.request_timeout) as response:
|
||||
parsed = json.loads(response.read().decode("utf-8"))
|
||||
if not isinstance(parsed, dict):
|
||||
raise ValueError("判定服务返回了无法识别的 JSON")
|
||||
return parsed
|
||||
|
||||
|
||||
def handle_message(message: dict) -> dict | None:
|
||||
method = message.get("method")
|
||||
msg_id = message.get("id")
|
||||
if method == "notifications/initialized" or method == "notifications/cancelled":
|
||||
return None
|
||||
if method == "initialize":
|
||||
return _result(
|
||||
msg_id,
|
||||
{
|
||||
"protocolVersion": _PROTOCOL,
|
||||
"capabilities": {"tools": {}},
|
||||
"serverInfo": {"name": "laya", "version": __version__},
|
||||
},
|
||||
)
|
||||
if method == "ping":
|
||||
return _result(msg_id, {})
|
||||
if method == "tools/list":
|
||||
return _result(msg_id, {"tools": [_tool_schema()]})
|
||||
if method == "tools/call":
|
||||
return _call_tool(msg_id, message.get("params") or {})
|
||||
if msg_id is None:
|
||||
return None
|
||||
return _error(msg_id, -32601, f"未知方法: {method}")
|
||||
|
||||
|
||||
def _call_tool(msg_id: object, params: dict) -> dict:
|
||||
name = params.get("name")
|
||||
arguments = params.get("arguments") or {}
|
||||
if name != "laya_judge":
|
||||
return _error(msg_id, -32602, f"未知工具: {name}")
|
||||
if not isinstance(arguments, dict):
|
||||
return _error(msg_id, -32602, "arguments 必须是对象")
|
||||
try:
|
||||
result = judge(arguments)
|
||||
except Exception as exc:
|
||||
return _result(
|
||||
msg_id,
|
||||
{
|
||||
"content": [{"type": "text", "text": f"Laya judge failed: {exc}"}],
|
||||
"isError": True,
|
||||
},
|
||||
)
|
||||
return _result(msg_id, {"content": [{"type": "text", "text": json.dumps(result, ensure_ascii=False)}]})
|
||||
|
||||
|
||||
def _result(msg_id: object, result: dict) -> dict:
|
||||
return {"jsonrpc": "2.0", "id": msg_id, "result": result}
|
||||
|
||||
|
||||
def _error(msg_id: object, code: int, message: str) -> dict:
|
||||
return {"jsonrpc": "2.0", "id": msg_id, "error": {"code": code, "message": message}}
|
||||
|
||||
|
||||
def read_message(stream) -> tuple[dict, str] | None:
|
||||
line = stream.readline()
|
||||
if not line:
|
||||
return None
|
||||
if line.lower().startswith(b"content-length:"):
|
||||
length = int(line.split(b":", 1)[1].strip())
|
||||
while True:
|
||||
extra = stream.readline()
|
||||
if extra in (b"\r\n", b"\n", b""):
|
||||
break
|
||||
body = stream.read(length)
|
||||
return json.loads(body.decode("utf-8")), "framed"
|
||||
stripped = line.strip()
|
||||
if not stripped:
|
||||
return read_message(stream)
|
||||
return json.loads(stripped.decode("utf-8")), "line"
|
||||
|
||||
|
||||
def write_message(stream, message: dict, framing: str) -> None:
|
||||
raw = json.dumps(message, ensure_ascii=False).encode("utf-8")
|
||||
if framing == "framed":
|
||||
stream.write(f"Content-Length: {len(raw)}\r\n\r\n".encode("ascii"))
|
||||
stream.write(raw)
|
||||
else:
|
||||
stream.write(raw + b"\n")
|
||||
stream.flush()
|
||||
|
||||
|
||||
def serve(stdin, stdout) -> int:
|
||||
while True:
|
||||
incoming = read_message(stdin)
|
||||
if incoming is None:
|
||||
return 0
|
||||
message, framing = incoming
|
||||
if not isinstance(message, dict):
|
||||
continue
|
||||
reply = handle_message(message)
|
||||
if reply is not None:
|
||||
write_message(stdout, reply, framing)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
return serve(sys.stdin.buffer, sys.stdout.buffer)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
140
src/laya_router/policy.py
Normal file
140
src/laya_router/policy.py
Normal file
@@ -0,0 +1,140 @@
|
||||
"""Turn model scores and deterministic signals into a hook decision.
|
||||
|
||||
Active mode injects a short hint at each PostToolUse checkpoint. The gate only
|
||||
tightens: it never returns allow. Codex does not support ``permissionDecision:
|
||||
ask`` (it marks the hook failed), so a medium risk there becomes a warning
|
||||
instead of an ask.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
STUCK_THRESHOLD = 0.7
|
||||
PREMATURE_THRESHOLD = 0.7
|
||||
GATE_HIGH = 0.75
|
||||
GATE_SPIKE = 0.85
|
||||
GATE_MEDIUM = 0.45
|
||||
STOP_THRESHOLD = 0.55
|
||||
|
||||
_IGNORE = "Ignore this if it does not fit what you are doing."
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Decision:
|
||||
action: str
|
||||
context: str = ""
|
||||
reason: str = ""
|
||||
|
||||
@property
|
||||
def silent(self) -> bool:
|
||||
return self.action == "silent"
|
||||
|
||||
|
||||
SILENT = Decision("silent")
|
||||
|
||||
|
||||
def _mean(scores: dict, names: tuple[str, ...]) -> float:
|
||||
values = [float(scores.get(name) or 0.0) for name in names]
|
||||
if not values:
|
||||
return 0.0
|
||||
return sum(values) / len(values)
|
||||
|
||||
|
||||
def decide_post_tool(
|
||||
*,
|
||||
enabled: bool,
|
||||
signals: dict,
|
||||
scores: dict,
|
||||
last_hint: str,
|
||||
) -> Decision:
|
||||
if not enabled:
|
||||
return SILENT
|
||||
failures = int(signals.get("consecutive_failures") or 0)
|
||||
stuck = float(scores.get("is_stuck") or 0.0)
|
||||
if failures >= 2 or stuck >= STUCK_THRESHOLD:
|
||||
text = (
|
||||
"Local routing hint: this run looks stuck "
|
||||
f"(consecutive command failures {failures}, is_stuck {stuck:.2f}). "
|
||||
f"Stop and replan before the next edit. {_IGNORE}"
|
||||
)
|
||||
else:
|
||||
bits: list[str] = []
|
||||
if float(scores.get("should_replan") or 0.0) >= STUCK_THRESHOLD:
|
||||
bits.append("consider a short replan")
|
||||
if float(scores.get("needs_more_context") or 0.0) >= STUCK_THRESHOLD:
|
||||
bits.append("read more of the repository before editing")
|
||||
if int(signals.get("repeated_command") or 0) >= 2:
|
||||
bits.append("that command just ran again")
|
||||
if not bits:
|
||||
bits.append("continue, and verify after you edit")
|
||||
text = "Local routing hint: " + "; ".join(bits) + f". {_IGNORE}"
|
||||
if text == last_hint:
|
||||
return SILENT
|
||||
return Decision("inject", context=text)
|
||||
|
||||
|
||||
def decide_pre_tool(
|
||||
*,
|
||||
trajectory_enabled: bool,
|
||||
gate_enabled: bool,
|
||||
client: str,
|
||||
tool_name: str,
|
||||
signals: dict,
|
||||
edit_scores: dict,
|
||||
gate_scores: dict,
|
||||
) -> Decision:
|
||||
risk_names = ("destructive", "irreversible", "touches_secrets_or_prod")
|
||||
risk = _mean(gate_scores, risk_names) if gate_enabled else 0.0
|
||||
spike = max((float(gate_scores.get(name) or 0.0) for name in risk_names), default=0.0)
|
||||
edit_context = ""
|
||||
if trajectory_enabled and signals.get("is_edit"):
|
||||
premature = float(edit_scores.get("edit_premature") or 0.0)
|
||||
if signals.get("target_unread") or premature >= PREMATURE_THRESHOLD:
|
||||
edit_context = (
|
||||
"Local routing hint: read the relevant code before this edit. "
|
||||
f"{_IGNORE}"
|
||||
)
|
||||
|
||||
if gate_enabled and (risk >= GATE_HIGH or spike >= GATE_SPIKE):
|
||||
reason = (
|
||||
"Local routing hint: this tool call looks destructive or hard to undo "
|
||||
f"(risk {risk:.2f}). Do not run it as-is. {_IGNORE}"
|
||||
)
|
||||
return Decision("deny", context=edit_context, reason=reason)
|
||||
|
||||
if gate_enabled and risk >= GATE_MEDIUM:
|
||||
reason = (
|
||||
"Local routing hint: this tool call may be risky "
|
||||
f"(risk {risk:.2f}) on {tool_name}. Confirm before running it. {_IGNORE}"
|
||||
)
|
||||
if client == "codex":
|
||||
text = reason if not edit_context else f"{edit_context}\n{reason}"
|
||||
return Decision("inject", context=text, reason=reason)
|
||||
return Decision("ask", context=edit_context, reason=reason)
|
||||
|
||||
if edit_context:
|
||||
return Decision("inject", context=edit_context)
|
||||
return SILENT
|
||||
|
||||
|
||||
def decide_stop(
|
||||
*,
|
||||
enabled: bool,
|
||||
stop_hook_active: bool,
|
||||
already_blocked: bool,
|
||||
signals: dict,
|
||||
scores: dict,
|
||||
) -> Decision:
|
||||
if not enabled or stop_hook_active or already_blocked:
|
||||
return SILENT
|
||||
claims = float(scores.get("claims_done_without_verification") or 0.0)
|
||||
if signals.get("edited_without_verification") and claims >= STOP_THRESHOLD:
|
||||
reason = (
|
||||
"The edited code has not been verified yet "
|
||||
f"(claims_done_without_verification {claims:.2f}). "
|
||||
"Run a test, build, or typecheck before stopping. "
|
||||
f"{_IGNORE}"
|
||||
)
|
||||
return Decision("block", reason=reason)
|
||||
return SILENT
|
||||
@@ -1,15 +1,20 @@
|
||||
"""Routing questions sent to Laya in one forward pass.
|
||||
|
||||
``choice`` is used for task level because Laya's ordinal ``score`` primitive is
|
||||
the weakest of the three. Bump ``QUESTIONS_VERSION`` when the wording changes
|
||||
the weakest of the three. Bump the matching version when the wording changes
|
||||
so logged decisions stay interpretable.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
QUESTIONS_VERSION = "1"
|
||||
POST_TOOL_VERSION = "1"
|
||||
PRE_EDIT_VERSION = "1"
|
||||
GATE_VERSION = "1"
|
||||
STOP_VERSION = "1"
|
||||
SKILL_VERSION = "1"
|
||||
|
||||
QUESTIONS: dict = {
|
||||
PROMPT_QUESTIONS: dict = {
|
||||
"task_level": {
|
||||
"type": "choice",
|
||||
"instructions": (
|
||||
@@ -59,3 +64,80 @@ QUESTIONS: dict = {
|
||||
),
|
||||
},
|
||||
}
|
||||
|
||||
POST_TOOL_QUESTIONS: dict = {
|
||||
"is_stuck": {
|
||||
"type": "noul",
|
||||
"instructions": (
|
||||
"Is the coding agent stuck: repeating itself, looping on the same "
|
||||
"error, or making no progress on the user's request?"
|
||||
),
|
||||
},
|
||||
"should_replan": {
|
||||
"type": "noul",
|
||||
"instructions": (
|
||||
"Should the coding agent stop editing and make a new plan before "
|
||||
"the next change?"
|
||||
),
|
||||
},
|
||||
"needs_more_context": {
|
||||
"type": "noul",
|
||||
"instructions": (
|
||||
"Does the agent need to read more of the repository before it can "
|
||||
"safely continue?"
|
||||
),
|
||||
},
|
||||
}
|
||||
|
||||
PRE_EDIT_QUESTIONS: dict = {
|
||||
"edit_premature": {
|
||||
"type": "noul",
|
||||
"instructions": (
|
||||
"Is the agent about to edit code it has not understood yet, without "
|
||||
"reading the relevant file or its callers?"
|
||||
),
|
||||
},
|
||||
}
|
||||
|
||||
GATE_QUESTIONS: dict = {
|
||||
"destructive": {
|
||||
"type": "noul",
|
||||
"instructions": (
|
||||
"Would this tool call delete data, overwrite unrelated files, wipe "
|
||||
"history, or otherwise destroy work?"
|
||||
),
|
||||
},
|
||||
"irreversible": {
|
||||
"type": "noul",
|
||||
"instructions": (
|
||||
"Is this tool call hard to undo, such as a force push, a published "
|
||||
"release, dropping a database, or sending a message?"
|
||||
),
|
||||
},
|
||||
"touches_secrets_or_prod": {
|
||||
"type": "noul",
|
||||
"instructions": (
|
||||
"Does this tool call read or write secrets, credentials, production "
|
||||
"configuration, or a production system?"
|
||||
),
|
||||
},
|
||||
}
|
||||
|
||||
STOP_QUESTIONS: dict = {
|
||||
"claims_done_without_verification": {
|
||||
"type": "noul",
|
||||
"instructions": (
|
||||
"Is the agent claiming the task is finished even though the edited "
|
||||
"code has not been verified by a test, build, or typecheck?"
|
||||
),
|
||||
},
|
||||
"task_incomplete": {
|
||||
"type": "noul",
|
||||
"instructions": (
|
||||
"Is the user's request still unfinished in the work shown here?"
|
||||
),
|
||||
},
|
||||
}
|
||||
|
||||
# Backward-compatible alias used by the prompt route and the warmup call.
|
||||
QUESTIONS: dict = PROMPT_QUESTIONS
|
||||
|
||||
@@ -57,3 +57,30 @@ def render_context(result: dict) -> str:
|
||||
"- high needs_web (around 0.7 or above): consult current documentation when available.\n"
|
||||
"- high needs_deep_reasoning (around 0.7 or above): reason through tradeoffs before editing.\n"
|
||||
)
|
||||
|
||||
|
||||
def render_skill(suggestion: dict | None) -> str:
|
||||
if not suggestion or not suggestion.get("name"):
|
||||
return ""
|
||||
name = suggestion["name"]
|
||||
path = suggestion.get("path") or ""
|
||||
mode = suggestion.get("mode") or "suggest"
|
||||
if mode == "content" and suggestion.get("body"):
|
||||
return (
|
||||
"\n\n<skill_relevance>\n"
|
||||
f"Relevant to the current request: {name}. Ignore this if it does not fit what the user actually asked for.\n"
|
||||
"Its instructions follow. Do not load it again with the Skill tool.\n"
|
||||
f"<skill name=\"{name}\" dir=\"{path}\">\n"
|
||||
f"{suggestion['body'].strip()}\n"
|
||||
"</skill>\n"
|
||||
"</skill_relevance>\n"
|
||||
)
|
||||
if mode == "named":
|
||||
return (
|
||||
f"\n\nSkill {name} is already loaded above; instructions unchanged. "
|
||||
"Ignore this if it does not fit.\n"
|
||||
)
|
||||
return (
|
||||
f"\n\nRelevant skill: {name} ({path}). "
|
||||
"Load it if it fits this request. Ignore this if it does not fit.\n"
|
||||
)
|
||||
|
||||
@@ -9,6 +9,7 @@ from __future__ import annotations
|
||||
import asyncio
|
||||
import gc
|
||||
import hashlib
|
||||
import itertools
|
||||
import json
|
||||
import os
|
||||
import threading
|
||||
@@ -16,6 +17,7 @@ import time
|
||||
from contextlib import asynccontextmanager
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
# Transformers can stall while probing TensorFlow. Set this before laya imports it.
|
||||
os.environ.setdefault("USE_TF", "0")
|
||||
@@ -25,12 +27,18 @@ from fastapi.responses import HTMLResponse, StreamingResponse
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from laya_router.config import Settings, load_settings
|
||||
from laya_router.evaluate import evaluate_event
|
||||
from laya_router.feed import LiveHub, read_routes
|
||||
from laya_router.questions import QUESTIONS, QUESTIONS_VERSION
|
||||
from laya_router.result import normalize_answers
|
||||
from laya_router.skills import SkillIndex, suggest_skill
|
||||
from laya_router.trajectory import SessionStore
|
||||
|
||||
_log_lock = threading.Lock()
|
||||
_hub = LiveHub()
|
||||
_sessions = SessionStore()
|
||||
_skill_index = SkillIndex()
|
||||
_seq = itertools.count(1)
|
||||
_DASHBOARD = Path(__file__).with_name("dashboard.html")
|
||||
|
||||
|
||||
@@ -41,6 +49,28 @@ class RouteRequest(BaseModel):
|
||||
session_id: str | None = None
|
||||
|
||||
|
||||
class EventRequest(BaseModel):
|
||||
event: str = Field(min_length=1)
|
||||
client: str | None = None
|
||||
cwd: str | None = None
|
||||
session_id: str | None = None
|
||||
turn_id: str | None = None
|
||||
prompt: str | None = None
|
||||
tool_name: str | None = None
|
||||
tool_input: Any = None
|
||||
tool_response: Any = None
|
||||
stop_hook_active: bool = False
|
||||
last_assistant_message: str | None = None
|
||||
|
||||
|
||||
class JudgeRequest(BaseModel):
|
||||
state: str = Field(min_length=1)
|
||||
question: str = Field(min_length=1)
|
||||
options: list[str] | None = None
|
||||
client: str | None = "mcp"
|
||||
session_id: str | None = None
|
||||
|
||||
|
||||
def candidate_devices(preference: str) -> list[str]:
|
||||
import torch
|
||||
|
||||
@@ -90,6 +120,42 @@ def load_agent(settings: Settings):
|
||||
raise RuntimeError("无法加载 Laya: " + "; ".join(errors))
|
||||
|
||||
|
||||
def _judge_questions(question: str, options: list[str] | None) -> dict:
|
||||
if options:
|
||||
criteria = {option: option for option in options[:12] if option.strip()}
|
||||
if len(criteria) < 2:
|
||||
raise HTTPException(status_code=400, detail="choice 至少需要两个选项")
|
||||
return {
|
||||
"question": {
|
||||
"type": "choice",
|
||||
"instructions": question,
|
||||
"criteria": criteria,
|
||||
}
|
||||
}
|
||||
return {"question": {"type": "noul", "instructions": question}}
|
||||
|
||||
|
||||
def _suggest(app: FastAPI, settings: Settings, req: RouteRequest) -> dict | None:
|
||||
traj = _sessions.get(req.session_id)
|
||||
traj.remember_prompt(req.prompt)
|
||||
if not settings.skills or app.state.agent is None:
|
||||
return None
|
||||
try:
|
||||
suggestion = suggest_skill(
|
||||
app.state.agent,
|
||||
_skill_index.get(req.cwd),
|
||||
req.prompt,
|
||||
inject=settings.skill_inject,
|
||||
already=set(traj.injected_skills),
|
||||
)
|
||||
except Exception as exc:
|
||||
print(f"Skill suggestion failed: {exc}", flush=True)
|
||||
return None
|
||||
if isinstance(suggestion, dict) and suggestion.get("name"):
|
||||
traj.remember_skill(str(suggestion["name"]))
|
||||
return suggestion
|
||||
|
||||
|
||||
def append_route_log(settings: Settings, record: dict) -> None:
|
||||
settings.data_dir.mkdir(parents=True, exist_ok=True)
|
||||
line = json.dumps(record, ensure_ascii=False)
|
||||
@@ -139,6 +205,8 @@ def create_app(settings: Settings | None = None) -> FastAPI:
|
||||
except Exception as exc:
|
||||
raise HTTPException(status_code=500, detail=f"predict failed: {exc}") from exc
|
||||
elapsed_ms = round((time.perf_counter() - started) * 1000, 1)
|
||||
skill = _suggest(app, settings, req)
|
||||
result["skill"] = skill
|
||||
prompt_hash = hashlib.sha256(req.prompt.encode("utf-8")).hexdigest()
|
||||
record = {
|
||||
"ts": datetime.now(timezone.utc).isoformat(),
|
||||
@@ -150,6 +218,11 @@ def create_app(settings: Settings | None = None) -> FastAPI:
|
||||
"elapsed_ms": elapsed_ms,
|
||||
"device": app.state.device,
|
||||
"questions_version": QUESTIONS_VERSION,
|
||||
"event": "UserPromptSubmit",
|
||||
"tool_name": None,
|
||||
"action": "inject",
|
||||
"seq": next(_seq),
|
||||
"skill": skill.get("name") if isinstance(skill, dict) else None,
|
||||
"level": result["level"],
|
||||
"level_confidence": result["level_confidence"],
|
||||
"level_probabilities": result["level_probabilities"],
|
||||
@@ -167,6 +240,92 @@ def create_app(settings: Settings | None = None) -> FastAPI:
|
||||
result["device"] = app.state.device
|
||||
return result
|
||||
|
||||
@app.post("/event")
|
||||
def event(req: EventRequest):
|
||||
if app.state.agent is None:
|
||||
raise HTTPException(status_code=503, detail="model not loaded")
|
||||
started = time.perf_counter()
|
||||
try:
|
||||
outcome = evaluate_event(app.state.agent, settings, _sessions, req.model_dump())
|
||||
except Exception as exc:
|
||||
raise HTTPException(status_code=500, detail=f"event failed: {exc}") from exc
|
||||
elapsed_ms = round((time.perf_counter() - started) * 1000, 1)
|
||||
summary = str(outcome.get("summary") or req.event)
|
||||
record = {
|
||||
"ts": datetime.now(timezone.utc).isoformat(),
|
||||
"client": req.client,
|
||||
"cwd": req.cwd,
|
||||
"session_id": req.session_id,
|
||||
"prompt": summary if settings.log_prompts else None,
|
||||
"prompt_sha256": hashlib.sha256(summary.encode("utf-8")).hexdigest(),
|
||||
"elapsed_ms": elapsed_ms,
|
||||
"device": app.state.device,
|
||||
"questions_version": outcome.get("questions_version") or "",
|
||||
"event": outcome.get("event"),
|
||||
"tool_name": outcome.get("tool_name"),
|
||||
"action": outcome.get("action"),
|
||||
"hint": outcome.get("context") or outcome.get("reason") or None,
|
||||
"scores": outcome.get("scores") or {},
|
||||
"seq": next(_seq),
|
||||
}
|
||||
try:
|
||||
append_route_log(settings, record)
|
||||
except Exception as exc:
|
||||
print(f"Failed to write route log: {exc}", flush=True)
|
||||
_hub.publish(record)
|
||||
outcome["elapsed_ms"] = elapsed_ms
|
||||
return outcome
|
||||
|
||||
@app.post("/judge")
|
||||
def judge(req: JudgeRequest):
|
||||
if app.state.agent is None:
|
||||
raise HTTPException(status_code=503, detail="model not loaded")
|
||||
question = _judge_questions(req.question, req.options)
|
||||
started = time.perf_counter()
|
||||
try:
|
||||
predicted = app.state.agent.predict(req.state, question)
|
||||
answers = predicted["answers"]["question"]
|
||||
except Exception as exc:
|
||||
raise HTTPException(status_code=500, detail=f"judge failed: {exc}") from exc
|
||||
elapsed_ms = round((time.perf_counter() - started) * 1000, 1)
|
||||
if req.options:
|
||||
result = {
|
||||
"type": "choice",
|
||||
"choice": answers.get("choice"),
|
||||
"confidence": float(answers.get("confidence") or 0.0),
|
||||
"probabilities": dict(answers.get("probabilities") or {}),
|
||||
}
|
||||
else:
|
||||
result = {
|
||||
"type": "noul",
|
||||
"probability": float(answers.get("noul") or 0.0),
|
||||
"confidence": float(answers.get("confidence") or answers.get("noul") or 0.0),
|
||||
}
|
||||
summary = req.question if settings.log_prompts else None
|
||||
record = {
|
||||
"ts": datetime.now(timezone.utc).isoformat(),
|
||||
"client": req.client,
|
||||
"cwd": None,
|
||||
"session_id": req.session_id,
|
||||
"prompt": summary,
|
||||
"prompt_sha256": hashlib.sha256(req.question.encode("utf-8")).hexdigest(),
|
||||
"elapsed_ms": elapsed_ms,
|
||||
"device": app.state.device,
|
||||
"questions_version": "judge",
|
||||
"event": "judge",
|
||||
"tool_name": "laya_judge",
|
||||
"action": "inject",
|
||||
"hint": json.dumps(result, ensure_ascii=False),
|
||||
"seq": next(_seq),
|
||||
}
|
||||
try:
|
||||
append_route_log(settings, record)
|
||||
except Exception as exc:
|
||||
print(f"Failed to write route log: {exc}", flush=True)
|
||||
_hub.publish(record)
|
||||
result["elapsed_ms"] = elapsed_ms
|
||||
return result
|
||||
|
||||
@app.get("/", response_class=HTMLResponse)
|
||||
def dashboard():
|
||||
return HTMLResponse(_DASHBOARD.read_text(encoding="utf-8"))
|
||||
|
||||
370
src/laya_router/skills.py
Normal file
370
src/laya_router/skills.py
Normal file
@@ -0,0 +1,370 @@
|
||||
"""Pick at most one installed skill for a user prompt.
|
||||
|
||||
The two-pass shape follows the Jev skill-suggestion cookbook: one request ranks
|
||||
the roster and asks whether any skill is warranted, a second request re-reads
|
||||
the top three and may reject all of them.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
from laya_router.questions import SKILL_VERSION
|
||||
|
||||
GATE_THRESHOLD = 0.30
|
||||
FITS_THRESHOLD = 0.30
|
||||
RANK_LIMIT = 24
|
||||
EXCERPT_CHARS = 700
|
||||
SKILL_QUESTIONS_VERSION = SKILL_VERSION
|
||||
|
||||
_GATE_NAMES = (
|
||||
"acts_on_user_system",
|
||||
"would_follow_documented_procedure",
|
||||
"prose_suffices",
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Skill:
|
||||
name: str
|
||||
description: str
|
||||
path: Path
|
||||
body: str
|
||||
|
||||
|
||||
def skill_roots(cwd: str | None, home: Path | None = None) -> list[Path]:
|
||||
base = home or Path.home()
|
||||
roots = [
|
||||
base / ".claude" / "skills",
|
||||
base / ".codex" / "skills",
|
||||
base / ".agents" / "skills",
|
||||
]
|
||||
if cwd:
|
||||
project = Path(cwd)
|
||||
roots.append(project / ".claude" / "skills")
|
||||
roots.append(project / ".codex" / "skills")
|
||||
roots.extend(_plugin_roots(base / ".claude" / "plugins" / "installed_plugins.json"))
|
||||
return roots
|
||||
|
||||
|
||||
def _plugin_roots(manifest: Path) -> list[Path]:
|
||||
if not manifest.is_file():
|
||||
return []
|
||||
try:
|
||||
loaded = json.loads(manifest.read_text(encoding="utf-8"))
|
||||
except (OSError, json.JSONDecodeError):
|
||||
return []
|
||||
found: list[Path] = []
|
||||
|
||||
def walk(node: object) -> None:
|
||||
if isinstance(node, dict):
|
||||
for key, value in node.items():
|
||||
if key in {"installPath", "install_path"} and isinstance(value, str) and value:
|
||||
found.append(Path(value))
|
||||
else:
|
||||
walk(value)
|
||||
elif isinstance(node, list):
|
||||
for item in node:
|
||||
walk(item)
|
||||
|
||||
walk(loaded)
|
||||
return found
|
||||
|
||||
|
||||
def parse_frontmatter(text: str) -> tuple[dict[str, str], str]:
|
||||
if not text.startswith("---"):
|
||||
return {}, text
|
||||
end = text.find("\n---", 3)
|
||||
if end < 0:
|
||||
return {}, text
|
||||
meta: dict[str, str] = {}
|
||||
for line in text[3:end].splitlines():
|
||||
if not line.strip() or line[:1].isspace() or ":" not in line:
|
||||
continue
|
||||
key, value = line.split(":", 1)
|
||||
meta[key.strip()] = value.strip().strip("\"'")
|
||||
body = text[end + 4 :].lstrip("\n")
|
||||
return meta, body
|
||||
|
||||
|
||||
def _disabled(meta: dict[str, str]) -> bool:
|
||||
return meta.get("disable-model-invocation", "").lower() in {"true", "yes", "1"}
|
||||
|
||||
|
||||
def discover_skills(cwd: str | None, home: Path | None = None) -> list[Skill]:
|
||||
found: list[Skill] = []
|
||||
seen: set[Path] = set()
|
||||
for root in skill_roots(cwd, home):
|
||||
if not root.is_dir():
|
||||
continue
|
||||
for path in root.rglob("SKILL.md"):
|
||||
resolved = path.resolve()
|
||||
if resolved in seen:
|
||||
continue
|
||||
seen.add(resolved)
|
||||
try:
|
||||
text = path.read_text(encoding="utf-8")
|
||||
except OSError:
|
||||
continue
|
||||
meta, body = parse_frontmatter(text)
|
||||
if _disabled(meta):
|
||||
continue
|
||||
description = meta.get("description") or _first_line(body)
|
||||
if not description:
|
||||
continue
|
||||
name = meta.get("name") or path.parent.name
|
||||
found.append(Skill(name=name, description=description, path=path, body=body))
|
||||
return _dedupe_names(found)
|
||||
|
||||
|
||||
def _first_line(body: str) -> str:
|
||||
for line in body.splitlines():
|
||||
stripped = line.strip().lstrip("#").strip()
|
||||
if stripped:
|
||||
return stripped[:240]
|
||||
return ""
|
||||
|
||||
|
||||
def _dedupe_names(skills: list[Skill]) -> list[Skill]:
|
||||
counts: dict[str, int] = {}
|
||||
for skill in skills:
|
||||
counts[skill.name] = counts.get(skill.name, 0) + 1
|
||||
taken: set[str] = set()
|
||||
renamed: list[Skill] = []
|
||||
for skill in skills:
|
||||
name = skill.name
|
||||
if counts[skill.name] > 1:
|
||||
name = f"{skill.name} ({skill.path.parent.name})"
|
||||
if name in taken:
|
||||
name = f"{name} #{len(taken)}"
|
||||
taken.add(name)
|
||||
renamed.append(Skill(name, skill.description, skill.path, skill.body))
|
||||
return renamed
|
||||
|
||||
|
||||
def directory_stamp(cwd: str | None, home: Path | None = None) -> tuple:
|
||||
stamp = []
|
||||
for root in skill_roots(cwd, home):
|
||||
if not root.exists():
|
||||
stamp.append((str(root), None))
|
||||
continue
|
||||
try:
|
||||
stamp.append((str(root), root.stat().st_mtime_ns))
|
||||
except OSError:
|
||||
stamp.append((str(root), None))
|
||||
return tuple(stamp)
|
||||
|
||||
|
||||
class SkillIndex:
|
||||
def __init__(self) -> None:
|
||||
self._stamp: tuple | None = None
|
||||
self._skills: list[Skill] = []
|
||||
|
||||
def get(self, cwd: str | None, home: Path | None = None) -> list[Skill]:
|
||||
stamp = directory_stamp(cwd, home)
|
||||
if stamp != self._stamp:
|
||||
self._skills = discover_skills(cwd, home)
|
||||
self._stamp = stamp
|
||||
return self._skills
|
||||
|
||||
|
||||
def gate_mean(scores: dict) -> float:
|
||||
acts = float(scores.get("acts_on_user_system") or 0.0)
|
||||
procedure = float(scores.get("would_follow_documented_procedure") or 0.0)
|
||||
prose = float(scores.get("prose_suffices") or 0.0)
|
||||
return (acts + procedure + (1.0 - prose)) / 3.0
|
||||
|
||||
|
||||
def rank_questions(skills: list[Skill]) -> dict:
|
||||
criteria = {skill.name: _clip(skill.description, 240) for skill in skills}
|
||||
return {
|
||||
"which": {
|
||||
"type": "choice",
|
||||
"instructions": "Which single installed skill best matches this request? Pick the closest even if none is perfect.",
|
||||
"criteria": criteria,
|
||||
},
|
||||
"acts_on_user_system": {
|
||||
"type": "noul",
|
||||
"instructions": (
|
||||
"Is the assistant being asked to act on the user's files, accounts, "
|
||||
"devices, or services, rather than only to explain or advise?"
|
||||
),
|
||||
},
|
||||
"would_follow_documented_procedure": {
|
||||
"type": "noul",
|
||||
"instructions": (
|
||||
"Would a careful expert consult a specific documented procedure or "
|
||||
"set of commands, rather than answer from general understanding?"
|
||||
),
|
||||
},
|
||||
"prose_suffices": {
|
||||
"type": "noul",
|
||||
"instructions": (
|
||||
"Could a knowledgeable generalist fully satisfy this in prose, with "
|
||||
"no tools and no access to the user's files?"
|
||||
),
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def rerank_questions(skills: list[Skill]) -> dict:
|
||||
criteria = {}
|
||||
questions: dict = {
|
||||
"which": {
|
||||
"type": "choice",
|
||||
"instructions": "Which of these skills does the specific thing the request asks for?",
|
||||
"criteria": criteria,
|
||||
}
|
||||
}
|
||||
for index, skill in enumerate(skills):
|
||||
excerpt = _clip(skill.body, EXCERPT_CHARS)
|
||||
criteria[skill.name] = _clip(f"{skill.description}\n{excerpt}", EXCERPT_CHARS + 240)
|
||||
questions[f"fits_{index}"] = {
|
||||
"type": "noul",
|
||||
"instructions": (
|
||||
f"Does the skill {skill.name} do the specific thing this request asks for? "
|
||||
f"{_clip(skill.description, 240)}"
|
||||
),
|
||||
}
|
||||
return questions
|
||||
|
||||
|
||||
def decide_skill(
|
||||
*,
|
||||
ranked_names: list[str],
|
||||
gate_scores: dict,
|
||||
rerank_choice: str | None,
|
||||
fits: list[float],
|
||||
) -> str | None:
|
||||
"""Return the winning skill name, or None when the gate or the shortlist rejects all."""
|
||||
if not ranked_names or gate_mean(gate_scores) < GATE_THRESHOLD:
|
||||
return None
|
||||
if not fits or max(fits) < FITS_THRESHOLD:
|
||||
return None
|
||||
if rerank_choice in ranked_names[: len(fits)]:
|
||||
return rerank_choice
|
||||
best = max(range(len(fits)), key=lambda index: fits[index])
|
||||
if best < len(ranked_names):
|
||||
return ranked_names[best]
|
||||
return None
|
||||
|
||||
|
||||
def suggestion_payload(skill: Skill, mode: str, *, already: bool) -> dict:
|
||||
if mode == "content" and not already:
|
||||
return {
|
||||
"name": skill.name,
|
||||
"path": str(skill.path),
|
||||
"mode": "content",
|
||||
"body": skill.body,
|
||||
"questions_version": SKILL_QUESTIONS_VERSION,
|
||||
}
|
||||
return {
|
||||
"name": skill.name,
|
||||
"path": str(skill.path),
|
||||
"mode": "named" if already else "suggest",
|
||||
"body": "",
|
||||
"questions_version": SKILL_QUESTIONS_VERSION,
|
||||
}
|
||||
|
||||
|
||||
def _clip(text: str, limit: int) -> str:
|
||||
text = " ".join(text.split())
|
||||
if len(text) <= limit:
|
||||
return text
|
||||
return text[: limit - 1] + "…"
|
||||
|
||||
|
||||
def _answers_to_scores(predicted: dict) -> dict:
|
||||
answers = predicted.get("answers") if isinstance(predicted, dict) else None
|
||||
if not isinstance(answers, dict):
|
||||
raise ValueError("skill predict 没有返回 answers")
|
||||
scores: dict = {}
|
||||
for name, entry in answers.items():
|
||||
if not isinstance(entry, dict):
|
||||
continue
|
||||
if "noul" in entry:
|
||||
scores[name] = float(entry["noul"])
|
||||
elif "choice" in entry:
|
||||
scores[name] = entry.get("choice")
|
||||
probs = entry.get("probabilities")
|
||||
if isinstance(probs, dict):
|
||||
scores[f"{name}_probabilities"] = probs
|
||||
return scores
|
||||
|
||||
|
||||
def _top_names(scores: dict, skills: list[Skill], k: int = 3) -> list[str]:
|
||||
names = [skill.name for skill in skills]
|
||||
probs = scores.get("which_probabilities")
|
||||
if isinstance(probs, dict) and probs:
|
||||
ordered = sorted(names, key=lambda name: float(probs.get(name) or 0.0), reverse=True)
|
||||
return ordered[:k]
|
||||
choice = scores.get("which")
|
||||
if not isinstance(choice, str) or choice not in names:
|
||||
return names[:k]
|
||||
ordered = [choice] + [name for name in names if name != choice]
|
||||
return ordered[:k]
|
||||
|
||||
|
||||
def suggest_skill(agent: object, skills: list[Skill], prompt: str, *, inject: str, already: set[str]) -> dict | None:
|
||||
if not skills or not prompt.strip():
|
||||
return None
|
||||
try:
|
||||
roster, scores = _first_pass(agent, skills, prompt)
|
||||
except Exception:
|
||||
return None
|
||||
if gate_mean(scores) < GATE_THRESHOLD:
|
||||
return None
|
||||
top_names = _top_names(scores, roster, 3)
|
||||
by_name = {skill.name: skill for skill in roster}
|
||||
shortlist = [by_name[name] for name in top_names if name in by_name]
|
||||
if not shortlist:
|
||||
return None
|
||||
try:
|
||||
second = agent.predict(prompt, rerank_questions(shortlist)) # type: ignore[attr-defined]
|
||||
rerank_scores = _answers_to_scores(second)
|
||||
except Exception:
|
||||
return None
|
||||
fits = [float(rerank_scores.get(f"fits_{index}") or 0.0) for index in range(len(shortlist))]
|
||||
winner_name = decide_skill(
|
||||
ranked_names=[skill.name for skill in shortlist],
|
||||
gate_scores=scores,
|
||||
rerank_choice=rerank_scores.get("which") if isinstance(rerank_scores.get("which"), str) else None,
|
||||
fits=fits,
|
||||
)
|
||||
if winner_name is None:
|
||||
return None
|
||||
skill = by_name.get(winner_name)
|
||||
if skill is None:
|
||||
return None
|
||||
mode = "content" if inject == "content" else "suggest"
|
||||
if skill.name in already and mode != "content":
|
||||
return None
|
||||
return suggestion_payload(skill, mode, already=skill.name in already)
|
||||
|
||||
|
||||
def _first_pass(agent: object, skills: list[Skill], prompt: str) -> tuple[list[Skill], dict]:
|
||||
if len(skills) <= RANK_LIMIT:
|
||||
predicted = agent.predict(prompt, rank_questions(skills)) # type: ignore[attr-defined]
|
||||
return skills, _answers_to_scores(predicted)
|
||||
try:
|
||||
from laya.shortlist import embed_fn_from_agent, predict_shortlist
|
||||
|
||||
predicted = predict_shortlist(
|
||||
agent,
|
||||
prompt,
|
||||
rank_questions(skills),
|
||||
embed_fn_from_agent(agent),
|
||||
k=RANK_LIMIT,
|
||||
)
|
||||
labels = predicted.get("shortlist", {}).get("which", {}).get("labels") or []
|
||||
by_name = {skill.name: skill for skill in skills}
|
||||
kept = [by_name[label] for label in labels if label in by_name]
|
||||
if kept:
|
||||
return kept, _answers_to_scores(predicted)
|
||||
except Exception:
|
||||
pass
|
||||
trimmed = sorted(skills, key=lambda skill: len(skill.description), reverse=True)[:RANK_LIMIT]
|
||||
predicted = agent.predict(prompt, rank_questions(trimmed)) # type: ignore[attr-defined]
|
||||
return trimmed, _answers_to_scores(predicted)
|
||||
232
src/laya_router/trajectory.py
Normal file
232
src/laya_router/trajectory.py
Normal file
@@ -0,0 +1,232 @@
|
||||
"""In-memory session trace and the deterministic signals derived from it.
|
||||
|
||||
The hook process exits after every event. The resident server keeps one
|
||||
``SessionTrajectory`` per session so later tool calls can see what was already
|
||||
read, edited, or verified.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
import threading
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
EDIT_TOOLS = {"Edit", "Write", "MultiEdit", "NotebookEdit", "apply_patch"}
|
||||
READ_TOOLS = {"Read", "Grep", "Glob"}
|
||||
BASH_TOOLS = {"Bash", "Shell", "bash"}
|
||||
|
||||
_VERIFY = re.compile(
|
||||
r"(?i)(?:^|[;&|]\s*)(?:pytest|python3? -m pytest|uv run pytest|npm(?: run)? test|"
|
||||
r"pnpm(?: run)? test|yarn test|bun test|cargo test|go test|vitest|jest|"
|
||||
r"npm run build|pnpm(?: run)? build|yarn build|tsc\b|mypy\b|ruff\b|"
|
||||
r"make (?:test|check)|cargo build)\b"
|
||||
)
|
||||
_PATCH_PATH = re.compile(r"^\*\*\* (?:Update|Add|Delete) File: (.+)$", re.M)
|
||||
_EXIT_CODE = re.compile(r"(?i)(?:exit code[: ]+[1-9]|command failed|traceback \(most recent)")
|
||||
|
||||
|
||||
@dataclass
|
||||
class Step:
|
||||
kind: str
|
||||
tool_name: str
|
||||
detail: str
|
||||
failed: bool = False
|
||||
paths: tuple[str, ...] = ()
|
||||
|
||||
|
||||
@dataclass
|
||||
class SessionTrajectory:
|
||||
prompt: str = ""
|
||||
steps: list[Step] = field(default_factory=list)
|
||||
last_hint: str = ""
|
||||
injected_skills: set[str] = field(default_factory=set)
|
||||
blocked_turns: set[str] = field(default_factory=set)
|
||||
|
||||
def remember_prompt(self, prompt: str) -> None:
|
||||
if prompt:
|
||||
self.prompt = prompt
|
||||
|
||||
def add(self, step: Step, *, limit: int = 40) -> None:
|
||||
self.steps.append(step)
|
||||
if len(self.steps) > limit:
|
||||
del self.steps[:-limit]
|
||||
|
||||
def remember_hint(self, text: str) -> None:
|
||||
self.last_hint = text
|
||||
|
||||
def remember_skill(self, name: str) -> None:
|
||||
if name:
|
||||
self.injected_skills.add(name)
|
||||
|
||||
def mark_stop_blocked(self, turn_id: str) -> None:
|
||||
if turn_id:
|
||||
self.blocked_turns.add(turn_id)
|
||||
|
||||
def stop_already_blocked(self, turn_id: str) -> bool:
|
||||
return bool(turn_id) and turn_id in self.blocked_turns
|
||||
|
||||
|
||||
class SessionStore:
|
||||
def __init__(self, *, max_sessions: int = 200) -> None:
|
||||
self._lock = threading.Lock()
|
||||
self._items: dict[str, SessionTrajectory] = {}
|
||||
self._order: list[str] = []
|
||||
self._max_sessions = max_sessions
|
||||
|
||||
def get(self, session_id: str | None) -> SessionTrajectory:
|
||||
key = session_id or "anonymous"
|
||||
with self._lock:
|
||||
traj = self._items.get(key)
|
||||
if traj is None:
|
||||
traj = SessionTrajectory()
|
||||
self._items[key] = traj
|
||||
self._order.append(key)
|
||||
overflow = len(self._order) - self._max_sessions
|
||||
if overflow > 0:
|
||||
for old in self._order[:overflow]:
|
||||
self._items.pop(old, None)
|
||||
del self._order[:overflow]
|
||||
return traj
|
||||
|
||||
|
||||
def command_text(tool_name: str, tool_input: object) -> str:
|
||||
if tool_name not in BASH_TOOLS and tool_name != "apply_patch":
|
||||
return ""
|
||||
if isinstance(tool_input, str):
|
||||
return tool_input
|
||||
if isinstance(tool_input, dict):
|
||||
for key in ("command", "cmd", "script"):
|
||||
value = tool_input.get(key)
|
||||
if isinstance(value, str) and value.strip():
|
||||
return value
|
||||
return ""
|
||||
|
||||
|
||||
def target_paths(tool_name: str, tool_input: object) -> tuple[str, ...]:
|
||||
found: list[str] = []
|
||||
if isinstance(tool_input, dict):
|
||||
for key in ("file_path", "filePath", "path", "notebook_path"):
|
||||
value = tool_input.get(key)
|
||||
if isinstance(value, str) and value.strip():
|
||||
found.append(value.strip())
|
||||
edits = tool_input.get("edits")
|
||||
if isinstance(edits, list):
|
||||
for edit in edits:
|
||||
if isinstance(edit, dict):
|
||||
found.extend(target_paths(tool_name, edit))
|
||||
text = command_text(tool_name, tool_input) if tool_name == "apply_patch" else ""
|
||||
if isinstance(tool_input, str) and tool_name == "apply_patch":
|
||||
text = tool_input
|
||||
if text:
|
||||
found.extend(match.group(1).strip() for match in _PATCH_PATH.finditer(text))
|
||||
deduped: list[str] = []
|
||||
for path in found:
|
||||
if path not in deduped:
|
||||
deduped.append(path)
|
||||
return tuple(deduped)
|
||||
|
||||
|
||||
def response_failed(tool_response: object) -> bool:
|
||||
if isinstance(tool_response, dict):
|
||||
for key in ("exit_code", "exitCode", "code"):
|
||||
value = tool_response.get(key)
|
||||
if isinstance(value, int) and value != 0:
|
||||
return True
|
||||
if tool_response.get("is_error") or tool_response.get("isError"):
|
||||
return True
|
||||
if tool_response.get("error"):
|
||||
return True
|
||||
text = json.dumps(tool_response, ensure_ascii=False)
|
||||
elif isinstance(tool_response, str):
|
||||
text = tool_response
|
||||
else:
|
||||
return False
|
||||
return _EXIT_CODE.search(text) is not None
|
||||
|
||||
|
||||
def is_verify_command(command: str) -> bool:
|
||||
return bool(command) and _VERIFY.search(command) is not None
|
||||
|
||||
|
||||
def step_from_event(tool_name: str, tool_input: object, tool_response: object, *, after: bool) -> Step | None:
|
||||
"""Build a step. Pre-tool events do not know the outcome, so they are not recorded."""
|
||||
if not after or not tool_name:
|
||||
return None
|
||||
paths = target_paths(tool_name, tool_input)
|
||||
if tool_name in READ_TOOLS:
|
||||
return Step("read", tool_name, paths[0] if paths else "", paths=paths)
|
||||
if tool_name in EDIT_TOOLS:
|
||||
detail = paths[0] if paths else ""
|
||||
return Step("edit", tool_name, detail, paths=paths)
|
||||
if tool_name in BASH_TOOLS:
|
||||
command = command_text(tool_name, tool_input).strip()
|
||||
return Step("bash", tool_name, command, failed=response_failed(tool_response))
|
||||
return Step("other", tool_name, "")
|
||||
|
||||
|
||||
def consecutive_failures(steps: list[Step]) -> int:
|
||||
count = 0
|
||||
for step in reversed(steps):
|
||||
if step.kind != "bash":
|
||||
continue
|
||||
if not step.failed:
|
||||
break
|
||||
count += 1
|
||||
return count
|
||||
|
||||
|
||||
def repeated_command(steps: list[Step]) -> int:
|
||||
commands = [step.detail for step in steps if step.kind == "bash" and step.detail]
|
||||
if not commands:
|
||||
return 0
|
||||
last = commands[-1]
|
||||
return sum(1 for command in commands[-4:] if command == last)
|
||||
|
||||
|
||||
def edited_without_verification(steps: list[Step]) -> bool:
|
||||
pending = False
|
||||
for step in steps:
|
||||
if step.kind == "edit":
|
||||
pending = True
|
||||
elif step.kind == "bash" and is_verify_command(step.detail) and not step.failed:
|
||||
pending = False
|
||||
return pending
|
||||
|
||||
|
||||
def was_inspected(steps: list[Step], target: str) -> bool:
|
||||
if not target:
|
||||
return False
|
||||
for step in steps:
|
||||
if step.kind != "read":
|
||||
continue
|
||||
for path in step.paths:
|
||||
if not path:
|
||||
continue
|
||||
if target == path or target.startswith(path.rstrip("/") + "/"):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def pre_edit_targets(tool_name: str, tool_input: object) -> tuple[str, ...]:
|
||||
if tool_name not in EDIT_TOOLS:
|
||||
return ()
|
||||
return target_paths(tool_name, tool_input)
|
||||
|
||||
|
||||
def summarize_steps(steps: list[Step], *, limit: int = 6) -> list[dict]:
|
||||
recent = steps[-limit:]
|
||||
rows = []
|
||||
for step in recent:
|
||||
detail = step.detail
|
||||
if len(detail) > 300:
|
||||
detail = detail[:300] + "…"
|
||||
rows.append(
|
||||
{
|
||||
"kind": step.kind,
|
||||
"tool": step.tool_name,
|
||||
"detail": detail,
|
||||
"failed": step.failed,
|
||||
}
|
||||
)
|
||||
return rows
|
||||
176
tests/test_cli.py
Normal file
176
tests/test_cli.py
Normal file
@@ -0,0 +1,176 @@
|
||||
"""Tests for ``laya_router.cli`` PATH / symlink helpers.
|
||||
|
||||
These cover the path exposure logic added so ``laya-router`` and
|
||||
``laya-hook`` work from any shell without manual ``PATH`` setup.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
from laya_router.cli import (
|
||||
_apply_path_symlinks,
|
||||
_bin_link_targets,
|
||||
_remove_path_symlinks,
|
||||
_user_path_bin_dir,
|
||||
ensure_path_symlinks,
|
||||
remove_path_symlinks,
|
||||
)
|
||||
|
||||
|
||||
class _FakeSettings:
|
||||
"""Minimal stand-in for ``Settings`` carrying just the paths we touch."""
|
||||
|
||||
def __init__(self, router: Path, hook: Path):
|
||||
self.router_bin = router
|
||||
self.hook_bin = hook
|
||||
self.mcp_bin = router.with_name("laya-mcp")
|
||||
|
||||
|
||||
def _make_fake_binaries(tmp_path: Path) -> tuple[Path, Path, _FakeSettings]:
|
||||
"""Build two venv-style binaries on disk and return (router, hook, settings)."""
|
||||
src_dir = tmp_path / "venv_bin"
|
||||
src_dir.mkdir()
|
||||
router = src_dir / "laya-router"
|
||||
hook = src_dir / "laya-hook"
|
||||
router.write_text("#!/bin/sh\necho router\n")
|
||||
hook.write_text("#!/bin/sh\necho hook\n")
|
||||
return router, hook, _FakeSettings(router, hook)
|
||||
|
||||
|
||||
def test_bin_link_targets_returns_both_binaries():
|
||||
from laya_router.config import load_settings
|
||||
|
||||
targets = _bin_link_targets(load_settings())
|
||||
names = [name for name, _ in targets]
|
||||
assert names == ["laya-router", "laya-hook", "laya-mcp"]
|
||||
assert targets[0][1].is_file()
|
||||
assert targets[1][1].is_file()
|
||||
|
||||
|
||||
def test_user_path_bin_dir_prefers_local_bin(monkeypatch, tmp_path):
|
||||
monkeypatch.setenv("PATH", f"{tmp_path}:/usr/bin")
|
||||
preferred = Path.home() / ".local" / "bin"
|
||||
if preferred.is_dir() and os.access(preferred, os.W_OK):
|
||||
assert _user_path_bin_dir() == preferred
|
||||
else:
|
||||
assert _user_path_bin_dir() == tmp_path
|
||||
|
||||
|
||||
def test_user_path_bin_dir_skips_system_dirs(monkeypatch):
|
||||
# Force "no writable dir exists" by overriding to a non-existent path.
|
||||
monkeypatch.setenv("LAYA_ROUTER_BIN_DIR", "/this/path/does/not/exist")
|
||||
monkeypatch.setenv("PATH", "/usr/local/bin:/usr/bin:/bin:/sbin")
|
||||
assert _user_path_bin_dir() is None
|
||||
|
||||
|
||||
def test_user_path_bin_dir_respects_env_override(monkeypatch, tmp_path):
|
||||
monkeypatch.setenv("LAYA_ROUTER_BIN_DIR", str(tmp_path))
|
||||
assert _user_path_bin_dir() == tmp_path
|
||||
|
||||
|
||||
def test_user_path_bin_dir_env_override_rejects_unwritable(monkeypatch, tmp_path):
|
||||
monkeypatch.setenv("LAYA_ROUTER_BIN_DIR", "/this/path/does/not/exist")
|
||||
assert _user_path_bin_dir() is None
|
||||
|
||||
|
||||
def test_apply_path_symlinks_creates_links(tmp_path):
|
||||
router, hook, settings = _make_fake_binaries(tmp_path)
|
||||
bin_dir = tmp_path / "bin"
|
||||
bin_dir.mkdir()
|
||||
|
||||
_apply_path_symlinks(settings, bin_dir)
|
||||
|
||||
assert (bin_dir / "laya-router").is_symlink()
|
||||
assert (bin_dir / "laya-router").resolve() == router.resolve()
|
||||
assert (bin_dir / "laya-hook").is_symlink()
|
||||
assert (bin_dir / "laya-hook").resolve() == hook.resolve()
|
||||
|
||||
|
||||
def test_apply_path_symlinks_is_noop_when_correct(tmp_path):
|
||||
_, _, settings = _make_fake_binaries(tmp_path)
|
||||
bin_dir = tmp_path / "bin"
|
||||
bin_dir.mkdir()
|
||||
|
||||
_apply_path_symlinks(settings, bin_dir)
|
||||
first = (bin_dir / "laya-router").lstat()
|
||||
|
||||
_apply_path_symlinks(settings, bin_dir)
|
||||
second = (bin_dir / "laya-router").lstat()
|
||||
|
||||
# Re-running on a correct symlink must not rewrite it.
|
||||
assert first.st_mtime_ns == second.st_mtime_ns
|
||||
|
||||
|
||||
def test_apply_path_symlinks_repairs_wrong_target(tmp_path):
|
||||
router, _, settings = _make_fake_binaries(tmp_path)
|
||||
bin_dir = tmp_path / "bin"
|
||||
bin_dir.mkdir()
|
||||
(bin_dir / "laya-router").symlink_to("/etc/passwd")
|
||||
|
||||
_apply_path_symlinks(settings, bin_dir)
|
||||
assert (bin_dir / "laya-router").resolve() == router.resolve()
|
||||
|
||||
|
||||
def test_apply_path_symlinks_repairs_broken_symlink(tmp_path):
|
||||
router, _, settings = _make_fake_binaries(tmp_path)
|
||||
bin_dir = tmp_path / "bin"
|
||||
bin_dir.mkdir()
|
||||
(bin_dir / "laya-router").symlink_to("/nonexistent/path")
|
||||
|
||||
_apply_path_symlinks(settings, bin_dir)
|
||||
assert (bin_dir / "laya-router").resolve() == router.resolve()
|
||||
|
||||
|
||||
def test_apply_path_symlinks_skips_unrelated_file(tmp_path, capsys):
|
||||
router, _, settings = _make_fake_binaries(tmp_path)
|
||||
bin_dir = tmp_path / "bin"
|
||||
bin_dir.mkdir()
|
||||
existing = bin_dir / "laya-router"
|
||||
existing.write_text("user's binary")
|
||||
|
||||
_apply_path_symlinks(settings, bin_dir)
|
||||
|
||||
captured = capsys.readouterr()
|
||||
assert "已被占用" in captured.err
|
||||
# The user's file must not be clobbered.
|
||||
assert existing.read_text() == "user's binary"
|
||||
# And the other one should still get a fresh symlink.
|
||||
assert (bin_dir / "laya-hook").is_symlink()
|
||||
assert (bin_dir / "laya-router").resolve() != router.resolve()
|
||||
|
||||
|
||||
def test_remove_path_symlinks_removes_both(tmp_path):
|
||||
_, _, settings = _make_fake_binaries(tmp_path)
|
||||
bin_dir = tmp_path / "bin"
|
||||
bin_dir.mkdir()
|
||||
_apply_path_symlinks(settings, bin_dir)
|
||||
assert (bin_dir / "laya-router").is_symlink()
|
||||
assert (bin_dir / "laya-hook").is_symlink()
|
||||
|
||||
_remove_path_symlinks(settings, bin_dir)
|
||||
assert not (bin_dir / "laya-router").exists()
|
||||
assert not (bin_dir / "laya-hook").exists()
|
||||
|
||||
|
||||
def test_remove_path_symlinks_idempotent(tmp_path):
|
||||
_, _, settings = _make_fake_binaries(tmp_path)
|
||||
bin_dir = tmp_path / "bin"
|
||||
bin_dir.mkdir()
|
||||
|
||||
# No symlinks created — remove should still be safe.
|
||||
_remove_path_symlinks(settings, bin_dir)
|
||||
_remove_path_symlinks(settings, bin_dir)
|
||||
|
||||
|
||||
def test_public_helpers_warn_when_no_writable_path(monkeypatch, capsys):
|
||||
monkeypatch.setenv("LAYA_ROUTER_BIN_DIR", "/this/path/does/not/exist")
|
||||
settings = _FakeSettings(Path("/nonexistent/router"), Path("/nonexistent/hook"))
|
||||
|
||||
ensure_path_symlinks(settings)
|
||||
captured = capsys.readouterr()
|
||||
assert "没有可写的目录" in captured.err
|
||||
|
||||
# remove_path_symlinks is silent in the same situation.
|
||||
remove_path_symlinks(settings)
|
||||
@@ -43,6 +43,21 @@ def test_claude_merge_is_idempotent_and_keeps_existing_hooks(tmp_path):
|
||||
for item in group["hooks"]
|
||||
]
|
||||
assert commands == [f'"{tmp_path / "runtime" / ".venv" / "bin" / "laya-hook"}" --client claude']
|
||||
for event, matcher in (
|
||||
("PreToolUse", "Bash|Edit|Write|MultiEdit|apply_patch"),
|
||||
("PostToolUse", "Bash|Edit|Write|MultiEdit|apply_patch|Read|Grep|Glob"),
|
||||
):
|
||||
ours = [
|
||||
group
|
||||
for group in saved["hooks"][event]
|
||||
if any("laya-hook" in item.get("command", "") for item in group.get("hooks", []))
|
||||
]
|
||||
assert ours[-1]["matcher"] == matcher
|
||||
assert any(
|
||||
"laya-hook" in item.get("command", "")
|
||||
for group in saved["hooks"]["Stop"]
|
||||
for item in group["hooks"]
|
||||
)
|
||||
assert list(path.parent.glob("settings.json.bak-*"))
|
||||
|
||||
assert client.uninstall() is True
|
||||
|
||||
60
tests/test_evaluate.py
Normal file
60
tests/test_evaluate.py
Normal file
@@ -0,0 +1,60 @@
|
||||
from laya_router.config import Settings
|
||||
from laya_router.evaluate import evaluate_event
|
||||
from laya_router.mcp import handle_message
|
||||
from laya_router.trajectory import SessionStore
|
||||
|
||||
|
||||
class _Agent:
|
||||
def predict(self, state, questions):
|
||||
answers = {}
|
||||
for name, spec in questions.items():
|
||||
if spec["type"] == "noul":
|
||||
value = 0.9 if name in {"is_stuck", "destructive", "claims_done_without_verification"} else 0.1
|
||||
answers[name] = {"noul": value}
|
||||
return {"answers": answers}
|
||||
|
||||
|
||||
def test_stop_blocks_only_once_per_turn(tmp_path):
|
||||
settings = Settings(home=tmp_path, skills=False)
|
||||
store = SessionStore()
|
||||
body = {
|
||||
"event": "PostToolUse",
|
||||
"session_id": "s",
|
||||
"tool_name": "Edit",
|
||||
"tool_input": {"file_path": "src/app.py"},
|
||||
"client": "claude",
|
||||
}
|
||||
evaluate_event(_Agent(), settings, store, body)
|
||||
stop = {
|
||||
"event": "Stop",
|
||||
"session_id": "s",
|
||||
"turn_id": "t1",
|
||||
"client": "codex",
|
||||
"stop_hook_active": False,
|
||||
}
|
||||
first = evaluate_event(_Agent(), settings, store, stop)
|
||||
assert first["action"] == "block"
|
||||
second = evaluate_event(_Agent(), settings, store, stop)
|
||||
assert second["action"] == "silent"
|
||||
|
||||
|
||||
def test_read_is_recorded_without_a_hint(tmp_path):
|
||||
settings = Settings(home=tmp_path)
|
||||
outcome = evaluate_event(
|
||||
_Agent(),
|
||||
settings,
|
||||
SessionStore(),
|
||||
{
|
||||
"event": "PostToolUse",
|
||||
"session_id": "s",
|
||||
"tool_name": "Read",
|
||||
"tool_input": {"file_path": "src/app.py"},
|
||||
},
|
||||
)
|
||||
assert outcome["action"] == "silent"
|
||||
|
||||
|
||||
def test_mcp_lists_the_judge_tool():
|
||||
listed = handle_message({"jsonrpc": "2.0", "id": 2, "method": "tools/list"})
|
||||
assert listed["result"]["tools"][0]["name"] == "laya_judge"
|
||||
assert handle_message({"jsonrpc": "2.0", "method": "notifications/initialized"}) is None
|
||||
@@ -79,6 +79,52 @@ def test_success_output(monkeypatch, tmp_path):
|
||||
assert "hint only" in context
|
||||
|
||||
|
||||
def test_pre_tool_deny_and_post_tool_context(monkeypatch, tmp_path):
|
||||
monkeypatch.setenv("LAYA_ROUTER_HOME", str(tmp_path))
|
||||
|
||||
def respond(settings, path, body, timeout):
|
||||
assert path == "/event"
|
||||
assert timeout == settings.event_timeout
|
||||
if body["event"] == "PreToolUse":
|
||||
return {"action": "deny", "reason": "do not delete", "context": ""}
|
||||
return {"action": "inject", "context": "replan", "reason": ""}
|
||||
|
||||
monkeypatch.setattr("laya_router.hook.post_json", respond)
|
||||
denied = json.loads(
|
||||
handle(
|
||||
{"hook_event_name": "PreToolUse", "tool_name": "Bash", "tool_input": {"command": "rm -rf /"}},
|
||||
{},
|
||||
"claude",
|
||||
)
|
||||
)
|
||||
assert denied["hookSpecificOutput"]["permissionDecision"] == "deny"
|
||||
posted = json.loads(
|
||||
handle(
|
||||
{"hook_event_name": "PostToolUse", "tool_name": "Bash", "tool_input": {"command": "pytest"}},
|
||||
{},
|
||||
"codex",
|
||||
)
|
||||
)
|
||||
assert posted["hookSpecificOutput"]["hookEventName"] == "PostToolUse"
|
||||
assert posted["hookSpecificOutput"]["additionalContext"] == "replan"
|
||||
|
||||
|
||||
def test_stop_block_and_fail_open(monkeypatch, tmp_path):
|
||||
monkeypatch.setenv("LAYA_ROUTER_HOME", str(tmp_path))
|
||||
monkeypatch.setattr(
|
||||
"laya_router.hook.post_json",
|
||||
lambda settings, path, body, timeout: {"action": "block", "reason": "run pytest"},
|
||||
)
|
||||
blocked = json.loads(handle({"hook_event_name": "Stop", "stop_hook_active": False}, {}, "codex"))
|
||||
assert blocked == {"decision": "block", "reason": "run pytest"}
|
||||
|
||||
def boom(settings, path, body, timeout):
|
||||
raise URLError("connection refused")
|
||||
|
||||
monkeypatch.setattr("laya_router.hook.post_json", boom)
|
||||
assert handle({"hook_event_name": "Stop"}, {}, "claude") == "{}"
|
||||
|
||||
|
||||
def _payload():
|
||||
return {
|
||||
"level": "normal",
|
||||
|
||||
102
tests/test_policy.py
Normal file
102
tests/test_policy.py
Normal file
@@ -0,0 +1,102 @@
|
||||
from laya_router.policy import decide_post_tool, decide_pre_tool, decide_stop
|
||||
|
||||
|
||||
def test_post_tool_dedupes_and_escalates_when_stuck():
|
||||
signals = {"consecutive_failures": 0, "repeated_command": 1}
|
||||
scores = {"is_stuck": 0.1, "should_replan": 0.1, "needs_more_context": 0.1}
|
||||
first = decide_post_tool(enabled=True, signals=signals, scores=scores, last_hint="")
|
||||
assert first.action == "inject"
|
||||
again = decide_post_tool(enabled=True, signals=signals, scores=scores, last_hint=first.context)
|
||||
assert again.action == "silent"
|
||||
stuck = decide_post_tool(
|
||||
enabled=True,
|
||||
signals={"consecutive_failures": 2, "repeated_command": 2},
|
||||
scores={"is_stuck": 0.9},
|
||||
last_hint="",
|
||||
)
|
||||
assert "replan" in stuck.context
|
||||
|
||||
|
||||
def test_gate_never_allows_and_codex_does_not_ask():
|
||||
quiet = decide_pre_tool(
|
||||
trajectory_enabled=True,
|
||||
gate_enabled=True,
|
||||
client="claude",
|
||||
tool_name="Bash",
|
||||
signals={"is_edit": False, "target_unread": False},
|
||||
edit_scores={},
|
||||
gate_scores={"destructive": 0.1, "irreversible": 0.1, "touches_secrets_or_prod": 0.1},
|
||||
)
|
||||
assert quiet.action == "silent"
|
||||
medium = decide_pre_tool(
|
||||
trajectory_enabled=False,
|
||||
gate_enabled=True,
|
||||
client="claude",
|
||||
tool_name="Bash",
|
||||
signals={},
|
||||
edit_scores={},
|
||||
gate_scores={"destructive": 0.6, "irreversible": 0.5, "touches_secrets_or_prod": 0.4},
|
||||
)
|
||||
assert medium.action == "ask"
|
||||
codex = decide_pre_tool(
|
||||
trajectory_enabled=False,
|
||||
gate_enabled=True,
|
||||
client="codex",
|
||||
tool_name="Bash",
|
||||
signals={},
|
||||
edit_scores={},
|
||||
gate_scores={"destructive": 0.6, "irreversible": 0.5, "touches_secrets_or_prod": 0.4},
|
||||
)
|
||||
assert codex.action == "inject"
|
||||
denied = decide_pre_tool(
|
||||
trajectory_enabled=False,
|
||||
gate_enabled=True,
|
||||
client="codex",
|
||||
tool_name="Bash",
|
||||
signals={},
|
||||
edit_scores={},
|
||||
gate_scores={"destructive": 0.95, "irreversible": 0.2, "touches_secrets_or_prod": 0.1},
|
||||
)
|
||||
assert denied.action == "deny"
|
||||
assert denied.action != "allow"
|
||||
|
||||
|
||||
def test_unread_edit_injects_hint():
|
||||
decision = decide_pre_tool(
|
||||
trajectory_enabled=True,
|
||||
gate_enabled=False,
|
||||
client="claude",
|
||||
tool_name="Edit",
|
||||
signals={"is_edit": True, "target_unread": True},
|
||||
edit_scores={"edit_premature": 0.2},
|
||||
gate_scores={},
|
||||
)
|
||||
assert decision.action == "inject"
|
||||
assert "read" in decision.context
|
||||
|
||||
|
||||
def test_stop_blocks_once_and_respects_the_loop_guard():
|
||||
scores = {"claims_done_without_verification": 0.8}
|
||||
signals = {"edited_without_verification": True}
|
||||
blocked = decide_stop(
|
||||
enabled=True,
|
||||
stop_hook_active=False,
|
||||
already_blocked=False,
|
||||
signals=signals,
|
||||
scores=scores,
|
||||
)
|
||||
assert blocked.action == "block"
|
||||
assert decide_stop(
|
||||
enabled=True,
|
||||
stop_hook_active=True,
|
||||
already_blocked=False,
|
||||
signals=signals,
|
||||
scores=scores,
|
||||
).action == "silent"
|
||||
assert decide_stop(
|
||||
enabled=True,
|
||||
stop_hook_active=False,
|
||||
already_blocked=True,
|
||||
signals=signals,
|
||||
scores=scores,
|
||||
).action == "silent"
|
||||
76
tests/test_skills.py
Normal file
76
tests/test_skills.py
Normal file
@@ -0,0 +1,76 @@
|
||||
from laya_router.skills import decide_skill, discover_skills, suggest_skill
|
||||
|
||||
|
||||
def _skill(root, name, description, *, disabled=False, body="Do the documented steps."):
|
||||
folder = root / name
|
||||
folder.mkdir(parents=True)
|
||||
flag = "true" if disabled else "false"
|
||||
(folder / "SKILL.md").write_text(
|
||||
f"---\nname: {name}\ndescription: {description}\ndisable-model-invocation: {flag}\n---\n{body}\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
|
||||
def test_discover_skips_disabled_skills(tmp_path):
|
||||
home = tmp_path / "home"
|
||||
_skill(home / ".claude" / "skills", "commit", "Create a git commit")
|
||||
_skill(home / ".agents" / "skills", "hidden", "Should stay hidden", disabled=True)
|
||||
found = discover_skills(None, home)
|
||||
assert [skill.name for skill in found] == ["commit"]
|
||||
|
||||
|
||||
def test_gate_and_fits_thresholds():
|
||||
assert decide_skill(
|
||||
ranked_names=["commit", "review"],
|
||||
gate_scores={"acts_on_user_system": 0.1, "would_follow_documented_procedure": 0.1, "prose_suffices": 0.9},
|
||||
rerank_choice="commit",
|
||||
fits=[0.9, 0.1],
|
||||
) is None
|
||||
assert decide_skill(
|
||||
ranked_names=["commit", "review"],
|
||||
gate_scores={"acts_on_user_system": 0.8, "would_follow_documented_procedure": 0.8, "prose_suffices": 0.1},
|
||||
rerank_choice="review",
|
||||
fits=[0.2, 0.1],
|
||||
) is None
|
||||
assert decide_skill(
|
||||
ranked_names=["commit", "review"],
|
||||
gate_scores={"acts_on_user_system": 0.8, "would_follow_documented_procedure": 0.8, "prose_suffices": 0.1},
|
||||
rerank_choice="review",
|
||||
fits=[0.4, 0.7],
|
||||
) == "review"
|
||||
|
||||
|
||||
class _Agent:
|
||||
def __init__(self, answers):
|
||||
self.answers = list(answers)
|
||||
self.calls = 0
|
||||
|
||||
def predict(self, state, questions):
|
||||
payload = self.answers[self.calls % len(self.answers)]
|
||||
self.calls += 1
|
||||
return {"answers": payload}
|
||||
|
||||
|
||||
def test_suggest_skips_a_skill_already_named(tmp_path):
|
||||
home = tmp_path / "home"
|
||||
_skill(home / ".claude" / "skills", "commit", "Create a git commit")
|
||||
skills = discover_skills(None, home)
|
||||
agent = _Agent(
|
||||
[
|
||||
{
|
||||
"which": {"choice": "commit", "probabilities": {"commit": 0.9}},
|
||||
"acts_on_user_system": {"noul": 0.9},
|
||||
"would_follow_documented_procedure": {"noul": 0.9},
|
||||
"prose_suffices": {"noul": 0.1},
|
||||
},
|
||||
{
|
||||
"which": {"choice": "commit"},
|
||||
"fits_0": {"noul": 0.8},
|
||||
},
|
||||
]
|
||||
)
|
||||
first = suggest_skill(agent, skills, "commit these changes", inject="suggest", already=set())
|
||||
assert first["name"] == "commit"
|
||||
assert first["mode"] == "suggest"
|
||||
again = suggest_skill(agent, skills, "commit these changes", inject="suggest", already={"commit"})
|
||||
assert again is None
|
||||
44
tests/test_trajectory.py
Normal file
44
tests/test_trajectory.py
Normal file
@@ -0,0 +1,44 @@
|
||||
from laya_router.trajectory import (
|
||||
Step,
|
||||
consecutive_failures,
|
||||
edited_without_verification,
|
||||
repeated_command,
|
||||
response_failed,
|
||||
step_from_event,
|
||||
target_paths,
|
||||
was_inspected,
|
||||
)
|
||||
|
||||
|
||||
def test_patch_and_edit_paths():
|
||||
assert target_paths("Edit", {"file_path": "src/app.py"}) == ("src/app.py",)
|
||||
patch = "*** Update File: src/app.py\n*** Add File: src/new.py\n"
|
||||
assert target_paths("apply_patch", {"command": patch}) == ("src/app.py", "src/new.py")
|
||||
|
||||
|
||||
def test_failure_signals():
|
||||
assert response_failed({"exit_code": 1})
|
||||
assert not response_failed({"exit_code": 0})
|
||||
assert response_failed("command failed: boom")
|
||||
steps = [
|
||||
Step("bash", "Bash", "pytest", failed=True),
|
||||
Step("bash", "Bash", "pytest", failed=True),
|
||||
]
|
||||
assert consecutive_failures(steps) == 2
|
||||
assert repeated_command(steps) == 2
|
||||
steps.append(Step("bash", "Bash", "pytest", failed=False))
|
||||
assert consecutive_failures(steps) == 0
|
||||
|
||||
|
||||
def test_unread_edit_and_verification():
|
||||
read = step_from_event("Read", {"file_path": "src/app.py"}, None, after=True)
|
||||
edit = step_from_event("Edit", {"file_path": "src/app.py"}, None, after=True)
|
||||
other = step_from_event("Edit", {"file_path": "src/other.py"}, None, after=True)
|
||||
verify = step_from_event("Bash", {"command": "pytest -q"}, {"exit_code": 0}, after=True)
|
||||
assert read is not None and edit is not None and other is not None and verify is not None
|
||||
steps = [read, edit]
|
||||
assert was_inspected(steps, "src/app.py")
|
||||
assert not was_inspected(steps, "src/other.py")
|
||||
assert edited_without_verification([edit, other])
|
||||
assert not edited_without_verification([edit, verify])
|
||||
assert step_from_event("Edit", {"file_path": "a.py"}, None, after=False) is None
|
||||
Reference in New Issue
Block a user