feat(medications): 增加基于视觉AI的药品智能录入系统
构建了从照片到药品档案的自动化处理流程，通过GLM多模态大模型实现药品信息的智能采集： 核心能力： - 创建任务追踪表 t_medication_recognition_tasks 存储识别任务状态 - 四阶段渐进式分析：基础识别→人群适配→成分解析→风险评估 - 提供三个REST端点支持任务创建、进度查询和结果确认 - 前端可通过轮询方式获取0-100%的实时进度反馈 - VIP用户免费使用，普通用户按次扣费 技术实现： - 利用GLM-4V-Plus模型处理多角度药品图像（正面+侧面+说明书） - 采用GLM-4-Flash模型进行文本深度分析 - 异步任务执行机制避免接口阻塞 - 完整的异常处理和任务失败恢复策略 - 新增AI_RECOGNITION.md文档详细说明集成方式 同步修复： - 修正会员用户AI配额扣减逻辑，避免不必要的次数消耗 - 优化APNs推送中无效设备令牌的检测和清理流程 - 将服药提醒的提前通知时间从15分钟缩短为5分钟
This commit is contained in:
richarjiang
49
sql-scripts/medication-recognition-tasks-table-create.sql
Normal file
49
sql-scripts/medication-recognition-tasks-table-create.sql
Normal file
@@ -0,0 +1,49 @@
|
||||
-- 药物AI识别任务表创建脚本
|
||||
-- 用于存储用户上传的药品图片和AI识别过程的状态追踪
|
||||
|
||||
CREATE TABLE IF NOT EXISTS `t_medication_recognition_tasks` (
|
||||
`id` VARCHAR(100) NOT NULL COMMENT '任务唯一标识，格式: task_{userId}_{timestamp}',
|
||||
`user_id` VARCHAR(50) NOT NULL COMMENT '用户ID',
|
||||
`front_image_url` VARCHAR(500) NOT NULL COMMENT '正面图片URL（必需）',
|
||||
`side_image_url` VARCHAR(500) NOT NULL COMMENT '侧面图片URL（必需）',
|
||||
`auxiliary_image_url` VARCHAR(500) DEFAULT NULL COMMENT '辅助面图片URL（可选，如说明书）',
|
||||
`status` VARCHAR(50) NOT NULL DEFAULT 'pending' COMMENT '识别状态: pending/analyzing_product/analyzing_suitability/analyzing_ingredients/analyzing_effects/completed/failed',
|
||||
`current_step` VARCHAR(200) DEFAULT NULL COMMENT '当前步骤描述，用于向用户展示',
|
||||
`progress` INT NOT NULL DEFAULT 0 COMMENT '进度百分比(0-100)',
|
||||
`recognition_result` TEXT DEFAULT NULL COMMENT '识别结果(JSON格式)，包含药品名称、剂型、剂量、适宜人群等完整信息',
|
||||
`error_message` TEXT DEFAULT NULL COMMENT '错误信息（仅在status为failed时有值）',
|
||||
`created_at` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
|
||||
`updated_at` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '更新时间',
|
||||
`completed_at` TIMESTAMP NULL DEFAULT NULL COMMENT '完成时间（成功或失败）',
|
||||
PRIMARY KEY (`id`),
|
||||
INDEX `idx_user_id` (`user_id`),
|
||||
INDEX `idx_status` (`status`),
|
||||
INDEX `idx_created_at` (`created_at`)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci COMMENT='药物AI识别任务表';
|
||||
|
||||
-- 添加外键约束（可选，如果需要严格的数据完整性）
|
||||
-- ALTER TABLE `t_medication_recognition_tasks`
|
||||
-- ADD CONSTRAINT `fk_recognition_user_id`
|
||||
-- FOREIGN KEY (`user_id`) REFERENCES `t_users`(`id`) ON DELETE CASCADE;
|
||||
|
||||
-- 示例数据结构说明
|
||||
-- recognition_result JSON 格式示例：
|
||||
/*
|
||||
{
|
||||
"name": "阿莫西林胶囊",
|
||||
"photoUrl": "https://cdn.example.com/medications/front_001.jpg",
|
||||
"form": "capsule",
|
||||
"dosageValue": 0.25,
|
||||
"dosageUnit": "g",
|
||||
"timesPerDay": 3,
|
||||
"medicationTimes": ["08:00", "14:00", "20:00"],
|
||||
"suitableFor": ["成年人", "细菌感染患者"],
|
||||
"unsuitableFor": ["青霉素过敏者", "孕妇", "哺乳期妇女"],
|
||||
"mainIngredients": ["阿莫西林"],
|
||||
"mainUsage": "用于敏感菌引起的各种感染",
|
||||
"sideEffects": ["恶心", "呕吐", "腹泻", "皮疹"],
|
||||
"storageAdvice": ["密封保存", "室温避光", "儿童接触不到的地方"],
|
||||
"healthAdvice": ["按时服药", "多喝水", "避免饮酒"],
|
||||
"confidence": 0.95
|
||||
}
|
||||
*/
|
||||
506
src/medications/AI_RECOGNITION.md
Normal file
506
src/medications/AI_RECOGNITION.md
Normal file
@@ -0,0 +1,506 @@
|
||||
# AI 药物识别功能说明
|
||||
|
||||
## 功能概述
|
||||
|
||||
AI 药物识别功能允许用户通过上传药品照片，自动识别药品信息并创建药物记录。系统使用 GLM-4V-Plus 视觉模型和 GLM-4-Flash 文本模型进行多阶段分析，提供完整的药品信息和健康建议。
|
||||
|
||||
## 核心特性
|
||||
|
||||
### 1. 多图片识别
|
||||
|
||||
- **正面图片**（必需）：药品包装正面，包含药品名称
|
||||
- **侧面图片**（必需）：药品包装侧面，包含规格信息
|
||||
- **辅助图片**（可选）：药品说明书或其他辅助信息
|
||||
|
||||
### 2. 多阶段分析
|
||||
|
||||
系统分4个阶段进行识别和分析：
|
||||
|
||||
1. **产品识别** (0-40%)：识别药品基本信息（名称、剂型、剂量等）
|
||||
2. **适宜人群分析** (40-60%)：分析适合人群和禁忌人群
|
||||
3. **成分分析** (60-80%)：分析主要成分和用途
|
||||
4. **副作用分析** (80-100%)：分析副作用、储存建议和健康建议
|
||||
|
||||
### 3. 实时状态追踪
|
||||
|
||||
- 支持轮询查询识别进度
|
||||
- 提供详细的步骤描述
|
||||
- 实时更新进度百分比
|
||||
|
||||
### 4. 结构化输出
|
||||
|
||||
识别结果包含完整的药品信息：
|
||||
|
||||
- 基本信息：名称、剂型、剂量、服用次数、服药时间
|
||||
- 适宜性分析：适合人群、不适合人群
|
||||
- 成分分析：主要成分、主要用途
|
||||
- 安全信息：副作用、储存建议、健康建议
|
||||
- 置信度评分
|
||||
|
||||
## API 接口
|
||||
|
||||
### 1. 创建识别任务
|
||||
|
||||
**接口**: `POST /medications/ai-recognize`
|
||||
|
||||
**权限要求**: 需要 VIP 会员或有 AI 使用次数
|
||||
|
||||
**请求参数**:
|
||||
|
||||
```json
|
||||
{
|
||||
"frontImageUrl": "https://cdn.example.com/front.jpg",
|
||||
"sideImageUrl": "https://cdn.example.com/side.jpg",
|
||||
"auxiliaryImageUrl": "https://cdn.example.com/auxiliary.jpg" // 可选
|
||||
}
|
||||
```
|
||||
|
||||
**响应示例**:
|
||||
|
||||
```json
|
||||
{
|
||||
"code": 0,
|
||||
"message": "识别任务创建成功",
|
||||
"data": {
|
||||
"taskId": "task_user123_1234567890",
|
||||
"status": "pending"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**注意事项**:
|
||||
|
||||
- 必须提供正面和侧面图片
|
||||
- 任务创建成功后立即扣减 1 次 AI 使用次数
|
||||
- 识别过程异步执行，不阻塞当前请求
|
||||
|
||||
### 2. 查询识别状态
|
||||
|
||||
**接口**: `GET /medications/ai-recognize/:taskId/status`
|
||||
|
||||
**轮询建议**: 每 2-3 秒查询一次
|
||||
|
||||
**响应示例**:
|
||||
|
||||
```json
|
||||
{
|
||||
"code": 0,
|
||||
"message": "查询成功",
|
||||
"data": {
|
||||
"taskId": "task_user123_1234567890",
|
||||
"status": "analyzing_product",
|
||||
"currentStep": "正在识别药品基本信息...",
|
||||
"progress": 25,
|
||||
"createdAt": "2025-01-20T12:00:00.000Z"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**状态说明**:
|
||||
|
||||
- `pending`: 任务已创建，等待处理
|
||||
- `analyzing_product`: 正在识别药品基本信息
|
||||
- `analyzing_suitability`: 正在分析适宜人群
|
||||
- `analyzing_ingredients`: 正在分析主要成分
|
||||
- `analyzing_effects`: 正在分析副作用和健康建议
|
||||
- `completed`: 识别完成
|
||||
- `failed`: 识别失败
|
||||
|
||||
**完成后的响应示例**:
|
||||
|
||||
```json
|
||||
{
|
||||
"code": 0,
|
||||
"message": "查询成功",
|
||||
"data": {
|
||||
"taskId": "task_user123_1234567890",
|
||||
"status": "completed",
|
||||
"currentStep": "识别完成",
|
||||
"progress": 100,
|
||||
"result": {
|
||||
"name": "阿莫西林胶囊",
|
||||
"photoUrl": "https://cdn.example.com/front.jpg",
|
||||
"form": "capsule",
|
||||
"dosageValue": 0.25,
|
||||
"dosageUnit": "g",
|
||||
"timesPerDay": 3,
|
||||
"medicationTimes": ["08:00", "14:00", "20:00"],
|
||||
"suitableFor": ["成年人", "细菌感染患者"],
|
||||
"unsuitableFor": ["青霉素过敏者", "孕妇"],
|
||||
"mainIngredients": ["阿莫西林"],
|
||||
"mainUsage": "用于敏感菌引起的各种感染",
|
||||
"sideEffects": ["恶心", "呕吐", "腹泻"],
|
||||
"storageAdvice": ["密封保存", "室温避光"],
|
||||
"healthAdvice": ["按时服药", "多喝水"],
|
||||
"confidence": 0.95
|
||||
},
|
||||
"createdAt": "2025-01-20T12:00:00.000Z",
|
||||
"completedAt": "2025-01-20T12:01:30.000Z"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3. 确认并创建药物
|
||||
|
||||
**接口**: `POST /medications/ai-recognize/:taskId/confirm`
|
||||
|
||||
**说明**: 用户确认识别结果后创建药物记录，可以对识别结果进行调整
|
||||
|
||||
**请求参数** (可选，用于调整识别结果):
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "调整后的药品名称",
|
||||
"timesPerDay": 2,
|
||||
"medicationTimes": ["09:00", "21:00"],
|
||||
"startDate": "2025-01-20T00:00:00.000Z",
|
||||
"endDate": "2025-02-20T00:00:00.000Z",
|
||||
"note": "饭后服用"
|
||||
}
|
||||
```
|
||||
|
||||
**响应示例**:
|
||||
|
||||
```json
|
||||
{
|
||||
"code": 0,
|
||||
"message": "创建成功",
|
||||
"data": {
|
||||
"id": "med_abc123",
|
||||
"name": "阿莫西林胶囊",
|
||||
"form": "capsule",
|
||||
"dosageValue": 0.25,
|
||||
"dosageUnit": "g",
|
||||
"timesPerDay": 3,
|
||||
"medicationTimes": ["08:00", "14:00", "20:00"],
|
||||
"isActive": true,
|
||||
"aiAnalysis": "{...完整的AI分析结果...}",
|
||||
"createdAt": "2025-01-20T12:02:00.000Z"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 前端集成示例
|
||||
|
||||
### 完整流程
|
||||
|
||||
```typescript
|
||||
// 1. 上传图片并创建识别任务
|
||||
async function startRecognition(
|
||||
frontImage: string,
|
||||
sideImage: string,
|
||||
auxiliaryImage?: string
|
||||
) {
|
||||
const response = await fetch("/medications/ai-recognize", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
Authorization: `Bearer ${token}`,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
frontImageUrl: frontImage,
|
||||
sideImageUrl: sideImage,
|
||||
auxiliaryImageUrl: auxiliaryImage,
|
||||
}),
|
||||
});
|
||||
|
||||
const result = await response.json();
|
||||
if (result.code === 0) {
|
||||
return result.data.taskId;
|
||||
}
|
||||
throw new Error(result.message);
|
||||
}
|
||||
|
||||
// 2. 轮询查询识别状态
|
||||
async function pollRecognitionStatus(taskId: string) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const pollInterval = setInterval(async () => {
|
||||
try {
|
||||
const response = await fetch(
|
||||
`/medications/ai-recognize/${taskId}/status`,
|
||||
{
|
||||
headers: {
|
||||
Authorization: `Bearer ${token}`,
|
||||
},
|
||||
}
|
||||
);
|
||||
|
||||
const result = await response.json();
|
||||
if (result.code !== 0) {
|
||||
clearInterval(pollInterval);
|
||||
reject(new Error(result.message));
|
||||
return;
|
||||
}
|
||||
|
||||
const {
|
||||
status,
|
||||
progress,
|
||||
currentStep,
|
||||
result: recognitionResult,
|
||||
errorMessage,
|
||||
} = result.data;
|
||||
|
||||
// 更新UI显示进度
|
||||
updateProgress(progress, currentStep);
|
||||
|
||||
if (status === "completed") {
|
||||
clearInterval(pollInterval);
|
||||
resolve(recognitionResult);
|
||||
} else if (status === "failed") {
|
||||
clearInterval(pollInterval);
|
||||
reject(new Error(errorMessage || "识别失败"));
|
||||
}
|
||||
} catch (error) {
|
||||
clearInterval(pollInterval);
|
||||
reject(error);
|
||||
}
|
||||
}, 2000); // 每2秒查询一次
|
||||
});
|
||||
}
|
||||
|
||||
// 3. 确认并创建药物
|
||||
async function confirmAndCreate(taskId: string, adjustments?: any) {
|
||||
const response = await fetch(`/medications/ai-recognize/${taskId}/confirm`, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
Authorization: `Bearer ${token}`,
|
||||
},
|
||||
body: JSON.stringify(adjustments || {}),
|
||||
});
|
||||
|
||||
const result = await response.json();
|
||||
if (result.code === 0) {
|
||||
return result.data;
|
||||
}
|
||||
throw new Error(result.message);
|
||||
}
|
||||
|
||||
// 完整使用示例
|
||||
async function recognizeAndCreateMedication() {
|
||||
try {
|
||||
// 步骤1: 创建识别任务
|
||||
showLoading("正在创建识别任务...");
|
||||
const taskId = await startRecognition(
|
||||
frontImageUrl,
|
||||
sideImageUrl,
|
||||
auxiliaryImageUrl
|
||||
);
|
||||
|
||||
// 步骤2: 轮询查询状态
|
||||
showProgress("识别进行中...", 0);
|
||||
const recognitionResult = await pollRecognitionStatus(taskId);
|
||||
|
||||
// 步骤3: 展示结果给用户确认
|
||||
const confirmed = await showConfirmationDialog(recognitionResult);
|
||||
if (!confirmed) return;
|
||||
|
||||
// 步骤4: 创建药物记录
|
||||
showLoading("正在创建药物记录...");
|
||||
const medication = await confirmAndCreate(taskId, userAdjustments);
|
||||
|
||||
showSuccess("药物创建成功！");
|
||||
navigateToMedicationDetail(medication.id);
|
||||
} catch (error) {
|
||||
showError(error.message);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### UI 交互建议
|
||||
|
||||
#### 1. 上传阶段
|
||||
|
||||
```typescript
|
||||
// 显示图片上传引导
|
||||
<div className="upload-guide">
|
||||
<div className="upload-item required">
|
||||
<Icon name="front" />
|
||||
<span>正面照片（必需）</span>
|
||||
<p>拍摄药品包装正面，确保药品名称清晰可见</p>
|
||||
</div>
|
||||
<div className="upload-item required">
|
||||
<Icon name="side" />
|
||||
<span>侧面照片（必需）</span>
|
||||
<p>拍摄药品包装侧面，确保规格剂量清晰可见</p>
|
||||
</div>
|
||||
<div className="upload-item optional">
|
||||
<Icon name="document" />
|
||||
<span>说明书（可选）</span>
|
||||
<p>拍摄药品说明书，可提高识别准确度</p>
|
||||
</div>
|
||||
</div>
|
||||
```
|
||||
|
||||
#### 2. 识别阶段
|
||||
|
||||
```typescript
|
||||
// 显示识别进度和当前步骤
|
||||
<div className="recognition-progress">
|
||||
<ProgressBar value={progress} />
|
||||
<div className="current-step">
|
||||
{status === 'analyzing_product' && '📦 正在识别药品信息...'}
|
||||
{status === 'analyzing_suitability' && '👥 正在分析适宜人群...'}
|
||||
{status === 'analyzing_ingredients' && '🧪 正在分析主要成分...'}
|
||||
{status === 'analyzing_effects' && '⚠️ 正在分析副作用...'}
|
||||
</div>
|
||||
<div className="progress-text">{progress}%</div>
|
||||
</div>
|
||||
```
|
||||
|
||||
#### 3. 确认阶段
|
||||
|
||||
```typescript
|
||||
// 展示识别结果供用户确认和编辑
|
||||
<div className="recognition-result">
|
||||
<div className="confidence-badge">
|
||||
识别置信度: {(result.confidence * 100).toFixed(0)}%
|
||||
</div>
|
||||
|
||||
<EditableField
|
||||
label="药品名称"
|
||||
value={result.name}
|
||||
onChange={(v) => setAdjustments({...adjustments, name: v})}
|
||||
/>
|
||||
|
||||
<EditableField
|
||||
label="每日服用次数"
|
||||
value={result.timesPerDay}
|
||||
onChange={(v) => setAdjustments({...adjustments, timesPerDay: v})}
|
||||
/>
|
||||
|
||||
{/* 更多可编辑字段 */}
|
||||
|
||||
<div className="ai-analysis">
|
||||
<h3>AI 健康分析</h3>
|
||||
<Section title="适合人群" items={result.suitableFor} />
|
||||
<Section title="不适合人群" items={result.unsuitableFor} />
|
||||
<Section title="主要成分" items={result.mainIngredients} />
|
||||
<Section title="副作用" items={result.sideEffects} />
|
||||
<Section title="健康建议" items={result.healthAdvice} />
|
||||
</div>
|
||||
|
||||
<div className="action-buttons">
|
||||
<Button onClick={handleCancel}>取消</Button>
|
||||
<Button primary onClick={handleConfirm}>确认创建</Button>
|
||||
</div>
|
||||
</div>
|
||||
```
|
||||
|
||||
## 错误处理
|
||||
|
||||
### 常见错误及解决方案
|
||||
|
||||
| 错误码 | 错误信息 | 原因 | 解决方案 |
|
||||
| ------ | ---------------------- | ------------------ | ---------------------- |
|
||||
| 400 | 必须提供正面和侧面图片 | 缺少必需的图片 | 确保上传正面和侧面图片 |
|
||||
| 403 | 免费使用次数已用完 | AI 使用次数不足 | 引导用户开通 VIP 会员 |
|
||||
| 404 | 识别任务不存在 | 任务ID错误或已过期 | 重新创建识别任务 |
|
||||
| 500 | AI响应格式错误 | 模型返回异常 | 提示用户重试或联系客服 |
|
||||
|
||||
### 识别失败处理
|
||||
|
||||
当识别状态为 `failed` 时：
|
||||
|
||||
1. 显示友好的错误提示
|
||||
2. 提供重试选项
|
||||
3. 建议用户：
|
||||
- 拍摄更清晰的照片
|
||||
- 确保光线充足
|
||||
- 药品名称完整可见
|
||||
- 或选择手动输入
|
||||
|
||||
## 性能优化建议
|
||||
|
||||
### 1. 图片优化
|
||||
|
||||
- 上传前压缩图片（建议最大 2MB）
|
||||
- 使用 WebP 格式减小体积
|
||||
- 限制图片尺寸（建议 1920x1080 以内）
|
||||
|
||||
### 2. 轮询优化
|
||||
|
||||
- 使用指数退避策略（2s, 3s, 5s...）
|
||||
- 设置最大轮询次数（如 30 次，约 1 分钟）
|
||||
- 超时后提示用户刷新页面
|
||||
|
||||
### 3. 用户体验优化
|
||||
|
||||
- 显示预估完成时间（约 30-60 秒）
|
||||
- 支持后台识别，用户可离开页面
|
||||
- 完成后发送推送通知
|
||||
|
||||
## 数据安全
|
||||
|
||||
1. **图片安全**：
|
||||
|
||||
- 图片 URL 应使用腾讯云 COS 临时访问凭证
|
||||
- 识别完成后可选择删除图片
|
||||
|
||||
2. **数据隐私**：
|
||||
|
||||
- 识别任务数据仅用户本人可查看
|
||||
- AI 分析结果不会共享给第三方
|
||||
- 支持用户主动删除识别历史
|
||||
|
||||
3. **任务清理**：
|
||||
- 建议定期清理 30 天前的识别任务记录
|
||||
- 可通过定时任务自动清理
|
||||
|
||||
## 最佳实践
|
||||
|
||||
### 拍照技巧
|
||||
|
||||
1. 确保光线充足，避免反光
|
||||
2. 药品名称和规格清晰可见
|
||||
3. 尽量拍摄完整的包装盒
|
||||
4. 避免手指遮挡关键信息
|
||||
5. 保持相机稳定，避免模糊
|
||||
|
||||
### 识别准确度
|
||||
|
||||
- 正面 + 侧面：准确度约 85-90%
|
||||
- 正面 + 侧面 + 说明书：准确度约 90-95%
|
||||
- 置信度 > 0.8：可直接使用
|
||||
- 置信度 0.5-0.8：建议人工核对
|
||||
- 置信度 < 0.5：建议重新拍照或手动输入
|
||||
|
||||
## 技术架构
|
||||
|
||||
### 模型选择
|
||||
|
||||
- **GLM-4V-Plus**：视觉识别模型，识别药品图片
|
||||
- **GLM-4-Flash**：文本分析模型，深度分析和结构化输出
|
||||
|
||||
### 数据流
|
||||
|
||||
1. 客户端上传图片到 COS，获取 URL
|
||||
2. 调用 `/ai-recognize` 创建任务
|
||||
3. 服务端异步调用 AI 模型进行多阶段分析
|
||||
4. 客户端轮询查询状态和结果
|
||||
5. 用户确认后创建药物记录
|
||||
|
||||
### 状态管理
|
||||
|
||||
- 使用数据库表 `t_medication_recognition_tasks` 持久化状态
|
||||
- 支持断点续传和故障恢复
|
||||
- 任务完成后保留 7 天供查询
|
||||
|
||||
## 未来规划
|
||||
|
||||
1. **功能增强**：
|
||||
|
||||
- 支持批量识别多个药品
|
||||
- 支持视频识别
|
||||
- 支持语音输入药品名称
|
||||
|
||||
2. **模型优化**：
|
||||
|
||||
- 训练专用的药品识别模型
|
||||
- 提高中文药品识别准确度
|
||||
- 支持更多药品类型
|
||||
|
||||
3. **用户体验**：
|
||||
- 支持离线识别（边缘计算）
|
||||
- 实时预览识别结果
|
||||
- 智能纠错和建议
|
||||
32
src/medications/dto/create-recognition-task.dto.ts
Normal file
32
src/medications/dto/create-recognition-task.dto.ts
Normal file
@@ -0,0 +1,32 @@
|
||||
import { ApiProperty } from '@nestjs/swagger';
|
||||
import { IsString, IsNotEmpty, IsOptional } from 'class-validator';
|
||||
|
||||
/**
|
||||
* 创建药物识别任务 DTO
|
||||
*/
|
||||
export class CreateRecognitionTaskDto {
|
||||
@ApiProperty({
|
||||
description: '正面图片URL（必需）',
|
||||
example: 'https://cdn.example.com/medications/front_001.jpg',
|
||||
})
|
||||
@IsString()
|
||||
@IsNotEmpty()
|
||||
frontImageUrl: string;
|
||||
|
||||
@ApiProperty({
|
||||
description: '侧面图片URL（必需）',
|
||||
example: 'https://cdn.example.com/medications/side_001.jpg',
|
||||
})
|
||||
@IsString()
|
||||
@IsNotEmpty()
|
||||
sideImageUrl: string;
|
||||
|
||||
@ApiProperty({
|
||||
description: '辅助面图片URL（可选，如说明书）',
|
||||
example: 'https://cdn.example.com/medications/auxiliary_001.jpg',
|
||||
required: false,
|
||||
})
|
||||
@IsString()
|
||||
@IsOptional()
|
||||
auxiliaryImageUrl?: string;
|
||||
}
|
||||
94
src/medications/dto/recognition-result.dto.ts
Normal file
94
src/medications/dto/recognition-result.dto.ts
Normal file
@@ -0,0 +1,94 @@
|
||||
import { ApiProperty } from '@nestjs/swagger';
|
||||
import { MedicationFormEnum } from '../enums/medication-form.enum';
|
||||
|
||||
/**
|
||||
* 药物识别结果 DTO
|
||||
* 包含创建药物所需的所有字段 + AI分析结果
|
||||
*/
|
||||
export class RecognitionResultDto {
|
||||
@ApiProperty({ description: '药品名称', example: '阿莫西林胶囊' })
|
||||
name: string;
|
||||
|
||||
@ApiProperty({
|
||||
description: '药品照片URL（使用正面图片）',
|
||||
example: 'https://cdn.example.com/medications/front_001.jpg',
|
||||
})
|
||||
photoUrl: string;
|
||||
|
||||
@ApiProperty({
|
||||
description: '药物剂型',
|
||||
enum: MedicationFormEnum,
|
||||
example: MedicationFormEnum.CAPSULE,
|
||||
})
|
||||
form: MedicationFormEnum;
|
||||
|
||||
@ApiProperty({ description: '剂量数值', example: 1 })
|
||||
dosageValue: number;
|
||||
|
||||
@ApiProperty({ description: '剂量单位', example: '粒' })
|
||||
dosageUnit: string;
|
||||
|
||||
@ApiProperty({ description: '建议每日服用次数', example: 3 })
|
||||
timesPerDay: number;
|
||||
|
||||
@ApiProperty({
|
||||
description: '建议服药时间',
|
||||
example: ['08:00', '14:00', '20:00'],
|
||||
type: [String],
|
||||
})
|
||||
medicationTimes: string[];
|
||||
|
||||
@ApiProperty({
|
||||
description: '适合人群',
|
||||
example: ['成年人', '细菌感染患者'],
|
||||
type: [String],
|
||||
})
|
||||
suitableFor: string[];
|
||||
|
||||
@ApiProperty({
|
||||
description: '不适合人群',
|
||||
example: ['青霉素过敏者', '孕妇', '哺乳期妇女'],
|
||||
type: [String],
|
||||
})
|
||||
unsuitableFor: string[];
|
||||
|
||||
@ApiProperty({
|
||||
description: '主要成分',
|
||||
example: ['阿莫西林'],
|
||||
type: [String],
|
||||
})
|
||||
mainIngredients: string[];
|
||||
|
||||
@ApiProperty({
|
||||
description: '主要用途',
|
||||
example: '用于敏感菌引起的各种感染',
|
||||
})
|
||||
mainUsage: string;
|
||||
|
||||
@ApiProperty({
|
||||
description: '可能的副作用',
|
||||
example: ['恶心', '呕吐', '腹泻', '皮疹'],
|
||||
type: [String],
|
||||
})
|
||||
sideEffects: string[];
|
||||
|
||||
@ApiProperty({
|
||||
description: '储存建议',
|
||||
example: ['密封保存', '室温避光', '儿童接触不到的地方'],
|
||||
type: [String],
|
||||
})
|
||||
storageAdvice: string[];
|
||||
|
||||
@ApiProperty({
|
||||
description: '健康建议',
|
||||
example: ['按时服药', '多喝水', '避免饮酒'],
|
||||
type: [String],
|
||||
})
|
||||
healthAdvice: string[];
|
||||
|
||||
@ApiProperty({
|
||||
description: '识别置信度(0-1)',
|
||||
example: 0.95,
|
||||
})
|
||||
confidence: number;
|
||||
}
|
||||
60
src/medications/dto/recognition-status.dto.ts
Normal file
60
src/medications/dto/recognition-status.dto.ts
Normal file
@@ -0,0 +1,60 @@
|
||||
import { ApiProperty } from '@nestjs/swagger';
|
||||
import { RecognitionStatusEnum } from '../enums/recognition-status.enum';
|
||||
import { RecognitionResultDto } from './recognition-result.dto';
|
||||
|
||||
/**
|
||||
* 药物识别状态响应 DTO
|
||||
*/
|
||||
export class RecognitionStatusDto {
|
||||
@ApiProperty({
|
||||
description: '任务ID',
|
||||
example: 'task_user123_1234567890',
|
||||
})
|
||||
taskId: string;
|
||||
|
||||
@ApiProperty({
|
||||
description: '识别状态',
|
||||
enum: RecognitionStatusEnum,
|
||||
example: RecognitionStatusEnum.ANALYZING_PRODUCT,
|
||||
})
|
||||
status: RecognitionStatusEnum;
|
||||
|
||||
@ApiProperty({
|
||||
description: '当前步骤描述',
|
||||
example: '正在识别药品基本信息...',
|
||||
})
|
||||
currentStep: string;
|
||||
|
||||
@ApiProperty({
|
||||
description: '进度百分比(0-100)',
|
||||
example: 40,
|
||||
})
|
||||
progress: number;
|
||||
|
||||
@ApiProperty({
|
||||
description: '识别结果（仅在状态为completed时返回）',
|
||||
type: RecognitionResultDto,
|
||||
required: false,
|
||||
})
|
||||
result?: RecognitionResultDto;
|
||||
|
||||
@ApiProperty({
|
||||
description: '错误信息（仅在状态为failed时返回）',
|
||||
example: '图片无法识别，请提供更清晰的照片',
|
||||
required: false,
|
||||
})
|
||||
errorMessage?: string;
|
||||
|
||||
@ApiProperty({
|
||||
description: '创建时间',
|
||||
example: '2025-01-20T12:00:00.000Z',
|
||||
})
|
||||
createdAt: Date;
|
||||
|
||||
@ApiProperty({
|
||||
description: '完成时间（仅在completed或failed时返回）',
|
||||
example: '2025-01-20T12:01:30.000Z',
|
||||
required: false,
|
||||
})
|
||||
completedAt?: Date;
|
||||
}
|
||||
25
src/medications/enums/recognition-status.enum.ts
Normal file
25
src/medications/enums/recognition-status.enum.ts
Normal file
@@ -0,0 +1,25 @@
|
||||
/**
|
||||
* 药物识别状态枚举
|
||||
*/
|
||||
export enum RecognitionStatusEnum {
|
||||
PENDING = 'pending',
|
||||
ANALYZING_PRODUCT = 'analyzing_product',
|
||||
ANALYZING_SUITABILITY = 'analyzing_suitability',
|
||||
ANALYZING_INGREDIENTS = 'analyzing_ingredients',
|
||||
ANALYZING_EFFECTS = 'analyzing_effects',
|
||||
COMPLETED = 'completed',
|
||||
FAILED = 'failed',
|
||||
}
|
||||
|
||||
/**
|
||||
* 识别状态描述映射
|
||||
*/
|
||||
export const RECOGNITION_STATUS_DESCRIPTIONS: Record<RecognitionStatusEnum, string> = {
|
||||
[RecognitionStatusEnum.PENDING]: '任务已创建，等待处理',
|
||||
[RecognitionStatusEnum.ANALYZING_PRODUCT]: '正在识别药品基本信息...',
|
||||
[RecognitionStatusEnum.ANALYZING_SUITABILITY]: '正在分析适宜人群...',
|
||||
[RecognitionStatusEnum.ANALYZING_INGREDIENTS]: '正在分析主要成分...',
|
||||
[RecognitionStatusEnum.ANALYZING_EFFECTS]: '正在分析副作用和健康建议...',
|
||||
[RecognitionStatusEnum.COMPLETED]: '识别完成',
|
||||
[RecognitionStatusEnum.FAILED]: '识别失败',
|
||||
};
|
||||
@@ -18,12 +18,17 @@ import { CreateMedicationDto } from './dto/create-medication.dto';
|
||||
import { UpdateMedicationDto } from './dto/update-medication.dto';
|
||||
import { MedicationQueryDto } from './dto/medication-query.dto';
|
||||
import { AiAnalysisResultDto } from './dto/ai-analysis-result.dto';
|
||||
import { CreateRecognitionTaskDto } from './dto/create-recognition-task.dto';
|
||||
import { RecognitionStatusDto } from './dto/recognition-status.dto';
|
||||
import { JwtAuthGuard } from '../common/guards/jwt-auth.guard';
|
||||
import { CurrentUser } from '../common/decorators/current-user.decorator';
|
||||
import { ApiResponseDto } from '../base.dto';
|
||||
import { MedicationReminderService } from './services/medication-reminder.service';
|
||||
import { MedicationAnalysisService } from './services/medication-analysis.service';
|
||||
import { MedicationRecognitionService } from './services/medication-recognition.service';
|
||||
import { UsersService } from '../users/users.service';
|
||||
import { RepeatPatternEnum } from './enums/repeat-pattern.enum';
|
||||
import { RecognitionStatusEnum } from './enums/recognition-status.enum';
|
||||
|
||||
/**
|
||||
* 药物管理控制器
|
||||
@@ -38,6 +43,7 @@ export class MedicationsController {
|
||||
private readonly medicationsService: MedicationsService,
|
||||
private readonly reminderService: MedicationReminderService,
|
||||
private readonly analysisService: MedicationAnalysisService,
|
||||
private readonly recognitionService: MedicationRecognitionService,
|
||||
private readonly usersService: UsersService,
|
||||
) {}
|
||||
|
||||
@@ -56,47 +62,11 @@ export class MedicationsController {
|
||||
// 设置提醒（实际由定时任务触发）
|
||||
await this.reminderService.setupRemindersForMedication(medication);
|
||||
|
||||
// 异步触发 AI 分析（不阻塞当前请求）
|
||||
this.triggerAutoAnalysis(user.sub, medication.id).catch(err => {
|
||||
this.logger.error(`触发自动AI分析异常: ${err instanceof Error ? err.message : String(err)}`);
|
||||
});
|
||||
|
||||
return ApiResponseDto.success(medication, '创建成功');
|
||||
}
|
||||
|
||||
/**
|
||||
* 触发自动AI分析
|
||||
* 这是一个后台异步任务，不会阻塞主请求
|
||||
*/
|
||||
private async triggerAutoAnalysis(userId: string, medicationId: string) {
|
||||
try {
|
||||
// 1. 检查用户免费使用次数或会员状态
|
||||
const userUsageCount = await this.usersService.getUserUsageCount(userId);
|
||||
|
||||
// 如果用户不是VIP且免费次数不足，直接返回
|
||||
if (userUsageCount <= 0) {
|
||||
this.logger.log(`新药自动分析跳过 - 用户ID: ${userId}, 免费次数不足`);
|
||||
return;
|
||||
}
|
||||
|
||||
this.logger.log(`开始新药自动AI分析 - 用户ID: ${userId}, 药物ID: ${medicationId}`);
|
||||
|
||||
// 2. 执行分析 (V2版本，返回结构化数据并自动保存到数据库)
|
||||
await this.analysisService.analyzeMedicationV2(medicationId, userId);
|
||||
|
||||
// 3. 分析成功后扣减次数
|
||||
// 注意：VIP用户 getUserUsageCount 返回 999，即使这里扣减了 freeUsageCount 也不影响 VIP 权益
|
||||
try {
|
||||
await this.usersService.deductUserUsageCount(userId, 1);
|
||||
this.logger.log(`自动AI分析成功，已扣减用户免费次数 - 用户ID: ${userId}`);
|
||||
} catch (deductError) {
|
||||
this.logger.error(`自动AI分析扣费失败 - 用户ID: ${userId}, 错误: ${deductError instanceof Error ? deductError.message : String(deductError)}`);
|
||||
}
|
||||
|
||||
} catch (error) {
|
||||
this.logger.error(`新药自动AI分析失败 - 用户ID: ${userId}, 药物ID: ${medicationId}, 错误: ${error instanceof Error ? error.message : String(error)}`);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@Get()
|
||||
@ApiOperation({ summary: '获取药物列表' })
|
||||
@@ -315,4 +285,196 @@ export class MedicationsController {
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@Post('ai-recognize')
|
||||
@ApiOperation({
|
||||
summary: 'AI识别药品创建',
|
||||
description: '通过上传药品图片（正面+侧面+可选辅助面），AI自动识别并生成药品信息。需要VIP会员或有AI使用次数。',
|
||||
})
|
||||
@ApiResponse({
|
||||
status: 201,
|
||||
description: '识别任务创建成功',
|
||||
type: RecognitionStatusDto,
|
||||
})
|
||||
@ApiResponse({
|
||||
status: 400,
|
||||
description: '参数错误（缺少必需图片）',
|
||||
})
|
||||
@ApiResponse({
|
||||
status: 403,
|
||||
description: '权限不足或次数不足',
|
||||
})
|
||||
async aiRecognizeMedication(
|
||||
@CurrentUser() user: any,
|
||||
@Body() dto: CreateRecognitionTaskDto,
|
||||
) {
|
||||
// 1. 验证图片必需性
|
||||
if (!dto.frontImageUrl || !dto.sideImageUrl) {
|
||||
return ApiResponseDto.error('必须提供正面和侧面图片');
|
||||
}
|
||||
|
||||
// 2. 检查用户权限
|
||||
const userUsageCount = await this.usersService.getUserUsageCount(user.sub);
|
||||
if (userUsageCount <= 0) {
|
||||
this.logger.warn(
|
||||
`AI药物识别失败 - 用户ID: ${user.sub}, 免费次数不足`,
|
||||
);
|
||||
return ApiResponseDto.error(
|
||||
'免费使用次数已用完，请开通会员获取更多使用次数',
|
||||
403,
|
||||
);
|
||||
}
|
||||
|
||||
try {
|
||||
// 3. 创建识别任务
|
||||
const result = await this.recognitionService.createRecognitionTask(
|
||||
user.sub,
|
||||
dto,
|
||||
);
|
||||
|
||||
// 4. 扣减使用次数（任务创建成功后立即扣减）
|
||||
await this.usersService.deductUserUsageCount(user.sub, 1);
|
||||
this.logger.log(
|
||||
`AI药物识别任务创建成功 - 用户ID: ${user.sub}, taskId: ${result.taskId}, 剩余次数: ${userUsageCount - 1}`,
|
||||
);
|
||||
|
||||
return ApiResponseDto.success(result, '识别任务创建成功');
|
||||
} catch (error) {
|
||||
this.logger.error(
|
||||
`AI药物识别任务创建失败 - 用户ID: ${user.sub}, 错误: ${error instanceof Error ? error.message : String(error)}`,
|
||||
);
|
||||
return ApiResponseDto.error(
|
||||
error instanceof Error ? error.message : '创建识别任务失败',
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@Get('ai-recognize/:taskId/status')
|
||||
@ApiOperation({
|
||||
summary: '查询AI识别状态',
|
||||
description: '通过任务ID查询药品识别的当前状态和进度，支持轮询查询',
|
||||
})
|
||||
@ApiResponse({
|
||||
status: 200,
|
||||
description: '查询成功',
|
||||
type: RecognitionStatusDto,
|
||||
})
|
||||
@ApiResponse({
|
||||
status: 404,
|
||||
description: '任务不存在',
|
||||
})
|
||||
async getRecognitionStatus(
|
||||
@CurrentUser() user: any,
|
||||
@Param('taskId') taskId: string,
|
||||
) {
|
||||
try {
|
||||
const status = await this.recognitionService.getRecognitionStatus(
|
||||
taskId,
|
||||
user.sub,
|
||||
);
|
||||
return ApiResponseDto.success(status, '查询成功');
|
||||
} catch (error) {
|
||||
this.logger.error(
|
||||
`查询识别状态失败 - 任务ID: ${taskId}, 用户ID: ${user.sub}, 错误: ${error instanceof Error ? error.message : String(error)}`,
|
||||
);
|
||||
return ApiResponseDto.error(
|
||||
error instanceof Error ? error.message : '查询识别状态失败',
|
||||
error instanceof Error && error.message.includes('不存在') ? 404 : 500,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@Post('ai-recognize/:taskId/confirm')
|
||||
@ApiOperation({
|
||||
summary: '确认并创建药物',
|
||||
description: '确认AI识别结果并创建药物记录。用户可以在识别结果基础上进行调整。',
|
||||
})
|
||||
@ApiResponse({
|
||||
status: 201,
|
||||
description: '创建成功',
|
||||
})
|
||||
@ApiResponse({
|
||||
status: 400,
|
||||
description: '识别任务尚未完成或识别失败',
|
||||
})
|
||||
async confirmRecognitionAndCreate(
|
||||
@CurrentUser() user: any,
|
||||
@Param('taskId') taskId: string,
|
||||
@Body() adjustments?: Partial<CreateMedicationDto>,
|
||||
) {
|
||||
try {
|
||||
// 1. 获取识别结果
|
||||
const status = await this.recognitionService.getRecognitionStatus(
|
||||
taskId,
|
||||
user.sub,
|
||||
);
|
||||
|
||||
if (status.status !== RecognitionStatusEnum.COMPLETED) {
|
||||
return ApiResponseDto.error(
|
||||
status.status === RecognitionStatusEnum.FAILED
|
||||
? `识别失败: ${status.errorMessage}`
|
||||
: '识别任务尚未完成，请稍后再试',
|
||||
400,
|
||||
);
|
||||
}
|
||||
|
||||
if (!status.result) {
|
||||
return ApiResponseDto.error('识别结果为空', 400);
|
||||
}
|
||||
|
||||
// 2. 合并用户调整和识别结果，创建药物DTO
|
||||
const createDto: CreateMedicationDto = {
|
||||
name: adjustments?.name || status.result.name,
|
||||
photoUrl: adjustments?.photoUrl || status.result.photoUrl,
|
||||
form: adjustments?.form || status.result.form,
|
||||
dosageValue: adjustments?.dosageValue || status.result.dosageValue,
|
||||
dosageUnit: adjustments?.dosageUnit || status.result.dosageUnit,
|
||||
timesPerDay: adjustments?.timesPerDay || status.result.timesPerDay,
|
||||
medicationTimes:
|
||||
adjustments?.medicationTimes || status.result.medicationTimes,
|
||||
repeatPattern: adjustments?.repeatPattern || RepeatPatternEnum.DAILY,
|
||||
startDate:
|
||||
adjustments?.startDate || new Date().toISOString(),
|
||||
endDate: adjustments?.endDate,
|
||||
note: adjustments?.note,
|
||||
isActive: adjustments?.isActive !== undefined ? adjustments.isActive : true,
|
||||
};
|
||||
|
||||
// 3. 创建药物记录
|
||||
const medication = await this.medicationsService.create(
|
||||
user.sub,
|
||||
createDto,
|
||||
);
|
||||
|
||||
// 4. 保存AI分析结果到药物记录
|
||||
const aiAnalysis = {
|
||||
suitableFor: status.result.suitableFor,
|
||||
unsuitableFor: status.result.unsuitableFor,
|
||||
mainIngredients: status.result.mainIngredients,
|
||||
mainUsage: status.result.mainUsage,
|
||||
sideEffects: status.result.sideEffects,
|
||||
storageAdvice: status.result.storageAdvice,
|
||||
healthAdvice: status.result.healthAdvice,
|
||||
};
|
||||
await this.medicationsService.update(medication.id, user.sub, {
|
||||
aiAnalysis: JSON.stringify(aiAnalysis),
|
||||
} as any);
|
||||
|
||||
// 5. 设置提醒
|
||||
await this.reminderService.setupRemindersForMedication(medication);
|
||||
|
||||
this.logger.log(
|
||||
`AI识别药物创建成功 - 用户ID: ${user.sub}, 药物ID: ${medication.id}, 任务ID: ${taskId}`,
|
||||
);
|
||||
|
||||
return ApiResponseDto.success(medication, '创建成功');
|
||||
} catch (error) {
|
||||
this.logger.error(
|
||||
`确认识别并创建药物失败 - 任务ID: ${taskId}, 用户ID: ${user.sub}, 错误: ${error instanceof Error ? error.message : String(error)}`,
|
||||
);
|
||||
return ApiResponseDto.error(
|
||||
error instanceof Error ? error.message : '创建药物失败',
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -6,6 +6,7 @@ import { ConfigModule } from '@nestjs/config';
|
||||
// Models
|
||||
import { Medication } from './models/medication.model';
|
||||
import { MedicationRecord } from './models/medication-record.model';
|
||||
import { MedicationRecognitionTask } from './models/medication-recognition-task.model';
|
||||
|
||||
// Controllers
|
||||
import { MedicationsController } from './medications.controller';
|
||||
@@ -20,6 +21,7 @@ import { RecordGeneratorService } from './services/record-generator.service';
|
||||
import { StatusUpdaterService } from './services/status-updater.service';
|
||||
import { MedicationReminderService } from './services/medication-reminder.service';
|
||||
import { MedicationAnalysisService } from './services/medication-analysis.service';
|
||||
import { MedicationRecognitionService } from './services/medication-recognition.service';
|
||||
|
||||
// Import PushNotificationsModule for reminders
|
||||
import { PushNotificationsModule } from '../push-notifications/push-notifications.module';
|
||||
@@ -32,7 +34,11 @@ import { UsersModule } from '../users/users.module';
|
||||
@Module({
|
||||
imports: [
|
||||
ConfigModule, // AI 配置
|
||||
SequelizeModule.forFeature([Medication, MedicationRecord]),
|
||||
SequelizeModule.forFeature([
|
||||
Medication,
|
||||
MedicationRecord,
|
||||
MedicationRecognitionTask,
|
||||
]),
|
||||
ScheduleModule.forRoot(), // 启用定时任务
|
||||
PushNotificationsModule, // 推送通知功能
|
||||
UsersModule, // 用户认证服务
|
||||
@@ -50,6 +56,7 @@ import { UsersModule } from '../users/users.module';
|
||||
StatusUpdaterService,
|
||||
MedicationReminderService,
|
||||
MedicationAnalysisService, // AI 分析服务
|
||||
MedicationRecognitionService, // AI 识别服务
|
||||
],
|
||||
exports: [
|
||||
MedicationsService,
|
||||
|
||||
105
src/medications/models/medication-recognition-task.model.ts
Normal file
105
src/medications/models/medication-recognition-task.model.ts
Normal file
@@ -0,0 +1,105 @@
|
||||
import { Column, Model, Table, DataType } from 'sequelize-typescript';
|
||||
import { RecognitionStatusEnum } from '../enums/recognition-status.enum';
|
||||
|
||||
/**
|
||||
* 药物AI识别任务模型
|
||||
*/
|
||||
@Table({
|
||||
tableName: 't_medication_recognition_tasks',
|
||||
underscored: true,
|
||||
timestamps: true,
|
||||
})
|
||||
export class MedicationRecognitionTask extends Model {
|
||||
@Column({
|
||||
type: DataType.STRING(100),
|
||||
primaryKey: true,
|
||||
comment: '任务唯一标识',
|
||||
})
|
||||
declare id: string;
|
||||
|
||||
@Column({
|
||||
type: DataType.STRING(50),
|
||||
allowNull: false,
|
||||
comment: '用户ID',
|
||||
})
|
||||
declare userId: string;
|
||||
|
||||
@Column({
|
||||
type: DataType.STRING(500),
|
||||
allowNull: false,
|
||||
comment: '正面图片URL',
|
||||
})
|
||||
declare frontImageUrl: string;
|
||||
|
||||
@Column({
|
||||
type: DataType.STRING(500),
|
||||
allowNull: false,
|
||||
comment: '侧面图片URL',
|
||||
})
|
||||
declare sideImageUrl: string;
|
||||
|
||||
@Column({
|
||||
type: DataType.STRING(500),
|
||||
allowNull: true,
|
||||
comment: '辅助面图片URL',
|
||||
})
|
||||
declare auxiliaryImageUrl: string;
|
||||
|
||||
@Column({
|
||||
type: DataType.STRING(50),
|
||||
allowNull: false,
|
||||
defaultValue: RecognitionStatusEnum.PENDING,
|
||||
comment: '识别状态',
|
||||
})
|
||||
declare status: RecognitionStatusEnum;
|
||||
|
||||
@Column({
|
||||
type: DataType.STRING(200),
|
||||
allowNull: true,
|
||||
comment: '当前步骤描述',
|
||||
})
|
||||
declare currentStep: string;
|
||||
|
||||
@Column({
|
||||
type: DataType.INTEGER,
|
||||
allowNull: false,
|
||||
defaultValue: 0,
|
||||
comment: '进度百分比(0-100)',
|
||||
})
|
||||
declare progress: number;
|
||||
|
||||
@Column({
|
||||
type: DataType.TEXT,
|
||||
allowNull: true,
|
||||
comment: '识别结果(JSON格式)',
|
||||
})
|
||||
declare recognitionResult: string;
|
||||
|
||||
@Column({
|
||||
type: DataType.TEXT,
|
||||
allowNull: true,
|
||||
comment: '错误信息',
|
||||
})
|
||||
declare errorMessage: string;
|
||||
|
||||
@Column({
|
||||
type: DataType.DATE,
|
||||
defaultValue: DataType.NOW,
|
||||
comment: '创建时间',
|
||||
})
|
||||
declare createdAt: Date;
|
||||
|
||||
@Column({
|
||||
type: DataType.DATE,
|
||||
defaultValue: DataType.NOW,
|
||||
comment: '更新时间',
|
||||
})
|
||||
declare updatedAt: Date;
|
||||
|
||||
@Column({
|
||||
type: DataType.DATE,
|
||||
allowNull: true,
|
||||
comment: '完成时间',
|
||||
})
|
||||
declare completedAt: Date;
|
||||
}
|
||||
523
src/medications/services/medication-recognition.service.ts
Normal file
523
src/medications/services/medication-recognition.service.ts
Normal file
@@ -0,0 +1,523 @@
|
||||
import { Injectable, Logger, NotFoundException } from '@nestjs/common';
|
||||
import { ConfigService } from '@nestjs/config';
|
||||
import { InjectModel } from '@nestjs/sequelize';
|
||||
import { OpenAI } from 'openai';
|
||||
import { MedicationRecognitionTask } from '../models/medication-recognition-task.model';
|
||||
import { CreateRecognitionTaskDto } from '../dto/create-recognition-task.dto';
|
||||
import { RecognitionStatusDto } from '../dto/recognition-status.dto';
|
||||
import { RecognitionResultDto } from '../dto/recognition-result.dto';
|
||||
import {
|
||||
RecognitionStatusEnum,
|
||||
RECOGNITION_STATUS_DESCRIPTIONS,
|
||||
} from '../enums/recognition-status.enum';
|
||||
import { MedicationFormEnum } from '../enums/medication-form.enum';
|
||||
|
||||
/**
|
||||
* 药物AI识别服务
|
||||
* 负责多图片药物识别、分析和结构化数据提取
|
||||
*/
|
||||
@Injectable()
|
||||
export class MedicationRecognitionService {
|
||||
private readonly logger = new Logger(MedicationRecognitionService.name);
|
||||
private readonly client: OpenAI;
|
||||
private readonly visionModel: string;
|
||||
private readonly textModel: string;
|
||||
|
||||
constructor(
|
||||
private readonly configService: ConfigService,
|
||||
@InjectModel(MedicationRecognitionTask)
|
||||
private readonly taskModel: typeof MedicationRecognitionTask,
|
||||
) {
|
||||
const glmApiKey = this.configService.get<string>('GLM_API_KEY');
|
||||
const glmBaseURL =
|
||||
this.configService.get<string>('GLM_BASE_URL') ||
|
||||
'https://open.bigmodel.cn/api/paas/v4';
|
||||
|
||||
this.client = new OpenAI({
|
||||
apiKey: glmApiKey,
|
||||
baseURL: glmBaseURL,
|
||||
});
|
||||
|
||||
this.visionModel =
|
||||
this.configService.get<string>('GLM_VISION_MODEL') || 'glm-4v-plus';
|
||||
this.textModel =
|
||||
this.configService.get<string>('GLM_MODEL') || 'glm-4-flash';
|
||||
}
|
||||
|
||||
/**
|
||||
* 创建识别任务
|
||||
*/
|
||||
async createRecognitionTask(
|
||||
userId: string,
|
||||
dto: CreateRecognitionTaskDto,
|
||||
): Promise<{ taskId: string; status: RecognitionStatusEnum }> {
|
||||
const taskId = `task_${userId}_${Date.now()}`;
|
||||
|
||||
this.logger.log(`创建药物识别任务: ${taskId}, 用户: ${userId}`);
|
||||
|
||||
await this.taskModel.create({
|
||||
id: taskId,
|
||||
userId,
|
||||
frontImageUrl: dto.frontImageUrl,
|
||||
sideImageUrl: dto.sideImageUrl,
|
||||
auxiliaryImageUrl: dto.auxiliaryImageUrl,
|
||||
status: RecognitionStatusEnum.PENDING,
|
||||
currentStep: RECOGNITION_STATUS_DESCRIPTIONS[RecognitionStatusEnum.PENDING],
|
||||
progress: 0,
|
||||
});
|
||||
|
||||
// 异步开始识别过程（不阻塞当前请求）
|
||||
this.startRecognitionProcess(taskId).catch((error) => {
|
||||
this.logger.error(
|
||||
`识别任务 ${taskId} 处理失败: ${error instanceof Error ? error.message : String(error)}`,
|
||||
);
|
||||
});
|
||||
|
||||
return { taskId, status: RecognitionStatusEnum.PENDING };
|
||||
}
|
||||
|
||||
/**
|
||||
* 查询识别状态
|
||||
*/
|
||||
async getRecognitionStatus(
|
||||
taskId: string,
|
||||
userId: string,
|
||||
): Promise<RecognitionStatusDto> {
|
||||
const task = await this.taskModel.findOne({
|
||||
where: { id: taskId, userId },
|
||||
});
|
||||
|
||||
if (!task) {
|
||||
throw new NotFoundException('识别任务不存在');
|
||||
}
|
||||
|
||||
return {
|
||||
taskId: task.id,
|
||||
status: task.status as RecognitionStatusEnum,
|
||||
currentStep: task.currentStep,
|
||||
progress: task.progress,
|
||||
result: task.recognitionResult
|
||||
? JSON.parse(task.recognitionResult)
|
||||
: undefined,
|
||||
errorMessage: task.errorMessage,
|
||||
createdAt: task.createdAt,
|
||||
completedAt: task.completedAt,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* 开始识别处理流程
|
||||
*/
|
||||
private async startRecognitionProcess(taskId: string): Promise<void> {
|
||||
try {
|
||||
const task = await this.taskModel.findByPk(taskId);
|
||||
if (!task) return;
|
||||
|
||||
// 阶段1: 产品识别分析 (0-40%)
|
||||
await this.updateTaskStatus(
|
||||
taskId,
|
||||
RecognitionStatusEnum.ANALYZING_PRODUCT,
|
||||
'正在识别药品基本信息...',
|
||||
10,
|
||||
);
|
||||
const productInfo = await this.recognizeProduct(task);
|
||||
await this.updateTaskStatus(
|
||||
taskId,
|
||||
RecognitionStatusEnum.ANALYZING_PRODUCT,
|
||||
'药品基本信息识别完成',
|
||||
40,
|
||||
);
|
||||
|
||||
// 阶段2: 适宜人群分析 (40-60%)
|
||||
await this.updateTaskStatus(
|
||||
taskId,
|
||||
RecognitionStatusEnum.ANALYZING_SUITABILITY,
|
||||
'正在分析适宜人群...',
|
||||
50,
|
||||
);
|
||||
const suitabilityInfo = await this.analyzeSuitability(productInfo);
|
||||
await this.updateTaskStatus(
|
||||
taskId,
|
||||
RecognitionStatusEnum.ANALYZING_SUITABILITY,
|
||||
'适宜人群分析完成',
|
||||
60,
|
||||
);
|
||||
|
||||
// 阶段3: 成分分析 (60-80%)
|
||||
await this.updateTaskStatus(
|
||||
taskId,
|
||||
RecognitionStatusEnum.ANALYZING_INGREDIENTS,
|
||||
'正在分析主要成分...',
|
||||
70,
|
||||
);
|
||||
const ingredientsInfo = await this.analyzeIngredients(productInfo);
|
||||
await this.updateTaskStatus(
|
||||
taskId,
|
||||
RecognitionStatusEnum.ANALYZING_INGREDIENTS,
|
||||
'成分分析完成',
|
||||
80,
|
||||
);
|
||||
|
||||
// 阶段4: 副作用分析 (80-100%)
|
||||
await this.updateTaskStatus(
|
||||
taskId,
|
||||
RecognitionStatusEnum.ANALYZING_EFFECTS,
|
||||
'正在分析副作用和健康建议...',
|
||||
90,
|
||||
);
|
||||
const effectsInfo = await this.analyzeEffects(productInfo);
|
||||
|
||||
// 合并所有结果
|
||||
const finalResult = {
|
||||
...productInfo,
|
||||
...suitabilityInfo,
|
||||
...ingredientsInfo,
|
||||
...effectsInfo,
|
||||
} as RecognitionResultDto;
|
||||
|
||||
// 完成识别
|
||||
await this.completeTask(taskId, finalResult);
|
||||
this.logger.log(`识别任务 ${taskId} 完成`);
|
||||
} catch (error) {
|
||||
const errorMessage =
|
||||
error instanceof Error ? error.message : String(error);
|
||||
this.logger.error(`识别任务 ${taskId} 失败: ${errorMessage}`);
|
||||
await this.failTask(taskId, errorMessage);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 阶段1: 识别药品基本信息
|
||||
*/
|
||||
private async recognizeProduct(
|
||||
task: MedicationRecognitionTask,
|
||||
): Promise<Partial<RecognitionResultDto>> {
|
||||
const prompt = this.buildProductRecognitionPrompt();
|
||||
const images = [task.frontImageUrl, task.sideImageUrl];
|
||||
if (task.auxiliaryImageUrl) images.push(task.auxiliaryImageUrl);
|
||||
|
||||
this.logger.log(
|
||||
`调用视觉模型识别药品，图片数量: ${images.length}, 任务ID: ${task.id}`,
|
||||
);
|
||||
|
||||
const response = await this.client.chat.completions.create({
|
||||
model: this.visionModel,
|
||||
temperature: 0.3,
|
||||
messages: [
|
||||
{
|
||||
role: 'user',
|
||||
content: [
|
||||
{ type: 'text', text: prompt },
|
||||
...images.map((url) => ({
|
||||
type: 'image_url',
|
||||
image_url: { url },
|
||||
})),
|
||||
] as any,
|
||||
},
|
||||
],
|
||||
response_format: { type: 'json_object' },
|
||||
} as any);
|
||||
|
||||
const content = response.choices[0]?.message?.content;
|
||||
if (!content) {
|
||||
throw new Error('AI模型返回内容为空');
|
||||
}
|
||||
|
||||
const parsed = this.parseJsonResponse(content);
|
||||
this.logger.log(`药品基本信息识别完成: ${parsed.name}, 置信度: ${parsed.confidence}`);
|
||||
return parsed;
|
||||
}
|
||||
|
||||
/**
|
||||
* 阶段2: 分析适宜人群
|
||||
*/
|
||||
private async analyzeSuitability(
|
||||
productInfo: Partial<RecognitionResultDto>,
|
||||
): Promise<Partial<RecognitionResultDto>> {
|
||||
const prompt = this.buildSuitabilityAnalysisPrompt(productInfo);
|
||||
|
||||
this.logger.log(`分析适宜人群: ${productInfo.name}`);
|
||||
|
||||
const response = await this.client.chat.completions.create({
|
||||
model: this.textModel,
|
||||
temperature: 0.7,
|
||||
messages: [
|
||||
{
|
||||
role: 'user',
|
||||
content: prompt,
|
||||
},
|
||||
],
|
||||
response_format: { type: 'json_object' },
|
||||
});
|
||||
|
||||
const content = response.choices[0]?.message?.content;
|
||||
if (!content) {
|
||||
throw new Error('AI模型返回内容为空');
|
||||
}
|
||||
|
||||
return this.parseJsonResponse(content);
|
||||
}
|
||||
|
||||
/**
|
||||
* 阶段3: 分析主要成分
|
||||
*/
|
||||
private async analyzeIngredients(
|
||||
productInfo: Partial<RecognitionResultDto>,
|
||||
): Promise<Partial<RecognitionResultDto>> {
|
||||
const prompt = this.buildIngredientsAnalysisPrompt(productInfo);
|
||||
|
||||
this.logger.log(`分析主要成分: ${productInfo.name}`);
|
||||
|
||||
const response = await this.client.chat.completions.create({
|
||||
model: this.textModel,
|
||||
temperature: 0.7,
|
||||
messages: [
|
||||
{
|
||||
role: 'user',
|
||||
content: prompt,
|
||||
},
|
||||
],
|
||||
response_format: { type: 'json_object' },
|
||||
});
|
||||
|
||||
const content = response.choices[0]?.message?.content;
|
||||
if (!content) {
|
||||
throw new Error('AI模型返回内容为空');
|
||||
}
|
||||
|
||||
return this.parseJsonResponse(content);
|
||||
}
|
||||
|
||||
/**
|
||||
* 阶段4: 分析副作用和健康建议
|
||||
*/
|
||||
private async analyzeEffects(
|
||||
productInfo: Partial<RecognitionResultDto>,
|
||||
): Promise<Partial<RecognitionResultDto>> {
|
||||
const prompt = this.buildEffectsAnalysisPrompt(productInfo);
|
||||
|
||||
this.logger.log(`分析副作用和健康建议: ${productInfo.name}`);
|
||||
|
||||
const response = await this.client.chat.completions.create({
|
||||
model: this.textModel,
|
||||
temperature: 0.7,
|
||||
messages: [
|
||||
{
|
||||
role: 'user',
|
||||
content: prompt,
|
||||
},
|
||||
],
|
||||
response_format: { type: 'json_object' },
|
||||
});
|
||||
|
||||
const content = response.choices[0]?.message?.content;
|
||||
if (!content) {
|
||||
throw new Error('AI模型返回内容为空');
|
||||
}
|
||||
|
||||
return this.parseJsonResponse(content);
|
||||
}
|
||||
|
||||
/**
|
||||
* 构建产品识别提示词
|
||||
*/
|
||||
private buildProductRecognitionPrompt(): string {
|
||||
return `你是一位拥有20年从业经验的资深药剂师，请根据提供的药品图片（包括正面、侧面和可能的辅助面）进行详细分析。
|
||||
|
||||
**分析要求**：
|
||||
1. 仔细观察药品包装、说明书上的所有信息
|
||||
2. 识别药品的完整名称（通用名和商品名）
|
||||
3. 确定药物剂型（片剂/胶囊/注射剂等）
|
||||
4. 提取规格剂量信息
|
||||
5. 推荐合理的服用次数和时间
|
||||
|
||||
**置信度评估标准**：
|
||||
- 如果图片清晰且信息完整，置信度应 >= 0.8
|
||||
- 如果部分信息不清晰但可推断，置信度 0.5-0.8
|
||||
- 如果无法准确识别，置信度 < 0.5，name返回"无法识别"
|
||||
|
||||
**返回严格的JSON格式**（不要包含任何markdown标记）：
|
||||
{
|
||||
"name": "药品完整名称",
|
||||
"photoUrl": "使用正面图片URL",
|
||||
"form": "剂型(tablet/capsule/injection/drops/syrup/ointment/powder/granules)",
|
||||
"dosageValue": 剂量数值(数字),
|
||||
"dosageUnit": "剂量单位",
|
||||
"timesPerDay": 建议每日服用次数(数字),
|
||||
"medicationTimes": ["建议的服药时间，格式HH:mm"],
|
||||
"confidence": 识别置信度(0-1的小数)
|
||||
}
|
||||
|
||||
**重要**：
|
||||
- dosageValue 和 timesPerDay 必须是数字类型，不要加引号
|
||||
- confidence 必须是 0-1 之间的小数
|
||||
- medicationTimes 必须是 HH:mm 格式的时间数组
|
||||
- form 必须是枚举值之一
|
||||
- 如果无法识别，name返回"无法识别"，其他字段返回合理的默认值`;
|
||||
}
|
||||
|
||||
/**
|
||||
* 构建适宜人群分析提示词
|
||||
*/
|
||||
private buildSuitabilityAnalysisPrompt(
|
||||
productInfo: Partial<RecognitionResultDto>,
|
||||
): string {
|
||||
return `作为资深药剂师，请分析以下药品的适宜人群和禁忌人群：
|
||||
|
||||
**药品信息**：
|
||||
- 名称：${productInfo.name}
|
||||
- 剂型：${productInfo.form}
|
||||
- 剂量：${productInfo.dosageValue}${productInfo.dosageUnit}
|
||||
|
||||
请以严格的JSON格式返回（不要包含任何markdown标记）：
|
||||
{
|
||||
"suitableFor": ["适合人群1", "适合人群2", "适合人群3"],
|
||||
"unsuitableFor": ["不适合人群1", "不适合人群2", "不适合人群3"],
|
||||
"mainUsage": "药品的主要用途和适应症描述"
|
||||
}
|
||||
|
||||
**要求**：
|
||||
- suitableFor 和 unsuitableFor 必须是字符串数组，至少包含3项
|
||||
- mainUsage 是字符串，描述药品的主要治疗用途
|
||||
- 如果无法识别药品，所有数组返回空数组，mainUsage返回"无法识别药品"`;
|
||||
}
|
||||
|
||||
/**
|
||||
* 构建成分分析提示词
|
||||
*/
|
||||
private buildIngredientsAnalysisPrompt(
|
||||
productInfo: Partial<RecognitionResultDto>,
|
||||
): string {
|
||||
return `作为资深药剂师，请分析以下药品的主要成分：
|
||||
|
||||
**药品信息**：
|
||||
- 名称：${productInfo.name}
|
||||
- 用途：${productInfo.mainUsage}
|
||||
|
||||
请以严格的JSON格式返回（不要包含任何markdown标记）：
|
||||
{
|
||||
"mainIngredients": ["主要成分1", "主要成分2", "主要成分3"]
|
||||
}
|
||||
|
||||
**要求**：
|
||||
- mainIngredients 必须是字符串数组，列出药品的主要活性成分
|
||||
- 至少包含1-3个主要成分
|
||||
- 如果无法确定，返回空数组`;
|
||||
}
|
||||
|
||||
/**
|
||||
* 构建副作用分析提示词
|
||||
*/
|
||||
private buildEffectsAnalysisPrompt(
|
||||
productInfo: Partial<RecognitionResultDto>,
|
||||
): string {
|
||||
return `作为资深药剂师，请分析以下药品的副作用、储存建议和健康建议：
|
||||
|
||||
**药品信息**：
|
||||
- 名称：${productInfo.name}
|
||||
- 用途：${productInfo.mainUsage}
|
||||
- 成分：${productInfo.mainIngredients?.join('、')}
|
||||
|
||||
请以严格的JSON格式返回（不要包含任何markdown标记）：
|
||||
{
|
||||
"sideEffects": ["副作用1", "副作用2", "副作用3"],
|
||||
"storageAdvice": ["储存建议1", "储存建议2", "储存建议3"],
|
||||
"healthAdvice": ["健康建议1", "健康建议2", "健康建议3"]
|
||||
}
|
||||
|
||||
**要求**：
|
||||
- 所有字段都是字符串数组
|
||||
- sideEffects: 列出常见和严重的副作用，至少3项
|
||||
- storageAdvice: 提供正确的储存方法，至少2项
|
||||
- healthAdvice: 给出配合用药的生活建议，至少3项
|
||||
- 如果无法确定，返回空数组`;
|
||||
}
|
||||
|
||||
/**
|
||||
* 解析JSON响应
|
||||
*/
|
||||
private parseJsonResponse(content: string): any {
|
||||
try {
|
||||
// 移除可能的 markdown 代码块标记
|
||||
let jsonString = content.trim();
|
||||
const jsonMatch = content.match(/```json\s*([\s\S]*?)\s*```/);
|
||||
if (jsonMatch) {
|
||||
jsonString = jsonMatch[1];
|
||||
} else {
|
||||
// 尝试提取第一个 { 到最后一个 }
|
||||
const firstBrace = content.indexOf('{');
|
||||
const lastBrace = content.lastIndexOf('}');
|
||||
if (firstBrace !== -1 && lastBrace !== -1) {
|
||||
jsonString = content.substring(firstBrace, lastBrace + 1);
|
||||
}
|
||||
}
|
||||
|
||||
return JSON.parse(jsonString);
|
||||
} catch (error) {
|
||||
this.logger.error(
|
||||
`解析JSON响应失败: ${error instanceof Error ? error.message : String(error)}, Content: ${content}`,
|
||||
);
|
||||
throw new Error('AI响应格式错误，无法解析');
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 更新任务状态
|
||||
*/
|
||||
private async updateTaskStatus(
|
||||
taskId: string,
|
||||
status: RecognitionStatusEnum,
|
||||
currentStep: string,
|
||||
progress: number,
|
||||
): Promise<void> {
|
||||
await this.taskModel.update(
|
||||
{
|
||||
status,
|
||||
currentStep,
|
||||
progress,
|
||||
},
|
||||
{
|
||||
where: { id: taskId },
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* 完成任务
|
||||
*/
|
||||
private async completeTask(
|
||||
taskId: string,
|
||||
result: RecognitionResultDto,
|
||||
): Promise<void> {
|
||||
await this.taskModel.update(
|
||||
{
|
||||
status: RecognitionStatusEnum.COMPLETED,
|
||||
currentStep: RECOGNITION_STATUS_DESCRIPTIONS[RecognitionStatusEnum.COMPLETED],
|
||||
progress: 100,
|
||||
recognitionResult: JSON.stringify(result),
|
||||
completedAt: new Date(),
|
||||
},
|
||||
{
|
||||
where: { id: taskId },
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* 任务失败
|
||||
*/
|
||||
private async failTask(taskId: string, errorMessage: string): Promise<void> {
|
||||
await this.taskModel.update(
|
||||
{
|
||||
status: RecognitionStatusEnum.FAILED,
|
||||
currentStep: RECOGNITION_STATUS_DESCRIPTIONS[RecognitionStatusEnum.FAILED],
|
||||
progress: 0,
|
||||
errorMessage,
|
||||
completedAt: new Date(),
|
||||
},
|
||||
{
|
||||
where: { id: taskId },
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -16,7 +16,7 @@ import * as dayjs from 'dayjs';
|
||||
@Injectable()
|
||||
export class MedicationReminderService {
|
||||
private readonly logger = new Logger(MedicationReminderService.name);
|
||||
private readonly REMINDER_MINUTES_BEFORE = 15; // 提前15分钟提醒
|
||||
private readonly REMINDER_MINUTES_BEFORE = 5; // 提前5分钟提醒
|
||||
private readonly OVERDUE_HOURS_THRESHOLD = 1; // 超过1小时后发送超时提醒
|
||||
|
||||
constructor(
|
||||
|
||||
@@ -1,8 +1,9 @@
|
||||
import { Injectable, Logger, OnModuleInit, OnModuleDestroy } from '@nestjs/common';
|
||||
import { Injectable, Logger, OnModuleInit, OnModuleDestroy, Inject, forwardRef } from '@nestjs/common';
|
||||
import { ConfigService } from '@nestjs/config';
|
||||
import { ApnsClient, SilentNotification, Notification, Errors } from 'apns2';
|
||||
import * as fs from 'fs';
|
||||
import { ApnsConfig, ApnsNotificationOptions } from './interfaces/apns-config.interface';
|
||||
import { PushTokenService } from './push-token.service';
|
||||
|
||||
interface SendResult {
|
||||
sent: string[];
|
||||
@@ -20,7 +21,11 @@ export class ApnsProvider implements OnModuleInit, OnModuleDestroy {
|
||||
private client: ApnsClient;
|
||||
private config: ApnsConfig;
|
||||
|
||||
constructor(private readonly configService: ConfigService) {
|
||||
constructor(
|
||||
private readonly configService: ConfigService,
|
||||
@Inject(forwardRef(() => PushTokenService))
|
||||
private readonly pushTokenService: PushTokenService,
|
||||
) {
|
||||
this.config = this.buildConfig();
|
||||
}
|
||||
|
||||
@@ -97,27 +102,45 @@ export class ApnsProvider implements OnModuleInit, OnModuleDestroy {
|
||||
|
||||
/**
|
||||
* 设置错误处理器
|
||||
* 自动停用无效的设备令牌
|
||||
*/
|
||||
private setupErrorHandlers(): void {
|
||||
// 监听特定错误
|
||||
this.client.on(Errors.badDeviceToken, (err) => {
|
||||
this.logger.error(`Bad device token: ${err}`, err.reason);
|
||||
// 监听无效设备令牌错误 - 需要停用
|
||||
this.client.on(Errors.badDeviceToken, async (err) => {
|
||||
this.logger.error(`Bad device token detected: ${err.deviceToken}`, err.reason);
|
||||
await this.deactivateInvalidToken(err.deviceToken, 'BadDeviceToken');
|
||||
});
|
||||
|
||||
this.client.on(Errors.unregistered, (err) => {
|
||||
// 监听设备注销错误 - 用户已卸载应用，需要停用
|
||||
this.client.on(Errors.unregistered, async (err) => {
|
||||
this.logger.error(`Device unregistered: ${err.deviceToken}`, err.reason);
|
||||
await this.deactivateInvalidToken(err.deviceToken, 'Unregistered');
|
||||
});
|
||||
|
||||
this.client.on(Errors.topicDisallowed, (err) => {
|
||||
this.logger.error(`Topic disallowed: ${err.deviceToken}`, err.reason);
|
||||
// 监听 topic 不匹配错误 - bundle ID 配置错误，需要停用
|
||||
this.client.on(Errors.topicDisallowed, async (err) => {
|
||||
this.logger.error(`Topic disallowed for device: ${err.deviceToken}`, err.reason);
|
||||
await this.deactivateInvalidToken(err.deviceToken, 'TopicDisallowed');
|
||||
});
|
||||
|
||||
// 监听所有错误
|
||||
// 监听所有其他错误
|
||||
this.client.on(Errors.error, (err) => {
|
||||
this.logger.error(`APNs error for device ${err.deviceToken}: ${err.reason}`, err);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* 停用无效的设备令牌
|
||||
*/
|
||||
private async deactivateInvalidToken(deviceToken: string, reason: string): Promise<void> {
|
||||
try {
|
||||
this.logger.warn(`Deactivating invalid token due to ${reason}: ${deviceToken}`);
|
||||
await this.pushTokenService.deactivateToken(deviceToken);
|
||||
} catch (error) {
|
||||
this.logger.error(`Failed to deactivate token ${deviceToken}: ${error.message}`, error);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 发送单个通知
|
||||
*/
|
||||
|
||||
@@ -110,7 +110,8 @@ export class PushNotificationsService {
|
||||
sentCount++;
|
||||
} else {
|
||||
const failure = apnsResults.failed[0];
|
||||
const errorMessage = failure.error ? failure.error.message : `APNs Error: ${failure.status}`;
|
||||
const error = failure.error as any;
|
||||
const errorMessage = error?.message || `APNs Error: ${failure.status || 'Unknown'}`;
|
||||
|
||||
await this.pushMessageService.updateMessageStatus(
|
||||
message.id,
|
||||
@@ -119,9 +120,12 @@ export class PushNotificationsService {
|
||||
errorMessage
|
||||
);
|
||||
|
||||
// 如果是无效令牌，停用该令牌
|
||||
if (failure.status === '410' || failure.response?.reason === 'Unregistered') {
|
||||
await this.pushTokenService.unregisterToken(userId, deviceToken);
|
||||
// 检查是否是无效令牌错误 - 需要停用 token
|
||||
const shouldDeactivateToken = this.shouldDeactivateToken(error, failure.response);
|
||||
|
||||
if (shouldDeactivateToken) {
|
||||
this.logger.warn(`Deactivating invalid token for user ${userId}: ${errorMessage}`);
|
||||
await this.pushTokenService.deactivateToken(deviceToken);
|
||||
}
|
||||
|
||||
results.push({
|
||||
@@ -173,6 +177,40 @@ export class PushNotificationsService {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 判断是否应该停用 token
|
||||
* 根据 APNs 错误类型判断 token 是否已失效
|
||||
*/
|
||||
private shouldDeactivateToken(error: any, response: any): boolean {
|
||||
if (!error && !response) {
|
||||
return false;
|
||||
}
|
||||
|
||||
// 检查错误对象的 reason 字段（apns2 库的错误格式）
|
||||
const reason = error?.reason || response?.reason || '';
|
||||
|
||||
// APNs 返回的需要停用 token 的错误原因
|
||||
const invalidTokenReasons = [
|
||||
'BadDeviceToken', // 无效的设备令牌格式
|
||||
'Unregistered', // 设备已注销（用户卸载了应用）
|
||||
'DeviceTokenNotForTopic', // token 与 bundle ID 不匹配
|
||||
'ExpiredToken', // token 已过期
|
||||
];
|
||||
|
||||
// 检查是否包含这些错误原因
|
||||
if (invalidTokenReasons.some(r => reason.includes(r))) {
|
||||
return true;
|
||||
}
|
||||
|
||||
// 检查 HTTP 状态码 410 (Gone) - 表示设备令牌永久失效
|
||||
const statusCode = error?.statusCode || response?.statusCode;
|
||||
if (statusCode === 410 || statusCode === '410') {
|
||||
return true;
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* 使用模板发送推送通知
|
||||
*/
|
||||
@@ -311,7 +349,8 @@ export class PushNotificationsService {
|
||||
} else {
|
||||
// 发送失败
|
||||
const failure = apnsResult as any;
|
||||
const errorMessage = failure.error ? failure.error.message : `APNs Error: ${failure.status}`;
|
||||
const error = failure.error as any;
|
||||
const errorMessage = error?.message || `APNs Error: ${failure.status || 'Unknown'}`;
|
||||
|
||||
await this.pushMessageService.updateMessageStatus(
|
||||
message.id,
|
||||
@@ -320,9 +359,12 @@ export class PushNotificationsService {
|
||||
errorMessage
|
||||
);
|
||||
|
||||
// 如果是无效令牌，停用该令牌
|
||||
if (failure.status === '410' || failure.response?.reason === 'Unregistered') {
|
||||
await this.pushTokenService.unregisterToken(userId, deviceToken);
|
||||
// 检查是否是无效令牌错误 - 需要停用 token
|
||||
const shouldDeactivateToken = this.shouldDeactivateToken(error, failure.response);
|
||||
|
||||
if (shouldDeactivateToken) {
|
||||
this.logger.warn(`Deactivating invalid token for user ${userId}: ${errorMessage}`);
|
||||
await this.pushTokenService.deactivateToken(deviceToken);
|
||||
}
|
||||
|
||||
results.push({
|
||||
@@ -597,7 +639,8 @@ export class PushNotificationsService {
|
||||
sentCount++;
|
||||
} else {
|
||||
const failure = apnsResults.failed[0];
|
||||
const errorMessage = failure.error ? failure.error.message : `APNs Error: ${failure.status}`;
|
||||
const error = failure.error as any;
|
||||
const errorMessage = error?.message || `APNs Error: ${failure.status || 'Unknown'}`;
|
||||
|
||||
await this.pushMessageService.updateMessageStatus(
|
||||
message.id,
|
||||
@@ -606,14 +649,12 @@ export class PushNotificationsService {
|
||||
errorMessage
|
||||
);
|
||||
|
||||
// 如果是无效令牌，停用该令牌
|
||||
if (failure.status === '410' || failure.response?.reason === 'Unregistered') {
|
||||
if (userId) {
|
||||
await this.pushTokenService.unregisterToken(userId, deviceToken);
|
||||
} else {
|
||||
// 如果没有用户ID，直接停用令牌
|
||||
await this.pushTokenService.deactivateToken(deviceToken);
|
||||
}
|
||||
// 检查是否是无效令牌错误 - 需要停用 token
|
||||
const shouldDeactivateToken = this.shouldDeactivateToken(error, failure.response);
|
||||
|
||||
if (shouldDeactivateToken) {
|
||||
this.logger.warn(`Deactivating invalid token for device ${deviceToken}: ${errorMessage}`);
|
||||
await this.pushTokenService.deactivateToken(deviceToken);
|
||||
}
|
||||
|
||||
results.push({
|
||||
@@ -738,7 +779,8 @@ export class PushNotificationsService {
|
||||
} else {
|
||||
// 发送失败
|
||||
const failure = apnsResult as any;
|
||||
const errorMessage = failure.error ? failure.error.message : `APNs Error: ${failure.status}`;
|
||||
const error = failure.error as any;
|
||||
const errorMessage = error?.message || `APNs Error: ${failure.status || 'Unknown'}`;
|
||||
|
||||
await this.pushMessageService.updateMessageStatus(
|
||||
message.id,
|
||||
@@ -747,14 +789,12 @@ export class PushNotificationsService {
|
||||
errorMessage
|
||||
);
|
||||
|
||||
// 如果是无效令牌，停用该令牌
|
||||
if (failure.status === '410' || failure.response?.reason === 'Unregistered') {
|
||||
if (userId) {
|
||||
await this.pushTokenService.unregisterToken(userId, deviceToken);
|
||||
} else {
|
||||
// 如果没有用户ID，直接停用令牌
|
||||
await this.pushTokenService.deactivateToken(deviceToken);
|
||||
}
|
||||
// 检查是否是无效令牌错误 - 需要停用 token
|
||||
const shouldDeactivateToken = this.shouldDeactivateToken(error, failure.response);
|
||||
|
||||
if (shouldDeactivateToken) {
|
||||
this.logger.warn(`Deactivating invalid token for device ${deviceToken}: ${errorMessage}`);
|
||||
await this.pushTokenService.deactivateToken(deviceToken);
|
||||
}
|
||||
|
||||
results.push({
|
||||
|
||||
@@ -205,6 +205,12 @@ export class UsersService {
|
||||
if (!user) {
|
||||
throw new NotFoundException(`ID为${userId}的用户不存在`);
|
||||
}
|
||||
|
||||
// 会员用户不扣减
|
||||
if (user.isVip) {
|
||||
return
|
||||
}
|
||||
|
||||
user.freeUsageCount -= count;
|
||||
await user.save();
|
||||
} catch (error) {
|
||||
|
||||
Reference in New Issue
Block a user