diff --git a/.codebuddy/memory/2026-06-01.md b/.codebuddy/memory/2026-06-01.md new file mode 100644 index 0000000..e53bf9e --- /dev/null +++ b/.codebuddy/memory/2026-06-01.md @@ -0,0 +1,29 @@ +# 2026-06-01 工作记录 + +## 快手代发订单详情新增 +- 新增 `open.dropshipping.order.merchant.detail` 到 `seed_data_kuaishou.sql` +- 第19个接口，POST、prefetch 自 dropshipping_order_list、single_record + +## 快手接口全面审查 +发现 prefetch 相关7个接口存在3个严重Bug： +1. Prefetch 阶段业务参数未包入 param JSON（body_wrapper_field 未生效） +2. Prefetch 阶段 `parseResp(nil config)` 无法解析 data.items/data.cpsOrderList 等路径 +3. Prefetch 循环未处理游标分页（固定页码递增） + +## Bug 修复（dynamic_sync.go） +1. `syncWithPrefetch` 改用 `buildReqBody(prefetchIface)` 构建请求，body_wrapper_field 正确包装 +2. 改用 `parseRespExt(resp.Body, prefetchIface.ResponseConfig)` 解析响应 +3. 支持游标分页的 prefetch 循环（cursor/pcursor） +4. `buildPrefetchParams` 增加过滤 body_wrapper_field/exclude_from_wrapper/cursor_pagination/time_field_mode +5. 新增 `collectPrefetchEntities` 辅助函数 +6. 修复并发阶段 `inQuery` 变量缺失的问题 + +## 同步调度并发锁修复 +1. `SyncByConfig` 新增 `syncRunningMap` 内存锁（sync.Map.LoadOrStore），防止同一接口并发执行 +2. 调度器从 `time.NewTicker` 改为 `for { run(); time.Sleep(interval) }`，前一次完成后才开始计时 + +## MEMORY.md 更新 +- response_config 增加 `single_record` 字段说明 +- prefetch 流程补充游标分页、参数构建、响应解析的详细说明 +- 新增「并发保护」章节，记录内存锁和调度器时序 +- 调度器章节补充 for+sleep 模式说明 diff --git a/.codebuddy/memory/MEMORY.md b/.codebuddy/memory/MEMORY.md new file mode 100644 index 0000000..6a467a9 --- /dev/null +++ b/.codebuddy/memory/MEMORY.md @@ -0,0 +1,479 @@ +# 数据引擎 - 项目记忆 (MEMORY.md) + +## 项目概述 +Golang + GoFrame v2 + PostgreSQL 的配置化数据同步引擎。通过 `api_datasource_platform` + `api_interface` 两表驱动，支持多平台多接口的数据同步，无需重新编译代码。 + +## 核心表结构 +见 `sql/init_core_tables.sql`： +- `api_datasource_platform` — 数据源平台配置 +- `api_interface` — API 接口配置（request_config / response_config / table_definition 三大 JSONB） +- `sync_tracker` — 同步跟踪（last_sync_time, sync_status） +- `sync_task_log` — 同步任务日志（用于补偿重试） + +## 认证类型（auth_type）模板 + +### 1. OAuth2（腾讯广告） +``` +auth_type: OAUTH2 +token: access_token 值 +client_id / client_secret: OAuth2 凭证 +auth_config: { + "token_in_query": true, // token 放 URL 查询参数 + "query_key": "access_token", // 参数名 + "refresh_token": "xxx", // 刷新 token + "extra_query_params": { + "timestamp": "{timestamp}", + "nonce": "{nonce}" + } +} +``` +安全校验：`applyAuthHeader` 中 OAUTH2/TOKEN 模式会设 `Authorization: Bearer {token}`，但 `token_in_query=true` 时跳过 Header，转加 URL 参数。 + +### 2. API_KEY + 签名（快手电商） +``` +auth_type: API_KEY +token/access_token: 调用凭证 +auth_config: { + "sign_algorithm": "md5_upper", // 签名算法: md5 / md5_upper + "app_key": "xxx", + "app_secret": "xxx", + "token_in_query": true, + "query_key": "access_token", + "extra_query_params": { + "timestamp": "{timestamp_ms}", + "signMethod": "MD5" + } +} +``` +签名过程：所有查询参数按 key 排序 → `k1=v1&k2=v2&...&key=app_secret` → MD5 摘要 → 追加 `sign` 参数。 +`{timestamp_ms}` 替换为毫秒时间戳，`{timestamp}` 替换为秒时间戳，`{nonce}` 替换为随机数。 + +### 3. TOKEN（简单 Bearer Token） +``` +auth_type: TOKEN +token: "xxx" +``` +`applyAuthHeader` 设 `Authorization: Bearer {token}`。 + +### 4. SIGN（仅签名，无额外 token） +``` +auth_type: SIGN +auth_config: { "app_key": "...", "app_secret": "..." } +``` +`applyAuthHeader` 不设特殊 Header，仅通过 `applySignature` 追加签名参数。`PlatformManager` 从 `auth_config` 读取 app_key/app_secret。 + +## request_config 字段完整参考 + +| 字段 | 类型 | 含义 | 适用平台 | +|------|------|------|---------| +| `parameters_location` | string | `"query"`=URL查询参数，不设则POST走JSON body | 通用 | +| `page_param` | string | 分页参数名，默认`"page"` | 通用 | +| `page_size_param` | string | 每页条数参数名，默认`"page_size"` | 通用 | +| `page_size` | int | 每页条数，**覆盖 config.yml 全局默认值**，需按各平台最大限制设置（如快手最大50，腾讯100） | 通用 | +| `cursor_pagination` | bool | 是否游标分页（true=游标，false/无=普通分页） | 通用 | +| `time_field` | string | 增量时间字段名 | 通用 | +| `time_field_mode` | string | `"range"`=beginTime/endTime模式（快手），`"filtering"`=filtering数组模式（腾讯，默认） | 通用 | +| `prefetch` | object | 预取配置（见下方） | 需遍历实体的接口 | +| `body_wrapper_field` | string | 业务参数包装字段名，如`"param"` | 快手 | +| `exclude_from_wrapper` | string[] | body_wrapper 时不包装的字段 | 快手 | +| `top_level_params` | string[] | 保留在顶层的字段（备用） | 通用 | +| `fields` | string[] | API 的 fields 参数 | 腾讯 | + +### prefetch 预取配置 +```json +"prefetch": { + "url": "/advertiser/get", // 预取接口的相对路径 + "method": "GET", // HTTP 方法 + "response_path": "data.list", // 从响应中取列表的 JSON 路径 + "target_param": "account_id", // 预取值注入到数据接口的参数名 + "value_field": "account_id" // 从预取列表里取哪个字段的值 +} +``` +流程：先分页拉取预取接口 → 提取所有 ID → 逐个 ID 并发拉取数据接口 → 将 ID 注入每行数据。 + +**预取阶段分页**：自动适配预取来源接口的分页方式： +- **游标分页**（`cursor_pagination: true`）：首次 `cursor=""`，后续从响应 `cursor_field` 取值，直到 `""` 或 `"nomore"` 停止 +- **普通分页**：读取 `page_info.total_page` 遍历全部页码 + +**预取阶段参数构建**：使用 `buildReqBody(prefetchIface)` 构建，自动处理 `body_wrapper_field` 包装（param JSON）、时间过滤（增量时传 beginTime/endTime）、分页参数名。响应解析使用 `prefetchIface.ResponseConfig`，正确识别各种 `list_path`（`data.orderList`、`data.items`、`data.cpsOrderList` 等）。 + +**预取阶段时间过滤**：增量同步时自动复用预取来源接口的 `time_field` + `time_field_mode`（如果来源支持增量，prefetch 会自动带上时间范围只拉增量数据）。 + +## response_config 字段完整参考 + +| 字段 | 类型 | 含义 | +|------|------|------| +| `success_field` | string | 成功标识字段名，默认`"code"` | +| `success_value` | number | 成功标识值，默认`0` | +| `message_field` | string | 错误信息字段名，默认`"message"` | +| `list_path` | string | 数据列表在 JSON 中的路径，如`"data.list"`、`"data.orderList"` | +| `cursor_field` | string | 游标字段路径，如`"data.cursor"` | +| `cursor_end_marker` | string | 游标结束标记，如`"nomore"`（代码中硬判断此字符串） | +| `single_record` | bool | 单记录模式：`list_path` 指向的路径是单个对象而非数组，会自动包装为单元素数组 | + +**响应解析逻辑**：`parseRespExt` 函数 +1. 检查 `success_field` 的值是否等于 `success_value` | **注意**：`success_value` 仅支持数字类型（内部通过 `toFloat64` 转换），不支持字符串如 `"SUCCESS"` 或布尔值 `true` +2. 按 `list_path` 遍历到数据列表数组 | 支持路径如 `data.orderList`，最后一段是数组 OK（代码有兼容处理） +3. 每条数据调用 `flattenRow` 展平嵌套 field（`orderBaseInfo.oid` → `oid`） +4. 提取 `cursor_field` 用于下一页，为空或等于 `cursor_end_marker` 时停止循环 +5. 从 `page_info.total_page` 获取总页数（普通分页用） + +**游标分页首次请求**：代码自动传 `cursor=""`（首页不传游标），由服务端返回第一页数据 + 下一页游标值。 + +**单记录模式**：`"single_record": true` 时，`list_path` 指向单个对象（如 `"data"`），自动包装成单元素数组用于统一处理。 + +## table_definition 字段完整参考 + +| 字段 | 类型 | 含义 | +|------|------|------| +| `table_name` | string | 目标表名，如`"kuaishou_order_list"` | +| `columns` | array | 列定义数组，每个元素 `{"name":"字段名","type":"PG类型","comment":"说明"}` | +| `conflict_keys` | string[] | upsert 冲突键，如`["oid"]`或`["image_id","account_id"]` | + +注意： +- 列名必须和 API 响应中的字段名**完全一致**（camelCase/snake_case 取决于 API） +- 嵌套字段会被 `flattenRow` 自动展平到顶层 +- `raw_data` 列自动包含，存储完整原始 JSON +- 自动建表时会额外添加 id/tenant_id/creator/created_at/updater/updated_at/deleted_at 审计字段 +- **必须写全**：`columns` 中需列出 API 响应中所有平铺的标量字段（字符串/数字/布尔），数组和嵌套对象字段保留在 `raw_data` 中即可 + +## 同步策略原则 +- **API 支持时间过滤**（有 beginTime/endTime 或 filtering 参数）→ 配 `time_field` + `time_field_mode`，实现增量同步 +- **API 不支持时间过滤** → 不配 `time_field`，走全量同步 +- **详情/子接口**（通过 prefetch 遍历）：prefetch 阶段自动复用预取来源接口的 `time_field_mode`（如果来源接口支持增量，prefetch 也会带上时间过滤只拉增量数据） + +## 全量同步流程 + +``` +SyncByConfig(platformCode, interfaceCode, isFullSync=false) + │ + ├─ getLastSyncTime() → 无记录返回 0 + │ lastSyncTime=0 → 全量 + │ + ├─ markSyncRunning() 标记 running + │ + ├─ buildReqBody(page=1, cursor="", lastSyncTime=0) + │ ├─ 游标分页: cursor="" 首次空游标 + │ ├─ range 模式: beginTime=90天前, endTime=now + │ └─ filtering 模式: 不添加时间过滤 + │ + ├─ API → parseRespExt → flattenRow → savePage(upsert) + │ + └─ 游标循环 → 直到 cursor="" 或 "nomore" + updateSyncTime() 记录最大时间戳 +``` + +## 增量同步流程 + +``` +SyncByConfig(platformCode, interfaceCode, isFullSync=false) + │ + ├─ getLastSyncTime() → 返回上次 maxTime + │ lastSyncTime>0 → 增量 + │ + ├─ buildReqBody(cursor="", lastSyncTime=xxx) + │ ├─ range 模式: beginTime=lastSyncTime, endTime=now + │ └─ filtering 模式: filtering=[{field=time_field, operator=GREATER_EQUALS, values=[lastSyncTime]}] + │ + └─ 后续同全量（只拉取时间范围内的数据） +``` + +## 并发保护 + +### 内存锁（同一进程内） +`SyncByConfig` 入口使用 `sync.Map`（`syncRunningMap`）记录正在执行的接口。若同一接口的同步请求再次到达（如调度器重叠），直接跳过并打印警告： +``` +WARN 接口 [kuaishou/order_list] 正在同步中，跳过重复请求 +``` +无论接口类型（prefetch/非prefetch、腾讯/快手、现有/新增）均自动生效。 + +### 异常中断检测（DB 状态） +如果 `sync_tracker.sync_status = "running"`，说明上次同步异常中断（进程崩溃），自动回退全量。 + +## 补偿机制 +`compensation.go` 定时扫描 `sync_task_log` 中 status="failed" 的任务，自动调用 `SyncByConfig` 重试（增量），支持指数退避（5, 15, 30, 60, 120 分钟）。 + +## 自动同步调度 +`sync_scheduler.go` 的 `runAutoSync` 遍历所有 ACTIVE 平台下的所有配置了 table_definition 的活跃接口，自动调用同步。 + +**调度器时序**：使用 `for { runAutoSync(); time.Sleep(interval) }` 模式，而非 `time.NewTicker`。每次同步**完全结束后**才开始计时 interval，避免前一次未跑完就启动下一次导致重叠。无论全量跑 30 分钟还是 70 分钟，下一次都在「本次完成时间 + interval」后执行。 + +## 新增平台 + 接口的操作步骤 + +### 操作步骤 +1. 在 `sql/` 下创建 `seed_data_{平台编码}.sql` +2. 先执行 `sql/init_core_tables.sql`（仅首次需要） +3. 执行新建的 seed SQL 文件（INSERT 平台 → INSERT 接口） +4. 配置认证凭据（token / app_key / app_secret 等） +5. 重启服务，管理后台验证：`http://localhost:3002/admin` + +### 种子 SQL 标准模板 + +#### 模板 A：OAuth2 + 普通分页（腾讯广告风格） +```sql +-- ============================================= +-- 1. 创建 {平台编码} 平台 +-- ============================================= +INSERT INTO api_datasource_platform ( + tenant_id, creator, created_at, updater, updated_at, + platform_code, platform_name, description, status, + api_base_url, auth_type, + token, client_id, client_secret, + auth_config, + rate_limit_per_minute, rate_limit_per_hour, + concurrency_limit, request_timeout_ms, max_retries, retry_delay_ms +) VALUES ( + 1, 'admin', NOW(), 'admin', NOW(), + '{平台编码}', '{平台名称}', '{描述}', 'ACTIVE', + '{API_BASE_URL}', 'OAUTH2', + '{access_token}', '{client_id}', '{client_secret}', + '{ + "token_in_query": true, + "query_key": "access_token", + "refresh_token": "{refresh_token}", + "extra_query_params": { + "timestamp": "{timestamp}", + "nonce": "{nonce}" + } + }'::jsonb, + 60, 3600, 10, 30000, 3, 1000 +); + +-- 2. 创建 {接口名} 接口（普通分页 + 增量 filtering） +INSERT INTO api_interface ( + tenant_id, creator, created_at, updater, updated_at, + platform_id, name, code, url, method, status, auth_type, + request_config, response_config, table_definition +) VALUES ( + 1, 'admin', NOW(), 'admin', NOW(), + (SELECT id FROM api_datasource_platform WHERE platform_code = '{平台编码}'), + '{接口名称}', '{接口编码}', + '{相对路径}', 'GET', 'active', 'inherit', + '{ + "parameters_location": "query", + "page": 1, + "page_size": 100, + "page_param": "page", + "page_size_param": "page_size", + "pagination_mode": "PAGINATION_MODE_NORMAL", + "time_field": "last_modified_time", + "fields": ["field1", "field2"] + }'::jsonb, + '{ + "success_field": "code", + "success_value": 0, + "message_field": "message", + "list_path": "data.list" + }'::jsonb, + '{ + "table_name": "{表名}", + "columns": [ + {"name": "id", "type": "BIGINT", "comment": "主键"}, + {"name": "name", "type": "VARCHAR(200)", "comment": "名称"}, + {"name": "created_time", "type": "BIGINT", "comment": "创建时间"} + ], + "conflict_keys": ["id"] + }'::jsonb +); +``` + +#### 模板 B：OAuth2 + prefetch 预取（遍历账户拉取子数据，腾讯广告风格） +```sql +-- 预取接口（先拉取实体列表） +INSERT INTO api_interface ( + tenant_id, creator, created_at, updater, updated_at, + platform_id, name, code, url, method, status, auth_type, + request_config, response_config, table_definition +) VALUES ( + 1, 'admin', NOW(), 'admin', NOW(), + (SELECT id FROM api_datasource_platform WHERE platform_code = '{平台编码}'), + '{实体列表名称}', '{实体编码}', + '{实体列表相对路径}', 'GET', 'active', 'inherit', + '{ + "parameters_location": "query", + "page": 1, + "page_size": 100, + "page_param": "page", + "page_size_param": "page_size", + "fields": ["id", "name"] + }'::jsonb, + '{ + "success_field": "code", + "success_value": 0, + "list_path": "data.list" + }'::jsonb, + '{ + "table_name": "{实体表名}", + "columns": [ + {"name": "id", "type": "BIGINT", "comment": "实体ID"}, + {"name": "name", "type": "VARCHAR(200)", "comment": "实体名称"} + ], + "conflict_keys": ["id"] + }'::jsonb +); + +-- 数据接口（遍历每个实体拉取数据） +INSERT INTO api_interface ( + tenant_id, creator, created_at, updater, updated_at, + platform_id, name, code, url, method, status, auth_type, + request_config, response_config, table_definition +) VALUES ( + 1, 'admin', NOW(), 'admin', NOW(), + (SELECT id FROM api_datasource_platform WHERE platform_code = '{平台编码}'), + '{数据接口名称}', '{数据接口编码}', + '{数据接口相对路径}', 'GET', 'active', 'inherit', + '{ + "parameters_location": "query", + "page": 1, + "page_size": 100, + "page_param": "page", + "page_size_param": "page_size", + "time_field": "last_modified_time", + "prefetch": { + "url": "{实体列表相对路径}", + "method": "GET", + "response_path": "data.list", + "target_param": "{参数名}", + "value_field": "{取值字段}" + } + }'::jsonb, + '{ + "success_field": "code", + "success_value": 0, + "list_path": "data.list" + }'::jsonb, + '{ + "table_name": "{数据表名}", + "columns": [ + {"name": "id", "type": "BIGINT", "comment": "ID"}, + {"name": "{参数名}", "type": "BIGINT", "comment": "实体ID（预取注入）"}, + {"name": "created_time", "type": "BIGINT", "comment": "创建时间"} + ], + "conflict_keys": ["id", "{参数名}"] + }'::jsonb +); +``` + +#### 模板 C：API_KEY + MD5 签名 + 游标分页 + param 包装（快手风格） +```sql +INSERT INTO api_datasource_platform ( + tenant_id, creator, created_at, updater, updated_at, + platform_code, platform_name, description, status, + api_base_url, auth_type, + auth_config, + rate_limit_per_minute, rate_limit_per_hour, + concurrency_limit, request_timeout_ms, max_retries, retry_delay_ms +) VALUES ( + 1, 'admin', NOW(), 'admin', NOW(), + '{平台编码}', '{平台名称}', '{描述}', 'ACTIVE', + '{API_BASE_URL}', 'API_KEY', + '{ + "sign_algorithm": "md5_upper", + "app_key": "{YOUR_APP_KEY}", + "app_secret": "{YOUR_APP_SECRET}", + "token_in_query": true, + "query_key": "access_token", + "extra_query_params": { + "timestamp": "{timestamp_ms}", + "signMethod": "MD5" + } + }'::jsonb, + 100, 3600, 5, 30000, 3, 1000 +); + +INSERT INTO api_interface ( + tenant_id, creator, created_at, updater, updated_at, + platform_id, name, code, url, method, status, auth_type, + request_config, response_config, table_definition +) VALUES ( + 1, 'admin', NOW(), 'admin', NOW(), + (SELECT id FROM api_datasource_platform WHERE platform_code = '{平台编码}'), + '{接口名称}', '{接口编码}', + '{相对路径}', 'GET', 'active', 'inherit', + '{ + "parameters_location": "query", + "page_param": "cursor", + "page_size_param": "pageSize", + "cursor_pagination": true, + "method": "{API方法名}", + "version": 1, + "signMethod": "MD5", + "pageSize": 50, + "time_field": "updateTime", + "time_field_mode": "range", + "body_wrapper_field": "param", + "exclude_from_wrapper": ["method", "version", "signMethod"], + "{其他业务参数}": {值} + }'::jsonb, + '{ + "success_field": "result", + "success_value": 1, + "list_path": "data.{数组字段名}", + "cursor_field": "data.cursor", + "cursor_end_marker": "nomore" + }'::jsonb, + '{ + "table_name": "{表名}", + "columns": [ + {"name": "oid", "type": "BIGINT", "comment": "订单ID"}, + {"name": "createTime", "type": "BIGINT", "comment": "创建时间"}, + {"name": "updateTime", "type": "BIGINT", "comment": "更新时间"}, + {"name": "{字段名}", "type": "{类型}", "comment": "{说明}"} + ], + "conflict_keys": ["oid"] + }'::jsonb +); +``` + +#### 模板 D：POST + JSON Body（腾讯音频风格，无 prefetch，无增量） +```sql +INSERT INTO api_interface ( + tenant_id, creator, created_at, updater, updated_at, + platform_id, name, code, url, method, status, auth_type, + request_config, response_config, table_definition +) VALUES ( + 1, 'admin', NOW(), 'admin', NOW(), + (SELECT id FROM api_datasource_platform WHERE platform_code = '{平台编码}'), + '{接口名称}', '{接口编码}', + '{相对路径}', 'POST', 'active', 'inherit', + '{ + "page": 1, + "page_size": 100, + "page_param": "page", + "page_size_param": "page_size", + "fields": ["field1", "field2"] + }'::jsonb, + '{ + "success_field": "code", + "success_value": 0, + "message_field": "message", + "list_path": "data.list" + }'::jsonb, + '{ + "table_name": "{表名}", + "columns": [ + {"name": "id", "type": "BIGINT", "comment": "ID"}, + {"name": "name", "type": "VARCHAR(200)", "comment": "名称"} + ], + "conflict_keys": ["id"] + }'::jsonb +); +``` +注意：POST 方法且无 `parameters_location: "query"` 时，参数以 JSON body 形式发送。 + +## 新增接口步骤总结 + +1. **确定平台**：已有平台直接跳到第3步，否则先新增平台（参考对应认证类型的模板） +2. **确定平台 auth_type**：OAUTH2 / TOKEN / API_KEY / SIGN → 复制对应模板的 auth_config +3. **分析 API**： + - 请求方式（GET/POST） + - 分页方式（普通分页 → page + total_page / 游标分页 → cursor + cursor_end） + - 成功标识（code=0 / result=1 / errcode=0 等） + - 数据列表路径（data.list / data.orderList / data.records 等） + - 数据字段名（注意大小写，嵌套结构会被展开） + - 是否需要遍历实体（prefetch） + - 增量时间字段（字段名 + 值类型） +4. **选择模板**并填入对应值 +5. **执行 seed SQL** +6. **在管理后台验证**：`http://{host}:3002/admin` + diff --git a/config.yml b/config.yml index f88ec5e..4277f2a 100644 --- a/config.yml +++ b/config.yml @@ -16,7 +16,7 @@ sync: retry_count: 3 # 最大重试次数 sync_interval_minutes: 60 # 自动同步间隔（分钟） compensation_interval_seconds: 300 # 补偿调度器扫描间隔（秒） - auto_sync_enabled: true # 是否启用自动同步 + auto_sync_enabled: false # 是否启用自动同步 # Database. database: diff --git a/controller/debug/admin_controller.go b/controller/debug/admin_controller.go new file mode 100644 index 0000000..c8aab5d --- /dev/null +++ b/controller/debug/admin_controller.go @@ -0,0 +1,355 @@ +package debug + +import ( + "github.com/gogf/gf/v2/net/ghttp" +) + +var DebugController = new(debugCtrl) + +type debugCtrl struct{} + +// DebugPage 调试页面 +func (c *debugCtrl) DebugPage(r *ghttp.Request) { + r.Response.Header().Set("Content-Type", "text/html; charset=utf-8") + r.Response.Write(adminHTML) + r.Exit() +} + +var adminHTML = ` + + + + +数据引擎管理后台 + + + +

📊 数据引擎管理后台

+
+
+
平台管理
+
接口管理
+
+ +
+

平台列表

+
加载中...
+
+
+ +
+

接口列表

+
加载中...
+
+
+
+ + + + + + +` diff --git a/controller/dict/api_datasource_platform_controller.go b/controller/dict/api_datasource_platform_controller.go index 2a26ab7..faeda7c 100644 --- a/controller/dict/api_datasource_platform_controller.go +++ b/controller/dict/api_datasource_platform_controller.go @@ -15,25 +15,25 @@ var DatasourcePlatform = new(datasourcePlatformController) // CreateDatasourcePlatform 创建数据源平台 func (c *datasourcePlatformController) CreateDatasourcePlatform(ctx context.Context, req *dto.CreateDatasourcePlatformReq) (res *dto.CreateDatasourcePlatformRes, err error) { - ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin"}) + ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1}) return service.DatasourcePlatform.Create(ctx, req) } // ListDatasourcePlatforms 获取数据源平台列表 func (c *datasourcePlatformController) ListDatasourcePlatforms(ctx context.Context, req *dto.ListDatasourcePlatformReq) (res *dto.ListDatasourcePlatformRes, err error) { - ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin"}) + ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1}) return service.DatasourcePlatform.List(ctx, req) } // GetDatasourcePlatform 获取数据源平台详情 func (c *datasourcePlatformController) GetDatasourcePlatform(ctx context.Context, req *dto.GetDatasourcePlatformReq) (res *dto.GetDatasourcePlatformRes, err error) { - ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin"}) + ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1}) return service.DatasourcePlatform.GetOne(ctx, req) } // GetPlatformByCode 根据平台编码获取平台信息 func (c *datasourcePlatformController) GetPlatformByCode(ctx context.Context, req *dto.GetPlatformByCodeReq) (res *dto.GetPlatformByCodeRes, err error) { - ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin"}) + ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1}) platform, err := service.DatasourcePlatform.GetByPlatformCode(ctx, req.PlatformCode) if err != nil { return nil, err @@ -45,27 +45,27 @@ func (c *datasourcePlatformController) GetPlatformByCode(ctx context.Context, re // UpdateDatasourcePlatform 更新数据源平台 func (c *datasourcePlatformController) UpdateDatasourcePlatform(ctx context.Context, req *dto.UpdateDatasourcePlatformReq) (res *beans.ResponseEmpty, err error) { - ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin"}) + ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1}) err = service.DatasourcePlatform.Update(ctx, req) return } // UpdateDatasourcePlatformStatus 更新数据源平台状态 func (c *datasourcePlatformController) UpdateDatasourcePlatformStatus(ctx context.Context, req *dto.UpdateDatasourcePlatformStatusReq) (res *beans.ResponseEmpty, err error) { - ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin"}) + ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1}) err = service.DatasourcePlatform.UpdateStatus(ctx, req) return } // DeleteDatasourcePlatform 删除数据源平台 func (c *datasourcePlatformController) DeleteDatasourcePlatform(ctx context.Context, req *dto.DeleteDatasourcePlatformReq) (res *beans.ResponseEmpty, err error) { - ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin"}) + ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1}) err = service.DatasourcePlatform.Delete(ctx, req) return } // GetPlatformStatistics 获取平台统计信息 func (c *datasourcePlatformController) GetPlatformStatistics(ctx context.Context, req *dto.GetPlatformStatisticsReq) (res *dto.GetPlatformStatisticsRes, err error) { - ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin"}) + ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1}) return service.DatasourcePlatform.GetStatistics(ctx) } diff --git a/controller/dict/api_interface_controller.go b/controller/dict/api_interface_controller.go index 6850e60..77deac2 100644 --- a/controller/dict/api_interface_controller.go +++ b/controller/dict/api_interface_controller.go @@ -15,39 +15,39 @@ var ApiInterface = new(apiInterfaceController) // CreateApiInterface 创建接口 func (c *apiInterfaceController) CreateApiInterface(ctx context.Context, req *dto.CreateApiInterfaceReq) (res *dto.CreateApiInterfaceRes, err error) { - ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin"}) + ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1}) return service.ApiInterface.Create(ctx, req) } // ListApiInterface 获取接口列表 func (c *apiInterfaceController) ListApiInterface(ctx context.Context, req *dto.ListApiInterfaceReq) (res *dto.ListApiInterfaceRes, err error) { - ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin"}) + ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1}) return service.ApiInterface.List(ctx, req) } // GetApiInterface 获取接口详情 func (c *apiInterfaceController) GetApiInterface(ctx context.Context, req *dto.GetApiInterfaceReq) (res *dto.GetApiInterfaceRes, err error) { - ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin"}) + ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1}) return service.ApiInterface.GetOne(ctx, req) } // UpdateApiInterface 更新接口 func (c *apiInterfaceController) UpdateApiInterface(ctx context.Context, req *dto.UpdateApiInterfaceReq) (res *beans.ResponseEmpty, err error) { - ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin"}) + ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1}) err = service.ApiInterface.Update(ctx, req) return } // UpdateApiInterfaceStatus 更新接口状态 func (c *apiInterfaceController) UpdateApiInterfaceStatus(ctx context.Context, req *dto.UpdateApiInterfaceStatusReq) (res *beans.ResponseEmpty, err error) { - ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin"}) + ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1}) err = service.ApiInterface.UpdateStatus(ctx, req) return } // DeleteApiInterface 删除接口 func (c *apiInterfaceController) DeleteApiInterface(ctx context.Context, req *dto.DeleteApiInterfaceReq) (res *beans.ResponseEmpty, err error) { - ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin"}) + ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1}) err = service.ApiInterface.Delete(ctx, req) return } diff --git a/main.go b/main.go index 744594b..71fb87a 100644 --- a/main.go +++ b/main.go @@ -1,6 +1,7 @@ package main import ( + "dataengine/controller/debug" "dataengine/controller/dict" syncCtrl "dataengine/controller/sync" syncSvc "dataengine/service/sync" @@ -10,6 +11,7 @@ import ( _ "gitea.com/red-future/common/k3sconfig" _ "github.com/gogf/gf/contrib/drivers/pgsql/v2" _ "github.com/gogf/gf/contrib/nosql/redis/v2" + "github.com/gogf/gf/v2/frame/g" "golang.org/x/net/context" ) @@ -27,5 +29,9 @@ func main() { // 平台同步引擎 syncCtrl.PlatformSyncController, }) + + // 管理后台页面 + g.Server().BindHandler("/admin", debug.DebugController.DebugPage) + select {} } diff --git a/service/sync/api_client.go b/service/sync/api_client.go index 398c0e7..864eed4 100644 --- a/service/sync/api_client.go +++ b/service/sync/api_client.go @@ -3,13 +3,16 @@ package sync import ( "bytes" "context" + "crypto/md5" "crypto/rand" + "encoding/hex" "encoding/json" "fmt" "io" "math/big" "net/http" "net/url" + "sort" "strings" "time" @@ -24,8 +27,9 @@ type ApiResult struct { // ApiClient 通用 API 客户端 type ApiClient struct { - config *PlatformConfig - client *http.Client + config *PlatformConfig + client *http.Client + rateLimiter <-chan time.Time // 限流 ticker } // NewApiClient 创建客户端 @@ -34,10 +38,17 @@ func NewApiClient(config *PlatformConfig) *ApiClient { if config.RequestTimeoutMs > 0 { timeout = time.Duration(config.RequestTimeoutMs) * time.Millisecond } - return &ApiClient{ + ac := &ApiClient{ config: config, client: &http.Client{Timeout: timeout}, } + // 初始化限流 + if config.RateLimitPerMinute > 0 { + interval := time.Minute / time.Duration(config.RateLimitPerMinute) + ac.rateLimiter = time.Tick(interval) + logrus.Infof("限流已启用: %d 次/分钟, 间隔 %v", config.RateLimitPerMinute, interval) + } + return ac } // Get 发送 GET 请求（无参数） @@ -85,6 +96,15 @@ func (c *ApiClient) doRequest(ctx context.Context, method, path string, body int } func (c *ApiClient) execute(ctx context.Context, method, path string, body interface{}, paramsInQuery bool) (*ApiResult, error) { + // 限流等待 + if c.rateLimiter != nil { + select { + case <-c.rateLimiter: + case <-ctx.Done(): + return nil, ctx.Err() + } + } + start := time.Now() fullURL := c.config.GetApiUrl(path) @@ -104,6 +124,8 @@ func (c *ApiClient) execute(ctx context.Context, method, path string, body inter } } + // 计算签名并追加（如快手 API 的 MD5 签名） + fullURL = c.applySignature(fullURL, body, paramsInQuery) logrus.Infof("请求 URL: %s", fullURL) req, err := http.NewRequestWithContext(ctx, method, fullURL, reqBody) @@ -192,6 +214,7 @@ func (c *ApiClient) applyAuthURL(rawURL string) string { for k, v := range eq { val := fmt.Sprintf("%v", v) val = strings.ReplaceAll(val, "{timestamp}", fmt.Sprintf("%d", time.Now().Unix())) + val = strings.ReplaceAll(val, "{timestamp_ms}", fmt.Sprintf("%d", time.Now().UnixMilli())) val = strings.ReplaceAll(val, "{nonce}", generateNonce()) extraParams[k] = val } @@ -250,3 +273,58 @@ func generateNonce() string { r, _ := rand.Int(rand.Reader, big.NewInt(10000)) return fmt.Sprintf("%012d%04d", nanoPart, r.Int64()) } + +// applySignature 计算签名并追加到 URL（支持快手等平台的 MD5 签名） +func (c *ApiClient) applySignature(rawURL string, body interface{}, paramsInQuery bool) string { + cfg := c.config.AuthConfig + if cfg == nil { + return rawURL + } + + signAlgo, _ := cfg["sign_algorithm"].(string) + if signAlgo == "" { + return rawURL + } + appSecret, _ := cfg["app_secret"].(string) + if appSecret == "" && c.config.AppSecret != "" { + appSecret = c.config.AppSecret + } + if appSecret == "" { + return rawURL + } + + parsed, _ := url.Parse(rawURL) + q := parsed.Query() + + // 收集所有参数并按 key 排序 + keys := make([]string, 0, len(q)) + for k := range q { + if k == "sign" { + continue + } + keys = append(keys, k) + } + sort.Strings(keys) + + var signStr string + for _, k := range keys { + signStr += k + "=" + q.Get(k) + "&" + } + signStr += "key=" + appSecret + + var sign string + switch signAlgo { + case "md5": + h := md5.Sum([]byte(signStr)) + sign = hex.EncodeToString(h[:]) + case "md5_upper": + h := md5.Sum([]byte(signStr)) + sign = strings.ToUpper(hex.EncodeToString(h[:])) + default: + return rawURL + } + + q.Set("sign", sign) + parsed.RawQuery = q.Encode() + return parsed.String() +} diff --git a/service/sync/dynamic_sync.go b/service/sync/dynamic_sync.go index 3accc3c..aee26f3 100644 --- a/service/sync/dynamic_sync.go +++ b/service/sync/dynamic_sync.go @@ -17,6 +17,9 @@ import ( "github.com/sirupsen/logrus" ) +// syncRunningMap 防止同一个接口被并发执行同步 +var syncRunningMap sync.Map + // SyncResult 同步结果 type SyncResult struct { TableName string @@ -37,6 +40,14 @@ type PrefetchConfig struct { // SyncByConfig 执行同步 func SyncByConfig(ctx context.Context, platformCode, interfaceCode string, isFullSync bool) (*SyncResult, error) { + // 内存锁：防止同一个接口被并发执行（两个调度周期重叠时跳过） + lockKey := platformCode + "/" + interfaceCode + if _, loaded := syncRunningMap.LoadOrStore(lockKey, true); loaded { + logrus.Warnf("接口 [%s] 正在同步中，跳过重复请求", lockKey) + return nil, fmt.Errorf("接口 [%s] 正在同步中，跳过", lockKey) + } + defer syncRunningMap.Delete(lockKey) + start := time.Now() pm := &PlatformManager{} @@ -87,7 +98,7 @@ func SyncByConfig(ctx context.Context, platformCode, interfaceCode string, isFul if prefetch != nil { return syncWithPrefetch(ctx, api, platform, iface, ifaces, td, prefetch, isFullSync, lastSyncTime, start) } - return syncSingleAPI(ctx, api, platform, iface, td, lastSyncTime, start) + return syncSingleAPI(ctx, api, platform, iface, td, isFullSync, lastSyncTime, start) } // paramsInQuery 判断参数是否应放在 URL 查询字符串中 @@ -104,24 +115,39 @@ func paramsInQuery(iface *entity.ApiInterface) bool { } // syncSingleAPI 单接口分页同步 -func syncSingleAPI(ctx context.Context, api *ApiClient, platform *PlatformConfig, iface *entity.ApiInterface, td *TableDefinition, lastSyncTime int64, start time.Time) (*SyncResult, error) { +func syncSingleAPI(ctx context.Context, api *ApiClient, platform *PlatformConfig, iface *entity.ApiInterface, td *TableDefinition, isFullSync bool, lastSyncTime int64, start time.Time) (*SyncResult, error) { pageSize := GetSyncPageSize(ctx) if ps, ok := iface.RequestConfig["page_size"].(float64); ok { pageSize = int(ps) } + taskType := "incremental" + if isFullSync || lastSyncTime <= 0 { + taskType = "full" + } + inQuery := paramsInQuery(iface) method := string(iface.Method) - body := buildReqBody(iface, 1, pageSize, lastSyncTime, nil) + // 游标分页首次请求需要 cursor=""（通过 extraParams 覆盖 buildReqBody 的 page=1 赋值） + firstExtra := map[string]interface{}{} + if isCursorPagination(iface) { + cp := "cursor" + if p, ok := iface.RequestConfig["page_param"].(string); ok && p != "" { + cp = p + } + firstExtra[cp] = "" + } + body := buildReqBody(iface, 1, pageSize, lastSyncTime, firstExtra) resp, err := api.Request(ctx, method, iface.Url, body, inQuery) if err != nil { - recordFailure(ctx, platform.PlatformCode, iface.Code, err.Error()) + recordFailure(ctx, platform.PlatformCode, iface.Code, taskType, err.Error()) return nil, fmt.Errorf("获取第一页失败: %w", err) } - rows, totalPages, maxTime, err := parseResp(resp.Body, iface.ResponseConfig) + rows, totalPages, maxTime, nextCursor, err := parseRespExt(resp.Body, iface.ResponseConfig) if err != nil { + recordFailure(ctx, platform.PlatformCode, iface.Code, taskType, fmt.Sprintf("解析第一页响应失败: %v", err)) return nil, err } @@ -130,24 +156,68 @@ func syncSingleAPI(ctx context.Context, api *ApiClient, platform *PlatformConfig result.InsertedRows += inserted result.TotalRows += len(rows) - for page := 2; page <= totalPages; page++ { - body := buildReqBody(iface, page, pageSize, lastSyncTime, nil) - resp, err := api.Request(ctx, method, iface.Url, body, inQuery) - if err != nil { - logrus.Errorf("第 %d 页失败: %v", page, err) - continue + // 游标分页 + if isCursorPagination(iface) { + for nextCursor != "" && nextCursor != "nomore" { + cp := "cursor" + if p, ok := iface.RequestConfig["page_param"].(string); ok && p != "" { + cp = p + } + body := buildReqBody(iface, 1, pageSize, lastSyncTime, map[string]interface{}{ + cp: nextCursor, + }) + + resp, err := api.Request(ctx, method, iface.Url, body, inQuery) + if err != nil { + logrus.Errorf("游标 %s 请求失败: %v", nextCursor, err) + recordFailure(ctx, platform.PlatformCode, iface.Code, taskType, fmt.Sprintf("游标 %s 请求失败: %v", nextCursor, err)) + break + } + + rows, _, mt, nc, pe := parseRespExt(resp.Body, iface.ResponseConfig) + if pe != nil { + logrus.Errorf("游标 %s 解析失败: %v", nextCursor, pe) + recordFailure(ctx, platform.PlatformCode, iface.Code, taskType, fmt.Sprintf("游标 %s 解析失败: %v", nextCursor, pe)) + break + } + if len(rows) == 0 { + break + } + nextCursor = nc + + inserted, _ = savePage(ctx, td, rows) + result.InsertedRows += inserted + result.TotalRows += len(rows) + if mt > maxTime { + maxTime = mt + } + result.TotalPages++ + time.Sleep(100 * time.Millisecond) } - rows, _, mt, err := parseResp(resp.Body, iface.ResponseConfig) - if err != nil { - continue + } else { + // 普通分页 + for page := 2; page <= totalPages; page++ { + body := buildReqBody(iface, page, pageSize, lastSyncTime, nil) + resp, err := api.Request(ctx, method, iface.Url, body, inQuery) + if err != nil { + logrus.Errorf("第 %d 页请求失败: %v", page, err) + recordFailure(ctx, platform.PlatformCode, iface.Code, taskType, fmt.Sprintf("第 %d 页请求失败: %v", page, err)) + continue + } + rows, _, mt, _, pe := parseRespExt(resp.Body, iface.ResponseConfig) + if pe != nil { + logrus.Errorf("第 %d 页解析失败: %v", page, pe) + recordFailure(ctx, platform.PlatformCode, iface.Code, taskType, fmt.Sprintf("第 %d 页解析失败: %v", page, pe)) + continue + } + inserted, _ = savePage(ctx, td, rows) + result.InsertedRows += inserted + result.TotalRows += len(rows) + if mt > maxTime { + maxTime = mt + } + time.Sleep(100 * time.Millisecond) } - inserted, _ = savePage(ctx, td, rows) - result.InsertedRows += inserted - result.TotalRows += len(rows) - if mt > maxTime { - maxTime = mt - } - time.Sleep(100 * time.Millisecond) } if maxTime <= 0 { @@ -160,84 +230,150 @@ func syncSingleAPI(ctx context.Context, api *ApiClient, platform *PlatformConfig return result, nil } +func isCursorPagination(iface *entity.ApiInterface) bool { + if iface.RequestConfig == nil { + return false + } + cp, _ := iface.RequestConfig["cursor_pagination"].(bool) + return cp +} + +// collectPrefetchEntities 从 rows 中收集实体和行数据 +func collectPrefetchEntities(rows []map[string]interface{}, prefetch *PrefetchConfig, allEntities *[]interface{}, allRows *[]map[string]interface{}) { + for _, item := range rows { + *allRows = append(*allRows, item) + if prefetch.ValueField == "" { + *allEntities = append(*allEntities, item) + } else if v, ok := item[prefetch.ValueField]; ok { + if f, ok := v.(float64); ok { + *allEntities = append(*allEntities, int64(f)) + } else { + *allEntities = append(*allEntities, v) + } + } + } +} + // syncWithPrefetch 预取模式同步（先分页拉取全部实体列表，再并发处理每个实体） func syncWithPrefetch(ctx context.Context, api *ApiClient, platform *PlatformConfig, iface *entity.ApiInterface, allIfaces []entity.ApiInterface, td *TableDefinition, prefetch *PrefetchConfig, isFullSync bool, lastSyncTime int64, start time.Time) (*SyncResult, error) { logrus.Infof("预取模式: %s -> %s", prefetch.URL, iface.Url) - // 1. 查找匹配 prefetch URL 的接口配置（用于获取正确的请求参数） + taskType := "incremental" + if isFullSync || lastSyncTime <= 0 { + taskType = "full" + } + + // ====== 1. 预取阶段：分页拉取全部实体列表 ====== prefetchIface := findInterfaceByURL(allIfaces, prefetch.URL) - prefetchParams := buildPrefetchParams(iface) + + // 判断预取来源是否游标分页，以及分页参数名 + prefetchIsCursor := false + prefetchPageParam := "page" if prefetchIface != nil && prefetchIface.RequestConfig != nil { - // 使用 prefetch 目标接口的 request_config 重建参数（覆盖默认值） - for k, v := range prefetchIface.RequestConfig { - if k == "headers" || k == "prefetch" || k == "page_param" || - k == "page_size_param" || k == "time_field" || k == "parameters_location" || - k == "filtering" || k == "group_by" || k == "date_range" { - continue - } - prefetchParams[k] = v + if cp, ok := prefetchIface.RequestConfig["cursor_pagination"].(bool); ok { + prefetchIsCursor = cp + } + if p, ok := prefetchIface.RequestConfig["page_param"].(string); ok && p != "" { + prefetchPageParam = p } } - method := strings.ToUpper(prefetch.Method) - inQuery := paramsInQuery(iface) + + prefetchMethod := strings.ToUpper(prefetch.Method) + prefetchPageSize := 100 + if prefetchIface != nil && prefetchIface.RequestConfig != nil { + if ps, ok := prefetchIface.RequestConfig["pageSize"].(float64); ok { + prefetchPageSize = int(ps) + } + } + + // 使用 prefetch 来源接口自己的配置判断参数位置 + var prefetchInQuery bool + if prefetchIface != nil { + prefetchInQuery = paramsInQuery(prefetchIface) + } else { + prefetchInQuery = paramsInQuery(iface) + } + + // prefetch 来源接口的 response_config（用于正确解析列表路径） + var prefetchRespCfg map[string]interface{} + if prefetchIface != nil { + prefetchRespCfg = prefetchIface.ResponseConfig + } allEntities := make([]interface{}, 0) allRows := make([]map[string]interface{}, 0) - prefetchPage := 1 - prefetchTotalPages := 1 - for prefetchPage <= prefetchTotalPages { - params := make(map[string]interface{}) - for k, v := range prefetchParams { - params[k] = v - } - pageParam := "page" - if p, ok := iface.RequestConfig["page_param"].(string); ok { - pageParam = p - } - params[pageParam] = prefetchPage + // 第一页（游标分页首次 cursor=""） + firstExtra := make(map[string]interface{}) + if prefetchIsCursor { + firstExtra[prefetchPageParam] = "" + } + prefetchReqIface := prefetchIface + if prefetchReqIface == nil { + prefetchReqIface = iface + } + body := buildReqBody(prefetchReqIface, 1, prefetchPageSize, lastSyncTime, firstExtra) + resp, err := api.Request(ctx, prefetchMethod, prefetch.URL, body, prefetchInQuery) + if err != nil { + recordFailure(ctx, platform.PlatformCode, iface.Code, taskType, fmt.Sprintf("预取第一页请求失败: %v", err)) + return nil, fmt.Errorf("预取第一页失败: %w", err) + } - resp, err := api.Request(ctx, method, prefetch.URL, params, true) - if err != nil { - return nil, fmt.Errorf("预取第 %d 页失败: %w", prefetchPage, err) - } + rows, prefetchTotalPages, _, nextCursor, err := parseRespExt(resp.Body, prefetchRespCfg) + if err != nil { + recordFailure(ctx, platform.PlatformCode, iface.Code, taskType, fmt.Sprintf("解析预取响应失败: %v", err)) + return nil, fmt.Errorf("解析预取响应失败: %w", err) + } + collectPrefetchEntities(rows, prefetch, &allEntities, &allRows) - entities, _, _, err := parseResp(resp.Body, nil) - if err != nil { - return nil, fmt.Errorf("解析预取第 %d 页响应失败: %w", prefetchPage, err) - } - - // 收集完整数据行（用于存库）和提取的 ID 值（用于遍历） - for _, item := range entities { - allRows = append(allRows, item) - if prefetch.ValueField == "" { - allEntities = append(allEntities, item) - } else if v, ok := item[prefetch.ValueField]; ok { - // 将 float64 转 int64，避免后续 URL 参数中出现科学计数法 - if f, ok := v.(float64); ok { - allEntities = append(allEntities, int64(f)) - } else { - allEntities = append(allEntities, v) - } - } - } - - if prefetchPage == 1 { - if tp := getTotalPages(resp.Body); tp > 0 { - prefetchTotalPages = tp - } else { + // 分页循环 + if prefetchIsCursor { + // 游标分页 + for nextCursor != "" && nextCursor != "nomore" { + body := buildReqBody(prefetchReqIface, 1, prefetchPageSize, lastSyncTime, map[string]interface{}{ + prefetchPageParam: nextCursor, + }) + resp, err := api.Request(ctx, prefetchMethod, prefetch.URL, body, prefetchInQuery) + if err != nil { + logrus.Errorf("预取游标 %s 请求失败: %v", nextCursor, err) break } + rows, _, _, nc, pe := parseRespExt(resp.Body, prefetchRespCfg) + if pe != nil { + logrus.Errorf("预取游标 %s 解析失败: %v", nextCursor, pe) + break + } + if len(rows) == 0 { + break + } + nextCursor = nc + collectPrefetchEntities(rows, prefetch, &allEntities, &allRows) + time.Sleep(100 * time.Millisecond) + } + } else { + // 普通分页 + for page := 2; page <= prefetchTotalPages; page++ { + body := buildReqBody(prefetchReqIface, page, prefetchPageSize, lastSyncTime, nil) + resp, err := api.Request(ctx, prefetchMethod, prefetch.URL, body, prefetchInQuery) + if err != nil { + logrus.Errorf("预取第 %d 页请求失败: %v", page, err) + continue + } + rows, _, _, _, pe := parseRespExt(resp.Body, prefetchRespCfg) + if pe != nil { + logrus.Errorf("预取第 %d 页解析失败: %v", page, pe) + continue + } + collectPrefetchEntities(rows, prefetch, &allEntities, &allRows) + time.Sleep(100 * time.Millisecond) } - prefetchPage++ - time.Sleep(50 * time.Millisecond) } if len(allEntities) == 0 { logrus.Warn("预取结果为空列表，跳过同步") return &SyncResult{TableName: td.TableName, Duration: fmt.Sprintf("%.1fs", time.Since(start).Seconds())}, nil } - logrus.Infof("预取到 %d 个实体（共 %d 页）", len(allEntities), prefetchPage-1) + logrus.Infof("预取到 %d 个实体", len(allEntities)) // 2. 将预取的数据也存入库（如账户列表存入 tencent_account_relation） if prefetchIface != nil && prefetchIface.TableDefinition != nil { @@ -258,6 +394,7 @@ func syncWithPrefetch(ctx context.Context, api *ApiClient, platform *PlatformCon } dataMethod := string(iface.Method) + inQuery := paramsInQuery(iface) concurrency := GetSyncConcurrency(ctx) var mu sync.Mutex @@ -346,21 +483,24 @@ func syncWithPrefetch(ctx context.Context, api *ApiClient, platform *PlatformCon // getTotalPages 从响应中提取总页数 func getTotalPages(raw []byte) int { - var r struct { - Data map[string]interface{} `json:"data"` - } - if err := json.Unmarshal(raw, &r); err != nil { + rows, tp, _, _, err := parseRespExt(raw, nil) + if err != nil || len(rows) == 0 { return 0 } - if r.Data == nil { - return 0 + return tp +} + +func toFloat64(v interface{}) (float64, bool) { + switch val := v.(type) { + case float64: + return val, true + case int: + return float64(val), true + case int64: + return float64(val), true + default: + return 0, false } - if pi, ok := r.Data["page_info"].(map[string]interface{}); ok { - if tp, ok := pi["total_page"].(float64); ok { - return int(tp) - } - } - return 0 } // buildPrefetchParams 构建预取接口的请求参数 @@ -382,7 +522,9 @@ func buildPrefetchParams(iface *entity.ApiInterface) map[string]interface{} { for k, v := range iface.RequestConfig { if k == "headers" || k == "prefetch" || k == "page_param" || k == "page_size_param" || k == "time_field" || k == "parameters_location" || - k == "filtering" || k == "group_by" || k == "date_range" { + k == "filtering" || k == "group_by" || k == "date_range" || + k == "body_wrapper_field" || k == "exclude_from_wrapper" || + k == "cursor_pagination" || k == "time_field_mode" { continue } if k == pageParam || k == psParam { @@ -467,7 +609,10 @@ func buildReqBody(iface *entity.ApiInterface, page, pageSize int, lastSyncTime i if iface.RequestConfig != nil { for k, v := range iface.RequestConfig { if k == "time_field" || k == "headers" || k == "prefetch" || - k == "page_param" || k == "page_size_param" || k == "parameters_location" { + k == "page_param" || k == "page_size_param" || k == "parameters_location" || + k == "cursor_pagination" || k == "time_field_mode" || + k == "body_wrapper_field" || k == "exclude_from_wrapper" || + k == "top_level_params" { continue } body[k] = v @@ -485,16 +630,33 @@ func buildReqBody(iface *entity.ApiInterface, page, pageSize int, lastSyncTime i } body[pageParam] = page body[psParam] = pageSize - // 增量同步：将 time_field 转为 API 期望的 filtering 格式 - // 如 filtering=[{"field":"last_modified_time","operator":"GREATER_EQUALS","values":["1780037982"]}] - if lastSyncTime > 0 { - if tf, ok := iface.RequestConfig["time_field"].(string); ok && tf != "" { + + // 时间过滤处理：支持两种模式 + // 1. "filtering" 模式（默认）：生成 filtering=[{"field":"...","operator":"GREATER_EQUALS","values":["..."]}]（腾讯） + // 2. "range" 模式：生成 beginTime/endTime + queryType（快手） + if tf, ok := iface.RequestConfig["time_field"].(string); ok && tf != "" { + timeMode := "filtering" + if tm, ok := iface.RequestConfig["time_field_mode"].(string); ok && tm != "" { + timeMode = tm + } + + if timeMode == "range" { + // 快手模式：beginTime/endTime（毫秒时间戳） + timeMs := lastSyncTime + if timeMs <= 0 { + // 全量：默认90天前 + timeMs = time.Now().Add(-90 * 24 * time.Hour).UnixMilli() + } + body["queryType"] = 2 + body["beginTime"] = timeMs + body["endTime"] = time.Now().UnixMilli() + } else if lastSyncTime > 0 { + // 腾讯 filtering 模式（仅增量时） timeFilter := map[string]interface{}{ "field": tf, "operator": "GREATER_EQUALS", "values": []interface{}{fmt.Sprintf("%d", lastSyncTime)}, } - // 合并已有的 filtering（如果 request_config 中已定义其他过滤条件） if existing, ok := body["filtering"].([]interface{}); ok { body["filtering"] = append(existing, timeFilter) } else { @@ -502,64 +664,197 @@ func buildReqBody(iface *entity.ApiInterface, page, pageSize int, lastSyncTime i } } } + for k, v := range extraParams { body[k] = v } + + // body_wrapper_field 支持：将业务参数包装到指定字段（如快手 API 的 param JSON） + if iface.RequestConfig != nil { + if wf, ok := iface.RequestConfig["body_wrapper_field"].(string); ok && wf != "" { + excludeSet := map[string]bool{"method": true, "version": true, "signMethod": true} + if excl, ok := iface.RequestConfig["exclude_from_wrapper"].([]interface{}); ok { + for _, v := range excl { + if s, ok := v.(string); ok { + excludeSet[s] = true + } + } + } + wrapperObj := make(map[string]interface{}) + for k, v := range body { + if !excludeSet[k] && k != wf { + wrapperObj[k] = v + delete(body, k) + } + } + b, _ := json.Marshal(wrapperObj) + body[wf] = string(b) + } + } + return body } -// parseResp 解析同步接口返回值 -func parseResp(raw []byte, responseConfig map[string]interface{}) ([]map[string]interface{}, int, int64, error) { - var r struct { - Code int `json:"code"` - Message string `json:"message"` - Data map[string]interface{} `json:"data"` +// parseRespExt 解析响应，支持自定义成功判断和数据路径 +func parseRespExt(raw []byte, rc map[string]interface{}) ([]map[string]interface{}, int, int64, string, error) { + var respMap map[string]interface{} + if err := json.Unmarshal(raw, &respMap); err != nil { + return nil, 0, 0, "", fmt.Errorf("JSON解析失败: %w", err) } - if err := json.Unmarshal(raw, &r); err != nil { - return nil, 0, 0, fmt.Errorf("解析响应失败: %w", err) - } - if r.Code != 0 { - return nil, 0, 0, fmt.Errorf("API错误: code=%d, message=%s", r.Code, r.Message) - } - - var rows []map[string]interface{} - totalPages := 1 - maxTime := int64(0) - - var listData []interface{} - if lp, ok := r.Data["list"]; ok { - listData, _ = lp.([]interface{}) - } else if lp, ok := r.Data["data"]; ok { - if m, ok := lp.(map[string]interface{}); ok { - if l, ok := m["list"].([]interface{}); ok { - listData = l + successField, successVal := "code", float64(0) + msgField, listPath, cursorPath := "message", "data", "" + singleRecord := false + if rc != nil { + if sf, _ := rc["success_field"].(string); sf != "" { + successField = sf + } + if sv, ok := rc["success_value"]; ok { + if f, ok := toFloat64(sv); ok { + successVal = f } } + if mf, _ := rc["message_field"].(string); mf != "" { + msgField = mf + } + if lp, _ := rc["list_path"].(string); lp != "" { + listPath = lp + } + if cf, _ := rc["cursor_field"].(string); cf != "" { + cursorPath = cf + } + if sr, _ := rc["single_record"].(bool); sr { + singleRecord = true + } + } + if v, ok := respMap[successField]; ok { + actual, _ := toFloat64(v) + if actual != successVal { + msg, _ := respMap[msgField].(string) + return nil, 0, 0, "", fmt.Errorf("API错误: %s=%v, %s=%s", successField, v, msgField, msg) + } } + // 解析 list_path，支持最后一段是数组的情况（如 data.orderList） + var listData []interface{} + var dataContainer map[string]interface{} + if listPath != "" { + parts := strings.Split(listPath, ".") + cur := respMap + for i, p := range parts { + if i == len(parts)-1 { + // 最后一段：可能直接是数组，也可能是包含 list/orderList 的 map + listData, _ = cur[p].([]interface{}) + if listData == nil { + if m, ok := cur[p].(map[string]interface{}); ok { + dataContainer = m + if l, ok := m["list"]; ok { + listData, _ = l.([]interface{}) + } + if listData == nil { + if ol, ok := m["orderList"]; ok { + listData, _ = ol.([]interface{}) + } + } + } + } else { + dataContainer = cur + } + } else { + next, ok := cur[p].(map[string]interface{}) + if !ok { + return nil, 0, 0, "", fmt.Errorf("路径 %s 在 %s 处中断", listPath, p) + } + cur = next + } + } + } + if listData == nil { + if singleRecord && listPath != "" { + // 详情接口：list_path 指向单个对象，包装为单元素数组 + parts := strings.Split(listPath, ".") + cur := respMap + ok := true + for _, p := range parts { + if m, exists := cur[p].(map[string]interface{}); exists { + cur = m + } else { + ok = false + break + } + } + if ok { + listData = []interface{}{cur} + dataContainer = cur + } + } + } + if listData == nil { + // 回退到根级字段 + listData, _ = respMap["list"].([]interface{}) + if listData == nil { + listData, _ = respMap["orderList"].([]interface{}) + } + dataContainer = respMap + } + var rows []map[string]interface{} + totalPages, maxTime := 1, int64(0) for _, item := range listData { if m, ok := item.(map[string]interface{}); ok { + // 展平嵌套 map：将子 map 的字段合并到顶层（如 orderBaseInfo.oid → oid） + flat := flattenRow(m) j, _ := json.Marshal(m) - m["raw_data"] = string(j) - if t, ok := m["last_modified_time"].(float64); ok && int64(t) > maxTime { - maxTime = int64(t) + flat["raw_data"] = string(j) + for _, tf := range []string{"last_modified_time", "created_time", "update_time", "createTime", "updateTime", "lastModifiedTime"} { + if t, ok := flat[tf].(float64); ok && int64(t) > maxTime { + maxTime = int64(t) + } } - if t, ok := m["created_time"].(float64); ok && int64(t) > maxTime { - maxTime = int64(t) - } - rows = append(rows, m) + rows = append(rows, flat) } } - - if pi, ok := r.Data["page_info"].(map[string]interface{}); ok { + nextCursor := "" + if cursorPath != "" { + cp := strings.Split(cursorPath, ".") + cc := respMap + for i, p := range cp { + if i == len(cp)-1 { + if s, ok := cc[p].(string); ok { + nextCursor = s + } + } else if m, ok := cc[p].(map[string]interface{}); ok { + cc = m + } + } + } + if pi, ok := dataContainer["page_info"].(map[string]interface{}); ok { if tp, ok := pi["total_page"].(float64); ok { totalPages = int(tp) - } else if tp, ok := pi["total_page"].(int); ok { - totalPages = tp } } + return rows, totalPages, maxTime, nextCursor, nil +} - return rows, totalPages, maxTime, nil +// flattenRow 展平嵌套 map：将子 map 的字段递归合并到顶层 +// 数组类型的字段保持原样不展平 +func flattenRow(m map[string]interface{}) map[string]interface{} { + result := make(map[string]interface{}, len(m)) + for k, v := range m { + if sub, ok := v.(map[string]interface{}); ok { + // 子 map 递归展平后合并到顶层 + for sk, sv := range flattenRow(sub) { + result[sk] = sv + } + } else { + result[k] = v + } + } + return result +} + +// parseResp 兼容旧版，保持4个返回值 +func parseResp(raw []byte, responseConfig map[string]interface{}) ([]map[string]interface{}, int, int64, error) { + rows, tp, mt, _, err := parseRespExt(raw, responseConfig) + return rows, tp, mt, err } func savePage(ctx context.Context, td *TableDefinition, rows []map[string]interface{}) (int, error) { @@ -631,18 +926,17 @@ func updateSyncTime(ctx context.Context, platformCode, interfaceCode string, t i Save() } -func recordFailure(ctx context.Context, platformCode, interfaceCode, errMsg string) { +func recordFailure(ctx context.Context, platformCode, interfaceCode, taskType, errMsg string) { dao.SyncTaskLog.Create(ctx, &taskDto.CreateSyncTaskLogReq{ TaskID: fmt.Sprintf("%s_%s_%d", platformCode, interfaceCode, time.Now().UnixNano()), - TaskType: fmt.Sprintf("%s_%s", platformCode, interfaceCode), + TaskType: taskType, PlatformCode: platformCode, InterfaceCode: interfaceCode, Status: "failed", MaxRetry: 3, + StartTime: time.Now(), RequestParams: map[string]interface{}{ - "platform_code": platformCode, - "interface_code": interfaceCode, - "error": errMsg, + "error": errMsg, }, }) } diff --git a/service/sync/platform_manager.go b/service/sync/platform_manager.go index 86f171d..8f362f7 100644 --- a/service/sync/platform_manager.go +++ b/service/sync/platform_manager.go @@ -15,6 +15,8 @@ import ( type PlatformConfig struct { *entity.DatasourcePlatform AccessToken string + AppKey string + AppSecret string } // GetApiUrl 拼接完整 API URL @@ -52,6 +54,15 @@ func (m *PlatformManager) GetPlatform(ctx context.Context, platformCode string) } case "API_KEY": cfg.AccessToken = platform.ApiKey + case "SIGN": + if platform.AuthConfig != nil { + if ak, _ := platform.AuthConfig["app_key"].(string); ak != "" { + cfg.AppKey = ak + } + if as, _ := platform.AuthConfig["app_secret"].(string); as != "" { + cfg.AppSecret = as + } + } default: logrus.Warnf("平台 %s 认证类型 %s 未处理", platformCode, platform.AuthType) } diff --git a/service/sync/sync_scheduler.go b/service/sync/sync_scheduler.go index a65ccad..be9890c 100644 --- a/service/sync/sync_scheduler.go +++ b/service/sync/sync_scheduler.go @@ -12,26 +12,15 @@ import ( "github.com/sirupsen/logrus" ) -// StartAutoSync 启动自动同步（独立 goroutine，启动后自动循环执行） +// StartAutoSync 启动自动同步（独立 goroutine，每次完成后等待 interval 再执行下一次） func StartAutoSync(ctx context.Context) { interval := GetSyncInterval(ctx) - logrus.Infof("自动同步调度器启动，间隔: %d 分钟", interval) - - // 首次执行：根据 sync_tracker 是否有记录自动判断全量/增量 - // 无记录 → 全量，有记录 → 增量 - runAutoSync(ctx) - - ticker := time.NewTicker(time.Duration(interval) * time.Minute) - defer ticker.Stop() + logrus.Infof("自动同步调度器启动，间隔: %d 分钟（完成一次后开始计时）", interval) for { - select { - case <-ticker.C: - runAutoSync(ctx) - case <-ctx.Done(): - logrus.Info("自动同步调度器已停止") - return - } + runAutoSync(ctx) + logrus.Infof("自动同步完成，等待 %d 分钟后执行下一次", interval) + time.Sleep(time.Duration(interval) * time.Minute) } } diff --git a/service/sync/table_manager.go b/service/sync/table_manager.go index cb70f4d..d28c3e7 100644 --- a/service/sync/table_manager.go +++ b/service/sync/table_manager.go @@ -86,18 +86,29 @@ func buildCreateSQL(td *TableDefinition) string { } cols = append(cols, "raw_data JSONB DEFAULT '{}'") - // 添加复合唯一索引（用于 ON CONFLICT upsert，所有 conflict_keys 作为一个复合索引） + // 添加复合唯一索引（用于 ON CONFLICT upsert） var constraints []string if len(td.ConflictKeys) > 0 { - cols := strings.Join(td.ConflictKeys, ", ") + ck := strings.Join(td.ConflictKeys, ", ") indexName := fmt.Sprintf("uq_%s_conflict", td.TableName) constraints = append(constraints, - fmt.Sprintf("CREATE UNIQUE INDEX IF NOT EXISTS %s ON %s (%s)", indexName, td.TableName, cols)) + fmt.Sprintf("CREATE UNIQUE INDEX IF NOT EXISTS %s ON %s (%s)", indexName, td.TableName, ck)) } sql := fmt.Sprintf("CREATE TABLE IF NOT EXISTS %s (\n %s\n);\n", td.TableName, strings.Join(cols, ",\n ")) + + // 添加唯一索引 if len(constraints) > 0 { - sql += strings.Join(constraints, ";\n") + ";" + sql += strings.Join(constraints, ";\n") + ";\n" } + + // 添加字段注释（COMMENT ON COLUMN） + for _, c := range td.Columns { + if c.Comment != "" { + escaped := strings.ReplaceAll(c.Comment, "'", "''") + sql += fmt.Sprintf("COMMENT ON COLUMN %s.%s IS '%s';\n", td.TableName, c.Name, escaped) + } + } + return sql } diff --git a/sql/seed_data.sql b/sql/seed_data.sql index c600cf6..2b0dbba 100644 --- a/sql/seed_data.sql +++ b/sql/seed_data.sql @@ -40,13 +40,15 @@ INSERT INTO api_datasource_platform ( -- ============================================= -- 2.1 账户关系接口（先获取所有账户ID） +-- 出参：{"code":0, "message":"ok", "data":{"list":[...], "page_info":{...}}} +-- 列表路径：data.list，分页：data.page_info.total_page INSERT INTO api_interface ( tenant_id, creator, created_at, updater, updated_at, platform_id, name, code, url, method, status, auth_type, - request_config, table_definition + request_config, response_config, table_definition ) VALUES ( 1, 'admin', NOW(), 'admin', NOW(), - 1, + (SELECT id FROM api_datasource_platform WHERE platform_code = 'tencent'), '账户列表', 'account_relation', '/advertiser/get', 'GET', 'active', 'inherit', '{ @@ -58,6 +60,12 @@ INSERT INTO api_interface ( "pagination_mode": "PAGINATION_MODE_NORMAL", "fields": ["account_id", "corporation_name", "is_adx", "is_bid", "is_mp"] }'::jsonb, + '{ + "success_field": "code", + "success_value": 0, + "message_field": "message", + "list_path": "data.list" + }'::jsonb, '{ "table_name": "tencent_account_relation", "columns": [ @@ -72,13 +80,15 @@ INSERT INTO api_interface ( ); -- 2.2 图片素材接口（遍历每个账户拉取图片） +-- 出参：{"code":0, "message":"ok", "data":{"list":[...], "page_info":{...}}} +-- 列表路径：data.list，分页：data.page_info.total_page INSERT INTO api_interface ( tenant_id, creator, created_at, updater, updated_at, platform_id, name, code, url, method, status, auth_type, - request_config, table_definition + request_config, response_config, table_definition ) VALUES ( 1, 'admin', NOW(), 'admin', NOW(), - 1, + (SELECT id FROM api_datasource_platform WHERE platform_code = 'tencent'), '图片素材', 'image', '/images/get', 'GET', 'active', 'inherit', '{ @@ -96,6 +106,12 @@ INSERT INTO api_interface ( "value_field": "account_id" } }'::jsonb, + '{ + "success_field": "code", + "success_value": 0, + "message_field": "message", + "list_path": "data.list" + }'::jsonb, '{ "table_name": "tencent_image", "columns": [ @@ -120,7 +136,7 @@ INSERT INTO api_interface ( {"name": "quality_status", "type": "VARCHAR(50)", "comment": "质量状态"}, {"name": "aigc_flag", "type": "VARCHAR(50)", "comment": "AIGC标志"}, {"name": "aigc_type", "type": "INT", "comment": "AIGC类型"}, - {"name": "verify_status", "type": "VARCHAR(50) DEFAULT 'PENDING'", "comment": "校验状态"}, + {"name": "verify_status", "type": "VARCHAR(50)", "comment": "校验状态"}, {"name": "verified_at", "type": "TIMESTAMP WITH TIME ZONE", "comment": "校验时间"}, {"name": "verified_by", "type": "VARCHAR(64)", "comment": "校验人"} ], @@ -129,13 +145,15 @@ INSERT INTO api_interface ( ); -- 2.3 视频素材接口（遍历每个账户拉取视频） +-- 出参：{"code":0, "message":"ok", "data":{"list":[...], "page_info":{...}}} +-- 列表路径：data.list，分页：data.page_info.total_page INSERT INTO api_interface ( tenant_id, creator, created_at, updater, updated_at, platform_id, name, code, url, method, status, auth_type, - request_config, table_definition + request_config, response_config, table_definition ) VALUES ( 1, 'admin', NOW(), 'admin', NOW(), - 1, + (SELECT id FROM api_datasource_platform WHERE platform_code = 'tencent'), '视频素材', 'video', '/videos/get', 'GET', 'active', 'inherit', '{ @@ -153,6 +171,12 @@ INSERT INTO api_interface ( "value_field": "account_id" } }'::jsonb, + '{ + "success_field": "code", + "success_value": 0, + "message_field": "message", + "list_path": "data.list" + }'::jsonb, '{ "table_name": "tencent_video", "columns": [ @@ -182,7 +206,7 @@ INSERT INTO api_interface ( {"name": "quality_status", "type": "VARCHAR(50)", "comment": "质量状态"}, {"name": "aigc_flag", "type": "VARCHAR(50)", "comment": "AIGC标志"}, {"name": "muse_aigc_version", "type": "INT", "comment": "Muse AIGC版本"}, - {"name": "verify_status", "type": "VARCHAR(50) DEFAULT 'PENDING'", "comment": "校验状态"}, + {"name": "verify_status", "type": "VARCHAR(50)", "comment": "校验状态"}, {"name": "verified_at", "type": "TIMESTAMP WITH TIME ZONE", "comment": "校验时间"}, {"name": "verified_by", "type": "VARCHAR(64)", "comment": "校验人"} ], @@ -192,13 +216,15 @@ INSERT INTO api_interface ( -- 2.4 音频素材接口（POST + JSON Body，无需遍历账户） -- 注意：此接口不依赖 account_id，不依赖 prefetch，不支持增量 +-- 出参：{"code":0, "message":"ok", "data":{"list":[...], "page_info":{...}}} +-- 列表路径：data.list，分页：data.page_info.total_page INSERT INTO api_interface ( tenant_id, creator, created_at, updater, updated_at, platform_id, name, code, url, method, status, auth_type, - request_config, table_definition + request_config, response_config, table_definition ) VALUES ( 1, 'admin', NOW(), 'admin', NOW(), - 1, + (SELECT id FROM api_datasource_platform WHERE platform_code = 'tencent'), '音频素材', 'audio', '/muse_audios/get', 'POST', 'active', 'inherit', '{ @@ -208,6 +234,12 @@ INSERT INTO api_interface ( "page_size_param": "page_size", "fields": ["audio_id", "cover_image_url", "audio_name", "author", "duration", "expire_time", "feel_tags", "genre_tags"] }'::jsonb, + '{ + "success_field": "code", + "success_value": 0, + "message_field": "message", + "list_path": "data.list" + }'::jsonb, '{ "table_name": "tencent_audio", "columns": [ @@ -219,7 +251,7 @@ INSERT INTO api_interface ( {"name": "expire_time", "type": "BIGINT", "comment": "过期时间戳"}, {"name": "feel_tags", "type": "JSONB", "comment": "情感标签"}, {"name": "genre_tags", "type": "JSONB", "comment": "风格标签"}, - {"name": "verify_status", "type": "VARCHAR(50) DEFAULT 'PENDING'", "comment": "校验状态"}, + {"name": "verify_status", "type": "VARCHAR(50)", "comment": "校验状态"}, {"name": "verified_at", "type": "TIMESTAMP WITH TIME ZONE", "comment": "校验时间"}, {"name": "verified_by", "type": "VARCHAR(64)", "comment": "校验人"} ], diff --git a/sql/seed_data_kuaishou.sql b/sql/seed_data_kuaishou.sql new file mode 100644 index 0000000..cc4fa64 --- /dev/null +++ b/sql/seed_data_kuaishou.sql @@ -0,0 +1,60 @@ +-- ============================================= +-- 快手电商平台初始化数据（18个接口） +-- ============================================= + +INSERT INTO api_datasource_platform (tenant_id,creator,created_at,updater,updated_at,platform_code,platform_name,description,status,api_base_url,auth_type,auth_config,rate_limit_per_minute,rate_limit_per_hour,concurrency_limit,request_timeout_ms,max_retries,retry_delay_ms) VALUES +(1,'admin',NOW(),'admin',NOW(),'kuaishou','快手电商','快手电商开放平台数据同步','ACTIVE','https://openapi.kwaixiaodian.com','API_KEY','{"sign_algorithm":"md5_upper","app_key":"YOUR_APP_KEY","app_secret":"YOUR_APP_SECRET","token_in_query":true,"query_key":"access_token","extra_query_params":{"timestamp":"{timestamp_ms}","signMethod":"MD5"}}'::jsonb,100,3600,5,30000,3,1000); + +-- 2. 订单列表（游标分页，增量range） +INSERT INTO api_interface (tenant_id,creator,created_at,updater,updated_at,platform_id,name,code,url,method,status,auth_type,request_config,response_config,table_definition) VALUES (1,'admin',NOW(),'admin',NOW(),(SELECT id FROM api_datasource_platform WHERE platform_code='kuaishou'),'订单列表','order_list','/open/order/cursor/list','GET','active','inherit','{"parameters_location":"query","page_param":"cursor","page_size_param":"pageSize","cursor_pagination":true,"method":"open.order.cursor.list","version":1,"signMethod":"MD5","orderViewStatus":1,"pageSize":50,"sort":1,"queryType":2,"time_field":"updateTime","time_field_mode":"range","cpsType":0,"body_wrapper_field":"param","exclude_from_wrapper":["method","version","signMethod"]}'::jsonb,'{"success_field":"result","success_value":1,"list_path":"data.orderList","cursor_field":"data.cursor","cursor_end_marker":"nomore"}'::jsonb,'{"table_name":"kuaishou_order_list","columns":[{"name":"oid","type":"BIGINT","comment":"订单ID"},{"name":"status","type":"INT","comment":"订单状态码"},{"name":"createTime","type":"BIGINT","comment":"创建时间"},{"name":"updateTime","type":"BIGINT","comment":"更新时间"},{"name":"payTime","type":"BIGINT","comment":"支付时间"},{"name":"sendTime","type":"BIGINT","comment":"发货时间"},{"name":"recvTime","type":"BIGINT","comment":"收货时间"},{"name":"refundTime","type":"BIGINT","comment":"退款时间"},{"name":"totalFee","type":"BIGINT","comment":"总金额(分)"},{"name":"expressFee","type":"BIGINT","comment":"运费(分)"},{"name":"discountFee","type":"BIGINT","comment":"优惠金额(分)"},{"name":"originalPrice","type":"BIGINT","comment":"原价(分)"},{"name":"buyerOpenId","type":"VARCHAR(100)","comment":"买家ID"},{"name":"sellerOpenId","type":"VARCHAR(100)","comment":"卖家ID"},{"name":"buyerNick","type":"VARCHAR(100)","comment":"买家昵称"},{"name":"sellerNick","type":"VARCHAR(100)","comment":"卖家昵称"},{"name":"remark","type":"TEXT","comment":"买家留言"},{"name":"itemTitle","type":"VARCHAR(300)","comment":"商品标题"},{"name":"num","type":"INT","comment":"商品数量"},{"name":"price","type":"BIGINT","comment":"商品单价(分)"},{"name":"activityType","type":"INT","comment":"活动类型"},{"name":"cpsType","type":"INT","comment":"分销类型"},{"name":"payType","type":"INT","comment":"支付类型"},{"name":"payChannel","type":"VARCHAR(50)","comment":"支付渠道"},{"name":"channel","type":"VARCHAR(50)","comment":"分销渠道"},{"name":"commentStatus","type":"INT","comment":"评价状态"},{"name":"priorityDelivery","type":"BOOLEAN","comment":"是否优先发货"},{"name":"carrierType","type":"INT","comment":"承运商类型"},{"name":"carrierId","type":"BIGINT","comment":"承运商ID"},{"name":"province","type":"VARCHAR(100)","comment":"省份"},{"name":"city","type":"VARCHAR(100)","comment":"城市"},{"name":"district","type":"VARCHAR(100)","comment":"区县"},{"name":"provinceCode","type":"INT","comment":"省份编码"},{"name":"cityCode","type":"INT","comment":"城市编码"},{"name":"districtCode","type":"INT","comment":"区县编码"}],"conflict_keys":["oid"]}'::jsonb); + +-- 3. 订单详情（prefetch→order_list） +INSERT INTO api_interface (tenant_id,creator,created_at,updater,updated_at,platform_id,name,code,url,method,status,auth_type,request_config,response_config,table_definition) VALUES (1,'admin',NOW(),'admin',NOW(),(SELECT id FROM api_datasource_platform WHERE platform_code='kuaishou'),'订单详情','order_detail','/open/order/detail','GET','active','inherit','{"parameters_location":"query","pageSize":10,"page_param":"cursor","page_size_param":"pageSize","method":"open.order.detail","version":1,"signMethod":"MD5","body_wrapper_field":"param","exclude_from_wrapper":["method","version","signMethod"],"prefetch":{"url":"/open/order/cursor/list","method":"GET","response_path":"data.orderList","target_param":"oid","value_field":"oid"}}'::jsonb,'{"success_field":"result","success_value":1,"list_path":"data","single_record":true}'::jsonb,'{"table_name":"kuaishou_order_detail","columns":[{"name":"oid","type":"BIGINT","comment":"订单ID"},{"name":"status","type":"INT","comment":"订单状态码"},{"name":"createTime","type":"BIGINT","comment":"创建时间"},{"name":"updateTime","type":"BIGINT","comment":"更新时间"},{"name":"payTime","type":"BIGINT","comment":"支付时间"},{"name":"sendTime","type":"BIGINT","comment":"发货时间"},{"name":"recvTime","type":"BIGINT","comment":"收货时间"},{"name":"refundTime","type":"BIGINT","comment":"退款时间"},{"name":"totalFee","type":"BIGINT","comment":"总金额(分)"},{"name":"expressFee","type":"BIGINT","comment":"运费(分)"},{"name":"discountFee","type":"BIGINT","comment":"优惠金额(分)"},{"name":"originalPrice","type":"BIGINT","comment":"原价(分)"},{"name":"buyerOpenId","type":"VARCHAR(100)","comment":"买家ID"},{"name":"sellerOpenId","type":"VARCHAR(100)","comment":"卖家ID"},{"name":"buyerNick","type":"VARCHAR(100)","comment":"买家昵称"},{"name":"sellerNick","type":"VARCHAR(100)","comment":"卖家昵称"},{"name":"remark","type":"TEXT","comment":"买家留言"},{"name":"itemTitle","type":"VARCHAR(300)","comment":"商品标题"},{"name":"num","type":"INT","comment":"商品数量"},{"name":"price","type":"BIGINT","comment":"商品单价(分)"},{"name":"activityType","type":"INT","comment":"活动类型"},{"name":"cpsType","type":"INT","comment":"分销类型"},{"name":"payType","type":"INT","comment":"支付类型"},{"name":"payChannel","type":"VARCHAR(50)","comment":"支付渠道"},{"name":"channel","type":"VARCHAR(50)","comment":"分销渠道"},{"name":"commentStatus","type":"INT","comment":"评价状态"},{"name":"priorityDelivery","type":"BOOLEAN","comment":"是否优先发货"},{"name":"carrierType","type":"INT","comment":"承运商类型"},{"name":"carrierId","type":"BIGINT","comment":"承运商ID"},{"name":"province","type":"VARCHAR(100)","comment":"省份"},{"name":"city","type":"VARCHAR(100)","comment":"城市"},{"name":"district","type":"VARCHAR(100)","comment":"区县"},{"name":"provinceCode","type":"INT","comment":"省份编码"},{"name":"cityCode","type":"INT","comment":"城市编码"},{"name":"districtCode","type":"INT","comment":"区县编码"}],"conflict_keys":["oid"]}'::jsonb); + +-- 4. 商品列表（普通分页，全量） +INSERT INTO api_interface (tenant_id,creator,created_at,updater,updated_at,platform_id,name,code,url,method,status,auth_type,request_config,response_config,table_definition) VALUES (1,'admin',NOW(),'admin',NOW(),(SELECT id FROM api_datasource_platform WHERE platform_code='kuaishou'),'商品列表','item_list','/open/item/list/get','GET','active','inherit','{"parameters_location":"query","pageSize":20,"page_param":"pageNumber","page_size_param":"pageSize","method":"open.item.list.get","version":1,"signMethod":"MD5","itemStatus":1,"itemType":1,"onOfflineStatus":1,"body_wrapper_field":"param","exclude_from_wrapper":["method","version","signMethod"]}'::jsonb,'{"success_field":"result","success_value":1,"list_path":"data.items"}'::jsonb,'{"table_name":"kuaishou_item_list","columns":[{"name":"kwaiItemId","type":"BIGINT","comment":"商品ID"},{"name":"relItemId","type":"BIGINT","comment":"外部商品ID"},{"name":"title","type":"VARCHAR(300)","comment":"商品标题"},{"name":"details","type":"TEXT","comment":"商品详情描述"},{"name":"categoryId","type":"INT","comment":"类目ID"},{"name":"categoryName","type":"VARCHAR(200)","comment":"类目名称"},{"name":"price","type":"BIGINT","comment":"商品价格"},{"name":"volume","type":"INT","comment":"销量"},{"name":"status","type":"INT","comment":"商品状态"},{"name":"auditStatus","type":"INT","comment":"审核状态"},{"name":"auditReason","type":"VARCHAR(500)","comment":"审核原因"},{"name":"shelfStatus","type":"INT","comment":"上下架状态"},{"name":"itemType","type":"INT","comment":"商品类型"},{"name":"createTime","type":"BIGINT","comment":"创建时间"},{"name":"updateTime","type":"BIGINT","comment":"更新时间"},{"name":"expressTemplateId","type":"BIGINT","comment":"运费模板ID"},{"name":"linkUrl","type":"TEXT","comment":"商品链接"}],"conflict_keys":["kwaiItemId"]}'::jsonb); + +-- 5. 商品详情（prefetch→item_list） +INSERT INTO api_interface (tenant_id,creator,created_at,updater,updated_at,platform_id,name,code,url,method,status,auth_type,request_config,response_config,table_definition) VALUES (1,'admin',NOW(),'admin',NOW(),(SELECT id FROM api_datasource_platform WHERE platform_code='kuaishou'),'商品详情','item_detail','/open/item/get','GET','active','inherit','{"parameters_location":"query","pageSize":10,"page_param":"cursor","page_size_param":"pageSize","method":"open.item.get","version":1,"signMethod":"MD5","body_wrapper_field":"param","exclude_from_wrapper":["method","version","signMethod"],"prefetch":{"url":"/open/item/list/get","method":"GET","response_path":"data.items","target_param":"kwaiItemId","value_field":"kwaiItemId"}}'::jsonb,'{"success_field":"result","success_value":1,"list_path":"data","single_record":true}'::jsonb,'{"table_name":"kuaishou_item_detail","columns":[{"name":"itemId","type":"BIGINT","comment":"商品ID"},{"name":"relItemId","type":"BIGINT","comment":"关联商品ID"},{"name":"title","type":"VARCHAR(300)","comment":"商品标题"},{"name":"details","type":"TEXT","comment":"商品详情描述"},{"name":"categoryId","type":"INT","comment":"类目ID"},{"name":"categoryName","type":"VARCHAR(200)","comment":"类目名称"},{"name":"parentCategoryId","type":"INT","comment":"父类目ID"},{"name":"parentCategoryName","type":"VARCHAR(200)","comment":"父类目名称"},{"name":"rootCategoryId","type":"INT","comment":"根类目ID"},{"name":"rootCategoryName","type":"VARCHAR(200)","comment":"根类目名称"},{"name":"auditStatus","type":"INT","comment":"审核状态 2通过"},{"name":"auditReason","type":"VARCHAR(500)","comment":"审核原因"},{"name":"onOfflineStatus","type":"INT","comment":"上下架状态 1上架 0下架"},{"name":"createdTime","type":"BIGINT","comment":"创建时间"},{"name":"updateTime","type":"BIGINT","comment":"更新时间"},{"name":"expressTemplateId","type":"BIGINT","comment":"运费模板ID"},{"name":"linkUrl","type":"TEXT","comment":"商品链接"},{"name":"purchaseLimit","type":"BOOLEAN","comment":"是否限购"},{"name":"limitCount","type":"INT","comment":"限购数量"},{"name":"timeOfSale","type":"BIGINT","comment":"开售时间"},{"name":"itemRemark","type":"TEXT","comment":"商品备注"},{"name":"spuId","type":"BIGINT","comment":"SPU ID"},{"name":"shortTitle","type":"VARCHAR(300)","comment":"商品短标题"},{"name":"sellingPoint","type":"TEXT","comment":"商品卖点"},{"name":"instructions","type":"TEXT","comment":"使用说明"},{"name":"duplicationStatus","type":"INT","comment":"铺货状态"},{"name":"duplicationReason","type":"VARCHAR(500)","comment":"铺货原因"},{"name":"multipleStock","type":"BOOLEAN","comment":"是否多库存"},{"name":"contractPhone","type":"BOOLEAN","comment":"是否合约机"},{"name":"offlineReason","type":"TEXT","comment":"下架原因"},{"name":"whiteBaseImageUrl","type":"TEXT","comment":"白底图URL"},{"name":"transparentImageUrl","type":"TEXT","comment":"透明图URL"}],"conflict_keys":["itemId"]}'::jsonb); + +-- 6. SKU列表（prefetch→item_list） +INSERT INTO api_interface (tenant_id,creator,created_at,updater,updated_at,platform_id,name,code,url,method,status,auth_type,request_config,response_config,table_definition) VALUES (1,'admin',NOW(),'admin',NOW(),(SELECT id FROM api_datasource_platform WHERE platform_code='kuaishou'),'SKU列表','sku_list','/open/item/sku/list/get','GET','active','inherit','{"parameters_location":"query","pageSize":100,"page_param":"cursor","page_size_param":"pageSize","method":"open.item.sku.list.get","version":1,"signMethod":"MD5","skuStatus":1,"body_wrapper_field":"param","exclude_from_wrapper":["method","version","signMethod"],"prefetch":{"url":"/open/item/list/get","method":"GET","response_path":"data.items","target_param":"kwaiItemId","value_field":"kwaiItemId"}}'::jsonb,'{"success_field":"result","success_value":1,"list_path":"data.skuList"}'::jsonb,'{"table_name":"kuaishou_sku_list","columns":[{"name":"kwaiSkuId","type":"BIGINT","comment":"SKU ID"},{"name":"relSkuId","type":"BIGINT","comment":"外部SKU ID"},{"name":"kwaiItemId","type":"BIGINT","comment":"商品ID"},{"name":"skuStock","type":"INT","comment":"库存"},{"name":"imageUrl","type":"TEXT","comment":"SKU图片URL"},{"name":"skuSalePrice","type":"BIGINT","comment":"售价(分)"},{"name":"volume","type":"INT","comment":"销量"},{"name":"isValid","type":"INT","comment":"是否有效 1有效"},{"name":"createTime","type":"BIGINT","comment":"创建时间"},{"name":"updateTime","type":"BIGINT","comment":"更新时间"},{"name":"specification","type":"VARCHAR(200)","comment":"规格描述"},{"name":"appkey","type":"VARCHAR(100)","comment":"应用标识"},{"name":"skuNick","type":"VARCHAR(100)","comment":"SKU别名"},{"name":"gtinCode","type":"VARCHAR(100)","comment":"商品条形码"}],"conflict_keys":["kwaiSkuId","kwaiItemId"]}'::jsonb); + +-- 7. 售后单列表（游标pcursor，增量range，最大1天） +INSERT INTO api_interface (tenant_id,creator,created_at,updater,updated_at,platform_id,name,code,url,method,status,auth_type,request_config,response_config,table_definition) VALUES (1,'admin',NOW(),'admin',NOW(),(SELECT id FROM api_datasource_platform WHERE platform_code='kuaishou'),'售后单列表','refund_list','/open/seller/order/refund/pcursor/list','GET','active','inherit','{"parameters_location":"query","pageSize":50,"page_param":"pcursor","page_size_param":"pageSize","cursor_pagination":true,"method":"open.seller.order.refund.pcursor.list","version":1,"signMethod":"MD5","type":9,"sort":1,"queryType":2,"currentPage":1,"body_wrapper_field":"param","exclude_from_wrapper":["method","version","signMethod"],"time_field":"updateTime","time_field_mode":"range"}'::jsonb,'{"success_field":"result","success_value":1,"list_path":"data.refundOrderInfoList","cursor_field":"data.pcursor","cursor_end_marker":"nomore"}'::jsonb,'{"table_name":"kuaishou_refund_list","columns":[{"name":"refundId","type":"BIGINT","comment":"退款单ID"},{"name":"oid","type":"BIGINT","comment":"订单ID"},{"name":"itemId","type":"BIGINT","comment":"商品ID"},{"name":"skuId","type":"BIGINT","comment":"SKU ID"},{"name":"relSkuId","type":"BIGINT","comment":"外部SKU ID"},{"name":"skuNick","type":"VARCHAR(100)","comment":"SKU别名"},{"name":"handlingWay","type":"INT","comment":"售后方式"},{"name":"negotiateStatus","type":"INT","comment":"协商状态"},{"name":"refundFee","type":"BIGINT","comment":"退款金额(分)"},{"name":"refundReason","type":"INT","comment":"退款原因码"},{"name":"refundReasonDesc","type":"VARCHAR(500)","comment":"退款原因描述"},{"name":"refundDesc","type":"TEXT","comment":"退款描述"},{"name":"refundType","type":"INT","comment":"退款类型"},{"name":"status","type":"INT","comment":"退款状态"},{"name":"receiptStatus","type":"INT","comment":"收货状态"},{"name":"buyerId","type":"BIGINT","comment":"买家ID"},{"name":"sellerId","type":"BIGINT","comment":"卖家ID"},{"name":"logisticsId","type":"BIGINT","comment":"物流ID"},{"name":"relItemId","type":"BIGINT","comment":"关联商品ID"},{"name":"submitTime","type":"BIGINT","comment":"提交时间"},{"name":"createTime","type":"BIGINT","comment":"创建时间"},{"name":"updateTime","type":"BIGINT","comment":"更新时间"},{"name":"negotiateUpdateTime","type":"BIGINT","comment":"协商更新时间"},{"name":"endTime","type":"BIGINT","comment":"结束时间"},{"name":"expireTime","type":"BIGINT","comment":"过期时间"}],"conflict_keys":["refundId"]}'::jsonb); + +-- 8. 售后单详情（prefetch→refund_list） +INSERT INTO api_interface (tenant_id,creator,created_at,updater,updated_at,platform_id,name,code,url,method,status,auth_type,request_config,response_config,table_definition) VALUES (1,'admin',NOW(),'admin',NOW(),(SELECT id FROM api_datasource_platform WHERE platform_code='kuaishou'),'售后单详情','refund_detail','/open/seller/order/refund/detail','GET','active','inherit','{"parameters_location":"query","pageSize":10,"page_param":"cursor","page_size_param":"pageSize","method":"open.seller.order.refund.detail","version":1,"signMethod":"MD5","body_wrapper_field":"param","exclude_from_wrapper":["method","version","signMethod"],"prefetch":{"url":"/open/seller/order/refund/pcursor/list","method":"GET","response_path":"data.refundOrderInfoList","target_param":"refundId","value_field":"refundId"}}'::jsonb,'{"success_field":"result","success_value":1,"list_path":"data","single_record":true}'::jsonb,'{"table_name":"kuaishou_refund_detail","columns":[{"name":"refundId","type":"BIGINT","comment":"退款单ID"},{"name":"oid","type":"BIGINT","comment":"订单ID"},{"name":"itemId","type":"BIGINT","comment":"商品ID"},{"name":"skuId","type":"BIGINT","comment":"SKU ID"},{"name":"relSkuId","type":"BIGINT","comment":"外部SKU ID"},{"name":"relItemId","type":"BIGINT","comment":"关联商品ID"},{"name":"handlingWay","type":"INT","comment":"售后方式"},{"name":"negotiateStatus","type":"INT","comment":"协商状态"},{"name":"refundFee","type":"BIGINT","comment":"退款金额(分)"},{"name":"refundReason","type":"INT","comment":"退款原因码"},{"name":"refundReasonDesc","type":"VARCHAR(500)","comment":"退款原因描述"},{"name":"refundDesc","type":"TEXT","comment":"退款描述"},{"name":"refundType","type":"INT","comment":"退款类型"},{"name":"status","type":"INT","comment":"退款状态"},{"name":"receiptStatus","type":"INT","comment":"收货状态"},{"name":"buyerId","type":"BIGINT","comment":"买家ID"},{"name":"buyerOpenId","type":"VARCHAR(100)","comment":"买家openId"},{"name":"sellerId","type":"BIGINT","comment":"卖家ID"},{"name":"logisticsId","type":"BIGINT","comment":"物流ID"},{"name":"productNum","type":"INT","comment":"商品数量"},{"name":"specialRefundType","type":"INT","comment":"特殊退款类型"},{"name":"sellerDisagreeReason","type":"INT","comment":"卖家拒绝原因码"},{"name":"sellerDisagreeDesc","type":"TEXT","comment":"卖家拒绝描述"},{"name":"negotiateReason","type":"VARCHAR(500)","comment":"协商原因"},{"name":"address","type":"TEXT","comment":"收货地址"},{"name":"validNegotiateBuyerModifyTimeStamp","type":"BIGINT","comment":"有效协商修改时间戳"},{"name":"timeLimitNegotiateChange","type":"INT","comment":"协商变更时限"},{"name":"submitTime","type":"BIGINT","comment":"提交时间"},{"name":"createTime","type":"BIGINT","comment":"创建时间"},{"name":"updateTime","type":"BIGINT","comment":"更新时间"},{"name":"negotiateUpdateTime","type":"BIGINT","comment":"协商更新时间"},{"name":"endTime","type":"BIGINT","comment":"结束时间"},{"name":"expireTime","type":"BIGINT","comment":"过期时间"}],"conflict_keys":["refundId"]}'::jsonb); + +-- 9. 商品类目（全量，data直接是数组） +INSERT INTO api_interface (tenant_id,creator,created_at,updater,updated_at,platform_id,name,code,url,method,status,auth_type,request_config,response_config,table_definition) VALUES (1,'admin',NOW(),'admin',NOW(),(SELECT id FROM api_datasource_platform WHERE platform_code='kuaishou'),'商品类目','category','/open/item/category','GET','active','inherit','{"parameters_location":"query","pageSize":100,"page_param":"cursor","page_size_param":"pageSize","method":"open.item.category","version":1,"signMethod":"MD5","body_wrapper_field":"param","exclude_from_wrapper":["method","version","signMethod"]}'::jsonb,'{"success_field":"result","success_value":1,"list_path":"data"}'::jsonb,'{"table_name":"kuaishou_category","columns":[{"name":"categoryId","type":"BIGINT","comment":"类目ID"},{"name":"categoryName","type":"VARCHAR(300)","comment":"类目名称"},{"name":"categoryPid","type":"BIGINT","comment":"父级类目ID 0为根类目"}],"conflict_keys":["categoryId"]}'::jsonb); + +-- 10. 带货口碑分（单记录，全量） +INSERT INTO api_interface (tenant_id,creator,created_at,updater,updated_at,platform_id,name,code,url,method,status,auth_type,request_config,response_config,table_definition) VALUES (1,'admin',NOW(),'admin',NOW(),(SELECT id FROM api_datasource_platform WHERE platform_code='kuaishou'),'带货口碑分','score_master','/open/score/master/get','GET','active','inherit','{"parameters_location":"query","pageSize":10,"page_param":"cursor","page_size_param":"pageSize","method":"open.score.master.get","version":1,"signMethod":"MD5","body_wrapper_field":"param","exclude_from_wrapper":["method","version","signMethod"]}'::jsonb,'{"success_field":"result","success_value":1,"list_path":"data","single_record":true}'::jsonb,'{"table_name":"kuaishou_score_master","columns":[{"name":"hasData","type":"BOOLEAN","comment":"是否有数据"},{"name":"showName","type":"VARCHAR(100)","comment":"分值文案描述"},{"name":"scoreStr","type":"VARCHAR(20)","comment":"分值(保留两位小数)"}],"conflict_keys":["showName"]}'::jsonb); + +-- 11. 店铺体验分（单记录，全量） +INSERT INTO api_interface (tenant_id,creator,created_at,updater,updated_at,platform_id,name,code,url,method,status,auth_type,request_config,response_config,table_definition) VALUES (1,'admin',NOW(),'admin',NOW(),(SELECT id FROM api_datasource_platform WHERE platform_code='kuaishou'),'店铺体验分','score_shop','/open/score/shop/get','GET','active','inherit','{"parameters_location":"query","pageSize":10,"page_param":"cursor","page_size_param":"pageSize","method":"open.score.shop.get","version":1,"signMethod":"MD5","body_wrapper_field":"param","exclude_from_wrapper":["method","version","signMethod"]}'::jsonb,'{"success_field":"result","success_value":1,"list_path":"data","single_record":true}'::jsonb,'{"table_name":"kuaishou_score_shop","columns":[{"name":"hasData","type":"BOOLEAN","comment":"是否有数据"},{"name":"showName","type":"VARCHAR(100)","comment":"分值文案描述"},{"name":"scoreStr","type":"VARCHAR(20)","comment":"分值(保留两位小数)"}],"conflict_keys":["showName"]}'::jsonb); + +-- 12. 店铺信息（单记录，全量） +INSERT INTO api_interface (tenant_id,creator,created_at,updater,updated_at,platform_id,name,code,url,method,status,auth_type,request_config,response_config,table_definition) VALUES (1,'admin',NOW(),'admin',NOW(),(SELECT id FROM api_datasource_platform WHERE platform_code='kuaishou'),'店铺信息','shop_info','/open/shop/info/get','GET','active','inherit','{"parameters_location":"query","pageSize":10,"page_param":"cursor","page_size_param":"pageSize","method":"open.shop.info.get","version":1,"signMethod":"MD5","body_wrapper_field":"param","exclude_from_wrapper":["method","version","signMethod"]}'::jsonb,'{"success_field":"result","success_value":1,"list_path":"data","single_record":true}'::jsonb,'{"table_name":"kuaishou_shop_info","columns":[{"name":"shopName","type":"VARCHAR(300)","comment":"店铺全名"},{"name":"shopType","type":"INT","comment":"店铺类型 1旗舰 2专卖 3专营"},{"name":"shopExpScoreStr","type":"VARCHAR(20)","comment":"购物体验分"},{"name":"productQualityScoreStr","type":"VARCHAR(20)","comment":"商品品质分"},{"name":"contentQualifyScoreStr","type":"VARCHAR(20)","comment":"内容质量分"},{"name":"customerServiceScoreStr","type":"VARCHAR(20)","comment":"客服服务分"},{"name":"logisticsServiceScoreStr","type":"VARCHAR(20)","comment":"物流服务分"},{"name":"afterSalesServiceScoreStr","type":"VARCHAR(20)","comment":"售后服务分"}],"conflict_keys":["shopName"]}'::jsonb); + +-- 13. 门店POI详情（单记录，需outerPoiId+source） +INSERT INTO api_interface (tenant_id,creator,created_at,updater,updated_at,platform_id,name,code,url,method,status,auth_type,request_config,response_config,table_definition) VALUES (1,'admin',NOW(),'admin',NOW(),(SELECT id FROM api_datasource_platform WHERE platform_code='kuaishou'),'门店POI详情','poi_detail','/open/shop/poi/getPoiDetailByOuterPoi','GET','active','inherit','{"parameters_location":"query","pageSize":10,"page_param":"cursor","page_size_param":"pageSize","method":"open.shop.poi.getPoiDetailByOuterPoi","version":1,"signMethod":"MD5","body_wrapper_field":"param","exclude_from_wrapper":["method","version","signMethod"]}'::jsonb,'{"success_field":"result","success_value":1,"list_path":"data","single_record":true}'::jsonb,'{"table_name":"kuaishou_poi_detail","columns":[{"name":"outerPoiId","type":"VARCHAR(100)","comment":"图商poiId"},{"name":"source","type":"INT","comment":"图商来源 1高德 2百度 3腾讯"},{"name":"poiId","type":"VARCHAR(100)","comment":"快手poiId"},{"name":"name","type":"VARCHAR(300)","comment":"poi名称"},{"name":"longitude","type":"VARCHAR(50)","comment":"经度"},{"name":"latitude","type":"VARCHAR(50)","comment":"纬度"},{"name":"address","type":"TEXT","comment":"地址信息"},{"name":"country","type":"VARCHAR(100)","comment":"国家"},{"name":"province","type":"VARCHAR(100)","comment":"省份"},{"name":"city","type":"VARCHAR(100)","comment":"城市"},{"name":"district","type":"VARCHAR(100)","comment":"区县"},{"name":"provinceCode","type":"BIGINT","comment":"省份编码"},{"name":"cityCode","type":"BIGINT","comment":"城市编码"},{"name":"districtCode","type":"BIGINT","comment":"区县编码"}],"conflict_keys":["poiId"]}'::jsonb); + +-- 14. 分销订单列表（游标pcursor，增量range，最大3天） +INSERT INTO api_interface (tenant_id,creator,created_at,updater,updated_at,platform_id,name,code,url,method,status,auth_type,request_config,response_config,table_definition) VALUES (1,'admin',NOW(),'admin',NOW(),(SELECT id FROM api_datasource_platform WHERE platform_code='kuaishou'),'分销订单列表','cps_order_list','/open/seller/order/cps/list','GET','active','inherit','{"parameters_location":"query","pageSize":80,"page_param":"pcursor","page_size_param":"pageSize","cursor_pagination":true,"method":"open.seller.order.cps.list","version":1,"signMethod":"MD5","sort":1,"queryType":2,"type":1,"currentPage":1,"body_wrapper_field":"param","exclude_from_wrapper":["method","version","signMethod"],"time_field":"updateTime","time_field_mode":"range"}'::jsonb,'{"success_field":"result","success_value":1,"list_path":"data.cpsOrderList","cursor_field":"data.pcursor","cursor_end_marker":"nomore"}'::jsonb,'{"table_name":"kuaishou_cps_order_list","columns":[{"name":"oid","type":"BIGINT","comment":"订单ID"},{"name":"distributorId","type":"BIGINT","comment":"分销者ID"},{"name":"distributorName","type":"VARCHAR(200)","comment":"分销者昵称"},{"name":"sellerId","type":"BIGINT","comment":"卖家ID"},{"name":"status","type":"INT","comment":"分销单状态"},{"name":"settlementTime","type":"BIGINT","comment":"结算时间"},{"name":"settlementSuccessTime","type":"BIGINT","comment":"结算成功时间"},{"name":"refundTime","type":"BIGINT","comment":"退款时间"},{"name":"payTime","type":"BIGINT","comment":"付款时间"},{"name":"expressFee","type":"BIGINT","comment":"运费(分)"},{"name":"totalFee","type":"BIGINT","comment":"实际付款金额-运费(分)"},{"name":"commissionRate","type":"BIGINT","comment":"分销率"},{"name":"estimatedIncome","type":"BIGINT","comment":"分销金额(分)"},{"name":"createTime","type":"BIGINT","comment":"创建时间"},{"name":"updateTime","type":"BIGINT","comment":"更新时间"},{"name":"platformDpRate","type":"BIGINT","comment":"平台佣金比例 千分比"},{"name":"activityUserId","type":"BIGINT","comment":"团长ID"},{"name":"activityUserNickname","type":"VARCHAR(200)","comment":"团长昵称"},{"name":"investmentPromotionRate","type":"INT","comment":"团长服务费率 千分比"},{"name":"investmentPromotionAmount","type":"BIGINT","comment":"团长服务费金额(分)"},{"name":"buyerOpenId","type":"VARCHAR(100)","comment":"买家识别ID"},{"name":"settlementBizType","type":"INT","comment":"订单业务类型 1快分销 2聚力计划"},{"name":"promoterServiceInCome","type":"BIGINT","comment":"达人接单服务收入(分)"},{"name":"promoterExcitationInCome","type":"BIGINT","comment":"达人奖励收入(分)"},{"name":"investmentServiceInCome","type":"BIGINT","comment":"团长接单收入(分)"},{"name":"investmentExcitationInCome","type":"BIGINT","comment":"团长奖励收入(分)"},{"name":"orderChannel","type":"VARCHAR(100)","comment":"出单渠道"},{"name":"activityId","type":"BIGINT","comment":"活动ID"},{"name":"itemId","type":"BIGINT","comment":"商品ID"}],"conflict_keys":["oid"]}'::jsonb); + +-- 15. 分销订单详情（prefetch→cps_order_list） +INSERT INTO api_interface (tenant_id,creator,created_at,updater,updated_at,platform_id,name,code,url,method,status,auth_type,request_config,response_config,table_definition) VALUES (1,'admin',NOW(),'admin',NOW(),(SELECT id FROM api_datasource_platform WHERE platform_code='kuaishou'),'分销订单详情','cps_order_detail','/open/seller/order/cps/detail','GET','active','inherit','{"parameters_location":"query","pageSize":10,"page_param":"cursor","page_size_param":"pageSize","method":"open.seller.order.cps.detail","version":1,"signMethod":"MD5","body_wrapper_field":"param","exclude_from_wrapper":["method","version","signMethod"],"prefetch":{"url":"/open/seller/order/cps/list","method":"GET","response_path":"data.cpsOrderList","target_param":"orderId","value_field":"oid"}}'::jsonb,'{"success_field":"result","success_value":1,"list_path":"data","single_record":true}'::jsonb,'{"table_name":"kuaishou_cps_order_detail","columns":[{"name":"oid","type":"BIGINT","comment":"订单ID"},{"name":"distributorId","type":"BIGINT","comment":"分销者ID"},{"name":"distributorName","type":"VARCHAR(200)","comment":"分销者昵称"},{"name":"sellerId","type":"BIGINT","comment":"卖家ID"},{"name":"status","type":"INT","comment":"分销单状态"},{"name":"settlementTime","type":"BIGINT","comment":"结算时间"},{"name":"settlementSuccessTime","type":"BIGINT","comment":"结算成功时间"},{"name":"refundTime","type":"BIGINT","comment":"退款时间"},{"name":"payTime","type":"BIGINT","comment":"付款时间"},{"name":"expressFee","type":"BIGINT","comment":"运费(分)"},{"name":"totalFee","type":"BIGINT","comment":"实际付款金额-运费(分)"},{"name":"commissionRate","type":"BIGINT","comment":"分销率"},{"name":"estimatedIncome","type":"BIGINT","comment":"分销金额(分)"},{"name":"createTime","type":"BIGINT","comment":"创建时间"},{"name":"updateTime","type":"BIGINT","comment":"更新时间"},{"name":"platformDpRate","type":"BIGINT","comment":"平台佣金比例 千分比"},{"name":"activityUserId","type":"BIGINT","comment":"团长ID"},{"name":"activityUserNickname","type":"VARCHAR(200)","comment":"团长昵称"},{"name":"investmentPromotionRate","type":"INT","comment":"团长服务费率 千分比"},{"name":"investmentPromotionAmount","type":"BIGINT","comment":"团长服务费金额(分)"},{"name":"buyerOpenId","type":"VARCHAR(100)","comment":"买家识别ID"},{"name":"settlementBizType","type":"INT","comment":"订单业务类型 1快分销 2聚力计划"},{"name":"promoterServiceInCome","type":"BIGINT","comment":"达人接单服务收入(分)"},{"name":"promoterExcitationInCome","type":"BIGINT","comment":"达人奖励收入(分)"},{"name":"investmentServiceInCome","type":"BIGINT","comment":"团长接单收入(分)"},{"name":"investmentExcitationInCome","type":"BIGINT","comment":"团长奖励收入(分)"},{"name":"orderChannel","type":"VARCHAR(100)","comment":"出单渠道"},{"name":"activityId","type":"BIGINT","comment":"活动ID"},{"name":"itemId","type":"BIGINT","comment":"商品ID"}],"conflict_keys":["oid"]}'::jsonb); + +-- 16. 达人分销订单列表（游标pcursor，增量range，最大7天） +INSERT INTO api_interface (tenant_id,creator,created_at,updater,updated_at,platform_id,name,code,url,method,status,auth_type,request_config,response_config,table_definition) VALUES (1,'admin',NOW(),'admin',NOW(),(SELECT id FROM api_datasource_platform WHERE platform_code='kuaishou'),'达人分销订单列表','distributor_order_list','/open/distribution/cps/distributor/order/cursor/list','GET','active','inherit','{"parameters_location":"query","pageSize":100,"page_param":"pcursor","page_size_param":"pageSize","cursor_pagination":true,"method":"open.distribution.cps.distributor.order.cursor.list","version":1,"signMethod":"MD5","sortType":1,"queryType":2,"cpsOrderStatus":0,"body_wrapper_field":"param","exclude_from_wrapper":["method","version","signMethod"],"time_field":"updateTime","time_field_mode":"range"}'::jsonb,'{"success_field":"result","success_value":1,"list_path":"data.orderView","cursor_field":"data.pcursor","cursor_end_marker":"nomore"}'::jsonb,'{"table_name":"kuaishou_distributor_order_list","columns":[{"name":"oid","type":"BIGINT","comment":"订单ID"},{"name":"distributorId","type":"BIGINT","comment":"分销者ID"},{"name":"cpsOrderStatus","type":"INT","comment":"分销订单状态"},{"name":"orderCreateTime","type":"BIGINT","comment":"订单创建时间"},{"name":"payTime","type":"BIGINT","comment":"支付时间"},{"name":"sendTime","type":"BIGINT","comment":"发货时间"},{"name":"sendStatus","type":"INT","comment":"发货状态"},{"name":"recvTime","type":"BIGINT","comment":"收货时间"},{"name":"orderTradeAmount","type":"BIGINT","comment":"订单交易金额(分)"},{"name":"baseAmount","type":"BIGINT","comment":"结算基数"},{"name":"shareRateStr","type":"VARCHAR(20)","comment":"分佣比例"},{"name":"settlementBizType","type":"INT","comment":"订单业务类型"},{"name":"settlementAmount","type":"BIGINT","comment":"结算金额(分)"},{"name":"settlementSuccessTime","type":"BIGINT","comment":"结算成功时间"},{"name":"buyerOpenId","type":"VARCHAR(100)","comment":"买家识别ID"},{"name":"createTime","type":"BIGINT","comment":"创建时间"},{"name":"updateTime","type":"BIGINT","comment":"更新时间"}],"conflict_keys":["oid"]}'::jsonb); + +-- 17. 开票金额查询（单记录，prefetch→order_list） +INSERT INTO api_interface (tenant_id,creator,created_at,updater,updated_at,platform_id,name,code,url,method,status,auth_type,request_config,response_config,table_definition) VALUES (1,'admin',NOW(),'admin',NOW(),(SELECT id FROM api_datasource_platform WHERE platform_code='kuaishou'),'开票金额','invoice_amount','/open/invoice/amount/get','GET','active','inherit','{"parameters_location":"query","pageSize":10,"page_param":"cursor","page_size_param":"pageSize","method":"open.invoice.amount.get","version":1,"signMethod":"MD5","fromType":"1","body_wrapper_field":"param","exclude_from_wrapper":["method","version","signMethod"],"prefetch":{"url":"/open/order/cursor/list","method":"GET","response_path":"data.orderList","target_param":"orderId","value_field":"oid"}}'::jsonb,'{"success_field":"result","success_value":1,"list_path":"queryInvoiceAmountResponseData","single_record":true}'::jsonb,'{"table_name":"kuaishou_invoice_amount","columns":[{"name":"toReceiverInvoiceAmount","type":"VARCHAR(50)","comment":"达人开票金额"},{"name":"toBuyerInvoiceAmount","type":"VARCHAR(50)","comment":"买家开票金额"},{"name":"toPlatformInvoiceAmount","type":"VARCHAR(50)","comment":"平台开票金额"},{"name":"orderStatus","type":"BIGINT","comment":"订单状态"},{"name":"preSaleOrder","type":"BOOLEAN","comment":"是否定金预售"},{"name":"platformAllowanceAmount","type":"VARCHAR(50)","comment":"平台补贴"},{"name":"freightWhenOrder","type":"VARCHAR(50)","comment":"运费"},{"name":"orderId","type":"BIGINT","comment":"订单ID"},{"name":"userPayAmount","type":"VARCHAR(50)","comment":"用户实付"},{"name":"receiverSubsidyAmount","type":"VARCHAR(50)","comment":"达人补贴"},{"name":"queryTime","type":"VARCHAR(50)","comment":"查询时间"}],"conflict_keys":["orderId"]}'::jsonb); + +-- 18. 代发订单列表（游标cursor，POST，全量最大30天） +INSERT INTO api_interface (tenant_id,creator,created_at,updater,updated_at,platform_id,name,code,url,method,status,auth_type,request_config,response_config,table_definition) VALUES (1,'admin',NOW(),'admin',NOW(),(SELECT id FROM api_datasource_platform WHERE platform_code='kuaishou'),'代发订单列表','dropshipping_order_list','/open/dropshipping/order/merchant/list','POST','active','inherit','{"parameters_location":"query","pageSize":50,"page_param":"cursor","page_size_param":"pageSize","cursor_pagination":true,"method":"open.dropshipping.order.merchant.list","version":1,"signMethod":"MD5","dropshippingStatus":0,"orderStatus":0,"refundStatus":0,"orderType":200,"queryType":"PAT_TIME","sort":0,"body_wrapper_field":"param","exclude_from_wrapper":["method","version","signMethod"]}'::jsonb,'{"success_field":"result","success_value":1,"list_path":"data.orderList","cursor_field":"data.cursor","cursor_end_marker":"nomore"}'::jsonb,'{"table_name":"kuaishou_dropshipping_order_list","columns":[{"name":"oid","type":"BIGINT","comment":"交易订单ID"},{"name":"dropshippingOrderCode","type":"VARCHAR(100)","comment":"代发订单编码"},{"name":"payTime","type":"BIGINT","comment":"支付时间"},{"name":"buyerOpenId","type":"VARCHAR(100)","comment":"买家开放id"},{"name":"buyerNick","type":"VARCHAR(100)","comment":"买家昵称"},{"name":"sellerOpenId","type":"VARCHAR(100)","comment":"卖家开放id"},{"name":"sellerNick","type":"VARCHAR(100)","comment":"卖家昵称"},{"name":"orderStatus","type":"INT","comment":"订单状态"},{"name":"orderStatusDesc","type":"VARCHAR(100)","comment":"订单状态描述"},{"name":"refundStatus","type":"INT","comment":"售后状态"},{"name":"refundStatusDesc","type":"VARCHAR(100)","comment":"售后状态描述"},{"name":"orderType","type":"INT","comment":"订单类型"},{"name":"orderTypeDesc","type":"VARCHAR(100)","comment":"订单类型描述"},{"name":"deliveryTime","type":"BIGINT","comment":"发货时间"},{"name":"createTime","type":"BIGINT","comment":"创建时间"},{"name":"updateTime","type":"BIGINT","comment":"修改时间"},{"name":"waybillCode","type":"VARCHAR(100)","comment":"运单号"},{"name":"expressCompanyCode","type":"VARCHAR(50)","comment":"物流公司编码"},{"name":"expressCompanyName","type":"VARCHAR(100)","comment":"物流公司名称"},{"name":"factoryCode","type":"VARCHAR(100)","comment":"代发厂家编码"},{"name":"factoryName","type":"VARCHAR(200)","comment":"代发厂家名称"},{"name":"allocateTime","type":"BIGINT","comment":"分配时间"},{"name":"dropshippingStatus","type":"INT","comment":"代发状态"},{"name":"dropshippingStatusDesc","type":"VARCHAR(100)","comment":"代发状态描述"},{"name":"cancelAllocateTime","type":"BIGINT","comment":"取消分配时间"},{"name":"cancelAllocateReason","type":"TEXT","comment":"取消分配原因"},{"name":"name","type":"VARCHAR(100)","comment":"收货人姓名"},{"name":"mobile","type":"VARCHAR(50)","comment":"手机号码"},{"name":"provinceName","type":"VARCHAR(100)","comment":"省份"},{"name":"cityName","type":"VARCHAR(100)","comment":"城市"},{"name":"districtName","type":"VARCHAR(100)","comment":"区县"},{"name":"detailAddress","type":"TEXT","comment":"详细地址"}],"conflict_keys":["oid"]}'::jsonb); + +-- 19. 代发订单详情（prefetch→dropshipping_order_list，单记录） +INSERT INTO api_interface (tenant_id,creator,created_at,updater,updated_at,platform_id,name,code,url,method,status,auth_type,request_config,response_config,table_definition) VALUES (1,'admin',NOW(),'admin',NOW(),(SELECT id FROM api_datasource_platform WHERE platform_code='kuaishou'),'代发订单详情','dropshipping_order_detail','/open/dropshipping/order/merchant/detail','POST','active','inherit','{"parameters_location":"query","pageSize":10,"page_param":"cursor","page_size_param":"pageSize","method":"open.dropshipping.order.merchant.detail","version":1,"signMethod":"MD5","body_wrapper_field":"param","exclude_from_wrapper":["method","version","signMethod"],"prefetch":{"url":"/open/dropshipping/order/merchant/list","method":"POST","response_path":"data.orderList","target_param":"dropshippingOrderCode","value_field":"dropshippingOrderCode"}}'::jsonb,'{"success_field":"result","success_value":1,"list_path":"data","single_record":true}'::jsonb,'{"table_name":"kuaishou_dropshipping_order_detail","columns":[{"name":"dropshippingOrderCode","type":"VARCHAR(100)","comment":"代发订单编码"},{"name":"oid","type":"BIGINT","comment":"交易订单ID"},{"name":"payTime","type":"BIGINT","comment":"支付时间"},{"name":"buyerOpenId","type":"VARCHAR(100)","comment":"买家开放id"},{"name":"buyerNick","type":"VARCHAR(100)","comment":"买家昵称"},{"name":"sellerOpenId","type":"VARCHAR(100)","comment":"卖家开放id"},{"name":"sellerNick","type":"VARCHAR(100)","comment":"卖家昵称"},{"name":"orderStatus","type":"INT","comment":"订单状态"},{"name":"orderStatusDesc","type":"VARCHAR(100)","comment":"订单状态描述"},{"name":"refundStatus","type":"INT","comment":"售后状态"},{"name":"refundStatusDesc","type":"VARCHAR(100)","comment":"售后状态描述"},{"name":"orderType","type":"INT","comment":"订单类型"},{"name":"orderTypeDesc","type":"VARCHAR(100)","comment":"订单类型描述"},{"name":"deliveryTime","type":"BIGINT","comment":"发货时间"},{"name":"createTime","type":"BIGINT","comment":"创建时间"},{"name":"updateTime","type":"BIGINT","comment":"修改时间"},{"name":"waybillCode","type":"VARCHAR(100)","comment":"运单号"},{"name":"expressCompanyCode","type":"VARCHAR(50)","comment":"物流公司编码"},{"name":"expressCompanyName","type":"VARCHAR(100)","comment":"物流公司名称"},{"name":"factoryCode","type":"VARCHAR(100)","comment":"代发厂家编码"},{"name":"factoryName","type":"VARCHAR(200)","comment":"代发厂家名称"},{"name":"allocateTime","type":"BIGINT","comment":"分配时间"},{"name":"dropshippingStatus","type":"INT","comment":"代发状态"},{"name":"dropshippingStatusDesc","type":"VARCHAR(100)","comment":"代发状态描述"},{"name":"cancelAllocateTime","type":"BIGINT","comment":"取消分配时间"},{"name":"cancelAllocateReason","type":"TEXT","comment":"取消分配原因"},{"name":"buyerNote","type":"JSONB","comment":"买家备注列表"},{"name":"sellerNote","type":"JSONB","comment":"卖家备注列表"},{"name":"orderItemList","type":"JSONB","comment":"订单商品信息列表"},{"name":"detailAddress","type":"TEXT","comment":"详细地址"},{"name":"addressId","type":"VARCHAR(100)","comment":"地址id"},{"name":"cityCode","type":"VARCHAR(50)","comment":"城市编码"},{"name":"provinceCode","type":"VARCHAR(50)","comment":"省份编码"},{"name":"streetCode","type":"VARCHAR(50)","comment":"街道编码"},{"name":"streetName","type":"VARCHAR(100)","comment":"街道名称"},{"name":"provinceName","type":"VARCHAR(100)","comment":"省份"},{"name":"countryCode","type":"VARCHAR(50)","comment":"国家编码"},{"name":"districtName","type":"VARCHAR(100)","comment":"区县名称"},{"name":"cityName","type":"VARCHAR(100)","comment":"城市名称"},{"name":"districtCode","type":"VARCHAR(50)","comment":"区县编码"},{"name":"countryName","type":"VARCHAR(100)","comment":"国家名称"},{"name":"receiverName","type":"VARCHAR(100)","comment":"收货人姓名"},{"name":"receiverMobile","type":"VARCHAR(50)","comment":"收货人手机号码"},{"name":"receiverTelephone","type":"VARCHAR(50)","comment":"收货人电话"}],"conflict_keys":["dropshippingOrderCode"]}'::jsonb);