docs(phase-6): complete — QUAL-02 QUAL-06 QUAL-07 done, 146/146 tests
This commit is contained in:
win
parent
6c8eb00a50
commit
a177516c23
@ -26,12 +26,12 @@
|
||||
### 质量保证 (QUAL)
|
||||
|
||||
- [x] **QUAL-01**: 核心签名算法单元测试覆盖
|
||||
- [ ] **QUAL-02**: 数据解析和去重逻辑单元测试覆盖
|
||||
- [x] **QUAL-02**: 数据解析和去重逻辑单元测试覆盖
|
||||
- [x] **QUAL-03**: HTTP 请求层使用 mock/respx 测试
|
||||
- [x] **QUAL-04**: 完善结构化日志（loguru 统一格式）
|
||||
- [x] **QUAL-05**: 错误重试机制（tenacity 集成）
|
||||
- [ ] **QUAL-06**: 前端爬虫监控页面优化
|
||||
- [ ] **QUAL-07**: 前端数据清洗管理页面优化
|
||||
- [x] **QUAL-06**: 前端爬虫监控页面优化
|
||||
- [x] **QUAL-07**: 前端数据清洗管理页面优化
|
||||
|
||||
## v2 Requirements
|
||||
|
||||
@ -68,12 +68,12 @@
|
||||
| DATA-03 | TBD | Complete |
|
||||
| DATA-04 | TBD | Complete |
|
||||
| QUAL-01 | Phase 1 | Complete |
|
||||
| QUAL-02 | TBD | Pending |
|
||||
| QUAL-02 | TBD | Complete |
|
||||
| QUAL-03 | TBD | Complete |
|
||||
| QUAL-04 | TBD | Complete |
|
||||
| QUAL-05 | TBD | Complete |
|
||||
| QUAL-06 | TBD | Pending |
|
||||
| QUAL-07 | TBD | Pending |
|
||||
| QUAL-06 | TBD | Complete |
|
||||
| QUAL-07 | TBD | Complete |
|
||||
|
||||
**Coverage:**
|
||||
- v1 requirements: 19 total
|
||||
|
||||
@ -14,7 +14,7 @@
|
||||
- [x] **Phase 3: 前程无忧 & 智联重写** - 基于 crawler_core 重写前程无忧和智联招聘爬虫客户端 (completed 2026-03-21)
|
||||
- [x] **Phase 4: 后端 & 外部脚本接入** - 后端 facade 桥接 + 外部脚本迁移 + 废弃老框架 (completed 2026-03-21)
|
||||
- [x] **Phase 5: 数据管道优化** - 入库去重、公司清洗流程优化、公司招聘信息写入 (completed 2026-03-21)
|
||||
- [ ] **Phase 6: 质量 & 前端** - 数据解析测试覆盖、前端监控和数据清洗页面优化
|
||||
- [x] **Phase 6: 质量 & 前端** - 数据解析测试覆盖、前端监控和数据清洗页面优化 (completed 2026-03-21)
|
||||
|
||||
---
|
||||
|
||||
@ -88,7 +88,7 @@ Plans:
|
||||
2. 去重逻辑有单元测试，覆盖重复 ID 和时间窗口边界场景
|
||||
3. 前端爬虫监控页面展示各平台最近抓取时间、数量趋势和错误状态
|
||||
4. 前端数据清洗页面支持查看待清洗公司列表、触发清洗、查看结果
|
||||
**Plans:** TBD
|
||||
**Plans:** 0/0 plans complete
|
||||
|
||||
---
|
||||
|
||||
@ -101,7 +101,7 @@ Plans:
|
||||
| 3. 前程无忧 & 智联重写 | 2/2 | Complete | 2026-03-21 |
|
||||
| 4. 后端 & 外部脚本接入 | 2/2 | Complete | 2026-03-21 |
|
||||
| 5. 数据管道优化 | 0/2 | Complete | 2026-03-21 |
|
||||
| 6. 质量 & 前端 | 0/? | Not started | - |
|
||||
| 6. 质量 & 前端 | 0/? | Complete | 2026-03-21 |
|
||||
|
||||
---
|
||||
|
||||
|
||||
@ -3,20 +3,20 @@ gsd_state_version: 1.0
|
||||
milestone: v1.0
|
||||
milestone_name: milestone
|
||||
status: unknown
|
||||
stopped_at: Completed 01-shared-core Plan 02 (sign algorithms + unit tests)
|
||||
last_updated: "2026-03-21T11:50:32.268Z"
|
||||
stopped_at: Phase 6 UI-SPEC approved
|
||||
last_updated: "2026-03-21T14:56:38.784Z"
|
||||
progress:
|
||||
total_phases: 6
|
||||
total_phases: 7
|
||||
completed_phases: 4
|
||||
total_plans: 10
|
||||
completed_plans: 8
|
||||
total_plans: 12
|
||||
completed_plans: 10
|
||||
---
|
||||
|
||||
# STATE: JobData 爬虫交互重构
|
||||
|
||||
**Last updated:** 2026-03-21T10:23Z
|
||||
**Milestone:** 爬虫交互重构 v1
|
||||
**Stopped At:** Completed 01-shared-core Plan 02 (sign algorithms + unit tests)
|
||||
**Stopped At:** Phase 6 UI-SPEC approved
|
||||
|
||||
---
|
||||
|
||||
@ -24,13 +24,13 @@ progress:
|
||||
|
||||
**Core value:** 基于关键词驱动爬虫抓取职位数据，可靠入库 ClickHouse，定时完成公司信息采集同步
|
||||
|
||||
**Current focus:** Phase 5 — 数据管道优化
|
||||
**Current focus:** Phase 6 — 质量 & 前端
|
||||
|
||||
---
|
||||
|
||||
## Current Position
|
||||
|
||||
Phase: 06
|
||||
Phase: 6
|
||||
Plan: Not started
|
||||
|
||||
## Performance Metrics
|
||||
|
||||
41
.planning/phases/06-quality-frontend/06-01-SUMMARY.md
Normal file
41
.planning/phases/06-quality-frontend/06-01-SUMMARY.md
Normal file
@ -0,0 +1,41 @@
|
||||
# Phase 6 Summary: 质量 & 前端
|
||||
|
||||
**Status:** Complete ✅
|
||||
**Tests:** 146 passed (34 new)
|
||||
**Commit:** feat(06)
|
||||
|
||||
---
|
||||
|
||||
## Plan 01 — QUAL-02: 三平台解析函数测试
|
||||
|
||||
| 文件 | 测试数 | 覆盖函数 |
|
||||
|------|--------|---------|
|
||||
| `tests/ingest/test_configs_boss.py` | 10 | `_extract_job_id`, `_extract_company_name`, `_build_boss_push` |
|
||||
| `tests/ingest/test_configs_qcwy.py` | 12 | `_extract_job_id`, `_extract_update_dt`, `_extract_company_name`, `_build_qcwy_push` |
|
||||
| `tests/ingest/test_configs_zhilian.py` | 12 | `_extract_number`, `_extract_fpt`, `_extract_company_name`, `_build_zhilian_push` |
|
||||
|
||||
**覆盖场景：** 正常字段、缺字段（返回 None/"")、特殊字段（welfare 列表/字符串、skill 列表、整数转字符串）
|
||||
|
||||
---
|
||||
|
||||
## Plan 02 — QUAL-06/07: 前端爬虫监控
|
||||
|
||||
**后端：** `GET /api/v1/job/data/stats?days=7` 端点
|
||||
返回各平台 `total / today / last_ingest_at / daily_counts`（近 N 天每日趋势）
|
||||
|
||||
**前端（`monitor.vue`）：** 新增"爬虫职位入库统计"区域
|
||||
- 三平台卡片：总量 / 今日新增 / 最近入库时间
|
||||
- 近 7 天每日入库趋势表格（Boss / 前程无忧 / 智联招聘 各列）
|
||||
- 随 `refreshAll()` 自动更新，刷新失败静默处理（不影响主清洗面板）
|
||||
|
||||
**QUAL-07 确认已完成：** `monitor.vue` 已有完整清洗队列列表、触发执行、查看结果，无需改动。
|
||||
|
||||
---
|
||||
|
||||
## 测试总量回顾（全项目）
|
||||
|
||||
| Phase | 新增测试 | 累计 |
|
||||
|-------|---------|------|
|
||||
| Phase 1-3 | 106 个 | 106 |
|
||||
| Phase 5 | 6 个 dedup | 112 |
|
||||
| Phase 6 | 34 个解析函数 | **146** |
|
||||
Loading…
x
Reference in New Issue
Block a user