156 lines
4.5 KiB
Markdown
156 lines
4.5 KiB
Markdown
---
|
||
phase: 5
|
||
plan: 2
|
||
wave: 2
|
||
title: "公司招聘信息 channel 区分(DATA-04)+ 推送 API 确认(DATA-02)"
|
||
depends_on:
|
||
- "05-01-PLAN.md"
|
||
files_modified:
|
||
- app/services/ingest/configs/boss.py
|
||
- app/services/ingest/configs/qcwy.py
|
||
- app/services/ingest/configs/zhilian.py
|
||
- app/schemas/ingest.py
|
||
- app/services/company_jobs_sync.py
|
||
autonomous: true
|
||
requirements:
|
||
- DATA-02
|
||
- DATA-04
|
||
---
|
||
|
||
# Phase 5 Plan 02: company_job channel 区分(DATA-04 + DATA-02 确认)
|
||
|
||
## Objective
|
||
|
||
### DATA-02(确认已完成)
|
||
|
||
`app/api/v1/job/job.py` 已有三个推送接收端点:
|
||
- `POST /data/store`(单条)
|
||
- `POST /data/batch`(批量同步)
|
||
- `POST /data/batch-async`(批量异步)
|
||
|
||
全部走 `IngestService`,`channel` 和 `platform` 由调用方在请求体中指定。
|
||
**spiderJobs 只需调用这些 API,DATA-02 不需要后端改动。**
|
||
|
||
### DATA-04(需要修复)
|
||
|
||
`company_jobs_sync.py` 目前调用 `router.store_batch(source, "mini", "job", jobs)`,
|
||
公司职位和搜索职位混用同一个 `channel="mini"`,无法区分来源。
|
||
|
||
**修复:**
|
||
1. `ChannelType` 枚举加 `COMPANY = "company"` 值
|
||
2. 各平台 `ingest/configs/*.py` 注册 `channel="company"` 的配置
|
||
3. `company_jobs_sync.py` 调用改为 `channel="company"`
|
||
|
||
## Must Haves
|
||
|
||
- [ ] `app/schemas/ingest.py` 中 `ChannelType` 枚举加 `COMPANY = "company"`
|
||
- [ ] `ingest/configs/boss.py` 注册 `channel="company", data_type="job"` 配置(共用 boss_job 表)
|
||
- [ ] `ingest/configs/qcwy.py` 同样注册 `channel="company"` 配置
|
||
- [ ] `ingest/configs/zhilian.py` 同样注册 `channel="company"` 配置
|
||
- [ ] `company_jobs_sync.py` 中 `store_batch(source, "mini", "job", jobs)` → `store_batch(source, "company", "job", jobs)`
|
||
- [ ] `pipenv run python -c "from app.services.ingest import IngestService; print(IngestService.get_registry_info())"` 显示 channel=company 条目
|
||
- [ ] 全量回归 `pytest tests/` 无失败
|
||
|
||
---
|
||
|
||
## Wave 2(依赖 Plan 01 完成)
|
||
|
||
### Task 2.1: 更新 ChannelType 枚举
|
||
|
||
<read_first>
|
||
- `app/schemas/ingest.py`(需读取确认当前 ChannelType 定义)
|
||
</read_first>
|
||
|
||
<action>
|
||
在 `ChannelType` 枚举中添加:
|
||
```python
|
||
COMPANY = "company"
|
||
```
|
||
</action>
|
||
|
||
---
|
||
|
||
### Task 2.2: 各平台 configs 注册 channel="company" 配置
|
||
|
||
<read_first>
|
||
- `app/services/ingest/configs/boss.py`(当前有 mini+job、mini+company 两条注册)
|
||
- `app/services/ingest/configs/qcwy.py`
|
||
- `app/services/ingest/configs/zhilian.py`
|
||
</read_first>
|
||
|
||
<action>
|
||
在三个 configs 文件中各添加一条新注册:
|
||
|
||
**boss.py 追加:**
|
||
```python
|
||
register(PlatformConfig(
|
||
platform="boss", channel="company", data_type="job",
|
||
table="boss_job", # 共用同一张表,通过 channel 列区分来源
|
||
dedup_fields=(DedupFieldSpec(column="job_id", extractor=_extract_job_id),),
|
||
))
|
||
```
|
||
|
||
**qcwy.py 追加:**
|
||
```python
|
||
register(PlatformConfig(
|
||
platform="qcwy", channel="company", data_type="job",
|
||
table="qcwy_job",
|
||
dedup_fields=(
|
||
DedupFieldSpec(column="job_id", extractor=_extract_job_id),
|
||
DedupFieldSpec(column="update_date_time", extractor=_extract_update_dt),
|
||
),
|
||
))
|
||
```
|
||
|
||
**zhilian.py 追加:**(需读取 zhilian.py 确认 extractor 函数名)
|
||
```python
|
||
register(PlatformConfig(
|
||
platform="zhilian", channel="company", data_type="job",
|
||
table="zhilian_job",
|
||
dedup_fields=(DedupFieldSpec(column="job_number", extractor=_extract_job_number),),
|
||
))
|
||
```
|
||
</action>
|
||
|
||
---
|
||
|
||
### Task 2.3: 修改 company_jobs_sync.py
|
||
|
||
<action>
|
||
将 `sync_company_jobs()` 中:
|
||
```python
|
||
# 修改前
|
||
store_result = await router.store_batch(source, "mini", "job", jobs)
|
||
|
||
# 修改后
|
||
store_result = await router.store_batch(source, "company", "job", jobs)
|
||
```
|
||
</action>
|
||
|
||
<acceptance_criteria>
|
||
- `grep "company.*job" app/services/ingest/configs/boss.py` 有输出
|
||
- `grep '"company"' app/services/company_jobs_sync.py` 有输出
|
||
- `grep '"mini"' app/services/company_jobs_sync.py` 无输出(全部替换)
|
||
</acceptance_criteria>
|
||
|
||
---
|
||
|
||
## Verification
|
||
|
||
```bash
|
||
# 1. 确认 channel=company 注册存在
|
||
pipenv run python -c "
|
||
from app.services.ingest.configs import boss, qcwy, zhilian # 触发 register()
|
||
from app.services.ingest.registry import list_configs
|
||
company_configs = [c for c in list_configs() if c.channel == 'company']
|
||
print(f'channel=company 配置数: {len(company_configs)}')
|
||
for c in company_configs:
|
||
print(f' {c.platform} / {c.channel} / {c.data_type} → {c.table}')
|
||
assert len(company_configs) >= 3, '❌ 缺少注册配置'
|
||
print('✅ 注册配置完整')
|
||
"
|
||
|
||
# 2. 全量回归
|
||
pipenv run python -m pytest tests/ -v --tb=short
|
||
```
|