156 lines
4.5 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

---
phase: 5
plan: 2
wave: 2
title: "公司招聘信息 channel 区分DATA-04+ 推送 API 确认DATA-02"
depends_on:
- "05-01-PLAN.md"
files_modified:
- app/services/ingest/configs/boss.py
- app/services/ingest/configs/qcwy.py
- app/services/ingest/configs/zhilian.py
- app/schemas/ingest.py
- app/services/company_jobs_sync.py
autonomous: true
requirements:
- DATA-02
- DATA-04
---
# Phase 5 Plan 02: company_job channel 区分DATA-04 + DATA-02 确认)
## Objective
### DATA-02确认已完成
`app/api/v1/job/job.py` 已有三个推送接收端点:
- `POST /data/store`(单条)
- `POST /data/batch`(批量同步)
- `POST /data/batch-async`(批量异步)
全部走 `IngestService``channel``platform` 由调用方在请求体中指定。
**spiderJobs 只需调用这些 APIDATA-02 不需要后端改动。**
### DATA-04需要修复
`company_jobs_sync.py` 目前调用 `router.store_batch(source, "mini", "job", jobs)`
公司职位和搜索职位混用同一个 `channel="mini"`,无法区分来源。
**修复:**
1. `ChannelType` 枚举加 `COMPANY = "company"`
2. 各平台 `ingest/configs/*.py` 注册 `channel="company"` 的配置
3. `company_jobs_sync.py` 调用改为 `channel="company"`
## Must Haves
- [ ] `app/schemas/ingest.py``ChannelType` 枚举加 `COMPANY = "company"`
- [ ] `ingest/configs/boss.py` 注册 `channel="company", data_type="job"` 配置(共用 boss_job 表)
- [ ] `ingest/configs/qcwy.py` 同样注册 `channel="company"` 配置
- [ ] `ingest/configs/zhilian.py` 同样注册 `channel="company"` 配置
- [ ] `company_jobs_sync.py``store_batch(source, "mini", "job", jobs)``store_batch(source, "company", "job", jobs)`
- [ ] `pipenv run python -c "from app.services.ingest import IngestService; print(IngestService.get_registry_info())"` 显示 channel=company 条目
- [ ] 全量回归 `pytest tests/` 无失败
---
## Wave 2依赖 Plan 01 完成)
### Task 2.1: 更新 ChannelType 枚举
<read_first>
- `app/schemas/ingest.py`(需读取确认当前 ChannelType 定义)
</read_first>
<action>
`ChannelType` 枚举中添加:
```python
COMPANY = "company"
```
</action>
---
### Task 2.2: 各平台 configs 注册 channel="company" 配置
<read_first>
- `app/services/ingest/configs/boss.py`(当前有 mini+job、mini+company 两条注册)
- `app/services/ingest/configs/qcwy.py`
- `app/services/ingest/configs/zhilian.py`
</read_first>
<action>
在三个 configs 文件中各添加一条新注册:
**boss.py 追加:**
```python
register(PlatformConfig(
platform="boss", channel="company", data_type="job",
table="boss_job", # 共用同一张表,通过 channel 列区分来源
dedup_fields=(DedupFieldSpec(column="job_id", extractor=_extract_job_id),),
))
```
**qcwy.py 追加:**
```python
register(PlatformConfig(
platform="qcwy", channel="company", data_type="job",
table="qcwy_job",
dedup_fields=(
DedupFieldSpec(column="job_id", extractor=_extract_job_id),
DedupFieldSpec(column="update_date_time", extractor=_extract_update_dt),
),
))
```
**zhilian.py 追加:**(需读取 zhilian.py 确认 extractor 函数名)
```python
register(PlatformConfig(
platform="zhilian", channel="company", data_type="job",
table="zhilian_job",
dedup_fields=(DedupFieldSpec(column="job_number", extractor=_extract_job_number),),
))
```
</action>
---
### Task 2.3: 修改 company_jobs_sync.py
<action>
`sync_company_jobs()` 中:
```python
# 修改前
store_result = await router.store_batch(source, "mini", "job", jobs)
# 修改后
store_result = await router.store_batch(source, "company", "job", jobs)
```
</action>
<acceptance_criteria>
- `grep "company.*job" app/services/ingest/configs/boss.py` 有输出
- `grep '"company"' app/services/company_jobs_sync.py` 有输出
- `grep '"mini"' app/services/company_jobs_sync.py` 无输出(全部替换)
</acceptance_criteria>
---
## Verification
```bash
# 1. 确认 channel=company 注册存在
pipenv run python -c "
from app.services.ingest.configs import boss, qcwy, zhilian # 触发 register()
from app.services.ingest.registry import list_configs
company_configs = [c for c in list_configs() if c.channel == 'company']
print(f'channel=company 配置数: {len(company_configs)}')
for c in company_configs:
print(f' {c.platform} / {c.channel} / {c.data_type} → {c.table}')
assert len(company_configs) >= 3, '❌ 缺少注册配置'
print('✅ 注册配置完整')
"
# 2. 全量回归
pipenv run python -m pytest tests/ -v --tb=short
```