356 lines
13 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

---
phase: 2
plan: 2
wave: 1
title: "新增 Boss HTTP 层 mock 测试"
depends_on:
- "02-01-PLAN.md"
files_modified:
- tests/boss/__init__.py
- tests/boss/test_boss_client.py
autonomous: true
requirements:
- QUAL-03
---
# Phase 2 Plan 02: Boss HTTP 层 mock 测试
## Objective
使用 `unittest.mock.MagicMock` 为 Boss HTTP 层添加 mock 测试,
覆盖正常响应和错误响应两种场景,满足 QUAL-03。
测试文件位置:`tests/boss/test_boss_client.py`
**为什么不用 respx** `crawler_core.HTTPClient` 底层使用 `requests_go`(非 httpx
respx 只拦截 httpx无法适用。使用 `MagicMock` 直接 mock `http_client` 接口,
不依赖网络,测试更快更稳定。
## Must Haves
- [ ] `tests/boss/__init__.py` 存在(空文件即可)
- [ ] `tests/boss/test_boss_client.py` 存在并包含 mock 测试
- [ ] `pytest tests/boss/ -v` 全部通过(无 ImportError无失败
- [ ] 测试组SearchRecJobs正常响应、业务错误、HTTP 非 200
- [ ] 测试组GetBrandDetail正常响应
- [ ] 测试组SearchBrandJobs正常响应、hasMore=True 时 is_end_page=False
- [ ] 测试组GetJobDetail.fetch()(通过 batch 接口,正常合并子请求)
- [ ] 测试组BossClient验证每次请求注入 Traceid 头、mpt/wt2 头)
---
## Wave 1
### Task 2.1: 创建 tests/boss/ 目录和 __init__.py
<read_first>
- `tests/crawler_core/test_boss_sign.py`(参考现有测试风格和 conftest 依赖)
- `conftest.py`(项目根目录,确认 sys.path 设置)
</read_first>
<action>
1. 创建 `tests/boss/__init__.py`,内容为空文件(或仅含 `# tests/boss/`)。
确认 `conftest.py` 已在项目根目录Phase 1 已创建),无需重复创建。
</action>
<acceptance_criteria>
- `test -f tests/boss/__init__.py && echo "OK"` 输出 OK
- `pytest tests/boss/ --collect-only 2>&1 | head -5` 无 "ERROR collecting" 字样
</acceptance_criteria>
---
### Task 2.2: 编写 test_boss_client.py
<read_first>
- `spiderJobs/platforms/boss/api.py`迁移后版本Task 1.2 产物)
- `spiderJobs/platforms/boss/client.py`(迁移后版本)
- `tests/crawler_core/test_boss_sign.py`(参考测试风格)
- `crawler_core/base.py`Result 结构)
</read_first>
<action>
创建 `tests/boss/test_boss_client.py`,内容如下(完整内容,直接写入文件):
```python
"""
Boss 直聘 HTTP 层 mock 测试QUAL-03
使用 unittest.mock.MagicMock 替代真实 HTTP 客户端,
覆盖正常响应和错误响应场景,无网络依赖。
"""
from __future__ import annotations
from unittest.mock import MagicMock, patch, call
import pytest
from spiderJobs.platforms.boss.api import (
SearchRecJobs,
GetBrandDetail,
SearchBrandJobs,
GetJobDetail,
_parse_boss_response,
)
from spiderJobs.platforms.boss.client import BossClient
from crawler_core.base import Result
# ─────────────────────────────────────────────────────────
# 1. _parse_boss_response 纯函数测试
# ─────────────────────────────────────────────────────────
class TestParseBossResponse:
def test_http_error_returns_failure(self):
result = _parse_boss_response(500, {})
assert result.success is False
assert result.status_code == 500
def test_non_dict_raw_returns_failure(self):
result = _parse_boss_response(200, "not a dict")
assert result.success is False
def test_biz_error_code_35_returns_failure(self):
result = _parse_boss_response(200, {"code": 35, "message": "IP地址存在异常"})
assert result.success is False
assert result.status_code == 35
assert "IP" in result.error
def test_joblist_payload_parsed_correctly(self):
raw = {
"code": 0,
"zpData": {
"jobList": [{"title": "Python工程师"}],
"hasMore": True,
},
}
result = _parse_boss_response(200, raw)
assert result.success is True
assert len(result.list) == 1
assert result.list[0]["title"] == "Python工程师"
assert result.is_end_page is False # hasMore=True → is_end_page=False
def test_joblist_no_more_pages(self):
raw = {
"code": 0,
"zpData": {"jobList": [{"title": "测试"}], "hasMore": False},
}
result = _parse_boss_response(200, raw)
assert result.is_end_page is True
def test_detail_payload(self):
raw = {"code": 0, "zpData": {"companyName": "测试公司"}}
result = _parse_boss_response(200, raw)
assert result.success is True
assert result.data == {"companyName": "测试公司"}
# ─────────────────────────────────────────────────────────
# 2. SearchRecJobs
# ─────────────────────────────────────────────────────────
class TestSearchRecJobs:
def _make_mock_client(self, return_value):
mock_client = MagicMock()
mock_client.get.return_value = return_value
return mock_client
def test_search_success(self):
raw = {
"code": 0,
"zpData": {
"jobList": [{"title": "测试职位1"}, {"title": "测试职位2"}],
"hasMore": False,
},
}
searcher = SearchRecJobs(city_code="101010100", client=self._make_mock_client((200, raw)))
result = searcher.search(page_index=1)
assert result.success is True
assert len(result.list) == 2
assert result.is_end_page is True
def test_search_http_error(self):
searcher = SearchRecJobs(client=self._make_mock_client((403, {})))
result = searcher.search(page_index=1)
assert result.success is False
assert result.status_code == 403
def test_search_biz_error(self):
raw = {"code": 35, "message": "IP地址存在异常"}
searcher = SearchRecJobs(client=self._make_mock_client((200, raw)))
result = searcher.search(page_index=1)
assert result.success is False
def test_search_builds_correct_params(self):
mock_client = MagicMock()
mock_client.get.return_value = (200, {"code": 0, "zpData": {"jobList": [], "hasMore": False}})
searcher = SearchRecJobs(city_code="101280600", page_size=10, client=mock_client)
searcher.search(page_index=2)
called_args = mock_client.get.call_args
params = called_args[0][1] if len(called_args[0]) > 1 else called_args[1].get("params", called_args[0][1])
assert params["cityCode"] == "101280600"
assert params["page"] == 2
assert params["pageSize"] == 10
# ─────────────────────────────────────────────────────────
# 3. GetBrandDetail
# ─────────────────────────────────────────────────────────
class TestGetBrandDetail:
def test_fetch_success(self):
mock_client = MagicMock()
mock_client.get.return_value = (200, {
"code": 0,
"zpData": {"brandName": "测试公司", "brandId": "abc123"},
})
fetcher = GetBrandDetail(brand_id="abc123", client=mock_client)
result = fetcher.fetch()
assert result.success is True
assert result.data["brandName"] == "测试公司"
def test_fetch_404(self):
mock_client = MagicMock()
mock_client.get.return_value = (404, {})
fetcher = GetBrandDetail(brand_id="notexist", client=mock_client)
result = fetcher.fetch()
assert result.success is False
assert result.status_code == 404
# ─────────────────────────────────────────────────────────
# 4. SearchBrandJobs
# ─────────────────────────────────────────────────────────
class TestSearchBrandJobs:
def test_search_success_has_more(self):
mock_client = MagicMock()
mock_client.get.return_value = (200, {
"code": 0,
"zpData": {"list": [{"jobName": "测试岗位"}], "hasMore": True},
})
searcher = SearchBrandJobs(brand_id="abc123", client=mock_client)
result = searcher.search(page_index=1)
assert result.success is True
assert len(result.list) == 1
assert result.is_end_page is False
def test_search_success_no_more(self):
mock_client = MagicMock()
mock_client.get.return_value = (200, {
"code": 0,
"zpData": {"list": [], "hasMore": False},
})
searcher = SearchBrandJobs(brand_id="abc123", client=mock_client)
result = searcher.search(page_index=1)
assert result.is_end_page is True
# ─────────────────────────────────────────────────────────
# 5. GetJobDetailbatch 接口路径)
# ─────────────────────────────────────────────────────────
class TestGetJobDetail:
def test_fetch_success_merges_sub_requests(self):
mock_client = MagicMock()
mock_client.batch.return_value = (200, {
"code": 0,
"zpData": {
"/wapi/zpgeek/miniapp/job/detail.json": {
"zpData": {"jobName": "数据工程师"}
},
"/wapi/zpgeek/miniapp/jobdetail/improvement/query.json": {
"zpData": {"tags": ["Python", "大数据"]}
},
},
})
fetcher = GetJobDetail(security_id="sid123", job_id="jid456", client=mock_client)
result = fetcher.fetch()
assert result.success is True
assert result.data["detail"]["jobName"] == "数据工程师"
assert "Python" in result.data["improvement"]["tags"]
def test_fetch_biz_error(self):
mock_client = MagicMock()
mock_client.batch.return_value = (200, {"code": 35, "message": "IP地址存在异常"})
fetcher = GetJobDetail(security_id="sid", job_id="jid", client=mock_client)
result = fetcher.fetch()
assert result.success is False
def test_fetch_exception_handled(self):
mock_client = MagicMock()
mock_client.batch.side_effect = ConnectionError("连接超时")
fetcher = GetJobDetail(security_id="sid", job_id="jid", client=mock_client)
result = fetcher.fetch()
assert result.success is False
assert "连接超时" in result.error
# ─────────────────────────────────────────────────────────
# 6. BossClient — Traceid/mpt/wt2 请求头注入
# ─────────────────────────────────────────────────────────
class TestBossClientHeaders:
def test_get_injects_traceid(self):
"""每次 GET 请求头包含 Traceid"""
client = BossClient(tunnel_proxy=None)
headers = client._boss_headers()
assert "Traceid" in headers
assert headers["Traceid"].startswith("M-W")
def test_traceid_is_unique_per_call(self):
"""每次调用生成不同的 Traceid"""
client = BossClient()
t1 = client._boss_headers()["Traceid"]
t2 = client._boss_headers()["Traceid"]
# 大概率不同(极低碰撞概率可忽略)
assert len(t1) > 10
assert len(t2) > 10
def test_mpt_wt2_in_headers(self):
"""signer 的 mpt/wt2 注入到请求头"""
from crawler_core.boss.sign import BossSign
signer = BossSign(mpt="test_mpt_value", wt2="test_wt2_value")
client = BossClient(signer=signer)
headers = client._boss_headers()
assert headers["mpt"] == "test_mpt_value"
assert headers["wt2"] == "test_wt2_value"
```
</action>
<acceptance_criteria>
- `test -f tests/boss/test_boss_client.py && echo "OK"` 输出 OK
- `pytest tests/boss/ -v 2>&1 | tail -10` 输出包含 "passed",无 "failed" 或 "error"
- `pytest tests/boss/ -v 2>&1 | grep "PASSED" | wc -l` 输出 >= 15至少 15 个测试通过)
- `pytest tests/boss/ -v 2>&1 | grep "import"` 无输出(无 ImportError
</acceptance_criteria>
---
## Verification
```bash
# 运行所有 Boss mock 测试
pytest tests/boss/ -v
# 同时运行 Phase 1 签名测试,确认没有回归
pytest tests/crawler_core/ -v
# 汇总结果
pytest tests/ -v --tb=short
```