- ARCH-04: job51 migrated to crawler_core (no old deps) - ARCH-05: zhilian migrated to crawler_core (no old deps) - 34 new mock tests (17 job51 + 17 zhilian) - Added _parse_zhilian_response custom parser for zhilian API format - Fixed POST Searcher _request() overrides for job51/zhilian - Full regression: 98 passed in 0.12s
1.4 KiB
1.4 KiB
Plan 03-02 Summary: 迁移智联招聘(zhilian)层至 crawler_core
Status: Complete
Tasks: 6/6
Commit: 8c2c2d2
What was built
将 spiderJobs/platforms/zhilian/ 4 个文件迁移至 crawler_core:
- client.py:HTTPClient 改用
crawler_core.http_client,ZhilianSign 改用crawler_core.zhilian.sign - api.py:
- import 改为
crawler_core.base.BaseFetcher/BaseSearcher/Result/parse_response - 添加
_parse_zhilian_response(HTTP 200 即成功,适配智联无 statusCode 的响应格式) - 1 处
self._http.get(→self.http_client.get((SearchCompanyPositions) - 为所有 Fetcher/Searcher 类补充
_parse()方法(crawler_core 要求的抽象方法) - 为 POST Searcher(SearchPositions)添加
_request()POST 覆写
- import 改为
- main.py:BaseFetcher/BaseSearcher 改用
crawler_core.base - sign.py:向后兼容桩,重新导出
crawler_core.zhilian.sign.ZhilianSign - tests/zhilian/test_zhilian_client.py:17 个 mock 测试
Key Decisions
- 添加了
_parse_zhilian_response而非使用parse_response(后者检查statusCode==200,但智联接口不返回此字段) SearchCompanyPositions._build_params()通过self._client.signer.sign_params()调用 signer,迁移后不受影响
Verification
pytest tests/zhilian/ -v → 17 passed ✅