I use git for version controll and tag my releases of scrapy crawlers. Since adding git tags (v1.3.1 or v1_3_3), deploying to scrapyd results in unusable code. The deployment goes through, but every command (e.g. schedule) leads to an error similar to this one:
{"node_name": "spider", "status": "error", "message": "Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.8/dist-packages/scrapyd/runner.py", line 40, in
main()
File "/usr/local/lib/python3.8/dist-packages/scrapyd/runner.py", line 35, in main
with project_environment(project):
File "/usr/lib/python3.8/contextlib.py", line 113, in enter
return next(self.gen)
File "/usr/local/lib/python3.8/dist-packages/scrapyd/runner.py", line 16, in project_environment
version, eggfile = eggstorage.get(project, eggversion)
File "/usr/local/lib/python3.8/dist-packages/scrapyd/eggstorage.py", line 28, in get
version = self.list(project)[-1]
File "/usr/local/lib/python3.8/dist-packages/scrapyd/eggstorage.py", line 37, in list
return sorted(versions, key=LooseVersion)
File "/usr/lib/python3.8/distutils/version.py", line 52, in lt
c = self._cmp(other)
File "/usr/lib/python3.8/distutils/version.py", line 337, in _cmp
if self.version < other.version:
TypeError: '<' not supported between instances of 'int' and 'str'
"}
Here is a posting that seems to be related but without a solution:
https://github.com/scrapy/scrapyd/issues/281
The only resolution I found is to remove the entire project and deploy it again. This of course only works if no spiders are running. Does somebody have an idea on how to fix this?
与恶龙缠斗过久,自身亦成为恶龙；凝视深渊过久,深渊将回以凝视…
