Skip to content

Add Jobs Comparison Monitor#2

Open
ogiaquino wants to merge 1 commit into
scrapy-plugins:mainfrom
ogiaquino:main
Open

Add Jobs Comparison Monitor#2
ogiaquino wants to merge 1 commit into
scrapy-plugins:mainfrom
ogiaquino:main

Conversation

@ogiaquino
Copy link
Copy Markdown
Collaborator

No description provided.

# run: >-
# python -m pip install -e .
#
# - name: Publish distribution 📦 to Private PyPI
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a ChatGPT comment?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be alright IMHO.

@ogiaquino please add Adrian and Andrey to help us reviewing this PR

@ogiaquino ogiaquino requested review from Gallaecio and wRAR January 8, 2026 12:39
Comment thread tests/conftest.py
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it intentional that there are two conftest files?

)

__all__ = [
SPIDERMON_JOBS_COMPARISON,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this really work and pass ruff? __all__ is a list of strings

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I guess it works because those constants are strings with the same values as names...

You can also filter which jobs to compare based on their close reason using the
``SPIDERMON_JOBS_COMPARISON_CLOSE_REASONS`` setting. The default value is ``()``,
which doesn't filter any job based on close_reason. To only consider successfully finished jobs,
use ``("finished", ) instead.``
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
use ``("finished", ) instead.``
use ``("finished", )`` instead.

The job that will have all the desired arguments will be processed.
Example {"debug_url": "https://www.google.com"} or {"is_full_crawl": True}
You can enable this filter by setting SPIDERMON_JOBS_COMPARISON_ARGUMENTS_ENABLED as True in the settings.
Otherwise, this filter will not be applied
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Otherwise, this filter will not be applied
Otherwise, this filter will not be applied.

Comment thread tox.ini
@@ -0,0 +1,13 @@
[tox]
envlist = min,py39,py310,py311,py312
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no min env.

Comment on lines +23 to +28
- python-version: "3.9"
tox: min
- python-version: "3.10"
- python-version: "3.11"
- python-version: "3.12"
- python-version: "3.13"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't match the version range declared as supported in ‎README.rst and setup.py, and there is no min env.

Comment thread pyproject.toml
Comment on lines +11 to +12
quote-style = "double"
indent-style = "space"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are the default values.

Comment thread pyproject.toml
@@ -0,0 +1,13 @@
[tool.ruff]
line-length = 88
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the default value.

Comment thread setup.cfg
@@ -0,0 +1,2 @@
[bdist_wheel]
universal = 1
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is specific to Python 2.

Comment thread setup.py
long_description_content_type="text/x-rst",
license="BSD",
packages=find_packages(),
package_data={"zyte_spidermon": ["VERSION"]},
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this file exists.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants