Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,11 +115,11 @@ following steps in advance to help us fix any potential bug as fast as possible.
We use GitHub issues to track bugs and errors. If you run into an issue with the project:

- Open an [Issue](https://github.com/django-commons/django-tasks-scheduler/issues/new).
(Since we can't be sure at this point whether it is a bug or not, we ask you not to talk about a bug yet and
(Since we can't be sure at this point whether it is a bug or not, we ask you not to talk about a bug yet and
not to label the issue.)
- Explain the behavior you would expect and the actual behavior.
- Please provide as much context as possible and describe the *reproduction steps* that someone else can follow to
recreate the issue on their own. This usually includes your code.
recreate the issue on their own. This usually includes your code.
For good bug reports, you should isolate the problem and create a reduced test case.
- Provide the information you collected in the previous section.

Expand Down Expand Up @@ -228,7 +228,7 @@ contextual information and is contained within parenthesis, e.g., feat(parser):

## Join The Project Team

If you wish to be added to the project team as a collaborator, please create an issue
If you wish to be added to the project team as a collaborator, please create an issue
explaining why you wish to join the team and tag @cunla in the issue.

<!-- omit in toc -->
Expand Down
2 changes: 1 addition & 1 deletion .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

version: 2
updates:
- package-ecosystem: "pip"
- package-ecosystem: "pip"
directory: "/"
schedule:
interval: "daily"
2 changes: 1 addition & 1 deletion .github/release-drafter.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,4 +28,4 @@ version-resolver:
template: |
## Changes

$CHANGES
$CHANGES
2 changes: 1 addition & 1 deletion .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -78,4 +78,4 @@ jobs:
uses: pypa/gh-action-pypi-publish@v1.13.0
with:
repository-url: https://test.pypi.org/legacy/
skip-existing: true
skip-existing: true
2 changes: 1 addition & 1 deletion .github/workflows/test-workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,4 +43,4 @@ jobs:
echo "reason: ${{ github.event.inputs.reason }}"
echo "environment-optional: ${{ github.event.inputs.environment-optional }}"
echo "choices-input-optional: ${{ github.event.inputs.choices-input-optional }}"
echo "number-input-optional: ${{ github.event.inputs.number-input-optional }}"
echo "number-input-optional: ${{ github.event.inputs.number-input-optional }}"
2 changes: 1 addition & 1 deletion .github/zizmor.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,4 @@ rules:
github-env:
ignore:
- 'action.yml:36:7'
- 'action.yml:28:7'
- 'action.yml:28:7'
15 changes: 15 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v6.0.0
hooks:
- id: check-yaml
- id: check-json
- id: check-xml
- id: end-of-file-fixer
- id: trailing-whitespace
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.14.0
hooks:
- id: ruff-check
args: [--fix]
- id: ruff-format
9 changes: 8 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ from typing import Dict
from scheduler.types import SchedulerConfiguration, Broker, QueueConfiguration

INSTALLED_APPS = [
# ...
# ...
'scheduler',
# ...
]
Expand Down Expand Up @@ -67,6 +67,13 @@ python manage.py migrate
4. Check out the admin views:
![](./docs/media/admin-tasks-list.jpg)


# Local development environment

You can install [`pre-commit` hook](https://pre-commit.com/) in the repo to add it as a git hook by
running: `pre-commit install`. It is configured to check all change files based on configuration in
`.pre-commit-config.yaml`.

# Sponsor

django-tasks-scheduler is developed for free.
Expand Down
2 changes: 1 addition & 1 deletion SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@

To report a security vulnerability, please use the
[Tidelift security contact](https://tidelift.com/security).
Tidelift will coordinate the fix and disclosure.
Tidelift will coordinate the fix and disclosure.
2 changes: 1 addition & 1 deletion docs/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -300,4 +300,4 @@ Settings:
### 🐛 Bug Fixes

### 🧰 Maintenance
-->
-->
4 changes: 2 additions & 2 deletions docs/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ python manage.py run_job {callable} {callable args ...}
Run this to empty failed jobs registry from a queue.

```shell
python manage.py delete_failed_jobs
python manage.py delete_failed_jobs
```

## `scheduler_stats` - Show scheduler stats
Expand Down Expand Up @@ -157,4 +157,4 @@ options:
--force-color Force colorization of the command output.
--skip-checks Skip system checks.

```
```
30 changes: 15 additions & 15 deletions docs/drt-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,24 +6,24 @@ Running `python manage.py scheduler_worker --name 'X' --queues high default low`
```mermaid
sequenceDiagram
autonumber

participant worker as WorkerProcess
participant qlist as QueueHash<br/>name -> key

participant qlist as QueueHash<br/>name -> key
participant wlist as WorkerList
participant wkey as WorkerKey
participant queue as QueueKey
participant job as JobHash


note over worker,qlist: Checking sanity

break when a queue-name in the args is not in queue-list
worker ->>+ qlist: Query queue names
qlist -->>- worker: All queue names
worker ->> worker: check that queue names exists in the system
end

note over worker,wkey: register
worker ->> wkey: Create workerKey with all info (new id, queues, status)
worker ->> wlist: Add new worker to list, last heartbeat set to now()
Expand All @@ -34,37 +34,37 @@ sequenceDiagram
```mermaid
sequenceDiagram
autonumber

participant worker as WorkerProcess
participant qlist as QueueHash<br/>name -> key

participant qlist as QueueHash<br/>name -> key
participant wlist as WorkerList
participant wkey as WorkerKey
participant queue as QueueKey
participant job as JobHash

loop Until death
worker ->> wlist: Update last heartbeat
note over worker,job: Find next job

loop over queueKeys until job to run is found or all queues are empty
worker ->>+ queue: get next job name and remove it or None (zrange+zpop)
queue -->>- worker: job name / nothing
end

note over worker,job: Execute job or sleep
critical [job is found]
worker ->> wkey: Update worker status to busy
worker ->>+ job: query job data
job -->>- worker: job data

worker ->> job: update job status to running
worker ->> worker: execute job
worker ->> job: update job status to done/failed
worker ->> wkey: Update worker status to idle
option No job pending
worker ->> worker: sleep
end
worker ->> worker: sleep
end
end
```

Expand Down
2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,4 +226,4 @@ Please report issues via [GitHub Issues][issues] .

[django-rq]:https://github.com/rq/django-rq

[rq]:https://github.com/rq/rq
[rq]:https://github.com/rq/rq
9 changes: 4 additions & 5 deletions docs/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
2. In `settings.py`, add `scheduler` to `INSTALLED_APPS`:
```python
INSTALLED_APPS = [
# ...
# ...
'scheduler',
# ...
]
Expand All @@ -22,7 +22,7 @@
import os
from typing import Dict
from scheduler.types import QueueConfiguration

SCHEDULER_QUEUES: Dict[str, QueueConfiguration] = {
'default': QueueConfiguration(
HOST='localhost',
Expand All @@ -47,7 +47,7 @@
'password': 'secret',
}),
'high': QueueConfiguration(URL=os.getenv('REDISTOGO_URL', 'redis://localhost:6379/0')),
'low': QueueConfiguration(HOST='localhost', PORT=6379, DB=0, ASYNC=False),
'low': QueueConfiguration(HOST='localhost', PORT=6379, DB=0, ASYNC=False),
}
```

Expand Down Expand Up @@ -76,7 +76,7 @@
5. Add `scheduler.urls` to your django application `urls.py`:
```python
from django.urls import path, include

urlpatterns = [
# ...
path('scheduler/', include('scheduler.urls')),
Expand All @@ -87,4 +87,3 @@
```shell
python manage.py migrate
```

2 changes: 1 addition & 1 deletion docs/migrate_to_v3.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,4 +33,4 @@ python manage.py export > scheduled_tasks.json
python manage.py import --filename scheduled_tasks.json
```

[issues]: https://github.com/django-commons/django-tasks-scheduler/issues
[issues]: https://github.com/django-commons/django-tasks-scheduler/issues
8 changes: 4 additions & 4 deletions docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ It is possible to view list of executions of a task, as well as the details of a
* In the Repeat field, enter the number of times the job is to be run. Leaving the field empty, means the job
will be scheduled to run forever.
* In the cron string field, enter a cron string describing how often the job should run.
* In the **Callable** field, enter a Python dot notation path to the method that defines the job. For the example
* In the **Callable** field, enter a Python dot notation path to the method that defines the job. For the example
above, that would be `myapp.jobs.count`
* Choose your **Queue**.
The queues listed are defined in your app `settings.py` under `SCHEDULER_QUEUES`.
Expand Down Expand Up @@ -159,7 +159,7 @@ WorkingDirectory = {{ path_to_your_project_folder } }
ExecStart = /home/ubuntu/.virtualenv/{ { your_virtualenv } }/bin/python \
{{ path_to_your_project_folder } }/manage.py \
scheduler_worker high default low
# Optional
# Optional
# {{user to run scheduler_worker as}}
User = ubuntu
# {{group to run scheduler_worker as}}
Expand All @@ -180,11 +180,11 @@ After you are done editing the file, reload the settings and start the new worke

```shell
sudo systemctl daemon-reload
sudo systemctl start scheduler_worker@{1..3}
sudo systemctl start scheduler_worker@{1..3}
```

You can target a specific worker using its number:

```shell
sudo systemctl stop scheduler_worker@2
```
```
1 change: 0 additions & 1 deletion mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -107,4 +107,3 @@ nav:
- Usage: usage.md
- Management commands: commands.md
- Change log: changelog.md

1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ Funding = "https://github.com/sponsors/cunla"

[dependency-groups]
dev = [
"pre-commit>=4.2",
"time-machine>=2.19",
"ruff>=0.13",
"coverage[toml]>=7.10",
Expand Down
6 changes: 1 addition & 5 deletions scheduler/settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,7 @@
from django.core.exceptions import ImproperlyConfigured

from scheduler.types import SchedulerConfiguration, QueueConfiguration

try:
from annotationlib import get_annotations
except ImportError:
from typing_extensions import get_annotations
from typing_extensions import get_annotations

logger = logging.getLogger("scheduler")

Expand Down
2 changes: 1 addition & 1 deletion scheduler/static/admin/js/select-fields.js
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,4 @@
toggleVerified($(this).val());
});
});
})(django.jQuery);
})(django.jQuery);
2 changes: 1 addition & 1 deletion scheduler/templates/admin/scheduler/change_form.html
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@

{% block after_related_objects %}
{% include 'admin/scheduler/jobs-list.partial.html' %}
{% endblock %}
{% endblock %}
2 changes: 1 addition & 1 deletion scheduler/templates/admin/scheduler/change_list.html
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@

{% block object-tools %}
{{ block.super }}
{% endblock %}
{% endblock %}
Original file line number Diff line number Diff line change
Expand Up @@ -58,4 +58,4 @@ <h2>Job executions</h2>
<span id="counter">{{ executions.paginator.count }} {% blocktranslate count counter=executions.paginator.count %}entry{% plural %}entries{% endblocktranslate %}</span>
</p>
</fieldset>
{% endif %}
{% endif %}
2 changes: 1 addition & 1 deletion scheduler/templates/admin/scheduler/jobs-list.partial.html
Original file line number Diff line number Diff line change
Expand Up @@ -55,4 +55,4 @@ <h2>Job executions</h2>
<span id="counter">{{ executions.paginator.count }} {% blocktranslate count counter=executions.paginator.count %}entry{% plural %}entries{% endblocktranslate %}</span>
</p>
</fieldset>
{% endif %}
{% endif %}
2 changes: 1 addition & 1 deletion scheduler/templates/admin/scheduler/scheduler_base.html
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,4 @@
{% block extrahead %}
{{ block.super }}
<script type="text/javascript" src="{% static "admin/js/actions.js" %}"></script>
{% endblock %}
{% endblock %}
Original file line number Diff line number Diff line change
Expand Up @@ -62,4 +62,4 @@
{% endfor %}
</tbody>
</table>
</div>
</div>
4 changes: 2 additions & 2 deletions scheduler/types/settings_types.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import sys
from dataclasses import dataclass
from enum import Enum
from typing import Callable, Dict, Optional, List, Tuple, Any, Type, ClassVar
from typing import Callable, Dict, Optional, List, Tuple, Any, Type, ClassVar, Set

from scheduler.helpers.timeouts import BaseDeathPenalty, UnixSignalDeathPenalty

Expand Down Expand Up @@ -45,7 +45,7 @@ class SchedulerConfiguration:

@dataclass(slots=True, frozen=True, kw_only=True)
class QueueConfiguration:
__CONNECTION_FIELDS__: ClassVar[Dict] = {
__CONNECTION_FIELDS__: ClassVar[Set[str]] = {
"URL",
"DB",
"UNIX_SOCKET_PATH",
Expand Down
Loading
Loading