Skip to content

Commit a212311

Browse files
committed
feat: one interface for all brokers and result backends
1 parent 056910e commit a212311

File tree

21 files changed

+587
-423
lines changed

21 files changed

+587
-423
lines changed

.github/workflows/code-check.yml

Lines changed: 18 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -11,29 +11,29 @@ jobs:
1111
runs-on: ubuntu-latest
1212
strategy:
1313
matrix:
14-
python-version: [ "3.9", "3.10", "3.11", "3.12", "3.13" ]
14+
python-version: [ "3.10", "3.11", "3.12", "3.13" ]
1515
fail-fast: true
16-
# TODO: Uncomment when we have a tests with database
17-
# services:
18-
# postgres:
19-
# image: postgres:16
20-
# env:
21-
# POSTGRES_USER: taskiq_pg
22-
# POSTGRES_PASSWORD: taskiq_pg
23-
# POSTGRES_DB: taskiq_pg
24-
# options: >-
25-
# --health-cmd pg_isready
26-
# --health-interval 10s
27-
# --health-timeout 5s
28-
# --health-retries 5
29-
# ports:
30-
# - 5432:5432
16+
services:
17+
postgres:
18+
image: postgres:18
19+
env:
20+
POSTGRES_USER: taskiq_postgres
21+
POSTGRES_PASSWORD: look_in_vault
22+
POSTGRES_DB: taskiq_postgres
23+
options: >-
24+
--health-cmd pg_isready
25+
--health-interval 10s
26+
--health-timeout 5s
27+
--health-retries 5
28+
ports:
29+
- 5432:5432
3130
steps:
32-
- uses: actions/checkout@v4
31+
- uses: actions/checkout@v5
3332
- id: setup-uv
34-
uses: astral-sh/setup-uv@v6
33+
uses: astral-sh/setup-uv@v7
3534
with:
3635
enable-cache: true
36+
version: "latest"
3737
python-version: ${{ matrix.python-version }}
3838
- name: Install Dependencies
3939
if: steps.setup-uv.outputs.cache-hit != 'true'

.github/workflows/release_docs.yml

Lines changed: 27 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,45 @@
11
name: Release docs
22
on:
3-
push:
4-
branches: [ main ]
3+
workflow_dispatch:
4+
inputs:
5+
pr:
6+
description: Pull request number to release docs for
7+
required: true
8+
type: string
9+
510
pull_request:
11+
types: [closed]
612
branches: [ main ]
713

814
permissions:
915
contents: write
1016

1117
jobs:
1218
deploy:
19+
if: ${{ github.event_name == 'workflow_dispatch' || (github.event_name == 'pull_request' && github.event.pull_request.merged == true) }}
1320
runs-on: ubuntu-latest
1421
steps:
15-
- uses: actions/checkout@v4
22+
- name: Checkout merge commit (auto on merge to main)
23+
if: ${{ github.event_name == 'pull_request' }}
24+
uses: actions/checkout@v5
25+
with:
26+
ref: ${{ github.event.pull_request.merge_commit_sha }}
27+
28+
- name: Checkout PR merge ref (manual on PR)
29+
if: ${{ github.event_name == 'workflow_dispatch' }}
30+
uses: actions/checkout@v5
31+
with:
32+
ref: refs/pull/${{ inputs.pr }}/merge
33+
1634
- name: Configure Git Credentials
1735
run: |
1836
git config user.name github-actions[bot]
1937
git config user.email 41898282+github-actions[bot]@users.noreply.github.com
20-
- uses: astral-sh/setup-uv@v6
38+
39+
- uses: astral-sh/setup-uv@v7
40+
with:
41+
enable-cache: true
42+
python-version: "3.12"
43+
version: "latest"
2144
- run: uv sync --only-dev
2245
- run: uv run mkdocs gh-deploy --force

.github/workflows/release_pypi.yml

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,12 @@ jobs:
1515
permissions:
1616
id-token: write
1717
steps:
18-
- uses: actions/checkout@v4
19-
- uses: astral-sh/setup-uv@v6
18+
- uses: actions/checkout@v5
19+
- uses: astral-sh/setup-uv@v7
20+
with:
21+
enable-cache: true
22+
python-version: "3.12"
23+
version: "latest"
2024
- run: uv version "${{ github.ref_name }}"
2125
- run: uv build
2226
- run: uv publish --trusted-publishing always

Makefile

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,8 +35,8 @@ run_infra: ## Run rabbitmq in docker for integration tests
3535

3636
.PHONY: lint
3737
lint: ## Run linting
38-
@uv run ruff check .
39-
@uv run mypy
38+
@uv run ruff check src tests
39+
@uv run mypy src
4040

4141
.PHONY: format
4242
format: ## Run formatting

README.md

Lines changed: 19 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -9,26 +9,34 @@
99

1010
PostgreSQL integration for Taskiq with support for asyncpg, psqlpy and aiopg drivers.
1111

12-
See example of usage in documentation.
12+
See more example of usage in [the documentation](https://danfimov.github.io/taskiq-postgres/).
1313

1414
## Installation
1515

1616
Depend on your preferred PostgreSQL driver, you can install this library:
1717

18-
```bash
19-
# with asyncpg
20-
pip install taskiq-postgres[asyncpg]
18+
=== "asyncpg"
2119

22-
# with psqlpy
23-
pip install taskiq-postgres[psqlpy]
20+
```bash
21+
pip install taskiq-postgres[asyncpg]
22+
```
23+
24+
=== "psqlpy"
25+
26+
```bash
27+
pip install taskiq-postgres[psqlpy]
28+
```
29+
30+
=== "aiopg"
31+
32+
```bash
33+
pip install taskiq-postgres[aiopg]
34+
```
2435

25-
# with aiopg
26-
pip install taskiq-postgres[aiopg]
27-
```
2836

29-
## Usage
37+
## Usage example
3038

31-
Simple example of usage with asyncpg:
39+
Simple example of usage with [asyncpg](https://github.com/MagicStack/asyncpg):
3240

3341
```python
3442
# broker.py

docker-compose.yml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
services:
2+
postgres:
3+
container_name: taskiq_postgres
4+
image: postgres:18
5+
environment:
6+
POSTGRES_DB: taskiq_postgres
7+
POSTGRES_USER: taskiq_postgres
8+
POSTGRES_PASSWORD: look_in_vault
9+
ports:
10+
- "5432:5432"

mkdocs.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,15 +39,15 @@ theme:
3939

4040
- media: "(prefers-color-scheme: light)"
4141
scheme: default
42-
primary: blue grey
42+
primary: indigo
4343
accent: blue
4444
toggle:
4545
icon: material/brightness-4
4646
name: Use dark mode
4747

4848
- media: "(prefers-color-scheme: dark)"
4949
scheme: slate
50-
primary: blue grey
50+
primary: indigo
5151
accent: blue
5252
toggle:
5353
icon: material/brightness-7

pyproject.toml

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ asyncpg = [
4646
"asyncpg>=0.30.0",
4747
]
4848
psqlpy = [
49-
"psqlpy>=0.10.1",
49+
"psqlpy>=0.11.6",
5050
]
5151

5252
[dependency-groups]
@@ -59,6 +59,7 @@ dev = [
5959
# tests
6060
"pytest>=8.4.2",
6161
"pytest-asyncio>=1.1.0",
62+
"pytest-cov>=7.0.0",
6263
# pre-commit hooks
6364
"prek>=0.2.4",
6465
# docs
@@ -84,6 +85,24 @@ markers = [
8485
"integration: marks tests with real infrastructure env",
8586
]
8687

88+
89+
[tool.coverage.report]
90+
exclude_lines = [
91+
"# pragma: no cover",
92+
"def __repr__",
93+
"def __str__",
94+
"def __unicode__",
95+
"if tp.TYPE_CHECKING:",
96+
]
97+
show_missing = true
98+
skip_empty = true
99+
omit = [
100+
"*/__init__.py",
101+
"venv/*",
102+
"tests/*"
103+
]
104+
105+
87106
[tool.ruff]
88107
line-length = 120
89108
target-version = "py39"

src/taskiq_pg/_internal/__init__.py

Whitespace-only changes.

src/taskiq_pg/_internal/broker.py

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
from __future__ import annotations
2+
3+
import abc
4+
import typing as tp
5+
6+
from taskiq import AsyncBroker, AsyncResultBackend
7+
8+
9+
if tp.TYPE_CHECKING:
10+
import asyncio
11+
12+
13+
_T = tp.TypeVar("_T")
14+
15+
16+
class BasePostgresBroker(AsyncBroker, abc.ABC):
17+
"""Base class for Postgres brokers."""
18+
19+
def __init__(
20+
self,
21+
dsn: str | tp.Callable[[], str] = "postgresql://postgres:postgres@localhost:5432/postgres",
22+
result_backend: AsyncResultBackend[_T] | None = None,
23+
task_id_generator: tp.Callable[[], str] | None = None,
24+
channel_name: str = "taskiq",
25+
table_name: str = "taskiq_messages",
26+
max_retry_attempts: int = 5,
27+
read_kwargs: dict[str, tp.Any] | None = None,
28+
write_kwargs: dict[str, tp.Any] | None = None,
29+
) -> None:
30+
"""
31+
Construct a new broker.
32+
33+
Args:
34+
dsn: connection string to PostgreSQL, or callable returning one.
35+
result_backend: Custom result backend.
36+
task_id_generator: Custom task_id generator.
37+
channel_name: Name of the channel to listen on.
38+
table_name: Name of the table to store messages.
39+
max_retry_attempts: Maximum number of message processing attempts.
40+
read_kwargs: Additional arguments for read connection creation.
41+
write_kwargs: Additional arguments for write pool creation.
42+
43+
"""
44+
super().__init__(
45+
result_backend=result_backend,
46+
task_id_generator=task_id_generator,
47+
)
48+
self._dsn: str | tp.Callable[[], str] = dsn
49+
self.channel_name: str = channel_name
50+
self.table_name: str = table_name
51+
self.read_kwargs: dict[str, tp.Any] = read_kwargs or {}
52+
self.write_kwargs: dict[str, tp.Any] = write_kwargs or {}
53+
self.max_retry_attempts: int = max_retry_attempts
54+
self._queue: asyncio.Queue[str] | None = None
55+
56+
@property
57+
def dsn(self) -> str:
58+
"""
59+
Get the DSN string.
60+
61+
Returns:
62+
A string with dsn or None if dsn isn't set yet.
63+
64+
"""
65+
if callable(self._dsn):
66+
return self._dsn()
67+
return self._dsn

0 commit comments

Comments
 (0)