Skip to content

Commit 5ffea86

Browse files
danfimovdanfimov
authored
feat: add main result backends (#1)
feat: add main result backends
2 parents 390cee7 + 76b70ba commit 5ffea86

File tree

22 files changed

+2232
-1
lines changed

22 files changed

+2232
-1
lines changed

‎.github/workflows/code-check.yml‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
name: Tests
2+
on: [pull_request]
3+
4+
jobs:
5+
test:
6+
name: Tests
7+
runs-on: ubuntu-latest
8+
strategy:
9+
matrix:
10+
python-version: [ "3.9", "3.10", "3.11", "3.12", "3.13" ]
11+
fail-fast: true
12+
# TODO: Uncomment when we have a tests with database
13+
# services:
14+
# postgres:
15+
# image: postgres:16
16+
# env:
17+
# POSTGRES_USER: taskiq_pg
18+
# POSTGRES_PASSWORD: taskiq_pg
19+
# POSTGRES_DB: taskiq_pg
20+
# options: >-
21+
# --health-cmd pg_isready
22+
# --health-interval 10s
23+
# --health-timeout 5s
24+
# --health-retries 5
25+
# ports:
26+
# - 5432:5432
27+
steps:
28+
- uses: actions/checkout@v4
29+
- id: setup-uv
30+
uses: astral-sh/setup-uv@v6
31+
with:
32+
enable-cache: true
33+
python-version: ${{ matrix.python-version }}
34+
- name: Install Dependencies
35+
if: steps.setup-uv.outputs.cache-hit != 'true'
36+
run: uv sync --all-extras
37+
- name: Run tests
38+
run: uv run pytest

‎.github/workflows/release_pypi.yml‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
name: Release
2+
3+
on:
4+
push:
5+
tags:
6+
# Publish on any tag starting with a `v`, e.g. v1.2.3
7+
- v*
8+
9+
jobs:
10+
pypi:
11+
name: Publish to PyPI
12+
runs-on: ubuntu-latest
13+
environment:
14+
name: release
15+
permissions:
16+
id-token: write
17+
steps:
18+
- uses: actions/checkout@v4
19+
- uses: astral-sh/setup-uv@v6
20+
- run: uv build
21+
- run: uv publish --trusted-publishing always

‎README.md‎

Lines changed: 71 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,71 @@
1-
# taskiq-postgres
1+
# taskiq-postgres
2+
3+
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/taskiq-postgres?style=for-the-badge&logo=python)](https://pypi.org/project/taskiq-postgres/)
4+
[![PyPI](https://img.shields.io/pypi/v/taskiq-postgres?style=for-the-badge&logo=pypi)](https://pypi.org/project/taskiq-postgres/)
5+
[![Checks](https://img.shields.io/github/checks-status/danfimov/taskiq-postgres/main?style=for-the-badge&logo=pytest)](https://github.com/danfimov/taskiq-postgres)
6+
7+
<div align="center">
8+
<a href="https://github.com/danfimov/taskiq-postgres/"><img src="assets/logo.png" width=400></a>
9+
<hr/>
10+
</div>
11+
12+
PostgreSQL integration for Taskiq with support for asyncpg, psqlpy and aiopg drivers.
13+
14+
## Installation
15+
16+
Depend on your preferred PostgreSQL driver, you can install this library:
17+
18+
```bash
19+
# with asyncpg
20+
pip install taskiq-postgres[asyncpg]
21+
22+
# with psqlpy
23+
pip install taskiq-postgres[psqlpy]
24+
25+
# with aiopg
26+
pip install taskiq-postgres[aiopg]
27+
```
28+
29+
## Usage
30+
31+
Simple example of usage with asyncpg:
32+
33+
```python
34+
# broker.py
35+
import asyncio
36+
37+
from taskiq_pg.asyncpg import AsyncpgResultBackend, AsyncpgBroker
38+
39+
result_backend = AsyncpgResultBackend(
40+
dsn="postgres://postgres:postgres@localhost:5432/postgres",
41+
)
42+
43+
broker = AsyncpgBroker(
44+
dsn="postgres://postgres:postgres@localhost:5432/postgres",
45+
).with_result_backend(result_backend)
46+
47+
48+
@broker.task
49+
async def best_task_ever() -> None:
50+
"""Solve all problems in the world."""
51+
await asyncio.sleep(5.5)
52+
print("All problems are solved!")
53+
54+
55+
async def main():
56+
await broker.startup()
57+
task = await best_task_ever.kiq()
58+
print(await task.wait_result())
59+
await broker.shutdown()
60+
61+
62+
if __name__ == "__main__":
63+
asyncio.run(main())
64+
```
65+
66+
Your experience with other drivers will be pretty similar. Just change the import statement and that's it.
67+
68+
## Motivation
69+
70+
There are too many libraries for PostgreSQL and Taskiq integration. Although they have different view on interface and different functionality.
71+
To address this issue I created this library with a common interface for most popular PostgreSQL drivers that handle similarity across functionality of result backends and brokers.

‎assets/logo.png‎

3.54 MB
Loading

‎pyproject.toml‎

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
[project]
2+
name = "taskiq-postgres"
3+
version = "0.0.0"
4+
description = "PostgreSQL integration for taskiq"
5+
readme = "README.md"
6+
classifiers = [
7+
"Development Status :: 5 - Production/Stable",
8+
"Intended Audience :: Developers",
9+
"Intended Audience :: Information Technology",
10+
"Framework :: AsyncIO",
11+
"License :: OSI Approved :: MIT License",
12+
"Programming Language :: Python",
13+
"Programming Language :: Python :: 3",
14+
"Programming Language :: Python :: 3 :: Only",
15+
"Programming Language :: Python :: 3.9",
16+
"Programming Language :: Python :: 3.10",
17+
"Programming Language :: Python :: 3.11",
18+
"Programming Language :: Python :: 3.12",
19+
"Programming Language :: Python :: 3.13",
20+
"Programming Language :: Python :: Implementation :: CPython",
21+
"Topic :: Software Development :: Libraries :: Python Modules",
22+
"Topic :: Software Development :: Libraries",
23+
"Topic :: Software Development",
24+
"Topic :: System :: Networking",
25+
"Typing :: Typed",
26+
"Operating System :: OS Independent",
27+
]
28+
keywords = ["taskiq", "tasks", "distributed", "async", "postgresql"]
29+
authors = [
30+
{name = "Anfimov Dima", email = "lovesolaristics@gmail.com"}
31+
]
32+
requires-python = ">=3.9"
33+
dependencies = [
34+
"aiopg>=1.4.0",
35+
"taskiq>=0.11.17",
36+
]
37+
38+
[project.urls]
39+
"Bug Tracker" = "https://github.com/danfimov/taskiq-postgres/issues"
40+
"Repository" = "https://github.com/danfimov/taskiq-postgres/"
41+
42+
[dependency-groups]
43+
aiopg = [
44+
"aiopg>=1.4.0",
45+
]
46+
asyncpg = [
47+
"asyncpg>=0.30.0",
48+
]
49+
psqlpy = [
50+
"psqlpy>=0.10.1",
51+
]
52+
dev = [
53+
"ruff>=0.11.9",
54+
"pytest>=8.3.5",
55+
"pytest-asyncio>=0.26.0",
56+
"mypy>=1.15.0",
57+
"asyncpg-stubs>=0.30.1",
58+
]
59+
60+
[tool.pytest.ini_options]
61+
pythonpath = [
62+
"."
63+
]
64+
asyncio_mode = "auto"
65+
asyncio_default_fixture_loop_scope = "function"
66+
markers = [
67+
"unit: marks unit tests",
68+
"integration: marks tests with real infrastructure env",
69+
"linting: marks tests as linter/static analyzers",
70+
]
71+
72+
[tool.ruff]
73+
line-length = 120
74+
target-version = "py39"
75+
76+
[tool.ruff.lint]
77+
select = ["ALL"]
78+
ignore = [
79+
# TODO: enable this rules
80+
"TRY301",
81+
"PLR0913",
82+
"D401",
83+
"ANN401",
84+
"PERF203",
85+
86+
87+
# boolean args
88+
"FBT001",
89+
"FBT002",
90+
91+
# Docstrings
92+
"D104", # in public package
93+
"D100", # in public module
94+
95+
# Conflicted rules
96+
"D203", # with D211
97+
"D212", # with D213
98+
]
99+
100+
[tool.ruff.lint.per-file-ignores]
101+
"tests/*" = [
102+
# Docstrings
103+
"D103",
104+
"D100",
105+
106+
# Strange rules about boolean args
107+
"FBT001",
108+
"FBT002",
109+
110+
"S101", # assert usage
111+
"S311", # pseudo-random generators are not suitable for cryptographic purposes
112+
]
113+
"tests/test_linting.py" = [
114+
"S603", # subprocess usage
115+
]
116+
117+
[tool.ruff.lint.isort]
118+
known-local-folder = ["taskiq_pg"]
119+
lines-after-imports = 2
120+
121+
[tool.mypy]
122+
check_untyped_defs = true
123+
disallow_incomplete_defs = true
124+
disallow_untyped_defs = true
125+
ignore_missing_imports = false
126+
no_implicit_optional = true

‎src/taskiq_pg/__init__.py‎

Whitespace-only changes.

‎src/taskiq_pg/aiopg/__init__.py‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
from taskiq_pg.aiopg.result_backend import AiopgResultBackend
2+
3+
4+
__all__ = [
5+
"AiopgResultBackend",
6+
]

‎src/taskiq_pg/aiopg/queries.py‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
CREATE_TABLE_QUERY = """
2+
CREATE TABLE IF NOT EXISTS {} (
3+
task_id {} UNIQUE,
4+
result BYTEA
5+
)
6+
"""
7+
8+
CREATE_INDEX_QUERY = """
9+
CREATE INDEX IF NOT EXISTS {}_task_id_idx ON {} USING HASH (task_id)
10+
"""
11+
12+
INSERT_RESULT_QUERY = """
13+
INSERT INTO {} VALUES (%s, %s)
14+
ON CONFLICT (task_id)
15+
DO UPDATE
16+
SET result = %s
17+
"""
18+
19+
IS_RESULT_EXISTS_QUERY = """
20+
SELECT EXISTS(
21+
SELECT 1 FROM {} WHERE task_id = %s
22+
)
23+
"""
24+
25+
SELECT_RESULT_QUERY = """
26+
SELECT result FROM {} WHERE task_id = %s
27+
"""
28+
29+
DELETE_RESULT_QUERY = """
30+
DELETE FROM {} WHERE task_id = %s
31+
"""

0 commit comments

Comments
 (0)