Skip to content

Make pg_gpu pip-installable from PyPI#116

Merged
andrewkern merged 6 commits into
mainfrom
feature/pip-installable-deps
Jun 8, 2026
Merged

Make pg_gpu pip-installable from PyPI#116
andrewkern merged 6 commits into
mainfrom
feature/pip-installable-deps

Conversation

@andrewkern

Copy link
Copy Markdown
Member

Closes #115.

The easiest fix here is to copy our pixi env info (pixi.toml) to pyproject.toml. Currently pyproject.toml declared no dependencies at all -- every runtime requirement lived only in pixi.toml (by design), so a pip install produced a package that couldn't import.

Changes

  • pyproject.toml: declare the full runtime stack in [project.dependencies], with GPU support (CuPy, kvikio, nvcomp) as hard requirements rather than optional extras. Add [project.optional-dependencies] for dev / docs / moments.
    • Uses cupy-cuda12x[ctk] so the CUDA toolkit headers CuPy needs to JIT-compile its kernels are pulled from PyPI
    • Adds matplotlib, seaborn (imported by the plotting module via init) and tskit (used directly by HaplotypeMatrix)
    • Fixes the Homepage URL to the kr-colab org.
  • pixi.toml: pyproject is now the single source of truth for the PyPI/GPU stack; the duplicated deps are removed and come transitively via the editable install. Dropped the NVIDIA extra-index-url (kvikio-cu12 / nvidia-nvcomp-cu12 are now on the default PyPI index). Added conda cuda-cudart-dev so CuPy's NVRTC compilation finds toolkit headers inside the pixi/conda environment too.
  • Docs: README and docs/source/installation.rst gain an "Installation into an Existing Environment" section; pixi remains the recommended
    path.

Install paths

  # Recommended (pinned, reproducible)
  pixi install && pixi shell

  # Into an existing conda/venv environment
  pip install "git+https://github.com/kr-colab/pg_gpu.git"

@andrewkern andrewkern requested a review from mufernando June 5, 2026 00:27
@andrewkern

Copy link
Copy Markdown
Member Author

@mufernando will you try this branch out on your setup?

@nspope

nspope commented Jun 5, 2026

Copy link
Copy Markdown
Collaborator

Why are we pinned to python <3.13? This is going to be annoying when integrating pg_gpu into existing pipelines/environments. Is it because of CUDA toolchain or kvikio? Glancing through the lock file, it's not obvious to me where the constraint is.

@andrewkern

Copy link
Copy Markdown
Member Author

ah good catch. we're not in capping the install in pyproject.toml. this is vestigal and i'll lift it.

@andrewkern

Copy link
Copy Markdown
Member Author

also -- i'm 95% confident everything (including the cuda stack) installs via pip cleanly, so we can publish to pypi

@nspope

nspope commented Jun 8, 2026

Copy link
Copy Markdown
Collaborator

installs and imports fine for me with

python3 -m pip install "git+https://github.com/kr-colab/pg_gpu.git@feature/pip-installable-deps"
python3 -c "import pg_gpu"

(whereas main branch fails as expected on import)

@andrewkern

Copy link
Copy Markdown
Member Author

excellent. when we merge this we can publish to pypi

@mufernando mufernando left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry for the delay! it works for me as well. thanks, @andrewkern!

@andrewkern andrewkern merged commit df875df into main Jun 8, 2026
1 check passed
@andrewkern andrewkern deleted the feature/pip-installable-deps branch June 8, 2026 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Integrating pg-gpu into existing workflows

3 participants