From cbb1ddb01c17e015296cd1733ec446f44263c287 Mon Sep 17 00:00:00 2001 From: Pramod Gupta Date: Sat, 16 May 2026 19:22:07 -0600 Subject: [PATCH 01/13] edit README.md --- README.md | 35 ++++++++++++++++++++--------------- 1 file changed, 20 insertions(+), 15 deletions(-) diff --git a/README.md b/README.md index 936d3d9..fbcdd6c 100644 --- a/README.md +++ b/README.md @@ -16,13 +16,18 @@ First, follow the [uv installation instructions](https://docs.astral.sh/uv/getti It is recommended to work within an isolated virtual environment like `conda` or `venv`. You can create a uv venv with `uv venv valis --python 3.12`. See [uv Environments](https://docs.astral.sh/uv/pip/environments/) for more info. -### Developer Install +### valis Developer Install ``` git clone https://github.com/sdss/valis valis cd valis uv pip install ``` +### datamodel Developer Install +git clone git@github.com:sdss/datamodel.git +cd datamodel +pip install -e ".[test]" + ### Solara Dependencies The default install does not install any of the Solara dependencies for Jdaviz and the SDSS DataView Explorer. To install these dependencies, run @@ -47,16 +52,6 @@ To install new packages and add them to the `pyproject.toml` and `uv.lock` files uv add [package] ``` -### Local Development - -To run a local instance for development, run the following from the top level of the `valis` repo. -``` -uvicorn valis.wsgi:app --reload -``` -This will start a local web server at `http://localhost:8000/valis/`. The API documentation will be located at `http://localhost:8000/valis/docs`. Or to see the alternate documentation, go to `http://localhost:8000/valis/redoc/` - -By default, the app will try to cache some route responses to a Redis database in localhost. If you don't have a Redis instance running you can use `in-memory` for testing (this caches the response directly in RAM). To do so, edit `~/.config/sdss/valis.yaml` and add `cache_backend: in-memory` (this should only be used in development or it could quickly use all available memory; the memory is freed when the app is stopped). Caching can be completely disabled by setting `cache_backend: null`. The time the cache is kept can be set with the `cache_ttl` (time to live) setting option. - ### Database Connection Valis uses the `sdssdb` package for all connections to databases. The most relevant database for the API is the `sdss5db` on `pipelines.sdss.org`. The easiest way to connect is through a local SSH tunnel. To set up a tunnel, @@ -70,19 +65,19 @@ Host pipe ForwardX11Trusted yes ProxyCommand ssh -A [unid]@mwm.sdss.org nc %h %p ``` -2. In a terminal, create an ssh tunnel to the pipelines database localhost port 5432, to a some local port. E.g. this maps the remote db localhost port 5432 to local machine on port 6000. +1. In a terminal, create an ssh tunnel to the pipelines database localhost port 5432, to a some local port. E.g. this maps the remote db localhost port 5432 to local machine on port 6000. ``` ssh -L 6000:localhost:5432 pipe ``` -3. Update your `~/.pgpass` file with the following lines. Replace `port`, `unid`, and `password`, with your tunneled port (e.g. 6000 in step 2), Utah unid (e.g. u1234567), and db password, respectively. +2. Update your `~/.pgpass` file with the following lines. Replace `port`, `unid`, and `password`, with your tunneled port (e.g. 6000 in step 1), Utah unid (e.g. u1234567), and db password, respectively. ``` localhost:[port]:*:[unid]:[password] host.docker.internal:[port]:*:[unid]:[password] ``` -Alternatively, if you don't want to edit your .pgpass file then set the VALIS_DB_PASS environment variable with your database password. See step 4. +If for some reason, you do not want to edit your .pgpass file then set the VALIS_DB_PASS environment variable with your database password. This is shown in step 3. -4. Set the following environment variables. +3. Set the following environment variables. - export VALIS_DB_PORT=6000 - export VALIS_DB_USER={unid} @@ -99,6 +94,16 @@ db_user: {unid} Additionally, you can set the environment variable `VALIS_DB_RESET=false` or add `db_reset: false` to `valis.yaml`. This will prevent the DB connection to be closed after a query completes and should speed up new queries. This setting should not be used in production. +### Local Development + +To run a local instance for development, run the following from the top level of the `valis` repo. +``` +uvicorn valis.wsgi:app --reload +``` +This will start a local web server at `http://localhost:8000/valis/`. The API documentation will be located at `http://localhost:8000/valis/docs`. Or to see the alternate documentation, go to `http://localhost:8000/valis/redoc/` + +By default, the app will try to cache some route responses to a Redis database in localhost. If you don't have a Redis instance running you can use `in-memory` for testing (this caches the response directly in RAM). To do so, edit `~/.config/sdss/valis.yaml` and add `cache_backend: in-memory` (this should only be used in development or it could quickly use all available memory; the memory is freed when the app is stopped). Caching can be completely disabled by setting `cache_backend: null`. The time the cache is kept can be set with the `cache_ttl` (time to live) setting option. + ## Deployment This section describes a variety of deployment methods. Valis uses gunicorn as its From 17b2af5877d05dc62334c73c74bf116abcc8abf8 Mon Sep 17 00:00:00 2001 From: Pramod Gupta Date: Sat, 16 May 2026 19:24:16 -0600 Subject: [PATCH 02/13] edit README.md --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index fbcdd6c..51aefa2 100644 --- a/README.md +++ b/README.md @@ -24,9 +24,11 @@ uv pip install ``` ### datamodel Developer Install +``` git clone git@github.com:sdss/datamodel.git cd datamodel pip install -e ".[test]" +``` ### Solara Dependencies From c7cf4ca2e320fd63154bf1bb9dca0c37cfd82e24 Mon Sep 17 00:00:00 2001 From: Pramod Gupta Date: Sat, 16 May 2026 21:14:02 -0600 Subject: [PATCH 03/13] edit README.md --- README.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/README.md b/README.md index 51aefa2..11bacca 100644 --- a/README.md +++ b/README.md @@ -29,6 +29,11 @@ git clone git@github.com:sdss/datamodel.git cd datamodel pip install -e ".[test]" ``` +### fastapi and uvicorn Install +``` +pip install "fastapi[standard]" +pip install uvicorn +``` ### Solara Dependencies From 928bd0ef2a4b80892da7d5b333604b83e47ffb86 Mon Sep 17 00:00:00 2001 From: Pramod Gupta Date: Sat, 16 May 2026 21:17:01 -0600 Subject: [PATCH 04/13] edit README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 11bacca..7955cd6 100644 --- a/README.md +++ b/README.md @@ -32,7 +32,7 @@ pip install -e ".[test]" ### fastapi and uvicorn Install ``` pip install "fastapi[standard]" -pip install uvicorn +pip install "uvicorn[standard]" ``` ### Solara Dependencies From 0dea02eb5062d1b732bc9c2dbadc761bda8fa5f3 Mon Sep 17 00:00:00 2001 From: Pramod Gupta Date: Sun, 17 May 2026 06:41:54 -0600 Subject: [PATCH 05/13] edit README.md --- README.md | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 7955cd6..bb56e12 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ the SDSS API for delivering and accessing remote information. -This API is built using the [FastAPI](https://fastapi.tiangolo.com/) web server. Python depdendices are managed with [uv](https://docs.astral.sh/uv/). +This API is built using the [FastAPI](https://fastapi.tiangolo.com/) web server. Python dependencies are managed with [uv](https://docs.astral.sh/uv/). See [uv Features](https://docs.astral.sh/uv/getting-started/features/) for what can be done with `uv`. See their [Project Guide](https://docs.astral.sh/uv/guides/projects/) for an intro to working on python projects. @@ -16,6 +16,13 @@ First, follow the [uv installation instructions](https://docs.astral.sh/uv/getti It is recommended to work within an isolated virtual environment like `conda` or `venv`. You can create a uv venv with `uv venv valis --python 3.12`. See [uv Environments](https://docs.astral.sh/uv/pip/environments/) for more info. +### fastapi and uvicorn Install +As noted above, valis uses fastapi. So first install fastapi and uvicorn and get familiar with fastapi. +``` +pip install "fastapi[standard]" +pip install "uvicorn[standard]" +``` + ### valis Developer Install ``` git clone https://github.com/sdss/valis valis @@ -24,16 +31,12 @@ uv pip install ``` ### datamodel Developer Install +valis uses sdss/datamodel so install it as shown below. ``` git clone git@github.com:sdss/datamodel.git cd datamodel pip install -e ".[test]" ``` -### fastapi and uvicorn Install -``` -pip install "fastapi[standard]" -pip install "uvicorn[standard]" -``` ### Solara Dependencies @@ -76,7 +79,7 @@ Host pipe ``` ssh -L 6000:localhost:5432 pipe ``` -2. Update your `~/.pgpass` file with the following lines. Replace `port`, `unid`, and `password`, with your tunneled port (e.g. 6000 in step 1), Utah unid (e.g. u1234567), and db password, respectively. +2. Update your `~/.pgpass` file with the following lines. Replace `port`, `unid`, and `password`, with your tunneled local port (e.g. 6000 in step 1), Utah unid (e.g. u1234567), and db password, respectively. ``` localhost:[port]:*:[unid]:[password] host.docker.internal:[port]:*:[unid]:[password] From 527be3730fd1345a658bbd3eb9cfd3f4a95ce5b6 Mon Sep 17 00:00:00 2001 From: Pramod Gupta Date: Sun, 17 May 2026 06:52:27 -0600 Subject: [PATCH 06/13] edit README.md --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index bb56e12..7116dff 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ the SDSS API for delivering and accessing remote information. This API is built using the [FastAPI](https://fastapi.tiangolo.com/) web server. Python dependencies are managed with [uv](https://docs.astral.sh/uv/). -See [uv Features](https://docs.astral.sh/uv/getting-started/features/) for what can be done with `uv`. See their [Project Guide](https://docs.astral.sh/uv/guides/projects/) for an intro to working on python projects. +See [uv Features](https://docs.astral.sh/uv/getting-started/features/) for what can be done with `uv`. See their [Project Guide](https://docs.astral.sh/uv/guides/projects/) for an introduction to working on Python projects. ## Installation First, follow the [uv installation instructions](https://docs.astral.sh/uv/getting-started/installation/). @@ -17,7 +17,7 @@ First, follow the [uv installation instructions](https://docs.astral.sh/uv/getti It is recommended to work within an isolated virtual environment like `conda` or `venv`. You can create a uv venv with `uv venv valis --python 3.12`. See [uv Environments](https://docs.astral.sh/uv/pip/environments/) for more info. ### fastapi and uvicorn Install -As noted above, valis uses fastapi. So first install fastapi and uvicorn and get familiar with fastapi. +As noted above, valis uses fastapi. So first install fastapi and uvicorn and get familiar with using fastapi. ``` pip install "fastapi[standard]" pip install "uvicorn[standard]" From c71ab3ff357fa57e6671099ccca32a172cfbdffc Mon Sep 17 00:00:00 2001 From: Pramod Gupta Date: Sun, 17 May 2026 07:05:58 -0600 Subject: [PATCH 07/13] edit README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 7116dff..6ce3bb0 100644 --- a/README.md +++ b/README.md @@ -17,7 +17,7 @@ First, follow the [uv installation instructions](https://docs.astral.sh/uv/getti It is recommended to work within an isolated virtual environment like `conda` or `venv`. You can create a uv venv with `uv venv valis --python 3.12`. See [uv Environments](https://docs.astral.sh/uv/pip/environments/) for more info. ### fastapi and uvicorn Install -As noted above, valis uses fastapi. So first install fastapi and uvicorn and get familiar with using fastapi. +As noted above, valis uses fastapi. So first install fastapi and uvicorn. See [fastapi tutorial](https://fastapi.tiangolo.com/tutorial/) for an introduction to fastapi. ``` pip install "fastapi[standard]" pip install "uvicorn[standard]" From 95e1c3b0e90b29331127d8ac8e25a1dc5ab7982c Mon Sep 17 00:00:00 2001 From: Pramod Gupta Date: Sun, 17 May 2026 07:06:51 -0600 Subject: [PATCH 08/13] edit README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 6ce3bb0..7347e8f 100644 --- a/README.md +++ b/README.md @@ -17,7 +17,7 @@ First, follow the [uv installation instructions](https://docs.astral.sh/uv/getti It is recommended to work within an isolated virtual environment like `conda` or `venv`. You can create a uv venv with `uv venv valis --python 3.12`. See [uv Environments](https://docs.astral.sh/uv/pip/environments/) for more info. ### fastapi and uvicorn Install -As noted above, valis uses fastapi. So first install fastapi and uvicorn. See [fastapi tutorial](https://fastapi.tiangolo.com/tutorial/) for an introduction to fastapi. +As noted above, valis uses fastapi. So first install fastapi and uvicorn. See the [fastapi tutorial](https://fastapi.tiangolo.com/tutorial/) for an introduction to fastapi. ``` pip install "fastapi[standard]" pip install "uvicorn[standard]" From aa86d1a54c68a809ab598183894be54268aa76c9 Mon Sep 17 00:00:00 2001 From: Pramod Gupta Date: Sun, 17 May 2026 07:10:29 -0600 Subject: [PATCH 09/13] edit README.md --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 7347e8f..0d8af42 100644 --- a/README.md +++ b/README.md @@ -116,7 +116,9 @@ By default, the app will try to cache some route responses to a Redis database i ## Deployment -This section describes a variety of deployment methods. Valis uses gunicorn as its +This section describes a variety of valis deployment methods so you can skip this section if you are only interested in valis development. + +Valis uses gunicorn as its wsgi http server. It binds the app both to port 8000, and a unix socket. The default mode is to start valis with an awsgi uvicorn server, with 4 workers. Valis requires a Redis database running at the default location in `localhost:6379`. If this is not possible, caching can be done in memory by modifying `~/.config/sdss/valis.yaml` to use `cache_backend: in-memory`. From 6d7692b28ad99b94f218b1b286f6e43699a6bb63 Mon Sep 17 00:00:00 2001 From: Pramod Gupta Date: Mon, 18 May 2026 21:02:27 -0600 Subject: [PATCH 10/13] edit README.md --- README.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/README.md b/README.md index 0d8af42..1181f66 100644 --- a/README.md +++ b/README.md @@ -112,6 +112,11 @@ uvicorn valis.wsgi:app --reload ``` This will start a local web server at `http://localhost:8000/valis/`. The API documentation will be located at `http://localhost:8000/valis/docs`. Or to see the alternate documentation, go to `http://localhost:8000/valis/redoc/` +If you modify the valis code (e.g. by creating a new route) then +do the below steps to so that uvicorn uses the new valis code. +First stop uvicorn by CTRL+C and then run the above uvicorn command +to start valis. + By default, the app will try to cache some route responses to a Redis database in localhost. If you don't have a Redis instance running you can use `in-memory` for testing (this caches the response directly in RAM). To do so, edit `~/.config/sdss/valis.yaml` and add `cache_backend: in-memory` (this should only be used in development or it could quickly use all available memory; the memory is freed when the app is stopped). Caching can be completely disabled by setting `cache_backend: null`. The time the cache is kept can be set with the `cache_ttl` (time to live) setting option. ## Deployment From 7e837639f69a162651a25f8aac533c4f41a74e66 Mon Sep 17 00:00:00 2001 From: Pramod Gupta Date: Tue, 19 May 2026 15:41:45 -0600 Subject: [PATCH 11/13] edit README.md --- README.md | 53 +++++++++++++++++++++++------------------------------ 1 file changed, 23 insertions(+), 30 deletions(-) diff --git a/README.md b/README.md index 1181f66..f7066ee 100644 --- a/README.md +++ b/README.md @@ -16,14 +16,8 @@ First, follow the [uv installation instructions](https://docs.astral.sh/uv/getti It is recommended to work within an isolated virtual environment like `conda` or `venv`. You can create a uv venv with `uv venv valis --python 3.12`. See [uv Environments](https://docs.astral.sh/uv/pip/environments/) for more info. -### fastapi and uvicorn Install -As noted above, valis uses fastapi. So first install fastapi and uvicorn. See the [fastapi tutorial](https://fastapi.tiangolo.com/tutorial/) for an introduction to fastapi. -``` -pip install "fastapi[standard]" -pip install "uvicorn[standard]" -``` -### valis Developer Install +### Developer Install ``` git clone https://github.com/sdss/valis valis cd valis @@ -34,8 +28,7 @@ uv pip install valis uses sdss/datamodel so install it as shown below. ``` git clone git@github.com:sdss/datamodel.git -cd datamodel -pip install -e ".[test]" +uv pip install -e /your/path/to/sdss/datamodel/ ``` ### Solara Dependencies @@ -62,6 +55,21 @@ To install new packages and add them to the `pyproject.toml` and `uv.lock` files uv add [package] ``` +### Local Development + +To run a local instance for development, run the following from the top level of the `valis` repo. +``` +uvicorn valis.wsgi:app --reload +``` +This will start a local web server at `http://localhost:8000/valis/`. The API documentation will be located at `http://localhost:8000/valis/docs`. Or to see the alternate documentation, go to `http://localhost:8000/valis/redoc/` + +If you modify the valis code (e.g. by creating a new route) then +do the below steps to so that uvicorn uses the new valis code. +First stop uvicorn by CTRL+C and then run the above uvicorn command +to start valis. + +By default, the app will try to cache some route responses to a Redis database in localhost. If you don't have a Redis instance running you can use `in-memory` for testing (this caches the response directly in RAM). To do so, edit `~/.config/sdss/valis.yaml` and add `cache_backend: in-memory` (this should only be used in development or it could quickly use all available memory; the memory is freed when the app is stopped). Caching can be completely disabled by setting `cache_backend: null`. The time the cache is kept can be set with the `cache_ttl` (time to live) setting option. + ### Database Connection Valis uses the `sdssdb` package for all connections to databases. The most relevant database for the API is the `sdss5db` on `pipelines.sdss.org`. The easiest way to connect is through a local SSH tunnel. To set up a tunnel, @@ -75,23 +83,23 @@ Host pipe ForwardX11Trusted yes ProxyCommand ssh -A [unid]@mwm.sdss.org nc %h %p ``` -1. In a terminal, create an ssh tunnel to the pipelines database localhost port 5432, to a some local port. E.g. this maps the remote db localhost port 5432 to local machine on port 6000. +2. In a terminal, create an ssh tunnel to the pipelines database localhost port 5432, to a some local port. E.g. this maps the remote db localhost port 5432 to local machine on port 6000. ``` ssh -L 6000:localhost:5432 pipe ``` -2. Update your `~/.pgpass` file with the following lines. Replace `port`, `unid`, and `password`, with your tunneled local port (e.g. 6000 in step 1), Utah unid (e.g. u1234567), and db password, respectively. +3. Update your `~/.pgpass` file with the following lines. Replace `port`, `unid`, and `password`, with your tunneled local port (e.g. 6000 in step 1), Utah unid (e.g. u1234567), and db password, respectively. ``` localhost:[port]:*:[unid]:[password] host.docker.internal:[port]:*:[unid]:[password] ``` -If for some reason, you do not want to edit your .pgpass file then set the VALIS_DB_PASS environment variable with your database password. This is shown in step 3. +Alternatively, if you do not want to edit your .pgpass file then set the VALIS_DB_PASS environment variable with your database password. See step 4. -3. Set the following environment variables. +4. Set the following environment variables. - export VALIS_DB_PORT=6000 - export VALIS_DB_USER={unid} -- export VALIS_DB_PASS={password} (Do this if you skipped step 2.) +- export VALIS_DB_PASS={password} (Do this if you skipped step 3.) or optionally add them to the `~/.config/sdss/valis.yaml` configuration file. @@ -104,24 +112,9 @@ db_user: {unid} Additionally, you can set the environment variable `VALIS_DB_RESET=false` or add `db_reset: false` to `valis.yaml`. This will prevent the DB connection to be closed after a query completes and should speed up new queries. This setting should not be used in production. -### Local Development - -To run a local instance for development, run the following from the top level of the `valis` repo. -``` -uvicorn valis.wsgi:app --reload -``` -This will start a local web server at `http://localhost:8000/valis/`. The API documentation will be located at `http://localhost:8000/valis/docs`. Or to see the alternate documentation, go to `http://localhost:8000/valis/redoc/` - -If you modify the valis code (e.g. by creating a new route) then -do the below steps to so that uvicorn uses the new valis code. -First stop uvicorn by CTRL+C and then run the above uvicorn command -to start valis. - -By default, the app will try to cache some route responses to a Redis database in localhost. If you don't have a Redis instance running you can use `in-memory` for testing (this caches the response directly in RAM). To do so, edit `~/.config/sdss/valis.yaml` and add `cache_backend: in-memory` (this should only be used in development or it could quickly use all available memory; the memory is freed when the app is stopped). Caching can be completely disabled by setting `cache_backend: null`. The time the cache is kept can be set with the `cache_ttl` (time to live) setting option. - ## Deployment -This section describes a variety of valis deployment methods so you can skip this section if you are only interested in valis development. +This section describes a variety of valis deployment methods. Skip this section if you are only interested in valis development. Valis uses gunicorn as its wsgi http server. It binds the app both to port 8000, and a unix socket. The default mode is to start valis with an awsgi uvicorn server, with 4 workers. From 66810da2db04732e3d4b2f811a7f1efe147f76e4 Mon Sep 17 00:00:00 2001 From: Pramod Gupta Date: Tue, 19 May 2026 15:50:06 -0600 Subject: [PATCH 12/13] edit README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index f7066ee..7394f8e 100644 --- a/README.md +++ b/README.md @@ -21,7 +21,7 @@ It is recommended to work within an isolated virtual environment like `conda` or ``` git clone https://github.com/sdss/valis valis cd valis -uv pip install +uv sync --python 3.12 ``` ### datamodel Developer Install From 0d3fcf567b07b6528d1b9be6c175d7c751025904 Mon Sep 17 00:00:00 2001 From: Pramod Gupta Date: Thu, 21 May 2026 18:03:44 -0600 Subject: [PATCH 13/13] edit README.md --- README.md | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/README.md b/README.md index 7394f8e..64febe0 100644 --- a/README.md +++ b/README.md @@ -27,7 +27,7 @@ uv sync --python 3.12 ### datamodel Developer Install valis uses sdss/datamodel so install it as shown below. ``` -git clone git@github.com:sdss/datamodel.git +git clone https://github.com/sdss/datamodel.git uv pip install -e /your/path/to/sdss/datamodel/ ``` @@ -63,11 +63,6 @@ uvicorn valis.wsgi:app --reload ``` This will start a local web server at `http://localhost:8000/valis/`. The API documentation will be located at `http://localhost:8000/valis/docs`. Or to see the alternate documentation, go to `http://localhost:8000/valis/redoc/` -If you modify the valis code (e.g. by creating a new route) then -do the below steps to so that uvicorn uses the new valis code. -First stop uvicorn by CTRL+C and then run the above uvicorn command -to start valis. - By default, the app will try to cache some route responses to a Redis database in localhost. If you don't have a Redis instance running you can use `in-memory` for testing (this caches the response directly in RAM). To do so, edit `~/.config/sdss/valis.yaml` and add `cache_backend: in-memory` (this should only be used in development or it could quickly use all available memory; the memory is freed when the app is stopped). Caching can be completely disabled by setting `cache_backend: null`. The time the cache is kept can be set with the `cache_ttl` (time to live) setting option. ### Database Connection