Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 72 additions & 0 deletions .dae-devops/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# ********** Please don't edit this file!
# ********** It has been generated automatically by dae_devops version 0.5.2.
# ********** For repository_name chimpflow

# ---------------------------------------------------------------------
# These make targets are designed to be called from the command line and from .gitlab-ci.yml.
# ---------------------------------------------------------------------

# I put the package_pip artifacts int this place for now until I can use the corporate internal pipserver.
# Maybe /dls_sw/work/python3/RHEL7-x86_64/distributions would be a better place?
PIP_FIND_LINKS = /dls_sw/apps/bxflow/artifacts

# I got these from https://gitlab.diamond.ac.uk/controls/reports/ci_templates/-/blob/master/defaults.yml.
# Directory where docs are published.
DOCS_PUBLISH_ROOT = /dls/cs-www/reports/gitlab-ci/chimpflow

# ---------------------------------------------------------------------
# These are called from the default before_script in the gitlab CI.
# That meants it is normally called before any other job.
# It installs things not covered by pyproject.toml.

prepare_git_dependencies:
chmod a+x .dae-devops/prepare_git_dependencies.sh
.dae-devops/prepare_git_dependencies.sh

# ---------------------------------------------------------------------
# The validate stage makes sure code is ready to package and release.

validate: \
validate_pre_commit \
validate_mypy \
validate_pytest
echo "validation complete"

validate_pre_commit:
tox -q -e pre-commit

validate_mypy:
tox -q -e mypy

validate_pytest:
tox -q -e pytest

validate_docs:
tox -q -e docs

# ---------------------------------------------------------------------
# The pip state packages and publishes for pip.

package: \
package_pip

package_pip:
pip install pipx twine
pipx run build

# ---------------------------------------------------------------------
# The publish stage publishes things which have already been built or packaged.

publish: \
publish_pip \
publish_docs

publish_pip:
cp -v -p dist/*.whl $(PIP_FIND_LINKS)

publish_docs:
mkdir -p $(DOCS_PUBLISH_ROOT)
cp -r build/html/* $(DOCS_PUBLISH_ROOT)


# dae_devops_fingerprint e7dd1051580ffa0ed1b6a5a9ce3e6290
34 changes: 34 additions & 0 deletions .dae-devops/docs/conventions.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
.. # ********** Please don't edit this file!
.. # ********** It has been generated automatically by dae_devops version 0.5.2.
.. # ********** For repository_name chimpflow

Naming conventions
=======================================================================

Here are the naming conventions used within the source code.

variables
lowercase, underscores

constants
uppercase, underscores

classes
camel case

class methods
lowercase, underscores

imports
require fully qualified package name (import <package> does nothing)

exception: top-level packages __init__.py will have __version__ defined in them

python packages
lowercase, underscores

repository
lowercase, hyphens


.. # dae_devops_fingerprint 1584b2ca3962a352d751931fba1af516
39 changes: 39 additions & 0 deletions .dae-devops/docs/developing.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
.. # ********** Please don't edit this file!
.. # ********** It has been generated automatically by dae_devops version 0.5.2.
.. # ********** For repository_name chimpflow

Developing
=======================================================================

If you plan to make change to the code in this repository, you can use the steps below.

Clone the repository::

$ git clone https://github.com/diamondlightsource/chimpflow/chimpflow.git

It is recommended that you install into a virtual environment so this
installation will not interfere with any existing Python software.
Make sure to have at least python version 3.9 then::

$ python3 -m venv /scratch/$USER/myvenv
$ source /scratch/$USER/myvenv/bin/activate
$ pip install --upgrade pip

Install the package in edit mode which will also install all its dependencies::

$ cd chimpflow
$ export PIP_FIND_LINKS=/dls_sw/apps/bxflow/artifacts
$ pip install -e .[dev]

Now you may begin modifying the code.

|

If you plan to modify the docs, you will need to::

$ pip install -e .[docs]




.. # dae_devops_fingerprint 2778ced07515a8064c66e3978215aae0
46 changes: 46 additions & 0 deletions .dae-devops/docs/devops.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
.. # ********** Please don't edit this file!
.. # ********** It has been generated automatically by dae_devops version 0.5.2.
.. # ********** For repository_name chimpflow

Devops
=======================================================================

There exists a a configuration file called ``.dae-devops/project.yaml``.

This file defines the project information needed for CI/CD.

It is parsed by the ``dae_devops.force`` command which creates these files:

- pyproject.toml
- .gitlab-ci.yml
- .dae-devops/Makefile
- .dae-devops/docs/*

Local CI/CD execution
-----------------------------------------------------------------------

All the CI/CD ops which are run by the git server can be run at the command line.

Running these ops before pushing to the git server can make the turnaround quicker to fix things.

Follow the steps in the Developing section. Then you can run the following commands.

Validation of the code::

$ make -f .dae-devops/Makefile validate_pre_commit
$ make -f .dae-devops/Makefile validate_mypy
$ make -f .dae-devops/Makefile validate_pytest
$ make -f .dae-devops/Makefile validate_docs

Packaging::

$ make -f .dae-devops/Makefile package_pip

Publishing::

$ make -f .dae-devops/Makefile publish_pip
$ make -f .dae-devops/Makefile publish_docs



.. # dae_devops_fingerprint d1782f75b785c0194b6666f894b338a6
24 changes: 24 additions & 0 deletions .dae-devops/docs/docs_structure.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
.. # ********** Please don't edit this file!
.. # ********** It has been generated automatically by dae_devops version 0.5.2.
.. # ********** For repository_name chimpflow

About the documentation
-----------------------

:material-regular:`format_quote;2em`

The Grand Unified Theory of Documentation

-- David Laing

There is a secret that needs to be understood in order to write good software
documentation: there isn't one thing called *documentation*, there are four.

They are: *tutorials*, *how-to guides*, *technical reference* and *explanation*.
They represent four different purposes or functions, and require four different
approaches to their creation. Understanding the implications of this will help
improve most documentation - often immensely.

`More information on this topic. <https://documentation.divio.com>`_

.. # dae_devops_fingerprint b2ecceb2057aa129a1ee55c76c58e867
43 changes: 43 additions & 0 deletions .dae-devops/docs/installing.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
.. # ********** Please don't edit this file!
.. # ********** It has been generated automatically by dae_devops version 0.5.2.
.. # ********** For repository_name chimpflow

Installing
=======================================================================


You will need python 3.9 or later.

On a Diamond Light Source internal computer, you can achieve Python 3.9 by::

$ module load python/3.9

You can check your version of python by typing into a terminal::

$ python3 --version

It is recommended that you install into a virtual environment so this
installation will not interfere with any existing Python software::

$ python3 -m venv /scratch/$USER/myvenv
$ source /scratch/$USER/myvenv/bin/activate
$ pip install --upgrade pip


You can now use ``pip`` to install the library and its dependencies::

$ export PIP_FIND_LINKS=/dls_sw/apps/bxflow/artifacts
$ python3 -m pip install chimpflow

If you require a feature that is not currently released you can also install
from git::

$ python3 -m pip install git+https://github.com/diamondlightsource/chimpflow/chimpflow.git

The library should now be installed and the commandline interface on your path.
You can check the version that has been installed by typing::

$ chimpflow --version
$ chimpflow --version-json

.. # dae_devops_fingerprint f4209c13d354b457d8cc21ddf076df26
38 changes: 38 additions & 0 deletions .dae-devops/docs/testing.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
.. # ********** Please don't edit this file!
.. # ********** It has been generated automatically by dae_devops version 0.5.2.
.. # ********** For repository_name chimpflow

Testing
=======================================================================

The package uses pytest for unit testing.

If you want to run the tests, first get a copy of the code per the instructions in the Developing section.

Then you can run all tests by::

$ pytest

Or this, which is the command used by the CI runner.

$ make -f .dae-devops/Makefile validate_pytest

To run a single test you can do::

$ pytest tests/the_test_you_want.py

If you want to see more output of the test while it's running you can do:

$ pytest -sv -ra --tb=line tests/the_test_you_want.py

Each test will write files into its own directory::

/tmp/chimpflow/tests/....

The tests clear their directory when they start, but not when they finish.
This allows peeking in there to see what's been written by the test.




.. # dae_devops_fingerprint 3787294ef4eef4f87161e07806f1f417
16 changes: 16 additions & 0 deletions .dae-devops/prepare_git_dependencies.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/usr/bin/env bash

# ********** Please don't edit this file!
# ********** It has been generated automatically by dae_devops version 0.5.2.
# ********** For repository_name chimpflow

me=${BASH_SOURCE}
echo "${me}: installing 0 dependencies for repository_name chimpflow"

function __install {
echo ${me}: "$@"
"$@"
}


# dae_devops_fingerprint 74a0f47063a55760445b6dba99ef5c79
21 changes: 21 additions & 0 deletions .dae-devops/project.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Primary information needed for devops.
primary:
repository_name: chimpflow
package_name: chimpflow_lib
one_line_description: "XChem service to apply CHIMP detection for discovery of well center, crystal locatins and suggested drop target."
author:
name: David Erb
email: david.erb@diamond.ac.uk
project_urls:
GitLab: https://github.com/diamondlightsource/chimpflow
project_scripts:
chimpflow: "chimpflow_lib.__main__:main"
chimpflow.cli: "chimpflow_cli.main:main"
dependencies:
- type: pypi
list:
- xchembku
- dls_servbase
- dls_mainiac
- dls_multiconf
- dls_utilpack
43 changes: 43 additions & 0 deletions .devcontainer/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# ********** Please don't edit this file!
# ********** It has been generated automatically by dae_devops version 0.5.2.
# ********** For repository_name chimpflow

# This file is for use as a devcontainer and a runtime container
#
# The devcontainer should use the build target and run as root with podman
# or docker with user namespaces.
#
FROM python:3.11 as build

ARG PIP_OPTIONS

# Add any system dependencies for the developer/build environment here e.g.
# RUN apt-get update && apt-get upgrade -y && \
# apt-get install -y --no-install-recommends \
# desired-packages \
# && rm -rf /var/lib/apt/lists/*

# set up a virtual environment and put it in PATH
RUN python -m venv /venv
ENV PATH=/venv/bin:$PATH

# Copy any required context for the pip install over
COPY . /context
WORKDIR /context

# install python package into /venv
RUN pip install ${PIP_OPTIONS}

FROM python:3.11-slim as runtime

# Add apt-get system dependecies for runtime here if needed

# copy the virtual environment from the build stage and put it in PATH
COPY --from=build /venv/ /venv/
ENV PATH=/venv/bin:$PATH

# change this entrypoint if it is not the same as the repo
ENTRYPOINT ["chimpflow"]
CMD ["--version"]

# dae_devops_fingerprint 5785b1775d09431a8a064cb67aa564cf
Loading