diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..788ac02 --- /dev/null +++ b/.gitignore @@ -0,0 +1,2 @@ +# ignore the secret +master_secret.txt diff --git a/GUIDE.md b/GUIDE.md new file mode 100644 index 0000000..260d551 --- /dev/null +++ b/GUIDE.md @@ -0,0 +1,126 @@ +# Madek Hosting Guide + +*For more details, see [Madek-Deploy Project](https://github.com/Madek/madek-deploy) +and the [general Madek Documentation](https://madek.readthedocs.io/)* + +--- + +## setup & install + +1. [Generate your own inventory repository by using this template](https://github.com/Madek/madek-instance/generate) + +1. set up inventory on a computer running Linux or macOS (will be the "control machine") + ```sh + which ansible-playbook || echo "install ansible first!" + git clone git@github:yourUserName/madek-instance my-madek + cd my-madek + sh -c 'git submodule update --init Madek && cd Madek && git submodule update --init --recursive deploy' + ``` + +1. prepare a server running [Debian `jessie`](https://www.debian.org/releases/jessie/), + log in as root via SSH and do `apt-get install python` + +1. inventory configuration + - prepare inventory files + ``` + # set hostname + export MADEK_HOSTNAME="madek.example.com" + # create hosts file + sh -c "echo \"$(cat examples/hosts_example)\"" > hosts + # create host_vars + sh -c "echo \"$(cat examples/host_vars_example.yml)\"" > "host_vars/${MADEK_HOSTNAME}.yml" + ``` + - edit global config in file `group_vars/madek.yml` + - edit per-host config in file `host_vars/madek.example.com.yml` + +1. install with ansible + ```sh + ansible-playbook -i hosts Madek/deploy/play_setup-and-deploy.yml + ``` + +1. setup initial configuration & admin account (choose a better password and save it): + ```sh + ansible-playbook -i hosts Madek/deploy/play_first-time-setup.yml -e "admin_password=supersecret" + ``` + +1. Log in as the admin user and go to the admin interface. + Change the password to a stronger one, customize the name of the instance and other settings. + Add Users and Groups and start using Madek! 🎉 + +## backup + +A `master_secret` was created during the installation and put in a text file +in your repository. +By default it is git-ignored, so it won't be accidentially pushed to a public +host (like GitHub). +You should either back up your local repository with the secret to a secure place; +or use [`git-crypt`](https://www.agwa.name/projects/git-crypt/) to add the +secret to the repository in encrypted form (*recommended*). + +## upgrade + +1. update `Madek` submodule reference to latest release + - either by accepting a Pull Request (when enabled) + - or manually: `./scripts/update_madek_latest stable` + +1. run the setup playbook again: `ansible-playbook -i hosts Madek/deploy/play_setup-and-deploy.yml` + +## automatic deployments + +***Prerequisite:*** All changed files (configuration etc) must be committed back into the repository, +so that it can be shared with other computers. +That means `git-crypt` must be set up (see below). + +*Note* that you can use this fork normally, with one caveat: +**don't edit any files that came with this repository**, or you will have to deal with merge conflicts later on! +The only exception is `README.md`, we won't touch it because you'll likely want to customize it. + +1. add GPG of your trusted CI machine to the repo: + ``` + git crypt add-gpg-user ${CI_GPG_KEY_ID} + ``` + +1. add SSH public key of CI executor to `authorized_keys` of target server + +1. set up your CI to `git crypt unlock` und run the deploy script. + See `examples/cider-ci.yml` for a working [Cider-CI](https://cider-ci.info) configuration. + +## git-crypt + +set up and add master secret: + +```sh +which git-crypt || echo 'install `git-crypt` first!' +cp examples/git-crypt/.git{ignore,attributes} . +git commit .gitignore .gitattributes -m 'setup git-crypt' +git crypt init +git crypt add-gpg-user you@example.com +git add master_secret.txt && git commit -m 'add encrypted secret' +git crypt status +``` + +if needed, set up secret variables: + +```sh +# create hosts file +sh -c "echo \"$(cat examples/git-crypt/hosts_example)\"" > hosts +# create host_vars +sh -c "echo \"$(cat examples/git-crypt/group_vars_secret_example.yml)\"" > group_vars/secrets.yml +git add group_vars/secrets.yml && git commit -m 'add encrypted secrets' +git crypt status +``` + +## HTTPS + +Secure Communications for your users (HTTPS) can be enabled +by obtaining a TLS certificate and configure apache to use it. +This can be done easily using `certbot` by [LetsEncrypt](https://letsencrypt.org). + + +1. Install `certbot`: `sudo apt-get install python-certbot-apache -t jessie-backports` +2. Get cert: `certbot certonly --apache -d madek.example.com` +3. Configure apache: `certbot run -n --apache --redirect --apache-vhost-root /etc/apache2/madek -d madek.example.com` + - even more secure (SSL Labs `A+` instead of `A`): `certbot run -n --apache --redirect --hsts --uir --strict-permissions --apache-vhost-root /etc/apache2/madek -d madek.example.com` + +If a certificate set up this way is found on the server, the deploy process will automatically use `certbot` for configuration with recommended settings. +You only have to re-run `certbot` yourself after each deploy if you prefer other settings. diff --git a/Madek b/Madek index 349723c..10e31d1 160000 --- a/Madek +++ b/Madek @@ -1 +1 @@ -Subproject commit 349723c40638b9aea4746dd3e9f88880277a331c +Subproject commit 10e31d1897949502f53c6c6cacfd1de370fc8365 diff --git a/README.md b/README.md index abc44fe..8ec7d30 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ -# madek-instance +# Madek hosting template -## config +This is a template to set up a [Madek](https://zhdk.ch/madek) server, +providing a web-based media archive. -- global config in `group_vars/madek.yml` -- per-host config in `host_vars/madek.example.com.yml` +## [Read the Guide](https://github.com/Madek/madek-instance/blob/master/GUIDE.md) diff --git a/ansible.cfg b/ansible.cfg new file mode 120000 index 0000000..adf3f72 --- /dev/null +++ b/ansible.cfg @@ -0,0 +1 @@ +Madek/deploy/ansible.cfg \ No newline at end of file diff --git a/examples/cider-ci.yml b/examples/cider-ci.yml new file mode 100644 index 0000000..60eba44 --- /dev/null +++ b/examples/cider-ci.yml @@ -0,0 +1,44 @@ +# working example configuration to deploy a madek instance using Cider-CI + +jobs: + deploy_madek: + name: Deploy Madek + + context: + task_defaults: + max_trials: 1 + aggregate_state: satisfy-last + traits: + Ansible 2: yes + git-crypt: yes + ci-executor.madek: yes + + environment_variables: + LANG: "en_US.UTF-8" + + tasks: + deploy: + exclusive_global_resources: + "madek.example.com": true + environment_variables: + HOSTS_FILE: hosts + + scripts: + deploy: + start_when: + submodules are checked out: {script_key: checkout-submodules} + git-crypt is unlocked: {script_key: unlock} + only when we are on the head of master branch: {script_key: check-head-of-master} + + timeout: 30 minutes + body: cd Madek/deploy && ansible-playbook -i "../../${HOSTS_FILE}" play_setup-and-deploy.yml + + unlock: + body: git crypt unlock && ls -la . && ls -R group_vars + + check-head-of-master: + body: git fetch && test $(git log -n 1 --pretty=%H HEAD -- ) == $(git log -n 1 --pretty=%H origin/master -- ) + + checkout-submodules: + start_when: {after: {script_key: check-head-of-master}} + body: git submodule update --init Madek && cd Madek && git submodule update --init --recursive deploy diff --git a/examples/git-crypt/.gitattributes b/examples/git-crypt/.gitattributes new file mode 100644 index 0000000..d17db48 --- /dev/null +++ b/examples/git-crypt/.gitattributes @@ -0,0 +1 @@ +*secret* filter=git-crypt diff=git-crypt diff --git a/examples/git-crypt/.gitignore b/examples/git-crypt/.gitignore new file mode 100644 index 0000000..ced161b --- /dev/null +++ b/examples/git-crypt/.gitignore @@ -0,0 +1,2 @@ +# DONT ignore the secret - its encrypted with git-crypt! +!master_secret.txt diff --git a/examples/git-crypt/group_vars_example.yml b/examples/git-crypt/group_vars_example.yml new file mode 100644 index 0000000..afde5f5 --- /dev/null +++ b/examples/git-crypt/group_vars_example.yml @@ -0,0 +1,3 @@ +# ansible_ssh_port: 12345 +# ansible_ssh_user: my_ssh_user +# zencoder_api_key: "my-key" diff --git a/examples/git-crypt/hosts_example b/examples/git-crypt/hosts_example new file mode 100644 index 0000000..0e2115b --- /dev/null +++ b/examples/git-crypt/hosts_example @@ -0,0 +1,5 @@ +[secrets] +${MADEK_HOSTNAME} + +[madek] +${MADEK_HOSTNAME} diff --git a/examples/host_vars_example.yml b/examples/host_vars_example.yml new file mode 100644 index 0000000..c8fdbbe --- /dev/null +++ b/examples/host_vars_example.yml @@ -0,0 +1,28 @@ +--- +# connection +ansible_ssh_host: ${MADEK_HOSTNAME} +madek_external_hostname: ${MADEK_HOSTNAME} +ansible_ssh_port: 22 +ansible_ssh_user: root + +# storage +setup_storage_directories: true +madek_storage_dir: /srv/madekdata +madek_file_storage_dir: /srv/madekdata/attachments +madek_thumbnail_storage_dir: /srv/madekdata/attachments +madek_tmp_dir: /srv/madekdata/tmp + +# audio and videos previews +zencoder_enabled: false +# zencoder_api_key: '{{zhdk_zencoder_api_key}}' +# zencoder_test_mode: True + +# backups +# db_backups_enabled: False + +# customize +# madek_webapp_html_extra_head_start: | +# + +# madek_webapp_html_extra_head_end: | +# diff --git a/examples/hosts_example b/examples/hosts_example new file mode 100755 index 0000000..bbf05ed --- /dev/null +++ b/examples/hosts_example @@ -0,0 +1,2 @@ +[madek] +${MADEK_HOSTNAME} diff --git a/group_vars/madek.yml b/group_vars/madek.yml deleted file mode 100644 index aee6a57..0000000 --- a/group_vars/madek.yml +++ /dev/null @@ -1,28 +0,0 @@ -# audio and videos previews -zencoder_enabled: false -# zencoder_api_key: '{{zhdk_zencoder_api_key}}' -# zencoder_test_mode: True - -# backups -# db_backups_enabled: False - -# branding -madek_site_title: "Media Archive" -madek_welcome_title: "Welcome to Madek!" - -# madek_webapp_html_extra_head_start: | -# - -# madek_webapp_html_extra_head_end: | -# diff --git a/host_vars/.gitkeep b/host_vars/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/host_vars/madek.example.com.yml b/host_vars/madek.example.com.yml deleted file mode 100644 index 886f794..0000000 --- a/host_vars/madek.example.com.yml +++ /dev/null @@ -1,11 +0,0 @@ -# connection -# ansible_ssh_host: madek.example.com -# madek_external_hostname: madek.example.com -ansible_ssh_user: root - -# storage -setup_storage_directories: true -madek_storage_dir: /srv/madekdata -madek_file_storage_dir: /srv/madekdata/attachments -madek_thumbnail_storage_dir: /srv/madekdata/attachments -madek_tmp_dir: /srv/madekdata/tmp diff --git a/hosts b/hosts deleted file mode 100644 index 57c6f22..0000000 --- a/hosts +++ /dev/null @@ -1,2 +0,0 @@ -[madek] -madek.example.com diff --git a/scripts/update_madek_latest b/scripts/update_madek_latest new file mode 100755 index 0000000..b13a4c4 --- /dev/null +++ b/scripts/update_madek_latest @@ -0,0 +1,23 @@ +#!/bin/bash -exu + +RELEASE_NAME=$1 +CHANNEL=${2:-stable} + +git pull + +cd Madek +git fetch +git submodule foreach --recursive 'git reset --hard HEAD' +git reset --hard "origin/${CHANNEL}" +git submodule update --recursive --init --force + +cd - +git add Madek + +if [[ -n "${RELEASE_NAME}" ]]; then + git commit -m "[Madek] ${RELEASE_NAME}" +else + git commit -m "[Madek] x.y.z" --edit +fi + +git push