Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
97 commits
Select commit Hold shift + click to select a range
f0be69d
move mirmachine engine to own folder
selfjell Aug 23, 2021
2b667ae
set up django project with connected react-webpack
selfjell Aug 23, 2021
068006a
add documentation, hide secrets
selfjell Aug 24, 2021
e39c543
fix typos, implement rough outline of input fields
selfjell Aug 26, 2021
c45a5f8
implement draft input form, add header buttons and about page
selfjell Aug 27, 2021
ccf8d6c
add react router, abstract header
selfjell Aug 30, 2021
47a3e61
add db models and views for active jobs
selfjell Aug 30, 2021
1848bf8
add job page, implement job posting and fetching to/from DB
selfjell Sep 1, 2021
9ca1a2d
implement job status and retrieval pages, add some error handling
selfjell Sep 6, 2021
750a696
add newick parser to backend and endpoint for fetching tree, still is…
selfjell Sep 7, 2021
b3b9d6d
implement newick parser, DB initializer and visualization of tree
selfjell Sep 8, 2021
05775ae
implement graph selection mode in main form
selfjell Sep 8, 2021
9bfc60d
fix backend parsing issue, update form per request, implement searcha…
selfjell Sep 13, 2021
72e17fc
abstract searchable dropdown and main page components
selfjell Sep 13, 2021
60a47e5
add searchable dropdown for families, add db models and initialization
selfjell Sep 13, 2021
0e32fe9
add backend for determining families included in search
selfjell Sep 15, 2021
713cc03
modify display of included family, add handling for single -node and …
selfjell Sep 17, 2021
6d3b2b3
fix searchable dropdown issue
selfjell Sep 20, 2021
4eb9a3e
add tree pruning before initializing db for nodes and edges
selfjell Sep 21, 2021
dcf707d
abstract panes from form components, refactor buttons, add demo button
selfjell Sep 21, 2021
ab2b17f
Create LICENSE
selfjell Sep 21, 2021
1a722e6
add adding of demo data and confirmation button
selfjell Sep 22, 2021
e99863e
add support for file upload
selfjell Sep 22, 2021
58889bd
add spinner for file uploads, add data validator in frontend
selfjell Sep 23, 2021
855ddcc
optimize for uploading of larger files
selfjell Sep 23, 2021
b2a6597
Merge pull request #1 from selfjell/file-upload
selfjell Sep 23, 2021
58595c4
add fetching from ncbi using accession numbers
selfjell Sep 24, 2021
ffb859c
temporary commit before converting to conda environment
selfjell Sep 29, 2021
d264696
implement job running, start implementing websocket connection
selfjell Oct 4, 2021
fe7a53a
implement websocket connection and channel layer dissemination
selfjell Oct 5, 2021
eff2d0d
add maintainer for removing temporary files when queue is empty
selfjell Oct 6, 2021
0b16e8e
add smarter fasta parsing, genome verification and visual feedback on…
selfjell Oct 6, 2021
e18ca3e
update readme for conda specifications
selfjell Oct 7, 2021
56e3e3b
cleanup structure, add more error handling and initial websocket conn…
selfjell Oct 7, 2021
8c9a319
add smarter file upload, various bugfixes
selfjell Oct 7, 2021
c8f28f6
add more feedback on upload, add validator for files before submition
selfjell Oct 11, 2021
bcaae3f
add help buttons on input form elements
selfjell Oct 12, 2021
2e73d79
implement cancellation of jobs, better feedback on user error, error …
selfjell Oct 14, 2021
1e2921f
implement backend for progress bar
selfjell Oct 15, 2021
0b84e66
implement progress bar in frontend
selfjell Oct 15, 2021
c8b3cef
add feedback on position in queue, and minor bug fix
selfjell Oct 18, 2021
156b6ab
add initiation time and completed time socket updates
selfjell Oct 18, 2021
3f25f83
add timer showing elapsed time of job
selfjell Oct 18, 2021
06eae1d
add time used for completed jobs, some visual changes
selfjell Oct 19, 2021
36b82ae
add backend for getting results, button for viewing results and crude…
selfjell Oct 19, 2021
6493664
add preliminary result page with tables of results and download button
selfjell Oct 21, 2021
1dd9f32
implement backend for zipping and downloading files
selfjell Oct 21, 2021
ca1fb4a
keep users from queuing multiple jobs
selfjell Oct 21, 2021
9ddfda7
fix frontend dependencies
selfjell Oct 22, 2021
5bad377
added modal content for 30nt sequence
selfjell Oct 25, 2021
7245488
dockerize the application and fix dependency and path issues
selfjell Oct 27, 2021
7303d50
updated dockerfile to use never versions of node and npm
selfjell Oct 28, 2021
b5c9bc0
add outline for mail functionality, more security and change timezone…
selfjell Nov 2, 2021
c6614d7
add throttling on job posting, fix bug with unspecified species tag
selfjell Nov 3, 2021
147cb60
add config file for users to change settings in the app
selfjell Nov 3, 2021
b18c461
add support for maintenance cronjob, more config options
selfjell Nov 4, 2021
2514ac6
configure for uwsgi, daphne and nginx
selfjell Nov 18, 2021
5f74338
add support for mysql db
selfjell Nov 18, 2021
f9e2d3d
remove input files, disable email services, start implementing footer
selfjell Nov 18, 2021
c54e74c
add uio to footer, implement cookie notification
selfjell Nov 19, 2021
3ace111
start implementing 0.2.11 patch
selfjell Jan 5, 2022
b61b555
change progress notification system, fix bug in snakemake, migration …
selfjell Jan 13, 2022
c8d2c43
add global variable for url used in frontend
selfjell Jan 24, 2022
4cf19e2
add genome fetching using assembly accnum, allow .fas files and - in …
selfjell Jan 25, 2022
05ae731
fix family fetching bug, progress reporter stability improved, add mo…
selfjell Jan 26, 2022
ec24a9b
bug fixes, add new tree and family file, update tree parser to recurs…
selfjell Jan 31, 2022
0f56a04
fix maintainer bug, add meta file to zipped download
selfjell Feb 1, 2022
934aaa1
add 404 page handling
selfjell Feb 1, 2022
d421643
minor bugfixes, add check if user can upload in frontend
selfjell Feb 16, 2022
5e7986f
add daphne to environment.yml
selfjell Feb 21, 2022
d118240
add dependency, move backend species check
selfjell Feb 25, 2022
8cec43c
update frontend design
selfjell Feb 25, 2022
a6acae4
redesign of frontend, add new logos
selfjell Feb 28, 2022
374e0db
update mirmachine engine, frontend bugfix
selfjell Mar 1, 2022
7633944
simplify modal exit, change species examples
selfjell Mar 1, 2022
2d45ac6
modify header bar, add return button to result
selfjell Mar 1, 2022
133495a
add check if file exists in ncbi fetch, visual update
selfjell Mar 9, 2022
c4fb1d2
update tree, fix bug
selfjell Mar 9, 2022
5116079
bug fix
selfjell Mar 9, 2022
e7b6174
frontend visual update
selfjell Mar 11, 2022
981bc17
add graphs and table to result page
selfjell Mar 14, 2022
1d52815
rename function, fix maintenance bug
selfjell Mar 28, 2022
ecebaa3
change miRNA to all-caps
selfjell Apr 1, 2022
71b7338
update results page, fix file hash bug, fix no results bug, remove un…
selfjell Apr 23, 2022
b917400
update settings, add about page contents, update logo
selfjell Apr 29, 2022
a1e78ed
add mail functionality
selfjell May 2, 2022
61187a3
add support for keyboard navigation
selfjell May 6, 2022
6007af6
add settings file for docker
selfjell May 8, 2022
6e07df6
add concurrency
selfjell May 9, 2022
59c10a2
fix result page overflow issue
selfjell May 9, 2022
1835d12
various bug fixes and enhancements
selfjell May 12, 2022
c287529
add additional information text
selfjell May 12, 2022
baf4dc4
add robots.txt, update admin page, fix typos
selfjell May 13, 2022
bed1bd7
fix issue with mail links, remove debugging
selfjell May 14, 2022
f98b00c
better user feedback for fetching error
selfjell May 27, 2022
5a16415
Updated citation after publication
selfjell Jul 4, 2023
65948eb
Update Form.js
sinanugur Jul 4, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
The diff you're trying to view is too large. We only load the first 3000 changed files.
52 changes: 52 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
engine/analyses/
engine/results/
engine/data/
data/
.snakemake
engine/dist/
engine/eggs/
engine/lib/
engine/build/
engine/lib64/
engine/__pycache__/

media/*

*.py[cod]
*.egg-info/
*.egg
.DS_Store

# dependencies
lookupService/frontend/node_modules
lookupService/frontend/.pnp
.pnp.js

# testing
/coverage

# production
/build

# misc
.DS_Store
.env.local
.env.development.local
.env.test.local
.env.production.local
.env

npm-debug.log*
yarn-debug.log*
yarn-error.log*

.idea/
db.sqlite3
lookupService/static/frontend/**
package-lock.json

secret.txt
Pipfile
Pipfile.lock
uwsgi.ini
uwsgi_params
54 changes: 46 additions & 8 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,13 +1,51 @@
analyses/
results/
engine/analyses/
engine/results/
engine/data/
data/
.snakemake
dist/
eggs/
lib/
build/
lib64/
__pycache__/
engine/dist/
engine/eggs/
engine/lib/
engine/build/
engine/lib64/
engine/__pycache__/

media/*

*.py[cod]
*.egg-info/
*.egg
*.log
.DS_Store

# dependencies
lookupService/frontend/node_modules
lookupService/frontend/.pnp
.pnp.js

# testing
/coverage

# production
/build

# misc
.DS_Store
.env.local
.env.development.local
.env.test.local
.env.production.local
.env
.envdock

npm-debug.log*
yarn-debug.log*
yarn-error.log*

.idea/
db.sqlite3
lookupService/static/frontend/**
package-lock.json
static/**
secret.txt

11 changes: 0 additions & 11 deletions .travis.yml

This file was deleted.

43 changes: 43 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# syntax=docker/dockerfile:1
FROM continuumio/miniconda3:latest AS backend

RUN apt update && \
apt install -y curl

#RUN curl -fsSL https://deb.nodesource.com/setup_lts.x | bash
#RUN apt install -y nodejs && \
# apt install -y npm

RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
ENV NODE_VERSION=16.13.0
ENV NVM_DIR=/root/.nvm

RUN . "$NVM_DIR/nvm.sh" && nvm install ${NODE_VERSION}
RUN . "$NVM_DIR/nvm.sh" && nvm use v${NODE_VERSION}
RUN . "$NVM_DIR/nvm.sh" && nvm alias default v${NODE_VERSION}
ENV PATH="/root/.nvm/versions/node/v${NODE_VERSION}/bin/:${PATH}"

WORKDIR /app

COPY environment.yml ./

RUN conda env create -f environment.yml -n mirmachine

COPY . ./

WORKDIR /app/lookupService/frontend

RUN npm install

RUN npm run build

WORKDIR /app

SHELL ["conda", "run", "-n", "mirmachine", "bin/bash", "-c"]

RUN ["conda", "run", "--no-capture-output", "-n", "mirmachine", \
"python", "manage.py", "migrate"]

EXPOSE 8000
ENTRYPOINT ["conda", "run", "--no-capture-output", "-n", "mirmachine", \
"python", "manage.py", "runserver", "0.0.0.0:8000"]
21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2021 Håvard Trondsen & Sinan U. Umu

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
File renamed without changes.
29 changes: 29 additions & 0 deletions MirMachineWebapp/asgi.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
"""
ASGI config for MirMachineWebapp project.

It exposes the ASGI callable as a module-level variable named ``application``.

For more information on this file, see
https://docs.djangoproject.com/en/3.2/howto/deployment/asgi/
"""

from django.core.asgi import get_asgi_application
import os

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'MirMachineWebapp.settings')
django_asgi_application = get_asgi_application()

from channels.routing import ProtocolTypeRouter, URLRouter
from channels.auth import AuthMiddlewareStack
import lookupService.urls
import django

django.setup()
application = ProtocolTypeRouter({
'http': django_asgi_application,
'websocket': AuthMiddlewareStack(
URLRouter(
lookupService.urls.websocket_patterns
)
)
})
166 changes: 166 additions & 0 deletions MirMachineWebapp/settings.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
"""
Django settings for MirMachineWebapp project.

Generated by 'django-admin startproject' using Django 3.2.6.

For more information on this file, see
https://docs.djangoproject.com/en/3.2/topics/settings/

For the full list of settings and their values, see
https://docs.djangoproject.com/en/3.2/ref/settings/
"""
import os
from pathlib import Path
from environ import Env

env = Env()

env.read_env(env_file='.env')

# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent


# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/3.2/howto/deployment/checklist/

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = env('DJANGO_SECRET_KEY')

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True

ALLOWED_HOSTS = ['mirmachine.org', 'localhost', '127.0.0.1']


# Application definition

INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'lookupService.apps.LookupserviceConfig',
'manifest_loader',
'rest_framework',
'channels',
'django_crontab',
]

MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

ROOT_URLCONF = 'MirMachineWebapp.urls'

TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [os.path.join(BASE_DIR, "lookupService", "templates")],
'APP_DIRS': True,
'OPTIONS': {
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
],
},
},
]

WSGI_APPLICATION = 'MirMachineWebapp.wsgi.application'
ASGI_APPLICATION = 'MirMachineWebapp.asgi.application'

# Database
# https://docs.djangoproject.com/en/3.2/ref/settings/#databases

DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'mirmachine_data',
'USER': env('DB_USER'),
'PASSWORD': env('DB_PASS'),
'HOST': 'localhost',
'PORT': '',
}
}

CHANNEL_LAYERS = {
"default": {
"BACKEND": "channels_redis.core.RedisChannelLayer",
"CONFIG": {
"hosts": [("127.0.0.1", 6379)]
},
},
}


# Password validation
# https://docs.djangoproject.com/en/3.2/ref/settings/#auth-password-validators

AUTH_PASSWORD_VALIDATORS = [
{
'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
},
]


# Internationalization
# https://docs.djangoproject.com/en/3.2/topics/i18n/

LANGUAGE_CODE = 'en-us'

TIME_ZONE = 'Europe/Oslo'

USE_I18N = True

USE_L10N = True

USE_TZ = True


# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/3.2/howto/static-files/
STATIC_ROOT = os.path.join(BASE_DIR, 'static')
STATIC_URL = 'static/'
STATICFILES_DIRS = (
os.path.join(BASE_DIR, 'lookupService', 'static', 'assets'),
os.path.join(BASE_DIR, 'lookupService', 'static', 'frontend'),
)

MEDIA_ROOT = os.path.join(BASE_DIR, 'media/')
MEDIA_URL = 'media/'
# Default primary key field type
# https://docs.djangoproject.com/en/3.2/ref/settings/#default-auto-field

DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'

EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'smtp.sendgrid.net'
EMAIL_USER = env('MAIL_USER')
EMAIL_HOST_USER = env('MAIL_HOST_USER')
EMAIL_HOST_PASSWORD = env('MAIL_PASS')
EMAIL_PORT = 587
EMAIL_USE_TLS = True

CRONJOBS = [
('0 0 * * 2,4,6', 'lookupService.helpers.maintainer.delete_expired_jobs')
]
Loading