-
Notifications
You must be signed in to change notification settings - Fork 3
feat: make migration view page #150
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
dlbrittain
wants to merge
3
commits into
master
Choose a base branch
from
admin_migration_views
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 2 commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,10 +1,105 @@ | ||
| from flask_admin import Admin | ||
| from flask_admin import Admin, AdminIndexView, BaseView, expose | ||
| from flask_admin.contrib.sqla import ModelView | ||
| from dynamicannotationdb.models import AnalysisVersion, AnalysisTable | ||
| from flask import request, current_app, g, render_template | ||
| from middle_auth_client import auth_required | ||
| from dynamicannotationdb.migration import DynamicMigration, run_alembic_migration | ||
| from sqlalchemy.engine.url import make_url | ||
| from materializationengine.info_client import ( | ||
| get_datastacks, | ||
| get_relevant_datastack_info, | ||
| ) | ||
|
|
||
|
|
||
| class MatAdminIndexView(AdminIndexView): | ||
| @expose("/", methods=["GET"]) | ||
| @auth_required | ||
| def index(self): | ||
| return super(MatAdminIndexView, self).index() | ||
|
|
||
| def is_accessible(self): | ||
| @auth_required | ||
| def helper(): | ||
| return True | ||
|
|
||
| return helper() | ||
|
|
||
|
|
||
| def setup_admin(app, db): | ||
| admin = Admin(app, name="materializationengine") | ||
| admin = Admin(app, name="materializationengine", index_view=MatAdminIndexView()) | ||
| admin.add_view(MigrationView(name="Migration")) | ||
| admin.add_view(ModelView(AnalysisVersion, db.session)) | ||
| admin.add_view(ModelView(AnalysisTable, db.session)) | ||
| return admin | ||
|
|
||
|
|
||
| def get_allowed_aligned_volumes(): | ||
| with current_app.app_context(): | ||
| datastacks = get_datastacks() | ||
| aligned_volumes = [] | ||
| for datastack in datastacks: | ||
| aligned_volume_name, pcg_table_name = get_relevant_datastack_info(datastack) | ||
| aligned_volumes.append(aligned_volume_name) | ||
| return aligned_volumes | ||
|
|
||
|
|
||
| class MigrationView(BaseView): | ||
| def is_accessible(self): | ||
| @auth_required | ||
| def helper(): | ||
| return True | ||
|
|
||
| return helper() and g.get("auth_user", {}).get("admin", False) | ||
|
|
||
| def inaccessible_callback(self, name, **kwargs): | ||
| return render_template("admin/403.html"), 403 | ||
|
|
||
| @expose("/") | ||
| @auth_required | ||
| def index(self): | ||
| aligned_volumes = get_allowed_aligned_volumes() | ||
| return self.render("admin/migration.html", aligned_volumes=aligned_volumes) | ||
|
|
||
| @expose("/migrate_static_schemas", methods=["POST"]) | ||
| @auth_required | ||
| def migrate_static_schemas(self): | ||
| sql_url = request.form.get("sql_url") | ||
| aligned_volume = request.form.get("aligned_volume") | ||
| sql_base_uri = sql_url.rpartition("/")[0] | ||
| sql_uri = make_url(f"{sql_base_uri}/{aligned_volume}") | ||
| migrator = run_alembic_migration(str(sql_uri)) | ||
| return self.render( | ||
| "admin/migration.html", | ||
| message=migrator, | ||
| aligned_volumes=get_allowed_aligned_volumes(), | ||
| ) | ||
|
|
||
| @expose("/migrate_annotation_schemas", methods=["POST"]) | ||
| @auth_required | ||
| def migrate_annotation_schemas(self): | ||
| sql_url = request.form.get("sql_url") | ||
| aligned_volume = request.form.get("aligned_volume") | ||
| dry_run = request.form.get("dry_run") == "true" | ||
| migrator = DynamicMigration(sql_url, aligned_volume) | ||
| migrations = migrator.upgrade_annotation_models(dry_run=dry_run) | ||
| return self.render( | ||
| "admin/migration.html", | ||
| message=migrations, | ||
| aligned_volumes=get_allowed_aligned_volumes(), | ||
| ) | ||
|
|
||
| @expose("/migrate_foreign_key_constraints", methods=["POST"]) | ||
| @auth_required | ||
| def migrate_foreign_key_constraints(self): | ||
| sql_url = request.form.get("sql_url") | ||
| aligned_volume = request.form.get("aligned_volume") | ||
| dry_run = request.form.get("dry_run") == "true" | ||
| migrator = DynamicMigration(sql_url, aligned_volume) | ||
| fkey_constraint_mapping = migrator.apply_cascade_option_to_tables( | ||
| dry_run=dry_run | ||
| ) | ||
| return self.render( | ||
| "admin/migration.html", | ||
| message=fkey_constraint_mapping, | ||
| aligned_volumes=get_allowed_aligned_volumes(), | ||
| ) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| <!DOCTYPE html> | ||
| <html lang="en"> | ||
| <head> | ||
| <meta charset="UTF-8"> | ||
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> | ||
| <title>Access Denied</title> | ||
| <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css"> | ||
| </head> | ||
| <body> | ||
| <div class="container mt-5"> | ||
| <div class="alert alert-danger" role="alert"> | ||
| <h4 class="alert-heading">Access Denied</h4> | ||
| <p>You do not have the necessary permissions to access this page.</p> | ||
| <hr> | ||
| <p class="mb-0">If you believe this is an error, please contact the administrator.</p> | ||
| </div> | ||
| <a href="{{ url_for('index') }}" class="btn btn-primary">Go to Home</a> | ||
| </div> | ||
| </body> | ||
| </html> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,102 @@ | ||
| <!DOCTYPE html> | ||
| <html lang="en"> | ||
| <head> | ||
| <meta charset="UTF-8"> | ||
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> | ||
| <title>Migrations</title> | ||
| <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css"> | ||
| </head> | ||
| <body> | ||
| <div class="container mt-5"> | ||
| <h1 class="mb-4">Database Migrations</h1> | ||
|
|
||
| <div class="card mb-4"> | ||
| <div class="card-header"> | ||
| Migrate Static Schemas | ||
| </div> | ||
| <div class="card-body"> | ||
| <form method="post" action="{{ url_for('migrationview.migrate_static_schemas') }}"> | ||
| <div class="form-group"> | ||
| <label for="sql_url">SQL URL:</label> | ||
| <input type="text" class="form-control" id="sql_url" name="sql_url" value="{{ config['SQLALCHEMY_DATABASE_URI'] }}"> | ||
| </div> | ||
| <div class="form-group"> | ||
| <label for="aligned_volume">Aligned Volume:</label> | ||
| <select class="form-control" id="aligned_volume" name="aligned_volume"> | ||
| {% for volume in aligned_volumes %} | ||
| <option value="{{ volume }}">{{ volume }}</option> | ||
| {% endfor %} | ||
| </select> | ||
| </div> | ||
| <button type="submit" class="btn btn-primary">Migrate Static Schemas</button> | ||
| </form> | ||
| </div> | ||
| </div> | ||
|
|
||
| <div class="card mb-4"> | ||
| <div class="card-header"> | ||
| Migrate Annotation Schemas | ||
| </div> | ||
| <div class="card-body"> | ||
| <form method="post" action="{{ url_for('migrationview.migrate_annotation_schemas') }}"> | ||
| <div class="form-group"> | ||
| <label for="sql_url">SQL URL:</label> | ||
| <input type="text" class="form-control" id="sql_url" name="sql_url" value="{{ config['SQLALCHEMY_DATABASE_URI'] }}"> | ||
| </div> | ||
| <div class="form-group"> | ||
| <label for="aligned_volume">Aligned Volume:</label> | ||
| <select class="form-control" id="aligned_volume" name="aligned_volume"> | ||
| {% for volume in aligned_volumes %} | ||
| <option value="{{ volume }}">{{ volume }}</option> | ||
| {% endfor %} | ||
| </select> | ||
| </div> | ||
| <div class="form-group form-check"> | ||
| <input type="checkbox" class="form-check-input" id="dry_run" name="dry_run" value="true" checked> | ||
| <label class="form-check-label" for="dry_run">Dry Run</label> | ||
| </div> | ||
| <button type="submit" class="btn btn-primary">Migrate Annotation Schemas</button> | ||
| </form> | ||
| </div> | ||
| </div> | ||
|
|
||
| <div class="card mb-4"> | ||
| <div class="card-header"> | ||
| Alter Foreign Key Constraints | ||
| </div> | ||
| <div class="card-body"> | ||
| <form method="post" action="{{ url_for('migrationview.migrate_foreign_key_constraints') }}"> | ||
| <div class="form-group"> | ||
| <label for="sql_url">SQL URL:</label> | ||
| <input type="text" class="form-control" id="sql_url" name="sql_url" value="{{ config['SQLALCHEMY_DATABASE_URI'] }}"> | ||
| </div> | ||
| <div class="form-group"> | ||
| <label for="aligned_volume">Aligned Volume:</label> | ||
| <select class="form-control" id="aligned_volume" name="aligned_volume"> | ||
| {% for volume in aligned_volumes %} | ||
| <option value="{{ volume }}">{{ volume }}</option> | ||
| {% endfor %} | ||
| </select> | ||
| </div> | ||
| <div class="form-group form-check"> | ||
| <input type="checkbox" class="form-check-input" id="dry_run" name="dry_run" value="true" checked> | ||
| <label class="form-check-label" for="dry_run">Dry Run</label> | ||
| </div> | ||
| <button type="submit" class="btn btn-primary">Alter Foreign Key Constraints</button> | ||
| </form> | ||
| </div> | ||
| </div> | ||
|
|
||
| {% if message %} | ||
| <div class="alert alert-info mt-4"> | ||
| <h3>Migration Result:</h3> | ||
| <pre>{{ message }}</pre> | ||
| </div> | ||
| {% endif %} | ||
| </div> | ||
|
|
||
| <script src="https://code.jquery.com/jquery-3.5.1.slim.min.js"></script> | ||
| <script src="https://cdn.jsdelivr.net/npm/@popperjs/core@2.5.3/dist/umd/popper.min.js"></script> | ||
| <script src="https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/js/bootstrap.min.js"></script> | ||
| </body> | ||
| </html> |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
shouldn't this require admin?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When I tested it if the user is not an admin it hides the migration tab in the admin view, but we should use the decorator as well, I forgot about the auth requires admin decorator, simple fix.