Dev/component graph

README.md

@@ -152,7 +152,7 @@ LOW PRIO
 ## Requirements
 
 - Python3.0+
-- Django2.0+
+- Django3.0+
 
 ## Installation
 
@@ -169,18 +169,19 @@ sudo -u postgres psql
 postgres=# create database panopticum;
 postgres=# create user panopticum with encrypted password 'my secret password';
 postgres=# grant all privileges on database panopticum to panopticum;
-sudo echo "host all all 127.0.0.1/32 md5" > /var/lib/pgsql/11/data/pg_hba.conf
+sudo sh -c 'host all all 127.0.0.1/32 md5" > /var/lib/pgsql/11/data/pg_hba.conf'
 sudo systemctl restart postgresql-11
 ```
 
 ### Install python3
 
 CentOS-7:
 ```
-sudo yum -y install https://centos7.iuscommunity.org/ius-release.rpm
+sudo yum -y install https://repo.ius.io/ius-release-el7.rpm
 sudo yum -y install python36
 sudo yum -y install python36-pip
 sudo yum -y install python36-psycopg2
+sudo yum -y install openssl-devel gcc python3-devel openldap-devel graphviz-devel
 
 ```
 

README_Graph.md

@@ -0,0 +1,111 @@
+# Component Graph Package
+This package adds a component graph feature to Panopticum
+
+
+## Usage
+### Files in this Package
+- /panopticum
+    - /static
+        - /css
+            - graph.css
+        - /fonts
+            - OpenSans-Regular.ttf
+        - /js
+            - graph.js
+            - graph-style.json
+        - /vendors
+            - /cytoscape.js-panzoom
+                - cytoscape.js-panzoom.css
+                - cytoscape-panzoom.js
+            - /cytoscape-klay
+                - cytoscape-klay.min.js
+                - klay.js
+        - /templates
+            - /graph
+                - graph.html
+    - graph.py
+- /panopticum_django
+    - urls.py
+- README_Graph.md (this file)
+
+
+### Adding to Panopticum
+Copy the folder ./Panopticum to <your git repository>/panopticum
+
+
+### Wiring 
+In <your git repository>/panopticum_django, adapt urls.py to expose the view and api.
+
+Add to the import statements:
+```
+from panopticum import graph
+```
+
+Add to the urlpatterns:
+```
+url('^graph.html', graph.graph_view, name='Graph'),
+path('api/graph/component/<int:componentId>/', graph.graph_component),
+```
+
+A sample is included in ./panopticum_django/urls.py
+
+
+## About
+The view is defined by ./panopticum/static/templates/graph.html, which in turn relies on ./panopticum/static/js/graph.js for dynamic rendering of graphical elements. This relies on the API on Panopticum /api/graph/component/<componentId> to deliver Panopticum data.
+
+[Cytoscape.js](https://js.cytoscape.org/) is used as the main library for visualisation. 
+
+
+### API
+This package introduces a new API at /api/graph/component/<componentId> which returns aggregated information about the component and its immediate parent & child dependencies, for use by the frontend.
+
+It accepts an optional parameter 'depth', to limit the search of parents & children.
+
+
+### Frontend
+./panopticum/static/js/graph.js handles the display of panopticum data into a cytoscape graph. Styling business logic is handled here, such as the mapping of the styling class type of a component, based on a Panopticum data field.
+
+The following JavaScript libraries are utilised:
+- [cytoscape.js](https://github.com/cytoscape/cytoscape.js)
+- [cytoscape-klay](https://github.com/cytoscape/cytoscape.js-klay)
+- [cytoscape-panzoom](https://github.com/cytoscape/cytoscape.js-panzoom)
+- [popper.js](https://github.com/popperjs/popper-core)
+- [cytoscape-popper](https://github.com/cytoscape/cytoscape.js-popper)
+- [tippy.js](https://github.com/atomiks/tippyjs/releases)
+
+
+### Configuring Styles
+Cytoscape graph style configuration is managed by ./panopticum/static/js/graph-style.json. This allows for configuration of cytoscape elements, i.e. the nodes, edges and corresponding text. Elements are able to apply multiple styles. Colors, images, text size and font are able to be styled, refer to [Cytoscape Style Documentation](https://js.cytoscape.org/#style) for more.
+
+This package provides the following style types:
+    - 'node', 'edge', default styles for nodes & edges
+    - '.compound', style for category nodes
+    - '.type<>', styles for different component types
+    - '.expandableNode', style for nodes that can be expanded
+    - '.securityAlert', style for nodes that have a security alert
+
+The latter 3 types are stackable on component nodes and are added/removed based on properties of the component and the graph.
+
+Regular CSS styling of the view is done in ./panopticum/static/css/graph.css. For example, styling of the popper elements.
+
+
+## TODO
+- Option for user to click on the component (within graph) and it should land to component detail page.
+- Print functionality
+- Dynamically plot components and chose line styles such that there is minimum cross over lines and better usage of canvas
+- Hover over the component and it shows some information about the component – make this configurable (which fields to be shown)
+- Double click should preserve the previous component position (since user may have taken effort to look it beautiful)
+- Security issue detection shouldn't be hardcoded and should be configurable. User should be able to select which fields to match and provide data to match against.
+- Configuration file details needs to be documented (Readme?)
+- Security issue red border improvement (hope this too is configurable)
+    - Make red border less thicker than it is at present
+    - Use cherry red color instead
+- when user selects a component
+    - and wants to expand parent node then
+        - by default, show only directly linked dependencies to selected component.
+        - show some icon/indicator around the parent indicating there are more components parent depends on. Once this icon/indicator is clicked then expand all other dependencies
+    - have an option to restrict the view to selected depth of the graph. e.g. if user has chosen depth=3 then at max we show 3 levels of connected graph ensuring selecting component is part of the graph.
+
+
+
+

panopticum/admin.py

@@ -72,6 +72,22 @@ class ComponentDependencyAdmin(admin.TabularInline):
     verbose_name = "Component Dependency"
     verbose_name_plural = "Component Dependencies"
 
+    def has_change_permission(self, request, obj=None):
+        """ Allow change requirement if user have permissions """
+        return request.user.has_perm(SIGNEE_STATUS_PERMISSION) or \
+               (request.user.has_perm(OWNER_STATUS_PERMISSION) and
+                obj and request.user == obj.owner_maintainer)
+
+    def _has_add_permission(self, request, obj=None):
+        return True;
+
+    def has_add_permission(self, request, obj=None):
+        return self.has_change_permission(request, obj)
+
+    def has_delete_permission(self, request, obj=None):
+        return False
+
+
 
 class TCPPortAdmin(admin.ModelAdmin):
     search_fields = ['port', ]
@@ -87,6 +103,9 @@ class ComponentDeploymentAdmin(admin.TabularInline):
     verbose_name_plural = "Component Deployments"
     autocomplete_fields = ['open_ports', ]
 
+    def _has_add_permission(self, request, obj=None):
+        return True;
+
 
 class RequirementInlineAdmin(admin.TabularInline):
     model = Requirement
@@ -208,6 +227,9 @@ class RequirementStatusEntryAdmin(admin.TabularInline):
     extra = 1
     verbose_name_plural = "Requirements"
 
+    def _has_add_permission(self, request, obj=None):
+        return True;
+
     def requirement(self, obj):
         return obj.name
 

panopticum/graph.py

@@ -0,0 +1,159 @@
+import django.http
+import rest_framework.decorators
+import rest_framework.status
+from django.shortcuts import render
+from django.conf import settings
+from django.http import JsonResponse
+from django.contrib.auth.decorators import login_required
+
+from rest_framework import viewsets, permissions, views
+from rest_framework.response import Response
+from rest_framework.decorators import action
+from rest_framework.views import APIView
+import rest_framework.filters
+
+import panopticum.filters
+from .models import *
+from .serializers import *
+
+import sys
+
+@login_required
+def graph_view(request):
+    return render(request, 'graph/graph.html')
+
+def graph_component(request, componentId):
+    depth = int(request.GET.get('depth', '1'))
+    if (depth < 0):
+        depth = 1
+
+    component_version_param = request.GET.get('component_version', None)
+
+    components_list = []
+    componentIds_set = set()
+    categoryIds_set = set()
+
+    start_component = ComponentModel.objects.get(id=componentId)
+    if component_version_param:
+        start_component_version = ComponentVersionModel.objects.filter(component=start_component.id, id=int(component_version_param)).first()
+    else:
+        start_component_version = ComponentVersionModel.objects.filter(component=start_component.id).order_by('-update_date').first()
+
+    # iterate forwards
+    process_queue = []
+    process_queue.append((start_component, start_component_version))
+    for i in range(depth + 1):
+        new_process_queue = []
+        for (curComponent, curVersion) in process_queue:
+            # add component
+            if curComponent.id not in componentIds_set:
+                componentIds_set.add(curComponent.id)
+                components_list.append(make_graph_component(curComponent, curVersion))
+                categoryIds_set.add(curComponent.category.id)
+
+            dependency_objects = ComponentDependencyModel.objects.filter(version = curVersion)
+            for dependency in curVersion.depends_on.all():
+                dependency_object = dependency_objects.filter(component=dependency).first()
+                dependency_version = ComponentVersionModel.objects.filter(component=dependency.id).order_by('-update_date').first()
+                new_process_queue.append((dependency, dependency_version))
+
+        process_queue = new_process_queue
+
+    # iterate backwards
+    process_queue = []
+    process_queue.append((start_component, start_component_version))
+    for i in range(depth + 1):
+        new_process_queue = []
+        for (curComponent, curVersion) in process_queue:
+            # add component
+            if curComponent.id not in componentIds_set:
+                componentIds_set.add(curComponent.id)
+                components_list.append(make_graph_component(curComponent, curVersion))
+                categoryIds_set.add(curComponent.category.id)
+
+            dependents = ComponentDependencyModel.objects.filter(component = curComponent.id) # find components that depends on this version
+            for dep in dependents:
+                dependency_object = dependents.filter(version=dep.version).first()
+                dependency_version = ComponentVersionModel.objects.filter(component=dep.version.component.id).order_by('-update_date').first()
+                new_process_queue.append((dep.version.component, dependency_version))
+
+        process_queue = new_process_queue
+
+    # return: component ID, name, parent, forward & backward edges
+
+    response = {
+        'components': components_list,
+        'categories': get_categories(categoryIds_set)
+    }
+
+    return JsonResponse(response)
+
+def get_categories(categoryIds):
+    categoriesIds_set = set(categoryIds)
+    categories_list = []
+    for catId in categoriesIds_set:
+        categoryObj = ComponentCategoryModel.objects.filter(id = catId).first()
+        categories_list.append({
+            "id": catId,
+            "name": categoryObj.name
+        })
+
+    return categories_list
+
+def make_graph_component(component, component_version):
+    # get forward dependencies
+    dependsOn_List = []
+    dependsOn_DepObjects = ComponentDependencyModel.objects.filter(version = component_version)
+    for dependsOn_Component in component_version.depends_on.all():
+        dependsOn_Dependency = dependsOn_DepObjects.filter(component = dependsOn_Component).first()
+        dependsOn_List.append({
+            "id": dependsOn_Component.id,
+            "type": dependsOn_Dependency.type
+        })
+
+    # get backward dependencies
+    dependedUpon_List = []
+    dependedUpon_IdSet = set()
+    dependendUpon_DepObjects = ComponentDependencyModel.objects.filter(component = component.id)
+    for dependedUpon_Dependency in dependendUpon_DepObjects:
+        if dependedUpon_Dependency.version.component.id not in dependedUpon_IdSet:
+            dependedUpon_IdSet.add(dependedUpon_Dependency.version.component.id)
+            dependedUpon_List.append({
+                "id": dependedUpon_Dependency.version.component.id
+            })
+
+    haReq = RequirementStatusEntry.objects.filter(component_version=component_version.id, requirement=8).first()
+    alerts = evaluate_security_alert(component, component_version)
+
+    obj = {
+        'id': component.id,
+        'name': component.name,
+        'category': component.category.id,
+        'dependsOn': dependsOn_List,
+        'dependedUpon': dependedUpon_List,
+        'type': component.type.name,
+        'ha': haReq.status.name,
+        'sec': alerts,
+        "info": {
+            "version": component_version.version,
+            "desc": component.description,
+            "privacy": component.data_privacy_class.name
+        }
+    }
+
+    return obj
+
+def evaluate_security_alert(component, component_version):
+    # rules
+    badPorts = [20, 21, 22, 23, 25, 53, 139, 445, 1433, 1434, 3306, 3389]
+
+    alerts = []
+    deployments = ComponentDeploymentModel.objects.filter(component_version=component_version.id)
+    for deployment in deployments:
+        # check ports
+        for open_port in deployment.open_ports.all():
+            if open_port.port in badPorts:
+                alerts.append(f"Vulnerable port {open_port.port} ({open_port.name})")
+
+    return alerts
+

panopticum/models.py

@@ -1,7 +1,5 @@
 import django.forms
 from django.db import models
-from datatableview.views import DatatableView
-from datatableview import helpers
 from django.forms.models import model_to_dict
 import datetime
 from django.contrib.auth.models import AbstractUser, Group