Skip to content

Add copy/pasting detection #3149

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
CarolineDenis wants to merge 12 commits into
base: main
Choose a base branch
from
Open

Add copy/pasting detection #3149

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
a4bded2
Add copy/pasting detection
CarolineDenis Mar 10, 2023
66a8a6c
Fixes duplicate in legacyTypes
CarolineDenis Apr 6, 2023
aafda56
fixes duplicate in plugins.ts
CarolineDenis Apr 6, 2023
7019a28
Disable copy paste detection for python packages folder
maxpatiiuk Apr 20, 2023
8c8af30
refactor(repository): cleanup .gitignore
maxpatiiuk Jun 22, 2024
5f36122
feat(tests): add .gitigonre to glob converstion util
maxpatiiuk Jun 22, 2024
2680813
feat(tests): improve jscpd config
maxpatiiuk Jun 22, 2024
6ec660f
docs(tests): add instructions in case copy paste was detected
maxpatiiuk Jun 22, 2024
13b2633
refactor(repository): deduplicate copy/pasted code
maxpatiiuk Jun 22, 2024
77ecd68
Lint code with ESLint and Prettier
maxpatiiuk Nov 30, 2024
61bc8d7
refactor: do self-code-review
maxpatiiuk Nov 30, 2024
cd39af8
Lint code with ESLint and Prettier
maxpatiiuk Jan 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -279,6 +279,15 @@ jobs:
run: |
npx prettier --write --minify `echo "${{steps.changed.outputs.changed}}" | tr " " "\n"` || true

- name: Detect possible code duplications
working-directory: specifyweb/frontend/js_src
run: |
npx tsx lib/tests/jscpd.ts

echo "If code duplications were found, you have the following options:"
echo "1. Refactor the code to deduplicate the code"
echo "2. If you believe having the duplication is better for readability/simplicity, then you can ignore the duplication. Instructions: https://github.com/kucherenko/jscpd/tree/master/apps/jscpd#ignored-blocks"

- name: Commit linted files (if made any changes)
working-directory: specifyweb/frontend/js_src
# Creates a new commit. Amending an existing commit is a bad idea
Expand Down
50 changes: 31 additions & 19 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,33 +1,45 @@
# Setup
/seed-database/
*.env
.env
docker-compose.yml

# Python
/venv/
/ve/
*.pyc
*~
.vagrant

# OS
.DS_Store

# Editors
/.vscode/
/.idea/
Comment on lines +12 to +17
Copy link
Member

@maxpatiiuk maxpatiiuk Jun 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used to be against OS or editor specific gitignore patterns, and was recommending using global gitignore for that but I changed my mind.

Reasons:

  • Setting up global gitignore is added friction, especially for new team members or external contributors.
  • Automated tools (like jscpd, eslint, prettier) and some IDEs do not look into global gitignore file

.emacs.desktop*
.directory
testing
specifypanel/ve
specifyweb/virtenv
venv/
ve/
seed-database
*.env

# Back-end
/specifyweb/settings/local_settings.py
/specifyweb/settings/local_specify_settings.py
/specifyweb/settings/local_logging_settings.py
/specifyweb/settings/debug.py
/specifyweb/settings/secret_key.py
/specifyweb/settings/ldap_settings.py

# Build artifacts
/.mypy_cache
/specifyweb/frontend/locale/**/*.mo
/specifyweb/settings/build_version.py
/specifyweb/frontend/static/js
/specifyweb/frontend/static/css
/specifyweb/frontend/static/js/
/specifyweb/frontend/static/css/
/specifyweb/frontend/static/manifest.json
node_modules
/specifyweb/frontend/js_src/testBuild
/specifyweb/frontend/js_src/docs
/specifyweb/frontend/js_src/coverage
/specifyweb/frontend/js_src/node_modules
/specifyweb/frontend/js_src/testBuild/
/specifyweb/frontend/js_src/docs/
/specifyweb/frontend/js_src/coverage/
/local_specifyweb_apache.conf
/specifyweb/frontend/js_src/stats.json
/specifyweb/frontend/js_src/dist
.env
docker-compose.yml
/specifyweb/frontend/js_src/dist/

# Misc
*~
testing/
133 changes: 10 additions & 123 deletions specifyweb/context/collection_resources.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,76 +2,13 @@
from specifyweb.specify.models import Spappresource, Spappresourcedir
from specifyweb.specify.views import openapi
from specifyweb.context.resources import Resource, Resources
from specifyweb.context.user_resources import get_resources_endpoint_schema, get_resource_endpoint_schema

collection_resources = openapi(schema={
"get": {
"responses": {
"200": {
"description": "Returns list of public app resources in the logged in collection.",
"content": {
"application/json": {
"schema": {
"type": "array",
"items": {
"type": "object",
"properties": {
"id": { "type": "integer", "description": "The appresource id." },
"name": { "type": "string", "description": "The appresource name." },
"mimetype": { "type": "string" },
"metadata": { "type": "string" },
},
'required': ['id', 'name', 'mimetype', 'metadata'],
'additionalProperties': False
}
}
}
}
}
}
},
"post": {
"requestBody": {
"required": True,
"description": "Creates appresource in the logged in collection owned by the logged in user.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"name": { "type": "string", "description": "The appresource name." },
"mimetype": { "type": "string" },
"metadata": { "type": "string" },
"data": { "type": "string", "description": "The data to be stored in the appresource." },
},
'required': ['name', 'mimetype', 'metadata', 'data'],
'additionalProperties': False
}
}
}
},
"responses": {
"201": {
"description": "The user resource was created.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": { "type": "integer", "description": "The appresource id." },
"name": { "type": "string", "description": "The appresource name." },
"mimetype": { "type": "string" },
"metadata": { "type": "string" },
"data": { "type": "string", "description": "The data to be stored in the appresource." },
},
'required': ['name', 'mimetype', 'metadata', 'data'],
'additionalProperties': False
}
}
}
}
}
}
})(Resources.as_view(_spappresourcedirfilter= lambda request: {
collection_resources = openapi(schema=get_resources_endpoint_schema(
description_get="Returns list of public app resources in the logged in collection.",
description_create="Creates appresource in the logged in collection.",
description_created="The collection resource was created.",
))(Resources.as_view(_spappresourcedirfilter= lambda request: {
'ispersonal': False,
'specifyuser__isnull': True,
'usertype__isnull': True,
Expand All @@ -86,60 +23,10 @@
}))


collection_resource = openapi(schema={
"get": {
"responses": {
"200": {
"description": "The public app resource of the given id in the logged in collection ",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": { "type": "integer", "description": "The appresource id." },
"name": { "type": "string", "description": "The appresource name." },
"mimetype": { "type": "string" },
"metadata": { "type": "string" },
"data": { "type": "string", "description": "The data to be stored in the appresource." },
},
'required': ['id', 'name', 'mimetype', 'metadata', 'data'],
'additionalProperties': False
}
}
}
}
}
},
"put": {
"requestBody": {
"required": True,
"description": "Updates the appresource with the given id in the logged in collection",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"name": { "type": "string", "description": "The appresource name." },
"mimetype": { "type": "string" },
"metadata": { "type": "string" },
"data": { "type": "string", "description": "The data to be stored in the appresource." },
},
'required': ['name', 'mimetype', 'metadata', 'data'],
'additionalProperties': False
}
}
}
},
"responses": {
"204": { "description": "The resource was updated.", },
}
},
"delete": {
"responses": {
"204": {"description": "The resource was deleted.",}
}
}
})(Resource.as_view(_spappresourcefilter= lambda request: {
collection_resource = openapi(schema=get_resource_endpoint_schema(
description_get="The public app resource of the given id in the logged in collection",
description_update="Updates the appresource with the given id in the logged in collection"
))(Resource.as_view(_spappresourcefilter= lambda request: {
'spappresourcedir__ispersonal': False,
}))

Expand Down
Loading
Loading