-
Notifications
You must be signed in to change notification settings - Fork 30
Bump version to 0.3.1-SNAPSHOT and update documentation #468
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Bump version to 0.3.1-SNAPSHOT and update documentation #468
Conversation
Summary of ChangesHello @rohitkumarbhagat, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request advances the project's development lifecycle by updating its internal version to "0.3.1-SNAPSHOT". It significantly improves the build and deployment reliability by implementing dynamic JAR file discovery in various CI/CD scripts, eliminating dependencies on hardcoded version strings. Additionally, the project's Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request updates the project version and build scripts to dynamically discover JAR files, avoiding hardcoded versions, and updates documentation. While this improves flexibility, it introduces a command injection vulnerability in pipeline/ingestion/cloudbuild.yaml due to the insecure use of Cloud Build substitutions within a bash -c block. Furthermore, the current implementation for finding the JAR file using ls | head -n 1 is not robust and could lead to deploying an older artifact if the build directory isn't clean.
| gs://$_TEMPLATE_BUCKET/templates/flex/ingestion.json \ | ||
| --image-gcr-path "${_IMAGE_GCR_PATH}:$SHORT_SHA" \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This section introduces a command injection vulnerability due to the insecure use of Cloud Build substitutions ($_TEMPLATE_BUCKET, ${_IMAGE_GCR_PATH}) within a bash -c block. Cloud Build performs string replacement before passing the script to the shell, which can lead to command injection if an attacker can influence these substitutions. Specifically, on line 32, $_TEMPLATE_BUCKET is unquoted, allowing command breakout, and on line 33, ${_IMAGE_GCR_PATH} inside double quotes still allows command substitution (e.g., $(id)). To remediate this, pass the substitutions as environment variables to the build step and use them within the shell script using shell variable expansion (e.g., "$${TEMPLATE_BUCKET}").
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any thoughts on this?
| # Find the JAR file using wildcard | ||
| JAR_FILE=$(ls tool/target/datacommons-import-tool-*-jar-with-dependencies.jar | head -n 1) | ||
| echo "Found JAR: ${JAR_FILE}" | ||
| gsutil cp "${JAR_FILE}" "gs://datacommons_public/import_tools/import-tool.jar" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The use of ls ... | head -n 1 to find the JAR file is not robust. If the build directory is not clean, multiple JAR files with different versions might exist. ls sorts alphabetically, so this command could select an older version, leading to the deployment of an incorrect artifact. A safer approach is to ensure that exactly one JAR file matches the pattern and fail the build otherwise.
# Find the JAR file, ensuring there is exactly one.
JAR_FILES=(tool/target/datacommons-import-tool-*-jar-with-dependencies.jar)
if [ "${#JAR_FILES[@]}" -ne 1 ]; then
echo "Error: Expected 1 JAR file, but found ${#JAR_FILES[@]}. Aborting." >&2
exit 1
fi
JAR_FILE="${JAR_FILES[0]}"
echo "Found JAR: ${JAR_FILE}"
gsutil cp "${JAR_FILE}" "gs://datacommons_public/import_tools/import-tool.jar"There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we consider passing the version as a substitution variable (similar to API key)?
| # Find the JAR file using wildcard | ||
| JAR_FILE=$(ls target/differ-bundled-*.jar | head -n 1) | ||
| echo "Found JAR: ${JAR_FILE}" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The use of ls ... | head -n 1 to find the JAR file is not robust. If the build directory is not clean, multiple JAR files with different versions might exist. ls sorts alphabetically, so this command could select an older version. A safer approach is to ensure that exactly one JAR file matches the pattern.
| # Find the JAR file using wildcard | |
| JAR_FILE=$(ls target/differ-bundled-*.jar | head -n 1) | |
| echo "Found JAR: ${JAR_FILE}" | |
| # Find the JAR file, ensuring there is exactly one. | |
| JAR_FILES=(target/differ-bundled-*.jar) | |
| if [ "${#JAR_FILES[@]}" -ne 1 ]; then | |
| echo "Error: Expected 1 JAR file, but found ${#JAR_FILES[@]}. Aborting." >&2 | |
| exit 1 | |
| fi | |
| JAR_FILE="${JAR_FILES[0]}" | |
| echo "Found JAR: ${JAR_FILE}" |
| # Find the JAR file using wildcard | ||
| JAR_FILE=$(ls target/ingestion-bundled-*.jar | head -n 1) | ||
| echo "Found JAR: ${JAR_FILE}" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The use of ls ... | head -n 1 to find the JAR file is not robust. If the build directory is not clean, multiple JAR files with different versions might exist. ls sorts alphabetically, so this command could select an older version. A safer approach is to ensure that exactly one JAR file matches the pattern.
| # Find the JAR file using wildcard | |
| JAR_FILE=$(ls target/ingestion-bundled-*.jar | head -n 1) | |
| echo "Found JAR: ${JAR_FILE}" | |
| # Find the JAR file, ensuring there is exactly one. | |
| JAR_FILES=(target/ingestion-bundled-*.jar) | |
| if [ "${#JAR_FILES[@]}" -ne 1 ]; then | |
| echo "Error: Expected 1 JAR file, but found ${#JAR_FILES[@]}. Aborting." >&2 | |
| exit 1 | |
| fi | |
| JAR_FILE="${JAR_FILES[0]}" | |
| echo "Found JAR: ${JAR_FILE}" |
| <maven.compiler.target>17</maven.compiler.target> | ||
| <!-- Dependency versions --> | ||
| <revision>0.1-SNAPSHOT</revision> | ||
| <revision>0.3.1-SNAPSHOT</revision> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Curious why go from 0.1 to 0.3.1..shall we do 0.1.1 instead?
| gs://$_TEMPLATE_BUCKET/templates/flex/ingestion.json \ | ||
| --image-gcr-path "${_IMAGE_GCR_PATH}:$SHORT_SHA" \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any thoughts on this?
| # Find the JAR file using wildcard | ||
| JAR_FILE=$(ls tool/target/datacommons-import-tool-*-jar-with-dependencies.jar | head -n 1) | ||
| echo "Found JAR: ${JAR_FILE}" | ||
| gsutil cp "${JAR_FILE}" "gs://datacommons_public/import_tools/import-tool.jar" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we consider passing the version as a substitution variable (similar to API key)?
This PR bumps the project version to 0.3.1-SNAPSHOT, updates build scripts to support dynamic JAR discovery, and improves release documentation.