Skip to content

Replace dynamodb scan with query#31

Open
Igor Rodionov (goruha) wants to merge 12 commits intomainfrom
replace-dynamodb-scan-with-query
Open

Replace dynamodb scan with query#31
Igor Rodionov (goruha) wants to merge 12 commits intomainfrom
replace-dynamodb-scan-with-query

Conversation

@goruha
Copy link
Copy Markdown
Member

@goruha Igor Rodionov (goruha) commented Jun 12, 2024

what

  • Replace DynamoDB scan with query

why

  • Because scan works by batch 1Mb of data.
If the total size of scanned items exceeds the maximum dataset size limit of 1 MB, the scan completes and results are returned to the user. The LastEvaluatedKey value is also returned and the requestor can use the LastEvaluatedKey to continue the scan in a subsequent operation.
  • Reduce cost. query is cheaper then scan
  • Fix false negatives get plan cases

CleanShot 2024-06-12 at 14 33 55@2x

references

Migration

  • Create dynamodb index ``
          - name: commitSHA-index
            hash_key: commitSHA
            range_key: createdAt
            projection_type: ALL
            non_key_attributes: []
            read_capacity: null
            write_capacity: null

Igor Rodionov (goruha) and others added 3 commits June 12, 2024 12:36
…sse/github-action-terraform-plan-storage into replace-dynamodb-scan-with-query

* 'replace-dynamodb-scan-with-query' of github.com:cloudposse/github-action-terraform-plan-storage:
  Update contents of the dist directory
@goruha Igor Rodionov (goruha) added the major Breaking changes (or first stable release) label Jun 12, 2024
@goruha Igor Rodionov (goruha) added the do not merge Do not merge this PR, doing so would cause problems label Jun 13, 2024
KeyConditionExpression: "#commitSHA= :commitSHA",
FilterExpression:
"#owner = :owner and #repo = :repo and #commitSHA = :commitSHA and #component = :component and #stack = :stack",
"#owner = :owner and #repo = :repo and #component = :component and #stack = :stack",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we removing the condition to match the commit SHA? I don't see how this could be correct?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do not merge Do not merge this PR, doing so would cause problems major Breaking changes (or first stable release)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants