Skip to content

Feat/transport memory usage #401

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
alexglasertpx wants to merge 26 commits into
base: main
Choose a base branch
from
Open

Feat/transport memory usage #401

alexglasertpx wants to merge 26 commits into from

Conversation

@alexglasertpx
Copy link
Contributor

@alexglasertpx alexglasertpx commented May 12, 2025

What type of PR is this? (check all applicable)

  • Refactor
  • Feature
  • Bug Fix
  • Optimization
  • Documentation Update

Description

Following on from previous branch, correcting output_path

Added/updated tests?

  • Yes
  • No, and this is why: please replace this line with details on why tests
    have not been included
  • I need help with writing tests

alexglasertpx added 26 commits May 1, 2025 16:55
@alexglasertpx
Added time and memory check
7757371
@alexglasertpx
Added libraries required for testing
48e2b11
@alexglasertpx
Added libraries required for testing
fc94974
@alexglasertpx
Ran black
7656cd2
@alexglasertpx
Calling new table commands
05a1bad
@alexglasertpx
Edited logging message
724389a
@alexglasertpx
Bucketed window function
b6e9ad4
@alexglasertpx
Ran black
7c1d37a
@alexglasertpx
Tidy up of code
987ecd2
@alexglasertpx
Tidy up of code
285593b
@alexglasertpx
Using 'with' to ensure cleanup if any issues
fdd293b
@alexglasertpx
Added tests for new parquet functions
7ced24e
@alexglasertpx
Incorporating batching and bucketing into the calculation of facts, f…
77b80f8 
…act_resources and entities
@alexglasertpx
Removing temp_parquet tests as no longer using those functions
7393361
@alexglasertpx
Removing tempfile library
422485e
@alexglasertpx
Corrected typo
572a2de
@alexglasertpx
Edited comment
b784231
@alexglasertpx
Edited exit message
137c9a2
@alexglasertpx
Check strategy before calling function
a07b535
@alexglasertpx
Added test for batch processing of parquet files
58002c0
@alexglasertpx
Remove commented out test
635e5a5
@alexglasertpx
Removed old functions using a single temporary parquet file
0fb9035
@alexglasertpx
Removed comments
c669ba6
@alexglasertpx
Minor edit to comment
4f4c910
@alexglasertpx
Edits to logger info
8eea573
@alexglasertpx
Edit to output_path
f06d158
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@CarlosCoelhoSL CarlosCoelhoSL Awaiting requested review from CarlosCoelhoSL

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@alexglasertpx