Refactor DBTypeDatabase into duckdb and postgres #47
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In order to make fewer changes for the COPY FROM feature I created a single implementation of Database that utilized DBType. Now that sqlite support is gone I've created two implementations for duckdb and postgres. I wanted to use the dbapi types but duckdb doesn't quite play nicely with them. Doing this refactor makes the next few features easier to implement.
I was never happy with the iteration of records and ingestion of records. This moves the __id generation into the database logic away from the iteration logic. I also cleaned up the magic "first record is the total" behavior. As part of this I changed how tqdm was initialized to be closer to how I think it is intended to be used.