upgrade pg_walstream version to 0.5.00

[isdaniel]

• Updated all test cases in replica_identity_tests.rs, sqlite_comprehensive_tests.rs, sqlite_destination_tests.rs, and where_clause_fix_tests.rs to replace serde_json::Value with pg_walstream::ColumnValue for consistency and improved type safety.
• Modified the format_value function to format ColumnValue types correctly in SQL statements.
• Introduced a new ReplicationActor to manage the LogicalReplicationStream in a dedicated thread, ensuring thread safety and proper command handling.
• Enhanced the actor's command structure to support starting the replication stream, reading events, sending feedback, and gracefully stopping the actor.

[gemini-code-assist]

Summary of Changes

Hello @isdaniel, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request focuses on upgrading the pg_walstream dependency to the latest version and improving the architecture around the logical replication stream. By introducing a dedicated ReplicationActor, the changes ensure better thread safety and management of replication tasks. The update also standardizes data types used in test cases and SQL formatting, contributing to more robust and maintainable code.

Highlights

• pg_walstream Version Upgrade: The pg_walstream dependency was upgraded from version 0.4.1 to 0.5.0, incorporating new features and improvements.
• Replication Actor Introduction: Introduced a ReplicationActor to manage the LogicalReplicationStream in a dedicated thread, enhancing thread safety and command handling.
• Data Type Consistency: Test cases were updated to use pg_walstream::ColumnValue instead of serde_json::Value for improved type safety and consistency.
• SQL Formatting: The format_value function was modified to correctly format ColumnValue types in SQL statements.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• Cargo.lock
  • Updated pg_walstream version to 0.5.0 and updated chrono version to 0.4.44.
• Cargo.toml
  • Updated pg_walstream version to 0.5.0.
• pg2any-lib/src/client.rs
  • Introduced ReplicationActorHandle and removed Mutex for LogicalReplicationStream.
  • Modified CdcClient to use ReplicationActorHandle for managing the replication stream.
  • Updated the start function to use the replication actor to start the replication stream.
  • Modified the producer to use the actor command sender instead of the replication stream.
  • Updated the shutdown process to use the replication actor to stop the stream.
• pg2any-lib/src/lib.rs
  • Added replication_actor module.
  • Removed format_postgres_timestamp from exports.
• pg2any-lib/src/replication_actor.rs
  • Added a new module for managing the LogicalReplicationStream in a dedicated thread.
• pg2any-lib/src/transaction_manager.rs
  • Replaced serde_json::Value with pg_walstream::ColumnValue.
  • Modified generate_insert_sql, generate_update_sql, and build_where_clause to use iter() and ColumnValue.
  • Updated format_value function to handle ColumnValue types.
• pg2any-lib/tests/destination_integration_tests.rs
  • Replaced serde_json::Value with ColumnValue in test events.
• pg2any-lib/tests/event_type_refactor_tests.rs
  • Replaced serde_json::json with ColumnValue in test data.
• pg2any-lib/tests/metrics_logical_tests.rs
  • Replaced serde_json::Value with ColumnValue in test events.
• pg2any-lib/tests/mysql_edge_cases_tests.rs
  • Replaced serde_json::Value with ColumnValue in test data.
• pg2any-lib/tests/mysql_error_handling_simple_tests.rs
  • Replaced serde_json::Value with ColumnValue in test data.
• pg2any-lib/tests/mysql_where_clause_fix_tests.rs
  • Replaced serde_json::Value with ColumnValue in test data.
• pg2any-lib/tests/replica_identity_tests.rs
  • Replaced serde_json::Value with ColumnValue in test data.
• pg2any-lib/tests/sqlite_comprehensive_tests.rs
  • Replaced serde_json::json with ColumnValue in test data and format_column_value function.
• pg2any-lib/tests/sqlite_destination_tests.rs
  • Replaced serde_json::json with ColumnValue in test data and format_column_value function.
• pg2any-lib/tests/where_clause_fix_tests.rs
  • Replaced serde_json::Value with ColumnValue in test data.

Activity

• The pull request involves upgrading the pg_walstream dependency and refactoring the client to use a ReplicationActor for managing the logical replication stream.
• It also includes updates to test cases to use ColumnValue instead of serde_json::Value for data consistency.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request significantly upgrades pg_walstream and refactors the code to use the new ColumnValue type, enhancing type safety and performance. It also introduces the ReplicationActor for robust, thread-safe logical replication stream management. However, a critical security audit identified SQL injection vulnerabilities in the SQL generation logic within transaction_manager.rs due to improper escaping of database identifiers and insufficient escaping of string literals for MySQL destinations. It is highly recommended to implement a centralized identifier escaping helper and ensure MySQL-specific escaping rules are followed to prevent arbitrary SQL execution. Additionally, a high-severity concern exists regarding a potential deadlock in the drop implementation of the ReplicationActorHandle.

[isdaniel]

/gemini review

[gemini-code-assist]

Code Review

This pull request upgrades "pg_walstream" and refactors the codebase to use "ColumnValue" instead of "serde_json::Value", which is a solid improvement for type safety and consistency. However, the manual construction of SQL strings introduces several SQL injection vulnerabilities, specifically due to unescaped database identifiers and incomplete string escaping for MySQL. It is highly recommended to use parameterized queries or a robust SQL builder. Furthermore, the "format_value" function in "transaction_manager.rs" incorrectly formats numeric and boolean values as string literals, which could lead to runtime errors or incorrect data handling.

[gemini-code-assist]

Code Review

This pull request upgrades pg_walstream and refactors the code to use pg_walstream::ColumnValue instead of serde_json::Value. While these changes improve type safety and performance, a critical SQL injection vulnerability has been identified in the format_value function within transaction_manager.rs due to improper backslash escaping for MySQL destinations. Additionally, there's a regression in SQL value formatting that could lead to incorrect data types, and some duplicated code in the test suite could benefit from refactoring.