Skip to content

Commit fea5906

Browse files
authored
feat(encryption) [1/N] Support encryption: Add crypto for AES-GCM (#2026)
Add Core Encryption Primitives for Iceberg Encryption Support. Part of #2034 ## Summary This PR introduces the foundational cryptographic primitives needed for implementing encryption in iceberg-rust, providing AES-GCM encryption operations that match the Java implementation's behavior and data format. ## Motivation Iceberg's Java implementation supports table-level encryption to protect sensitive data at rest. To achieve feature parity and ensure interoperability between Java and Rust implementations, we need to build encryption support from the ground up. This PR provides the core cryptographic operations that will serve as the foundation for the complete encryption feature. ## Changes New Module: encryption Added a new encryption module with core AES-GCM cryptographic operations: - encryption/crypto.rs - Core encryption implementation - EncryptionAlgorithm enum supporting AES-128-GCM as this is the only algorithm currently supported in arrow parquet - SecureKey struct with automatic memory zeroization for security - AesGcmEncryptor providing encrypt/decrypt operations with AAD support Key Features 1. Java-Compatible Format: Ciphertext format matches Java's implementation exactly: [12-byte nonce][encrypted data][16-byte GCM authentication tag] 1. This ensures files encrypted by Java can be decrypted by Rust and vice versa. 2. Secure Key Handling: Uses the zeroize crate to automatically clear encryption keys from memory when dropped, preventing key material from lingering in memory. 3. Additional Authenticated Data (AAD): Full support for AAD to ensure integrity of associated metadata that isn't encrypted. 4. Comprehensive Testing: 8 tests covering: - Round-trip encryption/decryption for both AES-128 and AES-256 - AAD validation - Empty plaintext handling - Tamper detection - Format compatibility verification Dependencies Added - aes-gcm = "0.10" - Industry-standard AES-GCM implementation - zeroize = "1.7" - Secure memory cleanup for encryption keys Compatibility This implementation directly corresponds to Java's https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/encryption/Ciphers.java: | Java Class | Rust Implementation | |-----------------------------|------------------------------------------| | Ciphers.AesGcmEncryptor | AesGcmEncryptor::encrypt() | | Ciphers.AesGcmDecryptor | AesGcmEncryptor::decrypt() | | EncryptionAlgorithm.AES_GCM | EncryptionAlgorithm::Aes128Gcm| Testing Future Work This PR is the first in a series to implement full encryption support. Upcoming PRs will add: 1. Table properties for encryption configuration 2. Key management interfaces (KeyManagementClient trait) 3. EncryptionManager implementation 4. Native Parquet encryption integration 5. AWS KMS support 6. Integration with Table and FileIO Review Notes - This PR is intentionally minimal and self-contained - No existing code paths are modified - this is purely additive - The module is public but won't be used until future PRs wire it up - Format compatibility with Java has been prioritized to ensure interoperability ## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123. --> - Closes #. #2035 ## What changes are included in this PR? <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? Yes <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? -->
1 parent 89a9c90 commit fea5906

6 files changed

Lines changed: 788 additions & 122 deletions

File tree

0 commit comments

Comments
 (0)