Skip to content

DRAFT: MDEV-34705: Storing binlog in InnoDB #3775

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
knielsen wants to merge 167 commits into
base: 11.4
Choose a base branch
from
+32,721 −2,022
Draft

DRAFT: MDEV-34705: Storing binlog in InnoDB #3775

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
167 commits
Select commit Hold shift + click to select a range
18a4e58
MDEV-20586: Incorrect commit of transaction in GTID table processing
knielsen Oct 23, 2025
a52a325
MDEV-34705: Binlog in Engine: Very first sketch, able to create and w…
knielsen Feb 25, 2024
8b49aef
MDEV-34705: Binlog in Engine: Early draft, first binlogging of DML to…
knielsen Jun 18, 2024
62024ae
MDEV-34705: Binlog in Engine: Allocate next binlog tablespace as needed.
knielsen Jun 19, 2024
d79e1f2
MDEV-34705: Binlog in Engine: Fix re-using ids for binlog tablespaces
knielsen Jun 21, 2024
0570151
MDEV-34705: Binlog in Engine: Start of binlog reader (untested, incom…
knielsen Jul 27, 2024
476e1b2
MDEV-34705: Binlog in Engine: Change option to --binlog-storage-engin…
knielsen Jul 27, 2024
f133e7c
MDEV-34705: Binlog in Engine
knielsen Aug 7, 2024
ce518d5
MDEV-34705: Binlog in Engine
knielsen Aug 8, 2024
32a6ccc
MDEV-34705: Binlog in Engine: Also binlog standalone (eg. DDL) in the…
knielsen Aug 9, 2024
0e8cd2a
MDEV-34705: Binlog in Engine: Pre-allocate binlog tablespaces in back…
knielsen Aug 16, 2024
d5513e1
MDEV-34705: Binlog in Engine: Fix losing last part of event group > t…
knielsen Aug 24, 2024
99a8ec0
MDEV-34705: Binlog in Engine: Resume from existing binlogs
knielsen Sep 22, 2024
c5fb377
MDEV-34705: Binlog in Engine: Refactor rpl_gtid.h
knielsen Oct 1, 2024
a019c9c
MDEV-34705: Binlog in Engine: Refactor nlz() in shared header
knielsen Oct 1, 2024
5ebe287
MDEV-34705: Binlog in Engine: Add functions for compressed int
knielsen Oct 1, 2024
a86f97d
MDEV-34705: Binlog in Engine: Refactor to add rpl_binlog_state_base::…
knielsen Oct 2, 2024
e7a4cd1
MDEV-34705: Binlog in Engine: Searchability for GTID position
knielsen Oct 3, 2024
f22ac74
MDEV-34705: Binlog in Engine: Clean up read_binlog_data()
knielsen Oct 9, 2024
e6f73de
MDEV-34705: Code to restore binlog GTID state at restart
knielsen Oct 30, 2024
e1f0644
MDEV-34705: Fix race that could corrupt last mtr data in a tablespace
knielsen Oct 31, 2024
b17c8a4
MDEV-34705: Inplement starting from a specific GTID position
knielsen Nov 4, 2024
385bdc0
MDEV-34705: Improved error handling when searching GTID position
knielsen Nov 8, 2024
ccf35c0
MDEV-34705: Binlog-in-engine: Working replication to slave
knielsen Nov 12, 2024
ff43213
MDEV-34705: Binlog-in-engine: Fix chunk flags
knielsen Nov 14, 2024
8c36736
MDEV-34705: Binlog-in-engine: Refactor fsp_binlog_write_cache()
knielsen Nov 15, 2024
cd918ac
MDEV-34705: out-of band binlogging, partial untested commit to do a s…
knielsen Nov 17, 2024
c615641
MDEV-34705: out-of band binlogging, fix trx_cache handling for out-of…
knielsen Nov 20, 2024
fe7d6eb
MDEV-34705: out-of band binlogging, link to oob data from commit reco…
knielsen Nov 20, 2024
a9f89a3
MDEV-34705: Binlog-in-engine: Small visibility tweak in handler_binlo…
knielsen Nov 22, 2024
baae6ed
MDEV-34705: Binlog-in-engine: Read side of out-of-band binlogging
knielsen Dec 21, 2024
4519837
MDEV-34705: Binlog-in-engine: Refactor InnoDB part
knielsen Dec 30, 2024
33453c0
MDEV-34705: Binlog-in-engine: Fix incorrect binlog data
knielsen Jan 3, 2025
21969e0
MDEV-34705: Binlog-in-engine: Drop old X/X0Y.cc name convention for n…
knielsen Jan 5, 2025
e5c8ffe
MDEV-34705: Binlog-in-engine: Configurable binlog directory
knielsen Jan 6, 2025
216323b
MDEV-34705: Binlog-in-engine: Implement SHOW BINARY LOGS
knielsen Jan 7, 2025
e0d697e
MDEV-34705: Binlog-in-engine: Implement FLUSH BINARY LOGS
knielsen Jan 13, 2025
27bf755
MDEV-34705: Binlog-in-engine: Implement RESET MASTER
knielsen Jan 14, 2025
0da033e
MDEV-34705: Binlog-in-engine: Misc. small fixes to make normal test s…
knielsen Jan 17, 2025
f235c75
MDEV-34075: Binlog-in-engine: Some test and review fixes
knielsen Jan 17, 2025
cd059f0
MDEV-34705: Binlog-in-engine: Buildbot fixes
knielsen Jan 17, 2025
5d0c89b
MDEV-34705: Binlog-in-engine: Handful of fixes
knielsen Jan 22, 2025
3f74e94
MDEV-34705: Binlog-in-engine: Implement PURGE BINARY LOGS
knielsen Feb 4, 2025
3b38d71
MDEV-34705: Binlog-in-engine: Implement DELETE_DOMAIN_ID for FLUSH
knielsen Feb 11, 2025
206a5e3
MDEV-34705: Binlog-in-engine: No use of InnoDB tablespace and bufferpool
knielsen Feb 15, 2025
a2ef596
MDEV-34705: Binlog-in-engine: New binlog redo/recovery
knielsen Feb 25, 2025
aa670e2
MDEV-34705: binlog-in-engine: New recovery preparatory commit
knielsen Feb 25, 2025
387dcf5
MDEV-34705: Binlog-in-engine: Implement legacy SHOW MASTER STATUS
knielsen Mar 1, 2025
5db94a4
MDEV-34705: Binlog-in-engine: Implement SHOW BINLOG EVENTS
knielsen Mar 1, 2025
c50839e
MDEV-34705: Binlog-in-engine: Recovery intermediate commit
knielsen Mar 2, 2025
7bf7016
MDEV-34705: Binlog-in-engine: First working recovery
knielsen Mar 12, 2025
2db4703
MDEV-34705: Binlog-in-engine: Recovery testcase + few bugfixes
knielsen Mar 19, 2025
7bfcd9d
MDEV-34705: Binlog-in-engine: Implement page checksum
knielsen Mar 21, 2025
f00763e
MDEV-34705: Binlog-in-engine: Use the whole page for binlog data
knielsen Mar 21, 2025
08cde93
MDEV-34705: Binlog-in-engine: Use separate 4k pagesize for binlog files
knielsen Mar 24, 2025
4b562bf
MDEV-34705: Binlog-in-engine: Implement file header page
knielsen Apr 2, 2025
6d78c9b
MDEV-34705: Binlog-in-engine: Fix race that caused page corruption
knielsen Apr 2, 2025
84cb815
MDEV-34705: Binlog-in-engine: Fix race that caused page corruption
knielsen Apr 2, 2025
fa69ad9
MDEV-34705: Binlog-in-engine: Fix 3 race bugs
knielsen Apr 3, 2025
363c311
MDEV-34705: Binlog-in-engine: Fix uninitialized variable in binlog di…
knielsen Apr 6, 2025
2946352
MDEV-34705: Binlog-in-engine: Fix race between reader and flush
knielsen Apr 7, 2025
8567f9e
MDEV-34705: Binlog-in-engine: Work-around compiler warning
knielsen Apr 7, 2025
e62bd84
MDEV-34705: Binlog-in-engine: Fix MSAN uninitialized warning in binlo…
knielsen Apr 7, 2025
09c49ff
MDEV-34705: Binlog-in-engine: More compiler warning fixes
knielsen Apr 7, 2025
64437b7
MDEV-34705: Binlog-in-engine: Integration with server-layer code
knielsen Apr 10, 2025
9e3ef35
MDEV-34705: Binlog-in-engine: Implement refcounting outstanding OOB r…
knielsen Apr 19, 2025
311e2fa
MDEV-34705: Binlog-in-engine: mariadb-backup integration
knielsen Apr 25, 2025
fb410dd
MDEV-34705: Binlog-in-engine: Reduce struct fsp_binlog_page_entry size
knielsen May 5, 2025
a0aa2bf
MDEV-34705: Binlog-in-engine: Improved page fifo
knielsen May 12, 2025
05261cb
MDEV-34705: Binlog-in-engine: Fix leftover fsync of legacy binlog
knielsen May 15, 2025
423cd4c
MDEV-34705: Binlog-in-engine: Few bug fixes
knielsen May 20, 2025
6625cbb
MDEV-34705: Binlog-in-engine: Attempt to fix assertion in do_fdatasync()
knielsen May 21, 2025
2eec9be
MDEV-34705: Binlog-in-engine: Fix hang with event group of specific size
knielsen Jun 1, 2025
3f7c58c
MDEV-34705: Binlog-in-engine: Crash-safe slave
knielsen Jul 1, 2025
4926b09
MDEV-34705: Binlog-in-engine: Protect against concurrent RESET MASTER…
knielsen Jun 9, 2025
21526ba
MDEV-34705: Binlog-in-engine: Binlog reader to read whole page at a time
knielsen Jul 3, 2025
54f830c
Binlog-in-engine: Implement savepoint support
knielsen Jul 15, 2025
14c08e6
Binlog-in-engine: Implement dynamically changing binlog max size
knielsen Jul 16, 2025
2c50896
Binlog-in-engine: Handle single event writes larger than binlog size
knielsen Jul 18, 2025
c0a80e9
Binlog-in-engine: Make --binlog-storage-engine available as read-only…
knielsen Jul 18, 2025
4b12df5
Binlog-in-engine: Handle recovery when all but one binlog files have …
knielsen Jul 18, 2025
8defa38
Binlog-in-engine: Also binlog non-innodb event groups
knielsen Jul 22, 2025
9bdf20a
Binlog-in-engine: Handle mixing transactional and non-transactional t…
knielsen Jul 22, 2025
7644f8b
Binlog-in-engine: Embedded fix
knielsen Jul 23, 2025
8f2ecf4
Binlog-in-engine: Clean up gtid state reading
knielsen Jul 23, 2025
fae8665
Binlog-in-engine: Support for new binlog format in mysqlbinlog
knielsen Aug 14, 2025
1f71e19
Binlog-in-engine: Implement heartbeat support
knielsen Aug 14, 2025
0a0c787
Binlog-in-engine: BINLOG_GTID_POS() not available with --binlog-in-en…
knielsen Aug 14, 2025
bbb125a
Binlog-in-engine: Use 16kB page size
knielsen Aug 15, 2025
66557b5
Binlog-in-engine: Give error on --encrypt-binlog
knielsen Aug 15, 2025
8714cef
Binlog-in-engine: Test case for GTID state > 1 page
knielsen Aug 15, 2025
af69b1c
Binlog-in-engine: Report master restart to slave
knielsen Aug 25, 2025
07a8b0c
Binlog-in-engine: Bug fix around crash-safe slave
knielsen Aug 27, 2025
c662dc3
Binlog-in-engine: Fix mysqlbinlog bug when binlog files are in curren…
knielsen Sep 3, 2025
1d24265
Binlog-in-engine: Fix wrong scope of variable in show_binlog_info()
knielsen Sep 3, 2025
6e62bcf
Binlog-in-engine: Fix memory leaks and incorrect assert
knielsen Sep 3, 2025
c28b845
Binlog-in-engine: Fixes for some review comments
knielsen Sep 4, 2025
01bb3c3
Binlog-in-engine: Better error message for mysqlbinlog on empty, pre-…
knielsen Sep 4, 2025
ba67c98
Binlog-in-engine: Fix race with concurrent write and FLUSH/RESET
knielsen Sep 5, 2025
019b2e7
Binlog-in-engine: Assertion on parallel online alter
knielsen Sep 9, 2025
adfd2b4
Binlog-in-engine: Fix assertion on filtered events
knielsen Sep 9, 2025
e99b626
Binlog-in-engine: Make binlog file name available on slave
knielsen Sep 10, 2025
149e987
Binlog-in-engine: Fix out-of-bounds read
knielsen Sep 11, 2025
18cadb4
Binlog-in-engine: SHOW BINLOG EVENT improvements.
knielsen Sep 13, 2025
fe5ed81
Binlog-in-engine: Fix missing binlog of CREATE...SELECT in ROW mode
knielsen Sep 16, 2025
c2fb400
Binlog-in-engine: Fix binlogging of large DDL
knielsen Sep 19, 2025
8005646
Binlog-in-engine: Fix error/assertion when rolling back to implict sa…
knielsen Sep 25, 2025
acaea3c
Binlog-in-engine: Support for combined stmt/trx cache in binlog
knielsen Sep 27, 2025
008f593
Binlog-in-engine: Fix hang and assertion in page fifo flushing
knielsen Sep 28, 2025
c14e36f
Binlog-in-engine: Fix race causing checksum error in binlog reader
knielsen Sep 29, 2025
07659fa
Binlog-in-engine: Fix busy-wait in binlog reader
knielsen Sep 30, 2025
3db01b4
Binlog-in-engine: No error in mysqlbinlog on empty binlog file
knielsen Oct 4, 2025
5cbb20d
Binlog-in-engine: Fixes to FLUSH BINARY LOGS DELETE_DOMAIN_ID
knielsen Oct 6, 2025
40ce89c
Binlog-in-engine: Fix binary search for GTID position
knielsen Oct 6, 2025
586b49a
Binlog-in-engine: Fix inconsistency in --binlog-directory
knielsen Oct 6, 2025
52a4a85
Binlog-in-engine: Initial support for 2pc and XA
knielsen Oct 1, 2025
c8b90a8
Binlog-in-engine: XA fix for implicit rollback
knielsen Oct 15, 2025
b7fa7e4
Binlog-in-engine: Fix inconsistency in stmt/trx cache use
knielsen Oct 16, 2025
691a087
Binlog-in-engine: Fix 3bugs found during RQG testing
knielsen Oct 17, 2025
a65fc28
Binlog-in-engine: XA: Fix hang during server shutdown.
knielsen Oct 17, 2025
748de8c
Binlog-in-engine: mariabackup and concurrent FLUSH, PRUGE, or RESET M…
knielsen Oct 19, 2025
92ce708
Binlog-in-engine: Remove wrong assertion
knielsen Oct 20, 2025
94d4416
Binlog-in-innodb: Small compile/test fixes after 11.4 rebase
knielsen Oct 20, 2025
ddb7bfb
Binlog-in-engine: Small recovery fix
knielsen Oct 21, 2025
a9ca8b5
Binlog-in-engine: Fix incorrect handling of internal 2pc rollback
knielsen Oct 22, 2025
f776c02
Binlog-in-engine: Fix RESET MASTER with active XA or OOB
knielsen Oct 22, 2025
f268031
Binlog-in-engine: Fix bug when restarting on existing binlog
knielsen Oct 23, 2025
848d6c6
Binlog-in-innodb: Fix some review comments in InnoDB code
knielsen Oct 24, 2025
f177b76
Binlog-in-engine: Fix SAVEPOINT name re-use
knielsen Oct 24, 2025
ce9fcca
MDEV-34705: Binlog-in-engine: Update rpl.rpl_ddl to use GTIDs to sync
bnestere Oct 11, 2025
cda2add
MDEV-34705: Binlog-in-engine: Add MTR tests for rpl.rpl_ddl
bnestere Oct 11, 2025
fd8c0ca
MDEV-34705: Binlog-in-engine: Update rpl_row_basic_* MTR tests to use…
bnestere Oct 11, 2025
b90b299
MDEV-34705: Binlog-in-engine: Add tests for rpl_row_basic
bnestere Oct 11, 2025
1320a67
MDEV-34705: Binlog-in-engine: rpl_mysqlbinlog_slave_consistency tests
bnestere Sep 17, 2025
2b658de
Binlog-in-engine: Add recovery testcase
knielsen Oct 27, 2025
e8160b5
Binlog-in-engine: Fix two recovery bugs
knielsen Oct 29, 2025
3553889
Binlog-in-engine: Assertion 'sp == nullptr' with SAVEPOINT
knielsen Oct 29, 2025
56605c8
Binlog-in-engine: Fix race causing hang during shutdown
knielsen Oct 29, 2025
11732fb
Binlog-in-engine: Fix couple XA bugs
knielsen Oct 31, 2025
3c5bfd1
Binlog-in-engine: SAVEPOINT fixes.
knielsen Nov 3, 2025
1c767fd
Binlog-in-engine: Tweak mysqlbinlog error handling
knielsen Nov 6, 2025
ccd0afc
Binlog-in-engine: Fix rpl.rpl_circular_for_4_hosts after changes to r…
knielsen Nov 6, 2025
3fbb359
Binlog-in-engine: Implement internal 2pc and user XA recovery.
knielsen Oct 14, 2025
5388c37
Fix that InnoDB rolls back entire trx in failed CREATE TEMPORARY
knielsen Nov 9, 2025
2f551a0
Binlog-in-engine: Fix autopurge of still needed binlog file
knielsen Nov 13, 2025
8f972e8
Binlog-in-engine: Fix incorrect printf arguments
knielsen Nov 13, 2025
c14e3ad
Binlog-in-engine: Documentation
knielsen Nov 26, 2025
32b853e
Binlog-in-engine: Clean up few remaining todos, error handling
knielsen Nov 27, 2025
510d5cf
Binlog-in-engine: Small cleanup/formatting
knielsen Nov 27, 2025
52a75d4
Binlog-in-engine: Fix incorrect GTID position at slave connect
knielsen Nov 28, 2025
5df6dc0
Binlog-in-engine: More robust binlog recovery
knielsen Nov 28, 2025
2d2aa48
Binlog-in-engine: Add mariadb-backup option to omit binlog from the b…
knielsen Dec 2, 2025
80a87a2
Binlog-in-engine: Update docs with upgrading info
knielsen Dec 24, 2025
ae8ac6f
MDEV-38429: Atomic DDL can binlog DDL during ddl log crash recovery w…
knielsen Dec 26, 2025
b0464a7
MDEV-38429: Atomic DDL can binlog DDL during ddl log crash recovery w…
knielsen Dec 26, 2025
63a8a3d
Binlog-in-engine: Fix error handling for failure to write GTID event
knielsen Dec 27, 2025
6df4e0f
Binlog-in-engine: Fix sporadic failure of binlog-in-engine.mysqlbinlog
knielsen Dec 29, 2025
cd94eed
Binlog-in-engine: Remove useless info messages in error log
knielsen Dec 29, 2025
c7a550e
Binlog-in-engine: Fix race between FLUSH BINARY LOGS and GTID allocation
knielsen Dec 30, 2025
e4b0e59
Binlog-in-engine: MDEV-38462: Large binlog write can overwrite head o…
knielsen Jan 2, 2026
aef54e9
MDEV-38465: Savepoint in trigger causes transactional inconsistency
knielsen Jan 5, 2026
73110a1
Binlog-in-innodb: Small docs clarifications
knielsen Jan 14, 2026
e3331af
Binlog-in-engine: Fixes for mariabackup --binlog-directory
knielsen Jan 14, 2026
94a302d
Binlog-in-engine: Fix sporadic test after rebase to latest
knielsen Jan 16, 2026
3b7cb94
Binlog-in-engine: Fix test failure in binlog_in_engine.mariabackup_bi…
knielsen Jan 17, 2026
37253f7
Binlog-in-engine: Tweak redo flush when binlogging MyISAM etc.
knielsen Jan 19, 2026
04c56e8
Binlog-in-engine: Fixes to binlog state records
knielsen Jan 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
151 changes: 151 additions & 0 deletions Docs/replication/binlog.md
View file
Open in desktop

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion client/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ TARGET_LINK_LIBRARIES(mariadb-show ${CLIENT_LIB})
MYSQL_ADD_EXECUTABLE(mariadb-plugin mysql_plugin.c)
TARGET_LINK_LIBRARIES(mariadb-plugin ${CLIENT_LIB})

MYSQL_ADD_EXECUTABLE(mariadb-binlog mysqlbinlog.cc)
MYSQL_ADD_EXECUTABLE(mariadb-binlog mysqlbinlog.cc mysqlbinlog-engine.cc)
TARGET_LINK_LIBRARIES(mariadb-binlog ${CLIENT_LIB} mysys_ssl)

MYSQL_ADD_EXECUTABLE(mariadb-admin mysqladmin.cc ../sql/password.c)
Expand Down
Loading