draw inspiration from Netflix DBLog's lock-free algorithm to improve

[hantmac]

Problems and Points for improvement

• Due to the integration of Debezium Engine, full-load reading phase does not support checkpoint. After a failure, it needs to be re-read.
• Currently, only single concurrency is supported, and horizontal scaling is not supported.
• When ensuring data consistency, Debezium needs to apply locks to the read databases or tables. Global locks may cause the database to hang, while table-level locks can restrict table reads.

[hantmac]

https://arxiv.org/pdf/2010.12597.pdf

[codering]

How is it going now ?

[hantmac]

@codering Not started yet 😄