feat: parse `/stackerdb_chunks` Stacks node event #653

rafaelcr · 2024-10-14T20:33:09Z

Starts listening to the new /stackerdb_chunks Stacks node event and parses the incoming message. Parsers are taken from the stacks_codec crate, which has been updated with the latest serializers.

For future PRs:

Start integrating the signer messages into the predicates framework
Store signer messages in a local SQLite DB for replay
Expand unit tests to include full Stacks node integration tests

components/chainhook-sdk/src/indexer/stacks/mod.rs

components/chainhook-sdk/src/observer/http.rs

Co-authored-by: Matthew Little <zone117x@gmail.com>

rafaelcr · 2024-10-15T16:18:32Z

Thanks for the comments @zone117x I'll be adding some unit tests today with your test payloads so we can merge this one ASAP

rafaelcr · 2024-10-15T20:15:54Z

@zone117x can you take a look? I added a unit test to check the parser behavior, but will expand the tests in a future PR. Right now, Chainhook has a strange way of depending on a local install of redis to run tests so it's tricky to run them in a local setup. I'll move that over to docker in the next PR and start adding some integration tests.

zone117x

LGTM!

## [1.9.0-stacks-signers.1](v1.8.0...v1.9.0-stacks-signers.1) (2024-10-25) ### Features * add various new nakamoto block fields to `/new_block` ingestion and `StacksPayload` ([#659](#659)) ([f48cda5](f48cda5)) * include recovered signer pubkeys in new block payload ([#662](#662)) ([b5ad4ba](b5ad4ba)) * include signer messages in Stacks predicate payloads ([#656](#656)) ([aee14bc](aee14bc)) * parse `/stackerdb_chunks` Stacks node event ([#653](#653)) ([e44d84a](e44d84a)) * store signer messages in local sqlite database ([#664](#664)) ([d12acd9](d12acd9)) * support tenure-height in block messages ([#661](#661)) ([99e5a61](99e5a61)) * **ts-client:** add persisted predicates and periodic health check ([#658](#658)) ([535226a](535226a)) * **ts-client:** add signer messages to stacks payloads ([#657](#657)) ([ff66bb2](ff66bb2)) ### Bug Fixes * add a configurable max timeout for outgoing predicate payload requests ([#642](#642)) ([6c1dfa9](6c1dfa9)), closes [#643](#643) * increase stacks event server body size limit to 500MB ([#660](#660)) ([21234c9](21234c9)) * return 500 status code to Stacks node when event ingestion fails ([#648](#648)) ([63c753c](63c753c)), closes [#646](#646) * signer pubkey calculation ([#665](#665)) ([cc93873](cc93873)) * update SignerMessage deserializing ([#663](#663)) ([71364c1](71364c1)) * validate ts client predicates before registering ([#639](#639)) ([2d08f72](2d08f72))

github-actions · 2024-10-25T17:18:33Z

🎉 This PR is included in version 1.9.0-stacks-signers.1 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

## [1.9.0](v1.8.0...v1.9.0) (2025-06-16) ### Features * add mock signature pubkey ([#670](#670)) ([c414d79](c414d79)) * add mock signer message support ([#669](#669)) ([ad5fd54](ad5fd54)) * add various new nakamoto block fields to `/new_block` ingestion and `StacksPayload` ([#659](#659)) ([f48cda5](f48cda5)) * callback to intercept and modify predicates before re-registration ([#675](#675)) ([5f13912](5f13912)) * include recovered signer pubkeys in new block payload ([#662](#662)) ([b5ad4ba](b5ad4ba)) * include signer messages in Stacks predicate payloads ([#656](#656)) ([aee14bc](aee14bc)) * parse `/stackerdb_chunks` Stacks node event ([#653](#653)) ([e44d84a](e44d84a)) * store signer messages in local sqlite database ([#664](#664)) ([d12acd9](d12acd9)) * support tenure-height in block messages ([#661](#661)) ([99e5a61](99e5a61)) * **ts-client:** add persisted predicates and periodic health check ([#658](#658)) ([535226a](535226a)) * **ts-client:** add signer messages to stacks payloads ([#657](#657)) ([ff66bb2](ff66bb2)) ### Bug Fixes * add a configurable max timeout for outgoing predicate payload requests ([#642](#642)) ([6c1dfa9](6c1dfa9)), closes [#643](#643) * add missing fields to ordinal inscription reveals ([#703](#703)) ([4e49608](4e49608)) * bump stacks codec ([3c117b5](3c117b5)) * increase stacks event server body size limit to 500MB ([#660](#660)) ([21234c9](21234c9)) * mock signature retrieval ([#671](#671)) ([8bb59c1](8bb59c1)) * only import stacks tsv if chainstate is empty ([#684](#684)) ([1f9f37e](1f9f37e)), closes [#677](#677) * propagate error when trying to store peer info ([1bc949a](1bc949a)) * read contract_interface from Stacks node block events ([#712](#712)) ([6f9c718](6f9c718)) * read stacks-core http event POST payloads for ignored events ([#673](#673)) ([a01470e](a01470e)) * return 500 status code to Stacks node when event ingestion fails ([#648](#648)) ([63c753c](63c753c)), closes [#646](#646) * signer pubkey calculation ([#665](#665)) ([cc93873](cc93873)) * ts client missing matches_regex option for print_event ([#701](#701)) ([8247900](8247900)) * update SignerMessage deserializing ([#663](#663)) ([71364c1](71364c1)) * validate ts client predicates before registering ([#639](#639)) ([2d08f72](2d08f72))

hirosystems-release-bot · 2025-06-16T16:34:25Z

🎉 This PR is included in version 1.9.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

## [1.9.0-stacks-signers.1](hirosystems/chainhook@v1.8.0...v1.9.0-stacks-signers.1) (2024-10-25) ### Features * add various new nakamoto block fields to `/new_block` ingestion and `StacksPayload` ([#659](hirosystems/chainhook#659)) ([f48cda5](hirosystems/chainhook@f48cda5)) * include recovered signer pubkeys in new block payload ([#662](hirosystems/chainhook#662)) ([b5ad4ba](hirosystems/chainhook@b5ad4ba)) * include signer messages in Stacks predicate payloads ([#656](hirosystems/chainhook#656)) ([aee14bc](hirosystems/chainhook@aee14bc)) * parse `/stackerdb_chunks` Stacks node event ([#653](hirosystems/chainhook#653)) ([e44d84a](hirosystems/chainhook@e44d84a)) * store signer messages in local sqlite database ([#664](hirosystems/chainhook#664)) ([d12acd9](hirosystems/chainhook@d12acd9)) * support tenure-height in block messages ([#661](hirosystems/chainhook#661)) ([99e5a61](hirosystems/chainhook@99e5a61)) * **ts-client:** add persisted predicates and periodic health check ([#658](hirosystems/chainhook#658)) ([535226a](hirosystems/chainhook@535226a)) * **ts-client:** add signer messages to stacks payloads ([#657](hirosystems/chainhook#657)) ([ff66bb2](hirosystems/chainhook@ff66bb2)) ### Bug Fixes * add a configurable max timeout for outgoing predicate payload requests ([#642](hirosystems/chainhook#642)) ([6c1dfa9](hirosystems/chainhook@6c1dfa9)), closes [#643](hirosystems/chainhook#643) * increase stacks event server body size limit to 500MB ([#660](hirosystems/chainhook#660)) ([21234c9](hirosystems/chainhook@21234c9)) * return 500 status code to Stacks node when event ingestion fails ([#648](hirosystems/chainhook#648)) ([63c753c](hirosystems/chainhook@63c753c)), closes [#646](hirosystems/chainhook#646) * signer pubkey calculation ([#665](hirosystems/chainhook#665)) ([cc93873](hirosystems/chainhook@cc93873)) * update SignerMessage deserializing ([#663](hirosystems/chainhook#663)) ([71364c1](hirosystems/chainhook@71364c1)) * validate ts client predicates before registering ([#639](hirosystems/chainhook#639)) ([2d08f72](hirosystems/chainhook@2d08f72))

rafaelcr added 2 commits October 11, 2024 12:03

feat: start parsing from codec

60e529f

chore: start parsing stackerdb chunks

c4068ed