C: Use Arena Allocator for Lexing and Parsing

[marcoroth]

This pull request introduces arena allocation for the lexer and parser, replacing "per-object" malloc/free calls with bulk allocation from a memory arena.

All allocated AST nodes, tokens, and internal strings are placed into a single arena that is freed in one shot after the parse tree has been converted to the binding's native objects.

The arena is accessed through hb_allocator_T, a vtable-based allocator abstraction introduced in #1287. This pull request switches the default backend from malloc to hb_arena_T across all bindings and the CLI.

The malloc backend remains available as a fallback. The intent is to validate the arena approach in production and then, once confident, simplify the abstraction away and use hb_arena_T directly.

The hb_allocator_T interface was extended with a destroy function pointer and a high-level hb_allocator_init(allocator, type) constructor that takes an hb_allocator_type_T enum (HB_ALLOCATOR_ARENA, HB_ALLOCATOR_MALLOC).

A separate hb_allocator_init_with_size variant accepts a custom initial arena size for edge cases. The default arena size is controlled by a compile-time HB_ALLOCATOR_DEFAULT_ARENA_SIZE flag.

The extract API (herb_extract, herb_extract_ruby_*, herb_extract_html_*) was also updated to accept an hb_allocator_T*.

Depends on #1287

[citizen428]

Would it make sense to remove the unused argument or do you have future plans for it? The macro name doesn't suggest that.

[marcoroth]

The point was to self-document the variable that's going to be available inside the loop. But maybe that's confusing?

hb_arena_for_each_page(arena, page) {
  total += page->position;
}

vs:

hb_arena_for_each_page(arena) {
  total += page->position;
  //       ^ where is page coming from?
}

[citizen428]

I'd say it is slightly confusing because clangd sees it as unused. So to me it looks less self-documenting and more like a leftover from an older implementation. But there are bigger fish to fry. 😃

[marcoroth]

I updated it in a8e2f0c 🙌🏼

[pkg-pr-new]

npx https://pkg.pr.new/@herb-tools/formatter@726

npx https://pkg.pr.new/@herb-tools/language-server@726

npx https://pkg.pr.new/@herb-tools/linter@726

commit: f6fb2c1

[github-actions]

🌿 Interactive Playground and Documentation Preview

A preview deployment has been built for this pull request. Try out the changes live in the interactive playground:

• Playground Preview
• Documentation Preview
• Website Preview

---

_{🌱 Grown from commit f6fb2c1}

---

✅ Preview deployment has been cleaned up.