Feature: extension: icebug

[aheev]

add icebug-memory(arrow csr) graphs support. zero-copy is not achievable with other graphs right now

Plan

• We will be using projected graphs for icebug ext too
• Icebug algos are exposed with Icebug prefix
• on CALL IcebugAlgo(g) RETURN *;,

ArrowNodeTableData nodeTableData = // catalogEntry->getStorage()  + arrow_table_support::getArrowData
ArrowRelTableData relTableData = // catalogEntry->getStorage()  + arrow_table_support::getArrowRelData
// table validations
// combine arrow recordBatches
auto graph = icebug.Graph.fromCSR(...);
return graph.Algo();

Caveats

• We need to expose helpers to fetch static Arrow Registries from arrow_table_support in icebug ext
• since we are using arrow recordBatches for tables, combining them voids zero-copy

[aheev]

@adsharma could you PTAL?

[adsharma]

Icebug has 100-200 algorithms. Why not:

CALL Icebug(g, "algo") RETURN *;

Then there is the issue of different algorithms having different effects/return types. Have you thought about how to handle it?

[aheev]

Icebug has 100-200 algorithms. Why not:

CALL Icebug(g, "algo") RETURN *;

works

Then there is the issue of different algorithms having different effects/return types. Have you thought about how to handle it?

we can follow the algo ext path by encoding the results into table columns

since we are using arrow recordBatches for tables, combining them voids zero-copy

I am worried about this though

[aheev]

Icebug has 100-200 algorithms. Why not:

CALL Icebug(g, "algo") RETURN *;

works

Then there is the issue of different algorithms having different effects/return types. Have you thought about how to handle it?

we can follow the algo ext path by encoding the results into table columns

since we are using arrow recordBatches for tables, combining them voids zero-copy

I am worried about this though

@adsharma what do you think?

[adsharma]

Really not a choice given the performance of chunked arrays. Zero-copy is not an end goal in itself. It's a performance optimization.

Ok with having an escape hatch/knob for avoiding combine_chunks.