Custom maps implementation#4895
Conversation
Co-authored-by: Dave Thaler <dthaler1968@gmail.com>
Co-authored-by: Dave Thaler <dthaler1968@gmail.com>
Co-authored-by: Dave Thaler <dthaler1968@gmail.com>
Co-authored-by: Dave Thaler <dthaler1968@gmail.com>
Alan-Jowett
left a comment
There was a problem hiding this comment.
Code Review - Custom Maps Implementation
I've reviewed the changes for PR #4895. Overall the implementation is solid with good separation of concerns and proper use of NMR for provider registration. Here are my findings:
Issues Found
1. Missing NULL check for provider_dispatch in cleanup functions (Low severity)
In _clean_up_custom_hash_map\ (line 4282), \map->provider_dispatch->process_map_delete_element\ is called without a NULL check. While I verified that the current code flow ensures \provider_dispatch\ is set before this function is reachable (via \�bpf_custom_map_delete\ which is only called after successful map creation), this creates a maintenance hazard.
If future code changes introduce paths where the hash table cleanup is called before provider attachment completes, this will crash. Consider adding a defensive check:
\\c
if (map->provider_dispatch != NULL) {
map->provider_dispatch->process_map_delete_element(...);
}
\\
2. Potential partial initialization from undersized provider dispatch table (Medium severity)
In _ebpf_custom_map_client_attach_provider\ (lines 4683-4686):
\\c
memcpy(
provider_dispatch_table,
provider_data->base_provider_table,
min(sizeof(ebpf_base_map_provider_dispatch_table_t), provider_data->base_provider_table->header.size));
\\
While _ebpf_validate_map_provider_dispatch_table\ checks that \header.size == EBPF_BASE_MAP_PROVIDER_DISPATCH_TABLE_CURRENT_VERSION_SIZE, if a future version allows different sizes for backwards compatibility, the \min()\ could copy less than expected, leaving function pointers uninitialized (they'd be zeroed from \�bpf_allocate_cache_aligned_with_tag, but not validated as non-NULL for optional functions like \process_map_find_element).
Currently safe due to strict validation, but worth noting for future versioning support.
Questions/Suggestions
-
Documentation: The documentation mentions \BPF_MAP_TYPE_XSKMAP\ as an example custom map type, but I don't see it added to \�bpf_map_type_t\ in this PR. Is that intentional (just an example) or should it be added?
-
Test coverage: The sample extension tests look comprehensive. Are there any fault injection tests for the custom map path to verify error handling?
Overall the implementation looks good to ship. The validation is thorough and the NMR integration follows established patterns in the codebase.
| ## 1 Overview | ||
| An "eBPF extension" is a Windows kernel driver or component that implements eBPF hooks or helper functions. The design | ||
| of eBPF for Windows is such that an extension providing an implementation for hooks and helper functions can be | ||
| An "eBPF extension" is a Windows kernel driver or component that implements eBPF hooks, helper functions, and custom maps. The design |
There was a problem hiding this comment.
nit: should this be hooks, helpers, OR custom maps?
| ebpf_process_map_delete_element_t process_map_delete_element; | ||
| } ebpf_base_map_provider_dispatch_table_t; | ||
| ``` | ||
| This the dispatch table that the extension needs to implement and provide to eBPF runtime. It contains the following fields: |
There was a problem hiding this comment.
nit: This "is" the dispatch table
| * @param[in] out_value_size The size in bytes of the destination (stored) value buffer. | ||
| * @param[out] out_value Optional pointer to the destination (stored) value buffer to populate. | ||
| * @param[in] flags Update flags. Supported values: | ||
| * EBPF_MAP_OPERATION_HELPER - The update is invoked from a BPF program. |
There was a problem hiding this comment.
Let's document the caller context is the same as the original user mode process if EBPF_MAP_OPERATION_HELPER is not set. This allows providers to implicitly use the handle table of the current process when resolving parameters like file descriptors.
* add proposal doc * update doc * update doc * cr comments * implement extensible maps * patch * changes * fix build breaks * add unit tests * add program load tests * update tests, export epoch APIs * fix validation, add driver tests * add epoch api usage in unit tests * add array map * fix * add doc * update sample program, fix analysis failure * bugfix, add tests * add tests, fixes * fix * fix build break * fixes * cleanup * cleanup * fix asan build * add hash map to unit tests * add tests * fix tests * test code refactor * add tests * code cleanup * cleanup * backup * ebpfcore changes * update unit tests * fix * fix * bug fixes, fix tests * update driver tests * fix bad merge * fix memory leak * fix * add test files * fix * backup * fixes * fix * fixes * update doc, code cleanup * fix * fix * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix * fix * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fixes, cleanup * fix * fix * export epoch_enter,epoch_exit * add comments, code cleanup * fix * fix bad merge, update documentation * Apply suggestions from code review Co-authored-by: Dave Thaler <dthaler1968@gmail.com> * cr comments * Apply suggestions from code review Co-authored-by: Dave Thaler <dthaler1968@gmail.com> * cr comments * cr comments * update doc * Update docs/eBpfExtensions.md Co-authored-by: Dave Thaler <dthaler1968@gmail.com> * Update tests/end_to_end/end_to_end.cpp Co-authored-by: Dave Thaler <dthaler1968@gmail.com> * Update tests/end_to_end/end_to_end.cpp Co-authored-by: Dave Thaler <dthaler1968@gmail.com> * Update libs/execution_context/ebpf_maps.c Co-authored-by: Dave Thaler <dthaler1968@gmail.com> * Update tests/end_to_end/helpers.h Co-authored-by: Dave Thaler <dthaler1968@gmail.com> * Update tests/end_to_end/helpers.h Co-authored-by: Dave Thaler <dthaler1968@gmail.com> * Update include/ebpf_extension.h Co-authored-by: Dave Thaler <dthaler1968@gmail.com> * cr comment * cr comments --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Dave Thaler <dthaler1968@gmail.com>
* add proposal doc * update doc * update doc * cr comments * implement extensible maps * patch * changes * fix build breaks * add unit tests * add program load tests * update tests, export epoch APIs * fix validation, add driver tests * add epoch api usage in unit tests * add array map * fix * add doc * update sample program, fix analysis failure * bugfix, add tests * add tests, fixes * fix * fix build break * fixes * cleanup * cleanup * fix asan build * add hash map to unit tests * add tests * fix tests * test code refactor * add tests * code cleanup * cleanup * backup * ebpfcore changes * update unit tests * fix * fix * bug fixes, fix tests * update driver tests * fix bad merge * fix memory leak * fix * add test files * fix * backup * fixes * fix * fixes * update doc, code cleanup * fix * fix * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix * fix * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fixes, cleanup * fix * fix * export epoch_enter,epoch_exit * add comments, code cleanup * fix * fix bad merge, update documentation * Apply suggestions from code review Co-authored-by: Dave Thaler <dthaler1968@gmail.com> * cr comments * Apply suggestions from code review Co-authored-by: Dave Thaler <dthaler1968@gmail.com> * cr comments * cr comments * update doc * Update docs/eBpfExtensions.md Co-authored-by: Dave Thaler <dthaler1968@gmail.com> * Update tests/end_to_end/end_to_end.cpp Co-authored-by: Dave Thaler <dthaler1968@gmail.com> * Update tests/end_to_end/end_to_end.cpp Co-authored-by: Dave Thaler <dthaler1968@gmail.com> * Update libs/execution_context/ebpf_maps.c Co-authored-by: Dave Thaler <dthaler1968@gmail.com> * Update tests/end_to_end/helpers.h Co-authored-by: Dave Thaler <dthaler1968@gmail.com> * Update tests/end_to_end/helpers.h Co-authored-by: Dave Thaler <dthaler1968@gmail.com> * Update include/ebpf_extension.h Co-authored-by: Dave Thaler <dthaler1968@gmail.com> * cr comment * cr comments --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Dave Thaler <dthaler1968@gmail.com>
Fixes #4375
This pull request introduces support for custom map types in eBPF extensions for Windows. It updates both the documentation and the extension API to describe and enable how extensions can implement and register new map types, alongside hooks and helper functions. The changes define new NPI contracts, data structures, and dispatch tables for map providers, and clarify how helper functions interact with custom maps.
Key changes:
Documentation updates for custom maps
New API and data structures for custom map providers
ebpf_extension.hto support custom map operations (create, delete, find, add, associate with program types, etc.) and epoch-based memory management. This includes theebpf_map_provider_data_t,ebpf_base_map_provider_dispatch_table_t, and related client-side structures and macros. [1] [2]NPI contract and GUID for map extensions
EBPF_MAP_INFO_EXTENSION_IID) inebpf_extension_uuids.hto uniquely identify the NPI contract for map information providers.These changes collectively enable eBPF extensions to define and register new map types, provide the necessary interfaces for the eBPF core to interact with custom maps, and document the requirements and procedures for extension authors.##
Testing
This PR adds tests.
Documentation
PR updates documentation.
Installation
NA.