I have a library, where I can't precalculate the memory footpring in a GASPI segment. Thus the idea is to create segments on demand. Does the following code work in principle:
void* GetMemory(size_t size) // collective called by all members of group
{
void* p = Allocate(size, seg_id);
if (p == 0)
{
gaspi_segment_avail_local(&seg_id); // from the extension
gaspi_segment_alloc(seg_id, size, GASPI_MEM_UNINITIALIZED);
gaspi_segment_ptr(seg_id, &p);
for (auto i : group)
gaspi_segment_register(seg_id, i, GASPI_BLOCK);
}
std::vector<unsigned int> ids(group.size(), 0);
remote_ids[my_rank] = seg_id;
gaspi_allreduce(ids.data(), ids.data(), 1, GASPI_OP_SUM, GASPI_TYPE_UINT, group, GASPI_BLOCK));
}
What it does: it tries to allocate the requested size on the already existing segment seg_id. If there is not enough memory in that old segment, a new seg_id is locally retrieved, bind to newly allocated memory and then registered at some remote ranks (stored in group). Afterwards all members of the group enter the gaspi_allreduce to exchange their segment ids, so that the ids can be used for communcation. The allreduce serves also as a barrier here (see gaspi_segment_create).
Question: works gaspi_segment_register as written here? That is, is it a one-sided call, which does either nothing on the remote rank or the remote rank receives the register request somehow magically?
Wish: If I'd use gaspi_segment_bind (with some memory from the outside), I have difficulties to release the segment. I cannot use segment_delete, since the memory is not owned by the segment. I could of course just release the memory at the outside, which renders the segment id invalid. But then I cannot reuse the segment id, since something like gaspi_segment_unbind is missing, isn't it?
I have a library, where I can't precalculate the memory footpring in a GASPI segment. Thus the idea is to create segments on demand. Does the following code work in principle:
What it does: it tries to allocate the requested size on the already existing segment seg_id. If there is not enough memory in that old segment, a new seg_id is locally retrieved, bind to newly allocated memory and then registered at some remote ranks (stored in group). Afterwards all members of the group enter the gaspi_allreduce to exchange their segment ids, so that the ids can be used for communcation. The allreduce serves also as a barrier here (see gaspi_segment_create).
Question: works gaspi_segment_register as written here? That is, is it a one-sided call, which does either nothing on the remote rank or the remote rank receives the register request somehow magically?
Wish: If I'd use gaspi_segment_bind (with some memory from the outside), I have difficulties to release the segment. I cannot use segment_delete, since the memory is not owned by the segment. I could of course just release the memory at the outside, which renders the segment id invalid. But then I cannot reuse the segment id, since something like gaspi_segment_unbind is missing, isn't it?