Skip to content

Using rustler-precompiled so that users don't have to install rust toolchain to compile the blake3 library#7

Draft
madclaws wants to merge 4 commits intoThomas-Jean:masterfrom
madclaws:rustler-precompiled
Draft

Using rustler-precompiled so that users don't have to install rust toolchain to compile the blake3 library#7
madclaws wants to merge 4 commits intoThomas-Jean:masterfrom
madclaws:rustler-precompiled

Conversation

@madclaws
Copy link
Copy Markdown
Contributor

Currently if we have to use the blake3 library then the host needs to have rust toolchain as a prerequisite. Due to this we need to install rust in the dockerfiles etc. We can use rustler-precompiled, such that the nif will be already build (on most rust supported platforms) and the artificats will be uploaded already, and we basically just download the corresponding nif needed for the host.

This PR is kind of a draft, because the major work happens in CI. Things we have to do now are,

  • Configure Github actions.

  • We could add in the readme section that, to force local compilation we need to set the env to BLAKE_BUILD=1. This is for those who wants to contribute.

  • The release flow

    • I have added a build.yml which does the building , uploading etc. This could be opinionated so i have just separated it from the rest, we could maybe merge it with the release.yml. Also In the release flow doc, there is part on generating checksum which is mandatory for hex. So this could be also added to release.yml.

Since i needed this urgently i have tried adding this to a blake3 clone project and it worked !!. Other references i have used are,

Glad to help, if any doubts 👍

… - true

rustler optional is true, so that act as a fallback to compiled locally with rust env (helpful in
local lib development)
Here we specify the url where the build artifacts will available for download (the actual building
and uploading is handled in CI). foce_build option is true if BLAKE3_BUILD env exists and value is
either  1 or true, this is helpful in library local development, where we can actually compile
locally with rust when needed.
…ILD: true env

build.yml build the nif on multiple envs and create release on respective tag and also upload the
artifact.
@Thomas-Jean
Copy link
Copy Markdown
Owner

Hey @madclaws thanks for this. I don't see a problem with this approach itself. I just wonder what the preferred method of enabling it would be. i.e. do we want to enable it by default or allow it to be opt-in?

Also, would we rather read a config variable than directly read the environment variables?

I agree that we will want to modify the release.yml to include the work you have done in the build.yml

Please let me know if you have any opinions on the opt-in vs opt-out, and thanks again!

@madclaws
Copy link
Copy Markdown
Contributor Author

madclaws commented Feb 7, 2024

Hi @Thomas-Jean , i would prefer enabling it by default, which kind of makes the usage of library seamless across even containers.
Reading from config variable is also fine, that could in-turn help the users (non-contributors) who might have an issue with downloading the precompiled binaries for some reason.

@scrogson
Copy link
Copy Markdown

This change would be great to have merged. I forked this repo and made similar changes so that I could take advantage of precompiled artifacts here.

If anyone else wants to get it working using my unpublished fork:

# mix.exs
{:blake3, github: "royal-markets/blake3", sha: "b64c203"}
# config.exs
config :blake3, source_url: "https://github.com/royal-markets/blake3"

@juulSme
Copy link
Copy Markdown
Collaborator

juulSme commented Nov 27, 2024

@scrogson a PR would definitely be considered for merging, but you haven't opened one. Why not give it a go ;)

@scrogson scrogson mentioned this pull request Dec 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants