Skip to content

Conversation

@ntjohnson1
Copy link
Collaborator

@ntjohnson1 ntjohnson1 commented Nov 14, 2025

Marking as draft since I need to set up some more perf comparisons but local ones looked good in initial exploration.
Also want to check this on the MATLAB side to see how annoying it is to export to a similar format or open this dump in MATLAB.
Closes #466


📚 Documentation preview 📚: https://pyttb--467.org.readthedocs.build/en/467/

@ntjohnson1
Copy link
Collaborator Author

@gqcollins would be curious to see if this helps your use case. AFAICT both of these binary approaches don't use pickle so should be more generally compatible across systems/versions.

@ntjohnson1 ntjohnson1 changed the title Binary export import Binary export/import Nov 14, 2025
@gqcollins
Copy link

I'll try it next week and get back to you. Thanks!

@gqcollins
Copy link

The binary code does help my use case quite a bit. As I mentioned before, the exports I'm doing take ~80s with the .txt format (using the improved numpy version) and imports take ~20s. The numpy binary version takes ~1.9s for exports and ~3.3s for imports. The .mat version takes ~2.5s for exports and ~1.6s for imports. And I can easily port these files to other systems.

You'll have to decide whether it's worth adding these (or at least one of them) to the repo or not. In any case, I appreciate you offering me the code!

@ntjohnson1
Copy link
Collaborator Author

Great! I'm glad it helped. I think the primary difference you're seeing in time is that mat defaults to compression where numpy doesn't.

I'm pretty confident we'll add one, both, or an alternative. But it might not land for a little bit, I probably need to touch base with Danny. I still need to check how easy it is to interop with the MATLAB ttb.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Should we support a binary serialization format?

2 participants