Skip to content

Create some tests to use as a performance benchmark #321

@bryan-harter

Description

@bryan-harter

We should try to compare reading/writing some data to the C library. Spacepy CDF wraps the C library, they might make a good thing to compare to. We can time the differences and perhaps use cprofile to inspect the read/write time of various files. At that point, we could look for potential improvements to make.

A few performance ideas come to mind, which may or may not be done as a part of this ticket:

  1. Convert all input to numpy. When the library first came out, there was an aversion to relying on any library except pure-python to read and write CDF files. At this point, numpy is so standard to have for scientific computing that we should just convert all of our input into numpy format, and then work with it all from there. If absolutely necessary, we could make a separate version of the library that relies on nothing but the standard built-in stuff.

  2. mmap might provide faster in-memory reading

  3. Numba might be nice for compiling heavy functions into machine code prior to execution. This would add another dependency to the library, so this should be last resort.

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    Status

    Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions