Aeromancy

Aeromancy is an opinionated philosophy and open-sourced framework that closely tracks experimental runtime environments for more reproducible machine learning. In existing experiment trackers, it’s easy to miss important details about how an experiment was run, e.g., which version of a dataset was used as input or the exact versions of library dependencies. Missing these details can make replicability more difficult. Aeromancy aims to make this process smoother by providing both new infrastructure (a more comprehensive versioning scheme including both system runtimes and external datasets) and a corresponding set of best practices to ensure experiments are maximally trackable.

In its current form, Aeromancy requires a fairly specific software stack: (hey, we said it was opinionated)

• Experiment tracker: Weights and Biases
• Object storage (artifacts): S3-compatible, e.g., Ceph
• Virtualization: Docker
• Python Package Manager: pdm
• Revision Control: Git

Aeromancy at SciPy 2024

Check out our abstract and poster:

Documentation overview

• If you're new to Aeromancy, start here! Or check out our SciPy 2024 abstract for a high level overview.
• In the Developer Reference section of the documentation, we include some design docs which provide an architectural overview and a glossary of terms.
• Want to get involved? We have starting points in our Contributor Guidelines.
• Lastly, we have autogenerated documentation in Code Reference.

Note

Aeromancy documentation is still in a very early state. As this is a pre-release, support may be limited.