Precomputed tables of all revisions a content is in, and all origins a revision is in, indexed for reasonably efficient backward queries. Its current implementation is primarily a Parquet database, along with some external indexes for more efficient access. The swh-provenance Rust crate provides access to these indexes and a gRPC server to query the data remotely.
If you use any of the datasets indexed on this website for research purposes, please acknowledge Software Heritage as recommended in the publications page, that is:
Specific datasets might recommend additional citations, to credit their creators.
For Amazon S3 links, you'll need to install either awscli or swh.datasets.
aws s3 cp --recursive --no-sign-request s3://softwareheritage/derived_datasets/2024-12-06/provenance/all/ 2024-12-06-provenance_allFor Amazon S3 links, you'll need to install either awscli or swh.datasets.
aws s3 cp --recursive --no-sign-request s3://softwareheritage/derived_datasets/2024-12-06/provenance/heads/ 2024-12-06-provenance_headsBy accessing the datasets, you agree with the Software Heritage Ethical Charter for using the archive data, the terms of use for bulk access, and the Software Heritage principles for large language models.
To learn how to use the datasets read the documentation.