Provenance Index (releases and head revisions)

Precomputed tables of all revisions a content is in, and all origins a revision is in, indexed for reasonably efficient backward queries. Its current implementation is primarily a Parquet database, along with some external indexes for more efficient access. The swh-provenance Rust crate provides access to these indexes and a gRPC server to query the data remotely.All revisions that are not directly pointed by a snapshot or release (ie. not branch heads or tagged revisions) are omitted.

Dataset size
Unknown
Export date
Derived of
Compressed graph [2024-12-06]
S3 URL
s3://softwareheritage/derived_datasets/2024-12-06/provenance/heads/
Deprecated
False

Referencing Software Heritage

If you use any of the datasets indexed on this website for research purposes, please acknowledge Software Heritage as recommended in the publications page, that is:

  1. Add a footnote on the title page of your paper, formatted as: “This work was made possible by Software Heritage, the universal source code archive: https://www.softwareheritage.org”; and
  2. cite at least one of the following papers:

Specific datasets might recommend additional citations, to credit their creators.

Download the dataset

For Amazon S3 links, you'll need to install either awscli or swh.datasets.

aws s3 cp --recursive --no-sign-request s3://softwareheritage/derived_datasets/2024-12-06/provenance/heads/ 2024-12-06-provenance_heads

By accessing the datasets, you agree with the Software Heritage Ethical Charter for using the archive data, the terms of use for bulk access, and the Software Heritage principles for large language models.

To learn how to use the datasets read the documentation.