Compressed graph

A compact and highly-efficient representation of the graph dataset, suited for scale-up analysis on high-end machines with large amounts of memory. The graph is compressed in Boldi-Vigna representation, designed to be loaded by the WebGraph framework, specifically using our swh-graph library.

Comments: author and committer timestamps were shifted back 1 or 2 hours, based on the Europe/Paris timezone, see https://gitlab.softwareheritage.org/swh/devel/swh-graph/-/issues/4788
Dataset size: 7.1 TiB
Export date: 2022-12-07
Derived dataset: Path counts
S3 URL: s3://softwareheritage/graph/2022-12-07/compressed/
Deprecated: False

Referencing Software Heritage

If you use any of the datasets indexed on this website for research purposes, please acknowledge Software Heritage as recommended in the publications page, that is:

Add a footnote on the title page of your paper, formatted as: “This work was made possible by Software Heritage, the universal source code archive: https://www.softwareheritage.org”; and
cite at least one of the following papers:
- Roberto Di Cosmo, Stefano Zacchiroli. Software Heritage: Why and How to Preserve Software Source Code. iPRES 2017. (BibTeX)
- Jean-François Abramatic, Roberto Di Cosmo, Stefano Zacchiroli. Building the universal archive of source code. Commun. ACM 61(10): 29-31 (2018). (BibTeX)

Specific datasets might recommend additional citations, to credit their creators.

Specific citation instructions

If you use this dataset for research purposes, please cite the following paper: Antoine Pietri, Diomidis Spinellis, Stefano Zacchiroli. The Software Heritage Graph Dataset: Public software development under one roof. In proceedings of MSR 2019: The 16th International Conference on Mining Software Repositories, May 2019, Montreal, Canada. Co-located with ICSE 2019 (Preprint), (BibTeX).

Download the dataset

For Amazon S3 links, you'll need to install either awscli or swh.datasets.

aws s3 cp --recursive --no-sign-request s3://softwareheritage/graph/2022-12-07/compressed/ 2022-12-07-compressed
# OR
swh datasets download-graph 2022-12-07