Sites, Pages and Twins

Sites become linked when their pages are forked from other sites. In this dataset we list visible sites in the federation, the date their pages were most recently edited, and the remote sites mentioned in their journals.

{ site: { slug: { date: integer, twins: [site, ...] }, ... }, ... }

The json for a recent scrape is 5MB, compressed to 1MB.

pages/sites-pages-and-twins

We've included the deno script we use to construct the rollup from incremental file structured used in seran-scrape. github