Web Data Commons - Extracting Structured Data from Two Large Web Corpora

Publication Date: 
Monday, 16 April, 2012
Published in: 
4th Linked Data on the Web Workshop LDOW2012
Hannes Mühleisen, Christian Bizer (FUB)

More and more websites embed structured data describing for instance products, people, organizations, places, events, resumes, and cooking recipes into their HTML pages using encoding standards such as Microformats, Microdatas and RDFa. The Web Data Commons project extracts all Microformat, Microdata and RDFa data from the Common Crawl web corpus, the largest and most up-todata web corpus that is currently available to the public, and provides the extracted data for download in the form of RDF-quads. In this paper, we give an overview of the project and present statistics about the popularity of the different encoding standards as well as the kinds of data that are published using each format.