View source for Submissions:2019/4 Million in 4 Weeks: A case study on bulk import of cultural heritage metadata on Wikidata

{{WCNA 2019 Session Submission
|theme=Relationship Building & Support<br />+ Tech & Tools<br />
|type=Presentation
|abstract=In August, 2019 alone, the US National Archives added over 4 million statements—and 400,000 new items—to Wikidata. Primarily, the edits were made via a custom [[w:en:Python_(programming_language)|Python]] script using the [[mw:Manual:Pywikibot|Pywikibot library]], running on [[mw:PAWS|PAWS]], with some additional usage of [[d:Help:QuickStatements|QuickStatements]] batches.

The project was both a technical endeavor—requiring us to develop a method for accessing large amounts of NARA data, transforming it, and posting it to Wikidata—as well as an intellectual one, since not only had NARA data standard not been previously mapped to Wikidata properties, but archival description itself on Wikidata is still in its early stages, and so many of the types of data we were trying to add had never been modeled before.

This presentation will be a deep dive into just how I did it, with thoughts on tools, data modeling, working with data sources, and navigating the community. It will also cover technical challenges and other reflections from the ongoing work, including dealing with large-scale data, gathering metrics and tracking progress, and working on behalf of an institution and getting buy-in from non-technical staff.
|academic=No
|author=Dominic Byrd-McDevitt
|email=dominic@byrd-mcdevitt.com
|username=Dominic
|affiliates=National Archives and Records Administration
|time=30 minutes
|present-other=Yes
}}