User:Fæ/LOC maps

From Wikimedia Commons, the free media repository
Jump to navigation Jump to search
Political map of the world, CIA 2012
China in Nan Song Dynasty, print of stone rubbing 1903, original carving 1136

Introduction

[edit]

Map uploads from the Library of Congress "portal" for maps at https://www.loc.gov/maps. Requested by Jheald in April 2018.

All maps appear in Category:Maps in the Library of Congress, which pre-batch upload started with 602 images. A report of files in this batch upload project is Petscan, this may lag the live data by a day. This search shows only jpegs with LOC in the title in the bucket category, which gives a more relevant total count of uploads than both jpegs and tiffs.

GLAM dashboard reports are available:

The methods applied for the upload are similar to those discussed at the LOC Prints project page.

Useful searches of the maps collection

[edit]

Technical

[edit]

Formats

[edit]

Images are pulled using iiif url requests. In theory these can return many formats including jp2, png, gif and tiff. In practice the requests for full size png formats may indefinitely return server 503 errors, presumably because the pngs are not held natively and are probably transcoded on the fly from a jp2 source. Consequently tiff is requested, which unfortunately also frequently gives 503 errors.

As tiffs are rendered by Wikimedia Commons poorly, a jpeg format is created in parallel and cross-linked by a gallery on the image page.

Known errors

[edit]

Upload configuration

[edit]
L'Amérique septentrionale, 1783
[edit]

Based on the LOC description of the collection, the maps either date from before 1923, or are US Government works. Consequently the basic licenses are chosen as:

if year<1918:
  license = "PD-Art|PD-old-100-1923"
elif year<1923:
  license = "PD-Art|PD-old-80-1923"
else:
  license = "PD-USGov"

The generic PD-USGov may be refined automatically by the pre-existing maintenance task User:Fæ/code/PD-USGov.

Categorisation

[edit]
Deepwater Horizon incident. Forecast oil spill area in Gulf of Mexico, NOAA 2010

There are two forms of category checked, place and date:

  • Old maps of <place>
  • <year> maps

If these exist as categories, they are added to the file. <place> alternatives are found by checking for location_city, location_county, location_state and location_country. For multiple sheet maps, the metadata has a 'location' array, which unfortunately appears unpredictably sorted, consequently the place category may not be the most local possible.

A separate housekeeping task is picking up on categories which are generated from U.S. location names but accidentally match to existing categories for non-U.S. locations. The mappings are:

mapping = [
		["Birmingham|Alabama", "Old maps of Alabama"],
		["Birmingham|Connecticut", "Old maps of Connecticut"],
		["Bristol|Connecticut", "Old maps of cities in Connecticut"],
		["Derby|Connecticut","Old maps of cities in Connecticut"],
		["Norwich|Connecticut", "Old maps of Connecticut"],
		["Salisbury|Connecticut", "Old maps of Connecticut"],
		["Athens|Georgia", "Old maps of Georgia (U.S. state)"],
		["Georgia", "Old maps of Georgia (U.S. state)"],
		["Lincoln|Illinois", "Old maps of Illinois"],
		["Attica|Indiana", "Old maps of Indiana"],
		# M
		["Cumberland|Maryland", "Old maps of cities in Maryland"],
		["North East England|Maryland", "Old maps of cities in Maryland"],
		["Belfast|Maine", "Old maps of Maine"],
		["Cambridge|Maryland", "Old maps of Maryland"],
		["Bedford|Massachusetts", "Old maps of Massachusetts"],
		["Cambridge|Massachusetts", "Old maps of Cambridge, Massachusetts"],
		["Chester|Massachusetts", "Old maps of Massachusetts"],
		["Gloucester|Massachusetts", "Old maps of Massachusetts"],
		["Ipswich|Massachusetts","Old maps of Massachusetts"],
		["Worcester|Massachusetts", "Old maps of Massachusetts"],
		["Versailles|Missouri", "Old maps of Missouri"],
		# N
		["Bristol|New Hampshire", "Old maps of New Hampshire"],
		["Canterbury|New Hampshire", "Old maps of New Hampshire"],
		["Exeter|New Hampshire", "Old maps of New Hampshire"],
		["Portsmouth|New Hampshire", "Old maps of New Hampshire"],
		["Lincoln|Nebraska", "Old maps of Nebraska"],
		["Antwerp|New York", "Old maps of New York (state)"],
		["Cambridge|New York", "Old maps of New York (state)"],
		["Dundee|New York", "Old maps of New York (state)"],
		["Rome|New York", "Old maps of New York"],
		["Salamanca|New York", "Old maps of New York"],
		["Syracuse|New York", "Old maps of New York (state)"],
		["Cambridge|Ohio", "Old maps of cities in Ohio"],
		["Lancaster|Ohio", "Old maps of cities in Ohio"],
		["Portsmouth|Ohio", "Old maps of cities in Ohio"],
		# P
		["Alexandria|Pennsylvania", "Old maps of Pennsylvania"],
		["Berlin|Pennsylvania", "Old maps of Pennsylvania"],
		["Bradford|Pennsylvania", "Old maps of Pennsylvania"],
		["Bristol|Pennsylvania", "Old maps of Pennsylvania"],
		["Carlisle|Pennsylvania", "Old maps of Pennsylvania"],
		["Cornwall|Pennsylvania", "Old maps of Pennsylvania"],
		["Derry|Pennsylvania", "Old maps of Pennsylvania"],
		["Hamburg|Pennsylvania", "Old maps of Pennsylvania"],
		["Oxford|Pennsylvania", "Old maps of Pennsylvania"],
		["Somerset|Pennsylvania", "Old maps of Pennsylvania"],
		["Bristol|Rhode Island", "Old maps of Rhode Island"],
		# S
		["Bristol|Vermont", "Old maps of Vermont"],
		["Essex|Vermont", "Old maps of Vermont"],
		["Rutland|Vermont", "Old maps of Vermont"],
		["Suffolk|Virginia", "Old maps of Virginia"],
		["Alexandria|Virginia", "Old maps of Virginia"],
		["Bristol|Virginia", "Old maps of Virginia"],
		["Portsmouth|Virginia", "Old maps of Virginia"],
		["Winchester|Virginia", "Old maps of Virginia"],
		["Berlin|Wisconsin", "Old maps of cities in Wisconsin"],
 ]

Where the mapping is <mismatched category, without "Old maps of " prefix>|<optional wikitext conditional match> → <correct U.S. location category>. The extra conditional match is needed for locations like "Bristol" where the default matched to the U.K. location and Bristol exists as a (realistic LOC metadata) place in Virginia and Connecticut. This test only applies to LOC map files where the Location metadata, added to the wikitext image page, includes "United States".

Singletons

[edit]

Maps are separated in to those with multiple pages and those displayed as a gallery with the flag "hassegments" in the json data. The first run of uploads are solely with singletons to avoid the complexity of categorisation.

During 2018, once the uploads appeared stable, the uploading was put out to an always connected Raspberry Pi device, as an experiment with low-power headless loading, and to reduce demand on a primary desktop.

Multiple sheet maps

[edit]

Multiple sheet maps are added to an automatically created atlas category based on <title>. Example Category:Sanborn Fire Insurance Map from Newberry, Newberry County, South Carolina.

As the way the information template is used varies significantly from singletons, a separate batch upload script is used. Year and place categories are added to the generated atlas category.

As there are a lot of Sanborn maps, the categorization attempts to find a sub-category of Category:Sanborn maps, if there is a relevant match to place metadata.