6. Data Management

Data are submitted at the end of the breeding season with a submission deadline of 31 October each year. When data are received, they undergo a process of checking and archiving:

  • The data submissions are filed electronically in the format received. Up to and including 2019 most data have been received on the standard SRMS spreadsheet but following the launch of SRMS Online (https://app.bto.org/demography/srms/public/login.jsp) in April 2020 the expectation is for an increasing number of submissions to be online going forward.
  • A copy of each data submission is amalgamated with all other data received for that breeding season.
  • The amalgamated dataset is checked for spatial errors and erroneous grid references corrected, where identified.
  • The amalgamated dataset is checked for duplicate records.
  • The amalgamated dataset is run through a cleaning programme. Data which are incomplete, incorrect, inaccurate and irrelevant are located then replaced, amended or removed to produce a final quality assured dataset fit for purpose. Cleaning ensures, for example, that observer, species, site names and codes and the habitat information are standardised. Incomplete or inaccurate information on occupancy and nest contents are also intelligently populated, where this is feasible, based on information that has been provided[1].
  • The cleaned version of the data is then added to the SRMS master dataset.
  • A copy of the relevant subset of the cleaned version of the data is also sent to each original data provider for their records.
  • A copy of the cleaned dataset is uploaded to Atlas of Living Scotland at resolutions that SRMG agree are appropriate for public viewing (see sections 6.1 and 7.2).
  • A copy of the cleaned dataset is uploaded to the SRMS Legacy Data Portal to be available to SRMS partner organisations (see sections 6.2 and 8).

[1] Occupancy information will be upgraded based on the subsequent productivity information (e.g. where a raw record has occupancy captured as “single adult” but the productivity information indicates there was a nesting attempt, the cleaning programme upgrades the occupancy status to “adult pair”).  Productivity information at various stages (egg, small chick, large chick and fledged young) will be upgraded based on subsequent stages (e.g. where a raw record has the number of eggs captured as “outcome unknown” (perhaps as a raptor worker was unable to check the nest contents at this stage) but another part of the record indicates that chicks were present on a subsequent visit then the cleaning programme upgrades the eggs to “Yes”).

See:
6.1 Administration of SRMS data on Atlas of Living Scotland
6.2 Administration of SRMS data on a secure online interface

Next section:
7. Accessing SRMS data