Australasian Seabird Occurences (including migratory species) - Aggregated data product (1939 - ongoing) (NESP MaC 5.9, IMOS)

Full description

This aggregated product collates observed seabird sightings from multiple CSIRO‑hosted Ocean Biodiversity Information System (OBIS) datasets spanning historical to contemporary programs, unifying platform‑based visual surveys, voyage‑based watch observations (WOV), archival ship logs, and animal‑borne tracking into one analysis‑ready resource. Ship‑based visual surveys (including standardised watch-observations from research vessels) record species identity, counts and behaviour along vessel tracks, with observation effort and environmental context captured as measurement‑or‑fact where available (e.g., Research vessel voyages: Investigator, Falkor, Franklin, Southern Surveyor and Aurora Australis). Historical atlas and ship‑log compilations extend temporal coverage back to the 1970s and earlier through curated digitisation of paper records and atlases.

Complementary animal‑tracking datasets add individual movement ecology via light‑level geolocators (GLS) and satellite/GPS tags, capturing migration routes, staging areas and foraging ranges (e.g., short‑tailed shearwaters, Arctic terns). Tracking programs delivered under IMOS Animal Tracking and other initiatives contribute deployment metadata, phases (e.g., pre‑laying exodus, incubation), and repeated seasons, enabling cross‑study synthesis of space use across the Southern Ocean and beyond.

Across sources, records are standardised to Darwin Core with taxonomic alignment to World Register of Marine Species (WoRMS, https://www.marinespecies.org/about.php#what_is_worms), harmonising vessel‑based sightings, historical ship logs and telemetry fixes for robust ecological, biogeographic and conservation analyses. In addition, this product has been normalised with a H3 spatial index applied, and Australian Marine Region Tags have been added which allows spatial filtering of the data based on known marine regions (for example, commonwealth marine regions).

Lineage

Maintenance and Update Frequency: monthly

Statement: Data Ingestion Workflow The Seabird Aggregated Data Product is generated using a reproducible Extract–Transform–Load (ETL) workflow designed to unify the many heterogeneous seabird datasets published through the OBIS Australia Node into a single harmonised dataset suitable for scientific analysis. Data Extraction Source datasets are obtained from the OBIS Australia Node as Darwin Core Archives (DwC‑A). Each archive includes metadata (EML), structural definitions (meta.xml), and one or more core data tables—typically occurrence, event, and extendedMeasurementOrFact (EMOF). Relevant taxonomic and vocabulary references are retrieved from the World Register of Marine Species (WoRMS) and the GBIF Life Stage vocabulary to ensure globally consistent terminology. Schema Validation and Standardisation Darwin Core tables are validated against predefined schemas prior to further processing. Mandatory fields such as occurrenceID, scientificNameID, scientificName, and eventDate are checked for completeness, and records missing essential spatial data (latitude and longitude) are removed. EMOF tables must reference a valid occurrence or event record; those that do not are excluded. Dataset metadata (e.g., title, abstract, contacts, temporal/spatial extent) is extracted directly from eml.xml. Transformation and Harmonisation The transformation step integrates the Darwin Core content with WoRMS taxonomy and GBIF life‑stage vocabularies. Key steps include: • Taxonomic validation: Scientific names, scientificNameIDs, and verbatim identifications are cross‑checked against WoRMS to ensure each observation is linked to an accepted marine taxon with correct hierarchy and status. • Timestamp standardisation: eventDate values are checked against ISO‑8601 patterns and normalised to ensure consistent temporal interpretation. • Life‑stage validation: Reported life‑stage values are validated against the GBIF vocabulary to ensure controlled terminology. • EMOF pivoting: The long‑format EMOF table (which may contain thousands of measurement types) is pivoted into wide format to merge environmental and contextual measurements with the corresponding occurrence record. Measurement types with significant data coverage (e.g., sea state, sea‑surface temperature, depth) are retained; less common attributes fall below the threshold for inclusion. • Measurement harmonisation: Where distinct measurement types represent the same underlying variable (e.g., “Wind Speed (knt)” vs “Wind Force (Beaufort)”), they are merged only where units and interpretation allow consistent integration. After merging, all tables are joined—occurrence, EMOF, and dataset‑level metadata—into a single unified record for each seabird observation. Aggregation and Quality Control All validated and transformed datasets are aggregated vertically into a single parquet dataset. Additional filtering ensures: • Rows with missing spatial coordinates, missing organism quantity, or invalid EMOF relationships are removed. • Empty or unused columns are dropped. • Each record is enriched with WoRMS taxonomic hierarchy and dataset‑level metadata. An ETL timestamp is added to allow tracking of update cycles and reproducibility. Loading and Publication The final harmonised dataset is written to cloud storage in Parquet format. The columnar format and compression significantly reduce file size relative to the source archives and improve accessibility for analytical workflows and scheduled data refreshes. Update Schedule The underlying seabird datasets may be updated at OBIS Australia, therefore the ETL pipeline is executed once per month to capture any changes to the source data. Advantages of the Workflow • Reproducible: Provides a transparent and standardised method for joining complex Darwin Core content into a single table. • Harmonised: Ensures consistent species names, life‑stage terms, measurements, and temporal formats across disparate datasets. • Efficient: Original multi‑file Darwin Core archives are reduced to a compact ~5 MB Parquet file, optimising storage and compute. • Cloud‑native: The workflow supports automated updates and fast access for researchers.

Statement: Datasets ingested from OBIS Australia Please see the seabirds spreadsheet (link in distribution) for the following information about each individual dataset - Dataset ID - Dataset Title - Dataset Contact - Name - Organisation - Role/Title - Email address - Licence - OBIS data source link - Data Organisation Owner

Notes

Credit
Source data - obtained from the OBIS Australia Node (managed by the CSIRO National Collections and Marine Infrastructure Information and Data Centre)

Credit
Source data - all contributors of seabird data to the OBIS Australia Node (data providers retain ownership of the data provided - authoritative scientists and science organisations approved by OBIS)

Credit
Source data - collated for AODN aggregation purposes by Dave Watts and Sachit Rajbhandari

Credit
Australia's Integrated Marine Observing System (IMOS) is enabled by the National Collaborative Research Infrastructure Strategy (NCRIS). It is operated by a consortium of institutions as an unincorporated joint venture, with the University of Tasmania as Lead Agent.

Credit
The data collection described in this record was funded by the Australian Government Department of Climate Change, the Environment, Energy & Water (DCCEEW) through the NESP Marine and Coastal Hub. In addition to NESP (DCCEEW) funding, this project was supported by an equivalent amount of in-kind support and co-investment from project partners and collaborators.

Credit
CSIRO National Collections and Marine Infrastructure Information and Data Centre

Credit
Any users of CSIRO Marine National Facility (MNF) data (identified in 'Source' file - "Data Organisation Owner") are required to clearly acknowledge the source of the material in the format "We acknowledge the use of the CSIRO Marine National Facility (https://ror.org/01mae9353) in undertaking this research." Acknowledgment and citations should be in all forms of publication including presentations (where the acknowledgement should be on the closing slide), journals, books, reports and related research outputs.