Data

Australian Search and Rescue Region: Vessel Tracking - Aggregated data product (2012 - ongoing) (NESP MaC 5.9, IMOS)

Australian Ocean Data Network
Australian Maritime Safety Authority (AMSA) ; Integrated Marine Observing System (IMOS)
Viewed: [[ro.stat.viewed]] Cited: [[ro.stat.cited]] Accessed: [[ro.stat.accessed]]
ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Adc&rfr_id=info%3Asid%2FANDS&rft_id=https://catalogue-imos.aodn.org.au/geonetwork/srv/api/records/2a5739e7-0cb8-444a-b83b-b2bc841b0ce8&rft.title=Australian Search and Rescue Region: Vessel Tracking - Aggregated data product (2012 - ongoing) (NESP MaC 5.9, IMOS)&rft.identifier=https://catalogue-imos.aodn.org.au/geonetwork/srv/api/records/2a5739e7-0cb8-444a-b83b-b2bc841b0ce8&rft.description=The vessel tracking data in this product originates from the Australian Maritime Safety Authority’s (AMSA) Craft Tracking System (CTS) and Mariweb databases. These systems collect Automatic Identification System (AIS) signals from ships using both terrestrial and satellite receivers. AIS data from these sources, covering the period from September 2012 to the most recently completed month, provides vessel position reports across the entire Australian Search and Rescue Region. Each record represents a single vessel position defined by geographic coordinates and the time of transmission in Coordinated Universal Time (UTC).To create a consistent and usable dataset, the monthly AMSA vessel tracking files have been processed and standardised into a set of single annual datasets. This processing includes ingesting the source Shapefiles from AMSA’s operational data service, cleaning and validating the vessel position reports, converting timestamps to UTC, applying H3 spatial indexing, and ensuring all records follow a uniform data schema. Any invalid or duplicate reports identified during this workflow are separated into a dedicated rejected records dataset. The resulting product is a harmonised, reliable, and reproducible record of vessel movements throughout the Australian Search and Rescue Region over more than a decade.Maintenance and Update Frequency: monthlyStatement: Data Ingestion Workflow The AMSA Vessel Traffic Optimised Data Product is produced using an Extract–Transform–Load (ETL) workflow that converts the monthly vessel‑tracking Shapefile deliveries published by the Australian Maritime Safety Authority (AMSA) into a harmonised, analysis‑ready Parquet dataset. The workflow standardises coordinates, timestamps, vessel attributes, spatial indexing, and quality controls to support reliable spatial and temporal analyses. Data Extraction AMSA provides monthly vessel‑tracking datasets as nested ZIP archives containing Shapefiles. Each monthly package is downloaded and unpacked using resilient extraction logic to accommodate variations in archive structure or naming conventions. Shapefiles are read into Arrow tables and geometries are converted to WKB for safe handling. Metadata supplied by AMSA in the monthly packages is used as the reference for expected field types and spatial reference information. Data Transformation and Standardisation A series of harmonisation steps ensure consistent structure and interpretation across all monthly datasets: Column standardisation: All column names are uppercased and common misspellings (e.g., TMESTAMP, TIMETAMP) and variant coordinate field names are mapped to standard names. Coordinate recovery: Latitude and longitude are taken directly from source fields where present; otherwise they are reconstructed from geometry and rounded to five decimal places. Timestamp normalisation: Heterogeneous date and datetime formats are parsed and converted into a single UTC timestamp field. Enrichment: Additional provenance fields (SOURCE_FILE_NAME, PROCESSED_DATE) are added, and each record is assigned an H3 spatial index at a configured resolution to support fast spatial aggregation. Validation A strict, sequential validation (“waterfall”) ensures that only high‑quality, spatially valid records enter the final dataset. Records may be rejected for: Duplication Missing or invalid coordinates Global coordinate out‑of‑range values Falling outside the configured AMSA region of interest Occurring on land (using a global land mask for Australia and Antarctica) Rejected records are written to separate Parquet files with explicit reasons for traceability and diagnostic use. Schema Enforcement A target output schema defines field names, data types, and ordering. Before loading, missing columns are added as nulls, types are cast to the defined schema, and the cleaned data are rehydrated into Polars for writing. This ensures stable typing and structure across months and years, including reruns. Aggregation and Loading Validated monthly data are appended to a yearly master Parquet file using an idempotent upsert procedure: existing records from the same source file are removed before new ones are added. This ensures safe reprocessing without duplication. Rejected records are stored separately on a per‑source basis. Update Schedule AMSA releases vessel traffic data on a monthly basis, therefore the ETL pipeline is executed once per month to capture any additions to the source data. Advantages of the Workflow Reproducible: Automates the extraction and conversion of AMSA’s nested Shapefile deliveries into standardised, analysis‑ready formats. Consistent and quality‑controlled: Enforces uniform coordinate systems, timestamps, data types, and vessel attributes across all months and years. Spatially optimised: H3 indexing improves spatial joins and summarisation performance. Cloud‑optimised: Output Parquet files provide efficient storage and rapid access for downstream analytics workflows.&rft.creator=Australian Maritime Safety Authority (AMSA) &rft.creator=Integrated Marine Observing System (IMOS) &rft.date=2026&rft.coverage=westlimit=75.00; southlimit=-86.60; eastlimit=163.00; northlimit=-2.00&rft.coverage=westlimit=75.00; southlimit=-86.60; eastlimit=163.00; northlimit=-2.00&rft_rights=Creative Commons Attribution-Noncommercial 4.0 International License http://creativecommons.org/licenses/by-nc/4.0/&rft_rights=The citation in a list of references is: AMSA, IMOS [year-of-data-download], Australian Search and Rescue Region: Vessel Tracking - Aggregated data product (2012 - ongoing) (NESP MaC 5.9, IMOS), [data-access-URL], accessed [date-of-access].&rft_rights=Any users of IMOS data are required to clearly acknowledge the source of the material derived from IMOS in the format: Data was sourced from Australia?s Integrated Marine Observing System (IMOS) ? IMOS is enabled by the National Collaborative Research Infrastructure strategy (NCRIS). If relevant, also credit other organisations involved in collection of this particular datastream (as listed in 'credit' in the metadata record).&rft_rights=Data, products and services from IMOS are provided as is without any warranty as to fitness for a particular purpose.&rft_subject=oceans&rft_subject=transportation&rft_subject=environment&rft_subject=economy&rft_subject=location&rft_subject=society&rft_subject=Global / Oceans | Indian Ocean&rft_subject=Global / Oceans | Southern Ocean&rft_subject=Global / Oceans | Pacific Ocean&rft_subject=Marine Features (Australia) | Great Australian Bight, SA/WA&rft_subject=Marine Features (Australia) | Bass Strait, TAS/VIC&rft_subject=Regional Seas | Tasman Sea&rft_subject=Regional Seas | Timor Sea&rft_subject=Regional Seas | Arafura Sea&rft_subject=Regional Seas | Coral Sea&rft_subject=Continents | Antarctica&rft_subject=Countries | Australia&rft_subject=States, Territories (Australia) | New South Wales&rft_subject=Countries | Timor-Leste&rft_subject=States, Territories (Australia) | South Australia&rft_subject=Marine Features (Australia) | Great Barrier Reef, QLD&rft_subject=States, Territories (Australia) | Tasmania&rft_subject=States, Territories (Australia) | Victoria&rft_subject=States, Territories (Australia) | Northern Territory&rft_subject=States, Territories (Australia) | Western Australia&rft_subject=Countries | Indonesia&rft_subject=Countries | Papua New Guinea&rft_subject=States, Territories (Australia) | Queensland&rft_subject=vessel of opportunity&rft_subject=vessel of opportunity on fixed route&rft_subject=fishing vessel&rft_subject=ship&rft_subject=naval vessel&rft_subject=geostationary orbiting satellite&rft_subject=research vessel&rft_subject=orbiting satellite&rft_subject=satellite positioning systems&rft_subject=ANTHROPOGENIC/HUMAN INFLUENCED ECOSYSTEMS&rft_subject=TRANSPORTATION&rft_subject=MARINE ENVIRONMENT MONITORING&rft_subject=MARINE VESSELS&rft.type=dataset&rft.language=English Access the data
Cite Saved to MyRDA Save to MyRDA

Licence & Rights:

Other view details
Unknown

Creative Commons Attribution-Noncommercial 4.0 International License
http://creativecommons.org/licenses/by-nc/4.0/

The citation in a list of references is: "AMSA, IMOS [year-of-data-download], Australian Search and Rescue Region: Vessel Tracking - Aggregated data product (2012 - ongoing) (NESP MaC 5.9, IMOS), [data-access-URL], accessed [date-of-access]."

Any users of IMOS data are required to clearly acknowledge the source of the material derived from IMOS in the format: "Data was sourced from Australia?s Integrated Marine Observing System (IMOS) ? IMOS is enabled by the National Collaborative Research Infrastructure strategy (NCRIS)." If relevant, also credit other organisations involved in collection of this particular datastream (as listed in 'credit' in the metadata record).

Data, products and services from IMOS are provided "as is" without any warranty as to fitness for a particular purpose.

Access:

Other

Full description

The vessel tracking data in this product originates from the Australian Maritime Safety Authority’s (AMSA) Craft Tracking System (CTS) and Mariweb databases. These systems collect Automatic Identification System (AIS) signals from ships using both terrestrial and satellite receivers. AIS data from these sources, covering the period from September 2012 to the most recently completed month, provides vessel position reports across the entire Australian Search and Rescue Region. Each record represents a single vessel position defined by geographic coordinates and the time of transmission in Coordinated Universal Time (UTC).

To create a consistent and usable dataset, the monthly AMSA vessel tracking files have been processed and standardised into a set of single annual datasets. This processing includes ingesting the source Shapefiles from AMSA’s operational data service, cleaning and validating the vessel position reports, converting timestamps to UTC, applying H3 spatial indexing, and ensuring all records follow a uniform data schema. Any invalid or duplicate reports identified during this workflow are separated into a dedicated rejected records dataset. The resulting product is a harmonised, reliable, and reproducible record of vessel movements throughout the Australian Search and Rescue Region over more than a decade.

Lineage

Maintenance and Update Frequency: monthly
Statement: Data Ingestion Workflow The AMSA Vessel Traffic Optimised Data Product is produced using an Extract–Transform–Load (ETL) workflow that converts the monthly vessel‑tracking Shapefile deliveries published by the Australian Maritime Safety Authority (AMSA) into a harmonised, analysis‑ready Parquet dataset. The workflow standardises coordinates, timestamps, vessel attributes, spatial indexing, and quality controls to support reliable spatial and temporal analyses. Data Extraction AMSA provides monthly vessel‑tracking datasets as nested ZIP archives containing Shapefiles. Each monthly package is downloaded and unpacked using resilient extraction logic to accommodate variations in archive structure or naming conventions. Shapefiles are read into Arrow tables and geometries are converted to WKB for safe handling. Metadata supplied by AMSA in the monthly packages is used as the reference for expected field types and spatial reference information. Data Transformation and Standardisation A series of harmonisation steps ensure consistent structure and interpretation across all monthly datasets: Column standardisation: All column names are uppercased and common misspellings (e.g., TMESTAMP, TIMETAMP) and variant coordinate field names are mapped to standard names. Coordinate recovery: Latitude and longitude are taken directly from source fields where present; otherwise they are reconstructed from geometry and rounded to five decimal places. Timestamp normalisation: Heterogeneous date and datetime formats are parsed and converted into a single UTC timestamp field. Enrichment: Additional provenance fields (SOURCE_FILE_NAME, PROCESSED_DATE) are added, and each record is assigned an H3 spatial index at a configured resolution to support fast spatial aggregation. Validation A strict, sequential validation (“waterfall”) ensures that only high‑quality, spatially valid records enter the final dataset. Records may be rejected for: Duplication Missing or invalid coordinates Global coordinate out‑of‑range values Falling outside the configured AMSA region of interest Occurring on land (using a global land mask for Australia and Antarctica) Rejected records are written to separate Parquet files with explicit reasons for traceability and diagnostic use. Schema Enforcement A target output schema defines field names, data types, and ordering. Before loading, missing columns are added as nulls, types are cast to the defined schema, and the cleaned data are rehydrated into Polars for writing. This ensures stable typing and structure across months and years, including reruns. Aggregation and Loading Validated monthly data are appended to a yearly master Parquet file using an idempotent upsert procedure: existing records from the same source file are removed before new ones are added. This ensures safe reprocessing without duplication. Rejected records are stored separately on a per‑source basis. Update Schedule AMSA releases vessel traffic data on a monthly basis, therefore the ETL pipeline is executed once per month to capture any additions to the source data. Advantages of the Workflow Reproducible: Automates the extraction and conversion of AMSA’s nested Shapefile deliveries into standardised, analysis‑ready formats. Consistent and quality‑controlled: Enforces uniform coordinate systems, timestamps, data types, and vessel attributes across all months and years. Spatially optimised: H3 indexing improves spatial joins and summarisation performance. Cloud‑optimised: Output Parquet files provide efficient storage and rapid access for downstream analytics workflows.

Notes

Credit
Source data - Commonwealth of Australia (Australian Maritime Safety Authority)
Credit
Australia's Integrated Marine Observing System (IMOS) is enabled by the National Collaborative Research Infrastructure Strategy (NCRIS). It is operated by a consortium of institutions as an unincorporated joint venture, with the University of Tasmania as Lead Agent.
Credit
The data collection described in this record was funded by the Australian Government Department of Climate Change, the Environment, Energy & Water (DCCEEW) through the NESP Marine and Coastal Hub. In addition to NESP (DCCEEW) funding, this project was supported by an equivalent amount of in-kind support and co-investment from project partners and collaborators.

Created: 04 02 2026

Data time period: 2012-09-01

This dataset is part of a larger collection

Click to explore relationships graph
Help

Related People

163,-2 163,-86 75,-86 75,-2 163,-2

119,-44.3

text: westlimit=75.00; southlimit=-86.60; eastlimit=163.00; northlimit=-2.00

Subjects
ANTHROPOGENIC/HUMAN INFLUENCED ECOSYSTEMS | Continents | Antarctica | Countries | Australia | Countries | Indonesia | Countries | Papua New Guinea | Countries | Timor-Leste | Global / Oceans | Indian Ocean | Global / Oceans | Pacific Ocean | Global / Oceans | Southern Ocean | MARINE ENVIRONMENT MONITORING | MARINE VESSELS | Marine Features (Australia) | Bass Strait, TAS/VIC | Marine Features (Australia) | Great Australian Bight, SA/WA | Marine Features (Australia) | Great Barrier Reef, QLD | Regional Seas | Arafura Sea | Regional Seas | Coral Sea | Regional Seas | Tasman Sea | Regional Seas | Timor Sea | States, Territories (Australia) | New South Wales | States, Territories (Australia) | Northern Territory | States, Territories (Australia) | Queensland | States, Territories (Australia) | South Australia | States, Territories (Australia) | Tasmania | States, Territories (Australia) | Victoria | States, Territories (Australia) | Western Australia | TRANSPORTATION | economy | environment | fishing vessel | geostationary orbiting satellite | location | naval vessel | oceans | orbiting satellite | research vessel | satellite positioning systems | ship | society | transportation | vessel of opportunity | vessel of opportunity on fixed route |

User Contributed Tags    

Login to tag this record with meaningful keywords to make it easier to discover

Other Information
(AMSA - Automatic Identification System webpage)

url : https://www.amsa.gov.au/safety-navigation/navigation-systems/about-automatic-identification-system

(AMSA - Vessel Tracking Data (source data - annual and monthly shapefile downloads + associated metadata))

url : https://www.operations.amsa.gov.au/spatial/DataServices/DigitalData

(View and download data though the AODN Portal)

url : https://portal.aodn.org.au/search?uuid=2a5739e7-0cb8-444a-b83b-b2bc841b0ce8

(Access To AWS Open Data Program registry for the Cloud Optimised version of this dataset (link to be added))

url : https://registry.opendata.aws/

(Data files accessible via Amazon S3 (public access, S3 URI))

local : s3://data-uplift-public/stored/datauplift/amsa/

(Access to Jupyter notebook to query Cloud Optimised converted dataset)

url : https://github.com/aodn/imos-user-code-library/blob/master/NESP/amsa.ipynb

(Access to R Markdown notebook to query Cloud Optimised converted dataset)

url : https://github.com/aodn/imos-user-code-library/blob/master/NESP/amsa.Rmd

(Video tutorials demonstrating the use of the associated Python Jupyter notebook)

url : https://youtube.com/playlist?list=PLHCEbETnUz5yyykHxoV-LYfEksOZthCZE&si=Rq4gufLsonVEn_Ep

(Video tutorials demonstrating the use of the associated R notebook)

url : https://youtube.com/playlist?list=PLHCEbETnUz5y3373qW6azsuoAEO4iQ6NC&si=fXBpbaL-4G4NWl_4

(Technical description of product)

url : https://content.aodn.org.au/Documents/IMOS/Data_product/Vessel_v1.0.pdf

global : aeb0afce-7fc7-4d48-91fc-f7b8e730073c

local : 010x3gp67

local : 010x3gp67

NESP MaC Project 5.9 – Making marine environmental data more assessment ready, 2025 (UTAS, IMOS)

doi : 10.82210/a44c9c1d

Identifiers
  • global : 2a5739e7-0cb8-444a-b83b-b2bc841b0ce8