| Surveying | |
DEM delivers better approximate AHD heights on public record
This paper outlines how a state-wide Digital Elevation Model (DEM) has been used to deliver better approximate Australian Height Datum (AHD) heights, yielding a homogeneous dataset of known provenance and verifiable quality across New South Wales (NSW), Australia. |
 |
|
 |
|
The Survey Control Information Management System (SCIMS) is the state’s database containing more than 250,000 survey marks on public record across New South Wales (NSW). It was recently updated with Australian Height Datum (AHD; Roelse et al., 1971; Janssen and McElroy, 2021a) heights sourced from a state-wide Digital Elevation Model (DEM) for more than 127,000 survey marks with existing Class U (i.e. approximate) or null AHD height values. Almost 19,000 marks received an AHD height for the first time, while 100 gross AHD height errors in SCIMS were identified and corrected.
The updated AHD heights are classified as Class U and displayed in SCIMS to the nearest metre. They provide important benefits for industry such as enabling the calculation and reporting through SCIMS of the Combined Scale Factor (CSF) and supporting datum modernisation to further improve user access to survey information.
DCS Spatial Services, a unit of the NSW Department of Customer Service (DCS), provides a state-wide DEM with a vertical uncertainty of ±0.9 m at the 95% confidence level (CL) and a horizontal grid density of 5 m. While it has been available through the Intergovernmental Committee on Surveying and Mapping’s (ICSM’s) Elevation Information System (ELVIS; ICSM, 2022) as 2 km by 2 km data tiles for some time, the DEM can now be queried directly through a publicly accessible Application Programming Interface (API) to return an AHD height at a specified location.
This paper outlines how this new interface has been used to query the coordinates of all marks in SCIMS to retrieve AHD heights from the elevation model, assess the accuracy of these AHD heights and update SCIMS with DEM-sourced heights for survey marks with existing Class U or null AHD height values to yield a homogeneous dataset of known provenance and verifiable quality across NSW.
Background
As surveyors and spatial professionals, we know that height and elevation data is crucial for a vast number of applications and that the survey control we provide underpins this data. DCS Spatial Services delivers various imagery and elevation products as part of its ongoing custodial responsibilities regarding the NSW Foundation Spatial Data Framework.
Clarifying the terminology used, a Digital Elevation Model (DEM) represents the bare-earth surface void of all natural and built features, while a Digital Surface Model (DSM) captures both the natural and built/artificial features of the environment (i.e. including the top of vegetation and buildings).
Accurate and reliable orthorectified aerial imagery and high-resolution elevation data are critical to effective planning, decision making, change monitoring and risk mitigation across NSW and are utilised by government, industry and the community. Reliable and quality-assured survey control is fundamental to ensuring the integrity of this data, which contributes significantly to economic, social and environmental sustainability in NSW.
As such, the Imagery and Elevation program and project work conducted by the Survey Operations team at DCS Spatial Services supports the following (Powell, 2017):
- Digital Image Acquisition System (DIAS) program, which captures high-resolution 50 cm Ground Sample Distance (GSD) aerial imagery state-wide.
- Digital Town Imagery Capture (DTIC) program, which captures high-resolution 10 cm GSD aerial imagery over cities, towns and villages throughout NSW.
- Light Detection and Ranging (LiDAR) program, which captures highly accurate elevation data in high-risk areas across NSW.
- Surface Model Enhancement (SME) project (2014-19), which utilised a variety of technology including aerial imagery and LiDAR to create a high-resolution, state-wide DSM.
One of the products provided by DCS Spatial Services is a state-wide DEM with a vertical uncertainty of ±0.9 m (95% CL) and a horizontal grid density of 5 m. It was produced by a combination of category 1 LiDAR, category 3 LiDAR, 10 cm ground resolution imagery and 50 cm ground resolution imagery. The DEM can now be queried directly through a new publicly accessible API to return an AHD height at a specified location. An API is essentially a connection between computers or between computer programs, i.e. a software interface offering a service to other pieces of software. This is extremely useful when dealing with large amounts of data and/ or machine-to-machine processes.
AHD height retrieval from the DEM via API
In May 2021, AHD heights from the state-wide DEM were extracted via an in-house developed Python script using a publicly available API called ‘public/ NSW_5M_Elevation’, hosted by DCS Spatial Services on the NSW Spatial Information Exchange (SIX) platform (DCS Spatial Services, 2022). To obtain a sufficiently large dataset for evaluation of the accuracy of the returned data, every survey mark in SCIMS (including witness marks, destroyed marks and interstate marks along the borders) was submitted to the API for a height retrieval. This included 301,200 survey marks at the time, using their horizontal Geocentric Datum of Australia 2020 (GDA2020; ICSM, 2021) positions for interrogation.
Due to the enormous number of queries required, and to prevent overload of the server, the submission data was separated into sets of 50 marks for asynchronous retrieval, with a wait timer introduced between sets. Retrieval was an iterative process, as failure rates for the server’s identify function were as high as 50% of the submitted set at times (likely caused in part by inferior internet connections while working from home during the COVID-19 pandemic). Failed retrievals were re-added to the submission set for the next iteration and automatically resubmitted until completion. This process took approximately 8 days of continuous processor time.
We found that 872 of the submitted positions returned no data, with 46 of these located on Lord Howe Island (which is not covered by this DEM nor true AHD). The remainder (apart from a few anomalies) were located along the Queensland, South Australian and Victorian borders. Closer inspection revealed that all these locations were outside the extent of the DEM (too far into the neighbouring states), noting that SCIMS includes several interstate survey marks close to the NSW border and that the Australian Capital Territory is entirely covered by the DEM.
Quality assessment of the DEM across NSW
The lower-accuracy AHD heights retrieved from the DEM were compared to existing high-accuracy AHD heights of survey marks on public record in SCIMS that satisfied the following criteria:
- Established (Class D or better) horizontal GDA2020 coordinates.
- Accurate (Class B/LD or better) AHD height.
- Mark at or near ground level (no towers, fence posts, pillars, cairns, reference trees etc.).
The prerequisite for established horizontal coordinates at each survey mark is equally as important as an accurate AHD height, as local terrain undulations can quickly alter the height returned from the DEM. Similarly, selecting marks that are not on the natural surface (above or below ground) renders the comparison invalid. In this instance, 34% of the survey marks in SCIMS (102,437 of 300,328 that returned heights) met the required criteria to be included in the comparison.
Figure 1 illustrates the distribution of these survey marks across the state, while Figure 2 shows those in Greater Sydney. The threshold for a successful comparison was set at ±0.9 m, which is the quoted vertical uncertainty of the DEM (95% CL). It was found that the calculated height differences between the DEM and SCIMS were within this threshold for 95,866 survey marks, i.e. 93.6% of the comparison set (indicated in blue in Figures 1 & 2).
The obtained pass rate of 93.6% is slightly lower than the quoted vertical uncertainty of the model (95%), which can be attributed to two main reasons:
- The comparison did not consider the vertical position of the survey mark above or below ground level (typically up to 0.2 m in either direction for an appropriately placed mark). This was deemed unnecessary as the result was fit for purpose and this information was only available for 3.7% of the comparison set.
- The horizontal density of the DEM is 5 m, so AHD heights at survey marks located on undulating terrain may show some discrepancy, depending on the position of the mark relative to the sample points of the model (which are used to interpolate the height at the specified position).
For at least the last five years, DCS Spatial Services has recorded the markto-ground-level offset at each survey mark occupied or inspected as part of normal field operations. To examine the effect of including such metadata, the analysis was repeated for those 3,849 survey marks in the comparison set with available mark-to-ground-level information. This smaller sample exhibited a pass rate of 97.1% when the mark-toground-level correction was applied and a 96.2% pass rate when it was ignored. Together with the earlier analysis, this result was deemed fit for purpose, confirming the stated DEM uncertainty.
It is worth noting that 960 (14.6%) of the 6,571 marks that failed to meet the ±0.9 m threshold (indicated in orange in Figures 1 & 2) are located within 20 m of the centreline of a major highway or motorway. This can be explained by rapid changes in topography often occurring along the cross-section of the road corridor, including embankments and cuttings. Furthermore, some of these roads are extremely steep, such as the Great Western Highway leading out of Sydney between Penrith and Glenbrook. Finally, 1,082 (16.5%) of all the marks failing to meet the threshold are specified as ‘destroyed’ in SCIMS, indicating that their AHD height may relate to a time prior to road or other construction earthworks altering the topography.
Comparison to an independent, national DEM
Following initial height retrieval, we noted that the DEM returned an AHD height that was significantly different (> 20 m, e.g. a typical contour) from the value in SCIMS for 787 survey marks with an existing Class U AHD height. This was investigated by querying Geoscience Australia’s national 1-second Shuttle Radar Topography Mission (SRTM) DEM for the entire dataset via another API (GA, 2022). We then compared the two DEMs to each other and to SCIMS. Wherever the NSW DEM value differed from SCIMS by more than 10 m (in 4,690 cases), the 3-way comparison was recorded.
Any AHD height difference exceeding 20 m between the two DEMs was then manually investigated, resulting in 45 of 108 marks (41.7%) to be identified for exclusion from the following SCIMS update. All these excluded marks were located where an open pit mine had subsequently been created. Any other large differences between the two DEMs were a result of the coarser resolution of the SRTM DEM (1 arcsecond equates to approximately 30 m), e.g. for Trigonometric Station (TS) pillars located on the side of a cliff, the SRTM DEM sometimes returned the height partway down the cliff. Most remaining large differences between the NSW DEM and SCIMS appeared to be the result of transcription errors in the SCIMS height (e.g. 1,000 m instead of 100 m) or rounding to the nearest contour when the heights were initially entered into SCIMS
Implementation of DEMsourced AHD heights in SCIMS
Reliable and quality-assured survey control is fundamental to ensure the integrity of the imagery and elevation products delivered by DCS Spatial Services. However, this connection can work both ways as these products can then be used to improve survey control information on public record in SCIMS. In this case, we updated SCIMS with suitable DEM-sourced AHD heights (at Class U) to yield a homogeneous dataset of known provenance and verifiable quality across NSW. This essentially improved approximate AHD height values in SCIMS that were initially obtained from the nearest contour on 1:25,000, 1:50,000 and 1:100,000 topographic maps to DEM-sourced values with sub-metre uncertainty. Displaying these AHD values to the nearest metre in SCIMS (Class U resolution) fits well with their 0.9 m uncertainty.
Noting the uncertainty of the state-wide DEM, survey marks were only selected for AHD height update if their existing AHD height in SCIMS was null or Class U. Furthermore, it is important to consider that SCIMS holds records for a wide range of different monument types. A location descriptor also indicates whether the mark was placed in the ground or on a structure. As such, further filtering was applied to limit the height update to only those marks that are likely to be at (or near) ground level.
Consequently, several monument types (Table 1) and mark location descriptions (Table 2) were excluded from the update.
Table 1: Monument types excluded from the SCIMS update
| AERIAL | FIRE TOWER | OBELISK | RESERVOIR |
| BEACON LIGHT | LIGHT HOUSE | RADAR TOWER | SPIRE |
| CHIMNEY | LIGHTING ROD | RADIO MAST | TOWER |
| FLAGSTAFF | MAST | RADIO TOWER | WIND VANE |
Table 2: Mark locations excluded from the SCIMS update.
| BUILDING OR STRUCTURE |
| SILO |
| OTHER STRUCTURE |
Table 4: Descriptive statistics of the difference between DEM-sourced and existing AHD height in SCIMS for the update dataset (723 outliers exceeding ±20 m were excluded from the calculation of the standard deviation).
| Minimum (m) | -964.725 |
| Maximum (m) | 1022.783 |
| Mean (m | 0.185 |
| Median (m) | 0.013 |
| Std Dev (m) | 4.228 |
As an additional precaution, any TS whose name included the word ‘TOWER’ was also excluded from the update. It should be noted that RESERVOIR OR TANK should also have been included on the list of mark locations to be excluded in Table 2 but was unfortunately missed. This resulted in 30 trig stations located on reservoirs incorrectly receiving a height at ground level. These will be revisited and corrected during the upcoming 6-monthly GDA2020 SCIMS refresh.
Once the update set had been filtered in this way and before the SCIMS update was executed, a final test was performed to check for trends. In general, we found that the data was normally distributed and 99.3% of the AHD heights included in the update were within 20 m of their existing SCIMS values (Figure 3). A 20 m error in height corresponds to approximately a 3 parts per million (ppm) error in the reduction of ground distances to the ellipsoid between two marks, which was deemed acceptable and fit for purpose.
As previously mentioned, height differences larger than 20 m were generally attributed to either of the following two reasons:
- Transcription or rounding errors in the existing SCIMS value (corrected by the update).
- Marks located where an open pit mine had subsequently been created (excluded from the update with mark status updated in SCIMS as ‘destroyed’).
SCIMS update results
This SCIMS update delivered DEMsourced AHD heights at Class U for 127,154 survey marks, of which 18,854 marks (14.8%) were assigned an AHD height for the first time (Figure 4). Putting this large number into perspective, this means that 42.3% of the survey marks that returned a DEM-sourced AHD height during the initial retrieval were updated during this process. This is a huge improvement in the access to reliable, approximate AHD heights of known quality in SCIMS, ensuring that nearly every survey mark in NSW has an AHD value of 0.9 m uncertainty or better.
When inspecting Figure 4, it is worth noting the near-perfect straight line of AHD height updates to survey marks running from north-west to south-east through the centre of the state. Despite appearing to be an artefact, this is a series of marks located along a gas pipeline easement, which connects to the main distribution network on the east coast. Table 4 summarises descriptive statistics related to the update dataset, showing the minimum, maximum, mean and median differences between the new and existing AHD height in SCIMS along with the resulting standard deviation. The existence of large outliers, as previously discussed, is confirmed by the difference between the mean and median values.
Benefits
These updated AHD heights provide several important benefits such as enabling better calculation and reporting of the Combined Scale Factor (CSF) with confidence at virtually all survey marks (99.98%) in NSW through SCIMS. CSFs are now typically up to 1.5 ppm better because heights have been improved from 10-metre to sub-metre accuracy. The DEM-sourced AHD heights also support datum modernisation efforts through the ongoing readjustment of legacy terrestrial data hosted by DCS Spatial Services for inclusion in the growing GDA2020 state adjustment by facilitating the rigorous reduction of terrestrially measured distances to the ellipsoid. This translates into more survey marks in SCIMS being assigned a Positional Uncertainty (PU), directly benefitting the profession and our customers.
Furthermore, this process was able to identify and correct 101 extremely large (100-1,000 m) AHD height errors (Class U) in SCIMS. It follows that retrieved DEM-sourced heights can now also be used to identify gross errors on SCIMS marks with existing accurate (Class B/LD or better) AHD heights. This further contributes to our ‘Saving AHD’ efforts, which aim to ensure that users have continued and easy access to reliable physical heights and their uncertainties across NSW (Janssen and McElroy, 2021b).
Finally, with SCIMS now holding AHD heights of known quality state-wide, ellipsoidal height was derived by applying AUSGeoid2020 (e.g. Brown et al., 2018; Janssen and Watson, 2018; Featherstone et al., 2019) at all applicable survey marks with existing null ellipsoidal height values in SCIMS. During the 6-monthly GDA2020 SCIMS refresh in November 2021, this provided ellipsoidal heights for 267,581 survey marks for the first time, ensuring that virtually all marks in SCIMS now also have an ellipsoidal height. Publishing these values allows surveyors and other users to easily verify that they have set their height datum and/or applied AUSGeoid2020 correctly during both field operations and office processing and reductions.
Conclusion
DCS Spatial Services provides a statewide DEM to the public with a vertical uncertainty of ±0.9 m (95% CL) and a horizontal grid density of 5 m. This DEM can now be queried directly through a new publicly available API to return an AHD height at a specified location. This paper has described how the new API was used to retrieve AHD heights from the elevation model for about 300,000 survey marks in SCIMS and assess the accuracy of these DEM-sourced AHD heights, thereby confirming the stated DEM uncertainty.
We have outlined how SCIMS was updated with DEM-sourced heights (at Class U) for 127,154 survey marks withexisting Class U or null AHD height values to deliver a homogeneous dataset of known provenance and verifiable quality across NSW. This allowed 18,854 survey marks to be assigned an AHD height for the first time, while many gross AHD height errors in SCIMS were identified and corrected.
The updated AHD heights provide important benefits for industry such as enabling the calculation and reporting through SCIMS of the CSF with confidence across all of NSW and supporting the readjustment of legacy terrestrial data in the growing GDA2020 state adjustment. Lastly, ellipsoidal height values were derived using AUSGeoid2020 and published in SCIMS, allowing 267,581 survey marks to receive an ellipsoidal height for the first time. As a result, virtually all SCIMS marks now have both an AHD height and ellipsoidal height.
References
Brown N.J., McCubbine J.C., Featherstone W.E., Gowans N., Woods A. and Baran I. (2018) AUSGeoid2020 combined gravimetric-geometric model: Location-specific uncertainties and baseline-lengthdependent error decorrelation, Journal of Geodesy, 92(12), 1439-1456 & 1467.
DCS Spatial Services (2022) public/ NSW_5M_Elevation, API to retrieve NSW 5 m elevation data derived from stereo imagery, https://maps.six.nsw.gov.au/arcgis/ rest/services/public/NSW_5M_ Elevation/ImageServer/ (accessed Aug 2022).
Featherstone W.E., McCubbine J.C., Claessens S.J., Belton D. and Brown N.J. (2019) Using AUSGeoid2020 and its error grids in surveying computations, Journal of Spatial Science, 64(3), 363-380.
GA (2022) DEM_SRTM_1Second, API to retrieve elevation data from the national 1-second Shuttle Radar Topography Mission (SRTM) DEM, https:// services.ga.gov.au/site_9/rest/ services/DEM_SRTM_1Second/ MapServer/ (accessed Aug 2022).
ICSM (2021) Geocentric Datum of Australia 2020 technical manual, version 1.7, https:// www.icsm.gov.au/gda2020- and-gda94-technical-manuals (accessed Aug 2022).
ICSM (2022) Elvis – Elevation and depth – foundation spatial data, https://elevation.fsdf.org. au/ (accessed Aug 2022).
Janssen V. and McElroy S. (2021a) Australian Height Datum: Celebrating 50 years, Coordinates, 17(5), 13-19.
Janssen V. and McElroy S. (2021b) Australian Height Datum: Saving AHD in New South Wales, Coordinates, 17(6), 6-10.
Janssen V. and Watson T. (2018) Evaluating the performance of AUSGeoid2020 in NSW, Proceedings of Association of Public Authority Surveyors Conference (APAS2018), Jindabyne, Australia, 9-11 April, 3-18.
Powell L. (2017) Survey control and quality assurance for aerial imagery and elevation models across NSW, Proceedings of Association of Public Authority Surveyors Conference (APAS2017), Shoal Bay, Australia, 20-22 March, 32-47.
Roelse A., Granger H.W. and Graham J.W. (1971) The adjustment of the Australian levelling survey 1970-1971, Technical Report 12, Division of National Mapping, Canberra, Australia, 81pp.
(No Ratings Yet)
Loading...
Leave your response!