Managing inaccurate historical survey records in a future accurate digital world

We start a series of two papers, first being the underlying survey database technology and workflows and the second being how the process is facilitating the representation of the survey plan in a machine readable file that is allowing automation of manual plan examination processes and transactions. We present here the first paper. The second paper would be published in next issue

Ian Harper Director, Geodata Australia, Australia

Spatial definition of land ownership

The physical extents of land ownership have been defined by many methods over the years ranging from verbal agreements between adjoining owners to formats of written descriptions, maps or survey records. These informal and formal tenure definitions have then been managed at various local, regional or State levels of registries by manual methods. Technology is driving the agenda to digital land administration systems. The starting spatial representation will usually be digitised from indicative maps of ownership. These are adequate where resources are limited but where survey records of property ownership existed, those digital maps will gain a higher level of acceptance where there is a recognisable link in the system to the historical survey measurement and cadastral intelligence.

The challenge is to represent the intent of those inaccurate historical records in a future digital administration that demands specific structure in digital data formats.

Future digital efficiencies

The digital database world offers powerful tools that can require significant technical and commercial resources. The ‘big picture’ type of outcome indicated (Diagram 1) belies the amount of data, governance and management resources required for their creation.

Technology is providing greater capabilities in measurement tools – GNSS, Scanning, Lidar, terrestrial and aerial imagery, Unmanned Aerial Vehicles (UAV) or drones. These tools also raise the level of expectation in outcomes.

Some of these technologies could assist in providing cost effective mapping solutions but others require considerable commercial and technical resources to manage the data.

Computing technologies have also advanced with significant increases in computational and data storage capacity. Those advances underpin the automation of existing manual processes that allow efficiency in accessing, manipulation and examining data. These processes have rigid governance systems with standard data formats but do provide significant efficiencies.

If digital cadastral definition was straight forward, it would have been implemented by now. To get to a future simplified outcome, the spatial and legal issues are considerable. We are moving towards it and other aspirations like 3D cadastral modelling and boundary definition by coordinates are on the agenda. The concept is achievable but the devil is in the detail and a starting agenda would be to get a cadastral management process in place at a local level that replicates a State system. Where the spatial component is managed is then optional.

A big issue is the perception and expectation that technology will solve all the problems. The tools of the future will probably do that, but we have a unique set of circumstances in the cadastral definition domain to manage as part of the transition. It will also be important to recognise there may be better ways of doing things that are difficult to grasp in our minds that have been moulded in a manual world.

The technology transitions

When digital systems are being implemented there are two main transitions that can occur in representing boundaries. The manual to digital transition is the initial one, as any system needs to create a spatial representation of the cadastral boundaries in a digital format. All types of agreements, descriptive documents, maps and surveys need to be identified and represented for transaction requirements.

Accuracy is not critical in the first transition but as systems develop in complexity and look to represent and manage other information (political, environmental, infrastructure, assets etc) it increasingly becomes a necessity. Thus there is a need for a second transition from a mapping solution to a survey solution. Higher accuracy also invokes a new level of consideration of issues associated with measurements and coordinates in a dynamic geodetic system.

Manual to digital transition

Where no documented records exist such as verbal agreements between owners or areas of previously State owned property, decisions on boundary locations must be determined. Ideally, monuments are physically placed in agreed locations and then surveyed. Those survey records then allow accurate reinstatement at any time. Broader scale indicative maps of ownership can be scanned to provide a digital representation for administration purposes and can be linked to aerial imagery to enhance location accuracy.

These methods service strategic land administration and are useful when resources are limited but reinstatement may face disputes.

Historical survey records represent the local relationships between adjoining properties and are interpreted by Licensed Surveyors to determine an accurate boundary location on the ground. Those records can be difficult to represent accurately in a large database and can require significant resources to consider them on their exact spatial intent. This is not justified in initial land administration databases.

Diagram 2 outlines the processes in the Manual to Digital transition. Technology now provides the tools to generate a useful mapping boundary database and even though survey records exist, a representative database can be created without reference to the detail on those records. Survey field work or computations may be useful but not vital.

Mapping to survey transition

The mapping to survey transition is a consideration when stakeholders are finding that the spatial precision of an initial mapping land administration database is not meeting their business needs. This has been happening for some time as Geographic Information Systems (GIS) database technology now provide powerful tools to manage governance and infrastructure.

The complexity has grown in those tools but the outcomes are still dependent on the quality of the information within them and there has been increasing investment in upgrading the accuracy of the cadastral database to overcome those inadequacies. A combination of accurate survey data and survey data management tools are required.

Existing survey measurement records do not ‘fit’ easily into a structured digital environment that represents the real world cadastre. Diagram 3 is an example from Australia that indicates how poorly an historical survey record that depicts a current legal land title represents the true ‘on ground’ dimensions of the property. Simply adopting all the legal survey records and joining them together is not an effective solution.

Three Plenary Speakers at the Map Asia Conference – August 2008 spoke about the future and those comments are noted in Diagram 4. The points they make are the importance of the cadastre and accuracy, and that the rigour of survey processes to check and validate data is needed rather than GIS or mapping processes that generate indicative maps with a minimum of effort. These digital maps are only suitable for ‘big picture’ jurisdictional governance.

Systems for the future

Technology now provides the capacity for accuracy in land administration spatial systems but there are issues in pursuing that goal:

– A significant investment if pursuing spatial quality in the short term

– Implementation strategies – Governance & Integration of data and resources

– Stakeholder perceptions and expectations

– A high level of detail required across a significant dataset

Even in developed countries with extensive systems in place, these issues make it difficult for large scale SDI mapping systems to be spatially accurate at a local level, so many local authorities or utilities create their own systems to meet their operational needs, even though it is duplicating the same database managed by the State.

The Survey Database (SD) technology outlined below provides the same workflows and processes at both local and State level so with good management, similar duplication of resources and databases can be avoided.

The SD is a survey database of dimensions. Representing and computing boundaries in the past has always been measurement based, but with digital technology, storage of spatial information is coordinate based.

Whilst the general acceptance of coordinate point definition is increasing, most users outside the survey and GIS worlds relate to measurements, so cadastral boundaries are best represented by measurements, but must be digitally stored as coordinates in the database.

The survey database technology

The Survey Database (SD) technology was developed in Australia to facilitate the transition to a survey database to generate a higher level of accuracy and functionality underpinning the cadastral database. The initial mapping databases had been generated by digitising administrative maps and as greater accuracy in those administrative databases was being pursued, the process was developed to integrate the historical survey records with new accurate survey data, partly automating the intuitive processes that a Licensed/ Registered Land Surveyor would follow in determining the location of a boundary.

The SD Solution at a State level is available around the world as the ESRI Parcel Editor tool in the ArcGIS enterprise system and at a local level, the GeoCadastre (GC) technology is a standalone software solution capable of being used in the field or remote locations with the most basic computer resources.

The SD format is standard at all levels and ground measurements are the common unit used in computations, so all measurement data (compass traverse/pacing, theodolites/band/ chain & EDM) can be utilised with GNSS or other modern position based technology. It means the most basic field survey information as well as modern survey data contributes to a geodetic or planar spatial outcome on the spatial merits of the information.

Measurement data can be sourced from manual and digital systems that could include:

– Manually entering the measurements from record documents

– Importing new digital survey data outcomes from survey cogo software.

– Creating parcel measurements from existing mapping database coordinates

– Generating measurements from sets of GNSS coordinates.

When joined, the geometry of all the property parcels in the database creates a mesh or continuous ‘fabric’ that is adjusted to nominated Survey Control points. All the original survey dimensions and control points are stored in the main SD.

Diagram 5 extends the database creation workflows to now include the progression from Mapping Processes to Survey Processes. The measurements from the survey records are input directly into the SD technology and issues such as poor quality plans and the topology or ‘fit’ of the cadastral fabric area are managed and adjusted to achieve the highest spatial outcomes without considerable field survey that would normally be required.

The diagram identifies that the SD processes provide greater location accuracy and database management efficiencies and if field survey is utilised to increase accuracy, those costs would be considerably greater without the SD technology. In traditional databases, the coordinates of boundary corners are the defining spatial attributes of a parcel. In the SD, the original measurements (irrespective of accuracy) as noted on the legal survey document are be the defining attributes. The coordinates of a parcel in a dynamic SD model are an informed (or otherwise) estimate of the spatial location of a corner at a specific time.

Survey database workflows

Data Entry

Measurement data can be sourced by various methods as outlined above. The manual data entry of measurements from survey records achieves the highest outcome but is the most labour intensive. Once that entry is complete it is stored in the SD and no further manual data entry is required.

Miscloses remain in the parcels as basic adjustments (Bowditch, etc) distribute an error across all lines where it may only be in one line. Redundancies in the SD Least Squares Adjustment (LSA) can identify the specific line. Variation in azimuths between survey plans or scale corrections are also automatically managed in the SD process.

Parcel Joining or Topology Management

The parcels are joined to a fabric ensuring the correct topology with no overlaps or gaps. As the adjustment is a survey based process across a geometry mesh, survey measurement connectivity is required throughout any dataset. The topology or ‘fit’ of the cadastre parcels is important to ensuring the adjustment will operate.

Adjusting the database

Weighting based on the estimates of measurement quality is applied to each measurement on a per plan/survey basis (i.e. all lines on a specific plan are usually equally weighted). The adjustment is designed for cadastral survey data which is quite different to the type of rigorous measurements found in a geodetic survey adjustment. Error ellipses are valuable in geodetic adjustments but in the SD LSA, they are generally not a true indicator of precision. The only true test of the spatial precision of a point is a comparison between the SD coordinate and the coordinate of a boundary location determined in the field by a Licensed Surveyor after consideration of all the evidence.

Survey database resource allocations

Basic data entry requires the least amount of experience and the level of comparative resources required diminishes through stages 2 and 3 but the level of experience required grows as the need to troubleshoot and resolve survey data issues grows. Field survey greatly assists the quality of the outcome in the adjustment and the adjustment is used as a tool to direct operators to areas of problem data where field survey can be focused.

The various stages can be assigned to different levels of experience, but they can all be managed at a local jurisdiction. Each stage has levels of data checking and validation that can be invoked if a higher spatial outcome is being sought. The local SD output can then be incorporated into a central SDI and should not require further adjustment.

Diagram 6 highlights a cadastral upgrade business model where there is a differentiation of data management tasks between the Local and State jurisdictions and identifies the SD tools for each jurisdiction.

There is a significant difference in the application of the technology between the State and Local Jurisdictions. At a State level, the business needs and outcomes are strategic and the level of detail and accuracy is not critical. At a local level, the detail and accuracy becomes more important as a database map becomes a tool that provides information as to the approximate location of property boundaries on the ground and flows through to the representation of above and below ground assets or identifying political, environmental or other constraints.

The SD is firstly the repository of survey measurement data, survey connectivity and control surveys but then becomes the computational tool in modelling the complete cadastre. Once local computations are undertaken, the output to the GIS Land Administration database is only the model of the cadastre boundaries. All the original survey data remains in the SD where it can only be accessed by authorised resources when further upgrading is required.

Survey Database – Business Case

There are only a few States (or Countries) where coordinates are the point of legal truth in defining property boundaries, thus in most developed countries, historical survey records are still the prima facie evidence to determining the location of cadastral boundaries.

There are two ends of the business strategy in creating a SD to expedite the mapping to survey database transition:

1. Reverse engineering existing mapping databases to the SD data structure. This retains the existing spatial quality of the model by adopting the original measurements from the mapping database, but facilitates efficient upgrading with survey plan data as required. There is limited initial migration costs and spatial upgrading can undertaken as needs and budgets allow. However, as this method progresses, the rationalisation of new survey data and original poor quality digitised data can be a timely process

2. Entering and joining all the survey measurement and other cadastral intelligence available from current and historical survey records and applying all the rigour of the process in conjunction with a GNSS field survey of some cadastral monuments – this strategy involves a reasonable cost ‘up-front’, but those costs are amortised rapidly due to the savings in general efficiencies and having one accurate system (i.e. not duplicated).

Over time the cost of Option 1 will be significantly greater than Option 2. The difficult part is to identify the real cost burdens of how an inaccurate cadastral database can affect the efficiency and quality of operation. The accidental severing of an optic fibre cable can cost millions of dollars per minute in down time. If there is a high level of uncertainty in the State based cadastral model, it may be initially be part of the problem or it may put in doubt any opportunity to apportion liability to the perpetrator.

There is a considerable amount of scope between the two strategies outlined above. Our experience in creating SDs has proven that a flexible strategy with consideration of survey data quality and stakeholder needs being the determining factors in a practical and cost effective solution. This may involve applying Option 2 in urban areas or where specific infrastructure projects are being undertaken and applying Option 1 in rural or remote areas where the quality of survey data is poor. One strategy across a whole jurisdiction is usually not an effective solution.

Survey database outcomes – Technical

There should be a State approach to statutory governance, standard data structures and workflow guidelines however local management will have a greater understanding of the survey and mapping data and should find it easier to resolve boundary or survey uncertainties.

The basic workflows of data collection & compilation can be undertaken at a local level. The full rigour of the process is available in the local tools but limited experienced resources at that level may mean that the complete checking and validation of survey data may not occur, however, as that experience grows a higher level of data validation can be done locally.

Local Authorities will be micro managing the data that feeds into a macro State Land Administration system. Local data management systems can be initially set up independently so that they have the capacity to feed into a State system that still may be in its formative stages.

Survey database outcomes – Social / Professional

A high level of local SD management will flow through to a greater sense of ownership and pride in the data.

In Developing States, up-skilling local resources in basic measurement and computations will also boost employment in areas away from capital Cities and other main Government Departments. In those jurisdictions local manpower resources are usually much more accessible than commercial technology solutions.

If government resources permit, engaging the local survey profession has a considerable upside. It would firstly provide skilled resources and secondly provide a strong connection between the two sectors. This was evident in the Northern Territory (Australia) where the wide involvement of the private survey practitioners meant they better understood the new (and different) digital workflows and thus supported the government department in its digital initiatives.

Survey Database versus Survey Definition

The SD is the dynamic integration of many individual surveys, possibly thousands and will not produce the same outcome as a survey definition outcome represented by a survey plan.

An individual survey identifies the immediate relationship between monuments and other survey information. New surveys have a stronger weighting than older surveys and the adjustment will endeavour to maintain the geometry relationship identified in the newer survey, however the constraints of a significant number of adjoining surveys will generally slightly vary the new survey.

The only way to hold a specific survey fixed in the adjustment is to nominate a significant number of the points in the survey as control points and hold the coordinates of those points ‘fixed’ in the adjustment.

In the same way a surveyor makes an intuitive decision to adopt certain monuments and measurements, the surveyor will identify the points to hold fixed and the coordinates to adopt.

As modern surveys are gradually added to an SD, the number of points accurately surveyed on the ground will increase and the model representation will become closer to the true location of the boundary.

This is the progression that will eventually lead to consideration of the SD as having a higher weighting in the chain of evidence in survey definition and when parts of the SD are deemed to be of the highest spatial precision, the coordinates representing boundaries could be considered for legal definition. There are many aspects of governance to be considered before this will occur.

Alignment definition in a Digital Environment.

Representing the definition of a road or other alignments defined in historical survey plans is something that also needs care in the transition to a digital database.

A distant occupation or alignment stone provides evidence of a straight alignment extending beyond the immediate surveyed parcels but that alignment point must be referenced by a coordinated point in a database. This can be misleading in that the coordinated point only represents a direction rather than a specific point in the cadastre.

The true digital representations of alignments should only be utilised when two alignments can intersect to identify an angle or corner in the cadastre. The SD accommodates this by having a unique data feature called a LINE POINT that lies along a boundary or alignment. The line point is not classed as a node in the adjustment so that boundary or alignment is kept as a straight line and in the adjustment does not generate a slight angle at that point.

Adjustment of Associated Feature Datasets.

As a Land Administration SDI moves toward greater accuracy, greater importance is placed on the spatial accuracy of all data. The relationship between infrastructure and other layers was historically referenced to the cadastral layer and by spatially upgrading only the cadastre, that relationship changed. This can be a major issue as stakeholders become more dependent on the Land Administration database.

At the regional and State Level the ESRI Parcel Editor SD tool has the functionality to automatically re-align associated feature datasets as adjustments are made to the cadastral layer, however at some point in the creation of the SD the true relationship must be generated to start that automation process.

That initial synchronization of the cadastre and associated layers requires a strategy involving resources that understand the data and the history as to how it was entered into the system.

Case studies

Several States and many local authorities in Australia use the GeoCadastre tool as a data compilation tool to feed historical survey plan data into their ESRI Parcel Fabric Spatial Data Infrastructure or other industry standard database technologies.

In NSW the technology underpins the automation of the lodgement and quality examination of digital survey plan files creating new titles. A LandXML survey plan file is lodged at the titles office portal and it immediately undergoes many tests of the plan’s jurisdictional information and internal geometry quality. This is followed by an automated spatial precision validation process.

In some cases this has resulted in a reduction of the time taken to register a new survey plan from 1-2 months under the existing manual workflows to 3-5 days. Economic benefits then apply to all stakeholders, the state requires less staff resources, receives property based revenue sooner and those looking to create new titles have their development holding costs reduced and their revenue returns are also faster.

In the Northern Territory an operational SD is close to being completely populated with historical survey records. Legislation is in place to allow coordinates to be the prima face survey evidence to legally define Land Titles. This will only happen in areas where the Surveyor General is satisfied that the SD coordinates are of sufficient spatial precision for that purpose.

The Hunter Water Corporation is a local Water & Sewer utility that has been manually entering survey plans into GC for over 15 years. They now have an expansive accurate SD cadastre aligning with accurate location of all their assets. They have recently implemented an ESRI based SDI to allow a higher level of management of their assets and cadastral database.

References

Dr Michael Elfick, Wallalong, NSW, AUSTRALIA – elfick@geodata.com.au

Tony Sleigh Emergency Information Coordination Unit Department of Lands NSW