3D mapping from space?

Prof Dr Armin Gruen, Dr Kirsten Wolff

Lately we observe an amazing increase in earth-observation platforms equipped with ultra high-resolution imagers. With the recent deployment of WorldView-1 we have reached the 0.5 m footprint level.
This raises the issue of 3D topo-mapping from space, in a more pressing way than ever before. Topo-maps of medium and larger scales (1:50,000 and better) are still missing in some parts of the world, in others they are hopelessly outdated. Upto- date aerial images, as a traditional data source for mapping, are not always and everywhere available. In contrast, highresolution satellite images with stereo capabilities constitute an interesting tool for mapping and the image providers advertise their use quite extensively.

Topo-mapping is worldwide controlled by specifications, which may differ from country to country. Therefore it is difficult to give general recommendations with respect to the question which sensor would be feasible for which map scale. In addition, digital mapping is largely scale-free, which makes the issue even more controversial.

In the literature we find many predictions and recommendations on 3D mapping from space, but mostly without substantial empirical evidence. 3D mapping is very often reduced either to the generation of ortho-images or/and to the georeferencing accuracy and DTM generation accuracy. But mapping is much more, as we all know. 3D mapping from satellite imagery is still a topic which causes many misconceptions. We hope we can contribute with this paper to a clarification of some of the issues.

What is 3D mapping?

A consistent definition of 3D mapping is missing. We are well used to conventional 2D and 2.5D mapping, resulting in an analogue map as final product. 3D mapping however gives us many more options, but also raises more questions.

With such new technology of digital mapping we have to address a number of problems, which are not necessarily all new, but so far only sparsely treated in R&D. Among those are:
+ 3D mapping – how does this differ from traditional 2D and 2.5D mapping?
+ Which objects have to be mapped and at which resolution and accuracy?
+ How should truly 3D objects be modelled in terms of geometry, topology and possibly also texture?
+ How should these objects be represented in the database?
+ Digital mapping – how much automation is currently possible?
+ Image interpretation – which pixel size do we need in order to bee able to extract features and objects that are required for topomapping
at a certain scale?
+ Image quality – what are the differences in image quality (and thus interpretability) between aerial and satellite images of the same Ground Sampling Distance (GSD)?
+ Orientation/geo-referencing – how accurately can we georeference the new satellite images (with and without GCPs) in planimetry and height?
+ DSM generation – what are the expected accuracies in automated DSM generation, which parameters determine the accuracies of the DSMs and what is the reliability of the estimated surface models?
+ DSM-DTM reduction – what are the most successful approaches for
DSM to DTM reduction and what are the main problems to be solved?

3D mapping requires totally new approaches to modelling. Most of the
traditional procedures and commercial software packages, which have been developed under 2.5D assumptions will inevitably fail under strict 3D requirements.

Prof Dr Armin Gruen, Dr Kirsten Wolff

From 2d maps to 3d landscape models

Worldwide databases like Google Earth have given a wider audience access to georeferenced data. This data is useful for all kind of planning and inspection purposes, but usually does not satisfy the demands of a professional map user. However, it has sharpened the mind for what can be done with geo-data, in terms of visualization and interactivity. And it has set a certain standard in perception of terrain and other geo-related data. Has Google Earth even changed irreversibly our perception of a map? If yes, then what do we, as experts or occasional users, expect form modern maps after Google Earth?

Mapping has grown way beyond the traditional domain of topo-mapping and thematic mapping. Markets for geospatial technologies include nowadays applications in insurance and risk management, natural and man-made hazards, real estate, Location-based Services, environmental monitoring, car navigation, oil and gas, homeland security and many more. However, the data contained in topo-maps (or “landscape models”) can in the future serve as the backbone of these diverse applications.

The flexibility in handling of and in modifying digital data has brought up the issues of “real-time mapping” (see recent disasters in Myanmar and China), “mapping on demand” (individualized data collection), “personal mapping” (representations for a particular purpose or person). Mobile Mapping platforms on cars, trains and UAVs allow for real-time raw data collection at an unprecedented speed and flexibility. New devices like mobile phones and PDAs in LBS-related applications bring up the notion of “ubiquitous mapping”.

how does this all relate to the task of topo-mapping?

There is a large amount of data already now available which would qualify for “3D maps”: 3D city models, forestry models, Cultural Heritage models, etc. Should this data form a joint database with the more conventional content of a traditional topographic map?

This all requires us to reconsider radically our mapping goals, tasks, procedures and products.

First steps in this direction have been taken. The Swiss Department of Lands (“swisstopo”) has defined a countrywide map system on the basis of a Topographical Landscape Model (TLM). This Landscape Model includes all objects that are currently represented in topo-maps, but in truly 3D form. This includes terrain, buildings, water objects, public transportation, public spaces and facilities, landcover, administrative borders and “points of interest”. Compared to the situation before this model features some novelties, as for instance – It serves as the basemodel for the whole country. Subsequent level models for cartography and for representation (at varying scales) are defined, which are derived from this unique dataset of the basemodel. Therefore all the objects of the basemodel are geometrically correctly modelled. There are no displacements of elements and no generalizations.
– The data is always actual. It is continuously updated, and not only at certain fixed intervals.
– The accuracy is very high. The object accuracy is specified to 1 m.
– The data is truly 3D. Therefore all objects can be correctly modelled and no information has to be suppressed because of lack of modelling tools.
– The data model is set up such that it is ready for extensions.

At this point the data is still acquired from aerial images, as of January 2008 only from digital cameras like the Leica ADS40.

This Landscape Model is interesting from a conceptional point of view. It remains to be seen how it performs in practice.

3d landscape Models from aerials or satellite images?

With the increased availability of ultra high-resolution satellite images and (partially) dropping prices this becomes a burning question. Both data sources do have distinct advantages and disadvantages, which are briefly listed here:
Pro satellite images:
• The satellite platform is operational 365 days of the year
• Frequent re-visit times (e.g. every 4 days or even more)
• Imagery is post-processed relatively quickly
• There are no Air Traffic Control restrictions
• Large area footprints decrease the need for block adjustment and creation of image mosaics
• The satellite can easily access remote or restricted areas
• No aircraft, cameras or otherwise expensive equipments are required (by the end user)

Contra satellite images:
• The image acquisition geometry is not flexible
• The image resolution is fixed for a particular sensor and low compared to most aerial imagery
• The radiometric resolution is often too low (problems in shadows and saturation areas)
• The image quality is often impaired by different factors and artifacts
• The typical off-nadir viewing angle of up to 25˚ is problematic in image matching
• The reliability of capture and delivery of imagery can be poor at times
• Strong possibility of cloud cover and thus occlusions
• The cost of the imagery may be too high (when compared to aerials)
The selection of any one of the data sources depends on many factors. The decision can only be made efficiently when all the project parameters are available. We have reported about an extreme case in Bhutan (Fraser et al.,2008), where access to aerial images is impaired and where pilot projects are underway to use satellite imagery for the
generation of a new topo-map 1:25,000.

In our following pilot test for topomapping we compared map objects derived from IKONOS 1m GSD stereo images with map data from the Swiss topo-map 1:25,000, which is usually derived form aerial images at scale 1:30,000 (which in turn corresponds to an image pixel size of about 0.5 m).

Prof Dr Armin Gruen, Dr Kirsten Wolff

Geo-referencing

Today geo-referencing from satellite images is well understood and controlled. It is the least problem we encounter in 3D topo-mapping. In previous projects we have collected a lot of experiences in geo-referencing. We have used SPOT-5, ALOS/PRISM, Cartosat-1, IKONOS and Quickbird images over different testfields worldwide (Germany, Italy, Japan, South Africa, Switzerland, Turkey, Vietnam). We have developed the software SAT-PP (Satellite Image Precision Processing), which includes several strict models for the most important sensors and also the related Rational Polynomial Function (RPF) approaches. With this software we have obtained consistent results in the subpixel domain, both for planimetry and height and for all sensors, using few (2-5) GCPs only. We could show that RPCs usually provide good relative orientation, while the absolute orientation has substantial systematic errors. These kinds of errors depend on the satellite/sensor. In the best case they represent just a bias (shift in coordinates), in other cases we diagnosed higher order terms. In any case the distortions can be removed with the concept of biascorrected RPCs and the use of 1-3 GCPs.

DTM generation

DTM generation is a key issue in topomapping. If produced in manual mode this does not constitute a problem, it only needs time – a lot of time. Therefore we turn towards automated DSM generation by image matching. Image matching – in its essence – is still an unsolved problem. With our software SAT-PP, which includes an advanced matching module, we obtain height accuracies between 1 and 5 pixels from high-resolution satellite images, depending on the type of terrain, land cover, image texture and image quality. While RMS errors in such tests show good results we must note that in all these cases substantial blunders (10 times the RMSE and more) still exist in the data. This is not acceptable. This can only be solved by substantial and time-consuming postediting of the DSM or by efforts to better understand the reasons for such blunders in image matching, with the aim to get rid of them. Therefore the avoidance and/or detection of blunders in the automatically generated DSM is a critical point for future research and development.

The next problem we are faced with is the reduction of the DSM, produced by the image matcher, to the DTM, as represented in the landscape model. Although there are some attempts available to automatically perform the reduction, the results are not convincing, because these algorithms are purely based on geometrical considerations. What is needed however is an image or point cloud interpretation approach which lets us understand what kind of object we are dealing with in the reduction process at a particular location.

Object extraction

We have experience with automated and semi-automated feature and object extraction, primarily in 3D city modelling and 3D road extraction. In 3D city modelling we use our semiautomated procedure CyberCity Modeler (CC-Modeler) for building extraction. With some examples derived from IKONOS and Quickbird images we could show to what extent and at which resolution these objects can be modelled from satellite imagery.

In road extraction we have developed “LSB-Snakes” (Least Squares B-Spline Snakes), a semi-automated technique which allows us to model roads in 3D.

In addition, the well-known technique of monoplotting can be used for object extraction (Fraser et al., 2008). This procedure works usually well, but with limited accuracy, depending on the quality of the underlying DTM.

In the following test the measurements of the topographic features (buildings, forests, streets, lakes, single trees and contour lines) for the map scale 1:25,000 were done by an experienced stereo operator of our group. For the other special topographic features we got support from an experienced topographer from swisstopo.

Pilot mapping project Thun

A key issue in mapping is the interpretability of images of a particular resolution. Currently this topic is in the center of our interest, because we believe there are misconceptions on this issue. We are conducting investigations to find out which objects can be extracted under which geometrical resolutions. Here we present some preliminary results. For more details please see Gruen and Wolff, 2008.

For a first test we selected the test area of Thun, Switzerland. This is a fairly flat urban zone which is composed of areas with single family and apartment houses with parks, an industrial area, a military airport and forest areas. This area contains many of the important features of a topographic map. For the manual drawing of contour lines we extended the area to a hilly region, including forest and open areas without any substantial buildings.

The aim of this investigation was to analyze the possibilities to identify and map buildings, roads and other individual features for a 1:25,000 topographic map by using high resolution satellite images data. For such a mapping scale we assumed that a GSD of 1m or even higher is required. In conventional mapping and map updating aerial images of scale 1:30,000 are used (which corresponds to a GSD of about 0.5 m). For our test area Thun two IKONOS panchromatic stereo images (December 2003, GSD 1m) were available.

Prof Dr Armin Gruen, Dr Kirsten Wolff

(a) Reference data for mapping
As reference data we used a digital version of the national topographic map 1:25,000 (Pixelmap) and the vectorized map (VECTOR 25) of this area, both products of the Swiss Federal Office of Topography. The planimetric accuracy is given as 3-8 m (corresponding to the map accuracy), which is by far worse than the geo-referencing and object measurement accuracy in IKONOS imagery. For quality control of manually measured contour lines we digitized the 10 m contour lines of the map.

Figure 1. Comparison of the 10 m reference contour lines (black lines) and the manually drawn contour lines (dashed lines).Left: open area; middle: steep forest area with two small creeks; right: mixed area

(b) Manual drawing of contour lines
10 m contour lines were measured in an area of 4 x 5.5 sqkm by an experienced stereo operator of our group using Stereo Analyst, a 3D feature extraction tool of the ERDAS IMAGING system. For the visual quality evaluation we defined three sub-areas with special land use patterns (Figure 1: open area (left), forest (middle) and mixed flat area (right)). As expected, we got the best results for an open area with only
a few houses and less good results in the steeper woodland. The IKONOSderived contours resulted in a distinctive smoothing effect of the smaller geomorphological features. Therefore the problem is not so much the limited metric accuracy of the IKONOS-derived contours (the reference contours also had only an accuracy of 3-4 m), but the loss of geo-morphological detail. This of course is also caused by the fact that trees and bushes are restricting an accurate interpretation of the scene.

(c) Object extraction
We used the Stereo Analyst also for the extraction of all map features. The identification and mapping of buildings and roads/streets was done by our own stereo operator. But for the classification of roads/streets and for the identification of several individual features special knowledge of an experienced topographer is required. Here we received the support from our Federal Office of Topography (swisstopo) in form of one professional operator, familiar with the map legend of the Swiss topo-maps.

Both operators did not have any preinformation about the test area and did not use the reference data as preinformation for the measurements.

Figure 2 gives an overview of the measured buildings, roads/streets, railway, small airport, forests and single trees of the test area.

Figure 2: Overview of the measured buildings, roads/streets, railway, small airport, forests and single trees of the test area Thun.

Figure 3: Overview of a sub-area of the urban test area. Filled colour:
VECTO R 25, contours: IKONOS – extracted houses.

(c1) Identification and mapping of buildings
In the map buildings are presented by their footprints. Our operator measured the buildings in 3D, but for comparison only the 2D footprint of the roofs could be used. For a more detailed analysis we sub-divided the whole area in two kinds of sub-areas: industrial (160 buildings) and residential areas (165 buildings). For quality control we compared the extracted shapes of the houses with the VECTOR 25 data set visually. The main focus was on the identification of houses and their shape and less on the metric analysis of their correct position. The results are
classified into the following 6 categories:
Equal: shape and position of the houses considered as equal
Partial loss: parts of the houses are missing, same position
Total loss: the house could not be extracted
Forest: the house is covered by trees
Different position: same shape, different position
Improvement: the extracted shape or position is better.

In case of differences in position of an individual building it can be assumed that the IKONOS mapping gives better planimetric accuracy than the given, cartographically modified (generalized and shifted) building data (see Figure 3).

Table 1 gives an overview of the results of the visual quality control of the extracted buildings. For a fully detailed analysis of the differences, an inspection trip into the field would be necessary. The results clearly show that the object identification from IKONOS imagery is not reliable enough.

The approximate mapping of the centerline of the roads was done by our operator without cartographic experience. In urban areas it was more difficult to measure the centerline than outside the town. Without an in-depth background in national mapping it was not possible for her to classify the streets in relation to the official classes of the national map. Some parts of smaller streets were covered by high buildings and trees. In such areas the interpreter tried to guess the run of the streets by analyzing the surrounding structure of the houses and trees. Altogether 163 street segments were analyzed. 11 (7%) of them could not be detected by our operator.

The classification of the streets and roads was investigated by two cartographic specialists. The main criterion for the classification was the width of the streets. We have 6 classes of roads: R1: >6m, R2: 4.2 – 6m, R3: 3 – 4.2m, R4: <4m, R5: dirt roads and R6: trails. Because of the limiting 1m resolution of the used images, it is very often not possible to distinguish between two neighboring classes. The images were taken in December, a period when dirty roads and trails are not used regularly and therefore it was difficult to identify them in several cases. The highway, railway rails and traffic circles could be easily identified and measured.

In future, additional features like pavements will be taken into account by swisstopo for a detailed classification. Such small details cannot be extracted from 1m resolution satellite images any more.

Prof Dr Armin Gruen, Dr Kirsten Wolff

(c3) Identification and mapping of individual map features
For the evaluation of individual map features like forests, bridges, airports, power lines, churches etc. the two cartographic operators used the whole imaged area of Thun. Some features like forests, rivers > 2-3 m, fruit orchards, soccer grounds, a golf course, a camping ground and a harbor could be identified. Other features like churches where only observable in less dense areas or when they had a special, traditional architecture.

Conclusions

We have pointed out that neither the goals nor the procedures of 3D mapping are clearly defined yet. The available new technologies require a totally fresh approach to mapping.

Satellite images are an interesting source for 3D mapping. However, they still do have a number of substantial disadvantages when compared to aerial images. As spatial and hopefully also radiometric resolutions improve in the future their suitability for landscape model generation and for medium scale topo-mapping has to be continuously evaluated.

In our tests with manual mapping from IKONOS stereo images we found problems in reproducing small geo-morphological details in contours, especially in cases of vegetation covering the bare earth. From these satellite images less map features could be interpreted compared to aerial images, and definitely not with the same reliability. However, the experienced topographer, using empirical knowledge, including also the special characteristics of the country, found many more features than we expected. All in all the IKONOS images were not sufficient for the production of 1:25,000 map data. But even if we consider that satellite images are already now available at the same spatial resolution as aerial images (WorldView-1 with 0.5 m GSD) we still have to take into account the lower radiometric quality of satellite data. Especially digital aerial images provide us with a hitherto unsurpassed image quality, which is very crucial when it comes to the interpretation of map features.

However, these statements refer to the map specifications of Switzerland. In other countries these specifications may not be as stringent.All the previous results were obtained by manual measurements. We should clearly understand that we are currently still very far away from any reliable procedure of automated landscape model or map generation. This remains a key topic for further research.

References

• Fraser, C., Dorji, T.,Gruen, A., 2008: High resolution satellite mapping for spatial information generation in Bhutan. Paper accepted for presentation at the XXIth ISPRS Congress, Beijing, 3-11 July, Commission VI, WG 6.
• Gruen, A., Wolff, K, 2008. 3D mapping from highresolution satellite images. Paper accepted for presentation at the XXIth ISPRS Congress, Beijing, 3-11 July, Commission IV, WG 9.
• Wolff, K, Gruen, A., 2008: Upto- date DSM generation using highresolution satellite image data. Paper accepted for presentation at the XXIth ISPRS Congress, Beijing, 3-11 July, Theme Session 3.
• Zhang, L., Gruen, A., 2006. Multiimage matching for DSM generation from IKONOS imagery. ISPRS Journal of Photogrammetry and Remote Sensing, Vol. 60, pp. 195-211.