VISAT: Mapping what you see

With the continued growth of urban centers all around the world, city planners are required to keep up with up-to-date geographical at a faster rate. This has led to the establishment of spatially-referenced Geographic Information Systems (GIS) for a variety of municipal applications. This information, however, is expensive to obtain by conventional methods. In addition, conventional methods supply only point solutions and are therefore not suited to support the increasingly complex requirements posed by urban centers in a timely fashion. Satellite remote sensing and aerial photogrammetry are two methods which can provide various GIS information at high rates and reasonable cost. However, with the first method, the associated accuracy is not suitable for many applications, and in the second case the near vertical field of view provides only part of the information required.

Furthermore, the type and quality of information required by the user is changing, quite often the user prefers a cartographically less perfect product (e.g. map substitute) that contains the most recent information rather than a product of very high cartographic standard but with outdated information. Also, the demand for user specific maps or data and for noncartographic products such as reports, images, graphs, and frequently asked questions are steadily increasing. For example; 3D digital maps (with landmarks; road vectors; transportation infrastructure information; etc.) that offer more visual information to make car navigation easier will soon replace the conventional 2D maps. 3D digital maps with enhanced 3D visualization will make maps even more attractive, informative and attractive, informative and interesting, supporting a new range of Internet and consumer applications.

With the advances in satellite and inertial georeferencing techniques and the readily available of digital imaging sensors, a considerable portion of GIS information can be acquired from moving vehicles. The advantage of kinematic-mode data collection is that the survey can be performed much faster and therefore more economically, whilst gathering mapped data from a new dimension – in the same plane where the data is seen and needed, or what the human eye sees.

Mobile Mapping – The venue for 3D Mapping

The concept of mobile mapping or mapping from moving vehicles has been around for as long as photogrammetry has been practiced. Early incarnations of Mobile Mapping Systems (MMS) were however, restricted to applications that permitted the determination of the mapped from existing ground control points. Fifteen years ago, advances in satellite and inertial location technologies made it possible to develop mobile mapping system differently. Instead of using ground control as reference for orientating the images, the trajectory and attitude of the imaging platform could now be determined directly. This made mobile mapping to be independent of preset ground control points. Hand in hand with this development was the change from analog to digital imaging- a change that has considerably evolved over the past years (Schwarz and El-Sheimy, 2004). As a result, mobile mapping systems have evolved from a concept of academic interest to a commercially viable industry and are currently at industry and are currently at a point where they match classical survey techniques in accuracy but far surpass in economy, speed and efficiency. These systems integrate navigation sensors and imaging sensors to determine the positions of the imaged points.. Although, the idea of mobile mapping is based on a simple concept, the real world implementation brings a lot of challenging problems. These are a product of integrating the concepts of kinematic geodesy, navigation, remote sensing, machine vision, and digital photogrammetry sciences which have been always treated separately. For more details, the reader is advised to read the article by Skaloud (1999) on mobile mapping implementation problems.

The initial trials to build a Mobile mapping system was a van for highway inventory (HI) 1983 by the University of Calgary (Schwarz et. al. 1993), however real implementation of practical systems were developed by the Centre for Mapping at the Ohio State University and the University of Calgary in the mid nineties. The University of Calgary system development objective was “A mobile mapping system that positions all visible objects of interest for an urban GIS with an RMS accuracy of 0.3 m while moving through a road corridor at a speed of 60 km/h and a maximum distance to the desired objects of 50 m. Data acquisition must be automatic and should contain real-time quality control features. Data processing, except for quality control, will be done in post mission and should have separate modules for georeferencing, image data base management, imaging, and quality assessment.” (El-Sheimy, 1996) The outcome of this project was the VISATT Van. The VISATT system – in its initial form – was notable because of the large number of imaging sensors it employed. Where previous land-based MMS were simple stereovision systems employing only two forward facing cameras, VISAT had eight cameras – permitting more flexible data collection and better imaging geometry. A new generation of the VISATT Mobile Mapping System has been developed in cooperation with Absolute Mapping Solution (AMS) which truly delivers a mobile mapping platform that integrates multisensors subsystems (See Figure 1).

In this article, an overview of the VISATT van is given. The sensors on board the VISATT are described highlighting their system functionality while providing an overview of the system’s operational mapping cycle. System deliverables and accuracy are discussed. Finally, an outlook into the future development of the VISATT including hardware, software, and applications is presented.

VISATT – System components

Although all mobile mapping vans share the same concept of direct georeferencing, they carry different types and grades of sensors depending on the application, integration scheme, and the required accuracy. For example, vans which are used for highway maintenance are equipped with a single GPS receiver and a single camera to detect the locations of asphalt defects with accuracy of few meters. In general, a mobile mapping van integrates navigation sensors and imaging sensors that can be used to determine the position of imaged points. All the sensors are rigidly mounted together on a platform; the former sensors determine the position and orientation of the platform, and the latter sensors determine the position of points external to the platform. The sensors that are used for the external position determination are predominantly photographic sensors and thus are typically referred to as imaging sensors (El-Sheimy, 1999).

sensors (El-Sheimy, 1999). However, additional sensors such as laser rangefinders (Li et al., 1999) or laser scanners are also used in MMS and therefore the more general terms of mapping sensors or relative sensors may also be used when referring to the remote sensors (Ellum and El-Sheimy, 2001). Generally speaking, the final system quality depends on the accuracy of the used sensors and their hardware/ software integration schemes.

The core hardware components of the VISATT van are a Strap down Inertial Navigation System (SINS), a dualfrequency GPS receiver, and a cluster of digital color cameras. The primary purposes of these components are – the GPS provides the position of the van, the SINS provides the orientation of the van, and the cameras are used for relative positioning from the van. These components, however, also have important secondary functions. For the GPS, these secondary tasks include controlling the long-term error growth of the SINS through the GPS/SINS Kalman filter and providing the precise timing base for all data streams. The secondary tasks of the SINS stem from its ability to be used as a position sensor in addition to an orientation sensor; consequently, these tasks include bridging GPS signal outages, detecting and correcting GPS cycle slips, and precise interpolation between GPS positions. The latter task – interpolation between GPS positions – is possible because the SINS provide data at 200 Hz, while the GPS positions and velocities are only available at 1-2 Hz.

Figure 1:VISATTM Van Mobile Mapping System

In addition to the GPS, SINS, and cameras, the VISAT system also integrates a Distance Measuring Instrument (DMI). The pick-up from the DMI is used to trigger the acquisition of the images from the cameras at constant distance

Table 1: Primary and Secondary Functions of VISAT Sensors

Figure 2: VISATT Log Application Running in VISAT Van

intervals defined by the user. Table 1 summarizes the primary and secondary tasks of the sensor in VISAT. Currently, the VISATT imaging component consists of 6 to 12 progressive color digital cameras (1600 x 1200 pixels or 2048 x 2048 pixels) which provide a 280 to 3600 field of view. The images are captured at high sampling rates and can be controlled by either time or traveled distance (usually every 2-7 m).

The images are captured while the van is moving at the highway posted speed (up to 125 km/h). Essentially, all navigation and mapping data streams are synchronized to a common time frame using a high frequency synchronized multi-channel clock. VISATT has an efficient and robust data logging module which enables the collection of the data with minimum time delay.

The VISATT logging system (see Figure 2) also has an expert module for real-time quality control, which communicates with the system’s operator and provides useful information such as length of the survey, distance to master station, directions to specific routes, etc.

VISATT – Operational cycle

VISATT provide a task-oriented implementation of mapping concepts. Surveying by VISATT consists of three steps, which are essentially the same as for any mobile mapping system:
. Data acquisition
. Georeferencing of the images using data from the navigation sensors (GPS and SINS).
. Positioning of objects in a mapping frame using a photogrammetric workstation and two or more more

VISATTM Geolmages Server Architecture

more georeferenced images.

All data collected by the VISAT van is post-processed. During the post-processing, the digital images acquired from the VISAT van are georeferenced using the position and orientation as determined by the GPS and SINS data. The system position and orientation are interpolated at the instants of image exposures and then combined with the system calibration parameters, as described by lever arm and boresight angles, to relate the images to the real world coordinate system (El-Sheimy, 2005). The georeferenced images are hosted on VISATT GeoImage Servers along with the camera calibration that describes the inner orientation of the sensors, and the system calibrations that describe the lever arm and boresight angles. This new generation of servers allows client access via .NET Remoting on TCP, HTTP, or Named Pipe channels for desktop, workgroup, or internet deployment and distribution. VISATT GeoImage Servers also act as a Universal Description Discovery and Integration Service (UDDI Web Service). The distributed three layers architecture of the server, Login and Security, GeoImage Metadata, and Image File Server architecture can accommodate for mega size VISAT Image Libraries by using

VISATT Station Measurement

Network Load Balancing (NLB) and Server Clustering techniques.

The next stage is the extraction of 3-D coordinates from the images. In addition, geometric information and attributes of themed objects such as control points, utility lines, and land parcels may be needed to form GIS elements for themed layers. This task is performed using a photogrammetric workstation called the VISAT StationT that enables the measurement of objects appearing in the images and the generation of GIS elements. VISATT Station is the client application that enables the user to make use of the collected georeferenced images and perform mapping and GIS editing. The georeferenced images are accessed from a local file on the user’s desktop or by logging on to one of the VISATT GeoImage Servers on the enterprise’s Intranet or by subscription to public servers on the Internet. Point features are digitized by measuring the point in at least two images. The 3D point coordinates are obtained using simple photogrammetric intersection. Attribute information can be also collected by assigning the appropriate point symbol for the point feature. VISATT data are easily populated into GIS software like ARC-GIS.

Automated Road Vector Extraction Engine (ARVEE) is a VISAT software component, especially designed to automatically detect and extract road lane line markings and road edges. ARVEE is an automated component running as a service on VISATT GeoImages Server that automatically process or re-process new or update georeference image files placed on the server. ARVEE derived information contains the 3D lane line vectors, their color and line type attributes. The 3D Road Vectors are also used in the Automated Quality Control Service (AQCS) of the server infrastructure and can easily be integrated into GIS platforms providing a vital method for creating and/or updating an important GIS layer for the next generation of Advanced Car Navigation, Driver Assistance Systems, and Fleet Management Services. (Wang et al., 2007 and El-Sheimy et al., 2007).

VISAT™ Arvee Application

Mobile mapping -future outlook

Mobile mapping for land vehicles, the combination of digital imaging and georeferencing, has developed from a topic of academic interest to a commercially viable industry with several applications. The VISAT technology, presented in this paper as an example of MMS, offers a system which is unique in several aspects:

. it offers a high-accuracy (10 – 30 cm RMSE) georeferenced imagery-based data product
. its georeferenced imagery (6 to 12 progressive color digital cameras of 1600 x 1200 pixels or 2048 x 2048 pixels which provide a 280 to 3600 field of view) can be used for 3D visualization or mined for GIS applications,
. its images are captured while the van is moving at the highway posted speed (up to 125 km/ h) and therefore the cost to acquire the data is significantly lower than other methods
. the level of accuracy and resolution delivered opens the door to a host of applications for which there is a current shortage of data

With the rapid development of highresolution digital frame cameras and the current development of laser and other sensors, economy and efficiency will continue to improve for MMS. The future of MMS is nothing short of promising and exciting. For example, the next generation of VISATT, the VISATT Van 3D Modeler which is currently under development with a prototype expected in early 2008 will include a terrestrial laser scanner for 3D modeling applications, enabling the user to “view” a photorealistic 3D model of the streets, surrounding buildings, road surface, etc. Future extension of the VISATT Van 3D Modeler includes the integration of multi-spectral sensors, infrared, and ground penetrating radar (GPR) with the overall objectives of providing a system capable of producing 3D virtual cities. This is just the beginning – MMS, VISAT included, truly provide a faithful capture of mapping what we see.

References

* Ellum, C.M. and N. El-Sheimy. 2001. A mobile mapping system for the survey community. Proceedings of The 3rd International Symposium on Mobile Mapping Technology (MMS 2001). Cairo, Egypt. January 3-5, 2001. On CD-ROM.

* El-Sheimy N., Wang C., Hassan T., and Lavigne M. 2007. Mobile Mapping for Automatic Extraction of Highway 3D Linear Features. The 3rd Annual Map Middle East Conference, Dubai, (UAE), April 9th to 11th, 2007.

* El-Sheimy, N., Chiang, KW. , and Noureldin, A. 2005. The Utilization of Artificial Neural Networks for Multi-Sensor System Integration in Navigation and Positioning Instruments. IEEE Transactions on Instrumentation and Measurement.

* El-Sheimy, N 2005. An Overview of Mobile Mapping Systems. FIG Working Week 2005 and GSDI-8, Cairo, Egypt April 16-21 (24 pages, on-line publication, http://www.fig. net/pub/cairo/papers/ts_17/ts17_ 03_elsheimy.pdf – Invited Paper.

* El-Sheimy, N., 2002. Boresight Calibration of Mobile Mapping System. FIG XXII International Congress, TS5.12 Calibration of Survey Equipment, Washington, D.C. USA, April 19-26.

* El-Sheimy, N. (1999). Trends in Georeferencing of Mobile Mapping Data. The FIG Working Week, TS7: Trends in Positioning Measurements, pp A1-A23, Sun City, South Africa, April 1999.

* El-Sheimy N., 1996. The Development of VISATT – A Mobile Survey System for GIS Applications. Ph.D. thesis, UCGE Report No. 20101. Department of Geomatics, The University of Calgary.

* Li, D. S.-D. Zhong, S.X. He, and H. Zheng. 1999. A mobile mapping system based on GPS, GIS and multi-sensor. Proceedings International Workshop on Mobile Mapping Technology. Bangkok, Thailand. April 21- 23, 1999. pp. 1-3-1 – 1-3-5.

* Nassar S., and N. El-Sheimy, 2005. Wavelet Analysis for Improving INS and INS/DGPS Navigation Accuracy. Journal of Navigation, The Royal Institute of Navigation, 58(1), pp. 119-134.

* Schwarz, K. P., Martell, H., El-Schwarz, K. P., Martell, H., El- Sheimy, N., Li, R., Chapman, M., and Cosandier, D. (1993), “VISAT- A Mobile Highway Survey System of High Accuracy”, VNIS Conference ’93 Conference, Ottawa, October 12-15, pp. 476-481.

* Schwarz K.P. and El-Sheimy N. (2004). Mobile Mapping Technologies: State of the Art and Future Trends. The International Society for Photogrammetry and Remote Sensing (ISPRS) 2004 Congress, Commission I, Istanbul, Turkey, July 15-22, 1996.

* Skaloud J., 1999. Problems in Direct Georeferencing by INS/ DGPS in the Airborne Environment. Invited paper, ISPRS Workshop on ‘Direct versus Indirect Methods of sensor Orientation’ WGIII/1, Barcelona.

* Wang C., Hassan T, and El- Sheimy N 2006. Automatic Road Geometry Extraction System for Mobile Mapping. The 5th International Symposium on Mobile Mapping Technology, Padua Italy 28-31 May 2007.