Getting to your Destination: Part I – Data Gathering Methodology

Right click here and choose 'Save target as...' to download the article in .pdf format.

Hi. I am Neels van Rooyen and am responsible for auto-routing, data translation and data management procedures at Mapit, one of the companies involved in bringing you the Garmap products.

In this topic series I hope to bring you a few articles describing the data capturing, processing, route-cleaning and finalization processes involved in bringing you routable data from the field to your device. The articles will be fairly technical (no glossy magazine stuff here) but I hope that any layperson interested will be able to understand them and the rather involved process of brining maps to your GPS device.

In the first instalment called ‘Data Gathering Methodology’, I would like to describe how we go about collecting source information and the reasons behind our methodology. In future articles I hope to describe each step in the process to a final product.

Where can one get the base map data?

When we started the data capturing project back in 1999 we were faced with three choices on how to obtain base data for new areas:

Use cadastral information. I.e. use town and city council planning maps in both electronic and paper format. This is the traditional way of putting together a map set and is the main procedure used by one of our main competitors.
Drive all navigable roads with a good GPS and collect attribute information (i.e. road names etc.) as you go along. This method is used to great extent by the big European mapping companies like Navtech.
Use aerial and satellite photography to determine where roads are and use the maps from no. 1 to obtain other information like street names.
A combination of 2 & 3. This is the method used by Mapit and European mapping companies like Tele Atlas.

The cost involved increases significantly from procedure 1 to 3. Each process also has its inherent limitations and advantages which I will describe briefly.

Cadastral Information as source:

Collecting town and city council planning maps has the advantage that it is certainly the cheapest and allows one to grow coverage fairly rapidly. Unfortunately in South Africa this approach is just not feasible for the following reasons:

‘As Built’ more often than not, does not agree with ‘As Planned’. Please look at the following example from our recent Durban update. In this image the town planner’s data is drawn in red and the white lines represent the roads we have captured. If I add that the image is about three years more recent than the town planning data and that examples like this pop up all over the place I am sure that you will understand that one cannot rely on this source alone. How can one blissfully capture roads off town plans when even a big city’s internal plans show discrepancies like this and still expect the final data to be acceptable?

The town plans focus on stands (or erven in the formal terminology) and street lines are not drawn in. One can easily capture suburban streets, but throughroutes and freeways without any stands next to them and outlying roads, become a big problem.
Road closures, road upgrades, road changes and other upgrades are not reported to the central town planning departments and the official plans often lag up to decades behind the actual road network.

Driving with a GPS as source:

Driving along with a GPS to capture new roads certainly has many advantages:

‘As Built’ information is collected.
All attribute information such as street names and road conditions can be captured at the same time.
POI (Places of Interest) information can also be captured at the same time.

Unfortunately there are also several disadvantages to this approach:

It is very difficult to ensure that a town is covered completely and all streets are driven, without some background map information.
Urban sprawl is a dramatic phenomenon in South Africa and new suburbs, road closures, road restructuring happen all over the place, even in settled neighbourhoods. To give you an example: Over 3000 new roads were drawn into the Pretoria area alone in our new update and this was only an update from late 2003 to early 2006. This excludes existing roads that have been edited. It is certainly not feasible to re-drive all the Metropolitan areas every two years.
Most towns have large traditionally disadvantaged neighbourhoods surrounding the ‘old’ town. These suburbs range from formal, through informal, to squatter areas. Many of these areas may not be of interest to the casual recreational user but are vital to delivery and tracking companies. These neighbourhoods can be difficult to access by fieldwork teams and may even be dangerous to enter. Upgrades to these neighbourhoods often happen at breakneck speed with an accompanying sprawl into new areas. Surveying by GPS is just impossible. In the following picture you can see an example of the size of a traditional disadvantaged neighbourhood (circled in blue) next to ‘old’ Bethlehem (circled in red)

I am sure many of you are aware of the problem with signal loss by the GPS caused by buildings, trees, mountains and other conditions. This forces one to equip fieldwork teams with expensive differential GPS units.
Accurate mapping by GPS can be very difficult and time consuming. Consider the following example of two fieldwork tracks:

With the left track the fieldwork team has driven into a filling station and the road does not actually go like this. In the second example, when travelling from the bottom, the fieldwork team had taken a slipway when turning left and this is actually a normal intersection with some slipways present. It takes a great amount of repetitive driving to cover all ramps and slipways at intersections and to compensate for entrances

The reason why this approach is favoured by European mapping is because urbanisation has virtually ceased and new roads are a rarity over there. Furthermore, most European countries have procedures in place whereby new upgrades to the road network are reported centrally which can easily be used as a source of reference to only survey these areas.

Aerial and Satellite Photography as source:

Once again this approach has advantages:

The data captured is completely ‘As Built’.
It is easy to identify new areas and suburbs and to ensure complete coverage of all roads.
It is easy to identify changes in existing areas and update only these. One can even see many road closures.
It is easy to identify up to where navigable roads lead into informal settlements.
Intersections and freeways are easier to draw in and there is no need to drive every ramp and slipway.

But even this source has its drawbacks:

Aerial or satellite photography is prohibitively expensive. It ranges from a minimum of about R 20K for a small town to in excess of R 12 million for an area the size of Gauteng.
Because of the rapid urban sprawl in South Africa, photography of more than at most three years old is required and it is often difficult and expensive to schedule new photography exclusively.
One is still reliant on other sources for attribute information such as street names.
Heavy tree or cloud cover can obscure vital parts of the photo set.
In outlying areas, where no GPS points are available it is impossible to accurately reference the photography (called geo-correcting), necessitating at least some fieldwork with an accurate GPS before the photography is usable.

The Mapit Approach

Mapit (www.mapit.co.za) is responsible for gathering the data for, and building the Garmap products.

Through the past 8 years since the inception of the programme, we have devised a methodology of sourcing data, which we believe is unique in the world and best suited to the South African and African environments.

After evaluating the three methods mentioned above we decided that the only viable option open to us, is to use aerial or satellite photography exclusively. This was possible in spite of the disadvantages mentioned above because of the following reasons:

Due to a variety of reasons Mapit has a very close relationship with various town and city councils, other companies and institutions that utilize aerial or satellite photography as well as with all the South African sources and most of the world’s sources of aerial or satellite photography for Africa. Because of these relationships it is often possible to acquire the necessary photography for highly discounted prices or even for free by working out mutual beneficial deals or by pooling with other companies to source the photography. We believe that we are in a unique position as company to utilize photography for mapping purposes because of this relationship network established over many years dating back to years before this project even started.
MapStudio is involved in the ownership of MapIT and through this relationship we have access to over 50 years worth of mapping data and map research which provided an instant, country wide, set of attributes (such as street names).

With this approach we managed to produce our first mapsets with all of the metropolitan areas and some of the larger cities completely covered. It is possible to source complete photographic sets for these areas and update our mapsets every two years. We are very happy with this most efficient and complete approach and will continue to use it for all future updates.

When spreading the coverage to smaller towns and rural areas, this approach became problematic forcing a change in approach because of the following reasons:

It is almost always impossible to obtain partners in sharing the cost of photography (the South Coast area in the current ver.4 being an exception to this rule) for smaller towns, forcing Mapit to carry the cost on its own, making the addition of new towns to the mapset very expensive.
Even for the Metropolitan Areas, POI information (with an accurate GPS positions) was not obtainable from outside sources and good enough partnerships to obtain this information from other contributors were not forthcoming.
An update every two years to Metropolitan Areas is not sufficient to accurately reflect changes to the major road network. New intersections and changes to existing ones occur all the time.

These problems forced Mapit to come up with a combined approach which we believe to be unique to our company:

We still use photography for the metropolitan areas, areas and larger towns and will continue to do so regardless of the cost.
We still source attribute information from MapStudio and augment this with an extensive contacting process to obtain all possible information and plans from the various city and town councils. We do not use this information to draw in roads but use it exclusively for attribute information and to give us an idea of coverage for new towns.
We have put together several fieldwork teams that are active all across South Africa, Nigeria and soon selected countries in the rest of Africa. These fieldwork teams serve several purposes:

Drive smaller towns: For these smaller towns, recent photography is just too expensive. We try to use all data we can get, even older photography (we have large sets of photography dating back to the nineties or earlier), and use the fieldwork to confirm currency and to check for new additions and changes.
Capture POI information: We have an extensive scheme to capture accurate POI information for all the metropolitan areas and have progressed a long way down this road. New towns added to the dataset are also driven for POI information although we are currently forced to only drive the throughroutes (most of the POI are grouped around these routes in any event, and this approach allows us to cover a larger number of towns with the available resources without missing a significant number of POI).
Capture intersections and exits along regional roads. Photography with sufficient detail to capture ramps and slipways at these intersections are not available at current, and even if this will become available in future, it will not be updated frequently enough to reflect all changes.

We are building our user community actively:

At first our only clients were the larger tracking and scheduling companies. We received a lot of tracks from these companies during the past allowing us to update, especially our regional roads, and identifying the roads most used through the rest of Africa. These companies are still a major source of information and these tracks also allow us to reference photographs were no GPS points are available.
Since releasing the first Garmin products, without actively pursuing this avenue due to mainly resource constrains, a number of active contributors have sent in huge contributions (often growing into thousands of kilometers of roads). Although we received information from casual contributors, these select few contributors helped us enormously. I will not mention these people – you know who you are – and you will continue to be a most important part of our plans.
Since the end of last year we have launched a much more active user contribution initiative. Starting with the compilation of a complete set of user guidelines with much input from the people mentioned above we have continued by appointing our most active contributor
Peter Short (Shorty to most people) to coordinate and manage user contributions. We have also launched a new forum. We hope to grow this part of the data chain still further because (due to the South African and African situation) it is often only the end user that is aware of changes in the road network and mistakes in our current mapsets.

Finally

Phew! I was meaning to wrap this up in four pages, and now I am at seven. If you are still with me I hope that you understand more about where we get our data from.

In the next installment I hope to show you examples of the photography available to us, as well as the steps involved in capturing data off this photography.