From 2D to 3D: Community-based use of photogrammetry

“Evolution favours eyes that perceive the world in 3D”. True in nature, but 3D TV for entertainment has been less successful. It maybe that the early applications which gain traction will be more functional applications where the third dimension brings greater value that just better entertainment; such as in mapping terrain or modelling structures.

Until recently mapping the ground in 3D would have required a stereoscopic camera with the depth created by the differential depth of field between the two lenses. Now, with much greater processing power, the depth of field can be calculated from the overlap in images taken from two different angles.

This is not a simple process. Before aerial images can be stitched together it is important that they are adjusted for any distortions in the lens or the angle at which the photograph was taken. This is called ortho-mosaic processing.
Ortho –comes from the Greek word meaning straight, right, or proper.
Mosaic is the art of creating images from an assemblage of small pieces of coloured glass, stone, or other materials. It is a technique of decorative art or interior decoration. Most mosaics are made of small, flat, roughly square, pieces of stone or glass of different colours, known in Latin as tesserae.
In this case, the individual tesserae are photographs. Photograph is based on the Greek φῶς (phos), meaning “light”, and γραφή (graphê), meaning “drawing, writing”, together meaning “drawing with light”.

Whereas photomosaic processing can take lots of digital images – be they stills or frames from a video – and stitch them together, it is also possible to work out the height of objects if they were photographed from an oblique angle and not directly overhead or from the side. Adding height information gives us detail of the topography. Topography, the detailed description of a place, comes from the Greek words τόπος (topos, “place”) and -γραφία (-graphia, “writing”). In classical literature this refers to writing about a place or places, what is now largely called ‘local history’.

This whole process is called Photogrammetry. This is a measurement technique for which the co-ordinates of the points in 3D of an object are determined by the measurements made in two photographic images (or more) taken from different positions, usually from different passes of an aerial photography flight. In this technique, common points are identified on each image. A line of sight (or ray) can be built from the camera location to the point on the object. It is the intersection of its rays (triangulation) which determines the relative three-dimensional position of the point.

As drone technology becomes more accessible, it brings the possibility of using amateur footage to reconstruct heritage sites, as can be seen with this example of a Norman castle, created using imagery from an off-the-shelf system and flown by a young person.
The Penrice Castle model was created in a collaboration with Mark Allan and Bradley Sparkes of Northumbria University.

Video frames and 250 higher resolution stills were processed to create a 3D model with over 12.7m points and 1.2m faces. They have been put together to create a perfectly accurate scale model of the castle, faithful to the colour and texture of the walls even down to where they are covered in ivy. The Penrice Castle model demonstrates how this approach opens up opportunities for a citizen science based approach to engaging local communities in heritage stewardship.

Structure from Motion (SfM) shows how one can take this a step further. First developed within the field of computer vision technologies and was introduced for the purpose of modelling built structures using a vast database of imagery acquired online (image hosting services such as Flickr etc.) from a variety of cameras, capturing the object-of interest from a multitude of perspectives. It operates under a similar principle to that of traditional stereoscopic photogrammetry in that 3D structure can be resolved from a series of offset overlapping images however, it requires significantly less involvement from the end user as much of the alignment and model construction is performed automatically. SfM has been widely adopted within the geosciences for reconstructing landscape topography for quantitative analysis, whether that be simply examining landforms and structural features, or comparing with time-series datasets for the detection of topographical changes. Mark Allan and Bradley Sparkes, advisers to this project have been pioneering this approach in remote locations.

Such applications include, amongst many others, detecting and monitoring landslides, quantifying rates of mass loss from retreating glaciers and observing dessert dune migration. SfM requires relatively little capital and logistical investment compared to more established surveying techniques, and is therefore particularly well-suited to deployment in remote environments where high resolution analyses have previously been limited.

We have the building blocks for an approach where terrain or structures can be swiftly modelled from the air, and then greater detail can be added by people using their smartphone cameras, opening up a new chapter in citizen based heritage stewardship.