Photo and video have both been around for a long time. Recently we have seen interest in locating where photos were snapped, this is called Geotagging. There is a well developed standard for storing location in photos using Exif. As a result of this standard, all photos taken on cameras with a GPS can be uploaded to services such as Flickr and geolocated on a map without any intervention from the end-user.
While the geolocation technologies surrounding photos has matured, the same cannot be said for videos and this is what I want to focus on. Do we need a standard for geotagging videos? I think the answer is definite Yes!
Review
Videos are much more difficult to deal with compared to photos. Photos are a snapshot in time which one point represents well. Videos on the other hand span longer periods of time, a point represents where the video started or finished well but not the part in-between.
I have to admit I am no expert on the inner workings of video formats, but i do have a good idea on what capabilities I would like to see. Having done a bit of research there seems to be a number of ways that location is associated with video:
- Geotagging: Similar to photos, a point is used to locate the starting point of the video. I took a look at the Exif information for video produced by the iPhone, and it contains location information for the starting point of the video:
—- Composite —-
Avg Bitrate : 776 kbps GPS Altitude : 34 m GPS Altitude Ref : Above Sea Level GPS Latitude : 49 deg 15′ 38.52″ N GPS Longitude : 123 deg 4′ 24.60″ W GPS Position : 49 deg 15′ 38.52″ N, 123 deg 4′ 24.60″ W Image Size : 480×272 Rotation : 90 - GPX Syncing: Devices record a GPS track at the same time as recording a video. Applications then consume the GPS track and video, since the time matches on the GPS and video you can tie the two together.
- Native Storage: Contour have a helmet camera with a GPS built in which actually stores this information within the MOV file using the NMEA format rather than maintaining it in a separate GPX file.
Proposal
Contour is moving towards what I believe we need for video. A standard which allows us to very easily tie together individual frames with location. On a more technical level, I want to be able to programmatically extract the latitude and longitude for any frame in the video. If we had that then it would allow us to do some amazing things (thanks to Dave Campanas for these ideas):
- Spatial Queries: Ask questions like “Which videos are under 10km in length?” or “Which videos have a section shot within this bounding box?”. I think this is really important as one of the issues with video is the volume of data that is produced. If the whole video was spatially aware you could search and organise your videos much more efficiently. For example, people unfortunately decided to riot here in Vancouver last year after their beloved hockey team lost. In order to identify suspects the police encouraged people to send in photos/videos of incidents. With over 1 million submissions searching must have been a timely process. But with fully georeferenced videos they could have queried which videos fell within a certain location they were investigating.
- Geo clipping video by area of interest. You would now not only be able to clip a video by time, but by location or distance. Continuing on with an example above you could query all videos in the area you are investigating and then clip the video just to that area too. As a result you would have to review only spatially relevant video.
- Select/relating to other objects: Add a geographic buffer around a video and select objects that are within a proximity of where the footage was recorded. This could add huge value, allowing you to simply merge together your existing spatial data accurately with video. An example would be to select all organisations (so you can draw up a list of contact details) that were in proximity of videos containing vandalism.
- Extract individual frames: Extract closest frame/clip to a target point from multiple videos. An example would be if you know where an individual had committed an act of vandalism from a photo, extract the closest frames from the videos surrounding the incident.
Someone at OpenStreetMap has also seen the value in adding location to the entire length of video. The article walks through their attempt at video mapping using the Contour GPS. This is another great use case.
Conclusion
In my view the hardware is here, so let’s wrap a standard around it so we can start building awesome applications! It would be a missed opportunity if all of the different device manufacturers use their own methods to georeference video. Interoperability would be destroyed and will make it difficult for applications to spring up that wrap innovative solutions around what could be a powerful piece of technology.
Stewart Harper
Stewart is the Technical Director of Cloud Applications and Infrastructure at Safe. When he isn’t building location-based tools for the web, he’s probably skiing or mountain biking.
These are really wonderful thoughts and I’m really looking forward to see
More user examples and ideas around this topic!
I can see a great use in creating a video tour in an area of interest with the map
Following the video timeline. Great in city planning or businesses like themeparks.
Thanks for the comments Oscar. My colleague has just finished building a FME workflow that ties together GPX files and iPhone video. I will post an example over the next few weeks that allows you to play back the video and move a point on the map.
One of the best blog post I’ve read so far. Nice ideas in this post. User subconsciously captures video with detailed metadata. All this can be searched and filtered on demand.
I do not know if this technology is there but this has lot of practical uses.
Thanks for the comment Bala. Let’s hope the standards continue to move in the same direction as the technology is.
There is a standard in works for device orientiation (which could be Geotagged along with Geolocation and Time to a set of video frames). There was recent concept(technology) called Google Goggles which has potential to make use of such API with Video.
Challenges:
The Geolocation used by my home computer is not accurate enough and points to my neighbors house. There could be similar issues measuring speed/orientiation/gravity etc.
But, the way it looks, standards seem to be converging but we have issues with accuracy of device or reference.
I didn’t know about that standard thanks, it should enable web applications to access information related to the device. I shall keep an eye on it.
With regards to accuracy I have found it pretty good when the devices are in open space. But if you are inside and relying on the wireless network for locating or in think trees then accuracy falls off.
Also, Geolocation specs in future might evolve from “location on ground” to “location on space” (3-d) when devices are accurate enough .
[…] [fonte] The GPS e video: uno standard per geo-localizzare i video? by Luca Palli, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License. GPS, Italiano No Comments […]
We have been working on geospatial multimedia for the past few years. I agree, there is a need for standards…GeoVideo is an existing OGC draft but not that well suited to wide market applications….we have drafted a standard (and architecture) better suited to mass market applications. We have also developed a University spin-off technology based around geospatial multimedia: http://www.ubipix.com…this enables anyone with an Android or iPhone collect, tag, upload and share both geocoded video clips as well as image collections. We are presently upgrading the way users can tag both on App and on client. We have come up with a whole range of spatial functions together with applications that could be enabled with this technology.
Please try it out and let us know how you get on.
Tim
Hi Tim, I had a play with your app and it works well. I like the playback with the chart. How have you done it, are you storing the frame based location information within the video container or in a separate file and then referencing it?
Hi Stewart,
We store video and navigation data separately since these streams are sent through various server based processing modules on upload. The tagging functionality is pretty basic at the moment so, we’re upgrading the process to allow multiple in-frame tags and also the ability to tag the actual geographical coordinates of the object rather than the location of observer at the time of capture. Location and tag records are maintained in separate tables within the database.
tim.mccarthy@nuim.ie
Very interesting article. The proposals for querying and interacting with georeferenced video definitely highlight the potential rewards for investing further into spatially aware media.
In response to the needs of some of our clients we have developed our own software for managing and interacting with GPS-enabled videos. Whilst it can be run as a standalone piece of software it is designed to be a fully integrated tool that works with a variety of common mapping platforms (so far it has been used with ArcPad, ArcMap and ArcExplorer but it is not restricted to ESRI formats at all). Videos are dynamically linked to maps, allowing users to view, and analyse video in a GIS environment.
You can read more about it on our blog-post here.
http://riamobilegis.blogspot.com.au/2012/08/geodrive.html
The potential for this is rapidly increasing as we and other organisations find more and more applications for video. The primary driver has been asset management along linear infrastructure (roads, paths, pipelines, etc) but there are many more possibilities for this.
Hi Patrick. Looks like a nice app you have built there, especially like the street-view integration. Is it a web or desktop application?
It would be great to be able to search through a collection of videos by location – I think that would be a killer plugin for a GIS application as then you could spatially query your videos.
We have been searching for 1) a device that records video and GPS data, embedding the GPS data in the video (e.g. metadata) and 2) be able to “play” the GPS data out a port (or similar machine readable access) when the video is played. The idea of being able to sync video with sensor data played in another app.
ContourGPS seems like an obvious candidate, but as near as we can tell, they do not publish how their GPS data is embedded. Other possibilities are the new(ish) wifi enabled helmet cams on the market, one can imagine the cam receiving GPS data (from a separate GPS) over wifi and embedding the data in video.
As you may know, in professional video, there are well established standards for embedding data into video. Why this is so hard at the consumer level is a mystery, there is a market opportunity being missed here.
Any other ideas that may get us to a solution?
Hi Eric,
I am only aware of the Contour model and you are right it is a closed standard. I have no-doubt the GoPro will add GPS to it’s cameras soon, it seems to be the next logical step, so hopefully they adopt a standard we can work with.
The answer has to be a standard similar to Exif that all manufacturers adopt.
I think that MXF standard give a good answer.
There does not need to be a standard for geotagging videos, but a dominant platform. See http://www.ommageo.com video sharing with a Google Earth interface.
As you note separate Android and iOS Apps will be required to work on those devices. I will create automated geotagging for video recorded on those specific devices, but even then a manual geotagging process mill be available because users are not limited to geotagging their own videos, but users may also geotag any video from YouTube by through an embed feature.