Siddharth Jain, one of the devs on our team, wrote a tool using VE3D to place photos in 3D and save this information as a tag in the file itself.  In short, you open a picture in the program, fly to roughly where the picture was taken from, click on points in the world and in the photo that you know are the same (for example, the chimney of a house, or the edge of a lake), and the program tries to determine where the picture was taken from.  You can then view this picture in the world.

We have a picture that another dev on the team took in Las Vegas for you to play with.  This picture already has some rough information in it to help you get started, and it might be a good idea to start with it.  There's also an image that has already been placed, so you can see how it might look after you're done (load this picture into the tool instead of the first one to see it).  Generally speaking your own photos will probably not have any positioning information in them to begin with, unless you have a very fancy camera, so it is important that you know pretty much where you were when you took it.  Also keep in mind that this is a tool and demonstration, not a finished product.

Here’s Sid’s writeup of the tool.  Enjoy!

Install

Download the zip file and extract its contents to any folder (there is no installer program).  As long as all four dll/exe files are in the same directory it should just run by double-clicking the exe file.

Introduction

The Microsoft Photo Placement Tool can be used to find the geographic coordinates of the camera that took a photo. In this way a photo can be geolocated in the 3D world. There are seven parameters which need to be determined to geoposition a photo in the 3D world. Six of them are latitude, longitude, altitude, roll, pitch, yaw which define the position and orientation of the camera. The seventh parameter is the focal length of the camera that took the photo. The Photo Placement Tool allows determination of these seven parameters via manually specified correspondences between points in the photo and 3D world.

System Requirements

Virtual Earth 3D (VE3D) must be installed on user’s machine.

Usage

On running the exe following screen is displayed:

Step 1: User uploads a photo by clicking on Load Image button

Any exif information in the image such as GPS position and focal length is automatically loaded.

Step 2: User navigates in VE3D to a position from where s/he can begin adding correspondences

An example screen is shown below:

Step 3: User turns on calibration by clicking on Turn ON Calibration button

When you are ready to begin adding correspondences, click on the Turn ON Calibration button. This sets the initial camera position and orientation, which is then optimized as correspondences are added.  To get back to step 2, click on the Load Image button again.

Step 4: User adds point correspondences between photo and 3D world

As the user adds correspondences via left mouse click, the tool calibrates the photo in the background. The Preview Result button is automatically enabled after adding 3 or more correspondences. An example is shown below:

While in calibration mode it is still possible to navigate in VE3D using the arrow keys, and change the view by holding the shift and ctrl keys while dragging with the mouse. Normal mouse drag is disabled in this mode.

Step 5: Preview the result by clicking on Preview button

When the RMS reprojection error becomes small (say less than 20 pixels), preview the result by clicking on Preview button. The smaller the error, the better the preview would be.

Step 6: Either Accept the result or Revise

User can either Accept or Revise. Clicking on Accept will display the solution and prompt user to save the calibrated photo as a JPEG. The latitude, longitude, altitude, roll, pitch, yaw, and focal length are saved as EXIF tags in the JPEG. Roll, pitch, yaw are relative to the local coordinate system at (lat,lon,alt). All angles (lat, lon, roll, pitch, yaw) are stored in degrees. Next time the photo is opened in this tool, the camera will automatically fly to that location.

By clicking on Revise user can continue to add more correspondences and refine the result. Note that as more correspondences are added, the RMS reprojection error may become better or even worse.

Miscellaneous Features

Use Locate feature to quickly fly to a location e.g. New York

Use the Locate button on the Menu to quickly fly to a location e.g. New York

Deleting a pushpin

Move mouse pointer over pushpin. Its color changes. Press delete key to delete the pushpin. Note that deleting a pushpin also deletes the corresponding pushpin, if any, in the other view.

The Settings Window

By clicking on the Settings button, user can control the parameters to optimize. The user would generally not need to change the default settings. However, for completeness the options provided by this window are described here. If the altitude option is unchecked, the tool will not optimize the altitude of the camera. For example, if you took a photo from the ground and have positioned the Virtual Earth 3D camera on the ground, you may want to uncheck the altitude button as the correct altitude has already been set by you. Similarly, if the focal length option is left unchecked, the tool will not optimize the focal length of the camera. You may want to do this, for example, when you are sure that the EXIF focal length is the correct focal length.

Technical Note: user may find that when a calibrated photo is opened, the camera does not fly to the exact calibrated location. Usually when this happens the camera is positioned too high.  This is because the background data in the area, specifically the DEM, is not yet fully loaded, and the program wasn’t able to get to the right spot.  Loading the image again should correct the problem.

Acknowledgements

Bill Chen in VE3D Research originally developed a photo geopositioning tool.  His work can be found here.  The present tool has been developed from scratch and is 100% managed code.  I would like to acknowledge Nikolai Faaland, David Buerer, Bret Mulvey for helping me out on this project. David Buerer provided the Hilton Las Vegas image used in this writeup.  Jonathan Doughi provided the Smith Tower image used in the writeup.