To avoid any potential confusion, this is a personal, spare-time project that has nothing to do with the cool Seadragon technology.
Before you read my post, you should watch the short 4 1/2 minute video that demonstrates an image resizing technique called seam carving. The technique was presented at this year's SIGGRAPH 2007 conference by Shai Avidan and Ariel Shamir. If you want to know all of the details, check out the paper they presented called Seam Carving for Content-Aware Image Resizing (20MB PDF). Shai and Ariel outline a relatively simple algorithm for finding a "seam" of pixels that is least likely to be missed when it's removed from an image. The algorithm looks for connected pixels of low energy, where energy refers to a measure of visual detail. By iterating the algorithm, an image can be resized while maintaining its general structure. Unlike a typical stretch operation in your favorite graphics application, the seam carving technique resists squashing or distorting the image.
When I first saw their video, I thought it was magic. Not only do they demonstrate resizing, but they also show how their algorithm can easily remove specific content (like a person or object) from a scene. It's easy to imagine how images can be resized using this technique for display on smaller devices like a cell phone. In their paper, Shai and Ariel briefly describe a method that would allow an image format with extended seam carving data to easily resize on-the-fly. Very cool.
Inspired by their work, I thought I'd play around with the algorithm and methods in my spare time. So, over the course of the past week or so, I've put together a .NET-based implementation of seam carving that I affectionately call SEAMonster. The code has evolved from a very simple prototype to something a bit more robust. As these things go, the architecture is a bit shaky at the moment, but it does work. The early version of the application can only open JPG files, doesn't support image expansion, and the user interface is pretty minimal. However, it's far enough along that someone might find it interesting.
If you'd like to take it for a spin, download SEAMonster_0.1.zip (63KB). The application requires the Microsoft .NET Framework Version 2.0 if you don't already have it installed on your machine. In lieu of documentation, I encourage you to watch the 8 minute video introduction I recorded to quickly get up-to-speed (note that you may need to download the TechSmith Screen Capture Codec to properly view this video).
When I have the code under control, I plan to upload the project to CodePlex so that the developer community can contribute. All of the image manipulation functionality is encapsulated in a DLL, so it should be usable in most scenarios. There is a lot of opportunity for improvement and optimization in these algorithms, and I expect a lot of innovation to happen over the next few months. It should be fun to watch.
Here are some other seam carving resources that you may find interesting:
If you have any comments about this early version of SEAMonster, or if you have suggestions for the version I plan to release on CodePlex, please leave feedback (or contact me directly).
Vous avez peut être vu cette présentation d'une technique impressionnante de redimensionnement d'images
I just posted this on the Paint.Net forums. Do you have any plans of creating a plugin for PDN?
Brian...I don't have any plans to create a plug-in. However, once I've uploaded the SEAMonster code to CodePlex, it shouldn't be too much work for someone to do this. The logic is implemented as a library which should make it relatively easy to adapt to other uses.
Wow, you weren't kidding about it being CPU intensive. I have a 2.5 meg image and it's taking a long time to do operations on a quad top of line Intel cpu. Is the algorithm something that can be spread out across CPUs? And also, are there plans to put your app on codeplex as well (with source)? Using larger images than the screen do not work very well since there are no scrollbars :(
I am completely amazed however, Great work!! I've never seen this before and is REALLY interesting! I can't wait till the library lets you stretch too :>
Kevin...yes, it is very CPU intensive. This is the same story for every implementation I've seen. I'm sure the algorithm will see improvements over time. Also, as you mention, the algorithm is very well suited to multi-core/CPUs, and performance should increase dramatically when that functionality is added.
Related to CodePlex, yes, as mentioned in the post, I do plan to upload it so that people much smarter than me can make contributions. :)
Nice job. Great to see this in .NET. I think the Seam Carving just might be the killer app to drive home those dual and multi-core CPUs when photoshop or design can do this. C'mon don't tell me you aren't going to upgrade all your PCs then when your friends are Seam Carving with impunity.
Doing this for web would be nice as well, imagine all those sites with skewed imagery and distorted aspect ratios self correcting themselves stylistically with Seam Carving. A man can dream....
Very interesting concept. I look forward to trying out your source code.
This is neat, and is good for photo manipulation. I kind of doubt the use as suggested in the opening of the video though, that we supposedly need the ability of images to resize like this inside of browser windows.
Check this one out. Very smooth, shows seams being removed...
Hi Mike! Nice GUI. I was considering trying a .NET implementation, but ran into threading issues that I couldn't easily resolve, so I kind of gave up on that :)
A bunch of work later and the algorithm is getting fast. Down to about 15 seconds for 1024x678 down to 800x600. Unfortunetly, I don't see many more drastic performance increases.
Thanks for the compliment, Brain. It's really more of a "fancy test harness" than anything else. :) 15 seconds for that resize is great. It's definitely an expensive algorithm if you want the best fidelity.
Awesome! I saw that original demo and was amazed. Cool to see it in .NET. Can't wait to try it out.
Sometimes you read people's blogs and you get to know their hobbies and roughly what filed of work
I am amazed by this technique, and how quickly there are a million different implementations of it!