D3D10_CREATE_DEVICE_DEBUG is a really handy thing to use:
http://msdn2.microsoft.com/en-us/library/bb204909.aspx
Watch your debug output with this turned on and you will get a lot of helpful stuff. Try rendering without a viewport, for example.
One thing to watch out for though - this will only work on a machine with the SDK installed. The D3D10CreateDevice call will fail on a machine without the debug layer installed if you use this flag. This is documented, but you have to look around for it.
Since the work on Microsoft Max wound down, I have been working on things that I cannot discuss. Sorry, that is how it goes sometimes.
I have been working a lot with D3D10 lately, so I will endeavor to get more tips about it out as I find them.
I am playing with D3D10 tonight, and I am trying to get D3DX10GetImageInfoFromResource to work for me. I have beaten on it for a long time, and it still resists my powers.
On a whim I write some code to call FindResource myself to make sure that I can find the resource and it works, but that makes me think about the resource types on this page:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winui/winui/WindowsUserInterface/Resources/IntroductiontoResources/resourcereference/resourcetypes.asp
And then I remember that some other D3D resource functions like resources to be in RC_RCDATA format. Lo and behold, this is the problem - I had a bmp file as a resource, but it was marked with RC_BITMAP!
Lesson learned - D3D loads RC_RCDATA resources, even when they are bitmaps. I hope this saves somebody some time one day :)
One of the cool things about dependency properties is that they allow other things to be dependent on them and notified of changes to them (hence their name). However, one thing that people will sometimes stumble upon is that they want to know about a change to a particular property value but WPF does not expose this event.
Every time I have found the code for this I have thought "Cool!" and then a couple of months later I forget it again. Now that I don't write WPF code all day every day anymore (sniff) I have even less of a chance of remembering it. So I post it here for myself to find later and as a gift to anybody who reads it.
The trick is to use DependencyPropertyDescriptor. If you have the DependencyProperty already (normally a static member of the class, such as WidthProperty) then you can call DependencyPropertyDescriptor.FromProperty and pass that in. Once you have the DependencyPropertyDescriptor you can call AddValueChanged to subscribe to the change event.
Now Bob's your mum's brother. Remember though, for your own properties you can register a change event handler in the metadata yourself to avoid writing this.
After our last installment, which was written many moons ago (literally), the only remaining thing left to do (apart from clean up the code a bit) was to implement the IScrollInfo.MakeVisible method. Somebody has requested that I address this, so I am going to attempt to fulfill that.
The idea of MakeVisible is that it is called to ensure the visiblity of one of the children in the Panel. When will this be called? Well, if you fire up the AnnoyingPanel sample that has been written in the past three chapters, and click one of the buttons, you will notice that the application will crash because of the exception we are firing from MakeVisible. Pushing the tab key will also have this result. This makes sense - when you push a button or tab to it, WPF will ensure that the button is visible so that you can see what you have tabbed to or clicked on.
So how do we implement the method? The method takes a Visual and a Rect. The Visual is the thing that we need to make visible - it will be one of the children of the Panel. The Rect is the rectangle of the Visual that needs to be visible. The return value, another Rect, is the rectangle of the Visual that you managed to make visible.
So to implement this method for our sample should be fairly simple. We will search our internal child list for the Visual to see if it is there. Based on where it is in the child list, and the size of the children, we can calculate the offset that we need to scroll to in order to have the child at the top of the viewport. Then we can set the new vertical offset, and return the rectangle we were given, since we know that the children are always smaller than the viewport.
public Rect MakeVisible(Visual visual, Rect rectangle)
{
for (int i = 0; i < this.InternalChildren.Count; i++)
{
if ((Visual)this.InternalChildren[i] == visual)
{
// we found the visual! Let's scroll it into view. First we need to know how big
// each child is.
Size finalSize = this.RenderSize;
Size childSize = new Size(
finalSize.Width,
(finalSize.Height * 2) / this.InternalChildren.Count);
// now we can calculate the vertical offset that we need and set it
SetVerticalOffset(childSize.Height * i);
// child size is always smaller than viewport, because that is what makes the Panel
// an AnnoyingPanel.
return rectangle;
}
}
throw new ArgumentException("Given visual is not in this Panel");
}
If you build this and run it then things will look the same, but now when you click or tab to a button it will be forced to the top of the viewport (or as close as it can get to there, anyway). So now we have a minimal working version of MakeVisible.
There are several issues with our implementation, and they are things to think about if you are implementing this for real in a real world application:
- Our implementation is repeating the calculation of child size again. We have this code littered throughout the sample in many places. I have done this to try and make it easier to grok what is going on, but this stuff should really be in helpers to avoid bugs from having it being done slightly differently in places.
- We are assuming that the only difference in the coordinate system of the Visual and the Panel is the position. WPF allows much more interesting transforms, and I am unsure at this time what the input and output should look like. This is an interesting experiment for somebody to try one day.
- Our calculation of the rectangle that we scrolled into view is simple, because we always try to move the child to the top of the viewport, and the children are always smaller than the viewport. This shortcut might not apply depending on how your Panel works, and you may need to do some calculation to figure out how much you are showing.
- We are always changing the vertical offset, even if the child is already visible. This is a distracting way to implement things - the user does not expect the child to move around when it is already visible. A real implementation would look at the offset, and decide if any scrolling is needed or not. If scrolling is not needed, but the child is partially shown, then the returned rectangle would need to change as well.
All of the four things above are incremental, and typically change depending on your scenario, which is why I am not being too specific about what to do or demonstrating it. IScrollInfo is an extension point designed to allow a maximum of flexibility, and there is a huge number of ways that you can use it. All that I am trying to do is let you be aware of the possibilities.
I have attached the final AnnoyingPanel.cs file for you to look at if you want all of the code. You may need to change the namespace to use it in your app, but I figure that this small implementation might be a starting point that somebody would want to use.
As we can see now on the home page, the Max project has ended. It was a real blast, and I had the opportunity to learn a lot about Avalon (now WPF) and to create a really cool product.
As the project closed, I decided to make a move in the company to where I can continue to work on cutting edge graphics and user interfaces, namely the Windows Client team. As Vista has been completed, the team is busy figuring out what the next version of Windows will look like, and this is an exciting team to move over to the team and help with that figuring out.
Just because Max does not work anymore does not mean that I will not be posting about WPF or answering questions about it. I have been slack with my posts lately with all of the moving around and goings on, but I hope to get some answering done today.
Note: This is *really* late in being posted, but I had work to catch up on when I got back :)
The last day of SIGGRAPH 2006. I divided my time between papers and sketches, so there was a lot to push into my already overloaded and sleep deprived brain at this stage. But I gave it a college try.
The first paper that I caught was most of the "Streaming Computation of Delaunay Triangulations" paper. The main advantage of this approach was being able to do both the input and output of the triangulation as a stream without large intermediate storage, and a modest memory requirement. The algorithm took advantage of the typical locality of the input data and created a system for figuring out when partial quadtrees of the input data were done. The results of the time and memory taken for doing very large sets of data were very impressive. One thing that stuck in my mind about this was seeing a data set from an aeroplane flying over some area, and the comment from the presenter that the pilot forgot to turn the data collection off. Even with this issue hurting data locality the algorithm handled things well.
The next paper in the session that I caught was "Spectral Surface Quadrangulation", which presented a technique for taking an input mesh and creating large quad patches that could be further divided to create mostly undistorted quads across the whole mesh. I have read some papers of this kind of thing before, but never in huge detail. The approach here was to create a function that had maxima, minima and saddle points, and to select a function that created these extremes in appropriate places. After this the points are refined with various techniques and quads can be generated.
After the morning break I caught "Simulating Multiple Scattering in Hair Using a Photon-Mapping Approach". I have never looked into hair rendering before, but from what I gather the scattering is quite directional with hair and hence typical methods suffer from a very small quantity of samples, making even blonde hair look dark. The approach shown in this paper deposits photons uniformly along path lengths, rather than at interaction points. This effectively has photon density mapping directly to radiance, which is what is needed to be calculated. The results are very interesting - especially simulating hair viewed in front of a light.
Following that was "Statistical Acceleration for Animated Global Illumination", which demonstrated a mechanism for sampling temporally as well as spatially for global illumination. Since the changes in illumination are very small from frame to frame (as well as from pixel to pixel in most cases) and you need to render every frame anyway, this approach gives both better looking results as well as faster rendering. The paper also showed how dramatic changes in the scene geometry or lighting can be handled as well.
Next up was "Multidimensional Lightcuts", which showed how to extend the lightcuts algorithm to handle effects such as motion blur. Doing effects like this effectively adds another dimension to the lighting equation integral, and this paper showed how to use similar techniques to the original lightcut algorithm to bring the discretized version of the integral into a small enough set of samples to calculate the lighting effectively.
The final paper before lunch was "Direct to Indirect Transfer for Cinematic Relighting". This showed a system that assumed that the camera was fixed, and then calculated the direct to indirect transfer matrix for the scene at a viewpoint. The transfer matrix is compressed in a GPU friendly way using wavelets, which allows for the scene to have completely different lighting at interactive rates. Even though the viewpoint and geometry are fixed, this still shows a very effective way to relight a scene.
After lunch I decided to round out the SIGGRAPH experience with a couple of sketches. The first sketch session was titled "Fast & Cheap", which sounded great because I love real-time and interactive stuff, and to do that everything must be fast and cheap. The first presentation was about "Fast Approximation to Spherical Harmonics Rotation", which presented a fast and simple way to approximate a rotation in spherical harmonic space. At the sketch it was admitted that a previous method is actually about as fast (which was only discovered right before the conference) but the mathematics employed were very interesting regardless. The basic idea was to decompose the transform into smaller, simpler transforms (rotations around Z are cheap and easy) and then to approximate the non-simple transform (apparently Y rotations are very yucky). It is interesting to see here something which I think happens often in graphics - that expanding a simple formula into a big complex one sometimes makes it much faster.
Speaking of complex mathematical constructs, the next presentation in the sketch (Real-time All-Frequency Relighting in Local Frame) used some very interesting constructs in order to achieve their impressive results. BRDFs are stored in the local frame and compressed using a spherical wavelet approach. Since the BRDF is in local frame, effects such as bump mapping can be done more easily. The lighting is also done per-pixel, using a visibility texture for shadowing. As with many other approaches I saw at SIGGRAPH there is a pattern here of very targetted precalculation and mathematical transformation enabling things to be done in real time that would have been unthinkable five or ten years ago (by me, anyway).
After this Sing Bing Kang from MSR presented a technique to render rain in real time using the GPU. After seeing the ATI presentation on a similar topic it was interesting to contrast the approaches. This approach worked by first using some image processing techniques to extract rain from video, and then uses a PRT based rendering approach where raindrops are assumed to be spherical and non-interacting to render the drops. This is a less memory and shader intensive technique than the ATI one.
This concluded the last day for me. The whole conference was a blast, and it exceeded the expectations of what to expect from the years of wondering what it was all about.
The first thing that I saw on day four was the paper "Photo Tourism: Exploring Photo Collections in 3D". It demonstrated a system for taking a collection of images that were taken in a similar area (such as in a town square), extracting features common across the photos, and using those features to estimate the position of those features in space.
An application for efficiently browsing this space was also presented. This technology can be seen in Photosynth, from Live Labs. This is an awesome technology, and it combines several different aspects of image processing and 3D rendering.
At this point I was interested in seeing a sketch session (basically a mechanism at SIGGRAPH for showing work in progress) about unstructured progressive graphics. I caught the end of a presentation that looked interesting about real-time multi-perspective rendering on graphics hardware, which was seemed to combine several viewpoints of the scene dynamically to produce some effects such as reflection that are difficult to recreate in real time. I have wondered about using this approach before - it seems to have the bonus of doing rendering the way that GPUs want to do it, which is a good way to make things fast.
There was also a presentation on compressing dynamically generated textures on the GPU. This was a very novel technique for essentially doing the compression of S3TC on the fly on the GPU to get 6x compression. I think that techniques like this will become very handy when DX10 is commonplace, because DX10 has more support for changing the interpretation of data on the fly, as well as integer operation support.
- I also tried to see more of the exhibition on this day. There are a lot of exhibits to cover, but I did get to spend more time looking over various things. Some interesting highlights in a totally random order:
- Seeing a *huge* multitude of displays all showing volumetric medical data.
A system for combining multiple low resolution projectors into a larger high resolution display, without having to precisely align things.
- Lots of cool 3D printers.
- Lots of high quality car renderings (ray tracing + modern CPU power == photorealism)
- Many different 3D modelling packages.
After lunch there was a sketch session called "In the Shadows". I was keen to attend this because I have done a lot of reading about shadows in real time, and it is an especially hard problem. The first talk in the sketch presented a method of measuring what shadows are perceived more than others (using real people looking at large quantities of images) and using these metrics to decide whether to render high or low quality shadows across a scene. It was refreshing to see human perception measurement applied to the problem.
Another talk in this sketch was about using real-time shadows with cone culling, by Louis Bavoil and Claudio T. Silva. The basic idea here was to do a shadow map pass, and interpret the pixels in the shadow map as spheres. The volume between the shadow map spheres and the light can be thought of as a cone in space, and by adding up the contributions of all of the cones. The technique is fast enough to run on a GPU, and can give plausible soft shadows.
In the afternoon there was the precomputed transfer set of papers. It seems that the variety of techniques that use PRT in some fashion multiplies more and more every year. I am personally going to research this more because of the power of the techniques. For those who are not familiar with PRT, it leverages 2D -> 1D signal transforms to transform and approximate 2D signals (such as environment maps and radiance transfer maps) with 1D vectors. For those familar with fourier transforms, imagine taking an input signal, taking the fourier transform, and removing the higher frequency contribution. In this way you can represent a lower quality version of the signal with fewer numbers than the original number of samples. The common thread in most precomputed transfer techniques is to preprocess some quantity of the transformation, and to reverse the transformation after doing lighting calculation in the transformed space.
The first paper presented was "Real-time BRDF Editing in Complex Lighting". The idea of this technique was to fix the lighting and the camera, but to vary the BRDF. This changes the typical assumptions that are made with the rendering equation, and a cascade of optimizations follows from the initial assumptions. Even more interesting was the notion that this technique was incremental, so changing only a small number of lighting parameters at once increases the framerate of rendering. Since in editing scenarios people typically only change one parameter at a time, you can interactively edit the material of an object with realistic all frequency lighting.
The next paper was "Generalized Wavelet Product Integral for Rendering Dynamic Glossy Objects". The mathematical constructs in this paper appeared very complex, but the results were very interesting. The technique computes illumination per vertex, and has precomputation transforms done using wavelets and Haar integral coefficients. I need to re-read this paper in more detail to understand the transform mathematics better.
After that a paper was presented titled "All-Frequency Precomputed Radiance Transfer Using Spherical Radial Basis Functions and Clustered Tensor Approximation". This paper showed a combination of using spherical radial basis functions (SRBFs) for the transformation and representation of lighting data, with a compression technique called "Clustered Tensor Approximation" to compact the data down to a reasonable size. The Efficiency of the SRBFs in representing the lighting environment was particularly impressive, meaning this paper is due for a second read as well.
The last paper that I saw on the day was "Real-Time Soft Shadows in Dynamic Scenes Using Spherical Harmonic Exponentiation". The demonstration of what this technique could achieve was very impressive. The idea was to use a collection of spheres as an approximation for the blockers, and to accumulate the effect of the blockers in log space to make the calculation more efficient. For applications already using precomputed transfer, this provides a solid technique for adding soft shadows to the scene.
Today I tried to get an early start on the day by avoiding the infamous Boston Big Dig inspired traffic and walking to the convention center. It was a great route that took me past the Boston Common and through Chinatown, and past a very Bostonian train station (South Station). I don't know if the walk was faster than waiting for the shuttle, but I did make it to the first paper talk of the morning.
The first paper was "Real-Time GPU Rendering of Piecewise Algebraic Surfaces". This is a paper by Charles Loop and Jim Blinn that builds on a similar theory to their 2005 paper "Resolution independent curve rendering using programmable graphics hardware" to render surfaces on the GPU. Personally, I think that this is one of the most impressive developments in GPU use that I have seen. This paper shows how the triangles given to the GPU can be used as control points for algebraic surfaces (such as a cone or sphere) and hence have the GPU render them through something like ray casting and root finding. The most impressive part of things is that the curves are evaluated at the resolution that they are rendered at, thus removing subdivision entirely from the pipeline of curve rendering. The only big problem that I see (which was mentioned in the talk) is the lack of tools that can output these surfaces, but I hope that some tool vendor picks up on that at some point.
The paper following that was "Point-Sampled Cell Complexes". As with the Loop/Blinn paper, this shows a method of representing surfaces. The novel thing about the representation is that it allows for point samples to represent surface samples, curve samples or vertex definitions. The algorithms presented join the points together along the given curves, and join surfaces together between those curves. I think that this was very interesting because it allowed for modelling some things that traditional techniques find challenging, such as cutting an arbitarily shaped hole into a surface. One thing that I want to read more about with this paper is if the representation method was designed with real time evaluation in mind, because some of the attributes of the algorithm look like it might make that difficult. But then again I have been surprised before.
After these papers I decided to join the course "Advanced Real-Time Rendering in 3D Graphics and Games". This course looked very practical, and I noticed that Natalya Tatarchuk was chairing the course and doing some lectures there based on the Toy Store demo that ATI have done. I have read a bit about the Toy Store demo before so I thought that it would be good to hear the material covered by the people that worked on it.
The first lecture that I saw was from Alex Evans. He presented some techniques that he put blended together to create fast approximations of global illumination on dynamic scenes. This was a great talk - Alex covered some of the experimental ideas that he played with and explained what he did with them, even if they were just ideas and did not go anywhere. I liked the philosophy that he extolled around combining ideas from many different places to tackle problems that have specific constraints. I can tell that Alex loves volume textures, and several of his techniques involved using them in one way or another to try and store volumetric lighting information of some kind. After seeing this I want to play more in this area, because it looks like there are some powerful things that can be done here.
After Alex's talk Natalya Tatarchuk talked about the parallax occlusion mapping done in the Toy Store demo. I have read about this before, but you get so much more depth when you have the person to go with the slides. The basic idea is to store a height field with a normal map (in the alpha channel, for example) that allows you then do some rudimentary ray casting style calculation to figure out what you are hitting. While the idea sounds basic, Natalya had a lot of details about tuning that was done to remove artifacts and increase the quality of the rendering. This was good to see because it makes the difference between something that is an interesting experiment and something that you can use for real.
After lunch Jason Mitchell from Valve talked about the Source engine, and discussed how shading works in it. Some interesting aspects of the engine that were discussed included varying light map density across a scene, storing light from a part of the environment at a particular point for use later, how they do HDR with an RGBA back buffer, and how they use histograms of light intensity to simulate how your eye adjusts to different lighting conditions. Of particular interest was how pretty much every feature had some artist controlled component in their tools. I think that this is very important, because it allows the people with the pretty sticks to beat the content into submission more easily.
The session continued with Natalya again, this time discussing how the water effects in Toy Store were done. There were a lot of different things covered here (she mentions that they had about 300 shader routines for all of the water effects). She showed how they did rain, how they used various particle simulation systems on GPU to calculate water flow, the way that droplets of water were modelled with sprites, the usage of fins (similar to how fur is done) to do haze on the taxi. The detail of the demo is incredible - they even have the water dripping down the glass store front, and these water trails leave shadows and pretend caustics on the toys behind the window. It was interesting that they used an old movie trick (mixing milk with the water for rain scenes) to solve the same problem in a virtual world as the problem in the real one.
In the afternoon the opening talk was from Chris Oat, who presented a method for computing lighting for static meshes with dynamic area light sources. This was demonstated with a potential application, which was landscape lighting. The idea of the approach was to precompute visibility data for the mesh (hence the static mesh restriction) and then use this information at render time together with spherical cap intersection calculation to estimate the amount of the area light that illuminates the point.
After this Carsten Wenzel from Crytek gave a talk on real time atmospheric effects in games. There were many different techniques presented, and most of them built on the technique of using a scene depth buffer rendered before the scene is rendered (in a similar fashion to deferred shading) to get fast access to depth results. Some of the techniques shown included sky light rendering (done on both CPU and GPU, with a novel scheme for calculating new sky data across multiple frames), volumetric fog, locally refined fog and soft particles. This was a great demonstration of the work that Crytek has done, and I really appreciated how Carsten came to share this at the conference.
In the afternoon I checked out some of the exhibition. The exhibition was pretty big, and there was a wide variety of stuff there from animation schools, animation studios, 3D editing tools and rendering tools. It seems like there are a lot of people doing stuff with high quality renderings of cars. Several companies have photorealistic rendering of cars down pat, from my eye.
The second day was very different to the first, with some papers being presented, a trip through the art gallery and the animation theatre at night.
Due to some traffic snarling, I missed the first couple of papers in the morning (a situation that I rectified the next day by walking past the Boston Common to get to the convention center instead of taking a bus). I did manage to pick up two paper presentations from the morning session, though.
The first one was "A Spatial Structure for Fast Poisson-Disk Sample Generation" which showed a very good technique for generating Poisson sample sets. This was great to see because I have had the need to generate a Poisson set before (it comes up all the time in computer graphics, and even more in real time graphics these days with the increasing GPU complexity) so seeing a better method of generating the sets was very interesting.
The next paper was "Recursive Wang Tiles for Real-Time Blue Noise" which showed an interesting technique for generating blue noise of varying densities that could be arbitarily zoomed. The combination of techniques in the paper to achieve the result was interesting, and the method showing for combining blue noise sample sets was novel. I want to read more about this later just to understand how the components of the paper work.
At this point I met up with a friend who is here for the conference as well and we checked out the art gallery some. There was a mix of static prints, videos and stuff built from scratch that was all very interesting. One of the coolest things there was something that used some kind of oil saturated with iron particles and manipulated with magnets. Some of the renderings shown on the screens here were very impressive too.
After I checked out the gallery I went to see the presentation of "Perfect Spatial Hashing" which was a great paper that showed a technique for compressing sparse texture data with a hashing. The compression process looks too slow for allowing modification of the distribution of texels, but it does allow for changing the texel values themselves. What was especially cool was that the decompressing of the texture can be done later with a pixel shader, so this technique can be readily applied to real-time rendering, and allow techniques that would otherwise waste a lot of texture memory. Considering how valuable texture memory is these days this is an important property of the work.
In the afternoon I attended the image manipulation paper set. The first of the papers was "Color Harmonization", which showed a technique for taking an image and shifting the colors to make the colors more harmonious. It was interesting to see the work on attempting to formalize color harmony, and using that to create the algorithm.
The next paper was "Drag-and-Drop Pasting", which showed an application that was able to take a very roughly outlined object from one image and drop it into another image. The application could find the detailed outline of the image (with alpha, from what I could tell) and merge it with the target image in a way that showed no seams. Some of the examples were very good - this kind of thing could put some photo-chopping people out of work :)
After that they showed "Two-Scale Tone Management for Photographic Look" which demonstrated a technique for taking the 'look' of one photo and applying it to another photo. Again, in this case the components of the technique were as interesting to me as the end result itself. There were some very cool image transforms in this that could be probably applied to other problems in a similar area.
Next up was "Interactive Local Adjustment of Tonal Values", which showed a method for identifying regions of an image and performing local adjustments of the image tone in those regions. This allowed for doing things like changing just the tone of the sky in a photo, for example, without changing the ground. The identification of the regions that you want to modify is way easier than masking things yourself using a lasoo or something similar, and a comparison of using this technique instead of a conventional photo editing program shows somebody with no lasoo skills able to do better work in way less time.
The last paper of the day was "Image-Based Material Editing", which showed a way to take a photo, choose an object in the photo, and then change the material of the object. The algorithms shown could identify the shape of the object, estimate the content of the photo behind the object, estimate the lighting of the object, and then use these estimations to replace the material of the object. Some examples were changing something from porcelain into metal or glass, for example. Despite the amount of information being estimated (or created from nearly nothing) the effect was quite convincing.
Later on at night there was the animation theatre. This was basically a collection of shorts from a large group of people. There were short films made by independents as well as short documentary style clips from studios showing how some of their work was done. One of the coolest ones was "In a New York Minute" by Weta (mentioned here). 458nm and One Rat Short were very very cool, and looked especially good on the 4k SXRD projector that they were using.
Onto the next day...
This is the first of my posts describing my experiences at SIGGRAPH 2006. I will try to have a post every day summarizing what I see. Everything here is my personal opinion.
The first day of the conference is a little more low key than the other days, by the look of things. The exhibition is still being set up (you can see it when you walk across bridges) and there are no paper presentations. There are courses, however, and given that there was not much else going on I decided to go to one to get my learning on.
The course I chose was this one. Basically there were a bunch of different lectures from people all centered around doing shading and rendering on the GPU. I really think that this area is still growing really fast and this was a great course for hearing about the history of the area and different technologies and techniques.
The first lecture was from David Blythe, a fellow Microsoft guy who talked about DX10 and how the architecture differs from DX9, and what the motivations and improvements are. This was very informative - I have read some slide decks and tutorial code before, but having a talk targeted around explaining the differences really helps. One interesting thing about the talk was that it also helped understand DX9 more through the explanations of where the frustrations and bottlenecks are. I knew that state changes were expensive and had some idea why, but the slide deck was good about explaining how the bottlenecks relate to GPU architecture. From what I can tell the DX10 design is a lot cleaner, and has the potential to free up a lot of CPU time by reducing how much has to be shunted between the CPU and GPU. I can't wait until I get a chance to play with it some - geometry shaders look like they will open up a lot of algorithms that you could not do before.
The next talk from Michael McCool discussed RapidMind, which is a system for letting people write general purpose parallel programs, of which shaders are a subset. Some of the renderings that he showed were very impressive, and the idea that you can write your algorithms once and have them run on a variety of hardware and architectures was very cool. What was interesting was hearing about applications beyond just rendering, such as flocking algorithms. I love seeing where shader programs are doing things other than shading (I have seen a paper about doing sorting using a GPU, for example).
Next Marc Olano talked about the OpenGL shading language. This was interesting to me because I have written shaders for DX9, and I have written OpenGL code before, but I have never done shaders in OpenGL and I was curious to see how it is done. I think that the way that you can bind data from the OpenGL state to the shaders through the variable names like glColor was very elegant. I was also glad to see that for the most part the shaders looked just like their DX counterparts. It is nice to see that the simularity in shading languages allows for easy transfer of algorithms between them.
After lunch Mark Kilgard from NVIDIA talked about Cg as well as giving an overview of the development of NVIDIA hardware, starting way back more than 5 years ago. There were a lot of graphs demonstrating the increase in performance over the years, as well as breakdowns on where the performance was coming from (clock speed, number of shader units, etc). There were also some comparisons with current and soon to be CPUs from Intel to give an idea of the difference in speed (which is 1 or 2 orders of magitude in favor of the GPU, depending on how you measure things, apparently). Mark also gave a bit of an overview of Cg, and what I found interesting was the fact that this language could target multiple APIs. The idea that you can write a shader once and use it with both DX and OpenGL is pretty compelling.
Next up was Thorsten Scheuermann from ATI, who talked about using render to vertex buffer (R2VB) as a way to enable you to use the pixel shader hardware for doing vertex shading. Since the pixel shader is quite often more powerful with a richer instruction set, this allows for some things to be done much faster than using the vertex buffer. The examples that Thorsten showed involved skinning mesh animations with around 10000 instances of an animation. NVIDIA hardware does not support this operation (apparently) but you can use vertex textures to do a similar technique with NVIDIA hardware. What I liked about the presentation was seeing familiar things used with a bit of a twist. I find that seeing more things like this allows you to be more creative with similar constraints in the future.
At this point there was a break, and I went to another course on Procedural modelling of Urban environments, since the shading and rendering course took a path into the non-realtime world at this point. I actually regret doing this - even though I am not involved in production rendering, a lot of the same problem spaces are explored and the technology is very similar. When I went back later for the last Q&A session I caught the end of a talk from Fabio Pellacini which looked very compelling.
The Urban environment course was still interesting however. Pascal Mueller talked about a tool that he has developed for taking various forms of data (such as GIS information) and combining them with some algorithms he has devised in order to procedurally generate models of cities. The end results look amazing to me - considering the work that would be involved in building a detailed 3d model of a metropolis, the fact that you can generate so much modelling data from such small inputs is incredible. Some examples of applying the software included modelling cities where the only data was small amounts gleaned from an archeological dig.
Ben Watson also presented a technique for creating urban models (up to the street and parcel level) from nothing (or at least from very small inputs, depending on the desire of the user). This differed from Mueller's work in that it was not for making a model of an existing place. This was also amazing to me, because coming up with psuedo-random data sets that fit some characteristic (such as looking like a typical city map) is something that I have tried to do without success before. What I found quietly amusing was when I looked at a screenshot and thought "That looks like SimCity" and then he mentioned doing work with Maxis and using their engine.
The content of the Urban Environment talks differed from what I expected. Reading the description of the course now it seems obvious, but I was expecting more detail about how to render a dataset like a detailed city model in real time very quickly. It is interesting how our own biases cause us to see or expect things - obviously the problem of rendering the set is more interesting to me than the problem of creating it (mainly due to my lack of aptitude in the latter).
The last thing that I saw was the Fast-Forward papers preview. This is basically a form of torture for people presenting papers at SIGGraph - they make some representative party of each paper do a 50 second talk about their paper. This is a great idea - as well as being entertaining, you almost get a 2 hour preview of every paper at the conference (which is no mean feat considering how many papers there are). Less than a minute is often enough for most things for you to decide if you want to go, so it is very useful. There is so much content at this conference the problem is knowing how to sift through it and not miss anything.
I have posted before about the philosophy behind the ItemsControl, and now I will pull together some concepts from previous posts. I will use the RadialPanel and some ValueConverter magic to show how we can take a ListBox (the most famous of the ItemsControl family) and have it:
- Respond to changes to a data collection.
- Swap between using a radial and stacking panel.
- Swap between showing the text of the items and showing the hash of the items.
One of the first things we need to do in order to build this sample is make a subclass of the RadialPanel that does not require the items to have the angle set on them. We do this because we want the items to arrange in a neat circle by themselves.
The simplest way to do this is to derive from RadialPanel, and override the MeasureOverride method to set the angles on the children to be evenly spaced before doing the base measure pass. This is very straightforward:
using System;
using System.Windows;
namespace ItemsControlExample
{
public class ItemsRadialPanel : RadialPanel
{
protected override Size MeasureOverride(Size availableSize)
{
double angle = 0.0;
double spacing = (360.0 / this.InternalChildren.Count);
// set the angle of the children based on where they are in the list
foreach (UIElement element in this.InternalChildren)
{
SetAngle(element, angle);
angle += spacing;
}
return base.MeasureOverride(availableSize);
}
}
}
Now we have a panel that we can use in an ItemsControl. Another piece of the puzzle that we need is a data converter that takes any object and returns the hash of it:
using System;
using System.Windows;
using System.Windows.Data;
using System.Globalization;
namespace ItemsControlExample
{
public class TextToHashConverter : IValueConverter
{
public object Convert(object value, Type targetType, object parameter, CultureInfo culture)
{
return value.GetHashCode();
}
public object ConvertBack(object value, Type targetType, object parameter, CultureInfo culture)
{
throw new NotSupportedException();
}
}
}
As far as value converters go, this is a simple one. But it is enough to demonstrate the idea. Now we have most of the code for building our control. Let's build a window in XAML that has the pieces we need to show it off. First we need a window with a resource section:
<Window x:Class="ItemsControlExample.Window1"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:local="clr-namespace:ItemsControlExample"
Title="ItemsControlExample" Height="300" Width="300"
>
<Window.Resources>
</Window.Resources>
</Window>
What do we put in the resources? Well, first we need to have an instance of the converter that we will be using later. So we declare that in the resources:
<local:TextToHashConverter x:Key="TextToHashConverter" />
Now we can define some control templates for the ListBox. One of them will use the radial panel:
<ControlTemplate x:Key="radialTemplate">
<local:ItemsRadialPanel IsItemsHost="True" />
</ControlTemplate>
And the other will use a stacking panel:
<ControlTemplate x:Key="scrollTemplate">
<ScrollViewer ScrollViewer.HorizontalScrollBarVisibility="Disabled">
<VirtualizingStackPanel IsItemsHost="True" />
</ScrollViewer>
</ControlTemplate>
Now we can define some data templates for the ListBox. The DataTemplate is what is used to create UI for each piece of data in the ListBox. The first template is simple - we center the text of the item inside a Border element. The binding statement has nothing in it because that will return the DataContext of the element, which is the item:
<DataTemplate x:Key="textTemplate">
<Border Background="Red">
<TextBlock HorizontalAlignment="Center" VerticalAlignment="Center" Text="{Binding}" />
</Border>
</DataTemplate>
And now we can also define the other template, which will just have a different converter and no background:
<DataTemplate x:Key="hashTemplate">
<TextBlock HorizontalAlignment="Center" VerticalAlignment="Center" Text="{Binding Converter={StaticResource TextToHashConverter}}" />
</DataTemplate>
Now we all we have to do is to declare our ListBox in the window. We will make a DockPanel with a bunch of buttons along the top so that we can change stuff:
<DockPanel>
<UniformGrid DockPanel.Dock="Top" Rows="1" Columns="5">
<Button Content="Add item" Name="addButton" />
<Button Content="Radial" Name="radialButton" />
<Button Content="Scroll" Name="scrollButton" />
<Button Content="Text" Name="textButton" />
<Button Content="Hash" Name="hashButton" />
</UniformGrid>
<ListBox Name="listBox" Template="{StaticResource scrollTemplate}" ItemTemplate="{StaticResource textTemplate}" />
</DockPanel>
As you can see, we started off with the normal text template and the normal scrolling template. But if we run our code now we will have nothing in the ListBox and the buttons will also do nothing. We need to add some code to fix that. First of all we can add some initial content by adding a member to the window class which will be our collection:
/// <summary>
/// We use ObservableCollection because it will fire events to let the ListBox keep things in sync.
/// </summary>
ObservableCollection<string> _strings = new ObservableCollection<string>();
And adding some code after InitializeComponent to put some initial content in there and wire up to the ListBox:
// add some strings to start with
_strings.Add("string 1");
_strings.Add("string 2");
// wire up the string collection to the ListBox
listBox.ItemsSource = _strings;
All that is left now is handling events on the buttons. First we add some handlers in the window constructor:
// wire up handlers for the buttons
addButton.Click += new RoutedEventHandler(OnClickAdd);
radialButton.Click += new RoutedEventHandler(OnClickRadial);
scrollButton.Click += new RoutedEventHandler(OnClickScroll);
textButton.Click += new RoutedEventHandler(OnClickText);
hashButton.Click += new RoutedEventHandler(OnClickHash);
And now we can write the handlers. The handler for add is very simple - we just add a new item to the collection. Notice that we don't have to tell the ListBox about it explicitly - ObservableCollection will fire an event to tell the ListBox that it has changed and the ListBox will automatically create the UI for it and display it. This is why ItemsControl is so cool:
/// <summary>
/// When we click add, we want to add an item to the end of the list.
/// </summary>
private void OnClickAdd(object sender, RoutedEventArgs e)
{
int stringNumber = _strings.Count + 1;
_strings.Add("string " + stringNumber.ToString());
}
Now we can add handlers for changing the data template. They simply set the appropriate property to the appropriate resource:
/// <summary>
/// When we click on the hash button, we show the hash data template
/// </summary>
private void OnClickHash(object sender, RoutedEventArgs e)
{
listBox.ItemTemplate = (DataTemplate)FindResource("hashTemplate");
}