Robotic Pandas

Designing a GPU-oriented geometry abstraction – Part Two.

2009-11-21T04:39:32Z

My last post described the problem of crafting an appropriate geometry abstraction for Bling. Bling previously solved the code problem for vertex and pixel shading, but lacked decent geometry input abstractions as well as an abstraction that supported geometry shading. The last post proposes giving geometry its own abstraction, but I was a bit hesitant in including properties within the geometry, which leads to composition and addressing problems. I think its time to back up a bit and describe the abstractions that are relevant to a traditional programmable rendering pipeline:

Pixels as children of geometric primitives. The position of a pixel is fixed and a pixel shader only specifies each pixel’s color and optional depth. It often makes sense to have other properties beyond color and depth at the pixel level for better shading computations; e.g., a normal that is computed on a per-pixel basis via a parametric equation or via normal mapping. Other properties can be accessed from previous vertex and geometry shading phases, where interpolation is used to determine the value of the property from the value of the property in the enclosing primitive’s vertices. Quality improves when properties are computed on a per-pixel basis rather than interpolated; e.g., consider Phong vs. Gouraud shading.
Vertices as used to define primitives. A vertex shader specifies the position of each vertex, and will compute per-vertex properties that are later used in later geometry and pixel shading.
Primitive geometries that are used to form complete geometries. A primitive is either of a point, line, or triangle topology, with triangle being the primitive that most commonly reaches the rendering stage. A primitive is defined by multiple vertices, and enclosing multiple pixels. Geometry shading works as a primitive translator, where one primitive can be translated into zero or more primitives of possibly a different topology. Primitives during geometry shading are defined as vertices.
A geometry as a collection of primitives. Its initial definition is as a set of vertices, a topology for the vertices, and optionally an index buffer to allow for arbitrary vertex sharing between primitives, or an implicit adjacency relationship with explicit breaks (also used in geometry shading).

As described in the last post, we can define an abstraction for geometry that supports composition, duplication, and transformation. Ideally, rendering could then involve forming a geometry in a clean functional way through multiple compositions and transformations, and passing the resulting geometry into a render command. Transformations not only include modifying the layout of the geometry by rotating, scaling, and translating it, but also applying color and lighting to the geometry and the pixels contained within it, or whatever else is required for a complete rendering specification. Lighting could even be applied to the constituent parts of a geometry before they are composed. The various properties needed to make this possible include thinks like diffuse, specular, glass, and refractive materials as well as additional non-geometry constituents such as directional, ambient, and spot lights. Essentially, the geometry would then become a mini-scene graph.

Scene graphs are common in retained graphics APIs such as WPF 3D and Java 3D. Basically, a scene graph is a graph of the elements that affect scene rendering, and then becomes the basis for what is shown on the screen. Since we are only interested in what can be efficiently rendered in a shader pipeline, we have to keep the graph nodes mostly homogeneous: duplicating and transforming a geometry in the graph is fine, but composing completely different geometries with different lighting schemes is probably not going to work in the context of one rendering call (instead, render the geometries in separate rendering calls).

The primary difference then with my previous geometry abstraction is that this new geometry embeds properties and transformations at every level of composition. A level in a geometric composition can omit or duplicate properties from a higher level, and transformations (e.g., lighting or layout) should input a property from the lowest level that it exists in the geometry that the transformation is attached to; e.g., a normal property at the pixel level is preferred from a normal property at the vertex level as it is more accurate for lighting computations. The problem with this approach is that we can lose static typing: a property might not exist at any level, then what? Right now I’m willing to live with dynamic checking since it will occur relatively early in Bling when code is generated at application startup.

Designing a GPU-oriented geometry abstraction – Part One.

2009-11-20T04:46:16Z

One the inputs of rendering via programmable shading on a modern graphics card is a collection of vertices associated with some per-vertex properties used in shader computations. When programming the GPU, this collection of vertices is commonly abstracted as a vertex buffer, which is essentially just a bag of bytes. The collection of vertices describes a primitive point, line, or triangle topology along with an optional index buffer (if triangle) that describes vertex sharing. Again, the abstractions for describing these topologies are very weak and essentially amount to flags and arrays, although we can informally describe to the vertex buffer plus its primitive topology description as a geometry. Each vertex first individually undergoes processing in a vertex shader, and then (optionally) per-primitive (point, line, or triangle) in a geometry shader where a new geometry can be specified with fewer or more vertices or even a completely different topology. After vertex and geometry shading, the position of each vertex must be specified. Finally, each pixel of the rasterized primitive is processed by a pixel shader to determine its color. This is a simplification, the GPU supports other features such as instancing, where we reuse the vertex buffer multiple times in a single rendering, and geometry shader stream out, where we save the result of vertex/geometry shading for later re-use.

Bling does a good job of abstracting vertices and pixels: in Bling shader code one value represents both the vertex and pixel, where the vertex position and pixel color are specified with respect to this value. Bling infers whether the code is referring to a vertex or pixel based on a simple dependency analysis: we are referring to a pixel whenever we refer to anything that is only available in the pixel shader, otherwise we are referring to a vertex and the computation result (if used in the pixel shader) will be interpolated based on the analogous results in the primitive’s other vertices. However, Bling’s vertex/pixel abstraction breaks down when dealing with geometry: the input of a a rendering is simply the vertex count and primitive data and Bling has no high-level geometry abstraction. Bling’s lack of a geometry abstraction prevents geometries from being manipulated, composed, and constructed at a high-level, and prevents geometry shaders from being expressed at all unless shader partitioning is expressed explicitly. And this is a pity because geometry shaders can do so many cool things such as dynamically forming fur, hair, feathers, sparkles, blobs, and so on. We definitely need to make manipulating geometries easier.

So here is my new realization: give geometries their own abstraction in Bling, defined as one of the following:

The atomic case is a a collection of adjacent primitives. Actually, the only things that need to be specified for this primitive geometry is the number of vertices, vertex adjacency (if more than one primitive is defined), and primitive topology. Vertex properties such as positions and normals are not included as they will be inferred via dependency analysis within the shader. Examples: the face of a cube (four vertices to define two triangles) or the result of sampling a parametric surface such as a sphere.
A composition of two different geometries (gn = g0 + g1). The geometries composed must be of the same primitive topology. Example: we could compose a face with a four simple triangles to form a pyramid.
A duplication of the same geometry N times (gn = N * g0). We define this separately from geometry composition as it could correspond (but might not depending on efficiency) to geometry instancing in the GPU API. For duplication to be meaningful, each duplicated geometry will always undergo a transformation based on the duplication index. Example: a face is duplicated four times to form a cube, or duplicating a cube N times to do some crude voxel rendering.
A transformation of one geometry’s properties (gn = f(g0)). This is a bit difficult to define since there is no explicit common set of vertex properties, and indeed a vertex might not need a normal or even a position if it is not going to be rendered. However, the most common transformations will be on the position and normal, as the geometry is translated, scaled, and rotated. Example: rotating a face in 6 different ways to represent each face of a cube.

The benefit of this approach is immediately apparent: depending on our need we can define geometries mathematically or load them from a mesh. As properties aren’t included explicitly in the geometry, we can mix and match separate geometries as long as they share the same topology. We can then synthesize the index buffer (if needed) automatically, removing this burden from the user, and infer where geometry instancing occurs.

Unfortunately, vertex properties are more difficult to specify. Before, each vertex had one vertex ID and an optional instance ID, which we would use to compute the vertex’s position and pixel’s color. These could then be used as indexes into a table (in the case of a mesh) to lookup initial values for properties like position and normal, or they could be used in more rich computations; e.g., computing a per-pixel normal from the derivative of a parametric surface. At any rate, we inferred what properties to include in the vertex buffer in a way that allowed an expressed computation to span the CPU and GPU, which is a very good feature from an optimization and reuse perspective. Now if we allow geometries to be composed, duplicated, and transformed, vertex addressing obviously becomes much more difficult when inferring properties. There are few directions that we could go in to solve this problem:

We could assign an ID to a vertex based on the order that it was defined in a composition, but often vertex properties are defined relative to their geometries, and such a scheme would basically ruin reuse of code of that computes properties for a certain composed geometry.
We could come up with some hierarchical Id scheme. However, vertices are generally thrown all together (in the vertex shader at least), and as such it would be very difficult to reason about vertices whose IDs are not homogenous.
The other obvious option is to embed properties in geometries, which would solve the geometry reuse problem but then would tie geometries down to a common set of properties that they had to share in order to be combined. Yet this might be reasonable if the set of per-vertex properties are relatively fixed and it was easy to deal with exceptions easily. Whatever properties are declared do not necessarily correspond to what properties go in the vertex buffer; e.g., the position and normal properties of a sampled parametric surface can refer to dynamic CPU-only values (such as a slider’s position), where inference would synthesize the vertex buffer accordingly to include only static values. I’m still thinking about this problem but I guess I’m working in the direction of somehow re-introducing per-vertex properties, although I might have to throw away dynamic typing to make it work in C#’s type system (hopefully, generics will be enough).

Any solution that I come with has to play friendly with geometry shaders, which have the ability transform geometries based on dynamic information. I’ll get into this with my next post.

What is a programmable abstraction?

2009-07-30T07:50:43Z

In a recent blog post I described the principles behind Bling’s design. The primary principle is preferring programmable over fixed abstractions. But I feel that my definition of a programmable abstraction is so far unsatisfactory and undeveloped. To converge on a better definition, let’s first start at the beginning and discuss abstraction.

The process of abstraction (verb) is that of hiding details so that one can focus on fewer concepts at a time. An abstraction (noun) then is a hider of details. A programming language provides various primitives for constructing custom abstractions, such as procedures, classes, functions, properties, or rules—let’s refer to these as abstraction primitives to distinguish them from computation primitives, which are used to build up computations that use or are hidden by abstractions. Using abstraction primitives, a library then define and implement an API as a set of abstractions that permit access to some kind of service. Of course, real life is often more complex: one or more languages can be involved in the definition of a library (e.g., WPF and C#, Visual Basic, XAML, and HLSL), while libraries will often be built from or to interoperate with abstractions from other libraries.

Libraries are increasingly defined as frameworks that provide heavy large-grained abstractions for assembling systems in specific domains; for example consider a web framework (Ruby on Rails) or an IDE framework (Eclipse). Programmers customize a framework’s abstractions from the top-down according to their needs and assemble them together according to a rigid architecture. The trade off of using a frameworks includes the learning curve of the framework’s abstractions and architecture, as well as a sacrifice in flexibility where the framework becomes difficult or impossible to program outside of its intended purpose.

I am going to refer to framework abstractions as fixed abstractions because they are analogous to abstractions used to program a GPU through a fixed-function graphics pipeline as opposed to the abstractions used to program a GPU through a programmable pipeline. In the past, GPUs encoded in hardware only various fixed functions to manipulate vertices, light pixels, and so on. To program the GPU, the programmer manipulated various abstractions that represent fixed GPU functions through an API such as DirectX or OpenGL. As with a framework, these fixed-function APIs allowed for some customization of abstractions from the top-down, come with rigid architectures, and sacrifice flexibility in what they can express.

For the reason of flexibility alone, fixed-function pipelines are being rapidly replaced programmable pipelines that have the ability to express arbitrary custom effects through small programs known as shaders, which run directly on the GPU. The programmable pipeline completely supersedes the fixed-function pipeline. For example, whereas a specific kind of light (e.g., spotlight) and material (e.g., specular) are fixed abstractions that would be customized and installed in a fixed-function pipeline, now it is merely sufficient to take the position of the vertex or pixel being shaded and return a color! At first glance, such an API seems a lot harder to use in simple cases where well non-custom understood lighting is being used, but then such cases can rely on reusable procedures that package up the lighting math. Beyond dramatically increased flexibility, the programmable pipeline approach also has a better learning curve as their are only a few lightweight abstractions to learn with virtually no architecture. For an excellent write up what the move from fixed-function to programmable pipelines means, see Tim Sweeney’s Ars interview on the subject.

What do programmable pipelines and shaders have to do with software libraries and frameworks? They hint at an alternative to frameworks: rather design a library with many large-grained abstractions (fixed functions) that programmers customize down to what they want, instead we could design a library with a few small-grained versatile abstractions (shaders) that programmers then build up to what they want. We refer to the latter as programmable abstractions, which are smaller and more generally used than fixed abstractions. Large-grained reuse then comes from assembling the programmable abstractions together and packaging them up through language-level abstraction primitives (e.g., procedures). Programmable abstractions are not a new concept: they underpin many libraries, especially those built in functional programming languages. Additionally, programmable abstractions can be promoted to language-level abstractions with domain-specific languages (e.g., pipes in shell script), where the key to defining a good DSL is to base it on versatile programmable abstractions rather than fixed abstractions. I feel that a term for such abstractions is needed because of the confusion surrounding languages, frameworks, libraries, DSLs, and something in between (highly versatile libraries that are not quite DSLs).

I’m not exactly sure what the threshold between a programmable abstraction and a fixed abstraction is, it might not be very clear cut. Also, the benefits of programmable abstractions are negated if excessive amounts of boilerplate are needed to use them, languages can go a long way in eliminating this boilerplate and making programmable abstractions more feasible (e.g., see express tree lifting in LINQ). To be continued, in the next part I’ll provide some concrete examples of differences between fixed and programmable abstractions.

Announcing Bling 3!

2009-07-27T06:36:36Z

I’d like to announce a newly rewritten release of Bling with many improvements and exciting new features. Bling is a novel experiment in how WPF/UI programming can be enhanced via a lightweight domain-specific language hosted completely within C#. Bling basically changes the meaning of statements and expressions to do more that is otherwise possible with normal values in C#; e.g., a + b no longer means add the value of “a” to the value of “b,” but rather it creates tree that we can then transform into a databinding relationship, HLSL code, and so on. We exploit this to reduce or completely UI boilerplate code; e.g., no more value converters, no more HLSL code, no more C# shader classes, etc…Bling 3 goes farther with this style of programming through the addition of lifted functions (so f(a) also creates a tree!). For more details, see my blog.

New features in Bling 3.0:

Improved support for multiple forms in lighting in pixel shaders reusing WPF3D abstractions when possible. Now a lighting effect can be added to a widget by specifying a WPF3D light, a material (similar to but not a WPF3D material), a normal map and/or a height map loaded from a texture (some kinds of lights require an eye position). Lighting effects can be combined via addition. WPF shader examples:

Normal mapped diffuse material + red specular material point light effect.
Glass material + point light effect (also normal mapped).
Diffraction material + point light effect (again normal mapped).
Diffuse material + point light effect with parametric normals (computed vs. loaded from an image).

Pixel-level transition abstractions that support composition. Example:

Intermediate progress of a ripple + sin transition composed via Min (take the min color from each transition’s result, where start and end are the same).

Hacked prototype-quality WPF-friendly web browser that supports transformations and pixel shader effects. Browser will refresh every 100 milliseconds, so flash animations will work if a bit choppy. Link clicking is supported, but not keyboard input (need more interop code). Example:

Paper normal mapped browser navigated to Bling’s codeplex page.

Simplified physics engine (more examples in the future).
Initial experimental support for DirectX 10, focusing right now on parametric surfaces. DirectX compositions look very much like Bling WPF pixel shaders and can refer directly to WPF dependency properties (just like a WPF pixel shader can). Example:

A sphere, no distortion
A sphere + a egg crate distortion
(2) with more magnitude
(3) with more frequency
(4) with less magnitude + the WPF sliders that are controlling it all.

Caveats:

Bling 3 is not backwards compatible with the previous version of Bling. Existing code requires porting.

ToDo:

Work on DirectX abstractions (right now we just have a start).
Would like to support Silverlight 3.

A web browser suitable for Harry Potter in WPF!

2009-07-27T05:01:50Z

Daily Prophet eat your heart out! Here is a prototype web browser we threw together in Bling:

We (myself, my intern lighting-export Li SiYu, and a former intern Wang Chao who worked on the browser part) started with code from Chris Cavanagh's Blog to render a background IE instance into a WPF writeable bitmap, which we optimized using some sample code from David Teitlebaum. The advantage of this approach is that a writeable bitmap is a first class entity in WPF and so can be transformed and distorted, whereas a standard WPF web browser doesn’t support these features due to airspace issues. The disadvantage of our approach is that its a big hack suitable only for playing around (e.g., in the above pic) and we can only update the browser about once every 100 milliseconds, so videos and Flash are extremely choppy (but still very lively!).

The web browser content is projected onto a diffuse material that is lit by a point light. We bind the point light’s position to the mouse so the lighting changes dynamically with user interaction. The height of the light is bound to the slider beneath the address bar. This code sets up the lighting:

ThumbBl LightXY = new ThumbBl();
Action UpdateLightXY = PointBl.Assign(LightXY, canvas.MousePosition);
this.MouseMove += (xx, yy) => UpdateLightXY();
Lighting lighting = new DiffuseMaterialCl().Apply(new PointLightBl() {
  Position = new Point3DBl(LightXY, HeightOfLight),
  Color = Colors.White,
});

This code mostly involves databinding, except for tracking the mouse position, which we do manually. The “Apply” method of material exercises basic lighting equations according to the specified light (diffuse material + point light in our case) to create a lighting result. The lighting code for a diffuse material is encoded in Bling is something like this:

Point3DBl Dir = LightSource.DirectionToPosition(Position);
DoubleBl Dis = Dir.Length;
Dir = Dir.Normalize;
LightingResult = ((LightSource.Color.ScRGB * Dir.Dot(Normal).Max(0) * 
  LightSource.UseAttenuation(Dir, Dis))) * Input.ScRGB;

Lights are distinguished by their direction (constant for a directional light, difference between position being shaded and light position for point light) and their attenuation (a bit more math).

We apply the lighting result to the canvas containing the browser via normal mapping, which uses pre-baked texture to add depth and naturalness to the surface being lit. The textures used here make the browser look like it is printed on slightly folded paper! The texture scrolls as the scrollbar is used to scroll browser content, though not at the same magnitude. Code:

Texture HeightField = new ImageBrushBl(HeightImage);
Texture NormalMap = new ImageBrushBl(NormalImage);
HeightField = HeightField.Distort(uv => new PointBl(uv.X, (uv.Y + scrollBar.Value).Frac));
NormalMap =     NormalMap.Distort(uv => new PointBl(uv.X, (uv.Y + scrollBar.Value).Frac));
canvas.Effect.Custom = lighting.Apply((HeightField)HeightField, (NormalMap)NormalMap, 
                              canvas.Size.AddZ(0.015d), canvas.CenterSize.AddZ(100d));

Textures in WPF are brushes, so we load our normal mapping files (HeightImage and NormalImage) as image brushes. We distort the resulting textures by shifting them vertically by the scrollbar’s value, which are then converted into proper texture coordinates via Frac (an HLSL intrinsic defined as Math.Abs(Math.Round(n))). We then explicitly coerce the two textures into a height field and normal map respectively, for use in normal mapping. These explicit coercions convert the textures according to a popular normal map and height field texture representations (Crazy Bump and various Photoshop normal mapping plugins). The other two arguments used for normal mapping are the 3D size of the canvas and the eye position we want to use. The code for normal mapping is as follows:

public PixelEffect Apply(HeightField Heights, NormalMap Normals, Point3DBl Size, Point3DBl Eye) {
  return new PixelEffect(input => new Texture(UV => {
    Point3DBl Position = new Point3DBl(UV * Size.XY(), Heights[UV] * Size.Z);
    var eyeDir = (Eye - Position).Normalize;
    var UV0 = UV + eyeDir.XY() * Heights[UV] * Size.Z;
    return this[Position, Normals[UV0], Eye][input, UV0];
  }));
}

A pixel effect is simply a texture-to-texture function that is suitable for use as a pixel shader. A texture is defined as a point-to-color function that can be loaded from a file (as we did with HeightField and NormalMap) but can also be defined programmatically, which is what we are doing here. The math for the resulting texture is beyond the scope of this blog post but is not very difficult: basically we extract heights and normals from the height field and normal map according to the current texture coordinate UV, adjust them for the 3D size of our target, and bias UV by using an eye direction and the current height. This is then plugged into our lighting result, which in turn is used to compute the resulting pixel from the input texture and the biased texture coordinate.

This effect will work for any WPF application, not just our browser, notice the text field at the top, the slider below that, and the scrollbar tot he side, which are all WPF widgets!

The code for this sample is available in the Bling 3 distribution, which will hopefully be released soon!

Fun(ctional) graphics in C#!

2009-07-20T08:19:50Z

Graphics programming often involves customizing and combining well known techniques as mathematic formulas and algorithms related to geometry, lighting, physics, and so on. For performance and architecture reasons, realizing these formulas in a real programming involves drastic transformations that results in code that is hard to read and write. For example, multiple graphics techniques are encoded into low-level pixel and vertex shader code in way that renders them unrecognizable.

One solution is to encode graphic techniques in a high-level functional programming language, where functions allow us to effectively represent, combine, and manipulate graphic techniques. Take for example lighting computations that can be represented as a function Position3D –> Normal3D –> InputColor –> ResultColor, which can then be combined via addition: take two lighting computations l1 and l2, where a succinct l1 + l2 expands to p –> n –> c –> l1(p)(n)(c) + l2(p)(n)(c). As graphics is math-intensive, it benefits dramatically from function representations.

Unfortunately, their are a couple of drawbacks to this approach. First, the execution of a functional program is drastically slower than imperative code. However, as most graphics computations are moving to GPUs via memory-restricted shader code, this problem can be solved by generating shader code from functional programs. Such is the approach taken by Conal Elliott’s Haskell-hosted Vertigo library done at MSR. In Vertigo, geometries can be expressed as parametric surfaces, surface normals for lighting are computed via symbolic differentiation. The second drawback is more one of familiarity: the techniques seem to require abandoning our existing mainstream languages and tools for dramatically different and less familiar languages such as Haskell. However, while functional programming definitely benefits from functional languages, functional programming techniques can definitely be applied in more familiar languages. As a mainstream language, C# even provides a lambda construct to support programmers who want to dabble in functional programming; e.g., by using LINQ to query databases.

I’m implementing a functional graphics library in Bling for C#. Bling supports the creation and composition of expression trees as seemingly normal C# expressions, allowing us to easily compose code in a high level language and then ship the result to the GPU as a vertex or pixel shader. This allows us, among other things, to support a style of programming very similar to how Vertigo is used in Haskell. For example, consider the definition of a sphere parametric surface in Bling:

public static readonly PSurface Sphere = new PSurface((PointBl p) => {
  PointBl sin = p.SinU;
  PointBl cos = p.CosU;
  return new Point3DBl(sin[0] * cos[1], sin[0] * sin[1], cos[0]);
});

PSurface is a wrapper around a function from PointBl to Point3DBl, which are wrapper types around point (R2) and 3D point (R3) expression trees. SinU and CosU define scaled sine and cosine values over points between [0,0] and [1,1] by using the coordinates as percentage for 2PI angles. Rendering a sphere in Bling is simply a matter of creating some light and defining how many vertices we want to sample in the resulting vertex buffer:

PSurface Surface = Geometries.Sphere;
DirectionalLightBl dirLight = new DirectionalLightBl() {
  Direction = new Point3DBl(0, 0, 1),
  Color = Colors.White,
};
ParametricKeys SurfaceKeys = new ParametricKeys(200, 200);
Device.Render(SurfaceKeys, (IntBl n, PointBl uv, IVertex vertex) => {
  vertex.Position = (Surface * (world * view * project))[uv]; ;
  Point3DBl norm = (Point3DBl)(Surface.Normals[uv] * world).Normalize;
  Point3DBl usePosition = (Point3DBl)((Surface[uv] * (world)));
  vertex.Color = dirLight.Diffuse()[usePosition, norm](n.SelectColor());
});

where world, view, and project are standard 4x4 matrices used in translating position in 3D scenes. This code creates a directional light coming from the bottom (reused from WPF 3D no less!) and defines a set of surface keys over 200 by 200 vertices, meaning 40,000 vertices are sampled in the rendered sphere. The light is applied to the sphere in a render function using a world transformed position of each vertex along with a world transformed normal that is automatically computed via the derivative of the parametric surface. The color for each vertex is selected based on the vertex index, leading to a nice spiral pattern in the sphere that also gives us an idea of vertex topology. Here is the result:

As described in the Vertigo paper, interesting geometries can also be formed from surfaces via displacement by height fields. Copying the examples in the vertigo paper, consider an eggcrate height field that whose definition in Bling has a simple structure similar to sphere:

public static readonly HeightField EggCrate = new HeightField((PointBl p) => {
  p = p.SinCosU;
  return p.X * p.Y;
});

The displacement method on surfaces is then defined as follows:

public PSurface Displace(Func<PointBl, DoubleBl> HeightField) {
  return new PSurface((PointBl p) =>
    this[p] + Normals[p] * HeightField(p));
}

As with lighting computations, surface normals are used to determine the proper direction in which to apply the height field. We can then apply the egg crate height field to a sphere as follows:

PSurface Surface = Geometries.Sphere.
  Displace((Geometries.EggCrate.Frequency(20d).Magnitude(sliderZ.Value * .5)));

The frequency and magnitude of a height field are adjusted via methods that have one line implementations (again, this is described in the Vertigo paper). One twist is that we control the magnitude by a slider (Z), which is a standard WPF slider with a value dependency property. Now, by moving the slider up, the sphere becomes deformed via the egg crate height field:

Moving the slider even farther up, the sphere becomes spiky:

The change in deformation occurs in real time, so we see a smooth animation as the sphere becomes deformed.

Bling will automatically determine what code executes outside of the shader in the form of either uniforms (per-shader external variables) or vertex buffers (per-vertex data). Bling will also determine what code runs in pixel shaders vs. vertex shaders, and will automatically generate vertex input and output structures to accommodate communication between the two kinds of shaders. Bling will also generate code to handle update uniforms and vertex buffers during rendering, requiring absolutely no boilerplate code on behalf of the programmer. Instead, the programmer can just focus on what graphics techniques they want to use, without needing to worry about adapting them to fit on the underlying graphics architecture. For the above example, here is the HLSL code that Bling generates, which is then rendered using Direct3D 10:

float U0;

float4x4 U1;

float4x4 U2;

float3 U3;

float3 U4;

struct VS_Input {

  float P0 : POSITION;

  float3 P1 : POSITION1;

  float3 P2 : POSITION2;

  float P3 : NORMAL;

  float3 P4 : NORMAL1;

  float3 P5 : NORMAL2;

  float P6 : NORMAL3;

  float3 P7 : NORMAL4;

  float3 P8 : NORMAL5;

  float4 P9 : COLOR;

};

struct VS_Output {

  float4 Pos : SV_Position;

  float4 P0 : COLOR;

};

VS_Output VS(VS_Input input) {

  VS_Output output = (VS_Output) 0;

  float t0_0 = (U0 * input.P0);

  float3 t0_1 = t0_0.xxx;

  float3 t0_2 = (input.P1 * t0_1);

  float3 t0_3 = (input.P2 + t0_2);

  float3 t0_4 = t0_3.xyz;

  float4 t0_5 = float4(t0_4, 1);

  float4 t0_6 = t0_5;

  float4 t0_7 = mul(t0_6, U1);

  float4 t0_8 = t0_7;

  float3 t0_9 = t0_8.xyz;

  float3 t0_10 = t0_8.w.xxx;

  float3 t0_11 = (t0_9 / t0_10);

  float4 t0_12 = float4(t0_11, 1);

  float t0_13 = (U0 * input.P3);

  float3 t0_14 = t0_13.xxx;

  float3 t0_15 = (input.P1 * t0_14);

  float3 t0_16 = (t0_1 * input.P4);

  float3 t0_17 = (t0_15 + t0_16);

  float3 t0_18 = (input.P5 + t0_17);

  float3x1 t0_19 = float3x1(t0_18.x, t0_18.y, t0_18.z);

  float3 t0_20 = t0_19;

  float t0_21 = (U0 * input.P6);

  float3 t0_22 = t0_21.xxx;

  float3 t0_23 = (input.P1 * t0_22);

  float3 t0_24 = (t0_1 * input.P7);

  float3 t0_25 = (t0_23 + t0_24);

  float3 t0_26 = (input.P8 + t0_25);

  float3x1 t0_27 = float3x1(t0_26.x, t0_26.y, t0_26.z);

  float3 t0_28 = t0_27;

  float3 t0_29 = cross(t0_20, t0_28);

  float3 t0_30 = normalize(t0_29);

  float3 t0_31 = t0_30.xyz;

  float4 t0_32 = float4(t0_31, 1);

  float4 t0_33 = t0_32;

  float4 t0_34 = mul(t0_33, U2);

  float4 t0_35 = t0_34;

  float4 t0_36 = normalize(t0_35);

  float3 t0_37 = t0_36.xyz;

  float3 t0_38 = t0_36.w.xxx;

  float3 t0_39 = (t0_37 / t0_38);

  float t0_40 = dot(U3, t0_39);

  float t0_41 = max(t0_40, 0);

  float3 t0_42 = t0_41.xxx;

  float3 t0_43 = (U4 * t0_42);

  float4 t0_44 = float4(t0_43, 1);

  float4 t0_45 = (t0_44 * input.P9);

  output.Pos = t0_12;

  output.P0 = t0_45;

  return output;

}

float4 PS(VS_Output input) : SV_Target {

  float4 retV;

  retV = input.P0;

  return retV;

}

technique10 Render {

  pass P0 {

    SetVertexShader( CompileShader( vs_4_0, VS() ) );

    SetGeometryShader(NULL);

    SetPixelShader(CompileShader(ps_4_0, PS()));

  }

}

Since the code is auto-generated, it is not very pretty, but it doesn’t have to be. I still need to work on the conversion process to minimize the number of vertex parameters passed in as input, as these are limited to 16 and can easily be exhausted when more external values (e.g., sliders) are added to parameterize geometry and lighting.

The code for this example is currently available in our codeplex SVN repository at https://bling.svn.codeplex.com/svn/Bling3, and I’m currently working on cleaning things up for a new release of Bling (titled Bling 3) that will support rich UI construction in WPF and preliminary support for DirectX 10 (sorry XP users!) as described in this post.

Bling WPF hits V1

2009-02-20T15:04:54Z

I'd like to announce a new and improved version of Bling WPF. In this version, we have redone the wrappers around WPF databinding and pixel shading for better usability, while a lot of documentation and examples have been added to the distribution and the Codeplex page. Finally, we've also added some experimental support for UI physics with an example! A release for Visual Studio 2008/.NET 3.5 SP1 is available at http://www.codeplex.com/bling. For anyone unfamiliar with Bling, here are the primary features:

WPF Databinding without IValueConverters in C#! For example, "button.CenterPosition.X = slider.Value * MainCanvas.Width" is valid C# code in Bling that will setup a databinding relationship that binds button's LeftProperty to something that will move it with the slider.
WPF pixel shaders in C# without HLSL code or boilerplate! A pixel shader is simply a texture-to-pixel function, e.g., "canvas.CustomEffect = (input,uv) => slider.Value.Lerp(input[uv], ColorBl.FromScRgb(new PointBl(1,1,1) - input[uv].ScRGB, input[uv].A));" is a one line pixel shader that will invert all the colors in canvas interpolated with respect to a slider's current value. No need to write HLSL code, no need to write a custom effect class.writing a pixel shader is boiled down to its core function.
Bling defines many WPF convenience properties; e.g., Size is defined as (Width, Height), Right is defined as Left + Width, CenterPosition is defined as LeftTop + Size / 2. Convenience properties are just like properties that are backed directly by dependency properties; i.e., they can undergo databinding, be used in pixel shaders, and so on.
Bling code is completely compatible with conventional WPF code. Bling wrappers are stateless so you can use Bling functionality anywhere in your program regardless of architecture.
UI Physics! Did you wonder what would happen if property bindings were solved via a physics engine rather than a databinding engine? Well, ok, probably not J, but the result is cool and could possibly be the future of UI toolkits. I'll write more about this later.

New Bling WPF release with metaballs!

2008-12-29T12:54:08Z

No, not meatballs. I've done a lot of work on Bling this month, the first of which is writing a paper on the technique used to build Bling. I've also overhauled how pixel shader effects are expressed so that even less boilerplate is required than before. In the new release, when you want to add a signal parameter to a shader, you can simply call "Sh" on the signal and it will automatically be added to the list of the shader's parameters.

As an example, consider code:

Bling.Shaders.Shaders.MakeDirect((txt, input, uv) => {
  FloatSh value = 0f;
  Point3DSh rgb = Point3DSh.New(0, 0, 0);
  PointSg xyscaled = canvas.Size() / 
    (canvas.Size().X + canvas.Size().Y);

  uv = uv * xyscaled.Sh(txt);
  for (int i = 0; i < points.Length; i++) {
    var p = ((points[i] - canvas.LeftTop()) / canvas.Size());
    p = p * xyscaled;
    var v = (uv - p.Sh(txt));
    v = v * v;
    var at = 1f / (v.X + v.Y);
    value += at;
    rgb += (colors[i % colors.Length].Sh().RGB * at);
  }
  var area = canvas.Width() * canvas.Height();
  var at0 = (area / 400).Sh(txt);

  return ((value > at0)).Condition(
    ColorSh.New(rgb / value, 1), 
    Colors.White.Sh());
});

This code mixes signal code and shader code to create a nice meta-ball effect. The xyscaled variable is a point signal that scales X and Y coordinates according to the dimensions of the container. It is computed outside of the pixel shader but is multiplied the pixel coordinate (uv) by converting it to a shader parameter (xyscaled.Sh(txt)). For each point used to create the meta-ball effect (which is formed by 8 thumbs), the point is scaled according to the canvas then re-scaled according to x and y dimensions (so the ball generated is a circle) using xyscaled, all of these computations happen through data-binding and not in the pixel shader saving precious GPU instructions and not replicating shader operations on each pixel since they don't change. After these computations are performed outside of the shader, the point is brought into the shader (p.Sh(txt)) so it can be used in an operation with the pixel coordinate. Likewise, the area of the canvas is operated on outside of the GPU and brought into the shader using (area / 400).Sh(txt), where it is then used as the threshold for the metaball computation.

Check out the result (which is animated when you run it):

The meta-ball example is the main example in the new source code/distribution, which you can get from Bling's Codeplex page.

Shading Blobs with Bling WPF!

2008-10-28T10:47:20Z

I updated Bling WPF to version 0.6, get it at the normal place (http://www.codeplex.com/bling). Mostly, I changed the DSL to get rid of more boilerplate code. Now you can create multiple input and parameter pixel shader effects with only a few lines of C# code (sorry, no XAML yet). Example of a blob shader:

        var effect = new EightArgLiftedShader<Point>();
        effect.ShaderFunction0 = (input, uv, points) => {
          FloatSh d = 0;
          for (int i = 0; i < texture.SegmentCount; i++) 
            d += uv.Distance(points[i].LftSh());
          
          d = 1 - (d / texture.SegmentCount);
          d = d * 2;
          var color = input[uv];
          return ColorSh.New(color.RGB * d, color.A);
        };
        for (int i = 0; i < texture.SegmentCount; i++)
          effect[i].Bind = polygons[j].RelativePoint(thumbs[i].CenterPosition());
        polygons[j].Effect = effect;

An EightArgLiftedShader takes eight arguments of the same type (in this case Point). The parameters are then packaged up as an array of ShaderValue<Point> objects (points) where we then compute the average distance with the coordinate being processed (uv). The distance is then inverted and doubled to come up with a value to multiple the current color by. Outside of the shader, each point parameter of the shader is bound to the relative center point of each thumb that forms the skin of the polygon being shaded (basically, take the AABB of the polygon and compute the percentage that the thumb is inside the AABB). Result on shading three blobs:

A bit more 3D than a gradient brush!

Angles as doubles considered harmful

2008-10-25T08:57:44Z

One problem with the .NET library is the inconsistent use of radians and degrees to represent angles: System.Math angle functions assume angle are expressed in radian while WPF assumes angles are expressed in degrees. Trouble arises when you try to mix both APIs. While you can convert between degrees and radians using a couple simple equations (that I was forget and are not provided in System.Math as they are in java.lang.Math), a better way has to exist.

In Bling WPF, I’ve defined the struct Bling.Signals.Angle that abstracts the underlying angle representation. Assuming “using Bling.Signals,” you can go from a double or int to an Angle using three extensions methods: Degrees(), Radians(), or PI(). E.g., 90.Degrees() is an angle that is equivalent to .5.PI() and (.5 * Math.PI).Radians(). The Angle struct itself then has standard trigonometric functions on Angles that are otherwise in Math (e.g., Cos(), Sin(), Tan()) and some that Math left out that are useful in graphics programming; e.g., SinCos() that returns a unit vector pointing in the direction of the angle. Since there is no point in an Angle struct if no APIs will ever use it, I’ve wrapped all the angle-like dependency properties to return Angle signals (AngleSg) rather than double signals. Example of Rotate:

    public static AngleSg Rotate(this UIElement source, PointSg center) {
      RotateTransform rotate = source.Transform<RotateTransform>();
      rotate.Center().Bind = center;
      return rotate.Angle();
    }

    public static AngleSg Angle(this RotateTransform source) {
      return source.Signal<double>(RotateTransform.AngleProperty).Lft().Degrees();
    }

Basically, double signal (DoubleSg) has a function Degrees() that transforms it into a read/write angle signal. Rotation can now be expressed in angles or radians:

rectangle.Rotate(rectangle.Center()).Value = 45.Degrees();
rectangle.Rotate(rectangle.Center()).Value = .25.PI();

These can also be combined with trig functions:

rectangle.Rotate(rectangle.Center()).Value = x.ASin();
rectangle.Rotate(rectangle.Center()).Value = x.ACos();

This becomes really convenient when you are doing lots of geometric graphics programming.

WPF Signal Library is now Bling WPF!

2008-10-21T09:34:36Z

I’m rebranding the signal library as Bling WPF since it now contains support for expressing pixel shaders in C#. The release I’m putting out today polishes the pixel shader implementation that I released last week and fixes lots of outstanding issues:

I’m not using Microsoft naming conventions; all names begin with a capital. This was a serious sticking point for many potential users.
I renamed all the helper classes based on whether they were for signals or shaders. Example, DoubleSg is a signal that lifts doubles while FloatSh is a shader value that lifts floats.
I added Angle and Percentage types to better represent angles and percentages that are encoded as dependency properties. This was getting on my nerves: why do I have to convert an angle to radians, call Cosine, and then convert it back to degrees so I can plop it into a WPF dependency property? Anyways, this should make WPF programming a bit higher level although it borders on unit types. To get a percent, call Percent on a value or double signal; e.g., 100.Percent() or 57.Percent(). To get an angle, there are three methods—90.Degrees(), .5.PI(), and (.5 * Math.PI).Radians() all evaluate to a 90 degree angle.
Most HLSL 2.0 shader functions should be supported now on both scalars and vectors where appropriate. I also made it easier to define zero argument shaders (use LiftedShaderEffect.Apply) and one argument shaders (call Shader on the double/color/point signal that you want to be bound).

Writing WPF pixel shaders in C#!

2008-10-17T10:44:10Z

This week I saw a great project, Brahma, on expressing shader-based GPU computation in C#/LINQ. Using Brahma, one can write a simple LINQ query to transform a matrix without ever touching HLSL code or compiler, neat stuff! The advantage of this approach is that you don’t have to leave C# and the accompany tool support, and you don't have to setup your project to support pixel shader development.

So I was thinking: can we do this to write WPF pixel shaders? Sure! I found a nice project that allows WPF pixel shaders to be compiled on the fly by NyaRuRu (in Japanese only), and the author gave me permission to use his compiler in my WPF signals project. I then wrote a lifted-style meta-programming library to generate HLSL code from lifted C# code—this is the same way I wrapped WPF databinding using signals. Basically, the body of a pixel shader uses special lifted types for floats, doubles, and the other data types that can go into a pixel shader. Example:

    public static ColorShader Emboss(ShaderCompiler<NoArgLiftedShader> txt, PointShader uv) {
      var input = txt.ImplicitInput;
      var color = input[uv];
      var cA = txt.Cache(input[uv - 0.001] * 2).Lft();
      var cB = txt.Cache(input[uv + 0.001] * 2).Lft();
      var cC = txt.Cache(ColorShader.New(0.5f, 1) - cA + cB).Lft();
      return ColorShader.New(txt.Cache((cC.R + cC.G + cC.B) / 3f).Lft(), color.A);
    }

This pixel shader method implements an emboss effect (I found the algorithm here). “uv” is the current point, implicit input is obtained from the shader compiler, use standard bracket syntax to access elements of the array. A special “Cache” method in the shader compiler is needed to create new local variables, where otherwise expressions will be repeated if they are used more than once. All standard arithmetic operations are available along with various data type constructors; e.g., ColorShader.New. To apply this shader, simply call LiftedShaderEffect.Apply and assign the result to the Effect property of your widget:

      canvas.Effect = LiftedShaderEffect.Apply(ShaderFunctions.Emboss);

Now, isn’t that easy? A more complicated lifted shader can have one or more “registers” that are represented as dependency properties in WPF. Here is an example of a wave shader that I copied from a random XNA sample project:

  public class WaveShader : LiftedShaderEffect {
    private static readonly LiftedShaderConfig Config0 = CreateConfig<WaveShader>();
    protected override LiftedShaderConfig Config { get { return Config0; } }
    private static readonly DependencyProperty WaveProperty = Config0.AddParameter<double>("Wave", Math.PI / 0.75);
    private static readonly DependencyProperty DistortionProperty = Config0.AddParameter<double>("Distortion", 1d);
    private static readonly DependencyProperty CenterProperty = Config0.AddParameter<Point>("Center", new Point(0.5,0.5));
    public Bling.Signals.DoubleSg Wave() { return this.Signal<double>(WaveProperty).Lft(); }
    public Bling.Signals.DoubleSg Distortion() { return this.Signal<double>(DistortionProperty).Lft(); }
    public Bling.Signals.PointSg Center() { return this.Signal<Point>(CenterProperty).Lft(); }
    static WaveShader() { Config0.ShaderFunction = ShaderFunction0; }
    private static ColorSh ShaderFunction0(ShaderCompiler txt, PointSh uv) {
      var wave = txt.Parameter<double>(WaveProperty).LftSh();
      var distortion = txt.Parameter<double>(DistortionProperty).LftSh();
      var center = txt.Parameter<Point>(CenterProperty).LftSh();
      var distance = (uv - center).Abs();
      var scalar = distance.Length;
      // invert the scale so 1 is centerpoint
      scalar = (1 - scalar).Abs();
      // calculate how far to distort for this pixel    
      var offset = (wave / scalar).Sin();
      offset = offset.Clamp(0, 1);
      // calculate which direction to distort
      var sign = (wave / scalar).Cos();
      // reduce the distortion effect
      offset = offset * distortion / 32;
      var input = txt.ImplicitInput;
      return input[uv + (offset * sign)];
    }
  }

A custom lifted shader effect must extend LiftedShaderEffect with itself as the type parameter (a bit of a cludge, I did some type hacking). It must also statically create a LiftedShaderConfig that manages transparently the registers and inputs of the pixel shader effect. Then, again statically, you can create your registers using the AddParameter of this config object by specifying the type of the parameter (passed in as a type parameter) along with its name and default value. Finally, you have to override the abstract Config property of LiftedShaderEffect with the static config object that you created. I’ve then wrapped all these dependency properties as signals for convenience. The pixel shader function is defined in WaveShader’s static initializer by assigning ShaderFunctin0 to ShaderFunction of the config object. Inside the shader, each shader parameter from the shader compiler using the Parameter method with same property type that you used in AddParameter to create the dependency property (which only remember their type as a value). Finally, the LftSh() method has to be called on these parameters so that the appropriate operators are available. The rest of the code looks pretty standard, you can access methods like Abs (absolute value), Length (of a vector), Sin, Clamp, and Cos in the C# code as you would in HLSL code.

The code and a release is available at http://www.codeplex.com/wpfsignals. Download the latest release and build (should work ok if you have 3.5 SP1, although I’m not sure if the latest DirectX SDK is required or not…). Run the SignalTest project and you should see this picture:

The blue thumb in the middle as well as the two sliders are data bound to the wave shader parameters (via signals of course!). You can move these around to cause the pixel shader to refresh the screen. If you want to use this in your own code, just include Bling.Shaders in your project and add “using Bling.Shaders” to the set of used namespaces (add “using Bling.Signals” if you want to also use signals).

Twilight of the High-level UI Toolkit

2008-10-08T06:08:29Z

I was reading Tim Sweeney’s write up on how fixed function GPUs and the APIs that depend on them are quickly becoming obsolete. Basically, as GPUs move toward programmable “shaders,” graphic engine writers can easily whip up their own efficient graphic functions. We are already seeing this in new graphics APIs: mobile-oriented OpenGL ES is basically a shader-mostly version of OpenGL sans most of the fixed-function features.

There is an analog with this move in how I program WPF: with retained graphics and convenient access to data binding functionality through signals, I mostly find myself avoiding all the high-level widgets in WPF like StackPanels and Grids because using low-level Canvas is (a) easy and (b) incredibly flexible. For example, consider a grids: WPF has Grid class of course, but with data binding it is rather trivial to get cells to align with whatever row and column abstractions you want to define. This means you can have grids whose columns expand and collapse, scroll bars that move per-row and column, arbitrary highlights, and so on…you no longer have to be limited to Grid functionality. Example from a prototype I’m working on:

The little + signs represent rows that can be expanded:

We can also layout row headers vertically if we want. All of this is built on low-level widget, mostly Canvas, Border, Button and Scrollbar, connected together through data binding. Its not even much more code than configuring a Grid would have required.

So I have a prediction to make: high-level UI toolkits (much of WPF, Swing) will become unpopular within 5 years. They will instead be replaced in popularity by low-level UI toolkits (Java2D/JavaFX, much of WPF) with high-level abstractions (data binding, signals). WPF can already be used like this, and it is very liberating.

Reaction Diffusion using WriteableBitmap

2008-10-07T08:55:01Z

For a while, I’ve been experimenting with reaction diffusion to provide for more “life like” surfaces. This reaction diffusion algorithm spreads colors out for a diffuse effect. To ensure responsiveness in the UI, the diffusion logic is run in a timer thread that notifies the UI thread to re-do the WriteableBitmap (texture) when it is done. As for performance, 300 by 300 is about all the algorithm can handle on my rather beefy machine before visible artifacts begin to show up. Multi-threading through ParallelFor doesn’t seem to help, but I’ll keep working on it to see if we can get a bigger buffer. Here is a sample of what the diffusion looks like:

I’ve done this using RenderTargetBitmap and a pixel shader before, however the behavior of diffusion was very difficult to control, using WriteableBitmap is more manageable. Code below, .NET 3.5 SP1 is required to run this program.

DiffusionBuffer.cs:

using System.Threading;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Media;

namespace PhysicsLib {
  public class DiffusionBuffer : Canvas {
    private readonly System.Windows.Media.Imaging.WriteableBitmap texture;
    private byte[] bufferA;
    private byte[] bufferB;
    private readonly int stride_t;
    private readonly int PixelWidth;
    private readonly int PixelHeight;
    private readonly Timer timer;
    public DiffusionBuffer(Vector size, int updateTime) {
      this.Width = size.X;
      this.Height = size.Y;
      this.texture = new System.Windows.Media.Imaging.WriteableBitmap((int)size.X, (int)size.Y, 96, 96, PixelFormats.Bgra32, null);
      int bytesPerPixel_t = (texture.PixelWidth) / 8;//  bytesPerPixel=  25=200/8
      stride_t = texture.PixelWidth * bytesPerPixel_t;//  stride=  5000=200*25 
      int arraySize_t = stride_t * texture.PixelHeight;//  arraysize=  1000000=5000*200
      bufferA = new byte[arraySize_t];
      bufferB = new byte[arraySize_t];

      for (int ij = 0; ij < texture.PixelWidth * texture.PixelHeight; ij++) {
        int i = ij % (texture.PixelWidth);
        int j = ij / texture.PixelWidth;

        bufferA[(i * 4) + (j * stride_t) + 0] = 000;
        bufferA[(i * 4) + (j * stride_t) + 1] = 000;
        bufferA[(i * 4) + (j * stride_t) + 2] = 000;
        bufferA[(i * 4) + (j * stride_t) + 3] = 255;
      }
      {
        var rect0 = new Int32Rect(0, 0, (int)texture.PixelWidth, (int)texture.PixelHeight);
        texture.WritePixels(rect0, bufferA, stride_t, 0);
      }
      {
        var image = new Image() { Source = texture };
        Canvas.SetLeft(image, 0);
        Canvas.SetTop(image, 0);
        this.Children.Add(image);
      }
      this.PixelHeight = texture.PixelHeight;
      this.PixelWidth = texture.PixelWidth;


      timer = new Timer(new TimerCallback((x) => ((DiffusionBuffer)x).updateA()), this, updateTime, updateTime);
    }
    private int getPx(int i, int j, int k, int def) {
      if (i < 0 || i >= PixelWidth) return def;
      if (j < 0 || j >= PixelHeight) return def;

      return bufferA[(i * 4) + (j * stride_t) + k];
    }
    private byte getPx0(int i, int j, int sp, int k) {
      var at = bufferA[(i * 4) + (j * stride_t) + k];
      long value = 0;
      value += getPx(i - sp, j - 00, k, at);
      value += getPx(i + sp, j + 00, k, at);
      value += getPx(i - 00, j - sp, k, at);
      value += getPx(i + 00, j + sp, k, at);

      var tp = sp - 1;
      value += getPx(i - tp, j - tp, k, at);
      value += getPx(i + tp, j + tp, k, at);
      value += getPx(i + tp, j - tp, k, at);
      value += getPx(i - tp, j + tp, k, at);
      var ret = (value / 7.97);
      if (ret > 255) ret = 255;
      return (byte)ret;
    }
    private delegate void DummyDelegate(DiffusionBuffer buffer);

    private void updateA() {
      for (int ij = 0; ij < PixelWidth * PixelHeight; ij++) {
        int i = ij % (PixelWidth);
        int j = ij / PixelWidth;

        bufferB[(i * 4) + (j * stride_t) + 3] = bufferA[(i * 4) + (j * stride_t) + 3];
        bufferB[(i * 4) + (j * stride_t) + 0] = getPx0(i, j, 3, 0);
        bufferB[(i * 4) + (j * stride_t) + 1] = getPx0(i, j, 3, 1);
        bufferB[(i * 4) + (j * stride_t) + 2] = getPx0(i, j, 3, 2);
      }
      this.Dispatcher.BeginInvoke(new DummyDelegate((buffer) => buffer.updateB()), this);
    }
    public void rectangle(Point p, Vector size, Color c) {
      var e = p + size;
      for (int i = (int)p.X; i < (int)e.X; i++) {
        for (int j = (int)p.Y; j < (int)e.Y; j++) {
          add(bufferB, (i * 4) + (j * stride_t) + 0, c.B);
          add(bufferB, (i * 4) + (j * stride_t) + 1, c.G);
          add(bufferB, (i * 4) + (j * stride_t) + 2, c.R);
        }
      }
    }
    private static void add(byte[] buf, int idx, byte value) {
      if (idx < 0 || idx >= buf.Length) return;
      int x = buf[idx];
      if (x < 0) x = 255 + x;
      x += value;
      if (x > 255) x = 255;
      buf[idx] = (byte) x;
    }

    public virtual void updateB() {
      var rect0 = new Int32Rect(0, 0, (int)texture.PixelWidth, (int)texture.PixelHeight);
      texture.WritePixels(rect0, bufferB, stride_t, 0);
      var old = bufferA;
      bufferA = bufferB;
      bufferB = old;
    }
  }
}

Program.cs:

using System;
using System.Windows;
using System.Windows.Controls.Primitives;
using System.Windows.Controls;
using System.Windows.Media;
using System.Windows.Shapes;

namespace TestForWriteablebitmap {

  public class MyDiffusionBuffer : PhysicsLib.DiffusionBuffer {
    public readonly Thumb[] thumbs = new Thumb[4];
    public readonly Color[] colors = { Colors.Red, Colors.Green, Colors.Blue, Colors.White };
    public MyDiffusionBuffer()
      : base(new Vector(200,200), 15) {

      for (int i = 0; i < thumbs.Length; i++) {
        thumbs[i] = new Thumb() { Background = new SolidColorBrush() { Color = colors[i] }, Width = 10, Height = 10 };
        Canvas.SetZIndex(thumbs[i], 1);
        thumbs[i].DragDelta += (x, y) => {
          Canvas.SetLeft(((Thumb)x), Canvas.GetLeft((Thumb)x) + y.HorizontalChange);
          Canvas.SetTop(((Thumb)x), Canvas.GetTop((Thumb)x) + y.VerticalChange);
        };
        Canvas.SetLeft(thumbs[i], 0);
        Canvas.SetTop(thumbs[i], 0);
        Children.Add(thumbs[i]);
      }
    }
    public override void updateB() {
      foreach (var t in thumbs)
        rectangle(new Point(Canvas.GetLeft(t), Canvas.GetTop(t)), 
          new Vector(t.Width, t.Height), ((SolidColorBrush)t.Background).Color);
      base.updateB();
    }
  }
  class Program {
    [STAThread]
    static void Main(string[] args) {
      var buf = new MyDiffusionBuffer();
      buf.RenderTransform = new ScaleTransform() { ScaleX = 2, ScaleY = 2, CenterX = 150, CenterY = 150 };
      var win = new Window() { Content = buf };
      Application app = new Application();
      app.Run(win);
    }
  }
}

Curved Polygons in WPF!

2008-10-06T07:51:51Z

I found something on the web http://www.antigrain.com/research/bezier_interpolation/index.html about using Bezier curves to build polygons with smooth corners. Anyways, I ported this to WPF and it works very well! Basically, consider the last point (point0), current point (point1), next point (point2) and the point after that (point3), you can compute each Bezier segment as follows:

    public static BezierSegment PolygonSegment(Vector point0, Vector point1, Vector point2, Vector point3, double smoothValue) {
      var c1 = (point0 + point1) / 2d;
      var c2 = (point1 + point2) / 2d;
      var c3 = (point2 + point3) / 2d;

      var len1 = (point1 - point0).Length;
      var len2 = (point2 - point1).Length;
      var len3 = (point3 - point2).Length;

      var k1 = len1 / (len1 + len2);
      var k2 = len2 / (len2 + len3);

      var m1 = c1 + (c2 - c1) * k1;
      var m2 = c2 + (c3 - c2) * k2;

      var ctrl1 = m1 + (c2 - m1) * smoothValue + point1 - m1;
      var ctrl2 = m2 + (c2 - m2) * smoothValue + point2 - m2;
      var curve = new BezierSegment();
      curve.Point1 = (Point) ctrl1;
      curve.Point2 = (Point) ctrl2;
      curve.Point3 = (Point) point2;
      return curve;
    }

Simply loop through your points to create your polygon, here is some code in a physics engine I’m doing:

      figure.StartPoint = points[(0 + 1) % points.Count].centerPoint;
      var segments = new PathSegmentCollection();
      for (int i = 0; i < points.Count; i++) {
        var point0 = points[(i + 0) % points.Count].centerPoint;
        var point1 = points[(i + 1) % points.Count].centerPoint;
        var point2 = points[(i + 2) % points.Count].centerPoint;
        var point3 = points[(i + 3) % points.Count].centerPoint;
        segments.Add(Extensions.PolygonSegment((Vector) point0, (Vector) point1, (Vector)point2, (Vector) point3, .7));
        var cp = point0;
        topLeft = topLeft.Min(cp);
        bottomRight = bottomRight.Max(cp);
      }
      figure.Segments = segments;

where figure is the only path figure of a PathGeometry.

Check out the cool blob-like results: