Graphics programming often involves customizing and combining well known techniques as mathematic formulas and algorithms related to geometry, lighting, physics, and so on. For performance and architecture reasons, realizing these formulas in a real programming involves drastic transformations that results in code that is hard to read and write. For example, multiple graphics techniques are encoded into low-level pixel and vertex shader code in way that renders them unrecognizable.

One solution is to encode graphic techniques in a high-level functional programming language, where functions allow us to effectively represent, combine, and manipulate graphic techniques. Take for example lighting computations that can be represented as a function Position3D –> Normal3D –> InputColor –> ResultColor, which can then be combined via addition: take two lighting computations l1 and l2, where a succinct l1 + l2 expands to p –> n –> c –> l1(p)(n)(c) + l2(p)(n)(c). As graphics is math-intensive, it benefits dramatically from function representations.

Unfortunately, their are a couple of drawbacks to this approach. First, the execution of a functional program is drastically slower than imperative code. However, as most graphics computations are moving to GPUs via memory-restricted shader code, this problem can be solved by generating shader code from functional programs. Such is the approach taken by Conal Elliott’s Haskell-hosted Vertigo library done at MSR. In Vertigo, geometries can be expressed as parametric surfaces, surface normals for lighting are computed via symbolic differentiation. The second drawback is more one of familiarity: the techniques seem to require abandoning our existing mainstream languages and tools for dramatically different and less familiar languages such as Haskell. However, while functional programming definitely benefits from functional languages, functional programming techniques can definitely be applied in more familiar languages. As a mainstream language, C# even provides a lambda construct to support programmers who want to dabble in functional programming; e.g., by using LINQ to query databases.

I’m implementing a functional graphics library in Bling for C#. Bling supports the creation and composition of expression trees as seemingly normal C# expressions, allowing us to easily compose code in a high level language and then ship the result to the GPU as a vertex or pixel shader. This allows us, among other things, to support a style of programming very similar to how Vertigo is used in Haskell. For example, consider the definition of a sphere parametric surface in Bling:

public static readonly PSurface Sphere = new PSurface((PointBl p) => {
  PointBl sin = p.SinU;
  PointBl cos = p.CosU;
  return new Point3DBl(sin[0] * cos[1], sin[0] * sin[1], cos[0]);
});

PSurface is a wrapper around a function from PointBl to Point3DBl, which are wrapper types around point (R2) and 3D point (R3) expression trees. SinU and CosU define scaled sine and cosine values over points between [0,0] and [1,1] by using the coordinates as percentage for 2PI angles. Rendering a sphere in Bling is simply a matter of creating some light and defining how many vertices we want to sample in the resulting vertex buffer:

PSurface Surface = Geometries.Sphere;
DirectionalLightBl dirLight = new DirectionalLightBl() {
  Direction = new Point3DBl(0, 0, 1),
  Color = Colors.White,
};
ParametricKeys SurfaceKeys = new ParametricKeys(200, 200);
Device.Render(SurfaceKeys, (IntBl n, PointBl uv, IVertex vertex) => {
  vertex.Position = (Surface * (world * view * project))[uv]; ;
  Point3DBl norm = (Point3DBl)(Surface.Normals[uv] * world).Normalize;
  Point3DBl usePosition = (Point3DBl)((Surface[uv] * (world)));
  vertex.Color = dirLight.Diffuse()[usePosition, norm](n.SelectColor());
}); 

where world, view, and project are standard 4x4 matrices used in translating position in 3D scenes. This code creates a directional light coming from the bottom (reused from WPF 3D no less!) and defines a set of surface keys over 200 by 200 vertices, meaning 40,000 vertices are sampled in the rendered sphere. The light is applied to the sphere in a render function using a world transformed position of each vertex along with a world transformed normal that is automatically computed via the derivative of the parametric surface. The color for each vertex is selected based on the vertex index, leading to a nice spiral pattern in the sphere that also gives us an idea of vertex topology. Here is the result:

As described in the Vertigo paper, interesting geometries can also be formed from surfaces via displacement by height fields. Copying the examples in the vertigo paper, consider an eggcrate height field that whose definition in Bling has a simple structure similar to sphere:

public static readonly HeightField EggCrate = new HeightField((PointBl p) => {
  p = p.SinCosU;
  return p.X * p.Y;
});

The displacement method on surfaces is then defined as follows:

public PSurface Displace(Func<PointBl, DoubleBl> HeightField) {
  return new PSurface((PointBl p) =>
    this[p] + Normals[p] * HeightField(p));
}

As with lighting computations, surface normals are used to determine the proper direction in which to apply the height field. We can then apply the egg crate height field to a sphere as follows:

PSurface Surface = Geometries.Sphere.
  Displace((Geometries.EggCrate.Frequency(20d).Magnitude(sliderZ.Value * .5)));

The frequency and magnitude of a height field are adjusted via methods that have one line implementations (again, this is described in the Vertigo paper). One twist is that we control the magnitude by a slider (Z), which is a standard WPF slider with a value dependency property. Now, by moving the slider up, the sphere becomes deformed via the egg crate height field:

Moving the slider even farther up, the sphere becomes spiky:

 

The change in deformation occurs in real time, so we see a smooth animation as the sphere becomes deformed.

Bling will automatically determine what code executes outside of the shader in the form of either uniforms (per-shader external variables) or vertex buffers (per-vertex data). Bling will also determine what code runs in pixel shaders vs. vertex shaders, and will automatically generate vertex input and output structures to accommodate communication between the two kinds of shaders. Bling will also generate code to handle update uniforms and vertex buffers during rendering, requiring absolutely no boilerplate code on behalf of the programmer. Instead, the programmer can just focus on what graphics techniques they want to use, without needing to worry about adapting them to fit on the underlying graphics architecture. For the above example, here is the HLSL code that Bling generates, which is then rendered using Direct3D 10:

Since the code is auto-generated, it is not very pretty, but it doesn’t have to be. I still need to work on the conversion process to minimize the number of vertex parameters passed in as input, as these are limited to 16 and can easily be exhausted when more external values (e.g., sliders) are added to parameterize geometry and lighting.

The code for this example is currently available in our codeplex SVN repository at https://bling.svn.codeplex.com/svn/Bling3, and I’m currently working on cleaning things up for a new release of Bling (titled Bling 3) that will support rich UI construction in WPF and preliminary support for DirectX 10 (sorry XP users!) as described in this post.