Welcome to MSDN Blogs Sign in | Join | Help

Taking LINQ to Objects to Extremes: A fully LINQified RayTracer

Not too long ago I blogged about a C# raytracer which took advantage of a lot of C#3.0 language constructs.  However, you may have noticed that it did not actually use LINQ query expressions all that much.  Well, after discussing this with a coworker on the PLINQ team at lunch one day - I was convinced that it should be possible to express the logic of the raytracer in a LINQ to Objects query.  The result is below - a single 60-line LINQ query which captures the complete raytracing algorithm.

[DISCLAIMER:  I am by no means suggesting it's a good idea to write code like this.  Do so only at your own risk!  Just as you *could* write your entire application in one method body, or decide to replace all if statements with virtual method dispatches, doing what I show below is a clear abuse of this language construct.  But it's an interesting example to show that there is a lot of expresiveness available in C#3.0 query expressions.]

One Really Big Query Expression!

var pixelsQuery =
    from y in Enumerable.Range(0, screenHeight)
    let recenterY = -(y - (screenHeight / 2.0)) / (2.0 * screenHeight)
    select from x in Enumerable.Range(0, screenWidth)
           let recenterX = (x - (screenWidth / 2.0)) / (2.0 * screenWidth)
           let point = Vector.Norm(Vector.Plus(scene.Camera.Forward, 
Vector.Plus(Vector.Times(recenterX, scene.Camera.Right), Vector.Times(recenterY, scene.Camera.Up)))) let ray = new Ray { Start = scene.Camera.Pos, Dir = point } let computeTraceRay = (Func<Func<TraceRayArgs, Color>, Func<TraceRayArgs, Color>>) (f => traceRayArgs => (from isect in from thing in traceRayArgs.Scene.Things select thing.Intersect(traceRayArgs.Ray) where isect != null orderby isect.Dist let d = isect.Ray.Dir let pos = Vector.Plus(Vector.Times(isect.Dist, isect.Ray.Dir), isect.Ray.Start) let normal = isect.Thing.Normal(pos) let reflectDir = Vector.Minus(d, Vector.Times(2 * Vector.Dot(normal, d), normal)) let naturalColors =
from light in traceRayArgs.Scene.Lights let ldis = Vector.Minus(light.Pos, pos) let livec = Vector.Norm(ldis) let testRay = new Ray { Start = pos, Dir = livec } let testIsects = from inter in from thing in traceRayArgs.Scene.Things select thing.Intersect(testRay) where inter != null orderby inter.Dist select inter let testIsect = testIsects.FirstOrDefault() let neatIsect = testIsect == null ? 0 : testIsect.Dist let isInShadow = !((neatIsect > Vector.Mag(ldis)) || (neatIsect == 0)) where !isInShadow let illum = Vector.Dot(livec, normal) let lcolor = illum > 0 ? Color.Times(illum, light.Color) : Color.Make(0, 0, 0) let specular = Vector.Dot(livec, Vector.Norm(reflectDir)) let scolor = specular > 0 ? Color.Times(Math.Pow(specular, isect.Thing.Surface.Roughness), light.Color) : Color.Make(0, 0, 0) select Color.Plus(Color.Times(isect.Thing.Surface.Diffuse(pos), lcolor), Color.Times(isect.Thing.Surface.Specular(pos), scolor)) let reflectPos = Vector.Plus(pos, Vector.Times(.001, reflectDir)) let reflectColor =
traceRayArgs.Depth >= MaxDepth ? Color.Make(.5, .5, .5) : Color.Times(isect.Thing.Surface.Reflect(reflectPos), f(new TraceRayArgs(new Ray { Start = reflectPos, Dir = reflectDir }, traceRayArgs.Scene,
traceRayArgs.Depth + 1))) select naturalColors.Aggregate(reflectColor, (color, natColor) => Color.Plus(color, natColor)))
.DefaultIfEmpty(Color.Background).First()) let traceRay = Y(computeTraceRay) select new { X = x, Y = y, Color = traceRay(new TraceRayArgs(ray, scene, 0)) }; foreach (var row in pixelsQuery) foreach (var pixel in row) setPixel(pixel.X, pixel.Y, pixel.Color.ToDrawingColor());
Let Expressions

"let" query clauses allow you to introduce a variable into scope which can be used by the following query clauses.  Similar to local variables in a method body, this gives you a way to avoid evaluating a common expression multiple times by storing it in a variable.  This can be very useful even in much simpler queries.  Of course, in the query above - let is absolutely critical.

Recursion

A raytracer is inhenertly recursive - whenever a ray hits a reflective or transparent surface, new rays have to be cast.  Thus the query that expreses the raytracer needs to be recursive.  But the let clause doesn't support declaring recursive functions.  Not to let that stop us (as it really should!) - we can use the Y combinator to introduce recursion by hand.  Mads wrote a very detailed description of the recursive lambda expressions in C# recently - and the code example above uses my personal implementation of some of the same ideas he describes. 

Expression-based control structures

When you are trying to write things in a single query expression - you don't have the luxury of using any statements.  Since most of the C# control structures are  statement based (if, switch, for, while, etc.) - this can make things a little trickier.  One really valauble expression-based control structure is the ternary operator ( x ? y : z ).  This is used in a couple of places above to provide the necessary branching logic.  The rest of the control structuring is accomplished with LINQ Standard Query Operators - Select, SelectMany, Where, OrderBy, Aggregate and DefaultIfEmpty.  As functional programmers have shown for years, the expressiveness of these very general list processing operations is impressive.

Conclusion

In case you had any doubts - hopefully this convinces you that C#3.0 Query Expressions are really a very expressive language construct.  Of course - this expressiveness is probably best used in more moderation than shown here!

File iconLINQRayTracer.cs

kick it on DotNetKicks.com

Published Monday, October 01, 2007 10:34 PM by LukeH

Comment Notification

If you would like to receive an email when updates are made to this post, please register here

Subscribe to this post's comments using RSS

Comments

# Techy News Blog &raquo; Taking LINQ to Objects to Extremes: A fully LINQified RayTracer

# Ein Raytracer in einem Programmausdruck

Tuesday, October 02, 2007 3:21 AM by TheUndeadable entwickelt

LINQ als Renderer LINQ hat was...

# re: Taking LINQ to Objects to Extremes: A fully LINQified RayTracer

Tuesday, October 02, 2007 6:11 PM by dditweb

HARD CORE. Very impressive!

# Taking LINQ to Objects to Extremes: A fully LINQified RayTracer

Tuesday, October 02, 2007 6:11 PM by DotNetKicks.com

You've been kicked (a good thing) - Trackback from DotNetKicks.com

# re: Taking LINQ to Objects to Extremes: A fully LINQified RayTracer

Tuesday, October 02, 2007 8:02 PM by neuralocity

Amazing.  This is an eyeopening experience for myself, very interesting.  Thanks!

# re: Taking LINQ to Objects to Extremes: A fully LINQified RayTracer

Tuesday, October 02, 2007 8:03 PM by neuralocity

You should start a LINQ Cookbook site.

# re: Taking LINQ to Objects to Extremes: A fully LINQified RayTracer

Wednesday, October 03, 2007 4:32 AM by MatthieuMEZIL

Great!

# re: Taking LINQ to Objects to Extremes: A fully LINQified RayTracer

Wednesday, October 03, 2007 2:55 PM by MichaelGiagnocavo

Would be cool if let was available in normal expressions and could also define subexpressions... I can't wait to see what C# 4 will enable.

# re: Taking LINQ to Objects to Extremes: A fully LINQified RayTracer

Thursday, October 04, 2007 1:38 PM by rogerj

# The Weekly Source Code 7

Thursday, October 11, 2007 6:44 PM by Scott Hanselman's Computer Zen

# The Weekly Source Code 7

Thursday, October 11, 2007 7:51 PM by ASPInsiders

In my new ongoing quest to read source code to be a better developer , I now present the seventh in an

# Community Convergence XXXIII

Sunday, October 14, 2007 8:54 PM by Charlie Calvert's Community Blog

Welcome to the thirty-third edition of Community Convergence. This week we have a new video called Programming

# let = const var

Tuesday, November 06, 2007 5:03 AM by R.Tanaka.Ichiro's Blog

let = const var

# F#

Wednesday, November 14, 2007 6:57 PM by LukeH's WebLog

A few weeks back, Soma blogged about an increased investment by the Microsoft Developer Division in the

# F#, brought to you by Luke!

Thursday, November 15, 2007 9:34 AM by Don Syme's WebLog on F# and Other Research Projects

Luke Hoban is now full time as program manager on F#, and has just posted a short introduction about

# F#, brought to you by Luke

Thursday, November 15, 2007 9:35 AM by Don Syme's WebLog on F# and Other Research Projects

Luke Hoban is now full time as program manager on F#, and has just posted a short introduction about

# F#, brought to you by Luke

Thursday, November 15, 2007 10:06 AM by Noticias externas

Luke Hoban is now full time as program manager on F#, and has just posted a short introduction about

# Miguel de Icaza: RayTracing in one LINQ statement

Friday, November 16, 2007 1:15 PM by 工程師的雞排攤

Through Don Syme&#39;s blog I read about Luke Hoban moving from the C# team at Microsoft to the F# team

# re: Taking LINQ to Objects to Extremes: A fully LINQified RayTracer

Friday, November 16, 2007 4:34 PM by Bryan

You are insane.  My hats off to you.

# re: Taking LINQ to Objects to Extremes: A fully LINQified RayTracer

Friday, November 16, 2007 5:23 PM by Anonymous

Screenshot? :)

# Ray Tracing en une requête LINQ

Sunday, November 18, 2007 11:17 AM by CoqBlog

Je viens de tomber sur ça : Taking LINQ to Objects to Extremes: A fully LINQified RayTracer En clair

# re: Taking LINQ to Objects to Extremes: A fully LINQified RayTracer

Tuesday, November 20, 2007 8:01 AM by LukeH

Anonymous -

There is a screenshot of the image produced by the raytracer on my previous blog post (linked to above).  Probably more fun though to try it out for yourself :-).  The attached code file can be compiled with the C#3.0 compiler at the command line or imported into a VS 2008 project.

# Chunk partitioning vs range partitioning in PLINQ

Sunday, December 02, 2007 5:42 PM by Parallel Programming with .NET

If you look in the PLINQ samples in the December 2007 CTP , you'll see a parallel implementation of Luke

# Chunk partitioning vs range partitioning in PLINQ

Sunday, December 02, 2007 6:15 PM by Noticias externas

If you look in the PLINQ samples in the December 2007 CTP , you&#39;ll see a parallel implementation

# re: Taking LINQ to Objects to Extremes: A fully LINQified RayTracer

Wednesday, December 05, 2007 7:45 AM by Dimchansky

LINQRayTracer is slower, than first version of RayTracer. Why?

# Use the power of let in your LINQ queries

Wednesday, December 05, 2007 11:59 AM by Linq in Action News

Often, when you try to find out how to write the correct LINQ query you need, you end up being confused

# Use the power of let in your LINQ queries

Wednesday, December 05, 2007 12:06 PM by Fabrice's weblog

Often, when you try to find out how to write the correct LINQ query you need, you end up being confused

# Parallelizing a query with multiple “from” clauses

Friday, December 07, 2007 3:02 PM by Parallel Programming with .NET

Consider a simplified version of the parallel implementation of Luke Hoban's LINQ ray tracer var Xs =

# Parallelizing a query with multiple “from” clauses

Friday, December 07, 2007 3:19 PM by Noticias externas

Consider a simplified version of Luke Hoban&#39;s LINQ ray tracer var Xs = Enumerable .Range(1, screenWidth

# Функциональное программирование на C# 3.0 (часть 2)

Saturday, December 08, 2007 2:04 PM by Ivan

Продолжение беседы про прикладную функциональщину. Попытка реализовать парсер п

# re: Taking LINQ to Objects to Extremes: A fully LINQified RayTracer

Sunday, December 16, 2007 10:16 PM by LukeH

Dimchansky - Thanks for the comment on the performance difference between the query-based version and the original version.  As you point out, there is a fairly significant performance overhead for the version that uses one giant query.  I should have mentioned this as an additional reason to *not* recommend this extreme kind of code for a typical application.

The culprit in terms of performance in this case is primarily the many "let" clauses.  These each incur a method call, a delegate invocation and an allocation of a "SelectIterator".  Since many of these are run in tight loops which are hit close to millions of times during a typical run, the additional cost is magnified significantly.

# Community Convergence XXXVIII

Wednesday, January 02, 2008 4:15 PM by Charlie Calvert's Community Blog

Welcome to the thirty-eighth Community Convergence. These posts are designed to keep you in touch with

# Silly comments

Wednesday, January 09, 2008 1:11 AM by Anders

Impressive! Regarding the comments on this article, one should really do something about the crap! But I guess you don't have the option to delete any of 'em, do you LukeH?

# re: Taking LINQ to Objects to Extremes: A fully LINQified RayTracer

Wednesday, February 20, 2008 6:25 PM by SHArQ

Uhh... Is that thingie what you call a BIG query?

How about a 40KB-length, seven-level-nested one? :)

# Video of Luke Hoban's In-Depth Look at C# 3.0

Saturday, February 23, 2008 2:40 AM by Charlie Calvert's Community Blog

Last fall in Barcelona, Spain two PM's from the C# team gave talks on key parts of the new technology

# Una query expression in Linq veramente lunga ed interessante

Tuesday, February 26, 2008 9:06 AM by Paolo Possanzini

Vi riporto un link ad un post che potete trovare anche nei link della StartPage di VisualStudio. Sono

# 很酷的let clause的应用

Tuesday, April 15, 2008 7:22 AM by works guo

这里是LINQ to XML利用let暂时存放子节点的数据,再从查询let中的数据得到XML中子节点多个属性.

# re: Taking LINQ to Objects to Extremes: A fully LINQified RayTracer

Sunday, April 20, 2008 9:32 PM by CoryDambach

Do you have a version of the Ray Tracer distributed with the Parallel Extensions CTP where the code has more comments for people without 3D graphics experience?  The organization of the CTP version is superb(in comparison to the single file one), but without XML comments I found navigating it tiresome...  (I know teaching 3d programming was not the purpose of either, but I thought I'd ask.)  I've learned quite a bit from it already.

Also, I haven't looked at the code for this one, but did you use Parallel Linq in it, the post didn't seem to imply it.  If not, why didn't you?

# LINQ doesn't have to be about databases...

Thursday, June 12, 2008 3:33 AM by Yet Another Coding Blog

I'm finally getting into the groove with LINQ and lambda expressions, and am over the initial skepticism

# LINQ, LINQ, say no more

Thursday, June 19, 2008 1:04 PM by ctodx

An interesting question that someone asked of me recently, indicating perhaps that the information from

# Slight speedup?

Thursday, August 07, 2008 5:55 AM by Niall Connaughton

Pretty cool, fun reading through it all :P

I was just wondering about the performance issue raised. I haven't read the other version of the code so I don't know if this improvement would be just for this mega-LINQ version or not. But I was looking at the section of the query that works out if an object is in shadow from the light and I think the query could be tweaked a bit.

Couldn't you put in a where clause to filter out objects which intersect the ray from the light but are beyond the originally intersected object? Then you could remove the order by and use FirstOrDefault(), saving on intersecting all other objects in the scene once you found a shadow casting object.

I guess depending on how many objects you have and how much shadow you have this may have varying levels of improvement.

# Функциональное программирование на C# 3.0 (часть 2)

Tuesday, September 23, 2008 7:20 AM by Write ahead blog

# re: Taking LINQ to Objects to Extremes: A fully LINQified RayTracer

Wednesday, October 29, 2008 3:13 PM by Oliver Hallam

I have ported this raytracer to XQuery!  All the code explained line by line so you can understand what is going on.  It runs with XQSharp.

More information can be found here:

http://www.xqsharp.com/xqsharp/samples/raytracer/index.htm

# re: Taking LINQ to Objects to Extremes: A fully LINQified RayTracer

Friday, September 18, 2009 3:35 PM by David Betz (MVP)

Wow... this is the greatest piece of code written in a LONG time.  You need to get an award or something.

Leave a Comment

(required) 
required 
(required) 

  
Enter Code Here: Required
 
Page view tracker