PTaylor's WebLog

FSX Content Taxonomy

ESP News:Developer Conference later this month and new ESP Dev Center on MSDN

Caveat Emptor:FS9 port-overs may reduce your simming experience, Part II

I want to clarify my previous post on FS9 port-overs.

Let me start by saying my blog posts are my opinion and not the Studio's or MS's. As are my posts here. Can we not try to conflate my comments into WW3?

Let me restate where the issue is:

1)FSX content - not a problem.

2)FS9 content that
   a)works in FSX (SP2) - not a problem.

3)FS9 content that
   a) is labeled as such - not a problem

4)FS99 content that:
   a)is labeled as FSX
   b)does not work in FSX
   c)has no content or marketing blurb update.

It is the narrrowly defined case 4 that I am talking to. I am most decidedly not talking to content prior to SP2 and am specifically referring to content that has not adjusted since Acceleration/SP2. It is 7 months since Acceleration and 5 since SP2. That is plenty of time to at least re-label content or make a patch or whatever.

I hope that clarifies for any 3rd party developers who read my comments as talking to them when they are not in case 4, and I apologize if by me not being crystal clear they felt unfairly singled out.

Caveat Emptor:FS9 port-overs may reduce your simming experience

On the dozen or so forums I inhabit and post, I see many people struggling with some plane models especially with Acceleration/SP2.

It turns out that in 99.9999% of the cases, these are "warmed over" FS9 ports. When we released information about Acceleration/SP2 we were pretty clear that there were some FS9 compatibility issues, so this should not be a surprise.

The surprise is when the model is billed as an FSX model but turns out to be FS9 with some lipstick.

Sure, if its freeware you get what you pay for. But for payware vendors to do this, that is not right. Your customers are figuring this out, and in the famous words of Sheriff Woody - "who poisoned the waterhole? you did".

How to tell if a plane model is a "warmed over" FS9 port? See this thread, http://www.sim-outhouse.com/sohforum...ad.php?t=57005

Use a file viewer or hex editor to look at the file. Near the beginning of the file, you should see the characters "MDLXMDLH" if it's an FSX model. An FS2002/2004 model will show "MDL8MDLH" or something else, depending on what model creator program it was built with.

I just have to share

Lamont Clark has posted a way cool tweak to display descriptive labels about 1500 ft above airports in FSX.

See the HOWTO and a screenshot at:

http://lc0277.nerim.net/wiki/index.php?Displaying%20airport%20names 

As an extra bonus, he has a great tip on how to adapt FSX aircraft to be used with the carriers in Acceleration here:

http://lc0277.nerim.net/wiki/index.php?Adapting%20FSX%20aircrafts%20to%20acceleration%20carriers

 Great stuff, I hope you like it. And hats off to Lamont!

Back of the envelope

Everyone who writes software, and most who use it, should be familiar with "back of the envelope" calculations. I originally read about this style of thinking about problems in Jon Bentleys' Programming Pearls, Chapter 7

Part of the work we are doing in core is around rendering and performance. We are working to update our rendering abstraction, how we view rendering in general; as well as how we think about and measure performance.

As part of this we have started doing some calculations related to bus bandwidth and how much of the bus each subsystem can and should use. This is really our first attempt at these “back of the envelope” calculations and we should have done them long ago.

Anyway, enough with the recriminations and on to the data. First, here are the bus bandwidth rates we are using:

Bus                               Expected Perf Rate  (GB/s)                 Expected Rate ( MB/Frame at 60 FPS )                                                  Date introduced

PCIe 16x 2.0     4                           68                                2007

PCIe 16x 1.0     2                           34                                2004

PCIe 8x  1.0     1                           17                                2004

AGP  8x  1.0     1                           17                                2002

Ok, so let’s think about this and just the Autogen tree subsystem.  In FSX, most auto generated trees have 12 vertices that contain 32 bytes of data[1] thus giving the model a size of (12 * 32 bytes) = 384 bytes per instance.

When using batching, assuming 20% of the bus bandwidth is available for auto generated trees, it is possible to transfer (0.20 * 34 MB / 384 bytes per instance) = 18568 batched trees per frame at 60 Hz.

Now, a typical scene has 50 1km x 1km cells in the scene. Autogen trees are pegged at 4500 max per cell (when the slider is all the way to the right). You can set the max up to 6000, so let’s call it 5000 for easier math. 5000*50=250,000 trees.  

Holy Autogen Batman! Yes, this is why autogen brings the system to its knees. Crysis doesn’t try to render 250k trees. If you are having a major perf problem with Autogen, try using the “max in cell” tweak to reduce the max to 500. Then your max is 500*50=25,000.

It should be obvious the same holds true for Autogen buildings.

Part of our performance work is to turn the engine from a batching engine to an instancing engine to help bridge this gap. When using instancing and assuming 10% of the bus bandwidth is available for non-animated instance data (0.10 * 34MB / 48 bytes per instance) = 74, 274 instances per frame at 60 Hz can be sent across a PCIe 16x bus.  If we give Autogen Trees 20%, then we get 150,000 trees. 3000 trees per cell is certainly within those limits. So that change alone gets us close to "within bounds".

Given there are sliders and config entries, users can adjust their settings to local conditions of the hardware and the style of flying you do. That is why this gap isn’t “tragic”; but it still is a rude surprise to most people that FSX tries to do so much and that is a good part of the root of the problem and why the FSX engine is so different from other engines.

And there are some other things we are doing, so the engine isn’t so overcommitted. But that is another post. J.

PS.

http://support.microsoft.com/kb/555739 lists the Autogen max per cell tweak. 

PPS

The slider stops correspond to the following percentages:

0, 10, 20, 45, 70, 100

Guess who's back?

In an excellent piece of good news, Adam Szofran, late Terrain Dev lead of Aces, has returned to the fold. And we are glad to have him back!

Now back to your regularly scheduled programming, I mean flying.

“Holy Grail or Fools’ Errand”, Indeed!

There is a lot of discussion and buzz around Intel, Larrabee, and ray tracing on the GPU.

So I decided to investigate. First, we have some required reading J.

Resources to get your head wrapped around the current debate and the questions it raises:

Wikipedia on “what is Larrabee”

http://en.wikipedia.org/wiki/Larrabee_%28GPU%29

Deano Calver on Real-Time Ray Tracing

http://beyond3d.com/content/articles/94/5

PC Perspective articles on Daniel Pohl’s work on converting Quake 3 and Quake 4 to be a ray tracing graphics engine.

Ray Tracing and Gaming - Quake 4: Ray Traced Project

http://www.pcper.com/article.php?aid=334

Rendering Games with Raytracing Will Revolutionize Graphics http://www.pcper.com/article.php?aid=455

Ray Tracing and Gaming - One Year Later

http://www.pcper.com/article.php?aid=506

Intel briefing on Larrabee and Visual Computing

http://www.gamasutra.com/php-bin/news_index.php?story=17898

nVidia comments

http://www.pcper.com/article.php?aid=530

Carmack comments

http://games.slashdot.org/article.pl?sid=08/03/12/1918250

http://www.pcper.com/article.php?aid=532&type=overview

Next, some fundamental research papers:

Ray Tracing

Turner Whitted’s seminal 1980 paper “An Improved Illumination Model For Shaded Display”

http://delivery.acm.org/10.1145/360000/358882/p343-whitted.pdf?key1=358882&key2=3584595021&coll=portal&dl=ACM&CFID=33609274&CFTOKEN=12069724

Streaming and Computation

Accelerator

ftp://ftp.research.microsoft.com/pub/tr/TR-2005-184.pdf

Brook

http://graphics.stanford.edu/papers/brookgpu/brookgpu.pdf

SH

http://www.gamasutra.com/features/20040716/mccool_01.shtml

http://libsh.org/

Tracing on GPUs

Ray Tracing on a Stream Processor, PhD Dissertation

http://graphics.stanford.edu/papers/tpurcell_thesis/tpurcell_thesis.pdf

Ray Tracing on Programmable Graphics Hardware

http://graphics.stanford.edu/papers/rtongfx/rtongfx.pdf

Ray Tracing on GPU, slides

http://www.cs.unc.edu/~lastra/comp870/Slides/Ray%20Tracing%20on%20GPU.ppt

Ray Tracing on GPU ,paper with implementation

http://gpurt.sourceforge.net/DA07_0405_Ray_Tracing_on_GPU-1.0.5.pdf

Stackless KD-Tree Traversal for High Performance GPU Ray Tracing

http://graphics.cs.uni-sb.de/Publications/TR/2007/StatelessTrav.pdf

Hybrid Ray Tracing

http://www.iam.unibe.ch/~robert/doc/hybrid-rt-2007.pdf

A Hybrid CPU-GPU Renderer

http://www.stadtwald21.de/mcbeister/gpu-cpu/paper.pdf

Ray Tracing fully implemented on programmable graphics hardware, Master’s Thesis at Chalmers

http://www.ce.chalmers.se/edu/proj/raygpu/downloads/raygpu_thesis.pdf

GPU Performance

Understanding GPUs Through Benchmarking

http://www.gpgpu.org/sc2006/slides/10.houston-understanding.pdf

http://graphics.stanford.edu/projects/gpubench/

Paradigm Shift?

Interactive Rendering In The Post-GPU Era

http://www.graphicshardware.org/previous/www_2006/presentations/pharr-keynote-gh06.pdf

“What’s it all about Alfie?”

Now with that out of the way, what do I think?

Examining the stream programming model in all of this, but especially the Purcell dissertation and the SourceForge hosted paper, it is clear that streams are a powerful and expressive concept.  The hybrid paper shows how to mix use of the CPU and GPU within a ray-tracing algorithm in an approach that shows a lot of promise because it could easily be adapted to Larrabe where the GPU parts run on the Larrabee GPU and the CPU parts run on the Larrabee CPU. Without a hybrid approach, the acceleration techniques that rely on spatial data structures cannot be run on a GPU efficiently today. This is where Larrabee does present interesting possibilities.

For ray tracing, though, there are still some problems to be solved. The scene aliasing problem can be solved by jittering the primary rays, so that requires more performance. The static vs dynamic scene problem can be solved by the use of hybrid sw techniques or hybrid hw ( Larrabee ). The memory issues affecting overall algorithm throughput can be solved with hybrid hw ( Larrabee ) or upgraded hw ( traditional rasterization hw which is already moving this way ). These are the easy problemsJ.

The cognitive gap problem in teaching the masses how to do stream programming is perhaps the hardest problem to solve. A new toolkit and a dedicated evangelism team, plus the runway of a multi-year investment in the hardware and software toolkit is what that takes. It took DirectX 4+ years (DX3 in 1996 to DX8 in 2000) to reach its place as a solid, well-respected, and widely used standard. Will Intel stay in the game that long?

So the ray tracing idea is certainly interesting; however it is not clear all the problems are solved enough to gain critical mass - thus we’ll just have to wait. If the CPU+GPU hw that is Larrabee allows the hybrid approach where you can implement a tracing algorithm as a mix of rasterization phases on a GPU with algorithm phases on a CPU  and Intel have an API that makes that path approachable for the average game programmer -  that might be a win.

More generally, I think the move to a new abstraction around stream processing provides many benefits; the biggest one is it explicitly enables the expression of many more algorithms without performing unnatural acts with a graphics API. If we could perform both in the same frame, that is dangerously close to nirvana. Of course, I have had a copy of the SH book for almost 2 years, downloaded CUDA as soon as it was available, and am always alert for paradigm shifts so my opinion may be suspect. J

With all of that said, the point that stream processing is fundamental does not necessarily portend “doom and gloom” for current IHVs as it seems to me that nVidia and ATI both have stream processor designs and thus can certainly adapt where they have weak links in their respective food chains if Intel really does prove to have a rabbit in its hat. ATI may have an easier time of that since they and AMD are one. However, nVidia is not to be discounted, they are smart and aggressive and know how to play to win so it would not surprise me one iota if they pulled a 2nd rabbit out of their hat in response if and when Intel shows its cards as a strong hand.

Let the debate begin J.

Vista SP1 review and rollout timeframe

http://www.winsupersite.com/reviews/winvista_sp1.asp is an excellent and informative article.

I hear 3/18 ( tomorrow ) is the day for downloaders, but I could be wrong :-).

Updated:Issues that may affect 3rd party content for FSX and that will not be fixed in SP1

I forgot the name of the #define for handing the VS2005 SP1 redist dependency and accidentally deleted the post

-----

A couple of issues have come up that we think are going to affect 3rd party content, and that SP1 will not fix.

VS 2005 SP1 and VS 2005 SP1 redist bits

The first one is, the release of VS 2005 SP1 comes with a new redist. Any 3rd party who builds code intended for FSX with the VS 2005 SP1 bits installed will take a dependency on the new DLLs in the redist unless they take special care and compile with a special #define ( #DEFINE _USE_RTM_VERSION ). This code will fail to load in FSX RTM. It is the 3rd party developers' responsibility for shipping all the bits their package is dependent on. Because we cannot fix FSX RTM, we are not going to include installing the VS 2005 SP1 redist bits in the SP1 installer. FSX SP1 itself does not have this dependency. We will place a link to the VS 2005 SP1 redist bits on our site with a note to this effect.

FSX RTM and the "round earth correction" issue

The second issue is the "round earth correction" issue for 3rd party content like long runways. If we don’t "round earth correct", the ends of long runways end up floating and planes appear to go thru the runway. If we do "round earth correct" and the 3rd party content has very long polygons to model the runway, we get z-fighting with the terrain mesh since the tesselations don’t match. We believe the right thing to do is add the round earth correction to SP1 which will work for short polygons (the threshold is somewhat dependent on the current mesh tessellation but we think <100m is likely to be safe) and not cause z-fighting.  That leaves long polygons. We could go further and re-tesselate the polygons for 3rd parties "on the fly" at load time but we are not going to be able to get that work into SP1. The real answer is for the 3rd party to update the content for FSX using the FSX SDK. For content that hasnt done that, and has polygons with a long extent, we are not going to be able to fix this in SP1.

 I hope these issues and the explanations are clear, useful, and help the FSX community.

Interview up at SimPilot

Mike McCarthy of FS Open Components fame ( amongst other things ) takes his time as a raconteur to interview yours truly here.

GPU variants, performance envelopes, and being an informed consumer

Overall, performance is only as good as the weakest link in the chain. So if you skimp on the GPU and get a great CPU, the GPU becomes the bottleneck.

There is a reason the high-end cards cost so much. They provide more. Really.

Understanding how the variants differ is key.

The IHVs make 4 lines of

cards:

1) Super high-end for the enthusiast at super high prices

2) High-end at high prices

3) Mid-range at mid prices

4) Low-end at low prices.

For nVidia in the 8 series that translates to:

1)8800 Ultra overclocked 768m monster

2)8800 GTX/GTS 512m, 8800 GT 2nd gen

3)8600

4)8400

The manufacturing process is not perfect, and there are defects in almost every wafer burnt by the FAB. Where in the wafer the defects are is critical. The IHVs have come up with a clever way to avoid having to throw as many chips out due to defects.

One design approach is with the chip components layed out in quadrants, with the shader pipes and stream processors arrayed around a central memory controller. There is a single die design for all "family" variants using this approach.

When the wafers come out of the FAB, the parts are speed binned, by that I mean they are tested at the maximum rated performance to see if the part works. If it doesn’t work at that clock, they reduce the clock until they see if it does work. If it does not work at all, the chip is discarded. If it does work, the level at which it speed bins determines what variant the part can be sold as.

If the part doesn’t speed bin out for Ultra, they try it for High. Typically High has the same number of shader units and stream processors as Ultra and the same memory width but with lower clocks ( less memory bandwidth ) and less memory.

If it doesn’t speed bin out for High, they turn 1 quadrant off ( now 3/4 of the top end ) and try to bin it as a Mid-range. With fewer shader units and stream processors. at a lower clock, and with less memory and less memory bus width which means less memory bandwidth.

If that doesn’t work, they turn 2 quadrants off ( now 1/2 of the top end ) and try at Low-end. With fewer shader units and stream processors. at a lower clock, and with less memory and sometimes less memory bus width which means less memory bandwidth. Much less.

Low-end is really low-end, this is no joke. Really. How does this help with deciding what to purchase?

Do not buy below mid-range if you can afford to (x5xx or greater is usually how midrange is labeled), and understand that the 2nd place in the high end (GT, not GTS) is often the price/performance sweet spot.

So by that metric, there is no way a 7300 is better than a 7600, and both are substantially less than a 7800 or 7900. And an 8400 is substantially less than an 8600 or 8800.

Laptop parts are almost always cut down, even at the high end. This is for space, heat, and power. So a top-end laptop chip does not give the same performance as the chip in a top-end add-in card. Caveat Emptor there.

If you cannot afford the top end, then you can’t. And then it is good there are mid-range and low-end variants. But realize you are buying a lesser product. In some cases significantly lesser.

So here is some practical advice, gleaned by examining the nVidia site and the 8 series comparo chart.

Let us take an 8400 compared to an 8800, and do some percentage math:

       StreamProcs CoreClock ShaderClock MemClock Memory MemInterface MemBandwidth (GB/sec)TextureFillRate (B/s)

8800 Ultra____128____612______1500_____1080____768MB_384-bit_____103.7______________39.2

8400 GS______16____450_______900______400____256MB__64-bit_______6.4_______________3.6

Percent______12.5% _73.5%_____60%_____ 37%____33.3%__16.7%______6.2% ____________9.2%

Note the underbars are to make the tables line up, my blog software eats spaces. Sorry about that J.

So for key areas like memory bandwidth and fill rate, the 8400 is under 10% of what an 8800 offers.

Note the "new" 8800 GT has 112 stream processors so that is 87.5% of the top end. Lets look at the rest of that part:

       StreamProcs CoreClock ShaderClock MemClock Memory MemInterface MemBandwidth (GB/sec)TextureFillRate (B/s)

8800 Ultra____128____612______1500_____1080____768MB__384-bit____103.7______________39.2

8800 Gt______112____600______1500______900____512MB__256-bit_____57.6______________33.6

Percent______87.5%__98%______100%____83.33%__66.7% __66.7%____55.5%_____________85.7%

So the 8800 GT has only 55% of the memory bandwidth of the Ultra, and on an app like FSX that crunches through a lot of memory, that means even this part might have issues with some slider settings depending on the scene.

The 8800 GTX is the closest to the top if you cannot afford the Ultra. The GTS 640/320 has a 320-bit memory interface where the GTS/512 and GT have only 256.

Before anyone thinks I am picking on nVidia, I can do the same with the ATI specs here for 2xxx family and here for 3xxx family.

From this you can see the 2400 and 2600 are quite a bit cut down in terms of memory bus while the 3850 is much closer to the 3870. Generations can be quite different in how the spread is between the low and high.

This discussion is not about being negative. This is about being a smart shopper, and the differences between these parts can and does have a direct impact on performance.

In terms of what parts of these specs are more critical for FSX, at least 3 of these are critical:

·         Overall memory size ( don’t go below 256m if you can help it)

·         Overall memory bandwidth ( the max you can afford )

·         Core clock ( that helps with handling the amount of drawing FSX does )

Sure, the memory and shader clocks are important, but if the core clock is too low, then those don’t matter as much. Sure, texture fill is important but without memory bandwidth that won’t matter as you cannot get the textures and geometry across the bus. Note memory interface width and memory bandwidth have a direct relationship.

I hope this was useful.

Last Post was 100th Blog post

I thought it was noteworthy of mention that my last post on FSX version identifiers was my 100th post.

I suppose that makes me a confirmed blogger, and not one of the bloggers who starts a blog and doesn't really follow-thru which I was in danger of in my old job.

FSX Release Identifiers, so users can verify what they have installed

RTM:   10.0.60905

SP1:     10.0.61355 (Russian: 10.0.61357)

SP2:     10.0.61472

XPack: 10.0.61637

You can check this by bringing up the About dialog box from the Help menu.

Manual for F/A-18 VC in Acceleration