The good and the bad of exception filters

The good and the bad of exception filters

Rate This
  • Comments 15

Every so often we get asked questions about the CLR’s support for exception filters. Why do some languages support them and others do not? Is it a good idea to add them to my new language? When should I use a filter vs. catch/rethrow? Etc. I’ll try to answer some of these questions for you here, and while I won’t go into all of them hopefully you’ll walk away with enough info to form your own opinion on the rest. Like so many things there’s good things and bad things about exception filters…

So what’s a filter?

The CLR provides a number of exception handling primitives that higher level languages can build upon. Some are fairly obvious, and map readily to language constructs that most of us know and love: try/catch and try/finally, for instance. I’d hazard to guess that everyone knows what those do, but just in case, let’s consider a quick example in C#:

    try

    {

        try

        {

            Console.Write(“1”);

            if (P) throw new ArgumentException();

        }

        finally

        {

            Console.Write(“2”);

        }

    }

    catch (ArgumentException e)

    {

        Console.Write(“3”);

    }

    Console.Write(“4”);

If P is true, then that will print out “1234”, of course. If P is false, then it will print “124”. Groovy.

But the CLR also provides two more EH primitives: fault and filter. A fault clause is much like a finally clause; it runs when an exception escapes its associated try block. The difference is that a finally clause also runs when control leaves the try block normally, whereas the fault clause will only run when control leaves the try block due to an exception. In the case above, if we replaced the “finally” with “fault” (there’s no C# syntax for that, but suspend your disbelief for a moment) then it would print “1234” if P is true, and “14” is P is false. See the difference? Most languages don’t expose this as a first-class language construct, but we do have a few that use fault clauses under the covers for specific scenarios.

So that leaves us with filters. I suppose the simplest definition of a filter is that it is a construct that allows one to build a conditional catch clause. In fact, that’s exactly what VB uses filters for. Let’s consider a more complicated example in VB:

    Function Foo() As Boolean

        Console.Write("3")

        Return True

    End Function

    Sub Main()

        Dim P As Boolean = True

        Try

            Try

                Console.Write("1")

                If (P) Then

                    Throw New ArgumentNullException()

                End If

                Console.Write(“P was False!”)

            Finally

                Console.Write("2")

            End Try

        Catch ex As ArgumentNullException When Foo()

            Console.Write("4")

        Catch ex As ArgumentException

            Console.Write("5")

        End Try

        Console.Write("6")

    End Sub

Here you’ll note the “Catch ex As ArgumentNullException When Foo()” line is a conditional catch statement. The catch handler will only execute and print “4” when the exception is an ArgumentNullException and when Foo() returns true. If Foo() returns false, then the catch clause doesn’t execute, and the system continues to search for a catch clause that can handle the exception. In this case, the very next clause would handle the exception, and print “5”.

So, what do you think this program prints? Don’t cheat by attempting to compile and run it! Using what you know about exception handling and looking at the program structure and syntax, what would you imagine this program prints? I suspect most people would guess “12346”. I even gave you a clue with the numbering.

I think most of us would look at the example above and conclude that the result should be “12346” because when we look at the syntax above we, quite rightly, see lexically scoped language constructs. We expect that when the code in the inner finally clause starts executing that no more code anywhere in the associated try block will execute. For instance, in the example above, if P is true then when we enter the Finally we know that no more code will execute in the try block, and that we’ll never print “P was False!”. Likewise, when we evaluate one of the catch clauses, we expect that all the code in the associated try block is done executing.

And here comes the bad…

It turns out that program actually prints “13246”. My clue with the numbering was an evil ruse. After the throw, Foo() is executed first as part of evaluating the first catch clause, and then the finally within the associated try block executes. And that’s just freaky… what happened to our lexically scoped language constructs?!

This is a surprising result for most. It breaks our intuitive reasoning about the language based on the lexically scope exception handling constructs provided. Here, when we evaluate the conditional catch clause, all the code in the associated try block has not, in fact, finished executing.

Why does this happen?

The reason we see “3” before “2” is subtle, and founded in the CLR’s implementation of exception handling. The CLR’s exception handling model is actually a “two pass” model. When an exception is thrown, the runtime searches the call stack for a handler for the exception. The goal of the first pass is to simply determine if a handler for the exception is present on the stack. If it sees finally (or fault) clauses, it ignores them for a moment.

The handler may be in the form of a typed catch clause, i.e., “Catch ex as ArgumentException)”. When the runtime sees a typed catch clause, it can determine itself if this clause will handle the exception by performing a simple check to determine if the exception object is of a type that inherits from (or is) the type in the clause.

But when the runtime sees a filter, it must execute the filter in order to determine if the associated handler will handle the exception. If the filter evaluates to true, then a handler has been found. If it evaluates to false, then the runtime will keep searching for a handler.

Once a handler has been found, the first pass is over and the second pass begins. On the second pass, the runtime again runs the call stack from the point of throw, but this time it executes all finally (or fault) clauses it finds on the way to the handler it identified on the first pass. When the handler is reached, it is executed, and the exception has finally been handled.

But why is that bad?

“Okay”, you say, “I get it. I understand that filters run during the first pass. I can deal with that… what’s the big deal?” Well, let’s first consider what a finally clause is for. We typically use finally clauses to ensure that our program state remains consistent when exiting a function even in the face of an exception. We put back temporarily broken invariants. Consider that the C# “using” statement is built using try/finally, and then consider all the things one might do with that.

But when your filter runs, none of those finally clauses have executed. If you called into a library within the associated try block, you may have not actually completed the call when your filter executes. Can you call back into the same library in that case? I don’t know. It might work. Or it might yield an assert, or an exception, or, well, your guess is as good as mine. The point is that you can’t tell.

Using filters wisely (or, “the good”)

But the notion of a conditional catch clause really is quite appealing, and there are ways to use these without getting caught by the problems of when the filter actually executes. The key is to only read information from either the exception object itself, or from immutable global state, and to not change any global state. If you limit your actions in a filter to just those, then it doesn’t matter when the filter runs, and no one will be able to tell that the filter ran out of order.

For instance, if you have a fairly general exception, like COMException, you typically only want to catch that when it represents a certain HRESULT. For instance, you want to let it go unhanded when it represents E_FAIL, but you want to catch it when it represents E_ACCESSDEINED because you have an alternative for that case. Here, this is a perfectly reasonable conditional catch clause:

                Catch ex As System.Runtime.InteropServices.COMException When ex.ErrorCode() = &H80070005

The alternative is to place the condition within the catch block, and rethrow the exception if it doesn’t meet your criteria. For instance:

                Catch ex As System.Runtime.InteropServices.COMException

                                If (ex.ErrorCode != &H80070005) Then Throw

Logically, this “catch/rethrow” pattern does the same thing as the filter did, but there is a subtle and important difference. If the exception goes unhandled, then the program state is quite different between the two. In the catch/rethrow case, the unhandled exception will appear to come from the Throw statement within the catch block. There will be no call stack beyond that, and any finally blocks up to the catch clause will have been executed. Both make debugging more difficult. In the filter case, the exception goes unhandled from the point of the original throw, and no program state has been changed by finally clauses.

The problem is that we rely on programmer discipline in order to use filters correctly, but it’s easy to use them incorrectly and end up with infrequently executed code (exceptions, after all, are for exceptional circumstances) that has subtle and hard to diagnose bugs due to inconsistent program state that should have been cleaned up by finally clauses further down the stack.

Why does the CLR use a two-pass exception handling model?

The CLR implements a two-pass exception handling system in order to better interoperate with unmanaged exception handling systems, like Win32 Structured Exception Handling (SEH) or C++ Exception Handling. We must run finally (and fault) clauses on the second pass so they run in order with unmanaged equivalents. Likewise, we must not execute filters later (say, on the second pass) because one of those unmanaged systems may have remembered that it was supposed to be responsible for handling the exception. If we were to run a filter late on the second pass and decide that a managed clause really should catch the exception after previously having not declared that on the first pass, then we would violate our contract with those unmanaged mechanisms with unpredictable results.

So, in short, it’s for interop, and like many things involving interop, we have a compatibility burden that we can’t ignore.

Would a one-pass model be better?

Many have wondered over the years if perhaps the two-pass model in general is bad, and if a one-pass model would be better. Like so many things in the world, it’s just not that clear. A one-pass model would simplify the exception handling implementation, and it would make more sense in the cases shown above. However, there are advantages to the two-pass model that can’t be ignored. Perhaps the most important one is that if the search for a handler fails on the first-pass the exception goes unhandled and in general no program state has changed, even though filters are run since filters tend not to change things. The call stack is still intact, and all values that lead to the exception are still present on the stack and on the heap (assuming no race conditions.) This is frequently essential when debugging an unhandled exception. In a one-pass model, all of the finally clauses would have been run before the exception goes unhandled.

Wrapping up

Of the languages that MS ships, only VB and F# support filters, and both do so via conditional catch clauses. In F#, you have to really go out of your way to attempt to inspect mutable global state, or to actually have a side effect, so you’re fairly safe there. In VB, though, you can call a function from the “when” clause of their catch statement, and in there you have free reign to do whatever you please. You can, without a doubt, get yourself into trouble attempting to do too much complicated work within such a filter. To keep your world safe and simple try to limit yourself to expressions that only access the exception object, and don’t modify anything. If you go beyond that, you need to consider carefully all the code executed in the try block, and if your actions in the filter will work if the backout code below has not finished executing yet.

Mike Magruder,

Developer, CLR.

Leave a Comment
  • Please add 6 and 2 and type the answer here:
  • Post
  • > There will be no call stack beyond that

    This is actually a very obnoxious property of re-throwing existing exceptions, whether with `throw;` or `throw e;`. It gets in the way when one tries to write, e.g., a custom thread pool, or a mechanism for cross-thread invoke (no, standard ones that I know of don't fit the bill). We have little choice but resort to calling `Exception.InternalPreserveStackTrace()` with reflection.

  • Small note: "CLR implements a two-pass exception handling .. like .. C++ Exception Handling".  C++ exception handling isn't two pass -- C++ exception's don't have any user-visible "inbetween passes" state.  Did you mean the Visual C/C++ extensions __try/__except.

    In addition to VB and F#, Microsoft also ships Visual C/C++, which supports SEH exception filters (though does not provide much support for filters over C++ exceptions).

  • You might also want to mention the other reasonfor the two passes in Windows SEH -- an exception filter can choose to _resume_ the code that threw instead of unwinding.  It's not just convenient for debugging, it's a necessity for program correctness.

    It sounds like CLR exception handling doesn't provide support for resuming from an exception -- is that correct?

  • I wanted to resume after an exception in the CLR once, digged through SSCLI code and I think it's not possible (correct me if I'm wrong!) — even if you return EXCEPTION_CONTINUE_EXECUTION from a filter, it just gets treated like EXCEPTION_CONTINUE_SEARCH or EXCEPTION_EXECUTE_HANDLER (don't remember which one).

  • Anton - exactly. In C# I frequently write code like this:

    // start some transaction

    try

    {

      // use transaction

      // commit transaction

    }

    catch

    {

      // abort/rollback transaction

      throw;

    }

    The problem? FXCop (correctly) throws warnings for the unqualified "catch", and the "throw" recreates a new exception with a new stack, making debugging next to impossible.

    Very frustrating.

    Another case that's been agonizing:

    try

    {

     // sql action

    }

    catch (SqlException exception)

    {

      if (exception.ErrorCode != 1234)

      {

         throw;

      }

    }

    I can (almost) understand the lack of filters in C#, but I still can't understand why "throw" recreates the exception's trace stack and why "fault" is not supported. try {} fault {} seems like such an obvious choice for enterprise applications.

  • In the first case, there is an established workaround idiom — boolean "success" flag + finally block. In cases like the second one, try-catch blocks like these often directly wrap a library call, while in-library stack traces are usually of little interest, so the stack trace problem is not so acute.

  • Seems like long ago, was done a wrong decision to simplify the implementation of exceptions. With CLR, things get more complicated. Eventually something has to be done, longer it takes harder its going to be.

  • On C++ and a two-pass system: yes, I should have been clearer and said I meant __try/__except, since those are the most obvious ways to see the two-pass system via native C++. MS VC++ also uses the two-pass system for C++ exceptions. In fact CLR exceptions and standard C++ exceptions are both implemented on top of Windows Structured Exception Handling (SEH). With just standard C++ exceptions you can indeed see that they are two-pass: in an unhandled exception filter, or when stopped in a debugger on an unhandled exception event.

    On resumption: yes, Windows SEH does allow one to resume execution from an exception handler function (i.e., the expression of the __except clause). Execution continues based on the state of the CONTEXT passed to the handler, which if left unchanged by the handler means execution will resume at the faulting instruction. The CLR does not provide a similar facility at the IL level, and the CLR also does not support resumption after a managed exception.

    On the two code snippets above, Anton is correct: a work-around for the lack of fault clauses in a language is to use a Boolean success flag and a finally block. Set the success flag at the end of the try block, and use it to avoid taking action in the finally block. The compilers (source to IL, and IL to native (i.e., JITs)) will respect the order of operations in the try block to ensure this kind of thing works.

    Finally (no pun intended), on throw/rethrow and stacks: in the post above, when talking about how the catch/rethrow pattern alters the stack, I was referring to the actual thread stack and not the stack trace strings present in the managed exception objects. This makes live and dump debugging on an unhandled exception difficult. I do know there are confusing aspects to the exception stack trace strings and how they change on throw/rethrow, but I’m not familiar enough the details to give you any good suggestions right now. I’ll work on getting you that in the next couple of days. I do need to say, though, that using reflection to call an internal, private method like Exception.InternalPreserveStackTrace() is extremely brittle. No one here can guarantee that the method will continue to exist in future versions of the CLR, or that it will continue to have the same semantic. I do note from a quick inspection of our code that it was not designed to be called on every managed exception object that you might encounter. In particular, the system avoids calling this for cases where the runtime must preallocate and share the same exception instance across multiple threads. These are some, but not all, instances of the out of memory exception, and the thread abort exceptions (rude and normal).

  • @Anton and to add to what Mike mentioned above:

    StackTrace string is indeed preserved on rethrow. The point of rethrow will figure out as an entry in the stack trace string.

    "throw e" is not rethrow. This is equivalent to throwing a new exception and thus, will result in resetting the stack trace string in the exception object. Hence, if you use this semantic inside (or even outside) the catch clause, you will lose the corresponding previous stack trace that may have been collected in the exception that was caught. This is by design.

    Also, it should be noted that the notion of rethrow is applicable *only* from within the context of an active exception and while you are inside the catch block, exception context is deemed active. Thus, “throw” serves the purpose of rethrowing an exception.

  • @Anton: oops! We went to cleanup the extra comment posts that you added and asked us to remove, but we accidentally removed one too many. I’m very sorry about that. Could you please repost your comment? I believe this is the link to your excellent post about the issue with InternalPreserveStackTraceString: http://yama-mayaa.livejournal.com/14588.html

  • Mike: thanks! I don't remember my posts exactly, though.

    Mike: in this post (http://yama-mayaa.livejournal.com/14588.html) I propose a method of preserving the stack trace in an existing exception which uses only public APIs.

    Regarding preallocated exceptions: AFAIR these are the ones that cannot be caught — i.e., CLR re-raises them at the end of a catch clause which attempts to handle them. In fact, there are circumstances (rude thread abort being one) when only CER-protected exception handlers are executed, while after a stack overflow even those don't get a chance.

    Gaurav: you are right about "re-throw" vs "throw" — I have edited my blog post accordingly.

    > This is by design.

    Indeed, but sometimes this functionality is indispensable — the Remoting team's code supports my case.

  • Hello Mike,

    > Is it a good idea to add them to my new language?

    It's not about _a_ language. The question is always about _the specific_ language, C#. Exception filters are missing in C# whereas they're present in VB.NET, what causes many heart bleeding.

    It doesn't matter how hard you'll try, I refuse to understand why a feature is absent in C# when it is present in VB.NET. If something works in VB, it could even more easily work in C#. That's my way of thinking, despite the fact that it may be as stupid as thinking that the Earth is flat and the Sun orbits around it.

    It's just that over the years I got used to C#, learning it was a huge investment to me, I'm pretty much proficient about the language and the BCL, and articles like this make me feel like I bought wrong shares, because you stuff up another language with features, forgetting about this one.

    And I'm not going to switch to VB, because I'm genetically unable to learn it. I tried several times and I failed. I cannot go beyond some few-liners in Excel or Access. To me it's totally unreadable. Some say than Perl is unreadable, I think they haven't seen VB. It's cluttered with thousands of meaningless keywords which I can't remember or understand.

    I kindly ask you to fix this situation and allow usage of exception filters in C# as soon as you can. Same applies to all features missing from C# present in VB.

    Thanks.

  • @Anton: on the preallocated exceptions, these are used for many different things. In some cases, we preallocate one to ensure that we can throw something in low memory situations. OOM is a good example here, and it can be caught and handled just fine. You are correct, though, that the thread abort exceptions will be automatically re-raised at the end of a catch block, or not delivered to a non-CER catch depending on the circumstances. Note, however, that while in the catch block one could easily change the state of such an object, though, even though it will be re-raised. I do want to be clear, though: whether we throw a preallocated exception object is an implementation detail, and may change over time. For instance, all OOM’s are not the preallocated instance today.

    On the loss of the string representation of the stack trace, we’ll dig into this and see if there is something reasonable we can do in a future version of the CLR.

    @Levy: I’m sorry, but whether filters exist in C# or not isn’t up to us here on the CLR :) I can’t honestly say that I know exactly why filters are not exposed by C#, but I suspect it has something to do with “the bad” I mention in my post above. I do know that people frequently ask the C# team to expose filters, but they have clearly so far chosen not to do so. Knowing the people involved, I believe this has been something they’ve thought through carefully.

  • @Mike: actually, the whole issue of pre-allocated exceptions could be beside the point in this discussion: doesn't CLR itself modify this pre-allocated exception before throwing it?

    On stack trace string: that would be great even if available only in a future version of the CLR, although this fix (?) could just possibly go into a service pack for current/older versions of CLR — they aren't going to go away in the near future.

  • Good post.

Page 1 of 1 (15 items)