Last week I helped a colleague who was experiencing UnobservedTaskExceptions I his code. The problem was essentially that the code started several tasks and then in a loop checked each one if it was faulted or not. If a task was faulted the method threw an exception. This meant that if two tasks faulted in the collection of tasks then the second one was never observed causing an UnobservedTaskException that brought down the process. While this sounds simple it turned out to be a hard nut to crack for a number of reasons.
First of all you need to know some things about the UnobservedTaskExceptions; while they by default crash your process in .Net 4.0, they don't in .Net 4.5. The fun thing is that you get the 4.5 behavior on your 4.0 assemblies by just having 4.5 installed. There is a way to configure your application to use the old 4.0 behavior and you can read about that and why the default behavior changed here. Forgetting this can frustrate you if your build environment does not have 4.5 but you have it on your own machine.
Second you have to remember that even if you use async/await you can still end up writing code that have the same "problem". The "problem" is that you only bubble up the first error you see and not all errors. For many reasons I think this is what you actually want (WhenAllOrError anybody?) but if you really want to get all exceptions you can just use Task.WhenAll and you'll be good.