Here are the full details I received from Grant Richins and Fei Chen when I asked how the JIT decides whether to employ the tail call optimization. Note that these statements apply to the JITs as they were when Grant and Fei looked through the code base, and are prone to change at whim. You must not take dependencies on this behavior. Use this information for your own personal entertainment only.
First, Grant talked about the 64-bit JITs (one for x64, one for ia64):
For the 64-bit JIT, we tail call whenever we’re allowed to. Here’s what prevents us from tail calling (in no particular order):
If all of those conditions are satisfied, we will perform a tail call. Also note that for verifiability, if the code uses a “tail.” prefix, the subsequent call opcode must be immediately followed by a ret opcode (no intermediate nops or prefixs are allowed, although there might be additional prefixes between the “tail.” prefix and the actual call opcode).
Fei has this to add about the 32-bit JIT:
I looked at the code briefly and here are the cases I saw where tailcall is disallowed: