The world is a better place if you generate verifiable IL

re: The world is a better place if you generate verifiable IL

Sam — Sat, 05 Feb 2005 07:37:00 GMT

Good stuff,

Do you have any links to information on *how* to produce verifiable IL when using reflection emit?
I would like to learn more...

re: The world is a better place if you generate verifiable IL

David Notario — Thu, 10 Feb 2005 00:49:00 GMT

Sam, if your question is how to verify that the IL you generate is verifiable, 2 ways of doing it:

1) Save the assembly you emit to disk, run peverify on it. This will make sure that all methods in your assembly are verifiable

2) Refuse SkipVerification permission. You can add the following attribute to your assembly:

[assembly:PermissionSetAttribute(SecurityAction::RequestRefuse,Name="SkipVerification")];

The downside of this aproach is that verification will be done at runtime (ie, it verifies as you go, so maybe you won't cover all your code if not all is used).

Producing verifiable code is about following the IL verification rules as described in ECMA spec, they're mostly type safety rules, so I don't think you will find much stuff there that is suprising.

Let me know if you have any other questions.

re: The world is a better place if you generate verifiable IL

Jeroen Frijters — Mon, 14 Feb 2005 13:41:00 GMT

Hi,

I try to always generate verifiable code, but in some cases that isn't as easy as you might think. For example, is the following code verifiable? Whidbey's PEVerify doesn't think so.

.assembly extern mscorlib {}
.assembly test {}
.module test.exe

.class private auto ansi beforefieldinit Test
extends [mscorlib]System.Object
{
.method private hidebysig specialname rtspecialname
instance void .ctor() cil managed
{
.maxstack 1
.try
{
ldstr "try"
call void [mscorlib]System.Console::WriteLine(string)
leave.s label
}
finally
{
ldstr "finally"
call void [mscorlib]System.Console::WriteLine(string)
endfinally
}
label:
ldarg.0
call instance void [mscorlib]System.Object::.ctor()
ret
}

.method private hidebysig static void Main() cil managed
{
.entrypoint
.maxstack 1
newobj instance void Test::.ctor()
pop
ret
}
}

re: The world is a better place if you generate verifiable IL

David Notario — Wed, 16 Feb 2005 08:42:00 GMT

Jeroen, I believe the verifier wants the .ctor of Test to call its base constructor before doing anything else (to guarantee the state of the base class). If you just call Object::.ctor as the first thing in your .ctor you'll become verifiable.

I'll confirm this with verifier guys.

re: The world is a better place if you generate verifiable IL

David Notario — Wed, 16 Feb 2005 08:43:00 GMT

re: The world is a better place if you generate verifiable IL

Jeroen Frijters — Wed, 16 Feb 2005 19:22:00 GMT

Thanks. Believe me, I do understand the issues involved. I've written a Java VM for .NET and this is kind of annoying (I could work around it by hoisting the try/finally code into a static method).
BTW, the ECMA spec doesn't say anything about this (apart from the fact that you must call be base class constructor before returning from the constructor).
There's also a related issue. I'd like to be able to do (also in a constructor):

try
{
// constructor body
// including call to base class ctor
}
finally
{
GC.KeepAlive(this);
}

This is needed to make sure the object isn't GC'ed before the constructor ends (which is required by the JVM spec). However, this code isn't verifiable because this may be uninitialized and the verifier doesn't understand that GC.KeepAlive is a harmless method.
(This same issue also applies to Monitor.Enter/Exit, the verifier should also allow these methods to be called with an uninitialized this).

re: The world is a better place if you generate verifiable IL

David Notario — Thu, 17 Feb 2005 10:06:00 GMT

Jeroen, first, thanks for doing the right thing (being verifiable) I agree with you that the status quo is not the best it could be. I spoke with the verifier people. It turns out that some of our verifier rules are not in ECMA. Some of them are just implementation limitations we don't want to standarize and others that will eventually get there.

Let me know of other stuff you find, I'll make sure we're tracking it (I can start a post where I can add these as we find them, as a way of having them public until they get standarized)

I assume you need the KeepAlive for the case where the instance .ctor gets inlined and you have code like this

.ctor()
{
some_field = 0;
// if .ctor was inlined at this point, with no other references it could be collected.
Console.WriteLine("Hello");
}

Why do you need the GC.KeepAlive inside a finally? GC.KeepAlive should be treated as a side effect, so just adding the KeepAlive at the end should do what you want (assuming that in the case of an exception you don't care, as the .ctor will never complete)

If that doesn't work:

Translate

java_equivalent_of_newobj

to:

newobj instance_ctor
dup
call GC.KeepAlive

In both cases you also get rid of the EH overhead.

Let me know if you need anything else.

re: The world is a better place if you generate verifiable IL

Jeroen Frijters — Thu, 17 Feb 2005 12:46:00 GMT

Thanks. I think the exception scenario may also be important. For some constructor parameter values, the JIT may be able to inline the constructor and eliminate code in such a way that it can detect that an exception will be throw, this will affect the liveness of this the pointer. BTW, the reason this is important is not so much because of external side effects (like the Console.WriteLine example you give), but because the constructor might contain native method calls that use an unmanaged resource. If the finalizer runs before the constructor terminates, the unmanaged resource will be freed will the native code is running. This is obviously not good ;-)

I did consider your alternate suggestion (adding the KeepAlive after the newobj), but that doesn't satisfy me either, because other .NET languages obvious won't do that, so interop will suffer.

One potential solution I thought of yesterday is to add a call to KeepAlive after every call that occurs in the constructor (once the this has been initialized). That could still theoretically (in some obscure scenarios) cause problems when the base class constructor throws an exception, but I think it might be the best workaround.

re: The world is a better place if you generate verifiable IL

Jeroen Frijters — Thu, 17 Feb 2005 12:51:00 GMT

Oh, and additionally, the try/finally before the base class ctor call is actually a real issue as well (well, it's actually try/catch since the JVM doesn't have a try/finally construct).

Take the following Java code:

class Base {
Base(Class c) { ... }
}

class Derived {
Derived() {
super(SomeClass.class);
}
}

(The "SomeClass.class" construct is equivalent to C#'s typeof(SomeClass))

One particular Java compiler compiles the Derived ctor as:

Class c;
try {
c = Class.forName("SomeClass");
} catch(ClassNotFoundException) {
throw new NoClassDefFoundError();
}
super(c);

My current (straight-forward) translation of this construct is rejected by PEverify is not verifiable (however, note that the CLR verifier doesn't agree with PEVerify on this!)

What I'd like to know is whether PEVerify is right, or the runtime is right.

Dynamic Assemblies and Declarative Security

.Net Security Blog — Sat, 28 May 2005 02:50:25 GMT

Speaking of dynamic IL generation ...
Before Whidbey, the framework supplied two ways of creating code...

Managed C++ codegen for new, array manipulation, delete

Mike Stall's .NET Debugging Blog — Sun, 04 Sep 2005 06:37:42 GMT

Managed C++ (MC++) code generation is a cool accomplishment. I think it's
another good testimony...

Managed C++ codegen for new, array manipulation, delete

Mike Stall's .NET Debugging Blog — Sun, 04 Sep 2005 06:39:41 GMT

Managed C++ (MC++) code generation is a cool accomplishment. I think it's
another good testimony...

Managed C++ codegen for new, array manipulation, delete

Mike Stall's .NET Debugging Blog — Sun, 04 Sep 2005 06:52:29 GMT

Managed C++ (MC++) code generation is a cool accomplishment. I think it's
another good testimony...

Managed C++ codegen for new, array manipulation, delete

Mike Stall's .NET Debugging Blog — Thu, 08 Sep 2005 08:10:13 GMT

Managed C++ (MC++) code generation is a cool accomplishment. I think it's another good testimony to the...