If you have an opaque native structure that you want to pass through managed code, consider using IntPtr, not Byte[], in your marshalling signatures.  Opaque means that you don't care about the contents of the buffer. For example, if in C#, you pinvoke to get an opaque native structure that you then hand to some other native API, the pinvoke definitions should use IntPtr instead of Byte[]. 

1. IntPtr doesn't have extra copying. Byte[] lives on the GC heap and so the memory can be moved around. Thus you may need to copy from the unmanaged memory onto the GC heap (probably via Marsahl.Copy), and then back again.
2. IntPtr is the lowest-common-denominator because you can always get the address of managed object as an IntPtr via GCHandle.AddrOfPinnedObject.
3. IntPtr's aren't paying for extra functionality. If the opaque structure just comes into managed code and then gets passed back to native, then you don't need any functionality beyond an IntPtr.

Note I've been saying "opaque" native structure here, meaning that we just need the bytes to pass through managed code, but we don't need to actually look at them in managed code. That's in contrast to importing the native structure definition into managed code and then use normal marshalling techniques. This can work too, but may have some drawbacks:

1. Importing the structure may be painful.
2. If managed code just passes the native structure back to native methods without actually inspecting the structure, then importing it is unnecessary.
3. If the structure is not cross-platform, you may need different managed definitions for each platform. This could make your managed become platform dependent.

Another alternative may be to use unsafe IL, like C#'s fixed keyword, and the address-of operator. But now you've got unsafe code.

On a related note, check out Josh William's comments on Byte[] vs. byref byte.