If you have an opaque native structure that you want to pass through managed code, consider using IntPtr, not Byte, in your marshalling signatures. Opaque means that you don't care about the contents of the buffer. For example, if in C#, you pinvoke to get an opaque native structure that you then hand to some other native API, the pinvoke definitions should use IntPtr instead of Byte.
Note I've been saying "opaque" native structure here, meaning that we just need the bytes to pass through managed code, but we don't need to actually look at them in managed code. That's in contrast to importing the native structure definition into managed code and then use normal marshalling techniques. This can work too, but may have some drawbacks:
Another alternative may be to use unsafe IL, like C#'s fixed keyword, and the address-of operator. But now you've got unsafe code.
On a related note, check out Josh William's comments on Byte vs. byref byte.