Creating something from nothing, asynchronously [Developer-friendly virtual file implementation for .NET improved!]

re: Creating something from nothing, asynchronously [Developer-friendly virtual file implementation for .NET improved!]

David Anson — Fri, 24 May 2013 17:15:30 GMT

Joe,

Got it, thanks for clarifying! Unfortunately, you're working right at the boundary of my memory/experience, so I don't know the right answer. :( That said, I think you're on the right track - what I'd suggest is to look at the SetData method that calls CreateStreamOnHGlobal and see if there are other options (ex: dwAspect or tymed) that would allow you to do what you want (i.e., provide something other than an in-memory stream). Also, I think there's a deferred-render mode for clipboard data that might be relevant/useful here because it waits to provide the data until it's needed and the file name should be available at that time.

If you figure this out, please let me know what you ended up doing - I'd love to learn! :)

re: Creating something from nothing, asynchronously [Developer-friendly virtual file implementation for .NET improved!]

Joe — Fri, 24 May 2013 07:50:27 GMT

Sorry, chunks of 4096 bytes, not kb :)

re: Creating something from nothing, asynchronously [Developer-friendly virtual file implementation for .NET improved!]

Joe — Fri, 24 May 2013 07:40:32 GMT

Yeah, that is exactly what I'm doing. In the FileDescriptor.StreamContents callback, I open a stream to the file on disk, read it in chunks of 4096kb, and write each chunk to the stream provided by the VirtualFileDataObject in the callback. I guess that's the common way of doing such things. But it seems the problem is the line var iStream = NativeMethods.CreateStreamOnHGlobal(IntPtr.Zero, true); in the GetData function of the DataObject. This writes all chunks to the global memory, not to disc... so every time I write to the stream, I write to global memory and at some point this is full, which leads to a "Not enough storage" message. This even occurs, when I use the stream.copyTo function. I wish I could use SHCreateStreamOnFileEx instead of CreateStreamOnHGlobal, but this requires that I know the location of the file before dropping it to the explorer, but the file itself is created INSIDE the FileDescriptor.StreamContents callback.

So what I want to do is:

private void onStream(Stream stream)

{

// create the file after dropping some items of my application to the explorer to a temporary location

// waiting until file creation has finished, after this I know where I find the file and how it is named.

// Open a stream to the created file and "move" it to the location where it has been dropped. I do this by copying the bytes from the source stream to the target stream. But this leads to the problem I mentioned, because all is written to global memory. SHCreateStreamOnFileEx seems to work if I knew the filename in advance, but this is created after the stream has been opened.

}

re: Creating something from nothing, asynchronously [Developer-friendly virtual file implementation for .NET improved!]

David Anson — Thu, 23 May 2013 16:28:34 GMT

Joe,

It sounds like you're providing an implementation of FileDescriptor.StreamContents that reads from a file and writes it to a stream. When doing so, you want to read from the file in a series of small chunk and write each chunk to the stream before moving on to the next chunk. For example, create a 1000 byte buffer, read into it from the file, write that out to the stream, read the next 1000 bytes, etc.. In this manner, you'll only be using 1000 bytes of memory at a time no matter how large the file is.

This is such a common pattern that recent versions of .NET offer the Stream.CopyTo method (msdn.microsoft.com/.../system.io.stream.copyto.aspx) which handles this for you!

I hope this helps!

re: Creating something from nothing, asynchronously [Developer-friendly virtual file implementation for .NET improved!]

Joe — Thu, 23 May 2013 12:41:22 GMT

Hi,

I have a problem when trying to drag/drop large files using VirtualFileDataObject.

What I do is to open a filestream to the source file, reading the bytes from the source filestream and writing those bytes to the Stream,

which was delivered by the callback function of "fileDescriptor.StreamContents",

using the "stream.Write" function. But this seems to write all bytes from the source file to memory first, before dumping it all to the harddisc.

This sometimes leads to an OutOfMemoryException if the file is bigger than several hundered Megabyte.

Is there any solution to this? I have to admit that I'm not that skilled in COM programming, so I do not know what to change in the VirtualFileDataObject...

re: Creating something from nothing, asynchronously [Developer-friendly virtual file implementation for .NET improved!]

Vijay G — Thu, 23 May 2013 08:42:44 GMT

Thank you David for your valuable comment.

re: Creating something from nothing, asynchronously [Developer-friendly virtual file implementation for .NET improved!]

David Anson — Wed, 22 May 2013 16:41:01 GMT

Vijay G,

I'm not sure that's possible. If the target application doesn't want to opt-into asynchronous mode, I think that's its choice and that Windows doesn't try to shim the operation. As a drag-drop source, you should try to support both modes, but be happy to provide data however the target asks for it. :)

re: Creating something from nothing, asynchronously [Developer-friendly virtual file implementation for .NET improved!]

Vijay G — Wed, 22 May 2013 08:00:03 GMT

Hi David,

Really a great article, thanks a lot!!!

I am facing one issue, how to force synchronous process to asynchronous? My use case is - I am dragging file to Outlook in that case VirtualFileDataObject doesn't call the GetAsyncMode() function, like it get called when dragging to explorer. I've tried all option to set IsAsynchronous to true, but not succeed. Waiting for your valuable response.

Thank you.

re: Creating something from nothing, asynchronously [Developer-friendly virtual file implementation for .NET improved!]

David Anson — Fri, 26 Apr 2013 16:30:09 GMT

I don't see that you can - it's likely to be in a different process about which the drop source knows nothing, so it's not clear what kind of information could be provided to the source that would be generally useful.

re: Creating something from nothing, asynchronously [Developer-friendly virtual file implementation for .NET improved!]

Kei — Fri, 26 Apr 2013 09:59:29 GMT

How can I get Dropping location?