Opening Files from SkyDrive using .NET

Opening Files from SkyDrive using .NET

Rate This
  • Comments 9

Since Windows 8 I fell in love with SkyDrive and use it all the time now. Windows 8.1 has improved this a lot by introducing a new concept that requires some changes to the way you interact with files. In, this post Gaye Oncul Kok, Program Manager on the .NET Framework team, explains the key things .NET developers need to know when they read and write files stored on SkyDrive.

SkyDrive for Windows 8.1 introduced a new technology, called smart files, which gives access to the files in the cloud by providing their content on demand. The technology was designed to minimize the disk space utilization on your Windows 8.1 device. You can think of the smart files as the avatars of your cloud files on a device. They have the same appearance as regular files, allowing you to browse, search and do common file operations like viewing the properties or a thumbnail of the file without downloading the full content locally. When you want to open the file, or explicitly want to make it available offline, only then are the file’s contents streamed to your device.

The following image shows the properties of a smart file. You can see that this file uses 40KB space on the device even though its full size is 3.95MB in the cloud.

SkyDrive's placholder files are smaller

From a .NET developers’ perspective, if you are developing a Windows Store App or a desktop app targeting platforms that has Windows Runtime support, your app can consume smart files just like regular files by using Windows.Storage APIs. However, applications that depend on .NET Framework System.IO APIs, such as File.Open() or FileInfo.Open(), will have problems, when operating on a smart file unless its content is fully downloaded on the device. This is due to the fact that the smart files are supported in the Windows Shell layer and above, whereas System.IO APIs resides on the Win32 layer.

If you want your desktop app to run on non-Windows Runtime platforms, such as Windows 7, and also work with SkyDrive smart files available with Windows 8.1 you can either use the related Shell APIs through COM-interop and platform invocations or provide two different implementations and distribute two versions of your app; one for Windows Runtime and the other for non-Windows Runtime platforms. There is a third option, which is provided in sample code. This code uses reflection to utilize Windows.Storage APIs, such as StorageFile.GetFileFromPathAsync and StorageFile.OpenAsync(), if the Windows Runtime types are available on the platform or falls back to System.IO APIs otherwise.

The code provides the following public static methods in SmartFileLightUp class:

public static Task<Stream> OpenReadAsync(string filePath)
public static Task<Stream> OpenWriteAsync(string filePath)
public static Task<Stream> OpenWriteAsync(string filePath, bool createFile)
public static Stream OpenRead(string filePath)
public static Stream OpenWrite(string filePath)

All of these eventually call into the private async method:

private static Task<Stream> OpenStreamAsync (string filePath, FileAccess mode) 

In a nutshell, OpenStreamAsync calls StorageFile.GetFileFromPathAsync to get a StorageFile object as and then calls StorageFile.OpenAsync() to open a random-access stream over that file. The returned random-access stream is converted to System.IO.Stream so that you can continue using the System.IO APIs once it is open. These operations performed when the application is running on a platform where the required Windows.Storage types exist. Otherwise, OpenStreamAsync calls File.Open() to open the stream.

SmartFileLightUp class includes OpenRead and OpenWrite in addition to their asynchronous counterparts so that you can use them just as a replacement for File.OpenRead() and File.OpenWrite(). Both async and sync versions do not wait for the full content of the smart file to be downloaded. The difference is that Sync waits for the calls to WinRT functions to return but the Async does not.

Let’s look at an example usage. The code below simply opens a file for read and then writes its content to the console. You can call SmartFileLightUp.OpenRead() instead of File.OpenRead and it will work just fine with both regular files and smart files:

public void PrintFileContent(string path)
{
    //using(Stream s = File.OpenRead(path))
    using (Stream s = SmartFileLightUp.OpenRead(path))
    {
        byte[] b = new byte[1024];
        UTF8Encoding temp = new UTF8Encoding(true);
        while (s.Read(b, 0, b.Length) > 0)
        {
            Console.WriteLine(temp.GetString(b));
        }
}

Another example is the code below which uses SmartFileLightUp.OpenWrite() to open the stream for a given file path and then appends the text given as parameter to the end of the file:

public async void WriteToStream(string path, string text)
{
    if (string.IsNullOrEmpty(path))
        throw new ArgumentNullException("path");

    using (Stream s = SmartFileLightUp.OpenWrite(path))
    {
        s.Seek(0, SeekOrigin.End);
        UnicodeEncoding uniencoding = new UnicodeEncoding();
        byte[] result = uniencoding.GetBytes(text);
        await s.WriteAsync(result, 0, result.Length);
    }
}

We provided the synchronous versions of each method in case you want the minimum code change for some legacy code; however we recommend using asynchronous version OpenReadAsync, and OpenWriteAsync directly. Below is an example code from a Windows Forms Application, which reads the wallpaper image from a path for displaying it in a PictureBox:

private async void button_Click(object sender, EventArgs e)
{
    string path = InitializeWallpaperPath();
    var s = await SmartFileLightUp.OpenReadAsync(path);
    var img = Image.FromStream(s);
    pictureBox1.Image = img;
}

OpenWriteAsync(string filePath, bool createFile) is an overload that we provided to for creating the file before opening if it does not already exist. Calling this method with createFile = false is equivalent to calling OpenWriteAsync(string filePath).

Note that by using Windows Runtime APIs, you are accessing the smart files in a way that would not block the caller to wait for the full file content to be downloaded, which could take a long time if the file is large. Instead, sections of a smart file will be progressively downloaded and cached as they’re needed by the application reading the file.

It’s also important to mention that this sample code does not differentiate between smart files and normal files; for platforms that support Windows Runtime all files will be opened via Windows Runtime APIs. Also, the code does not use reflection every time to get the Type and MethodInfo objects for invoking. The first time these methods are retrieved, their delegates are cached so you will be invoking the delegates after the first call. This code also caches whether or not the WinRT types are available on the OS platform to minimize use of reflection on previous versions of Windows: After the first call, it will just fall back to System.IO.File.Open when running on non-WinRT platforms.

Since the calls to Windows Runtime APIs are through reflection, you can just include the provided *.cs files in your existing projects and compile without any additional settings required for using Windows Runtime APIs in desktop applications. If you choose to compile the class library and reference it in your projects, that is fine, too.

Summary

SkyDrive for Windows 8.1 introduced a new technology, called smart files. These files can be consumed just like regular files by using Windows Runtime APIs. However, applications that uses System.IO APIs will have problems when opening a smart file unless its content is fully downloaded on the device. With this blog post, we provided you a sample code that invokes Windows.Storage APIs by reflection. This sample code will enable you to write desktop apps that operates on smart files available on Windows 8.1 and also supports running on non-Windows Runtime platforms.

Enjoy consuming new smart files in your desktop apps and let us know if you have any feedback!

Leave a Comment
  • Please add 7 and 4 and type the answer here:
  • Post
  • A couple of things, I don't really understand where the reflection comes in, as there doesn't seem to be any reflection code in the above samples.

    Secondly, it feels like this is a temporary measure. Surely in the long term, it all needs to be transparent and work with functions like File.ReadAllLines(), which I use all the time.

  • Thanks for the insights .NET team. Please let us sync the file attributes, especially the hidden attribute. answers.microsoft.com/.../9c22caca-75b1-4c01-bb00-9cb6947b7315

    OAN, can anyone from .NET team reply to this issue //github.com/madskristensen/WebEssentials2013/issues/258?

    I am very excited by all the latest feature you guys have introduced with .NET 4.5.1 and VS 2013. But if there is one thing nagging the contributors of WebEssentials VS extension; its the Visual Studio's assembly Microsoft.Css.Core..  That one is weirdly implemented with no documentation, comments for function signatures, so many inline classes and using mixed patterns (factory, DI/IOC..)! Can you guys please update it and make it consistent with Microsoft.Html.Core (with Render(TextWritter writer in each class), so Emmet CSS (zencoding: //githu.com/madskristensen/zencoding/) and CSS minifier (AjaxMin) can leverage it. See issue on github //github.com/madskristensen/WebEssentials2013/issues/152 and codeplex //ajaxmin.codeplex.com/discussions/466818.

    Then there is a licensing issue. We need to have VS installed to use the CSS AST provided by Css.Core, which is not applicable for the scenarios like AjaxMin (http://ajaxmin.codeplex.com/). Is there any plan to provide Css, Web, Less and Html Cores under framework's namespace.. perhaps System.Web? Please at least provide a basic AST in .NET framework, a strip-down (but extendable) version of the full implementation with the ability to parse and render the artifacts.

  • @Mark: We didn't show the reflection code in this post because it's more than a few lines --- the post focused on how you would use the sample code we provided.

    You can find the details on how we implemented the SmartFileLightUp class on code.msdn.microsoft.com. If you want to look at the reflection code, you find the general logic here.

    With respect to whether this is a temporary issue: we talked a lot with the SkyDrive team to decide whether it would make sense to handle this transparently or whether we should provide explicit APIs. There are several reasons why we decided to go down the explicit API route for now:

    From a layering perspective, the SkyDrive smart files are implemented at the shell level, which is well above Win32. Today, the .NET file system APIs depend only on functionality that is implemented at the Win32 level. We consider general file I/O to be fairly low-level and are afraid that moving it above the shell layer may have some ripple effects.

    More importantly: even if a .NET application doesn't know anything about smart files, many scenarios will just work. For example, if a .NET app uses the regular OpenFileDialog, the shell will make sure that file will be fully downloaded before the path is returned to the app. Also, if an end user double clicks a file in File Explorer, the shell will also make sure the file is fully downloaded before a registered application will be invoked.

    One scenario that is affected by smart files is if an application gets the path by some other means, for example, by directly enumerating the files in a directory and opening them. We believe that the majority of these apps are unlikely to operate over SkyDrive files (which are mostly documents and pictures).

    Now, both arguments don't mean we'll never provide an implementation in the BCL. We just felt that the cases where the app has to be aware of smart files is small enough that providing a sample is good enough for now. Depending on feedback we may decide to either provide a "proper" NuGet package that will provide this functionality or by handling it transparently in the BCL.

    @Adeel: Thanks, I've forwarded your comments to the appropriate owners.

  • Thanks for clarifying. It's good that you have covered the main scenarios where an application would get a path from. My vote would be for it to be somehow transparent in the BCL, as I wouldn't like to debug a problem where I forgot to implement the skydrive workaround and then the user got some strange error message when pointing to a skydrive folder which I then enumerate.

  • A interesting post but a real shame this sample does not address a rather fundamental issue with accessing smart files.  If the user has no network connection and tries to access an online only smart file through the Windows Shell such as a file picker then the file is greyed out and cannot be selected.  This makes sense.  However this sample code provided awaits the OpenAsync method returning the file which will not complete until the user regains access to a network.  Consequently the application just sits and waits for the operation to complete with no indication to the user that something is wrong.

    The Windows.Storage APIs for Windows 8.1 introduced a property "IsAvailable" which returns false if the file cannot be downloaded.  This really should have been worked into the sample code.

    It would be helpful for Microsoft to provide full blown solution to this which maps all key methods in System.IO such as File.Copy, File.Move etc. to their WinRT API equivalents and handles the IsAvailable issue. This issue is starting to crop up on support forums as it has caught many developers off guard.

  • @Andrew: Thanks for the feedback, we'll look into this.

    To set expectations: we'll probably not be able to provide the full blown solution that you asked for. However, we are checking whether we can at least expose the IsAvailable API so that callers can handle the case meaningfully, for example by displaying an appropriate error message or by graying out as done by Windows Explorer.

    That being said, we still would like to hear what folks think so that we can prioritize the work appropriately in the future.

    For your convenience, I've filed the following user voice item. I'll also ask @dotnet tweet about it to make sure it gets visibility by the community. In addition, feel free to forward this link whenever the question comes up, for example, in the support forums you mentioned.

    Thanks!

    visualstudio.uservoice.com/.../5328111-support-system-io-apis-for-skydrive-s-smart-files

  • @Immo  Thanks for rasing the profile of this.  Making the IsAvailable property available would certainly be a very welcome step forward.

  • According to documentation msdn.microsoft.com/.../windows.storage.storagefile.aspx StorageFile, as well as other classes from Windows.Storage namespace, can be used from Store apps only. This article suggests to use StorageFile method from desktop .NET apps. Is documentation inconsistent or does this article describe officially unsupported scenario?

  • Hello,

    I suggest you to download new Long Path Tool software that simply allows you to work easily on Long Path files.

    Thank you........

Page 1 of 1 (9 items)