Paul's MCS Developer Blog

Sharing and blabbing on my findings as a Microsoft consultant

Posts
  • Paul's MCS Developer Blog

    SharePoint Foundation as a Search API Layer for hobbyist apps

    • 0 Comments

    I’ve been a SharePoint developer on a variety of customer projects. At least 1/4 of my work has been focused on Enterprise Search using SharePoint and previously FAST Search. Most recently, I worked on a common web content management platform using cross-site publishing and extensive UI work involving the SharePoint Search REST API.

    I make mobile apps professionally (as a Microsoft consultant, I’ve worked on Windows and mobile web apps) and as a hobby. My primary hobby app, which I maintain as a service for the military enthusiast, survivalist and Army communities, is Army Field Manuals. It provides a library of a bunch of PDF files published by the U.S. Army.

    I’ve also recently completed my MIM degree, in which had some close encounters with librarian- and archivist-type people. These are dedicated folks who organize and classify information, and many of them are extremely tech-savvy. This grad school work inspired me to want to do a better job than the Army has done to expose the public contents of Army doctrine. The primary ways I’ve come up with to do this are to integrate a variety of sources, provide some manual classification, and provide a robust search interface.

    How do I do full-text search in a mobile (Windows) application?

    Search in my app currently only searches the titles (and soon I’ll be adding descriptions). Unlocking the full text of the documents for search requires (at least in Microsoft lingo) an IFilter for PDFs. The technology is built-in with Windows 8.1, which provides hooks through IndexableContent,, but since the 3GB of field manuals are only downloaded to the app clients on demand, there’s no way to provide full text searching across the whole corpus.

    Enter SharePoint Search. SharePoint can search PDFs natively, can provide full text results with hit highlighting, and can return the metadata associated with the document in context. It can also provide rank tuning (in my case, I might want to do things like rank highly rated documents (LibraryThing has some ratings)). So, blending all these ideas I had an epiphany that SharePoint could provide some good value for my app community. So I’ve been figuring out how this might work and be affordable.

    What about SharePoint Online for Office 365?

    I’ve tried this, but the anonymous site you get seems to preclude using the search API from an anonymous user. When I run this query logged in, it works. As anonymous I get a file not found error. In the SharePoint Foundation solution below it works fine, so I’m assuming this is a specific exclusion of functionality in the public site on Office 365.

    /_api/search/query?querytext=%27army%27&QueryTemplatePropertiesUrl=%27spfile://webroot/queryparametertemplate.xml%27&SummaryLength=360&KeywordInclusion=%271%27

    What’s the cheapest way to use SharePoint’s API layer?

    Shared hosting of SharePoint Foundation is cheapest. Even though no CALs are required for External Users in the other versions, you’d need a CAL for yourself at least—and where would you purchase that as a hobbyist developer? And after that, the 3rd party hosting providers charge a lot more money for standard or enterprise licensed SKUs. I’m still discovering the features that SharePoint Foundation can do, since I typically work with the Enterprise SKU of SharePoint. However, I’ve signed up for a 3GB plan (enough to hold all the public Army doctrine PDF documents) on PlexHosted.com (affiliate link) for about $25/mo, and I’ve found that I can add site columns, access the search REST API anonymously, and access the lists API as well. So far so good. The Term Store is not available; I was hoping to use that to help tag content. However, a multiselect lookup will work for what I’m doing in the short term.

    What are the limitations?

    There’s more to APIs than anonymously exposing metadata and search results. This solution doesn’t do authentication or much of anything in the way of security or protections for the system against abuse. There is some logging, however, so hopefully that could prove useful. Also the system will be on a shared instance to keep the costs manageable. Not sure if that would scale. Another down side that I’ve found is that my search API access is lumped in with some other sites’ anonymous results. That doesn’t have any real impact for me, but it’s definitely not secured across site boundaries.

  • Paul's MCS Developer Blog

    Share rendered XAML content with Share contract and RenderTargetBitmap

    • 1 Comments

    A new feature of Windows 8.1 is that developers can render the XAML tree to a bitmap. This can be useful in exposing a screenshot of a part of an application with the image Share contract. Note that Windows 8.1 supports sharing a screenshot of any app through a new Share charm option. This is slightly more specific and also lets you control properties like Title.

    In order to make this happen, you have to go through a few hoops. Here is a sample event handler that wires up the pieces together including converting to a PNG image to bring the file size down. See MSDN’s “How to share an image” article for configuration.

    In this case, we have a ScrollViewer in an ListView control (it’s used in a continuous document viewing section of an app). The method shares what is currently visible on the screen.

    private async void OnDataRequested(DataTransferManager sender, DataRequestedEventArgs e)
    {
    var scrollViewer = RecurseChildren<ScrollViewer>(lvImages).FirstOrDefault();

    if (scrollViewer != null)
    {
    DataRequest request = e.Request;
    request.Data.Properties.Title = "Clip from an app";

    DataRequestDeferral deferral = request.GetDeferral();

    try
    {
    var bmp = new RenderTargetBitmap();
    await bmp.RenderAsync(scrollViewer);

    var pixelBuffer = await bmp.GetPixelsAsync();

    var reader = DataReader.FromBuffer(pixelBuffer);
    var bytes = new byte[pixelBuffer.Length];
    reader.ReadBytes(bytes);

    var randomAccessStream = new InMemoryRandomAccessStream();
    var encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.PngEncoderId, randomAccessStream);
    encoder.SetPixelData(BitmapPixelFormat.Rgba8, BitmapAlphaMode.Straight, (uint)bmp.PixelWidth, (uint)bmp.PixelHeight, 96, 96, bytes);
    await encoder.FlushAsync();
    await randomAccessStream.FlushAsync();

    request.Data.SetBitmap(RandomAccessStreamReference.CreateFromStream(randomAccessStream));
    }
    finally
    {
    deferral.Complete();
    }
    }
    }

    Here is an example from my Army Field Manuals app. Note that the share item doesn’t include the app border, which is handy in this case:

    share_rendertargetbitmap

    Additional references used:


    Note: This blog post is based on Windows 8.1 RTM and Visual Studio 2013 RC.

  • Paul's MCS Developer Blog

    PDF Viewing Components for Windows Store apps (WinRT) (XAML/C#)

    • 6 Comments

    Updated 4/7/2014 to reflect Windows 8.1 PDF API capabilities and new SDK samples

    Microsoft released a componentized, high-performing PDF viewing component for XAML/C# applications on 4/2/2014. Woohoo! This serves as a good reference sample of how this needs to be done (C# front-end with a C++ back-end).

    If you want more bells and whistles, a number of third parties add more to the reader experience such as alternate viewing methods (facing page vs. continuous view, etc). Figure 1 shows a breakdown of all the components that I have seen. These are all expensive (from $495 to “if you have to ask, it’s too expensive”), so they might not work well for a hobbyist app developer. But if you have a need in your commercial or LOB application, you may find one of these to be useful. 

    Vendor / Sample App Rendering Annotations Full Text Search License
    PDFTron Mobile PDF SDK / Drawboard Proprietary Yes Yes $$$$
    Foxit Embedded PDF SDK for Windows RT / Foxit Mobile PDF Proprietary Yes Yes $$$$
    PDF Xpansion SDK / Perfect PDF Proprietary Yes Yes $$$$
    DevExpress Windows 8 XAML Controls / Army Field Manuals Microsoft No No $$
    ComponentOne XAML Controls / ComponentOne XAML Controls Microsoft* No No $$
    Syncfusion Essential Studio for WinRT/XAML Proprietary No No $$
    Windows.Data.PDF Microsoft No No MS-LPL
    PdfShowcase Example Microsoft No No MS-LPL
    MuPDF WinRT / PDF Touch Proprietary Yes No? GPL / $$$$
    Offline Rasterization w/Ghostscript Proprietary No No GPL / $$$$
    Any .NET Brokered Component** Proprietary Not Sure Not Sure Varied

    Fig 1. PDF Viewing Components for Windows Store applications

    * Offers two rendering modes – one that converts to XAML and another that uses the PDF API
    ** Windows 8.1 Update offers the ability for enterprise apps to broker desktop components for use in WinRT applications.

    If you know of others, please post a reply or send me a tweet @paulwhit. I’m actively monitoring this topic.

    Also Ahmed-Faoud has ported MuPDF for use in Windows Store applications. This is very fast, but carries a GPL license.

    If high performance viewing of large or complex documents isn’t a consideration for your use case, you could provide rasterization to bitmap images either within your application at runtime or on a load event or background task, or by using a batch process like below. You could then load them in a ListView control and you should get pretty good performance. Rasterizing the PDF pages on the fly in a ListView is probably going to be too slow, especially on a low-powered device.

    As an example, I’m using some C# code in a batch application to generate thumbnails for my Army Field Manuals app. I’m distributing the images with my app, so there’s no processor use on the client.

    To rasterize the first page of a PDF document as a thumbnail, in a Windows Store application, add this to a button click event handler. The “file” variable is a file path. A quick way to access a local file is to put it in your %localappdata%\packages\{app id}\LocalState path and use ApplicationData.Current.LocalFolder to retrieve its file name.

    var pdfDoc = await PdfDocument.LoadFromFileAsync(file);
    var pdfPage = pdfDoc.GetPage(0);

    var pageImageFile = await ApplicationData.Current.LocalFolder.CreateFileAsync(
    "sm_" + Path.GetFileNameWithoutExtension(file.Path) + ".scale-100.png", CreationCollisionOption.ReplaceExisting);
    var randomStream = await pageImageFile.OpenAsync(FileAccessMode.ReadWrite);
    PdfPageRenderOptions options = new PdfPageRenderOptions();
    options.DestinationWidth = 250;
    await pdfPage.RenderToStreamAsync(randomStream, options);
    await randomStream.FlushAsync();

    randomStream.Dispose();
    pdfPage.Dispose();

    There is a team dedicated to the PDF APIs at Microsoft. You can find the PDF API team’s blog led by program manager Shalu Gupta on TechNet.

    For the record, I am currently using DevExpress in my Army Field Manuals app.

  • Paul's MCS Developer Blog

    Unzipping files with a progress bar in Windows Store applications (XAML/C#)

    • 0 Comments

    Here’s a method for unzipping a Zip file in a Windows Store application. ZipArchive is part of System.IO.Compression.

    private async Task UnzipWithProgress(StorageFolder outFolder, Stream zipFileStream, ProgressBar pb)
    {
        bool bDelete = false;
    
        using (var archive = new ZipArchive(zipFileStream))
        {
            try
            {
                // Ensure the UI update part runs in the UI thread
                var ignore = this.Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>
                {
                    this.tbDownloadStatus.Text = "Unzipping";
                    pb.Value = 0;
                    pb.Maximum = archive.Entries.Count;
                });
    
                foreach (ZipArchiveEntry entry in archive.Entries)
                {
                    if (entry.Name != "")
                    {
                        string fileName = entry.FullName.Replace("/", @"\");
                        StorageFile newFile = await CreateFile(outFolder, fileName);
                        Stream newFileStream = await newFile.OpenStreamForWriteAsync();
                        Stream fileData = entry.Open();
                        byte[] data = new byte[entry.Length];
                        int len = await fileData.ReadAsync(data, 0, data.Length);
                        while (len > 0)
                        {
                            await newFileStream.WriteAsync(data, 0, len);
                            len = await fileData.ReadAsync(data, 0, data.Length);
                        }
                        await newFileStream.FlushAsync();
    
                        var ignore2 = this.Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>
                        {
                            pb.Value++;
                        });
                    }
                }
    
                var itemId = OriginalUriString;
                this.Frame.Navigate(typeof(ItemDetailPage), itemId);
            }
            catch (Exception ex)
            {
                bDelete = true;
            }
            finally
            {
                if (bDelete)
                    outFolder.DeleteAsync();
            }
        }
    }

    To use it, call it this way. In my case I called it from the LoadState method.

    StorageFile file;
    try
    {
        file = await ApplicationData.Current.LocalFolder.GetFileAsync(OriginalUriString);
    }
    catch (FileNotFoundException)
    {
        file = null;
    }
    
    if (file != null)
    {
        using (var ws = await file.OpenReadAsync())
        using (var forReadRandomStream = ws.AsStreamForRead(0))
        {
            string path = ApplicationData.Current.LocalFolder.Path + @"\" + Path.GetFileNameWithoutExtension(file.Name) + @"\";
            await UnzipWithProgress(await StorageFolder.GetFolderFromPathAsync(path), forReadRandomStream, progressBar1);
        }
    }

    Apologize that this is not a File->New demo, but I thought it might be useful and Scott Hanselman made me do it.

  • Paul's MCS Developer Blog

    Office 2013 release day questions

    • 1 Comments

    So, being the good dogfooder that I am, I'm posting this using Word 2013 on release day.

    Looks like this may be the future of Windows Live Writer (according to the tea leaves). It feels a lot like Writer initially. It can do a lot out of the box but I'll spare you the needless includes from Flickr and other locations. Although I don't see Insert->Code, maybe cut and paste from Visual Studio is formatted well enough.

    Even though I'm a Microsoft employee, I haven't been privy to anything with this release—in fact, I don't think I was able to glean anything other than what's been posted online. I'm looking forward to learning more in the next few weeks and months. I'm particularly intrigued by the sync story with Skydrive and the investment in web versions of the apps. Also fascinating to think about is 3rd party apps integrated with Office 365, Skydrive, and Office Web Apps. Looks like there's an Office store where folks can acquire these add-ons.

    Here are a few questions that I have after an hour working with some of the release. There seems to be a cohesive connected-device picture coming together.


    Word 2013 Questions

    • Will the sharing features (such as blog posting) be available in the web app?
    • Will there be an RSS reader online to compete with Google Reader?
    • Will real-time collaboration be backward compatible with Office 2010?
    • How well does the PDF editor work?


    Outlook 2013 Questions

    • I imported my Google Reader RSS feeds OPML file. Now, when I click on RSS Feeds, it goes to my Google Reader. Is this a fluke?
    • Anything new in people merging? Will my merged contacts sync with the Metro Mail App and Windows Phone? It's a lot of work to redo these associations.
    • I see you can add Facebook and LinkedIn social connectors. Where's Twitter?
    • Are live tiles coming?


    OneNote 2013 Questions

    • Can OneNote edit PDFs like Word can?


    Other/Random

    • Why aren't we seeing the Metro style versions in this beta release?
    • Any new authentication mechanisms planned for Skydrive (Facebook?)


    I'll probably figure some of this out, but if anyone has some of this information already, feel free to include it in the comments.

    Thanks!

    p.s. Man this thing posts a LOT faster than Writer.

Page 1 of 4 (19 items) 1234