Visual automated transcription of piano music – innovation from Imagine Cup!


    Beethoven inspired two masters students at the University of Lethbridge to come up with a new solution to the challenge of automatically transcribing piano music. In this post they talk about their project and winning the Innovation category of the Canadian Imagine Cup.

    ClaVisionScreenshotWhat category did you enter in the Imagine Cup?

    We entered the 2014 Canada Microsoft Imagine Cup Innovation Competition, a contest for the best new innovative software. Actually, this category is a good opportunity for those who have new ideas for implementing software innovations.

    What is your project/idea?

    Our software claVision is a completely new way to perform automatic piano music transcription by visually analyzing the piano keyboard, without having any audio of the played music. This idea was inspired by the famous composer, Beethoven. By the last decade of his life, he was almost totally deaf, forcing him to rely on the visual relationship between his fingers and piano keys. claVision simulates what a deaf musician like Beethoven does. It can be used on any kind of musical keyboard. It can also be used as an educational aid for students learning to play a piano. You can see a short video demonstrating their software here.

    Who is on your team?

    ClaVisionTeamPhoto Team claVision includes two members: Mohammad Akbari and Hossein Naseri, along with mentor Dr. Howard Cheng, from the University of Lethbridge. Mohammad is a Masters student in Computer Science. He is working as the team manager, image and video processing expert, software developer and music expert (pianist) of the team. Hossein is currently a Masters of Science student in Physics. In the team claVision, he is working as an expert in mathematics and audio processing. Dr. Howard Cheng is a faculty member in the Department of Mathematics and Computer Science in University of Lethbridge.


    Why did you enter Imagine Cup?

    Last October, the geekStarter program from Alberta Innovates Technology Future hosted an information session talking about the Microsoft Imagine Cup, which made us interested in this competition. So, after the suggestion of our team’s mentor Dr. Howard Cheng, we found this competition as an amazing opportunity to present our innovative software claVision.

    Did you think you would get this far when you entered?

    We knew our software was a new technology for solving an old problem. However, because it was our first competition of this kind and we were not aware of the other competitors and their projects, we did not expect to be selected as the winner of the 2014 Canada Microsoft Imagine Cup Innovation Competition. So, when we heard this fantastic news, we got totally surprised.

    What do you think you did really well with your project?

    There are some significant points making our software stand out. Unlike other similar products that perform automatic music transcription by “listening” to the music, in claVision, the audio of the played music is ignored. As a result, many of the drawbacks of existing transcription techniques from audio are no longer present. Both live (real-time) and recorded video processing can be handled by our software accurately. claVision has a number of other useful applications such as automatic improvised music transcription, piano education especially for hearing-impaired people. Moreover, after analyzing the demand for automatic music transcription, we have designed a business plan to get claVision to the market.

    If you were starting over again from scratch what would you do differently?

    Thanks to this competition, we are more experienced than before. We think if we had this experience at first, we would have thought about the business aspect more instead of just the technical side.

    What’s next for you after Imagine Cup?

    After this competition, we are planning to get claVision to the market. So, we want to consult with technology transfer experts on steps to commercialize our software product. Additionally, we have some other interesting ideas that we are interested in presenting them to this Microsoft Imagine Cup next year.

    What advice would you share with a student who is thinking of entering Imagine Cup?

    Microsoft Imagine Cup as a global student technology competition is the best opportunity that every student may have in their lives to present new ideas to the world. If you are looking for a great opportunity helping you achieve your success, do not miss it!

    What’s next?

    Team claVision now advances to the world semi-finals. During this stage they will be judged against the other Canadian category winners (Innovation, Games, and World Citizenship) and against other entries in the Innovation category from around the world. Ten innovation teams from around the world will be selected to attend the World Imagine Cup finals this July in Seattle. In June, we find out which of the top three Canadian teams will be selected to represent Canada at the world finals. Good luck team claVision!


  • Go DevMENTAL

    Music notations for Office – innovation from University of Manitoba


    Guest post from Kenny Hong, a student at the University of Manitoba who is working with fellow student Eric Fehr on Muusika, an app for Office to allow musical notations. His idea was selected by the Microsoft Office team as one of the top five worldwide in the Imagine Cup Apps for Office challenge! Kenny takes a few minutes to tell us about his idea and the Imagine Cup experience so far…

    imageWhat category did you enter in the Imagine Cup?

    I entered the Imagine Cup under the Apps for Office category to create an application that is used by the Microsoft Office Suites (ie. Word, Excel, OneNote).

    What is your project/idea?

    The idea is to have an application available to Word/OneNote where you are able to create musical ideas. We want you to be able to notate music straight onto Word which is already a powerful piece of software. There are a lot of complicated music notation software out there that take a lot of resources. Our project, "Muusika", is an attempt to create a way to simplify jotting down quick music ideas without having to spend a lot of time figuring things out.

    Who is on your team?

    Kenny Hong - 2nd year General Science student at the University of Manitoba

    Eric Fehr - 2nd year Computer Science student at the University of Manitoba

    Why did you enter the Imagine Cup?

    There wasn't a particular reason why I entered. My professor had talked about it a bit during one of his lectures, and I thought it would be a fun idea to enter, so I worked on a proposal and submitted it.

    Did you think you would get this far when you entered?

    I actually did not think I would make it through round one. It was quite the shock... I immediately tried to look for teammates so that the idea could actually come alive.

    What do you think you did really well with your project?

    I believe that the idea/concept was properly submitted. There are not a lot applications for office that involve working with music and I just wanted to be a part of creating an application that lets users jot down a quick musical idea.

    If you were starting over again from scratch what would you do differently?

    I would have assembled a team immediately and started working on the project earlier. That would have made this a bit easier, especially for an app like this.

    What’s next for you after Imagine Cup?

    I am fairly new to the world of Computer Science, and I now believe that this is a career I would like to pursue. After the Imagine Cup, I would love to continue the project, and perfect it so anyone can use it. I believe I can improve on it as I learn more about the world of programming. For now, after the competition, I will be helping some fellow students bring Tedx to the University of Manitoba!

    What advice would you share with a student who is thinking of entering Imagine Cup?

    Do it. It is a great opportunity and it can be quite a challenge too ( if you like challenges), but like I said above, get a good head start on it, it will make your life less stressful (exams pile up! Smile with tongue out).

    What happens next?

    Kenny & Eric are now working to bring Muusika to life. May 26th they will submit their completed app to the judges who will select two of the five final entries to attend the Imagine Cup World Finals in Seattle, Washington in July. Those two teams will compete for first place and $5000.  Join us in wishing them good luck in the next round, we hope to see them bearing the Canadian flag to the Imagine Cup world finals in Seattle this July!

    Here is their winning proposal


    Music is becoming a critical part in the lives of many people and learning the basics of music can be a pain without the right tools. A simple music notation software can be very expensive and complicated to use but writing by hand can result in wasting a lot of paper. This project, Muusika, is an app aimed towards providing users a simple way to jot and take down musical ideas without having to waste any paper. Muusika can also provide the younger and/or beginner music students a way to learn the basic and fundamentals of the staves, notation, theory, and composition. It also works the other way around, where music teachers are able to use this tool to teach students the fundamentals of music.

    There are currently templates on Microsoft Word for empty staves but it can be more effective by adding a feature where users are able to input notes onto the staves without having to look it up on the ASCII tables constantly. Users can also benefit from the organizational aspect of Microsoft OneNote by having notebooks where they are able to quickly jot down a melody so they don’t forget. Muusika will take advantage of any power features Word and OneNote may have, including the Pens and Pencil tools.


    This was created on OneNote using the line tool and the pen and pencil tool. The lines where complicated to straighten and kept on sticking together. Using a Wacom Bamboo tablet I was able to draw out the notes, Muusika will enable the use of an easier way input notes in Word and OneNote.

    Office Support

    Muusika will support the following applications:

    Word – For ease of use and professional looking arrangements and compositions without having to break the bank for a complicated professional music notation software.

    OneNote – If you have an idea, if you like to keep musical journals, if you like to work on a song where mistakes should be make, OneNote will provide the resources needed to organize the songs your composing, lyrics you are creating, music you are arranging, etc…

    Planned Content & Technologies

    To ensure the simplicity of the Muusika, it must have enough functions to be usable in the world of music but limited enough not to overwhelm and confuse the users. Muusika will give users the choice of drawing the notes using a tablet/drawing table or a simple point and click input system involving a toolbar and mouse. The toolbar should have the basics needed for music, all the clefs, notes lengths, rests, and ties with a customizable option to add concepts like crescendos, and volume control for the more advanced musicians.

    Muusika may also include a MIDI feature where it will playback the notes inputted via the notes tool.

    User Experience Story

    Let’s say one user is an aspiring teenage musician writing songs, the OneNote version of the application will enable the user to jot down the ideas for both lyrics and melodies. They can organize their ideas by each notebook being a genre, each section being a song, and each page being an idea/version of that song. That is only one way to organize your music.

    Another way to use the application would be on Word. A music teacher can easily teach a student the basics and fundamentals of music by either drawing on the given staves or inputting the notes with the tools given and if the MIDI playback is available, they will also be able to play back what was inputted. This lets students copy what was inputted and practice what they have learned.

  • Go DevMENTAL

    Love Visual Studio? All we are saying…is give Blend a chance!


    VisualStudioLogoMany of us work in Visual Studio every day, but have never taken the time to learn Blend. Adding Blend to your toolset will make you a more productive developer. so check out the MVA Blend Jump Start!

    I have worked with Visual Studio for longer than I care to admit (do you remember when you could view the clipboard ring in a window?). I have worked in Blend a little bit here and there, but never had a chance to sit down and really figure out how it can help me with my coding. Mickey MacDonald tried it last year and in one day not only started learning Blend but built an entire game using just Blend! (read Mickey’s story)

    So as you look at the long list of technologies you want to learn, if you are doing anything with XAML, I highly recommend bumping Blend to the top of your list. There is a really good Jump Start for Designing your XAML UI with Blend on Microsoft Virtual Academy presented by Jerry Nixon a Technical Evangelist and Unni Ravindranathan a Program Manager from Microsoft.

    The jumpstart covers

    • XAML development in Visual Studio 2013 (there are some nice new features in 2013!)
    • XAML Design and Styling in Blend
    • How to build Windows Store Apps with Blend
    • a little Developer Candy :)
    • Some great suggested resources and next steps.

    Don’t forget If you are a student you get Blend when you download Visual Studio 2013 from DreamSpark!

    If you are in Canada, don’t forget that watching videos on Microsoft Virtual Academy could earn you rewards through Developer Movement!

    Put aside some time, and learn how to make the most out of a tool you already have in your toolbox!

  • Go DevMENTAL

    Step by Step Creating a Windows 8 app from Windows Phone App Studio


    Windows Phone App Studio now supports the creation of Windows 8 apps in additional to Windows Phone! This post will show you how to generate a Windows 8 app and get it ready to publish.

    If you haven’t used Windows Phone App Studio before, you should check out the other resources on our  Windows Phone App Studio Resources page which include a post which explains how to build an app using Windows Phone App Studio. Don’t forget if you are in Canada your apps could earn you rewards!

    This post will explain how to

    • Preview the Windows 8 app
    • Change the Windows 8 Splash Screen
    • Update the information for publishing the application
    • Associate your app with the Windows Store
    • Generate the app files for testing and publishing
    • Test your app and capture screenshots for publishing

    Preview the Windows 8 app

    When you have finished building your app. You can preview the appearance of your app as a Windows 8 app, select Windows Preview to change from the Windows Phone preview to the Windows 8 application preview.


    A pop up windows will appear giving you a preview of your application as a Windows 8 app.


    Changing the Windows 8 Splash screen

    Once you are happy with the basic appearance of your app (changed the background color, content, etc, as explained in the previous posts of this series, you need to specify a Windows 8 splash screen background.

    Select Tiles | Splash & Lock. Then you can choose between two splash screen designs.

    • Solid color background with the icon you selected for your tiles.
    • A photograph as a background.
    • image

    For a solid color background, simply select the desired background color from the color drop down box.


    If you wish to use a photograph as a background, Select Edit and then choose an image to use as a background. The image you use as a splash screen for a Windows 8 app should be 620x300 pixels.


    Tip: If the image you want to use isn’t 620x300 pixels, you can resize it using a tool such as MSPaint. You may need to turn off the “Maintain aspect ratio” checkbox if your original image isn’t the same proportions as 620x300 pixels. This may cause the image to be skewed when you resize it, so you may have to play around a bit to find a suitable image that looks good in those proportions.


    Updating the information for publishing the application

    Now you are ready to specify the information that will affect how the app appears in the Windows store.

    Select the Publish Info Tab

    Enter the app name

    Enter the app name you wish to use when you publish your app to the store. Do give this some thought, the app name is the first indication to a user of what your app can do. There is a nice MSDN article called “What to name your app” with some tips on selecting a good name.

    Enter the app description

    When a user brings up details of your app in the store, they will see the app description. This is your chance to convince a user that yes your app is the right app to download! The description should make it clear who should use this app and what the app can do. There is an MSDN article with tips on writing a good description “Your apps description

    Select your language

    Specify the language of your application. Although it is possible to create multilingual applications for both Windows Phone and Windows 8. The Windows Phone App Studio does not currently give you the ability to create multilingual applications, so you should specify the primary language of the application. If you want to make the app multilingual, you will need to export the source code into Visual Studio and add the necessary language support. For more information see localizing your apps why and how

    Specify whether you wish to include an About page for your application

    An about page allows the user to see basic information about your app

    Specify whether you wish to include advertisements in your application

    Advertising is one of the ways you can make money from your application. The more time advertisements spend displayed to users, the more money you earn. However, adding advertisements to your app isn’t quite as simple as flipping the toggle switch to enable ad clients. There are additional steps you must complete to include advertising. The Windows Phone App Studio ‘how to’ describes the additional steps. If this is your first application, I would suggest you get your app working and published, and then you come back and add the advertising later.


    Windows Store Association

    The Windows Store requires you to reserve your App Name before you submit to the store. Once you have reserved the name the store will assign your application a Package Identity. You need to know this information and your name and ID as the publisher before Windows Phone App Studio can generate a package you can publish for your Windows 8 app.

    Once Windows Phone App Studio has that information, it can update a file in the package called the application manifest. That’s why you have the “Associate the App with the Store” in the Publish Info tab within App Studio.

    So how exactly do we do all that?

    Reserve your app name in the Windows Store

    First you will need a Windows Dev Center account. If you are a student, you can get one for free through DreamSpark. If not, you’ll have to pay a registration fee of $20. We won’t look at the exact steps in this tutorial, but you can create an account here. It’s a single store account for Windows Phone and Windows Store

    Once you’ve registered and logged in to your account, go to the Dev Center home page. Select Dashboard | Submit an app.


    Select App Name


    Specify the app name you have selected and select Reserve Name.


    If someone else is already using your app name you will have to pick another name. If you have selected a valid app name then you get a message informing you that you can Save to reserve the selected app name.


    The app name you just reserved will be your App Display name! Make a note of it.

    Get the information from the store needed to associate your Windows Phone App Studio app with this reserved app name.

    Find out your app identity.

    While still in the Windows Store, go to Dashboard, and select Edit for your application


    Now select Services from the menu and in the text select Live services site.


    Under Application Identity, make a note of the Identity Name it will be something like 1234user.YourAppName. This is your App Identity.


    Now go back to the dashboard and select Profile to get your publisher information


    In your profile take a note of your publisher id and your publisher name


    Associate your Windows Phone Studio app with the Windows Store

    Now you have all the information you need to associate your app with the store. Return to Windows Phone App Studio, go to the Publish Info tab and select Associate app with the store.


    When you select Associate App with the store, a popup window is displayed. Use the information you just collected from the Windows Store to populate the fields.

    • App Identity: This was the App identity you got from the Store Dashboard, when you selected the app you reserved, then selected Edit | Select Services, and selected the “Live Services site”.
    • App Display Name: This is the name you defined when you reserve the App name in the store
    • Publisher ID: This is the id you got from your Account profile. It should have the following format “CN=AAAAA-1111-2222-BBBBBBBBB”
    • Publisher Name: This is the name you got from your Account profile.

    Add your privacy policy

    Whenever a Windows Store app collects personal information, you must declare a privacy policy. Apps that connect to the internet retrieve the IP address of the person using the app. This information is considered personal, so therefore any Windows Store application that connects to the internet must have a privacy policy. The privacy policy tells the user how the information collected is used by the application.

    Here is a link to a post entitled “Windows 8 Certification tips: The privacy statement” . This post tells you more about the privacy statement and how to set up the privacy statement for your application.

    Once you have set up a web page (you can set up a free website for your privacy policy using Azure Web Sites). Include the URL for your privacy policy on the Publish Info page in Windows Phone App Studio


    Generating the App Publishing Files

    First, open up the app you want to publish in the Windows Phone App Studio browser. It doesn’t matter what page you’re on. Select the Finish button.


    This will take you to a page where you can GENERATE your app packages. You’ll notice that there is a section labelled “Previous versions” – currently this is probably blank, because this is the first version you have created.

    Select Generate.


    A pop up will appear that gives you the choice of generating the code for a Windows Phone app or a package that contains code for a Windows Phone and Windows 8 application.


    Once you’ve generated your app, you will see a version listed in the “Previous versions” section. Now, in the main body of the page, you will see three major options:

    · Download package – choose this option if you want to download your app to your own phone only. This is useful if you want to use your app yourself without publishing to the store, or if you want to see how it works on your phone before publishing it.

    · Download publish package – choose this option to go straight to publishing without changing your code or taking screenshots. If you do not have Visual Studio, this is the option you should take.

    · Download source code – choose this option if you want to edit your code or take screenshots using an emulator.

    Test your app and capture screenshots for publishing

    Now we are going to install the app on our pc, so we can test it and capture screenshots. You will need a screenshot to publish your app to the store.

    Install the certificate

    First we need to install a certificate so that our PC or laptop will trust the app we are about to install. Expand Pre-requisites and choose Install Windows 8.1 Certificate. Save the downloaded file to somewhere you can find it afterwards. I created a subfolder called AppStudio.


    Navigate to the folder and you will find a file called Add-RootCertificate.zip. Select that file and either right click and choose Extract All, or select the file and choose Extract All from the menu to extract the contents of the zip file.


    Now right click the file Add-RootCertificate.ps1 and select Run as PowerShell script



    You will probably get a prompt warning you that this is a script file that could harm your computer. You must allow the program to run to continue and install the certificate.

    When the program runs you will see a window appear asking you to make sure you are logged in as the administrator on your machine. It will prompt you to hit Enter. When you hit Enter, you will receive one or two more warnings, after you select the option to run the script the certificate will be installed.

    NOTE: The reason you are getting so many warnings is because PowerShell is a tool system administators often use to change settings on your computer. It’s very useful, but because it can change computer settings, that means if a hacker managed to trick you into running a malicious powerscript file on your computer it could get your computer pretty messed up. As is always the case! Only download and run files from sources you trust!

    Install the application

    Now we have our certificate installed, we can install our application

    Under installable packages select the Windows 8.1 installable package


    Navigate to the folder and you will find a file called StoreApp.zip. Select that file and either right click and choose Extract All, or select the file and choose Extract All from the menu to extract the contents of the zip file. Just like you did with the certificate zip file.

    Now go into the extracted contents and find the file called App-AddDevPackage.ps1 . This is another PowerShell script, but this one will install your application. Right click on the file and choose Run with PowerShell just like you did with the script that installed your certificate. A window will open as the app runs and after it runs a message will display saying your app has been installed successfully

    NOTE: This app is a Windows 8.1 app so you must be running Windows 8.1 to install and run your application.

    Run the application and capture screenshots

    Now you can run your application. The easiest way to launch your application is to press the Windows Key button on your keyboard and then type the name of your application.



    Then select your application from the search results and you can see your app up and running!

    When you publish your app you will want a screenshot, you can use the keyboard combination of <Ctrl>+<Shift>+<PrtSc> to get a screenshot to your clipboard. Then open Paint, and choose Paste and now you have your screenshot in Paint where you can save it to use when you publish your application. You many want to create several screenshots showing off different aspects of your application. Here is a screenshot of my application in Windows 8.1


    I now have a working application for Windows 8.1 and screenshots so I have everything I need to publish my app to the store! Now it’s your turn!

  • Go DevMENTAL

    Meet the winners of the 2014 Canadian Microsoft Imagine Cup!


    Imagne Cup

    Intel logo

    So who won this year’s 2014 Canadian Imagine cup? Some amazing students with some amazing ideas!

    Congratulations to all the teams who participated in this year’s Imagine Cup Canada. Thanks to our great sponsors at intel, the members of each winning team will receive one of the latest 2-in-1 touch enabled PCs.

    Games Category

    Today, great games come from anywhere and people play them everywhere. On their phones, on their computers, on their tablets, in their browser, with their friends, with anyone, with everyone. Powerful game engines and libraries are available for free so students can get started right away. There’s really never been a better time to make the next great game.

    Congratulations to Team World Painters from the University of Calgary! Their project Funiverse is an amazing educational game that leverages Occulus Rift technology to immerse the player in a virtual reality. The members of Team World Painters are: Alper Aksac, Gabriela Jurca, Omar Addam, Omar Zaarour and their mentor Dr Reda Alhajj. You can see a video showing their project here.

    University of Calgary Logo World Painters

    World Citizenship Category

    Since Imagine Cup started in 2003, students all over the world have teamed up to make the world a better place. By creating impressive new technology projects in fields such as health, education, and the environment these students have shown the world new ways to think and to change. The World Citizenship category challenges students to find a problem in the world, in your own life or community, that affects many people, and then work to solve it.

    Congratulations to Team Marianopolis Brainstorm from Marianopolis college! Their project Mass Assaying Relaying Inquiring Survey Apparatus (MARISA) uses Kinect technology to help triage and evaluate disaster sites. The members of team Marianopolis Brainstorm are Mansib Rahman and mentor Elly Su. You can see a video showcasing their project here.

    Marianopolis college logo Team Marianopolis Brainstorn Team Marianopolis Brainstorn mentor

    Innovation Category

    Incredible, world-changing software innovations often come from students. Social networks, music services, digital photography apps, gadgets and robotics – the list goes on. In the innovation category we are looking for the next big thing!

    Congratulations to team claVision from University of Lethbridge! Their project claVision finds a new way to solve an old problem. They have developed software to do visual automatic transcription of piano music. The members of team claVision are Mohammad Akbari, Hossein Naseri and mentor Howard Cheng. You can see a short demonstration of their software here.

    University of Lethbridge logo Team ClaVision

    What happens next?

    Bill GatesEach of these teams is now preparing for the World Semifinals. During this round, one of these three amazing teams will be selected to represent Canada at the Imagine Cup 2014 Worldwide Finals this July in Seattle, Washington. At the worldwide finals they will compete for the coveted Imagine Cup trophy, up to $50,000 USD in prizes and a chance to meet Bill Gates!

    Other notable accomplishments by Canada in Imagine Cup

    I also wanted to highlight a couple of other great performances on the world stage by Canadians in the 2014 Imagine Cup!

    Pitch Video Challenge – Honourable Mention

    In September we asked students to dream up their next great Imagine Cup project and pitch it to us in a short video. We received submissions from around the world!

    Congratulations to team SocialDocTeam from Canada who earned an honourable mention for the pitch for their project Documentaries for Change in the Pitch Video challenge for World Citizenship.

    Project Blueprint Challenge – Honourable Mention

    Hundreds of teams of students from dozens of countries around the world competed in this contest. Each team took a long, hard look at their Imagine Cup project and produced a document that really spelled out what they were making: their concept, target audience, early feedback they've received, competitors, user personas, top user stories, business model, and core technologies.

    Congratulations to team Marianopolis Brainstorm who earned an honourable mention in the Project Blueprint challenge for Innovation.

Page 2 of 82 (409 items) 12345»