Wenming's Big Data and Big Compute Blog

All about running HPC and Big Data workload on the Microsoft Windows Azure platform. I have been and will continue to cover a variety of topics on Hadoop, HPC, Migration from / Interops with Unix, Tips and Tricks for running HPC and Big Data.

Browse by Tags

Tagged Content List
  • Blog Post: Data Science in a Box using IPython: Installing IPython notebook (2/4)

    In the previous blog, we demonstrated how to create a Windows Azure Linux VM in detail . We will continue the installation process for the IPython notebook and related packages. Python 2.7 or 3.3 One of the discussions that happened at the Python in Finance conference is which version of Python you should...
  • Blog Post: Enter the Big Data Matrix: analyzing meanings and relations of everything (2/2)

    Running the Python example step by step: We explained the basic idea behind LSA or latent semantic analysis in the first part of this blog . We built a matrix by word counting for each document. The set of document vectors are then sorted by words they appear in. Then we applied SVD (single...
  • Blog Post: Let there be Windows Azure HDInsight

    Windows Azure HDInsight Service, formerly known as Hadoop on Windows Azure, is now available inside the Windows Azure Preview portal. Hadoop-based big data tools are what I call the WMD(P), or Weapons of Mass Data Processing. (You heard it here first!) This is a very exciting development, and I would...
  • Blog Post: Running Weather Research Forecast as a Service on Windows Azure

    About 9 months ago, I deployed a Weather forecast demo at an internal Microsoft event, Techfest. The demo uses real data from NOAA and predicts high resolution weather forecast up to the next 3 days running a HPC modeling code called WRF . Since then, I’ve received a great deal of interest from...
  • Blog Post: Hadoop On WindowsAzure Updated

    HadoopOnAzure allows a user to run Hadoop on Microsoft Windows Azure as a service. It is currently in private CTP with limited capacity, and by invitation only. We did add more capacity today, you may attempt to sign up this free service at https://connect.microsoft.com/SQLServer/Survey/Survey.aspx?SurveyID...
  • Blog Post: Explicit and Implicit

    This is the final post of the simple HPC Math series I am composing. The previous two posts were on linear, and nonlinear solvers. Just like nonlinear solvers depend on linear solvers, time stepper solvers will depend on the previous two building blocks. Explicit and Implicit methods are used in numerical...
  • Blog Post: 5 min Intro to nonlinear solvers

    Nonlinear solvers are essentially solving a system of nonlinear equations. X^2 + x -1 =0 is a nonlinear equation. As we recall from Calculus 101, to solve these equations, one has to use some variants of Newton’s method because it has fast convergence to the solution property. For systems of nonlinear...
  • Blog Post: Math under the hood

    We recently had a company called Simulia visiting us at a deep dive lab. Their product, Abaqus, a commercial software package for finite element analysis prompted me to quickly brush up on my numerical computation knowledge. After all, you can't compute without numerical methods under the hood! While...
  • Blog Post: Problogue

    Super computing is crucial for science in the 21st century.  For a long time people have been theorists or experimentalists, those have worked in synergy to develop scientific theory and explain natural phenomenon.  The incredible development of super computer in recent years has enabled theory...
Page 1 of 1 (9 items)