Windows Azure Blob storage API provided following upload scenarios to upload a blob: Scenario [1]: You can upload a single blob in N parallel threads In your code if you set CloudBlobClient.ParallelOperationThreadCount = N; then N parallel threads...

Map Task (HDFS data localization): The unit of input for a map task is an HDFS data block of the input file. The map task functions most efficiently if the data block it has to process is available locally on the node on which the task is scheduled...

Recently I was working on an issue where the SSL certificate was expired and due to it, the user were warned to not to use site. The certificate expiration was visible as below: In this situation the following steps should be taken to get...

As you may have already know that Hadoop Cluster is network and disk, IO intensive. Recently I was trying to run a test scenario where I decided to change SATA hard disk to a high performance SSD Disk while keeping the cluster hardware the same. I was...

I was recently asked about if someone store their content at Windows Azure Storage, how secure is it and how many copies are there, so I decided to write this article. Technically there are 6 copies of your blob content , distributed in two separate...

I was recently asked to show that Pig scripts are actually MapReduce jobs so to explain it in very simple way I have created the following example: Read a text file using Pig Script Dump the content of the file As you can see below...

Technically they both will do the job, you are looking from "either hive or Pig" perspective, means you don't know what you are doing yet. However if you first define the data source, scope and the result representation and then look for which one to...

My recent development work with Windows Azure Node.js SDK 0.5.2 helped me to found two issues. Until next SDK update is available please use the given workaround to solve your problem. Issues 1: Exception while testing your node.js application in...

I was recently informed about a great “Ruby on Rails” companion package to run your Ruby on Rails application on Windows Azure. With this package you can: Run multiple instance of Ruby Application on Windows Azure Deploy your application...

In this article we will learn how to configure the same Hadoop cluster to do some customization. To learn about creating your own Hadoop cluster on Windows Azure by using your own Windows Azure Subscription account – Click here To add more...

Apache Hadoop distribution (currently in CTP) allows you to setup your own Hadoop cluster in Windows Azure cloud. This is article is written keeping those users in mind, who are very new to “Windows Azure” and “Hadoop on Windows Azure”...

When you have lots of instances running on Windows Azure, you may need to investigate issues on a specific instance however there is not direct way, you can offline specific instance from your total instances. To investigate a specific instance your first...

To get your Amazon S3 account setup with Apache Hadoop cluster on Windows Azure you just need you AWS security credentials which pretty much look like as below: After you completed creating Hadoop cluster in Windows Azure, you can log into your...

One of my teammate William Bellamy has written a great article “ Troubleshooting Best Practices for Developing Windows Azure Applications ” which is available for everyone to consume. Summary: The number one priority of Microsoft...

In this example I will create a node.js based blob application which will storage all the blog articles on Azure Storage. When application starts it reads blog article from Windows Azure table storage and then render it using EJS viewer. This sample is...

When you run Map/Reduce job in Hadoop cluster on Windows Azure you will get an aggregated progress and log directly on portal, so you can see what is happening with your job. This log is different the what see when you do check individual job status in...

Time to time, I get involved with our Windows Azure partners to discussion IP address configuration in Windows Azure Virtual Machine so I decided to write this article to point out most of the configuration: When you have your application running...

After you log into your Hadoop Portal and configured your cluster, you can select “Manage Data” tile as below: On the next screen you can select: “Set up ASV” to set your Windows Azure Blob Storage as data source...

By default the application drive in Windows Azure VM has maximum size limitation of 1GB with any kind of VM type (small, extra-large). So if you have a large or extra-large VM, and running multiple sites configured in your Web Role, you might want to...

In Windows Azure VM diagnostics is enabled by default so basic log can be collected and transferred to Azure Storage on demand. With Windows Azure SDK 1.6, the default application setting do two things: 1. It enable the diagnostics and transfer diagnostics...

Microsoft Distribution of Apache Hadoop comes with Pig Support along with an Interactive JavaScript shell where users can run their Pig queries immediately without adding specific configuration. The Apache distribution running on Windows Azure has built...

If you want to access Windows Azure Blob Storage same as a drive on your development machine you sure can do it. Gladinet provide the application to mount Windows Azure Blob Storage directly as a Network Drive so you can access blob directly from your...

Microsoft Distribution of Apache Hadoop comes with Hive Support along with an Interactive Hive shell where users can run their Hive queries immediately without adding specific configuration. The Apache distribution running on Windows Azure has built in...

In Hadoop cluster, namenode communicate with all the other nodes. Apache Hadoop on Windows Azure have the following XML file which includes all the primary settings for Hadoop: C:\Apps\Dist\conf\HDFS-SITE.XML <?xml version="1.0...

Microsoft distribution to Apache Hadoop comes by direct connectivity to cloud storage i.e. Windows Azure Blob storage or Amazon S3. Here we will learn how to connect your Windows Azure Storage directly from your Hadoop Cluster. To learn how to connect...