If you ever notice “99% of XX HDINSIGHT CORES” in your HDInsight dash board on wonder what it means? Or during cluster creation if you get error/warning “The cluster can have at most 0 data nodes” and wonder whatit means?
In this short article I’ll just explain what above means and show core calculation.
So let’s take an example if you have 740 HDInsight Core on your subscription and 740 HDInsight Core being used by 8 clusters (167 nodes). Each worker node consumes 4 cores and each head node consumes 8 cores.
Here is the calculation:-
Data Nodes x Cores
Head Nodes x Cores
Total Cores used
16 x 4 = 64
1 x 8 = 8
16 x 4 =64
1 x 8 = 8
32 x 4= 128
7 x 4 =28
So if you try to create cluster at least with 1 node it requires 12 cores which is not available to subscription. That’s why you get error “The cluster can have at most 0 data nodes” and message “99% of 740 HDInsight cores” in the dashboard.
In case it’s require to increase the core, one of the options is to raise ticket from the portal.
The other option is reuse existing cluster in case you are not using it.