Performance Testing using DbPopulator - Microsoft Dynamics CRM Team Blog - Site Home

DbPopulator is a tool used by MSCRM performance team to populate different types of data (qualitatively and quantitatively) in the MSCRM test boxes for performance testing. This tool is available as part of CRM Performace Toolkit (download available in http://www.codeplex.com/crmperftoolkit) and can be used to test SDK Create performance. I will describe how we can do this using DbPopulator.

Getting Started

Download CRM Performance Toolkit from the above mentioned Url and install it into the test machine. After successful installation, you will find DbPopulator tool available in <installroot>\Binaries\DbPopulator directory. You will also find some sample XML files are put under the <installroot>\ CRM4_Perf_Toolkit\dbPopulator\Sample_Xmls directory. These sample XMLs can be used to measure SDK Create performance for different entities. Also, you can measure how performance varies when you increase number of concurrent threads for record creation.

For example, consider unitTest_Account XML. The body of this XML looks like,

<?xml version="1.0" ?>

<DbPopConfig>

   <exec threadCount='1' />

      <userGroups domain="<DOMAINNMAME>" webserver="<WEBSERVERNAME>"

             organization="<ORGANIZATIONNAME>">

   <userGroup name='crmusr' startidx='1' count='1' password='<PASSWORD>'/>

</userGroups>

   <objectDist>

      <account count='10' save_1='settag(accountid,account,entityid)'

          save_2='settype(accountidtype)'>

      </account>

   </objectDist>

</DbPopConfig>

You need to set the domainname, webserver and organizationname parameters for your test environment. Also you need to set the test user name and password properly. These are explained in more details in the blog http://blogs.msdn.com/crm/archive/2008/02/29/microsoft-dynamicstm-crm-4-0-performance-toolkit.aspx. Also, set the account count as 1000 so that it takes sufficient time to calculate throughput properly.

Now run DbPopulator as

dbPopulator.exe ..\..\CRM4_Perf_Toolkit\dbPopulator\Sample_Xmls\unitTest_Account.xml –q

Remember to use –q switch to run DbPopulator in quiet mode, else the console logging will take some extra time and throughput measurement may not be accurate. After execution, DbPopulator will display the result in console as

Creating entities for crmusr1

User crmusr1 took 19641 ms

dbpop Runtime is 30969 ms

The time taken by crmusr1 is 19.641 second for creating 1000 records. DbPop runtime is more because it does some extra processing like creating CrmService etc. So here, we get single threaded account creation throughput as 1000/19.641 = ~51.

Now run the same test after setting threadCount = ‘2’ and count = ‘2’ in userGroup. The result will look something like

Creating entities for crmusr1

Creating entities for crmusr2

User crmusr2 took 21704 ms

User crmusr1 took 21704 ms

dbpop Runtime is 33250 ms

Each user has taken 21.704 second to create 1000 user. But they have run in parallel, so we can assume that in 21.704 second, we can create 2000 accounts using 2 threads in parallel. There may be slight non-overlapping timing among the threads, but for all practical estimation purpose, we can ignore them. This result will calculate the account create throughput as 2000/21.704 = ~92 when 2 threads were creating account records simultaneously.

Sample Results

I ran DbPopulator to measure the SDK create throughput for several entities. The below table summarizes the result.

Name of Entity	Thread Count = 1	Thread Count = 2	Thread Count = 3	Thread Count = 4	Thread Count = 5	Thread Count = 8
Account	51	93	100	100	100	97
Contact	50	90	99	100	99	95
Campaign	60	100	103	102	102	100
Task	46	83	90	91	89	89
Email	41	77	95	96	93	89
Quote	36	70	100	100	100	100
QuoteDetail	10	18	25	30	37	45
Invoice	40	76	101	101	101	101
InvoiceDetail	9	16	23	30	36	45

For most of the entities, the create throughput saturates after 3 to 4 threads, whereas few other entities had throughput increased even up to 8 concurrent threads. These results are obtained in a typical test environment (described in below section). The numbers may vary depending on your system’s hardware and software configuration but it will provide you a nice way to measure the Create performance and predict approximate time for uploading data in your system.

Hardware Details

This experiment was done using a four box On Premise configuration with the following server details.

Domain Controller: HP dc5750 with AMD Athlon 64X2 4800+ 2.5 GHz processor and 4 GB RAM
Windows Server 2008 Enterprise Edition 64 bit
Application Server: HP DL 360 with Intel Xeon X5450 3 GHz Dual processor Quad Core and 8 GB RAM
Windows Server 2008 Enterprise Edition 64 bit
SQL Server: HP DL 360 with Intel Xeon X5450 3 GHz Dual processor Quad Core and 16 GB RAM
Windows Server 2003 Enterprise Edition 64 bit with SQL Server 2005 sp2 Enterprise Edition 64 bit
Client Machine: HP dc5750 with AMD Athlon 64X2 4200+ 2.2 GHz processor and 3 GB RAM
Windows XP Professional Edition sp3 32 bit

All machines were on a gigabit network switch with gigabit NIC’s.

The DbPopulator tool, released as part of CRM Performance Toolkit, provides a nice and easy way to measure SDK Create performance. It can also be used to predict multi threaded upload speed for any MSCRM client application (which uses MSCRM Web Services).

Cheers,

Koushik Bhattacharjee