Slava Oks's WebLog : SQL Server Memory Management

Q and A: Ratio between AWE Allocated and VM Committed/Reserved in DBCC MEMORYSTATUS output

slavao — Wed, 03 Jan 2007 04:02:00 GMT

Q: What kind of ratio should you expect to see between AWE Allocated and VM Committed/Reserved? We are running 8GB with a cap of 7GB in the max server memory and yet the VM Committed is around 100MB. I was first shocked to see that the sqlservice in task...(read more)

Q & A: Does SQL Server always respond to memory pressure?

slavao — Mon, 13 Nov 2006 23:39:00 GMT

Q: I thought I've seen reference recently to problems with SQL2005 not releasing memory when under pressure, and this was possibly due to the Lock Pages In Memory option. We have a server where this seems to be happening, i.e. SQL2005 uses all available...(read more)

Talking points around SQL Server 2005 scalability

slavao — Thu, 03 Aug 2006 18:09:00 GMT

Couple of weeks ago I did a talk on SQL Server 2005 scalability. The actual talk was not about the enormous amount of features that you can leverage in SQL Serve 2005 to make your application scale but rather how SQL 2005 achieves its scalability. In...(read more)

Q & A: I can't make SQL Server 2005 32 bit version to use all memory on the machine

slavao — Thu, 03 Aug 2006 17:41:00 GMT

I’m currently using the RTM+SP1 x86-SQL2005 build on a server which has 32GB of physical memory. I notice that during ramp-up the memory usage (as seen under task manager) increases to about 28GB (i.e. “Available memory” reduces to 4GB) and stays there....(read more)

Set of new features in SQL Server 2005 enabled by SQLOS

slavao — Wed, 12 Apr 2006 21:57:00 GMT

Couple of weeks ago I had a chance to visit set of our customers and talk to them directly. It was an unforgettable experience. As a part of the visit I made set of presentations related to SQLOS and new features it enables in SQL Server 2005. I thought...(read more)

Q and A: Enabling AWE settings for SQL Server 32 bit edition in WOW

slavao — Wed, 12 Apr 2006 21:25:00 GMT

Q. A customer wants to run SQL 2000 SP4 in a Windows Server 2003 SP1 x64 edition. They did the installation without problems but SQL Server is not using more than 4GB of memory. Is it possible to enable AWE in this configuration? A. The answer is yes,...(read more)

Q and A: SQL Server memory allocated outside of Buffer Pool, formerly known as MemToLeave

slavao — Tue, 14 Mar 2006 21:14:00 GMT

Q. Under SQL Server 2000 Is the memtoleave concept still valid under 64-bit? A. Yes. If SQL Server’s component requests memory larger than 8 KB block, the block will be allocated outside of Buffer Pool. Remember, inside of SQL Server, Buffer Pool is preferable...(read more)

Q and A: Do I need to run "sp_configure 'awe enabled', 1" to make 64 bit version of SQL Server use locked pages?

slavao — Mon, 13 Mar 2006 20:44:00 GMT

The answer is No. The source of possible confusion came from one of my previous posts: http://blogs.msdn.com/slavao/archive/2005/11/15/493019.aspx . I apologize for the confusion. In order to make 64 bit version of SQL Server leverage locked pages all...(read more)

Be Aware: 4GB of VAS under WOW, does it really worth it?

slavao — Mon, 13 Mar 2006 03:12:00 GMT

By now you have heard a lot about the fact that under WOW a 32 bit process can get 4GB of VAS. I agree this is great, but I would like you to be aware of something that might hit you really hard. As it turns out enabling 4GB VAS can be problematic due...(read more)

Q and A: Clock Hands - what are they for?

slavao — Sun, 20 Nov 2005 21:56:00 GMT

Q1 I read in your blog about the new algorithm with two clock hands to control memory consumption, but I really would like to understand exactly what the hands are doing and if they relate to the lazywriter.

A1. In SQL Server 2005 we have separated database cache, Buffer Pool, and the rest of the caches. As in SQL Server 2000, in the SQL Server 2005 lazywriter continues to be responsible for controlling memory consumption by database page's. However it does no longer control memory consumption by procedure cache. In SQL Server 2005 we introduced new common caching framework. All caches except for Buffer Pool leverage it. The framework consist of set of stores and ResourceMonitor. There are three types of stores: cache store, user store (the name here is related to internal usage of the store - it does not have to do anything with a user) and object store. Both cache and user stores share common LRU mechanism and leverage common costing. Procedure cache is an example of a cache store and metadata cache is an example of a user store. Object stores are just pools of memory blocks and don't require either LRU nor costing. Network library leverage object store for pooling network buffers.

Store's LRU mechanism is a rather simple - we use variation of thoroughly discussed in operating system's literature clock algorithm. As you remember clock algorithm has a hand. When hand moves it decreases a cost of an entry. Once cost of an entry reaches 0, the entry can be discarded from the cache. Cost gets reset whenever an entry is reused. When dealing with multiple caches a designer has to take into account global and local eviction policies. Global policies consider global picture and enable running of LRU, clock algorithm, across all the caches. In its turn, local policies look at the local resource consumption only and run LRU for a given cache only.

In order to satisfy global and local policies SQL Server's stores implement two hands: external and internal (please don't confuse a clock hand names with types of memory pressure. There is no connection!). External clock hand implements global policy and internal clock hand implements local policy. Even though every cache has its own external hand, external hands move "synchronously" - one at a time. Their task is to simulate global hand. ResourceMonitor is in charge of moving external hands whenever it notices a memory pressure (Don't forget there are different types of memory pressures, you can find more information here http://blogs.msdn.com/slavao/archive/2005/02/01/364523.aspx). Internal clock is moved whenever a single cache needs to be trimmed. SQL Server attempts to keep each cache "reasonably" sized with respect to other caches. A task, thread, checks local policies and runs internal clock hand if an it needs to be run whenever the task accesses the store.

Q2 There is no documentation on the dmv sys.dm_os_memory_cache_clock_hands, so I would like to understand what the values mean in that view.

A2. Once you know what the clock hands are and what they are for suddenly this dmv makes sense :-). So here is the brief explanation:

cache_address - an address of the store in memory. You can use it to join with the rest of the cache/clerks related dmvs

name,type - name and a type of a store the same as in other dmvs.

clock_hand - type of a clock hand: external or internal. Remember, there will be two clock hands for every store.

clock_status - status of a clock hand: suspended or running. A clock hand runs whenever a corresponding policy kicks in.

rounds_count - a number of rounds a clock hand made. External clock hands should have this value almost the same.

removed_all_rounds_count - a number of entries removed by a clock hand in all rounds

removed_last_round_count - a number of entries removed by a clock hand in a last round

last_tick_time - time when a clock hand moved last

round_start_time - time when a clock hand started current round

last_round_start_time - overall time of the last, previous, round

In this dmv the most interesting columns are status and count of rounds. For example, if external clock hands constantly running, SQL Server is permanently under memory pressure. If you remove this condition chances are the server will perform better. Depending on a store you can improve overall server behavior as well. For example if you have plenty of adhoc queries, procedure cache's local policy will force run of its internal clock hand. It means that every access to the store will be "penalized". You can remove this condition by rewriting your application to either use parameterized queries or stored procedures.

Let me know if you have more questions!

Q and A: Virtual Address Space, VAS, reservation - why bother?

slavao — Tue, 15 Nov 2005 23:09:00 GMT

Seems like today is a question day :-). I will attempt to answer several questions about VAS I got today:

Q1. There is one aspect of Virtual Memory that is just not clear yet. I think I understand about VAS, but maybe not as much as I could, but I don’t understand about ‘reserving memory’.

When we talk about SQL Server having a certain amount of reserved, but uncommitted, memory, are we talking about VAS?

A1: Yes, We are talking about Virtual Address Space, VAS. SQL Server also has a different, unrelated, notion of reservation. Whenever query execution engine needs buffers to run a complex query, it goes to Buffer Pool, BP, and reserves buffers from it. If there is not enough buffers in BP to serve the reservation, the query doesn't get run until either buffers become available or time out expires. The bottom line is that VAS reservation and BP reservation are two different things.

Q2. What is the value of reserving before committing? Why not just commit at the beginning, or wait until we need to commit in order to reserve?

A2. I understand these questions are related to VAS not to BP's reservations :-). There are several reasons why one would reserve VAS region first and commit it latter.

The first reason is that there are different ways of committing reserved VAS regions. In order to understand this, we need to understand the difference between reserving and committing. Reserving is a process of allocating VAS regions, committing is a process of binding of VAS's regions to a physical memory. However committing is not the only way to perform the binding. Once VAS region is reserved it can be bound to physical memory by means such as committing, AWE mechanism, file mapping API's and etc. In some cases, i.e. simple commit, OS allows to reserve VAS region and commit it in one shot using VirtualAlloc API. In other cases, i.e. AWE mapping, VAS regions have to be reserved upfront and only then they can be bound to physical memory allocated through AWE mechanism.

VAS's limitation is another reason for reserving VAS upfront and then committing its regions latter on demand. On x86 VAS is limited resource - its size can't exceed 4GB and in most of the case it is 2GB. Everyone requires VAS's regions in order use memory. The high demand makes VAS valuable resource. Consequently many components, including SQL Server's Buffer Pool, reserve VAS regions upfront and bind them to physical memory latter on demand. In most of the cases there is no reason to bind reserved VAS regions to physical memory right a way especially if only worry about VAS availability for latter use. Moreover in some cases the premature binding can introduce unnecessary physical memory pressure

Q3. Can the reserved memory and/or VAS grow or shrink the way committed memory can?

A3. Though reserved VAS regions can't grow, they can be freed when a component no longer needs them. For example SQL Server 2005 when configured to use AWE mechanism reserves VAS regions in 4MB regions. If SQL Server notices that it runs out free VAS regions - basically it hits VAS pressure, SQL Server will attempt to free unused 4MB regions to release the pressure.

Q4. What happens if SQL Server can’t get the physical memory that it has reserved?

A4. In cases when system low on physical memory - SQL Server might fail binding giving VAS region to the physical memory. In most of the cases it will result in some kind of OOM error. It the error happens in the engine ERROLOG will have information that SQL Server failed to commit a region of the given size. In addition there is OOM ring buffer that will have a record about the failure. Keep in mind that SQL Server has internal mechanism to detect the condition up front so in normal scenario it will attempt to release its physical memory usage before it hits the failure. In most of the cases such failure means that there is either interaction problem between SQL Server, OS, and/or some other application; someone's leaking physical memory or given configuration can't sustain applied load.

Hope this helps. Let me know if you have more questions.

Q and A: x64 & AWE mechanism

slavao — Tue, 15 Nov 2005 21:15:00 GMT

Today I received following question:

Hello,

We will be running Windows Server 2003 x64 with SQL Server 2005 x64. I would think AWE would not come into the picture in this scenario.

I'm trying to find out more about SQL Server 2005 memory management on x64 systems using x64 versions of the OS and SQL Server.

What role if any does AWE play in this scenario?

Thanks,

In the post http://blogs.msdn.com/slavao/archive/2005/08/31/458545.aspx I did indicate that on 64 bit platform we recommend to enable SQL Server to use locked pages in memory. You can do it by giving SQL Server's account a right to lock pages in memory. By giving this right to SQL Server you enable SQL Server to allocate and hold on to physical memory so that OS can't reclaim the memory back through paging - swapping SQL Server to disk. Even though we have plenty of safeguards inside of the server, still the swapping in some cases can bring SQL Server's performance to its knees.

When running with locked pages enabled, SQL Server monitors box wide memory state and does release memory back to OS in the case of pressure. The key here is that internally SQL Server makes its own decision about what memory can be freed. Internal decision making significantly helps SQL Server to avoid performance degradation in cases of box wide memory pressure. Keep in mind that SQL Server won't release its memory below 'min server memory' specified through sp_configure.

So where does AWE mechanism comes in here? On 64 bit platforms when running with lock pages privilege enabled, we use OS's AWE mechanism http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dngenlib/html/awewindata.asp to allocate physical memory. However we map allocated memory only once since we have plenty of VAS, virtual address space. On 64 bit platform to make use of locked pages you don't have to turn on AWE through sp_configure.

Hope this answers the question!

Final changes in SQL Server 2005's memory manager

slavao — Sat, 05 Nov 2005 23:07:00 GMT

As you would expect in the last several months SQL 2005 went through rigorous testing. During the period we have identified and put several improvements that you might be interested to hear about.

Memory Broker

Analyzing SQL Server's behavior with respect to memory consumption from multiple sources we realized that we have an opportunity to improve dynamic memory distribution even further. We realized if we put a centralized mechanism that would dynamically distribute memory between buffer pool, query execution, query optimizer and caches, memory distribution would be more efficient and adaptable for different types of loads. So we did it.

Final release of SQL Server has a new memory component - Memory Broker. Memory Broker's responsibility is to distribute memory across different components based on their demands. One can think of the memory broker mechanism as a control mechanism with a feedback loop. The mechanism is simple: Memory broker monitors memory demand and consumption by each component. Based on the collected information it calculates "optimal" memory distribution for each components and broadcasts the information to the components. Each component uses the information to adapt its memory usage accordingly.

You can monitor memory broker behavior by querying memory broker ring buffer:

select * from sys.dm_os_ring_buffer

where

ring_buffer_type = 'RING_BUFFER_MEMORY_BROKER'

Ring buffer is only updated when memory broker wants behavior of a given component to change: shrink, grow or stay stable.

One can use dbcc memorystatus to find out last memory broker notification.

NUMA

In my post http://blogs.msdn.com/slavao/archive/2005/08/02/446648.aspx I outlined Windows and SQL Server NUMA support and issues associated with NUMA. In the final release we put several improvements that you might want to be aware of.

In SQL 2005 final release Buffer Pool will use remote pages if it can't get local. Before it wasn't the case - BP would stop allocating memory from the given node if it can no longer get local pages. Because of that we had to jump through hoops to make sure that BP gets local pages. It is no longer the case. If your application can tolerate latency of using foreign pages, and 99% of apps fall into this category, you don't have to run "warm up" load described in my previous post.

In addition we changed the meaning of min/max server memory setting on NUMA. On NUMA these settings get divided among nodes. For example if you set max server memory to be 16GB on the system with four nodes. Buffer Pool will allocate 4GB of memory on every node. If you take one of the nodes off line by changing SQL Server affinity, max server memory setting will be distributed among remaining nodes. For example in the previous case if you take two nodes offline the 8GB of memory will be evenly distributed amongst remaining nodes. Since Buffer Pool node is capable of using foreign pages, remote memory will be utilized if there not enough memory on remaining nodes. If you want the server not to use memory from the nodes it is no longer running on you will need to decrease max server memory setting as well after moving nodes offline.

We have exposed a set of new performance counters that you might find useful. SQL Server::Buffer Node. The set of counters shows a memory used by the Buffer Pool in every node.

Keep your questions coming!

Talking Points around Memory Manager in SQL Server 2005

slavao — Tue, 30 Aug 2005 19:59:00 GMT

Yesterday I gave a talk around SQL Server 2005's memory manager. The talk was rather chalk talk. It included memory manager changes in SQL 2005 as well as changes post CTP15. For the talk I prepared several slides - basically talking points that I thought might be interesting to everyone. If you were following my blog you will notice that slides below closely outline my previous posts. Please ask questions if you have any :-)!

Talking Points for Memory Manager In SQL Server 2005

Agenda

• Quick Intro

• Noticeable Changes from SQL 2000

• Memory Nodes

• Memory Clerks

• Memory Caches & Pools

• Memory Broker

• NUMA support

• Q&A

Quick Intro

• VAS – Virtual Address Space

• Physical Memory & Swap File

• AWE mechanism & Locked Pages in memory on 64 bit

• Memory Pressure

– Internal (Virtual & Physical)

– External (Virtual & Physical)

• SMP – Symmetric Multi Processing

• NUMA – Non Uniform Memory Access

Noticeable Changes from SQL 2000

• Responds to VAS pressure

– Thread pool truncation

– BP’s region truncation when AWE mechanism is enabled

• Supports dynamic memory management when using AWE mechanism

• Supports locked pages on 64 bit platform

• Implements Resource Monitor & Common caching framework

• Contains extended NUMA & Soft NUMA support

• Implements Memory Broker

• Lots of new memory DMVs

Memory Nodes

• A Memory Node is a software abstraction representing the memory attached to either to all CPUs in SMP configuration or to a single NUMA node

• Supports several different memory allocators

– Single Page Allocator - Buffer Pool Memory

– Multiple Page Allocator - Memory outside of Buffer Pool

– Reserved Page Allocator – DAC memory

• Currently is not exposed through either DMVS or

dbcc memorystatus

Memory Clerks

• Reflects memory usage for specific component

• Receives notifications about memory state changes and responds to pressure

– CLR’s GC is hooked into this mechanism

• Utilize Memory Node allocators for memory allocation

• Sys.dm_os_memory_clerks & dbcc memorystatus

Memory Object

• Memory Objects are heaps

• Used by components to allocate/free memory dynamically

• Plenty of memory objects in the system

• Sys.dm_os_memory_objects dumps all memory objects

• Sys.dm_os_memory_allocations dumps all allocations out of memory objects. (Supported with trace flag 3654 only)

Resource Monitor

• Mechanism to respond to memory pressure

– Implemented as a regular task

– Occupies its own hidden scheduler

– Invokes GC in CLR

• Select * from Sys.dm_os_ring_buffers where ring_buffer_type = “RING_BUFFER_RESOURCE_MONITOR”

Memory Caches

• Buffer Pool

– Data page cache (Not covered here)

• Cache Store

– Generic cache framework

– Examples: Procedure Cache & System Rowset Cache

• User Store

– Generic cache framework

– Examples: Schema Manager, Security & Metadata Caches

• Sys.dm_os_memory_cache_counters

Note: Caches are Memory Clerks

Cache Store

• Generic cache mechanism with the storage

– Sys.dm_os_memory_cache_hash_tables

– Sys.dm_os_memory_cache_entries

• Implements size control by utilizing LRU (clock) algorithm

– External Clock hand controls memory consumed by all caches

– Internal Clock hand controls memory consumed by a given cache

– Sys.dm_os_memory_cache_clock_hands

• User Store is exactly the same as Cache Store but doesn’t have storage

Memory Pools

• Object Store

– Generic pool framework

– Responds to memory pressure

– Example: SNI’ Network Packet Cache

• Object Store

– Sys.dm_os_memory_pools

Note: Pools are Memory Clerks

Memory Broker

• Mechanism to dynamically broker memory across large memory components such as Query Execution, Query Optimization & Caches

• dbcc memory status & Sys.dm_os_memory_ring_buffers where ring_buffer_type = “RING_BUFFER_MEMORY_BROKER”

NUMA support

• Automatic configuration according to hardware layout

• Application can be bound to a specific node

• Memory Nodes

– Collection of Scheduler (CPU) Nodes

– Memory locality

• Scheduler Nodes

– Collection of Schedulers

– Scheduling locality

– I/O port for network requests – preemptive I/O

– Resource & Scheduler Monitors

NUMA support cont

• Soft NUMA

– Soft Node configuration for large SMPs

– Soft Node configuration for NUMA

• Soft Nodes can’t span multiple hardware nodes

• Buffer Pool is NUMA aware

– Lazy Writer per NUMA node

– Partitioned free lists

– Perfmon, SQLServer.BufferNode

• Database pages

• Foreign pages

• Stolen pages

• Target pages

• TotalPages

Configuring SQL Server 2005 for Soft NUMA

slavao — Fri, 19 Aug 2005 02:30:00 GMT

Sometimes I am surprised how issues come up the moment you mention them J. Recently, I have discussed Soft NUMA configurations in the article http://blogs.msdn.com/slavao/articles/441058.aspx. This week one of our customers had an interesting problem. The customer wanted to partition single SQL server instance based on the load. Customer’s application is heterogynous. It consists of TPCH type queries and data loading applications. The customer has a system, which is NUMA, with 2 nodes and 4 CPU per Node. The customer wanted to give the loading application two CPUs and the rest of CPUs to the queries. Is it possible to achieve it?

As you might guess the answer is SQL2005's Soft NUMA support. We advised them to configure SQL Server and clients to treat system as three node NUMA system. (Surprised? Yes it is possible with SQL 2005 Soft NUMA support). The configuration looks like following: zero node has 4 CPUs, first node has 2 CPUs and last node has 2 CPUs. Keep in mind, when you configure SQL Server for Soft NUMA, soft nodes should fully be contained in the real nodes, i.e. a soft node can not span several real NUMA nodes. Customer’s TPCH queries were configured to utilize zero and first nodes and the load application was configured to utilize last node. Once configured and started this configuration worked as expected – load was fully partitioned across CPUs. It is important to notice that the customer was very delighted with the experience. Below is the example of node & network configuration we provided customer with:

A. Node configuration – provides node and their affinity information.

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\90\NodeConfiguration\Node0]

"CPUMask"=dword:0000000F

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\90\NodeConfiguration\Node1]

"CPUMask"=dword:00000030

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\90\NodeConfiguration\Node2]

"CPUMask"=dword:000000C0

- Keep in mind affinity mask for a soft node can’t span multiple hardware nodes. For example 000000CF is incorrect mask.

B. Network Configuration – provides port & node assignment.

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL.1\MSSQLServer\SuperSocketNetLib\Tcp\IPAll]

"TcpPort"="1433[0x3],2000[0x4]"

"TcpDynamicPorts"=""

"DisplayName"="Any IP Address"

- This configuration makes SQL Server to listen on two ports. One port serves two nodes. It means that port node relations ship could be one two many.

- Keep in mind that in the square brackets you specify node affinity not the CPU affinity, i.e. 3 in this case means that port 1433 will processes requests on node 0 and 1

- When port is assigned to multiple nodes it will assign connections to the nodes in a round robin fashion

Once registry is configured server have to be restarted for changes to take affect. In addition clients should be configured to connect to specific ports. In the case of our customer TPCH clients were directed to port 1433 and the loading application was assigned to port 2000.

SQL Server 2005 supports more complex configuration for Soft NUMA. Below is information along with an example provided to me by our performance team; describing how to configure SQL Server 2005 for Soft NUMA in general. Please notice that you can actually bind a specific NIC to the node.

This is default key where you need to change the value of TcpPort.

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL.1\MSSQLServer\SuperSocketNetLib\Tcp\IPAll]

"TcpPort"=""

"TcpDynamicPorts"=""

"DisplayName"="Any IP Address"

Thus if you need 3 ports (2000,2001,1433) on a four node machine such that 2000 controls connections such that they go nodes 0 and 1 ,2001 causes them to go to nodes 2 and 3 and 1433 to go to all nodes You use:

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL.1\MSSQLServer\SuperSocketNetLib\Tcp\IPAll]

"TcpPort"="2000[0x3],2001[0xc],1433"

Thus the values in the square brackets are node masks and NOT processor masks. The above key makes all NICs on the system listen to all the above ports. Now instead, if we want to control things on a NIC by NIC basis, we need to flip this switch

From

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL.1\MSSQLServer\SuperSocketNetLib\Tcp]

"ListenOnAllIPs"=dword:00000001

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL.1\MSSQLServer\SuperSocketNetLib\Tcp]

"ListenOnAllIPs"=dword:00000000

And then if we had Two NICs on the system besides the loopback adaptor. Note the Last part of the key “IP3” depend on the NIC number

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL.1\MSSQLServer\SuperSocketNetLib\Tcp\IP2]

"Enabled"=dword:00000001

"Active"=dword:00000001

"TcpPort"="2000[0x3],1433"

"TcpDynamicPorts"=""

"DisplayName"="Specific IP Address"

"IpAddress"="10.192.168.01"

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL.1\MSSQLServer\SuperSocketNetLib\Tcp\IP3]

"Enabled"=dword:00000001

"Active"=dword:00000001

"TcpPort"="2001[0xc],1433"

"TcpDynamicPorts"=""

"DisplayName"="Specific IP Address"

"IpAddress"="10.192.168.02"

This means we shall listen to 2000 only 10.192.168.01 and pass connections received from 2000 only to nodes 0 and 1. And similarly for 2001.

Below configuration which makes an 8 proc 2 node system behave as an 8 node system. It assumes that there are 8 NICs in the system

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL.1\MSSQLServer\SuperSocketNetLib\Tcp]

"Enabled"=dword:00000001

"ListenOnAllIPs"=dword:00000000

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL.1\MSSQLServer\SuperSocketNetLib\Tcp\IPAll]

"TcpPort"=""

"TcpDynamicPorts"="2001[0x1],2002[0x2],2003[0x4],2004[0x8],2005[0x10],2006[0x20],2007[0x40],2008[0x80]"

"DisplayName"="Any IP Address"

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL.1\MSSQLServer\SuperSocketNetLib\Tcp\IP3]

"Enabled"=dword:00000001

"Active"=dword:00000001

"TcpPort"="2001[0x1]"

"TcpDynamicPorts"=""

"DisplayName"="Specific IP Address"

"IpAddress"="10.192.168.01"

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL.1\MSSQLServer\SuperSocketNetLib\Tcp\IP4]

"Enabled"=dword:00000001

"Active"=dword:00000001

"TcpPort"="2002[0x2]"

"TcpDynamicPorts"=""

"DisplayName"="Specific IP Address"

"IpAddress"="10.192.168.02"

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL.1\MSSQLServer\SuperSocketNetLib\Tcp\IP5]

"Enabled"=dword:00000001

"Active"=dword:00000001

"TcpPort"="2003[0x4]"

"TcpDynamicPorts"=""

"DisplayName"="Specific IP Address"

"IpAddress"="10.192.168.03"

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL.1\MSSQLServer\SuperSocketNetLib\Tcp\IP6]

"Enabled"=dword:00000001

"Active"=dword:00000001

"TcpPort"="2004[0x8]"

"TcpDynamicPorts"=""

"DisplayName"="Specific IP Address"

"IpAddress"="10.192.168.04"

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL.1\MSSQLServer\SuperSocketNetLib\Tcp\IP7]

"Enabled"=dword:00000001

"Active"=dword:00000001

"TcpPort"="2005[0x10]"

"TcpDynamicPorts"=""

"DisplayName"="Specific IP Address"

"IpAddress"="10.192.168.05"

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL.1\MSSQLServer\SuperSocketNetLib\Tcp\IP8]

"Enabled"=dword:00000001

"Active"=dword:00000001

"TcpPort"="2006[0x20]"

"TcpDynamicPorts"=""

"DisplayName"="Specific IP Address"

"IpAddress"="10.192.168.06"

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL.1\MSSQLServer\SuperSocketNetLib\Tcp\IP9]

"Enabled"=dword:00000001

"Active"=dword:00000001

"TcpPort"="2007[0x40]"

"TcpDynamicPorts"=""

"DisplayName"="Specific IP Address"

"IpAddress"="10.192.168.07"

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL.1\MSSQLServer\SuperSocketNetLib\Tcp\IP10]

"Enabled"=dword:00000001

"Active"=dword:00000001

"TcpPort"="2008[0x80]"

"TcpDynamicPorts"=""

"DisplayName"="Specific IP Address"

"IpAddress"="10.192.168.08"

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\90\NodeConfiguration]

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\90\NodeConfiguration\Node0]

"CPUMask"=dword:00000001

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\90\NodeConfiguration\Node1]

"CPUMask"=dword:00000002

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\90\NodeConfiguration\Node2]

"CPUMask"=dword:00000004

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\90\NodeConfiguration\Node3]

"CPUMask"=dword:00000008

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\90\NodeConfiguration\Node4]

"CPUMask"=dword:00000010

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\90\NodeConfiguration\Node5]

"CPUMask"=dword:00000020

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\90\NodeConfiguration\Node6]

"CPUMask"=dword:00000040

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\90\NodeConfiguration\Node7]

"CPUMask"=dword:00000080