Slava Oks's WebLog : SQLOS

SQLOS's DMVs Continue

slavao — Fri, 29 Sep 2006 05:22:00 GMT

sys.dm_os_schedulers Q. Do I need to by more CPUs? In order to answer this question you have to find out if your load is really CPU bounded. Your load is really CPU bounded if a number of runnable tasks per each scheduler always greater than 1 and all...(read more)

SQLOS's DMVs

slavao — Wed, 23 Aug 2006 02:54:00 GMT

SQL Server 2005 has a large number of DMVs that you can take advantage of especially when you try to understand behavioral characteristics of your SQL Server 2005 installation. DMVs expose enormous amount of information but how can you make use of it?...(read more)

Set of new features in SQL Server 2005 enabled by SQLOS

slavao — Wed, 12 Apr 2006 21:57:00 GMT

Couple of weeks ago I had a chance to visit set of our customers and talk to them directly. It was an unforgettable experience. As a part of the visit I made set of presentations related to SQLOS and new features it enables in SQL Server 2005. I thought...(read more)

Retrieving information about CPUs' relationships

slavao — Mon, 13 Mar 2006 00:52:00 GMT

In order for server side applications to run efficiently on modern hardware they have to understand hardware layout. As we are moving towards multicore and multi node NUMA systems, understanding relationships amongst cores, logical and physical CPU has...(read more)

Q and A: Clock Hands - what are they for?

slavao — Sun, 20 Nov 2005 21:56:00 GMT

Q1 I read in your blog about the new algorithm with two clock hands to control memory consumption, but I really would like to understand exactly what the hands are doing and if they relate to the lazywriter.

A1. In SQL Server 2005 we have separated database cache, Buffer Pool, and the rest of the caches. As in SQL Server 2000, in the SQL Server 2005 lazywriter continues to be responsible for controlling memory consumption by database page's. However it does no longer control memory consumption by procedure cache. In SQL Server 2005 we introduced new common caching framework. All caches except for Buffer Pool leverage it. The framework consist of set of stores and ResourceMonitor. There are three types of stores: cache store, user store (the name here is related to internal usage of the store - it does not have to do anything with a user) and object store. Both cache and user stores share common LRU mechanism and leverage common costing. Procedure cache is an example of a cache store and metadata cache is an example of a user store. Object stores are just pools of memory blocks and don't require either LRU nor costing. Network library leverage object store for pooling network buffers.

Store's LRU mechanism is a rather simple - we use variation of thoroughly discussed in operating system's literature clock algorithm. As you remember clock algorithm has a hand. When hand moves it decreases a cost of an entry. Once cost of an entry reaches 0, the entry can be discarded from the cache. Cost gets reset whenever an entry is reused. When dealing with multiple caches a designer has to take into account global and local eviction policies. Global policies consider global picture and enable running of LRU, clock algorithm, across all the caches. In its turn, local policies look at the local resource consumption only and run LRU for a given cache only.

In order to satisfy global and local policies SQL Server's stores implement two hands: external and internal (please don't confuse a clock hand names with types of memory pressure. There is no connection!). External clock hand implements global policy and internal clock hand implements local policy. Even though every cache has its own external hand, external hands move "synchronously" - one at a time. Their task is to simulate global hand. ResourceMonitor is in charge of moving external hands whenever it notices a memory pressure (Don't forget there are different types of memory pressures, you can find more information here http://blogs.msdn.com/slavao/archive/2005/02/01/364523.aspx). Internal clock is moved whenever a single cache needs to be trimmed. SQL Server attempts to keep each cache "reasonably" sized with respect to other caches. A task, thread, checks local policies and runs internal clock hand if an it needs to be run whenever the task accesses the store.

Q2 There is no documentation on the dmv sys.dm_os_memory_cache_clock_hands, so I would like to understand what the values mean in that view.

A2. Once you know what the clock hands are and what they are for suddenly this dmv makes sense :-). So here is the brief explanation:

cache_address - an address of the store in memory. You can use it to join with the rest of the cache/clerks related dmvs

name,type - name and a type of a store the same as in other dmvs.

clock_hand - type of a clock hand: external or internal. Remember, there will be two clock hands for every store.

clock_status - status of a clock hand: suspended or running. A clock hand runs whenever a corresponding policy kicks in.

rounds_count - a number of rounds a clock hand made. External clock hands should have this value almost the same.

removed_all_rounds_count - a number of entries removed by a clock hand in all rounds

removed_last_round_count - a number of entries removed by a clock hand in a last round

last_tick_time - time when a clock hand moved last

round_start_time - time when a clock hand started current round

last_round_start_time - overall time of the last, previous, round

In this dmv the most interesting columns are status and count of rounds. For example, if external clock hands constantly running, SQL Server is permanently under memory pressure. If you remove this condition chances are the server will perform better. Depending on a store you can improve overall server behavior as well. For example if you have plenty of adhoc queries, procedure cache's local policy will force run of its internal clock hand. It means that every access to the store will be "penalized". You can remove this condition by rewriting your application to either use parameterized queries or stored procedures.

Let me know if you have more questions!

Talking Points around Memory Manager in SQL Server 2005

slavao — Tue, 30 Aug 2005 19:59:00 GMT

Yesterday I gave a talk around SQL Server 2005's memory manager. The talk was rather chalk talk. It included memory manager changes in SQL 2005 as well as changes post CTP15. For the talk I prepared several slides - basically talking points that I thought might be interesting to everyone. If you were following my blog you will notice that slides below closely outline my previous posts. Please ask questions if you have any :-)!

Talking Points for Memory Manager In SQL Server 2005

Agenda

• Quick Intro

• Noticeable Changes from SQL 2000

• Memory Nodes

• Memory Clerks

• Memory Caches & Pools

• Memory Broker

• NUMA support

• Q&A

Quick Intro

• VAS – Virtual Address Space

• Physical Memory & Swap File

• AWE mechanism & Locked Pages in memory on 64 bit

• Memory Pressure

– Internal (Virtual & Physical)

– External (Virtual & Physical)

• SMP – Symmetric Multi Processing

• NUMA – Non Uniform Memory Access

Noticeable Changes from SQL 2000

• Responds to VAS pressure

– Thread pool truncation

– BP’s region truncation when AWE mechanism is enabled

• Supports dynamic memory management when using AWE mechanism

• Supports locked pages on 64 bit platform

• Implements Resource Monitor & Common caching framework

• Contains extended NUMA & Soft NUMA support

• Implements Memory Broker

• Lots of new memory DMVs

Memory Nodes

• A Memory Node is a software abstraction representing the memory attached to either to all CPUs in SMP configuration or to a single NUMA node

• Supports several different memory allocators

– Single Page Allocator - Buffer Pool Memory

– Multiple Page Allocator - Memory outside of Buffer Pool

– Reserved Page Allocator – DAC memory

• Currently is not exposed through either DMVS or

dbcc memorystatus

Memory Clerks

• Reflects memory usage for specific component

• Receives notifications about memory state changes and responds to pressure

– CLR’s GC is hooked into this mechanism

• Utilize Memory Node allocators for memory allocation

• Sys.dm_os_memory_clerks & dbcc memorystatus

Memory Object

• Memory Objects are heaps

• Used by components to allocate/free memory dynamically

• Plenty of memory objects in the system

• Sys.dm_os_memory_objects dumps all memory objects

• Sys.dm_os_memory_allocations dumps all allocations out of memory objects. (Supported with trace flag 3654 only)

Resource Monitor

• Mechanism to respond to memory pressure

– Implemented as a regular task

– Occupies its own hidden scheduler

– Invokes GC in CLR

• Select * from Sys.dm_os_ring_buffers where ring_buffer_type = “RING_BUFFER_RESOURCE_MONITOR”

Memory Caches

• Buffer Pool

– Data page cache (Not covered here)

• Cache Store

– Generic cache framework

– Examples: Procedure Cache & System Rowset Cache

• User Store

– Generic cache framework

– Examples: Schema Manager, Security & Metadata Caches

• Sys.dm_os_memory_cache_counters

Note: Caches are Memory Clerks

Cache Store

• Generic cache mechanism with the storage

– Sys.dm_os_memory_cache_hash_tables

– Sys.dm_os_memory_cache_entries

• Implements size control by utilizing LRU (clock) algorithm

– External Clock hand controls memory consumed by all caches

– Internal Clock hand controls memory consumed by a given cache

– Sys.dm_os_memory_cache_clock_hands

• User Store is exactly the same as Cache Store but doesn’t have storage

Memory Pools

• Object Store

– Generic pool framework

– Responds to memory pressure

– Example: SNI’ Network Packet Cache

• Object Store

– Sys.dm_os_memory_pools

Note: Pools are Memory Clerks

Memory Broker

• Mechanism to dynamically broker memory across large memory components such as Query Execution, Query Optimization & Caches

• dbcc memory status & Sys.dm_os_memory_ring_buffers where ring_buffer_type = “RING_BUFFER_MEMORY_BROKER”

NUMA support

• Automatic configuration according to hardware layout

• Application can be bound to a specific node

• Memory Nodes

– Collection of Scheduler (CPU) Nodes

– Memory locality

• Scheduler Nodes

– Collection of Schedulers

– Scheduling locality

– I/O port for network requests – preemptive I/O

– Resource & Scheduler Monitors

NUMA support cont

• Soft NUMA

– Soft Node configuration for large SMPs

– Soft Node configuration for NUMA

• Soft Nodes can’t span multiple hardware nodes

• Buffer Pool is NUMA aware

– Lazy Writer per NUMA node

– Partitioned free lists

– Perfmon, SQLServer.BufferNode

• Database pages

• Foreign pages

• Stolen pages

• Target pages

• TotalPages

SQL Server 2005 NUMA support & troubleshooting

slavao — Tue, 02 Aug 2005 18:43:00 GMT

SQL Server 2005 has been designed to take advantage of NUMA features exposed by both hardware and the Windows operating system. There are several caveats that you need to be aware when attempting to run SQL Server on NUMA platforms.

In this post I would like to go over both Windows's and SQL Server's NUMA support, describe their possible configuration, and give some advises on troubleshooting some of the issues.

The latest SQL Server's CTP release has most of the NUMA support already built in so that you can play with it and see for yourself actual NUMA in action. If you are more interested in troubleshooting without understanding the actual issues you could jump to troubleshooting section right a way.

Background: I expect you understand classic cc-NUMA configuration so that I don't have to go into details of explaining them.

Windows NUMA Support

NUMA hardware platforms could be configured differently. The two major configurations are pure NUMA or Interleaved NUMA. Pure NUMA will appear to operating system as a set of CPU nodes, sometimes called pods, with local memory. Depending on a hardware manufacture there will be a different penalty to access memory residing on other nodes - remote memory. In this configuration for an application to perform well it needs to minimize a number of remote accesses.

When system is configured to use interleaved memory, to operating system it appears as a large SMP box where memory for cache lines is interleaved from different NUMA's nodes. This configuration is suitable for applications that don't have NUMA optimizations and might behave badly in pure NUMA configuration. For example if you don't have SQL2000 post SP3 QFE that has NUMA optimizations you shouldn't be running SQL Server on the pure NUMA configuration. In general SQL2000 doesn't have many NUMA optimization so that you might want always to configure system as interleaved.

When Windows starts up on pure NUMA hardware it recognizes system multi node configuration and boots accordingly. From our experience we have noticed that during the start OS will allocate memory mostly from a single node - keep in mind that this does depend on memory availability on the nodes as well as a number of applications OS has to start. This allocation pattern during OS startup could be problematic for a NUMA aware application - since memory is not distributed evenly across the nodes and there is no way to find out memory distributions across the nodes. As a system continue running the problem becomes more severe - there is less and less free memory available on the nodes. It could be due to memory hungry applications or to System File Cache, SFC. Depending on the system configuration it is possible for System File Cache to allocate significant amount of memory. Moreover on NUMA configurations it is quite possible for SFC to unevenly use memory across nodes. If a NUMA aware application only attempts to allocate local memory it might become a victim of memory starvation on the nodes in which it collided with SFC or with other memory hungry applications.

Windows exposes a set of APIs that allows applications to take advantage of NUMA:

GetNumaHighestNodeNumber

GetNumaProcessorNode

GetNumaNodeProcessorMask

GetNumaAvailableMemoryNode

There are several API's "gotchas" that you have to be aware of:

There is no explicit way to allocate memory from the given node - a- thread allocates a memory from a given memory node by changing its affinity to the given node and then calling VirtualAlloc (if running with swap file you will have to touch virtual address before it gets bound to physical page) or other low level APIs that will trigger physical page allocation. If OS doesn't have memory on a given node it will serve the memory from the different node. It won't fail the allocation.
GetNumaAvailableMemoryNode could return 0 even though there could be a still memory that OS could serve from the node.
Windows will serve memory in the following order: 1. From the node a thread is affinitized to 2. From node that has memory free and 3. From any node - memory freed up from standby lists, working sets and System File Cache.

The consequences of A & C is that NUMA aware applications should have more of special handling for remote memory. A & C might cause OS to return memory in random order such as remote, local, remote, and etc. Keep in mind if applications decides not to cache remote memory it might get in trouble because when memory is freed it gets put on the free list and will be given away on the next allocation request.

The consequences of B is that applications can't reliably decide whether they can or can't allocate memory from the node. Case B usually occurs after system runs for some time especially when large applications are present.

SQL Server 2005 NUMA Support

SQL Server 2005 attempts to take full advantage of NUMA architecture by leveraging SQLOS for more info see http://blogs.msdn.com/slavao/articles/441058.aspx. During startup SQL Server will configure itself base on underlying OS and hardware configuration. It will create the environment inside of itself to mimic the actual hardware - SQL Server will create a software abstraction, for purpose of our discussion we will call it a Node, around every NUMA node and its memory.

Every SQL Server's Node has a memory manager, set of schedulers each of which maps to a underlying CPU; I/O port and other components. SQL Server's Node could be considered as a separate SQL Server instance with its own I/O port that it listens on. Clients or separate applications can be configured to connect to a specific Node only. A Node can be online - accepting new work or offline, accepting neither new work or connections. State of a Node could be changed on the fly using affinity settings - see setting affinity below. A connection could be made to a given Node or set of Nodes - a network port could be associated with multiple Nodes. If no extra configuration specified a new connection is assigned to a next Node in a round robin fashion. If connection is made to a set of Nodes it will be assigned to a next Node in a round robin fashion across that set of Nodes. BOL has plenty of information on how to configure SQL Server's ports, nodes and clients to connect. Once connection is created it is bound to a Node until disconnected. If connection's Node is moved offline its work will be scheduled on other Nodes. All the memory that is required to serve the connection's needs is allocated locally from the memory that is attached to the Node. Database pages are always allocated locally from the Node's memory they accessed on unless they have been brought into the memory by connection assigned to another Node. Please keep in mind that in the latest CTP SQL Server attempts not to allocate memory from different Node. This behavior might cause premature OOM or significant query slow down.

To support different NUMA features SQL Server can be configured both on server and client site. On the server site DBAs can configure amount of memory, active Nodes, a number of threads, locked pages, network configuration - what NIC is bound to what Node. On the client side DBA can configure particular clients/applications to connect to specific Nodes.

Below are two settings that affect SQL Server behavior on NUMA

Affinity - Using sp_configure a DBA can easily change SQL Server affinity. The major difference between SQL Server 2000 and SQL Server 2005 is that affinity changes will take affect right away - no server restart is required. As you know from SQL 2000 if affinity is not set SQL Server will affinitize itself to all CPUs. In SQL Server 2005 when running on NUMA and affinity is not set SQL Server will affinitize each scheduler to a set of CPUs that a SQL Server Node maps to. For example for a NUMA node 1 (nodes are numbered starting with 0) with 4 CPUs, SQL Server's schedulers belonging to Node 1 will have affinity of 0xF0. When DBA does specify affinity, we call it hard affinity, every scheduler in a Node will be affinitized to its own CPU. For example in the above example first scheduler will have affinity 0x10. When a Node's scheduler maps to an underlying CPU we say that scheduler is online. It is possible for a scheduler to be offline. For example lets say on two NUMA node system, DBA sets server affinity to be 0x1F - only 5 schedulers a mapped to actual CPUs; 4 from the Node 0 and 1 from the Node 1. Last 3 schedulers from the Node 1 are moved offline - no new work will be submitted to them. If all schedulers from a given Node are offline then the whole Node is offline - it won't get any of work assigned moreover no new connections will be made to that Node.

Max Server Memory - These are the global settings. In the current CTP SQL Server 2005 will continue allocate memory from a given node until either node still has memory - allocation returns a memory from requested node or overall allocated memory doesn't exceed max server memory. SQL Server won't try to allocate/use remote memory - the issue will be addressed in the final version of SQL Server 2005.

Troubleshooting problems related to SQL Server running on NUMA

A query runs sporadically slow even if plan doesn't change or clients observes other significant perf degradations - is the most common problem. The major cause of the problem is memory exhaustion on the Node query is running on. The memory exhaustion could happen due to multiple reasons:

Node 0 problem - All memory on this node is consumed during OS startup.

Possible solution for this problem is to change SQL Server affinity to move Node 0 offline.

Node n problem - All memory on this node is consumed by other applications or by System File Cache (also see misconfigured SFC problem below). You can verify if you are hitting this problem by looking at the size of SFC, other applications' memory consumption and amount of memory consumed by SQL Server on every Node from dbcc memorystatus.

Possible solution for this problem is to grab all the memory for SQL Server before starting other applications. You can achieve it by utilizing max server memory, dynamic scheduling and clients affinity. The idea is to start one Node at a time, connect to the node, apply load to allocate memory on the Node, start next Node, apply load, force memory allocation and so on. At the end set min and max memory to be the same so that SQL Server doesn't shrink its memory usage under the load. Here are the steps you need to make to achieve it:

Configure SQL Server to run on multiple Nodes with a port per Node
Configure a client for each Node
Prepare large query, for example reindex, that will require at least DesirableMemory/#Nodes of memory
Start SQL Server
Move all Nodes offline except for Node 0 using sp_configure
Set "max server memory" to DesirableMemory/#Nodes using sp_configure
Connect to Node 0
Run the job
Increase "max server memory to "max server memory" + DesirableMaxMemory/#Nodes using sp_configure
Move next Node online using sp_configure
Repeat 7-10 for the rest of the Nodes
Set "max server memory" = "min server memory" = DesirableMaxMemory

Misconfigured System File Cache problem - This problem is related to Node n problem. If FSC is incorrectly configured it can consume all the memory on the machine. You can verify if you are hitting this issue by looking at the size of FSC through either TaskManager or perfmon. You can fix the issue by going to "My Computer"->"Properties"->"Advanced"->"Performance Settings"-> "Advanced"->"Memory Usage Programs"->"Ok" and reboot the box, if you changed the setting. It is possible for the applications even when configured for Programs to force large consumption in System File Cache. If it happens you could write an application that periodically trims the size of System File Cache. There is one available from SysInternals http://www.sysinternals.com/Utilities/CacheSet.html (I haven't tried it lately so use it at your own risk :-))

Your comments are welcome!

SQLOS helps SQL Server to leverage hardware capabilities

slavao — Wed, 20 Jul 2005 21:50:00 GMT

I just finished an article on how SQLOS helps SQL Server to leverage hardware capabilities http://blogs.msdn.com/slavao/articles/441058.aspx. It will be interesting to hear your comments, thoughts.

Thanks a lot in advance

SQLOS Caching

slavao — Fri, 18 Mar 2005 19:19:00 GMT

Yukon caching vs SQL2000

What makes Yukon memory management different from SQL2000 is elaborative caching framework. In SQL2000 there are two major caches: data page cache, called Buffer Pool and procedure cache, cache of query plans. Buffer Pool and procedure cache are very tightly coupled. For example procedure cache relies on buffer pool's eviction mechanism to control its size, even though both have separate costing policies. Having BP and procedure cache tied together significantly simplified caching machinery in SQL2000 when we had to deal with hard to solve problems.

This model worked perfectly in SQL200 but it didn't work for Yukon. With addition of new features and new requirements in Yukon a number of caches we have to support exploded. Tying all caches to the Buffer Pool became not only problematic but not even feasible. It became obvious that we need to create common caching framework. The key idea behind the framework is uniform mechanism and the common costing policies.

Common caching framework

To cache different type of data SQLOS implements common caching framework. There are several types of cache mechanisms: Cache Store, User Store and Object Store. Each store has its own properties and hence its usage. User Store is a bit awkward name for the cache but once I describe its usage it will make more sense to you.

Before we jump into further description of stores I would like to explain a difference between meanings of caches and pools. In SQLOS's world, cache is a mechanism to cache heterogeneous type of data with a given cost for each entry. Usually there is a given state associated with an entry. A cache implements lifetime control of an entry, its visibility, and provide some type of LRU policies. Depending on the type of the data cached each entry could be used by multiple clients at the same time. For example SQL Server procedure cache is a cache in SQLOS' s terms. A plan's lifetime, visibility and cost is controlled by SQLOS's cache mechanism. Each plan can be used by multiple batches at the same time.

In its turn, pool, in SQLOS's terms, is a mechanism for caching homogeneous data. In most cases cached data doesn't have neither state nor cost associated with it. A pool has limited control over lifetime of the entry and its visibility. Once an entry is taken from the pool it is actually removed from it and pool no longer has any control over the entry until the entry is pushed back to the pool. Only one client can use an entry at the time. An example of a pool is a pool of network buffers: no state, no cost , all buffers of the same size. Keep in mind SQL Server's Buffer Pool is a cache in SQLOS terms. Currently it doesn't use any of SQLOS's caching mechanism

Cache Store and User Store

Both stores are actual caches. They are very similar. Former implements support for hash tables and latter requires users of the cache, developers leveraging the framework, to implement their own storage semantics - hence is the name User Store.

-------------- ------------

| Cache Store | | User Store |

--------------\ / ------------

\ /

-----------------

| Clock Algorithm |

-----------------

----------------

| Clock Hands |

----------------

|Clock Entry Info|

----------------

Fig 1

Conceptually there are two major controls in SQLOS caches - life time control and visibility control. Life time control provides life time management of an entry. Visibility control manages visibility of an entry. It is important to understand that entry in a cache can exists but might not be visible. For example if cache is marked for single use only, entry won't be visible after it is given a way. In addition entry might be marked as dirty. It will continue exist, live, in the cache but won't be visible to any look up.

Life time of entries is controlled by store's mechanism themselves. In case of Cache Store the lifetime is fully controlled by SQLOS's caching framework. In case of User Store entrees' lifetime is only partially controlled by a store. Since user implements its own storage user's mechanism also participate in lifetime control. For both stores entrees' visibility is controlled by the caching framework.

Lifetime of an entry is managed by embedded reference count in Clock Entry Info. Once this count goes to zero, an entry will be destroyed. In case of User Store only Clock Entry Info but not the actual data will be destroyed.

Visibility of an entry is implemented by pin count embedded into Clock Entry Info. Keep in mind that pin count and reference count are different mechanisms. One manages lifetime and the other manages visibility. For an entry to be visible its pin count needs to be visible, have value larger than 0. It also need to be non dirty and not marked for single usage. Pin count of one means that entry is visible and currently is not in used.

Hash tables

Cache store's mechanism implements its own storage - hash tables. A single cache store can have several hash tables. This is very useful when cache store users need support for different types of lookup. For example SQL Server's procedure cache leverages this functionality to enable procedures' look up by name or by id. Lookup process in different hash tables is independent of each other and doesn't conflict with clock hand movement described below.

Clock Algorithm

SQLOS's caching framework implements LRU policy to control visibility and lifetime of entries in the cache. It simulates LRU by implementing clock algorithm. Clock algorithm object is embedded both in Cache and User Stores. Currently it contains two clock hands - internal and external. An external clock hand is moved by Resource Monitor when the whole process gets into memory pressure. (http://blogs.msdn.com/slavao/archive/2005/02/19/376714.aspx)

Internal clock hand is used to control a size of a cache relative to other caches. You can think of internal clock hand as a way to put a max cap on a single cache. If this mechanism didn't exists then it would be possible for a single cache to push the whole process into memory pressure. For example if you execute numerous ad hoc queries, they can be cached. Without having internal clock hand they would force the whole SQL Server's process into memory pressure. To avoid such situation internal clock hand will starts moving once framework predicts that procedure cache's max cap is reached.

Clock hand's movement doesn't interfere with store usage. Every time clock hand steps on not-in-use entry it divides its cost by 2 If entry is not in use and its cost is zero, a clock hand will first make entry invisible and then will attempt to remove it from the cache. The remove process can fail if there is another clock hand currently working with the same entry. Once both of clock hands move away from the entry, it will be removed.

It is possible that in a future we could add more clock hands so that we can get better control over a single cache or group of caches.

Object Store

Object store is a simple pool. It is used to cache homogeneous type of data. Currently it doesn't have any costing associated with its entries. It implements max cap to control its size relative to other caches. In addition on ever memory pressure notification by Resource Monitor, object store will remove preconfigured amount of entries. It is possible to implement more sophisticated algorithms in the future. For now we would like to keep things as simple as possible.

Store DMVs and Size control

Yukon's dmvs enable you to observe cache behaviors in action. In Beta2 there were few tables that exposed information about stores: sys.dm_os_memory_caches and sys.dm_os_memory_pools.

In Beta3 you get it much more:

sys.dm_os_memory_cache_counters- provides you with summary information for every store - amount of memory used, number of entries, number of entries in use and etc…

sys.dm_os_memory_cache_hash_tables - provides you with information about cache store's hash tables - max, min and average bucket length and etc..

Sys.m_os_memory_cache_clock_hands - provides you with information about clock hands for every cache and user store - hand is running, number of rounds, amount of entries removed and etc…

These tables are very useful in understanding, monitoring and tuning behavior of a single store and SQL Server overall. For example if you observe permanent hands' movement you might improve server performance by finding ways to stop them.

In my next post I will try to analyze different OOM errors. I will also take look into ways of avoiding getting server into different types of pressure. Stay tuned!

I hope this post was useful to you. Please let me know your comments.

Be aware: FLS-Fiber Local Storage

slavao — Mon, 28 Feb 2005 19:23:00 GMT

Problem:

In the last several months there were several publications describing usage of fibers. When covering SQLOS's scheduling I will go into more details about them. Today I would like to touch on the subject on how to make your dlls/libraries more robust in the fiber environment.

As you know, when running SQL server in a fiber mode you need to be very careful when using functionality that make heavy usage of TEB, thread environment block, or TLS, thread local storage. During a fiber switch TLS and many other fields in TEB don't get switched.

You will be surprised how many libraries leverage TLS and don't work correctly with Fibers. Even older versions of MSVCRT, c-runtime, leverage TLS to allocate their per thread local data. The list of broken libraries include:

COM
RPC
Side by Side
Older versions of CRT
Windows synchronization: Critical Section & Mutex
...

So how do you protect yourself? Starting with Windows XP/2003, operating system introduced FLS, Fiber local storage: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/flssetvalue.asp. It has exactly the same semantics as TLS. However, opposite from TLS, FLS is per a fiber and it does get switched during SwitchToFiber call.

This means that if your dll will be used in fiber mode, you might want to consider using FLS instead of TLS. If a thread is not converted to a fiber, FLS is still available and there is no much overhead in using it.

If you are running XP, extended stored procedure or dll that will be loaded into SQL Server, you must use FLS. Be careful though FLS is not available on Win2K. Applications that need to run on down level platforms have to use dynamic linking - leveraging GetProcAddress API to make sure that your dll loads correctly there.

A bit of history:

FLS was finally introduced in Windows XP and 2003 because latter version of c-runtime made heavy usage of TLS for exception handling. This turned fiber mode completely unusable.

How it works:

When a thread is created it doesn't have FLS set up. Windows uses lazy initialization for FLS. When FLSAlloc gets called for the first time, OS allocates FLS for this thread. Once thread is converted to fiber, FLS gets switched during SwitchToFiber call. You can use FLSCallBack to destroy FLS entries when a fiber/thread exists or FLS gets shrunk.

I will be glad to hear your comments!

SQLOS's memory manager: responding to memory pressure

slavao — Sun, 20 Feb 2005 00:25:00 GMT

Resource Monitor and Memory Pressure

When configuring SQL Server it is very important to understand how it reacts to memory pressure. I have already spent significant amount of time describing types of memory pressure. In this post you will understand why it is important. Memory pressure is categorized into two major groups: VAS and physical. Physical memory pressure could be imposed by OS, we called external or it could be imposed by the process itself we call it internal.

SQLOS implements a complete framework to enable process's handling any type of memory pressure. In the heart of the framework lies Resource Monitor task, RM. RM monitors state of the external and internal memory indicators. Once one of them changes, RM observes state of all indicators. Then it maps indicator's states into corresponding notification. Once notifications is calculated it broadcasts it to memory clerks.

` ------------------

| Resource Monitor|

/ ------------------\

/ | \

------------------------------ ------------- ------------------------------

------------------------------ ------------- ------------------------------

Resource Monitor and Memory Clerks

Remember SQLOS has two types of nodes: memory node and cpu nodes. Memory nodes provide locality of allocations and cpu nodes provide locality of scheduling. Currently every cpu node has its own resource monitor. The reason is to be able to react to memory pressure on a given node - I will talk more about cpu nodes when covering SQLOS scheduling subsystem. For now remember that depending on machine configuration there could be multiple RM tasks running at the same time.

Large memory consumers leverage memory clerks to allocate memory. One more important task of memory clerks is to process notifications from RM. A consumer can subscribe its clerk to receive memory pressure notifications and react to it accordingly.

Every cpu node has a list of memory clerks. First RM calculates notification it needs to send. Then it goes through the list and broadcast notification to each memory clerk one by one. During the broadcast caches receive notification as well since they are memory clerks.

------------------

------------------------| Resource Monitor| -------------------------

/ ------------------ \

/ / \ \

--------------------- ----------------------- -------------------------- ------------------

--------------------- ----------------------- -------------------------- ------------------

From RM's scheduling point of view there are couple of important points you need to be aware.

Resource Monitor runs on its own scheduler, we called it hidden scheduler
Resource Monitor runs in non-preemptive mode.
DAC node doesn’t have its own Resource Monitor

There are several memory clerks that can respond to memory pressure. We already talked about caches. In addition every cpu node leverages its clerk to trim worker and system thread pools under memory pressure. Full text leverages its memory clerk to shrink shared memory buffers it shares with MSSearch. CLR uses its clerk to trigger GC. Buffer pool leverages its clerk to respond to external and VAS memory pressure only. (Why?)

External Memory pressure: RM and Buffer Pool

From SQLOS perspective Buffer Pool is a single page allocator - extensively used memory manager. External memory pressure is signaled by Windows. RM wakes up and broadcasts corresponding notification to clerks. Upon receiving the notification BP recalculates its target commit, amount of physical memory BP is allowed to consume. Keep in mind that target commit can't be lower than configuration parameter specified through sp_configure min server memory and can't be higher of max server memory. If new target commit is lower than currently committed buffers, BP starts shrinking until external physical memory pressure disappears. During this process BP tries to decommit or in case of AWE free physical memory back to OS. Remember that in SQL2000 BP didn't react to physical memory pressure when running in AWE mode.

Internal Memory Pressure: BP and Resource Monitor

Shrinkage of BP causes internal memory pressure. This is one of the ways for BP to get process into internal physical memory pressure What components BP notifies about internal memory pressure? Yes, you guessed correctly, SQLOS exposes a mechanism for BP to turn on RM's indicator corresponding to internal memory pressure. As you learned RM translates the indicator's signal to notification it will broadcast to clerks. BP has its clerk and will get RM's notification back. Oh no, we get into infinite loop!? Actually this is not the case because BP only monitors external physical memory pressure. It ignores internal physical memory pressure altogether.

There are couple other ways for internal physical pressure to appear. It could be caused by dynamically changing max server memory. In addition it could raise when 75% of BP's pages are stolen using SQLOS's single page allocator interface. By triggering internal physical memory pressure BP reclaims its pages from caches and other components currently consuming them.

VAS Memory Pressure

So far I discussed how SQLOS and consequently SQL Server handles physical external and internal memory pressure. Handling VAS pressure is harder because on Windows it is difficult to recognize it. There are two ways how RM gets notified about VAS pressure. The first way is for memory node to notify RM. When memory node Virtual or Shared memory interfaces fail to allocate a region of 4MB and below (RM doesn't get notified if size of a region above 4MB), memory node turns on RM's VAS low indicator. There also exists proactive way, when RM is running it probes VAS for 4MB size if such region no longer exists RM itself turns on VAS low signal and starts broadcasting corresponding notification.

Responding to VAS pressure is what makes Yukon different from SQL2000. In SQL2000 for server is hard to recover once it gets into VAS pressure. In Yukon VAS pressure notification will be send to all memory clerks so they have opportunity to shrink. For example cpu node will shrink its threads, CLR might unload appdomains that currently not in use, network libs will shrink their network buffers.

You remember, when talking about SQLOS memory manager, I mentioned that in AWE mode BP is capable of reacting to VAS pressure? Here it all comes together. When BP receives VAS low notification it enumerates its 4MB VAS regions it reserved previously. If it finds 4MB region that is not currently in use or either used by database pages it can easily free it.

Monitoring memory pressure:

The subject won't be complete without taking a look at how one can monitor, diagnose, different types of pressures SQL Server gets exposed to. Yes, we made yours and our life simpler. There is dmv that you can take a look at to find out history of memory pressure.

Following query shows a set of last notification RM broadcasted:

select * from sys.dm_os_ring_buffers

where

ring_buffer_type='RING_BUFFER_RESOURCE_MONITOR'

(yes we have several different ring buffers that you can pick into :-), including schedulers, exceptions and OOMs, but these are subjects for different posts)

Here is the example of query output.

<Notification>RESOURCE_MEMPHYSICAL_HIGH</Notification>

</ResourceMonitor>

</MemoryNode>

</MemoryRecord>

</Record>

Following query shows when BP, single page allocator, turns on/off internal memory pressure

select * from sys.dm_os_ring_buffers

where

ring_buffer_type='RING_BUFFER_SINGLE_PAGE_ALLOCATOR'

</Pressure>

</Record>

Sorting the outputs from these two queries by time will allow you to observe the actual behavior of the SQL Server over time with respect to memory pressure.

If you are a careful reader most of the output from ring buffer queries should make sense to you by now. Some time latter on I will try to spend more time on detailed description of the output.

Conclusion:

Memory pressure might significantly impact server performance and stability. Especially when SQL Server shares a box with other applications or shares its VAS with xps or CLR. Memory pressure might triger extra I/Os, recompilies, and other unnecessary activities. Understanding and diagnosing the types of memory pressure SQL Server is exposed to is very important part of managing your server and writing applications for it. I hope the information provided in this post will enable you to do your job more efficiently.

Have a good weekend!

SQLOS's memory manager and SQL Server's Buffer Pool

slavao — Fri, 11 Feb 2005 18:33:00 GMT

SQLOS's memory manager consists of several components such as memory nodes, memory clerks, memory caches, and memory objects. Fig 1. depicts memory manager components and their relationship:

----------------

| Memory Object |

----------------

                                        |
                                        |
                                        V
                              ----------------

| Page Allocator |

----------------

/ \

\/ \/

--------------- ---------------

| Memory Clerk | | Caches |

--------------- ---------------

\ /

\/ \/

----------------

| Memory Node |

----------------

Fig. 1

Memory Node

A memory node is not exposed to memory manager clients. It is internal SQLOS's object. The major goal for a memory node is to provide locality of allocation. It consists of several memory allocators. There are three major types of allocators. The first type is a set of page allocators. The second type is virtual allocator leveraging Windows VirtualAlloc APIs. The third type is a shared memory allocator which is fully based on Window's file mapping APIs.

The page allocators are the most commonly used allocators in SQLOS memory manager. The reason they are called page allocators is because they allocate memory in multiple of SQLOS's page. A size of a page is 8k, the same as a size of database page in SQL Server. As you will learn further this is not coincidence.

There are four different types of page allocators. Single page allocator, multi page allocator, large page allocator and reserved page allocator. Single page allocator can only provide one page at the time. Multiple page allocator, as you might guess, provides a set of pages at the time. Large page allocator could be used to allocate large pages. SQLOS and SQL Server use large pages to minimize TLB misses when accessing hot data structures. Currently large pages are supported only on IA64 or x64 hardware with at least 8GB of RAM. A size of a large page on IA64 is 16MB. The last type of page allocators reserved could be used to allocate special set of pages reserved for emergency, i.e. when SQLOS is low on memory. Fig2. shows the memory node and its allocators.

           ----------------------              ----------------               ---------------------------
          | Large Page Allocator |<--------| Memory Node |--------->| Reserved Page Allocator |
           ---------------------           /   ---------------- \           ---------------------------
                                                 /               |              \

/ | \

\/ \/ \/

                     --------------------     ----------------------     ----------------------
                    | VM & SM Allocator |    | Single Page Allocator |    | Multi Page Allocator |
                    --------------------      ----------------------      ----------------------
           Fig. 2

At this point SQL Server doesn't have a dmv, dynamic management view, that would dump a set of all memory nodes and information about their allocators. Dbcc memorystatus, discussed further, comes pretty close but it dumps information about cpu nodes not about memory nodes. You might remember that CPU nodes are proper subset of memory nodes. It means that information presented by dbcc memorystatus is sufficient to understand memory distribution on the system.

Memory Clerks

Memory nodes are hidden from memory manager users. If a client of memory manager needs to allocate memory it first creates a memory clerk. There are four types of memory clerks such as generic, cache store, user store and object store. The latter three a bit convoluted. Along with memory clerk functionality they provide data caching.

One can think of a memory clerk as a bag of statistics. It supports the same type of allocators as memory nodes as well as it enables large memory consumers to hook into memory brokerage infrastructure. (I will describe infrastructure in one of the next posts). There are several global memory clerks provided by SQLOS. SQLOS's middle and large memory consumers are encouraged to use their own clerk so that one could understand memory consumption by a component. Memory clerks infrastructure enables us to track and control amount of memory consumed by a memory component. Each CPU node has a list of memory clerks that we can safely walk during runtime. SQL Server implements sys.dm_os_memory_clerks dmv to dump clerk information. In addition combined clerk information could be derived from dbcc memory status.

Memory objects

SQLOS's memory object is a heap. A memory object requires a memory clerk to allocate its memory. We support three types of memory objects. A variable memory objects is a regular heap. An incremental memory object is a mark/shrink heap. This allocation policy is very handy during compilations and execution processes. Usually both of the processes happen in two phases. First phase is to grow memory usage and the second is to shrink memory usage. If the process is isolated we don't have to call any of destructors when freeing memory. It significantly improves performance. The last type of memory object is fixed size. As you can guess components can use such policy when they need to allocate objects of a given size.

A payload for a given memory object is 8kb. It is exactly the same as a SQLOS's page size. It also means that a memory object could be created from memory clerk leveraging single page allocator. (This is yet another very important point! Keep this in mind until I will cover SQL Server's Buffer Pool) SQL Server exposes a dmv to dump all memory objects in its process: sys.dm_os_memory_objects.

If you notice both memory clerks and memory objects dmvs expose page allocator column. Also I depicted page allocator in Fig.1. Under the hood memory object uses memory clerks's page allocator interface to allocate pages. This is useful to know when you want to join memory clerk and memory object dmvs.

So far I have described how SQLOS's memory manager structured inside. Now it is time to start talking how all this fits into SQL Server.

Buffer Pool

Now we got to the point where the life becomes very interesting. In this part all the pieces that I covered so far including memory management should start fall in their places.

Remember SQL Server has two memory settings that you can control using sp_conifigure. They are max and min server memory. I am not sure if you know but these two setting really control the size of the buffer pool. They do not control overall amount of physical memory consumed by SQL Server. In reality we can't control amount of memory consumed by SQL Server because there could be external components loaded into server's process.

When SQL Server starts, during initialization, Buffer Pool first decides how much of VAS it needs to reserve for its usage. It bases its decision on the amount of physical memory, RAM, present on the box. If amount of physical memory is equal or larger than amount of VAS it can use, remember that VAS is limited resource especially on x86, it will leave 256MB of VAS for external components plus a number of threads SQL Server is configured to use multiplied by 512KB. You might remember that 512KB is SQL Server's thread stack size. In default configuration with physical memory larger than 2GB, Buffer Pool will leave 256MB+256*512KB = 384MB of VAS space. Some people name this region as MemToLeave but in reality it is in correct. SQL Server might end up using this part of VAS itself and I will show you how it could happen latter on. You might also remember -g parameter that some people recommend to use when SQL Server starts outputting "Can't Reserve Virtual Address Space" errors. First 256MB is exactly what -g parameter controls. If you specify -g 512MB, amount of VAS that BP won't use is 512MB+256*512KB = 640MB. There is no point in specifying -g 256MB. This input parameter is the same as default value.

Once BP decides amount of VAS it will use. It reserves all of it right a way. To observe such behavior you might want to monitor SQL Server's virtual bytes from perfmon or you could use vasummary view I talked about in my previous posts. In normal case Buffer Pool can't get this much memory in one chunk so if you take a closer look at SQL Server's VAS you will see several large regions reserved. This behavior is very different from many other servers that you might have seen. Some people report it as a VAS leak in SQL Server. In reality this behavior is by design.

Buffer Pool commits pages on demand. Depending on internal memory requirements and external memory state, it calculates its target, amount of memory it thinks it should commit before it can get into memory pressure. To keep system out of paging target is constantly recalculated. Target memory can't exceed max memory that represents max server memory settings. Even if you set min server memory equal to max server memory Buffer Pool will only commit its memory on demand. You can observe this behavior by monitoring corresponding profiler event.

The size of SQL Server database page is 8KB. Buffer Pool is a cache of data pages. Consequently Buffer Pool operates on pages of 8KB in size. It commits and decommits memory blocks of 8KB granularity only. If external components decide to borrow memory out of Buffer Pool they can only get blocks of 8KB in size. These blocks are not continues in memeory. Interesting, right? It means that Buffer Pool can be used as underneath memory manager forSQL Server components as long as they allocate buffers of 8KB. (Sometimes pages allocated from BP are referred as stolen)

Here is where SQLOS and Buffer Pool meet. See Fig.3

                -----------------
                | Memory Node   |
                -----------------
                          |
                          |
                          V
           ------------------------
           | Single Page Allocator |
           ------------------------
                          |
                          |
                          V
                -----------------
                |   Buffer Pool    |
                -----------------

Fig. 3

SQLOS' memory manager can be dynamically configured to use specific single page allocator. This is exactly what SQL Server does during a startup it configures Buffer Pool to be SQLOS's single page allocator. From that point on all dynamic single page allocations are provided by Buffer Pool. For example remember that memory object's payload is 8KB. When a component creates a memory object the allocation is served by SQLOS's single page allocator which is BP.

When describing the memory manager I mentioned that every large component has its own memory clerk. It means that Buffer Pool has its own memory clerk as well. How is it possible, BP leverages SQLOS memory clerk but SQLOS' memory manager relies on BP? This is common chicken and egg problem that you often can observe in operating systems. The key here is that Buffer Pool never uses any type of page allocator from SQLOS. It only leverages Virtual and AWE SQLOS's interfaces.

           -----------------
          |    Buffer Pool    |
          -----------------
                    |
                    |
                    V
     --------------------------
     | Memory Clerk (VM/AWE) |
     --------------------------
                    |
                    |
                    V
          -----------------
          | Memory Node   |
          -----------------

Fig. 4

All SQL Server's components optimized for 8KB allocations so that they can allocate memory through SQLOS's single page allocator and consequently through Buffer Pool. However there are cases when a component requires large buffers. If it happens allocation will be either satisfied by memory node's multi page allocator or by virtual allocator. As you might guess that memory will be allocated outside of Buffer Pool. This is exactly why I don’t like term MemToLeave, SQL Server does allocate memory out of that area!

Buffer Pool and AWE mechanism

When describing SQLOS memory manager and Buffer Pool, the discussion would be incomplete without describtion of how AWE fits in all of this. It is really important to understand how Buffer Pool allocates its memory when SQL Server configured to use AWE mechanisms. First, please remember, BP leverages SQLOS's memory clerk interfaces to allocate both VAS and physical pages through AWE. Second, there are several differences that you need to keep in mind. First BP reserves VAS in 4MB chunks instead of "single" large region. This enables SQL Server to release VAS when process is under VAS pressure. (We didn't have all bits and pieces to do this when server is not configured to use AWE mechanisms). Then it allocates all of its memory using AWE mechanism on demand. This is very big difference between SQL2000 and Yukon. In SQL Server 2000 BP would allocate all of its memory when using AWE mechanism right a way.

Buffer Pool is a preferred memory allocator for the whole server. In AWE mode it allocates its memory leveraging AWE mechanism. It means that all allocations allocated through SQLOS's single page allocator will come from pages allocated through AWE. This is what many people really missing. Let me make the point again: When Server is configured for AWE mode, most of it allocations are allocated through AWE mechanism. This is exactly the reason why you won't see private bytes and memory usage growing for SQL Server in this mode.

Since data pages are use relative addressing, i.e. self contained, Buffer Pool can map and unmap them into and out of process's VAS. Other components could have done the same if they were not relying on the actual allocation address. Unfortunately there are no components right now other than BP that can take advantage of AWE mechanism.

Future posts

I haven't completed discussion about SQLOS memory management yet . There is still much to talk about. In my next posts I will cover SQLOS caches and handling of memory pressure. It is also really important to look at dbcc memory status and related dmvs.

I understand that this material is not easy to grasp. Please let me know if you have any questions or you would like me to explain some areas in deeper details.

If you have any feedback I would love to hear that as well. This is really important to me so can adjust my posts accordingly.

The weekend is just a day away: Enjoy it!

SQLOS - unleashed

slavao — Sun, 06 Feb 2005 00:38:00 GMT

With SQL Server 2005 Beta 1 and Beta 2 you might have noticed that there is no ums.dll in the bin directory. The reason is simple it is no longer exists. In SQL2000 ums.dll provided SQL Server with user mode non-preemptive scheduling. So what happened in latest version? Have we removed non-preemptive scheduling? The answer is no. Yukon continues relying on non-preemptive scheduling. As adequate research shows for RDBMS engine to meet performance and scalability requirements it needs to leverage non-preemptive scheduling.

In Yukon we took idea of non-preemptive scheduling further and have created new component, library, SQLOS, or sometimes referred as SOS. SQLOS is a user mode operating system. It includes OS pieces such as non preemptive scheduling, memory management, resource monitoring, exception handling, I/O, synchronization and hosting subsystems. Keep in mind SQLOS doesn’t provide OS abstraction layer to SQL Server. It doesn’t warp any of Windows APIs for portability purposes. Contrary it continues further binding of SQL Server to Windows by exploiting scalability and performance features of the Operating System.

The goals of SQLOS include bringing all system components together, enabling further innovation of SQL Server’s scalability and performance, providing manageability and supportability features. It exposes cohesive API to developers so that they can easily exploit features of hardware and the operating system. SQLOS models its internals to simplify developing process as for high and as for low end platforms.

SOS’s scheduling subsystem consists of scheduling nodes, schedulers, tasks, workers and system threads. A scheduling node provides an abstraction layer over group of CPUs. On NUMA systems scheduling node represents a single NUMA node. On SMP box there is one scheduling node. It is possible to change a number of nodes by editing Windows registry. We call such configuration soft NUMA. As you would expect a node also is a collection of schedulers. A scheduler is anabstraction over a CPU. Schedulers deal with tasks that run by workers that get executed by system threads. Description of SQLOS scheduling could take pages and pages so I will cover it in depth in separate post.

SQLOS’s memory management consists of memory nodes, memory clerks, caches, pools, and memory objects. A memory node is an abstraction over the memory attached to a scheduling node. A scheduling node is a proper subset of memory node. Separating scheduling and memory nodes enables us to model different hardware configurations. Every component leverages its own memory clerk to allocate large blocks of memory. Memory objects, heaps, utilize memory clerks to allocate memory. When SQL Server runs it creates a number of different memory clerks and memory objects. One could monitor them by quering SQL Server's DMVs, Dynamic Management Views. In addition to nodes, clerks and memory objects, SQLOS implements unique caching and pooling frameworks. The whole memory subsystem deserves several posts and as promised I will start covering it pretty soon.

SQLOS contains several threads to provide self monitoring and adequate resource distribution. In addition SOS exposes hosting interfaces. Hosting API significantly simplifies integration of CLR and SQL Server engines. CLR leverages SQLOS’s hosting API for all of its OS needs including thread pooling and memory management.

There is plenty of information to cover about SQLOS. One probably could write a whole book describing it. In my next post I will start going over SQLOS memory manager in deeper details. If you haven’t read my previous posts about Windows memory management it is time to do so. You need to be ready so that you can fully enjoy the next article.

Please feel free to ask any questions. And let me know your comments.

Enjoy great weekend!