If broken it is, fix it you should

Using the powers of the debugger to solve the problems of the world - and a bag of chips    by Tess Ferrandez, ASP.NET Escalation Engineer (Microsoft)

ASP.NET Memory: Thou shalt not store UI objects in cache or session scope

ASP.NET Memory: Thou shalt not store UI objects in cache or session scope

  • Comments 28
ASP.NET Memory: Thou shalt not store UI objects in cache or session scope

Surprisingly the issue I wrote about in "the eventhandlers that made the memory baloon" (Jan 2006) is something that still happens very frequently, I reference it in cases at least a few times a month.  Just this last week I had different variations of it crop up in different cases so in this post I will show a different variation, what to look out for and how to identify it.

Problem description:

The issue here is a quick growth in memory, eventually causing OutOfMemory exceptions.

The problem:

The problem in this case was that the application stored UI controls in session scope, i.e. Labels, DataGrids etc. in order to keep relevant information about the user throughout the session.  The data they want to store is not that big, a subset of a dataset, some user related strings etc. and since it was already nicely contained in the controls there was no reason to extract them into a separate object to store in session scope.

WRONG!

If we look at a dropdownlist for example in memory, this is what a dropdownlist contains  (the Name column contains the member variable names, and the Type column shows the type of the member variable)

0:032> !do 06f4fb14
Name: System.Web.UI.WebControls.DropDownList
MethodTable: 663b95bc
EEClass: 663b952c
Size: 124(0x7c) bytes
(C:\WINDOWS\assembly\GAC_32\System.Web\2.0.0.0__b03f5f7f11d50a3a\System.Web.dll)
Fields:
      MT    Field   Offset                 Type VT     Attr    Value Name
790fd8c4  4002144        4        System.String  0 instance 02f1fd64 _id
790fd8c4  4002145        8        System.String  0 instance 02f1fd64 _cachedUniqueID
663a66b8  4002146        c ...em.Web.UI.Control  0 instance 06f4f47c _parent
6641d56c  4002147       2c         System.Int32  1 instance        6 _controlState
6641194c  4002148       10 ...m.Web.UI.StateBag  0 instance 06f51c5c _viewState
663a66b8  4002149       14 ...em.Web.UI.Control  0 instance 06f4dba0 _namingContainer
663a40c4  400214a       18   System.Web.UI.Page  0 instance 06f4dba0 _page
6641f33c  400214b       1c ...+OccasionalFields  0 instance 06f53958 _occasionalFields
663b63cc  400214c       20 ...I.TemplateControl  0 instance 00000000 _templateControl
6639b220  400214d       24 ...m.Web.VirtualPath  0 instance 00000000 _templateSourceVirtualDirectory
66417cf0  400214e       28 ...rs.ControlAdapter  0 instance 00000000 _adapter
663bfea4  400214f       30 ...SimpleBitVector32  1 instance 06f4fb44 flags
790fd0f0  400213e      d08        System.Object  0   shared   static EventDataBinding
    >> Domain:Value  00198418:NotInit  001c6fd0:02f06fd0 <<
790fd0f0  400213f      d0c        System.Object  0   shared   static EventInit
    >> Domain:Value  00198418:NotInit  001c6fd0:02f06fdc <<
790fd0f0  4002140      d10        System.Object  0   shared   static EventLoad
    >> Domain:Value  00198418:NotInit  001c6fd0:02f06fe8 <<
790fd0f0  4002141      d14        System.Object  0   shared   static EventUnload
    >> Domain:Value  00198418:NotInit  001c6fd0:02f06ff4 <<
790fd0f0  4002142      d18        System.Object  0   shared   static EventPreRender
    >> Domain:Value  00198418:NotInit  001c6fd0:02f07000 <<
790fd0f0  4002143      d1c        System.Object  0   shared   static EventDisposed
    >> Domain:Value  00198418:NotInit  001c6fd0:02f0700c <<
7912d8f8  4002150      d20      System.Object[]  0   shared   static automaticIDs
    >> Domain:Value  00198418:NotInit  001c6fd0:02f07018 <<
790fd8c4  40025c5       34        System.String  0 instance 00000000 tagName
663e95f0  40025c6       44         System.Int32  1 instance       76 tagKey
663eb1d0  40025c7       38 ...tributeCollection  0 instance 00000000 attrColl
6641194c  40025c8       3c ...m.Web.UI.StateBag  0 instance 00000000 attrState
66401a78  40025c9       40 ...WebControls.Style  0 instance 00000000 controlStyle
663bfea4  40025ca       48 ...SimpleBitVector32  1 instance 06f4fb5c _webControlFlags
790fd0f0  40025cc       4c        System.Object  0 instance 00000000 _dataSource
7910be50  40025cd       50       System.Boolean  1 instance        1 _requiresDataBinding
7910be50  40025ce       51       System.Boolean  1 instance        1 _inited
7910be50  40025cf       52       System.Boolean  1 instance        1 _preRendered
7910be50  40025d0       53       System.Boolean  1 instance        0 _requiresBindToNull
7910be50  40025d1       54       System.Boolean  1 instance        0 _throwOnDataPropertyChange
790fd0f0  40025cb      ec0        System.Object  0   shared   static EventDataBound
    >> Domain:Value  00198418:NotInit  001c6fd0:02f209c8 <<
66404954  40025d3       58 ...UI.DataSourceView  0 instance 06f51c90 _currentView
7910be50  40025d4       55       System.Boolean  1 instance        0 _currentViewIsFromDataSourceID
7910be50  40025d5       56       System.Boolean  1 instance        1 _currentViewValid
663c876c  40025d6       5c ...eb.UI.IDataSource  0 instance 06f51c80 _currentDataSource
7910be50  40025d7       57       System.Boolean  1 instance        1 _currentDataSourceValid
663ba948  40025d8       60 ...ceSelectArguments  0 instance 06f539fc _arguments
7910be50  40025d9       64       System.Boolean  1 instance        1 _pagePreLoadFired
7910be50  40025da       65       System.Boolean  1 instance        0 _ignoreDataSourceViewChanged
6641a730  4002653       68 ...istItemCollection  0 instance 06f4fb90 items
79102290  4002654       74         System.Int32  1 instance       -1 cachedSelectedIndex
790fd8c4  4002655       6c        System.String  0 instance 00000000 cachedSelectedValue
79104368  4002656       70 ...ections.ArrayList  0 instance 06f539b0 cachedSelectedIndices
7910be50  4002657       66       System.Boolean  1 instance        0 _stateLoaded
790fd0f0  4002651      ee0        System.Object  0   shared   static EventSelectedIndexChanged
    >> Domain:Value  00198418:NotInit  001c6fd0:02f20bfc <<
790fd0f0  4002652      ee4        System.Object  0   shared   static EventTextChanged
    >> Domain:Value  00198418:NotInit  001c6fd0:02f20c08 <<

So if I have a dropdownlist that contain all the US states and their abbreviations, and I want to store this in cache to avoid having to rebuild the list, a logical step to take might be to cache the dropdownlist.   

However, if I do so, anything that it references will be referenced as long as the dropdownlist is in cache, so it can't be garbage collected.  In the case of web controls, they all have a _parent member variable which links back to the parent control, and a _page that links back to the page where the control was originally instantiated.  In other words, this means that as long as your control is in cache, you will also be caching the page, any other controls that it uses, any data bindings to data grids etc. etc. 

As you can see above you will also be caching the statebag (viewstate), the IDataSource and anything it references, etc. etc.  all in all it builds up to quite a bit more data than we originally bargained for, when in reality, what I wanted to store was just the list of states and abbreviations and maybe the selected item (if storing it in session scope).

The resolution:

Whenever you store anything in cache or session scope, make sure that you store just what you need and nothing more.  If you store an object that is not a simple type like a string or an int, make sure that you know all about that object and it's member variables so that you know exactly what it is that you will be holding on to.  UI objects are never good to store in cache or session scope since they contain quite a bit of extra data about their appearance and location, instead just extracting the ItemsColection and the selected item would be appropriate in the case above.

How you can identify the problem in a memory dump: 

You can get a memory dump when memory is high either with Debug Diagnostics 1.1, or by running adplus -hang -pn w3wp.exe (from the debugging tools for windows).

If you open it up in windbg, and load sos (.loadby sos mscorwks) these issues are fairly easy to identify.

Running !dumpheap -stat you will see a lot of System.Web.UI items at the bottom of the output in this case. 

65412bb4    2,461    3,004,252 System.Data.RBTree`1+Node[[System.Data.DataRow, System.Data]][]
7a75a878  189,495    3,031,920 System.Collections.Specialized.NameObjectCollectionBase+NameObjectEntry
65408b8c   47,914    3,066,496 System.Data.DataRow
663c1f78   80,093    3,524,092 System.Web.UI.WebControls.TableStyle
6640776c  226,668    3,626,688 System.Web.UI.Pair
66412f04  140,180    3,925,040 System.Web.UI.WebControls.ListItem
663eb1d0  278,998    4,463,968 System.Web.UI.AttributeCollection
654359c8    3,203    4,682,276 System.Data.RBTree`1+Node[[System.Int32, mscorlib]][]
663c3f04  397,066    4,764,792 System.Web.UI.WebControls.FontInfo
663bb498  133,543    4,807,548 System.Web.UI.WebControls.TableRow+CellControlCollection
663d5cb0   64,087    5,126,960 System.Web.UI.WebControls.HyperLink
1c1165bc   51,295    5,129,500 SomeControlLibrary.WebControls.SomeControl
79102290  551,137    6,613,644 System.Int32
663d6ad4   78,635    6,919,880 System.Web.UI.WebControls.Table
79104368  312,558    7,501,392 System.Collections.ArrayList
66411ac8  390,338    9,368,112 System.Web.UI.WebControls.Unit
663f2408  112,314    9,434,376 System.Web.UI.WebControls.Image
66401a78  251,127   11,049,588 System.Web.UI.WebControls.Style
663ec8e8  133,546   11,217,864 System.Web.UI.WebControls.TableRow
663dd2b0  259,638   11,424,072 System.Web.UI.WebControls.TableItemStyle
663bd0a0  149,928   11,994,240 System.Web.UI.WebControls.Label
664140a4  263,577   15,814,620 System.Web.UI.LiteralControl
7912d9bc   42,506   16,614,336 System.Collections.Hashtable+bucket[]
663c7308  462,037   16,633,332 System.Web.UI.ControlCollection
6641194c 1,269,095  20,305,520 System.Web.UI.StateBag
663d7328  341,364   28,674,576 System.Web.UI.WebControls.TableCell
7a7580d0 1,494,283  29,885,660 System.Collections.Specialized.HybridDictionary
6641f33c  751,897   33,083,468 System.Web.UI.Control+OccasionalFields
7a75820c 1,309,784  36,673,952 System.Collections.Specialized.ListDictionary
663c1de8 2,427,266  38,836,256 System.Web.UI.StateItem
7912d8f8 1,016,027  47,522,388 System.Object[]
7a7582d8 2,671,736  53,434,720 System.Collections.Specialized.ListDictionary+DictionaryNode
000e1b50    6,519   69,179,268      Free
790fd8c4 1,674,890  213,206,880 System.String

And if you run !dumpheap -type aspx or !dumpheap -type ascx to list all the aspx/ascx pages on the heap you will likely find many of them

0:034> !dumpheap -type aspx
...
Statistics:
      MT    Count    TotalSize Class Name
1d32425c        8        5,088 ASP.register_aspx
1b39666c      175       86,800 ASP.login_aspx
1ceab19c      199       93,928 ASP.products_aspx
1c15b6cc      280      222,880 ASP.home_aspx
1cea17ac      532      261,744 ASP.default_aspx
Total 1,194 objects, Total size: 670,440

The next step is to take any of these pages, preferably the ones that are most plentiful like ASP.default_aspx in this case, and !gcroot some of them to figure out why they are still around

0:034> !dumpheap -mt 1cea17ac     
Using our cache to search the heap.
   Address         MT     Size  Gen
...
040d0eb4 1cea17ac      492    2 ASP.default_aspx
04110d38 1cea17ac      492    2 ASP.default_aspx
0412b918 1cea17ac      492    2 ASP.default_aspx
04201758 1cea17ac      492    2 ASP.default_aspx
04213ea8 1cea17ac      492    2 ASP.default_aspx
0421943c 1cea17ac      492    2 ASP.default_aspx
042496bc 1cea17ac      492    2 ASP.default_aspx
042ae71c 1cea17ac      492    2 ASP.default_aspx
042ec8f4 1cea17ac      492    2 ASP.default_aspx
043310b4 1cea17ac      492    2 ASP.default_aspx
043c7f74 1cea17ac      492    2 ASP.default_aspx
0442d27c 1cea17ac      492    2 ASP.default_aspx
0453cb08 1cea17ac      492    2 ASP.default_aspx
04617168 1cea17ac      492    2 ASP.default_aspx
...
 

0:034> !gcroot 042ae71c
...
ESP:1fbf894:Root:  109420b0(System.Threading._TimerCallback)->
  10942068(System.Threading.TimerCallback)->
  10940004(System.Web.Caching.CacheExpires)->
  10940024(System.Object[])->
  109407b4(System.Web.Caching.ExpiresBucket)->
  06f05928(System.Web.Caching.ExpiresPage[])->
  06f059ac(System.Web.Caching.ExpiresEntry[])->
  04616834(System.Web.Caching.CacheEntry)->
  04616804(System.Web.SessionState.InProcSessionState)->
  0e474dd4(System.Web.SessionState.SessionStateItemCollection)->
  0e474e34(System.Collections.Hashtable)->
  0e481110(System.Collections.Hashtable+bucket[])->
  04616698(System.Collections.Specialized.NameObjectCollectionBase+NameObjectEntry)->
  045ea5d8(System.Web.UI.WebControls.DropDownList)->
  042ae71c(ASP.default_aspx)->

If we follow the root chain from the bottom up we see that ASP.default_aspx is a member variable of the DropDownList (the _page member variable) and that this in turn is a member variable of a NameObjectEntry in a hash table in session state.  The NameObjectEntry is a key-value pair with the key and the value of a session variable, so through this we can gather that we are storing the DropDownlist in session scope.

If we look at the NameObjectEntry we can also see exactly which session variable it is (LstStates in this case)

0:034> !do 04616698
Name: System.Collections.Specialized.NameObjectCollectionBase+NameObjectEntry
MethodTable: 7a75a878
EEClass: 7a7c51c4
Size: 16(0x10) bytes
GC Generation: 2
(C:\WINDOWS\assembly\GAC_MSIL\System\2.0.0.0__b77a5c561934e089\System.dll)
Fields:
      MT    Field   Offset                 Type VT     Attr    Value Name
790fd8c4  400117c        4        System.String  0 instance 04968898 Key
790fd0f0  400117d        8        System.Object  0 instance 045ea5d8 Value
0:034> !do 04968898
Name: System.String
MethodTable: 790fd8c4
EEClass: 790fd824
Size: 38(0x26) bytes
GC Generation: 2
(C:\WINDOWS\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)
String: LstStates
Fields:
      MT    Field   Offset                 Type VT     Attr    Value Name
79102290  4000096        4         System.Int32  1 instance       11 m_arrayLength
79102290  4000097        8         System.Int32  1 instance       10 m_stringLength
790ff328  4000098        c          System.Char  1 instance       4c m_firstChar
790fd8c4  4000099       10        System.String  0   shared   static Empty
    >> Domain:Value  000db310:790d884c 00109200:790d884c <<
7912dd40  400009a       14        System.Char[]  0   shared   static WhitespaceChars
    >> Domain:Value  000db310:109203f4 00109200:109242f8 <<
 

In summary, be careful with what you store away,

Laters,

Tess

  • Hello Tess,

    is the !dumpheap -type aspx specific to a SOS version ?

    I tried that and i get 0 items, but in my dumpheap -stat i can see i have a lot aspx

    Thanks

  • in some versions you have to run !dumpheap -type *aspx*

  • Do you know in dumpheap -stat what -1 in GEN means ?

    0x0a402814 0x18d04bc4      256   -1 _ASP.ReportOpPendentes_aspx

  • You shouldnt get the gen column with !dumpheap -stat, probably with some other switch for !dumpheap since the gen is only displayed for individual objects.

    I believe it means it has already been garbage collected.

    You would usually see it if you try to dump out objects outside of the currently used areas of the heap.  

    If you get it with normal !dumpheap -type ... or !dumpheap -mt ..., make sure that you are using the sos that comes with the framework.

  • You are right, it was with dumpheap -mt and not -stat. This is something that always confuses me. If it´s already Garbage collected why is it still around ? Because if i do a gcroot i get no root for those objects

  • it just hasn't been marked as free yet, perhaps it is in the middle of a garbage collection, or there is something wrong so that you are enumerating items that are outside of the bounds of the currently allocated memory.

    I would have to look at the exact details to give you any further answers...

  • AJAX Cool ScriptManager stuff I missed... [Via: Jay Kimble ] ASP.NET ASP.NET Memory: Thou shalt not...

  • You'd think by now someone would have thought to turn this into a compiler warning or MDA as often as the issue  comes up.

  • many, many thanks. that was exactly our problem here. Now we fixed it with your help!!! Many thanks!

  • I'm wondering if this technique could be used to create a tool to evaluate the amount of memory required for one user's Session state?

    I'm working on an ASP.NET application that stores a lot of data in a Session and I'm concerned about how this will scale as the company adds customers. It would be great to know the exact resources a single user requires. That way, we'd know whether to rebuild or just add memory to cope with expansion.

    Thanks,

    Ken

    MVP [ASP.NET]

  • you can always use something like this

    http://blogs.msdn.com/tess/archive/2007/09/18/debugging-script-dumping-out-asp-net-session-contents.aspx

  • On a related theme.

    I have a question regarding the MVP pattern that has become so popular of late.

    One of the ideas behind the pattern is passing around a view. Often the view can be a whole ASP.net page.

    I was wondering what the performance implications of this are as there must be a lot of information being transfered around that does not need to.

    Any ideas?

  • are you talking about MVC (Model View Controller)?

    If so, I think the story is a little bit different.  You wouldnt be passing around controls that have ties back to the page necessarily.  If you do then that might be an issue...  to be really honest I haven't really looked into it enough to know what the details are and what the implications would be.  Once I get there I'll make sure to post about it though...

  • Thanks for the reply!

    Yes the MVP is a slightly different take on the MVC pattern but they have essentially the same goal; seperation of concerns.

    I guess I should stop being lazy and fire up windbg and have a look myself.

    Any tips on what to look for?

  • Would I see this issue if storing Control instances in the HttpContext.Item Collection.  I know the Context dies at the end of the request, but would it delay the garbage collection of the control and associated parent?

Page 1 of 2 (28 items) 12
Leave a Comment
  • Please add 3 and 7 and type the answer here:
  • Post