When it is a status message?
One of the challenges when interacting with an asynchronous queue is error handling. In a connected application it is easy to interact with the user and tell them what went wrong - but if they have submitted a timesheet and something about it isn't quite right then it is harder to return this information. In the Manage Queue screen the problem will show as a failed job, and an error will display, but in many cases it isn't really that anything broke - just that something about the timesheet wasn't right. You may experience this more using the PSI web services directly from a custom application than if you are using the normal timesheet interface, as we are aware of the various states and will not let you try and submit something that isn't going to go through. However, there could be timing issues that would even make our timesheet jobs fail - and generally this information will display with the timesheet..
If you are developing a timesheet application you should consider the best way to get this information back to the resource through your application - as they may not see their queue jobs if they don't generally use PWA.
Technorati Tags: Project Server 2007
Working on a case recently I could see from the application logs that a server was having a hard time. Timeouts, out of memory exceptions, lots of red - so something was up. The customer's comment was that this was period end processing so lots of timesheets and updates going through. The queue was meant to avoid this problem, so I looked at the queue settings and both project and timesheet queues were set to use 10 threads, rather than the default 4. This could well have been leading to their problems!
The queue is designed to limit the rate that work gets processed so that the peaks that can overload the server get spread out. In project management terms think of it as leveling for the server - stopping the server from trying to do many things at once. Another analogy is my weekend to-do list If I had a list of 10 things I would spend at least Saturday morning deciding what to do, and which my least favorite task would be. Then I might end up swapping tasks and my productivity would be poor. My wife has learned that I am "single-threaded" and best throughput is achieved by giving me one job at a time.
How many threads is the right number? This very much depends on your server configuration and also the workload mix in terms of both volumes of transactions and the size of each transaction. You can obviously publish many more 10 line projects than 1000 line projects in the same time. One rule of thumb some of our field guys use as a starting point is to set the number of threads to the number of available processors (or cores). So if you have a single dual processor machine as your application server then 2 threads might be a good starting point; if you have a quad dual-core then you might be able to use 8 threads. This will also depend on farm topology and other applications running on the same servers. You wouldn't want to use these figures and also have the server acting as a Web Front-End or running search or other processor intensive activities.
Monitoring performance counters and the application and ULS logs will enable you to fine tune the queue to work with your normal server loads - but please don't just increase the number of queue threads expecting to make things work faster. Time is nature's way of keeping everything from happening at once - for project server we achieve the same thing with the queue!
See http://blogs.msdn.com/brismith/archive/2008/03/24/slow-olap-cube-builds-and-large-tempdb-revisited.aspx for the latest information on this problem.
There is an issue with some of the SP1 and hotfix 939594 fixes which resolve earlier problems of data missing in the cubes. The query has become more complex and SQL Server 2005 is not coming up with the right execution plan for this new more complex query. The upshot of this is much use of tempdb - which can grow very large, and the cube build takes much longer than it did.
You may have already seen the blog which gives a workaround for this issue - but as many of you may see this anew after loading SP1 I wanted to raise awareness again. Thanks to Noel, Kermit, JF and Thuy for helping get to the bottom of this one.
The problem is mostly seen if you have added lots of dimensions to the various cubes and also if you specify a dynamic date range. A full build from earliest to latest is sometimes a workaround which reduces the impact on tempdb but is still slower than it should be. My suggested steps which are also mentioned in the referenced blog and given even more coverage in the MSDN article are:-
1. Create a cube set exactly how you want it (earliest to latest please - no dynamic date range) and while it is building capture a trace in SQL Profiler. SQL Database Engine should be profiled – can be restricted to the reporting db.
2. After a few minutes search in the running trace for MSP_EpmAssignmentByDay_OlapView. If it isn’t found then wait a little while longer and try again.
3. Once found then right click the line in the trace and select Extract Event Data. (DO NOT COPY FROM THE bottom pane). In the save as dialog save this as badquery.sql.
4. The current cube can be stopped at this point. If no one else is using Analysis Service then just restarting will kill this off. If you do restart then you will need to close out of the Cube Status window otherwise you can get errors with the next cube build.
5. In SQL Server Management Studio open a new query to the Database Engine and select the Reporting DB. Copy the following text into the query window.
EXEC sp_create_plan_guide N'guide_forceorder',
N'OPTION (force order)'
6. Open the badquery.sql file and select all (Ctrl-A or Edit, Select All) then Copy (CTRL_C or Edit Copy) and then select the <exact query> from the text in the query window and Paste (CTRL-V). You should have selected the < > character too. This may leave some space between the ‘ and the SELECT but this is fine.
7. Execute this command – which should finish successfully. Build a new cube and this should use this execution plan and process more quickly. To get an indication of the speed you could open badquery.sql and then at the end of the query paste OPTION (FORCE ORDER) and then execute.
8. You can also monitor in SQL Profiler to see if it is finishing quicker by looking for the MSP_EpmAssignmentByDay_OlapView and seeing if the SQL BatchCompleted comes in a reasonable time.
The reason to avoid dynamic date ranges is that this will make the query change every time it is run - and then the execution plan will not match - and will be ignored.
If you use a constant date range then this method can be applied - but read the MSDN article on the need to escape single quotations marks. The will be around the dates - so for instance '12/31/2008' would need to be ''12/31/2008''. Note that these are two individual quotes, not a double quote.
As an example this workaround can mean the tempdb hardly gets touched (rather than growing 30GB or more) and the cube builds in less than 1 tenth of the time. Your mileage may vary.
I hope this helps - and if you don't understand any of the steps above then you probably shouldn't attempt this - speak to your DBA. But if you are seeing this issue and need some assistance to address it then please open an incident with our support teams. Http://support.microsoft.comwill give you the options.
Thanks to our writers, we have a new publication for you. In their own words:-
It can sometimes be a challenge for new users to find their way through Project 2007 on their way to becoming project managers. The Project team has just produced another tool that will help you understand Project 2007 —The Project Management Quick Reference Guide. You can download and print this template for ease of use. Now you have another tool to help your organization achieve your project goals.
I have been waiting for SP1 before blogging this as it might have frustrated a few people as pre SP1 there were some broken pieces here. The filter that you use when creating a view for everyone through Server Settings, Manage Views behaves slightly differently from that found on the client side when a user creates their own filters. So why is this? To explain I will use the example of filtering for particular project types. I will base this on the out-of-the-box Project Center, Summary view, with the field Project Type added.
Here is a view of my starting point view - where "Maintenance Project" is the term for Proposals and Activity Plans.
When adding a filter on the client in Project Center the filter works as most people would expect - it is doing a text comparison. If I want to just see Maintenance Projects I can set a custom filter where the Field Name Project Type contains the word Maintenance.
And then we see:-
But if we want a view that users can choose that automatically has this filter then we probably want to define it on the server. Setting exactly the same filter in the view definition for Summary in Manage Views gives this:-
Why is nothing showing up? The reason is that server side filters are not working quite the same as on the client - and are doing their comparisons against an enumeration of the Project Type defined as a number. Maintenance Projects have a Project Type of 4 - so with a filter set like this - where Project Type equals 4
we get what we want on the client with no client side filters set.
Enumerations for various of these kind of things can be found in the SDK. This different behavior makes more sense when you look at different language versions. If you have created a PWA site in French for instance you can set a server side filter for Maintenance Project without needing to know the French term. Remember on the server you can filter on a field that is not displayed - so it isn't as silly as it sounds...
I know in this case it wouldn't make much difference - maintenance is maintenance - but other translations may be more challenging (at least to me). This is a site provisioned in Korean, and viewed with IE set to French - just so you can see we are still looking at the same field.
I hope this helps you understand why there are differences in the filters - and how to use them.