Cum Grano Salis

Optimizing life – Life in Async

Shahar — Wed, 01 Jul 2009 17:43:09 GMT

I was going to get coffee today at work, minding my own business, when someone from the test-team asked me if the order I was doing things when preparing coffee was intentional. Absentmindedly, I explained the reasoning behind the order of things and how I reached this way of doing things and belatedly realized that I may have gone too far in my explanation – I think it was the look of pity in his eyes that clued me in.

To explain, first you need to understand what our kitchenette looks like and the various steps in making coffee:

Here are the various things we are seeing here:

1) Access – that’s where I typically come to the kitchenette from.
2) The coffee machine
3) Counter
4) Cups
5) Lids
6) Fridge (Where milk is).

Our coffee machines basically give you an Americano – no milk. The way I take my cup’o’joe is to do 12 oz of machine generated coffee (into a 16 oz cup) and fill another 2-3 oz of milk – and then the cup needs to be lidded as well. Of course, there are a number of variations on how to make such a coffee with this layout.

Making Coffee – the naive way:

Step 1: Go to position (4) – grab a cup.
Step 2: Go to position (1) – place cup at machine.
Step 3: Press buttons - Wait for coffee to be made.
Step 4: Take milk from fridge (6)
Step 5: Fill cup with milk (possibly at position (3))
Step 6: Discard milk carton or, if still full, place back in fridge.
Step 7: Go to (5) and grab a lid.
Step 8: Place lid on cup.

And we are done. To understand how this can be optimized we first need to figure out how much each part of the coffee making process takes:

Making coffee (the machine phase): ~2 minutes (constant).
Getting milk (from position (1)): On average, takes about 20 seconds. Depending on traffic, can easily take as long as 1:00 minutes if there is a lot of traffic and even longer if you consider the possibility of people talking to you while you are making the coffee.
Getting a cup: Usually takes about 5 seconds from position 1 – can take longer due to traffic and chat. Rarely takes more than 30 seconds and requires little concentration.
Filling cup with milk: Takes about 5 secodns.
Discarding/Returning milk: Takes about 10 seconds. Worst case (traffic etc) can take up to a minute.
Grabbing a lid: 5 seconds from position (1).
Lidding the cup: 5 seconds at position (3).

So.. Translating this into pseudo code, this is what we get:

function NaiveCoffeeMaking()
{
WalkTo(4);
GetCup(); // 10s
WalkTo(2);
PlaceCupInMachine(); // 1s
MakeCoffee(); // 120s
GrabCupFromMachine(); // 1s
WalkTo(6);
GetMilk(); // 20s
PourMilk(); // 5s
DiscardOrReturnMilk(); // 10s
WalkTo(5);
GrabLid(); // 5s (probably closer to 3s in this case)
WalkTo(3);
LidCup(); // 5s
}

Phew.. Okay.. So we have that. Now lets calculate how much time this takes (best case):177 seconds!

The thing to consider though is that the machine phase can be asynchronous – that is to say – while the machine is making coffee, the CPU (me) is waiting Idle. The question is, how can we use that to optimize the time? For our pseudo code then, we will now separate MakeCoffee into two functions: BeginMakeCoffee() which essentially means pressing the buttons on the machine and immediately return to the caller and EndMakeCoffee() which completes the operation (and if not done, blocks until done). And so, the most optimized version of how to make coffee is:

function FullyOptimizedCoffeeMaking()
{
WalkTo(2);
BeginMakeCoffee(); // 120s – but returns immediately
WalkTo(4);
GrabCup(); // 5s
WalkTo(6);
GetMilk(); // 20s
PourMilk(); // 5s
DiscardOrReturnMilk(); // 10s
WalkTo(5);
GrabLid(); // 5s (probably closer to 3s in this case)
WalkTo(2);
PlaceCupInMachine();
EndMakeCoffee();
LidCup(); // 5s
}

Notice that when we call BeginMakeCoffee(), we immediately return and continue processing – going through the phases of getting the milk, preparing the cup and only then, when done, placing it in the machine. So, in this case, best case time would be: 125 seconds. That’s 120 seconds that it takes to make the coffee plus the 5 that it takes to lid the cup. The rest of the time is taken up in parallel to making the coffee.

But there’s a problem! BeginMakeCoffee() and EndMakeCoffee() do not work exactly like a machine async operation. Generally speaking, when you issue an async operation, the sub-system will take care to store the result for you when it’s done and wait patiently for you to call the second part of the operation (EndMakeCoffee) before it returns the result. A coffee machine does not work that way. When the coffee is ready, it’s coming out – whether there’s something to store it or not. That presents some problems for the FullyOptimizedCoffeeMaking() function! What happens if instead of talking about Best case, we were to talk about worst case. The GetCup() can sometimes take 30 seconds. GetMilk can sometimes take 120 seconds. As well as some of the other things that make take longer under high-stress cases. What happens then? Well, what happens is that the coffee starts pouring and there’s nowhere for it to go, so it gets wasted. Sacrilege!!

In essence, we need something that’s both optimized AND safe. Since life does not have convenient temporary memory (where the coffee would have been stored) or sync objects (where the act of pouring the coffee could have been locked and deferred until a cup is to be placed underneath the nozzle), it means we need to work around those limitations:

function OptimizedAndSafeCoffeeMaking()
{
WalkTo(2);
BeginMakeCoffee(); // 120s – but returns immediately
WalkTo(4);
GrabCup(); // 5s
WalkTo(2);
PlaceCupInMachine();
WalkTo(6);
GetMilk(); // 20s
WalkTo(2);
PourMilk(); // While cup is in the machine – 10s
WalkTo(6)
DiscardOrReturnMilk(); // 15s
WalkTo(5);
GrabLid(); // 5s (probably closer to 3s in this case)
WalkTo(2);
EndMakeCoffee();
LidCup(); // 5s
}

The CPU (me) now needs to work more – you will notice that there is more walking around to accommodate the new placement of the cup (it can no longer be carried around – one needs to walk to it to pour the milk and then back to the fridge to discard or put back). The time this now takes is still 125 seconds - 120s to make the coffee and prepare the cup and another 5s at the end when the cup is lidded. Switching the lines around and doing more leg work though has bought us a lot more safeness.

Note that this is still not perfect – if the 5s it takes to get a cup and place it in the machine would balloon from 5s to 120s, we would be right back in our original position of losing our coffee. However, that is far less likely to happen in this case. If you want to eliminate risk completely, simply switch BeginMakeCoffee() with GrabCup() and PlaceCupInMachine(). You will lose 5 seconds (making the total time 130 seconds), but you are then completely safe.

So that’s what I was trying to explain to the tester when that look came to his face. The thing is, I am sure everyone does this. But people probably do this unconsciously or at least in a less ~~obsessive compulsive~~ analytic manner than a software developer would.

Windows 7 RC – Media Streaming (“Stream” button in Windows Media Player)

Shahar — Thu, 07 May 2009 20:28:49 GMT

I found out about this feature a bit ago and.. Wow.. I am so impressed. The set-up procedure is a little convoluted, but once you do it, the thing just works.

Here’s the idea: Allow your identity (your Windows Live account, basically) to stream your media accross the interwebs. If my home computer has my entire outdated music library on it, I don’t need to copy the entire thing to my laptops to access it – I just make sure to associate my Windows Login user to my Live account user and voila – I can stream media from my house when I work!

Step by step:

You can associate your live-id to your windows user account by using the control panel, but Windows Media Player gives you a shortcut. In the Stream drop-down, choose “Allow Internet access to home media”:

Then, you will be presented with a dialog containing the options you have. You should click “Link online ID”:

Once you click that, WMP will bring you to a window that allows you to add an online ID provider. Click the “Add an online…” button:

Once clicked, you will be presented with providers. Click the “Windows Live” one – that will again redirect you to a download from Microsoft. Choose the correct architecture (x86 or x64) and download/install the package. Once you did that, going back to the “Link online IDs” dialog will look like this:

You now need to select the “Link online ID” on the WindowsLiveID highlighted item. That will ask you to log into Windows Live with your Live-ID. Once that’s done, the window will again change to show the linked account:

If you go back to Windows Media Player, you should choose the”Allow internet access to home media” option on the open dialog box (same as the second image in this post). Once clicked, the dialog should close and you should land back inside Windows Media Player. But now, you will have your home library available to choose from:

You can now play your music to your heart conent. Apparently, this also supports streaming of video. The quality seems kind’a random – I have seen it be very good (indistinguishable from the original) and pretty bad. I guess it takes your bandwidth into account:

Enjoy!!

Prish Resizer moved to another blog (Also, new version – jpg quality support + custom size through context menu)

Shahar — Tue, 20 Jan 2009 17:00:00 GMT

Image Resizer has moved to another blog – this way it does not interfere with this blog which is all about Excel Services – my actual daytime job.

Find it here!

What new features would you like to see in Prish Image Resizer?

Shahar — Wed, 14 Jan 2009 17:39:47 GMT

Now that the x64 issue and the non-latin characters issues are solved, what new features would you like to see in the resizer?

Post your comments.

New version of Prish Image Resizer (x64 support. Unicode support).. It’s been a while..

Shahar — Wed, 14 Jan 2009 17:00:00 GMT

Well.. It’s been a while since I updated..

Short and sweet – this is the new version of Image Resizer. There are 2 setups now – one for 32bit and one for 64bit.

This also solves the issues people have been having with foreign characters making resizing impossible.

You can find the new versions in here:

Direct links:

Tips on Excel Services article published on MSDN

Shahar — Fri, 23 May 2008 21:33:25 GMT

This article contains some basic troubleshooting/tips on using Excel Services.

New published paper - advanced usage of Excel Services and File Format manipulations

Shahar — Tue, 06 May 2008 17:46:34 GMT

Sergei Gundorov (whom you may remember from a previous post) has published a paper on MSDN where he talks about a very interesting solution his team wrote in the Microsoft IT department. The solution is probably one of the most advanced implementation of Excel Services I have seen to date and includes lots of ingenious customizations - the white paper, specifically, talks about how to use Excel Services as a landing pad for data and then get the file stream from it, customize it (add stuff into it) and serve it up to the user.

You can find the paper on MSDN.

Awesome mouse-pad

Shahar — Fri, 25 Apr 2008 03:28:32 GMT

We have a surplus computer store near my house which I like mostly because their pricing on cables is not equivalent to the GDP of a small island state. They have some real wonky stuff in there - very old printers, monitors, etc.

I passed through there today and saw this:

Most-awesome-mouse-pad evah!

It is so much better than the competition because it has space not for two, but FIVE 3.5 diskettes. I think the only way this could be even awesommer is if it had space for 5.25 disks.

Transitioning from TiVo to Windows Media Center

Shahar — Tue, 25 Mar 2008 20:55:36 GMT

I finally made the jump to Windows Media Center 2 months ago or so and, overall, I am much happier for it. I have had TiVos in my house since 2001 and it has gotten to be that I just never watch live TV anymore. I adore the concept of the DVR and am greatly annoyed when I dont have it at my fingertips.

Except for a few hiccups when I started (some due to me and others due to the fact that I bought the Dell 420 when it just came out, before Dell has fully gotten rid of the kinks), the machine has been working famously for about 2 months now. And it's just great.

There are a bunch of things I don't like about it, but for the most part they revolve around the programmability model. The software itself, out of the box, is absolutely awesome - it is SO much more responsive than TiVo (a Quad processor helps!) and the fact that you can upgrade HDDs and hardware generally is great too. The menu system is also miles ahead of TiVo, not to talk about integration with my home network and the Internet - totally and utterly sweet. The Movie Guide is a real nice touch and very easy to use. So is conflict resolution - I find it much better presented and handled.

The one feature I really miss though is grouped-shows by title ordered by date.

Who's Afraid of the big-bad cast operator?

Shahar — Mon, 24 Mar 2008 16:00:39 GMT

My previous blog entry (which was completely misunderstood by some people who commented - I blame the fact that English is not my native tongue) reminded me of another poor coding behavior. Just so we don't have a repeat of that last post, let me clarify that I have nothing against the as operator - I use it all the time - I just have a problem with certain usage patterns that employ it.

For whatever reason, people tend to use the "as" operator instead of the cast operator.

A lot of times you will see code like this:

MyClass c = o as MyClass;
c.DoSomething();

Where the following code makes MUCH more sense:

MyClass c = (MyClass)o;
c.DoSomething();

I have two theories as to why people do it this way:

1. People are afraid of Exceptions.
2. People are used to dynamic_cast<>() from C++ which is the easiest way to check if a class is of a certain runtime type.

The reason the example above with the"as" operator is a Bad Thing is because it defers the exception to a later point in your code. If the class is not what you expect it to be, your cast will succeed (resulting in NULL) only to fail a few lines later on a NullReferenceException - which in turn makes debugging and logging harder to understand because you are not really seeing the correct exception.

Now, there are many valid usages of the as operator - but they almost always rely on the fact that you actually do something with the fact that the class is different than what you expect. So for example, if you want your Equals method to be able to handle classes different than your own, that's a GREAT place to use as:

public override bool Equals(object o)
{
    MyClass other = o as MyClass;
    if (other == null) return false;
    //....
}

However, if you do not intend for your .Equals method to support comparison with other types (and have no other usage for the knowledge that the reference you got is not what you expected), you should definitely use the regular cast operator.

Using a Generic return type that does not appear in the parameter list - when to use it

Shahar — Sun, 23 Mar 2008 21:15:00 GMT

EDIT: Contrary to what people understood from this post, it is NOT a "Do not use Generics" post - I think Generics are a GREAT addition to the language/framework and I use them all the time. It is against a very specific usage of Generics

Short Answer: Rarely.

Slightly longer answer: When you are aware of the alternatives.

Yet longer answer:

I have seen this a couple of times recently and thought I'd drop a line about it. As always, take it with a grain of salt, but here goes...

You find yourself wanting to write a method that returns a value to the caller. Said value can be of many types and you decide to create a template out of it - a recent example I saw (which is very typical to the misuse of this mechanism):

T GetRegValue<T>(string path); // Get the value of a registry path

This method calls into the Win32 wrappers and gets back an Object, the code then returns it to the caller in the following manner:

T result = (T)objectResult;

I have heard many reasons as to why people choose to use this mechanism - the two that come up most often:

1. It's easier to understand/write/maintain or it is "cleaner".
2. It's more type-safe.

Lets go over those one by one...

It's easier to understand/write/maintain or it is "cleaner"

The amount of code that needs typing in this case is exactly the same as if you had returned an object. Compare and contrast:

string result = (string)GetRegValue("path"); // This is when using the "old and boring" way
string result = GetRegValue<string>("path"); // This is when using the "new and shiny" way

It's EXACTLY the same amount of code characters and I am pretty sure more people know how to read/parse casting operators (the (string) bit) than generic type parameters (the <string> bit). So it's the same amount of code (not easier to write), it's using a mechanism that less people know (not easier to understand). That leaves "easier to maintain" which I usually equate to "easier to understand".

We are left with it's "cleaner" but that's a whole new blog entry - to me both mechanisms look alike from the cleanliness point of view.

It's more type-safe

This is just not true. In the typical example, a cast is made to the requested type (T in this generic method) - casts in .NET are safe (that is to say, they will throw an exception of they do not work and if they do work they are guaranteed to work properly). The code doesn't even eliminate the cast - it just moves it to the inside of the method.

Here's my rationale behind why such a mechanism should not be used:

1. It's harder to debug.
2. It doesn't contribute anything.

It's harder to debug

In the cases where the type of the retrieved cannot be cast, your exception will be thrown inside the inner method which may throw you off when you are trying to figure out what's wrong. It's not a big deal, especially if you own the library that is making the call, but it's just a little harder to understand what went wrong.

It doesn't contribute anything

Generics are a more advanced language construct that casting and as such is harder to understand. At the end of the day, you are getting the same result and so should be using the simplest mechanism possible.

So when should I use this?

There are cases where this can be useful - for example, when some property of the return type is used to make the method smarter. This is where you go onto the gray line. The above snippet, when presented to me, actually looked a little different:

T result = Convert.ChangeType(objectResult, typeof(T)); // Used to be T result = (T)objectResult;

This is a subtle difference, but an important one. Here we are asking the runtime to force one type unto another. To understand the difference, consider the following call (first one is the generic version and the second the non-generic):

double d = GetRegValue<double>("path");
double d = (double)GetRegValue("path");

In this case, the fancy version will actually work whereas the non-generic one will fail with an InvalidCastException. The reason behind this is simple - the API method used to fetch the value from the registry returns a boxed integer. A boxed integer can only be cast into two things - an Object or an Integer - it can't be cast to any other type. For this to work, the caller would have had to do one of the following:

double d = (double)(int)GetRegValue("path"); // Cast the object into an int and then the int into a double.
double d = Conver.ChangeType(GetRegValue("path"), typeof(double)); // Coerce the value.

Both these choices ARE indeed somewhat less readable and perhaps less maintainable than the generic alternative - I personally could go both ways on this - not married to either of them. In the case of Registry access, the values are usually known beforehand and thus you probably know exactly what you are looking for so I would probably go with an object retval.

Hitting "Session Timeout" errors when you clearly are not supposed to be hitting them

Shahar — Tue, 18 Mar 2008 21:09:03 GMT

Scenario:

You are working on preparing dashboards or uploading workbooks to your SharePoint site for usage with Excel Services. Every now and again, even though you are clearly interacting with the session, EWA will suddenly tell you that the session has timed out.

Reason:

Under some circumstances, the server may decide that it needs to recycle itself or stop servicing requests targeted at a specific workbook. In those cases, the server has no way of distinguishing between a timed-out session, and invalid session (a session identified by an invalid string internally - the same thing used by Excel Web Services) or a session that died because of the reasons just cited.

Because of those reasons, the server gives the most generic session-related error it sees.

What to do:

Nine times out of ten, this occurs because a workbook is corrupt in some manner or contains something that Excel Services otherwise does not expect. Identifying the problematic workbook and then using the forums to send it (scrubbed of any private info of course) would be a great first step at trying to resolve the issue. Also, if the workbook has recently changed, it could be useful to go back to an earlier version and see what change actually caused the issue.

COM Library for Excel Web Services - Use EWS from VBA!

Shahar — Wed, 05 Mar 2008 17:00:14 GMT

In the ODC 2008, I gave a demo of how to use Excel Web Services from a VBA client - the demo was basically just a managed library that wrapped a generated Web Services proxy.

Since Microsoft no longer seems to supply a SOAP toolkit for office, this is the easiest way of achieving access to Excel Web Services.

The library gives access to all of the important methods exposed by EWS and will allow users to read data/set data and otherwise manipulate an Excel Services session. The following VBA example shows how to use the library:

Sub DoIt()

Dim Es As New EwsCom.ExcelServicesSession

' Open the workbook.

Es.Open "http://bluemonster/_vti_bin/ExcelService.asmx", "http://bluemonster/Test/Shared Documents/Wow.xlsx"

Debug.Print "Session is:" & Es.SessionId

' Get a single cell result.

Dim R

R = Es.GetCell("Range2", True)

Debug.Print "GetCell of Range2 got:" & R

' Set a single cell back.

Es.SetCell "Range2", "My New Value"

' Get a range and fill the workbook with the results gotten from it.

Dim MyRange As EwsCom.ExcelServicesRange

Set MyRange = Es.GetRange("Range1", False)

For Row = 0 To MyRange.RowCount - 1

For Column = 0 To MyRange.ColumnCount - 1

ActiveCell.Offset(Row, Column).Value2 = MyRange(Column, Row)

' Save the session workbook into a local file.

Es.SaveWorkbook WorkbookType_FullWorkbook, "c:\temp\MyFile.xlsx"

' Close the session.

Es.Close

End Sub

As you can see, the major difference between the way the library works and the EWS works is that the Session ID is wrapped so that it doesn't need to be used over and over again - as well as that annoying status array that's usually ignored.

I have published the library on MSDN Code Samples - it's called EwsCom - enjoy.

Chapter of the 'Professional Excel Services' book is on MSDN.

Shahar — Wed, 05 Mar 2008 00:49:42 GMT

MSDN has just published a chapter from my book "Professional Excel Services". You can find it on this link.

This wouldn't happened without the great work of Siew Moi Kohr. Thanks Siew!

Single-select Dimension member Analysis Services Filter Web Part

Shahar — Sun, 02 Mar 2008 05:09:00 GMT

A long long time ago, in a building far far away, two ~~Jedis~~ developers called Sreepada Santhegudda and Sergei Gundorov showed me a solution they came up with for a single-select Analysis Services filter web part.

While SharePoint ships with a set of filters that allow users to build interesting dashboards, and while one of them allows you to select dimension members for filtering, the solution is by no means a one-size-fits-all solution. Apart from the issue of having a somewhat convoluted way of doing single select (you essentially need to un-select all elements one way or another and then select the item you want - and then, when you want to switch from one member to another, you need to remember to un-select the previously selected one).

They have sent me the solution eons ago and one way or another, I kept procrastinating and never posted it - Mike Reese prodded me enough that I felt totally ashamed of myself and so I finally got off my fat behind and posted it. Anyway - here's the solution, including source code, for your usage. They also have a document that explain usage and installation.

Here's a screen-shot that shows the simplified selection mechanism:

I have created an MSDN Code Sample library and have uploaded it to there. Enjoy!!!