Bill Li

Move

MSDNArchive — Wed, 29 Jul 2009 07:20:00 GMT

My blog has moved to http://www.BaliOnWeb.com. Most of articles are also moved to new site. This is retired at this moment.

My PM talk in SJTU

MSDNArchive — Mon, 01 Jun 2009 18:44:00 GMT

Finally, I delivered my first talk. Kind of excited, you can imagine. :-)

PPT can be found @

http://cid-0c5963e0b1a0be5d.skydrive.live.com/self.aspx/Public/PM%20Talk.ppt

Let me know what you think of it.

Update 7/30/2009

This post has been moved to http://www.balionweb.com/my-pm-talk-in-sjtu.html

If I were designing a new email service, I would…

MSDNArchive — Wed, 11 Mar 2009 13:49:00 GMT

All great designs come from deep understanding to customers. In my case, I'd like to design the email service for information workers(IW) as I am one of them. Basically they are hired to get things done. Modern projects, or tasks in smaller granularity, are getting too complex to be accomplished individually. So people have to work together – we call it collaboration. Consequently they have to communicate. They are forced to communicate. Email solves the problem of logistics and synchronization so that communication could happen between different time frames and locations, so it is still indispensable currently. But it is far from perfect, sometimes it is a trouble. Can it be done any better? Let us have a try. This new thing is called Pmail.

Entities

First of all, let us keep in mind that tasks are actually what IWs really care about. They are much happier if they can complete assigned task without touching emails. Right? Completion of tasks is what they are trying to achieve. Second, when they have to communicate, they care about distilled information they are expecting while writing/reading emails. You will not get any additional credit through presenting your idea by writing a poem. Senders and recipients come last. Without them, communication can't happen. These two entities get lower priorities because if distilled information can be gained by other means, say search engine or anonymous DL, who cares them?

Again - email for IWs is all about tasks, not messages. This is the most fundamental philosophy differentiating Pmail's design from others.

Problems - an abused tool in workplace

By nature, current email systems allow one sends anything to any number of persons at anytime anywhere. Gradually it turns out true that something that can be anything can do nothing actually. Theoretically any email should be of your interest; otherwise sender will not send it to you. Realistically it is so easy to be over-used:

Email overload. You might probably find everyone around you is complaining about too many emails. Doubt? Open you email clients, see how many unread mails you have. NY Times reports, E-MAIL has become the bane of some people's professional lives. This occurs likely because of complicated job nature, but I'd say often it is actually because of poor email prioritization. Can you easily tell which email is much more important than others? Or emails tell its priority to you?
Hard to map emails to one's day-to-day jobs. How many times you have to search your emails to dig certain messages out because you need something?
Mail thread discussions often go wild. Do you find you are in trouble figuring out what is going on when suddenly looped in a thread?

At a glance

Pmail's primary design goal is to help team get jobs done efficiently – under right timing, priorities, order and resources. Everyone would have a clear picture about how his work contributes to team success.

Pmail would also aim to minimize communications as much as possible. Leave users alone please. In task-centered email design, all emails will fall into one of below four categories:

Category	Description	Example	Your Action
FYI	There is no immediate impact to any tasks in your plate at this moment	High level org changes announcement Knowledge sharing News letter	No – No immediately action required
Catch the ball	Someone requires you do something to unblock his task(s)	Sign Off request Fix broken printer Mandatory code review for check in	Action required – Do something and then back to requester with results
Here you go	Someone provides you what your task(s) require	Approval letter Team member phone number collection table	Action unblocked – You are ready to go complete specific task(s) if all dependencies are resolved
Collective efforts	Someone needs your inputs for task(s) he is working on	Execution plan review Brainstorming	Best efforts – do what you can to provide inputs

Persona/User scenarios

Key scenarios that Pmail addresses include:

Know big picture for better prioritization – Steven, who is a developer in a software development team, came to his office in a morning. He opens bird view of pmail to check how his tasks (or WBS in more general term) fits into the entire team progress. He notices one of his tasks, task8, is in critical path according to updated plan(never expect plan is really locked after lock-down). This task is automatically prioritized to p0(highest priority) by Pmail. It is about implementing a feature according to design specification by feature PM.
Maintain dependencies – Steven goes ahead to open task8 and find this task depends on two resources, 1) one LHS virtual machine, which has been completed by task7. VM address and credential are also attached. Nice! 2) PM spec. It is also claimed completed.
Main email threads around tasks –Steven opened the spec document in team Sharepoint server, and he finds several points are not clear for him. Then Steven starts a mail conversation with relative PM, Joanne, against this task. The mail thread is linked to task8 and its status is open.
Drive team work flow – Joanne received a catch-the-ball mail from Steven about p0 task8, so it is a shadow p0 task for her as well. Joanne can check task details in the Pmail. She quickly comes up with answers regarding Steven's questions and writes back a here-you-go email to Steven. Steven receives the email and get unblocked. Steven closed the email thread by several sentences. This mail thread is traceable in task8 and can't be replied any more as it is closed. Several hours later, Steven completes task8 and closes it in Pmail. When Steven re-visits the bird view page of Pmail, he finds task8's color turns green due to status change. Steven then picks up another one unblocked taks, task9.
Maintain team discussion – Task9 is about designing a new feature F9. Deliverable is reviewable dev design spec. Steven has two options about solving a technical problem, but not sure which way to go. He then sends a collective-effort discussion mail to the whole team. Since this is not a catch-the-ball mail, team mates will treat it with best efforts, but Steven still gets several great feedbacks. When he feels the problem is solved, he summarizes the thread with several sentences and closes the thread.
Check history – One year later, Steven transfers to another team and his replacement, Eric, would like to better understand why F9 is designed this way. He opened Task9 in Pmail, check the mail thread and better understand original design decision.

Demo/UI Mock-up

Bird view will look like above diagram. In following task view, you can see all mail threads about one task are at your finger tips.

Mail view will also a bit different. Every mail has a TaskID field to help you get the whole context conveniently.

Key features in Pmail

In addition to biggest differences we covered before, you might also get excited when seeing below features.

Sometimes feature-rich is not a good thing. It is so easy to reach this point when email system is designed for multiple purposes after several releases, say home usage included. But in any event, Pmail will provide a button to show/hide features(say, increase/decrease indent) which I don't use in last month. Get me a simple world.
Mouse hover will show abstract of email(user could assign, or automatically select) before deciding to read it
Send me SMS notification when a selected critical thread gets new replies. No special carrier service needed, input my cell phone and go
For discussion mails, send brainstorming results or decision made, and then close the thread by marking it as un-reply-able
Enforce sequential replies, for example fill in a excel table
Automatically memorize folder or url of files you are editing, and go to Pmail, it can be attached to the email by a hot key, say "ctrl+shift+v"
mailURL(say, steven-task9-show-me-money-88; don't use GUID, please) which can be shared across the clients. Don't to forward again
Automatically send out ping for dead catch-the-ball emails after pre-configured interval times out
Play video/audio right inside the mail client
Mail web client also provides web API like Facebook. OWA kills programmability.
Tag everywhere – one mail thread might relate to more than one task. It is not a good idea to put a mail in one folder exclusively.
Don't allow send active documents to team for review, put it on shared place(say, Sharepoint) and send a link

I am interested in your thoughs on it. Let me know. Thank you.

Building Global Development Team

MSDNArchive — Sat, 07 Mar 2009 12:58:00 GMT

Nowadays software is getting so complex that it needs incredibly more and more people to build it. For example, there are 9000 engineers working on Vista simultaneously. In certain sense, you can call that it is a labor-intensive industry. Ideally, it would be best to put all people in one place; however there are also lots of sound reasons to build global development teams.

Pros and Cons

Why bother to build global development team?

New talents pool – Software development needs so many persons with similar attributes. In Microsoft Company wide, top 3 hiring criteria are smart, passion for technology and fit to company values such as openness, continual self-improvement and mutual respect, etc. From various studies, supply of talented IT staffers isn't keeping up with demand. And it won't change anytime soon. Even when Microsoft is cutting 5000 jobs now, we still have another 2000~3000 job openings there. What to do if enough qualified hires can be not found within US? Go outside.
Lower cost – let us straightly go to data. Annual pay for Level "59" is about $74,000 in US, while same level is paid about $20,000(~RMB150K equally) in China. Another slight fact - It costs about $1.5 to go most common places in Shanghai…
Close to market – Most innovative ideas often comes out of interactions with customers. And customers' requirements vary from country to country. So best way to serve a local market is to be in the market.
Specific knowledge – Cross-nation acquisition for specific technologies is another reason to build global development. You can't(or not able to) move all folks to HQ.

Like many other things, challenges always go with benefits hand in hand. You can't take only part of it. Fair enough.

Distance – People who work on the same software can't work alone, they have to exchange information and make decisions. Less than 3 minutes conversation is often enough for slight but frequently arising communication needs, for example, "Could you show me the bug in your environment?" Distance makes it impossible to stop by one's office. Network bandwidth is another issue introduced by distance. Why it matters? Suppose one team has to copy large file sets from another remote team daily, it would be a problem. The file sets can be growing amazingly big – for example, daily SQL Server build is about 300GB.
Time zone – Would communication a real headache since modern technologies such as email and telephone have been there for quite a long time? But the things are no one is there when you are working due to time zone. You have to wait another day to get an email response. When you rush to office to check it, the most frustrating but concise reply could be - "What do you mean by <insert anything you assume others should understand>? How can I help you?"
Culture/Language – Master level of English is unbalanced in global teams. There are dozens of ways to say "A beats B" in English, but only several of them are understandable for general public; dialect is another obstacle. For example, Chinese folks often have hard time pronouncing letter "L", as a result, words like "girl" might sound weird sometimes.
Junior team – In my organization, most hires right come out of college. We are raw smart, but less experienced. It is hard to deny that some key ingredients for great engineers just take time – design skills, debugging techniques, influencing capabilities. This is not a cutting-corner example.
Conflicts between hierarchical management and local branding – By organization hierarchy definition, many business units have existence in China. STB, Live, Online Service, etc. They functions nearly independently of each other and reports to US. But from the perspective of customers/partners/talents, it looks a bit confusing because there are so many Microsoft's.
People development – folks in remote sites are not equally exposed to development resources such as face-to-face training, library, mentors, etc.
Governing issue – due to well known concerns, something confidential can't be moved out of US.

How-to

There are several factors playing critical role in deciding appropriate distributed development model. At least you should consider project type, team seniority, team size, team culture, communication cost and history.

Build trust first. No matter how adaptable a new team is, it always takes time to fit into a specific team culture. So starting with easy jobs to build team moral and credibility is the safest steps before moving forward. Beyond this, the new team can target increased ownership.
Decide on team coupling level. From highest to lowest, the level could be: pseudo random assignment, same branch same feature/component, same branch different feature/component, different sub branch but same main branch, different main branch & clearly defined data contract. Avoid circular dependency.
Come up with communication plan. Minimized communication is not always the best, especially for relatively junior teams. The best way to develop people is to work with senior folks on daily basis as much as possible. So you have to make tradeoffs here based on various inputs.
Shared lab if possible. If you have to copy large files across the ocean frequently, consider doing your job on a lab down in remote team. If you have to do copying, make sure the files meet most critical quality requirements before doing so, for example, a build verification test.
People development plan. Regular staff exchanging plan such as Marco Polo and Silk Road program, getting a mentor, record trainings, etc.
Local brand management. Externally shown as one image; internally run by functionality as usual.

#3, Hulu, Why?

MSDNArchive — Mon, 23 Feb 2009 11:10:00 GMT

Fast Company recently published its version of the world’s top 50 most innovative companies. Although I would question why Intel is among top10, what surprised me most is the fact that Hulu is listed #3. I know there might be political things about the particular order, as it goes with most ranking, but it would be also interesting to find out “why Hulu, not others”.

Origin

Hulu, an online video streaming company, managed to do something which YouTube failed to do. Copyright is one of YouTube’s headaches, but it is incredible positive thing for Hulu, because it is built intentionally aimed to server property content by two major stream media dogs, NBC Universal and Fox.

{YouTube, watch someone’s DIY video} VS { Hulu, watch TV & Movies online legally} -> Similar but different market niche.

People might be asking, why NBC and Fox executives don’t rely on YouTube to serve their plays? You can imagine following conversation which probably happened behind the scene:

NBC/Fox:	Hey, YouTube, are you interested in serving my video to the world?
YouTube:	Why not. Let’s sit down and take a look at this. Now we are owned by Google, and we are overwhelming dominator in online video market. We have great brand. We have great infrastructure. We have most talented engineers. Blabla… (down to the point) so you have to pay x dollar for every minute show.
NBC/Fox(think):	uh-um… let me do some math here. Plan A is to work with YouTube, plan B is to build up something myself. In next 3-5 years, if everything goes as predicted, plan B will bring much more money to our shareholders than plan A, and less risky.
YouTube(ping NBC/Fox):	What do you think of the plan?
NBC/Fox:	Nice plan, but no, thanks.

This is the first site who delivers property video to your computer for free. Traditional Media Company gradually realized that they have to embrace the changes if they are not able to prevent them. It is online streaming, in this case. Hulu has more than 120 sources now.

Key is property content sources are nonrenewable rare resources.

Independence

Hulu’s CEO said to capital angels, “I don't think you'll be seeing the name Fox or NBC on the site hardly at all, Hulu is about the shows, not the networks. The shows are the brands that users care about." Another quote, “the key to Hulu's success is its freedom to operate essentially as a stand-alone company…”

From startup’s perspective, captical can be good thing, or bad thing. It can help you grow much faster, but it can also easily enable you miss your initial goals. Capital often appoints some seemingly smart guy, who is with XYZ MBA degree or n years of experience in ABC company, to take over the company as one of its investment agreements.

That is indeed one of the worst investment risk controls, although it happens again and again. Give money to most passionate guys, and letting them be passionate always is the only way to maximize the probability of getting most out of your investment.

~~Feature?Solution?~~ Experience!

Question: If you are given a task to build a video streaming site within less than 3 months, what would you do?

We were taught this way:

1) Identify who will be using your site

2) Draw use case diagram

3) List scenarios for each user role

4) To support each scenarios, figure out needed features

5) Design/Code/test your features

6) Go live

7) Yeah! Party! :- )

If you follow this in your next interview, I can almost guarantee a pass. Do we miss anything? Actually we missed most critical one – Experience! Experience is a combination of brand/feeling/easy-to-use/enjoyable process. For example, given below requirement:

"Design something which is used to sit on, commonly for use by one person. It often has the seat raised above floor level, supported by legs."

People will respond immediately, "Chair!". You may notice that there are at least hundreds of types of chairs in the world, if not thousands of, if not millions of. Only most imaginative ones who deeply understand that particular set of users’ needs, care about their feelings and eventually apply those into product designs can do the best work. Let us take a look at "art of chair".

Be COOL, in show time, although you might have the similar hard time figuring out what some of them really are. :-) Just like someone said about iTune – “iTune is not selling features. iTune is selling experience.”

Hulu’s key experiences:

1) Simple

2) Larger screen

3) High-resolution video

4) Clutter-free

5) Quality control

6) Free to users

7) No download

8) Obsessed with users

Lead Without Authority

MSDNArchive — Fri, 20 Feb 2009 10:56:00 GMT

As to the complexity level of problems, developers in Microsoft, who deal with coding, are probably taking one of the most challenging jobs in the world. But another engineering role, program manager, is not easier in any sense. Before I explain why, let me illustrate what typical product teams look like.

In the diagram, group manager often runs a team owning individual components or feature areas, say, one spell checker DLL in office; a product unit manager owns a product, say Outlook; while a general manager runs a product family, say Office. One compelling advantage Microsoft possesses is that its products are designed to work together seamlessly. For example, in a recent CIO.com article, 10 Reasons to Use Microsoft Outlook for Your Company's E-Mail, 8 of 10 points is actually about interoperability with other software.

Managers define strategy, Developers write code, testers test code, what do PMs do? Well, it is really hard to define - there is even no consensus within Microsoft. However, one thing is clear – PMs do anything except coding(unless they have to :-) to drive projects meeting strategic goals. Enable right things get done right. Figure out gaps and fill them. The role is something like concrete among brick walls. (cartoons on the right)

If you take a look at PM's position, blocks with black bold border, in the organization hierarchy, you will get the challenges here. PMs are supposed to lead their peers to work towards the right direction.

"You are not my boss. Why do I have to listen to you?"

This question gets down the topic of "Lead without Authority". Before you read along, I have to clarify that I am not a PM expert, but I (0) I've been a PM for quite a while, (1) work with great PMs, (2) learn and try most fundamental things which work well, (3) I faced similar situations for many many times, either in schools, or various organizations. Here is my sharing.

Charisma. You might be incredibly raw smart – you can optimize an algorithm from O(N*N*N) down to O(logN), or you might be able to design a system of .9999 availability, extremely secure, and super cheap compared with competitors… That doesn't mean you can be a good leader. Instead, many of the best leaders, however, will point to the fact that they were "C" students(From Source).
Credibility. It is the key to make people follow you. It works like stock value – hard to build, easy to lose. Be careful of it always. Do right things right. Bring something valuable to the team. Your credibility will accumulate gradually. Do what you say, and say what you mean.
Consistency. People hated to be randomized. Don't always change directions. Clear goal.
Partnership mentality. Low ego.
Help others great. Most influences are indirect. People often complain about "over-managed, but less supported". (Check out photo on the right)
Lead by showing, not lead by talking only. Get your hands a bit dirty. Leaders should be willing to do anything they tell their reports to do. Show domo, mock UI, prototypes to other when presenting a new way to do things.
Has sense of people. Most basic thing is How to Win Friends and Influence People. Learn to inspire people. Always be positive in every aspect.
Communicate. Communicate. Communicate. Listen. Ask questions. You can be introverts, but have to be proactive. Be transparent. Understand your audience – strength, weakness, style, and figure out most effective approaches.
Relationships enhance communication. Invest time in talking to people interactively. When you need to request advice, an opinion, or a task, you can talk to people from a healthy and positive place.
Sharpen your vision. Look around. Look ahead. This helps in making sound decisions.
Start to drive teams with strong pseudo leaders
- Talk to them especially leaders or submarine leaders, get to know what are their likes/dislikes, Ask: How can I help you guys? Win their support.
- Build credibility. Bring something invaluable to the team, for example, build up strong relationships with a partner team, present a competitor research report, improve team visibility
- Before making changes, present it to the leaders in private, say 1on1, to reach rough consensus before it goes public.
- Help others great. Let them get the credit.
Conflict Management
- Focus on goals, not conflicts themselves. Not a question of smart or dumb.
- If you can control your emotion well enough, 25% of conflicts will go away.
- Overwhelming majority of conflicts is just because of perspectives and different values. Try to figure out stories of other people's side. People often come to the same solution when presented same set of information. Use negotiation tips. Keep win-win in mind.
- In worst case – when you have to resolve unavoidable conflicts, say 5 pies for 10 kids, make sure put a fair decision-making system in place, which is often a engineering way.
You can reward desired performance even without authority
- Promotions, raises, bonuses are not in your list, but you can affect them in indirect ways
- Fame. Complimenting an employee on a job well done at a staff meeting or in front of company officers can be extremely rewarding
- Increased trust. For example, give top performers the freedom to work in a more self-directed manner.

My AD Fun Experiment

MSDNArchive — Wed, 18 Feb 2009 18:28:00 GMT

Today I got a mail from lakequincy.com, saying:

“Hi Bill,

I noticed that you were never able to plug the Lake Quincy Media ad tags into your site. Are you still interested in earning revenue from displaying ads targeted to Microsoft developers? If so, you can get your tags from here: <certain link>

Let me know if you have any questions or if you’ve decided against running the ads and I’ll set your account to inactive for you.”

I even forgot that I resgistered in their site. To encourage such great customer service, I go ahead to try how the AD really works for me; finally decide to put a small square in the side bar. You can easily find it now if you scroll down a bit and pay attention to the left side bar. Let us see how many dollars I can make after one or two quarters. By putting an AD to the blog, it looks more of-the-business, doesn’t it? Of course, you can bid for that AD position. :- )

Update(2/26/2009)

The AD is disabled temporarily due to security alert.

P2P Backup System w/o SPOF for Work Group

MSDNArchive — Sun, 15 Feb 2009 18:23:00 GMT

This is also one of my half-completed ideas years ago. It was recalled recently by two stories:

Stories

#1: One of my team mates lost his Outlook email archive due to a mistaken operation. He is very upset because all of his emails in past two years are gone, unrecoverablly. Suggested Outlook mail file is 2GB, but people can easily drive it up to 10GB and often notice this too late. As what he said in MSN signature, “my archive, 555…”

#2: While I was browsing one of my favorite bloggers – Brad Abrams’s blog, ran into a shocking post, Help! My hard disk crashed. He can afford a new disk, but he need the data – especially “the pictures of the kids recent birthday party”. In the comments, people suggested various ways to recover it, unfortunately looks like no luck.

This leads to the most basic questions: why people don’t backup data even if they know they are in the risk of losing them? Probably because of:

1) The chance is very tiny

2) Current backup solutions are not “good enough”

For 1), although it might be true, but remember the result might be cataclysmic. Let us take a look at what is going on about 2).

Existing Backup Solutions

Traditional C/S – Most of current backup solutions are based on traditional C/S architecture. Assume one use 40% of 500GB disk drive on average; it is still a big capacity for enterprise with thousands of staff. Backup time is also another concern. Let me illustrate this with some math. For a backup server with two 1-Gbit NICs, its max throughput is about 200MB. If backup window(people are sleeping) is 10 hours, it means it can back up about 7.2TB(2*100*3600*10MB) data. Obviously it is hardly scalable along with business expansion.

Online backup – People can turn to AWS S3, or Azure SQL Data Services to simplify things on their own side. But this doesn’t make things even better if you look at current network speed. Calculating data is much easier than moving data around.

My P2P Backup System

p2p backup comes with several inherent advantages in handy:

1) No additional hardware expense. People often have at least 30% capacity left; we can leverage that for backup space.

2) No obvious network bottleneck.

Some people might also list “no single point of failure”, but that is not true. Let us take a look at how BT works, a general P2P network.

Basically here are what happens in P2P network - peers need talk to a central coordination server(i.e., Tracker in BT system) to get info about other peers, and then talk to individual peers about real data exchange. These two steps are essential. Single Point of Failure(SPOF) can happen in the coordination server. You may argue that you can enhance its reliability by techniques like mirroring, but does it look like over-heavyweight? We have many reliable services running by full time IT guys, why do we bother to re-invent the wheel? Several services can be utilized, such as AD, Sharepoint, exchange, which allow customized data writing. Data needed written into a central place would be:

1) Peer list by <IP, port>

2) Other meta data such as backup network name, Software RAID level

Of course, we need do compression, encryption, incremental backup, scheduled backup in the solution. I may do investigation later and update the post.

Also see other backup solutions

http://www.storegrid.com/index.html - select a folder, then backup, traditional

www.streamload.com - Large net disk, Upload your files to streamload, then view it, or email it to someone else.

http://www.beinsync.com/ - Access you computer: install a mini PHP web server onto it.

http://base.google.com/base/about.html - In about 15 minutes, your item will have a unique web address and be visible to the world.

http://www.openomy.com/ - openomy is an online file storage system designed to be a platform for Web 2.0 applications, built by Ian Sefferman

Memory Leaks Demo & Detection in .NET Application

MSDNArchive — Wed, 11 Feb 2009 07:32:00 GMT

Memory leaks are always headache of developers. Do .NET developers no longer bother to worry about memory leaks because of garbage collection? Yes and NO. GC periodically find objects that cannot be accessed in the future and then reclaim the resources used by the objects. GC achieves this by maintaining a list of references to live objects. When this mechanism is broken, memory leak happens.

There are many reasons to leak memory. In addition to calling unmanaged code from managed code, another one of general cases is about event handler. If you do this:

Foo.FooEvent += new EventHandler(MemoryLeaksHere.Method);

When you complete using MemoryLeaksHere, but you are still using Foo, then MemoryLeaksHere will still remain alive as well. MemoryLeaksHere object will leak memory as a result of failing to GC.

Let us take a look at one simple example first.

using System;

namespace MemoryLeakSample

{

class Foo

{

public static Foo myFoo;

public event EventHandler FooEvent;

public Foo()

{

myFoo = this;

}

public void FooMethod()

{

MemoryLeaksHere memLeak = new MemoryLeaksHere();

memLeak.TryQuit();

}

public void FireEvent()

{

FooEvent(null, null);

}

static void Main(string[] args)

{

Foo foo = new Foo();

for (int i = 0; i < 5; ++i)

{

foo.FooMethod();

}

GC.Collect();

GC.WaitForPendingFinalizers();

GC.Collect();

Console.WriteLine("Check memory leak here.");

}

/// <summary>

/// This object will cause memory leak

/// </summary>

public class MemoryLeaksHere

{

public MemoryLeaksHere()

{

Foo.myFoo.FooEvent += new EventHandler(OnMyFooEventFired);

Console.WriteLine("\nObject-{0}: Construct. Subscribe.", this.GetHashCode());

}

~MemoryLeaksHere()

{

Console.WriteLine("Object-{0}: Deconstruct.", this.GetHashCode());

}

public void TryQuit()

{

Console.Write("Object-{0}: leak me?", this.GetHashCode());

string input = Console.ReadLine();

if (string.Equals(input, "no"))

{

Foo.myFoo.FooEvent -= new EventHandler(OnMyFooEventFired);

Console.WriteLine("Object-{0}: Unsubscribe.", this.GetHashCode());

}

else

{

Console.WriteLine("Object-{0}: Not Unsubscribe", this.GetHashCode());

}

private void OnMyFooEventFired(object sender, EventArgs e)

{

// Do something

}

In MemoryLeaksHere object’s constructor, Foo starts to hold a reference to MemoryLeaksHere by registering event handler. In MemoryLeaksHere.TryQuit(), if we don't unregister, memory leak will happen.

To be more intuitive, you can copy/paste sample code to VS2008, and then enable unmanged code debugging by following:

Project->Properties->Debug->Enable Unmanaged Code debugging

Now set a breakpoint at “Check memory leak here”, and start build/debug. When being asked leak me or not, you can choose either yes or no. For example:

Here, looks like we leak two of them. Finally app will hit the breakpoint and stop. At this point, we can go to VS immedate window to load sos.dll, and then check how many objects in the heap:

!load sos.dll

extension C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\sos.dll loaded

!dumpheap -type MemoryLeaksHere

PDB symbol for mscorwks.dll not loaded

Address MT Size

0132e7d0 00983104 12

0132eba0 00983104 12

total 2 objects

Statistics:

MT Count TotalSize Class Name

00983104 2 24 MemoryLeakSample.MemoryLeaksHere

Total 2 objects

So now we know there are two object instances are not recycled. Why are they not GC-ed? Because someone has a reference to them. Choose one of them, and use gcroot command.

!gcroot 0132e7d0

Note: Roots found on stacks may be false positives. Run "!help gcroot" for

more info.

Error during command: Warning. Extension is using a callback which Visual Studio does not implement.

Scan Thread 7592 OSTHread 1da8

ESP:12f434:Root:01312d48(MemoryLeakSample.Foo)->

0132f704(System.EventHandler)->

0132f6ec(System.Object[])->

0132e7dc(System.EventHandler)->

0132e7d0(MemoryLeakSample.MemoryLeaksHere)

Scan Thread 4704 OSTHread 1260

Now we can see that MemoryLeakSample.Foo is still referencing MemoryLeakSample.MemoryLeaksHere via event handler. If it is not 5 iterations, image what would happen if every incoming request results in a slice of memory leak... Soon or later, you online service will be down.

Designing Your Own Recent Posts Widget for MSDN Blog

MSDNArchive — Sat, 07 Feb 2009 16:42:00 GMT

In my MSDN blog, I need “Recent Posts”, but I don’t need archive side bar. After having played with template for a while, still no luck. Hmmm, looks like I have to DIY it. Fortunately in News sidebar, you can fill in raw html including JavaScript. Then next question is where we can retrieve post tiles. The immediate idea is from current DOM document. Through experiments, I found this is impossible because the DOM is not fully loaded yet when the script is executed. Later, I figured it out that all posts title can be gotten from RSS. For my blog, the address is http://blogs.msdn.com/bali_msft/rss.xml. One thing worth noticing is the fact that RSS in MSDN blog is not up to date - Your post will not instantly appear in the RSS. After I get all posts in RSS format, things became much easier. And then I go ahead to add more interesting things:

Posts background use two colors in turn
Show a new tag for posts less than 3 days old
Show latest 8 posts only
Show posts' age

So, the final thing will look like this:

If you find it is useful, feel free to paste below code to you blog’s news section. Note to customize “Configurable params” to your own needs and leave other code intact. It works well at least in my IE 8 and Firefox 3.0.6.

// Configurable params

var recentPostNumber = 8;

var rssUrl = "http://blogs.msdn.com/bali_msft/rss.xml";

var title = "My Recent Posts";

var newPostAgeInHour = 72;

// Cacluate age of one post. It is all about getting time span in Javascript

// return formate: x min; x hour y min, x day y min, x days, x yeas (ago)

// Refer to: http://www.w3schools.com/jsref/jsref_obj_date.asp

function calculateAge(postDate)

{

var ret = "fresh!";

CurrentDate = new Date();

TimeSpan = new Date(CurrentDate - postDate);

var mySpanArray = new Array();

    mySpanArray[0] = TimeSpan.getUTCFullYear()-1970;
    mySpanArray[1] = TimeSpan.getUTCMonth();
    mySpanArray[2] = TimeSpan.getUTCDate()-1;
    mySpanArray[3] = TimeSpan.getUTCHours();
    mySpanArray[4] = TimeSpan.getUTCMinutes();

var TimeSpanTagArray_1 = new Array("years", "months", "days", "hours", "minutes");

var TimeSpanTagArray_2 = new Array("year", "month", "day", "hour", "minute");

// Starting from non-zero element and pick two significant values

for(i = 0; i < mySpanArray.length; i++) {

if(mySpanArray[i] != 0) {

var correctTag = (mySpanArray[i] == 1)?(TimeSpanTagArray_2[i]):(TimeSpanTagArray_1[i]);

ret = mySpanArray[i] + " " + correctTag;

if(i+1 < mySpanArray.length && mySpanArray[i+1] != 0) {

correctTag = (mySpanArray[i+1] == 1)?(TimeSpanTagArray_2[i+1]):(TimeSpanTagArray_1[i+1]);

ret = ret + ", " + mySpanArray[i+1] + " " + correctTag;

}

break;

}

return ret;

}

// Display the recent posts

// Refer to:

// http://www.w3schools.com/DOM/dom_node.asp

// http://www.w3schools.com/DOM/dom_methods.asp

function displayPosts (xmldoc)

{

var newTag = "<SPAN style=\"COLOR: red\">(New!)</SPAN>";

var posts = xmldoc.getElementsByTagName("item");

var displayText = "<h3>" + title + "</h3><UL>";

if (posts.length < recentPostNumber) {

recentPostNumber = posts.length;

}

for(var i = 0; i < recentPostNumber; i++)

{

PostTitle = posts[i].firstChild.firstChild.nodeValue;

PostLink = posts[i].firstChild.nextSibling.firstChild.nodeValue;

PostDateStr = posts[i].firstChild.nextSibling.nextSibling.firstChild.nodeValue;

PostDate = new Date(PostDateStr);

CurrentDate = new Date();

// Calculate age

var _PostAge = calculateAge(PostDate);

var PostAge = "<SPAN style=\"font-size: 80%; color: black\"> (" +_PostAge + " ago)</SPAN>";

// Show a new tag for posts happening last days defined by 'newPostAgeInHour'

var myNewTag = "";

if((CurrentDate.getTime() - PostDate.getTime())/1000/60 < newPostAgeInHour * 60) {

myNewTag = newTag;

}

// Get background color

var BKColor = (i%2 == 0)?("#B8CCE4"):("#DBE5F1");

displayText = displayText + "<LI style=\"background-color:" + BKColor + "\"><A href=\"" + PostLink + "\">" + myNewTag + PostTitle + PostAge + "</A></LI>"

}

displayText = displayText + "</UL>";

var target = document.getElementById("RecentPosts");

target.innerHTML=displayText;

}

// Call back

function complete(){

if (req.readyState == 4) {

　　if (req.status == 200) {

displayPosts (req.responseXML);

}

// Initial async call

function getPosts()

{

　　if (window.XMLHttpRequest) {

　　req = new XMLHttpRequest();

　　}else if (window.ActiveXObject) {

　　req = new ActiveXObject("Microsoft.XMLHTTP");

　　}

　　if(req){

　　req.open("GET", rssUrl, true);

　　req.onreadystatechange = complete;

　　req.send(null);

　　}

// Entry point

getPosts();

</Script>

Searching For a Number in Shifted Sorted Array within O(log(n)) time

MSDNArchive — Tue, 03 Feb 2009 18:50:00 GMT

Run into the algorithm problem long time ago. Now post my answer here. A sorted array, say: {1,2,3,4,5,6,7,8,9,10,11,12}, do right rotate through carry unknown times, and then it might become: {6,7,8,9,10,11,12,1,2,3,4,5}. Now we need get the index of a given number, say 4, from the array within O(log(n)) time. Apparently a 8-year-old can get it done with O(n) time.

We can think of it this way: take the middle element of array, if target is found, fine; if not, and then array become two parts, one is sorted array, the other is shifted sorted array. As illustrated as below diagram:

If the target falls into the sorted array half, we can simple do a binary search; otherwise, repeat this operation in the other half in recursive way. You can see this is divide-and-conquer algorithm. Obviously this is O(log(n)).

Code

// A typical binary search implementation

int _BinarySearch(unsigned int ShiftedArray[], unsigned int start, unsigned int end, unsigned int target)

{

// Not found

if( start == end && ShiftedArray[start] != target) {

return -1;

}

unsigned int middle = start + (end - start)/2;

if(target == ShiftedArray[middle])

{

return middle;

} else if (target > ShiftedArray[middle]) {

return _BinarySearch(ShiftedArray, middle + 1, end, target);

} else {

return _BinarySearch(ShiftedArray, start, middle - 1, target);

}

// Select a given number from shifted array.

// ShiftedArray is something like = {6,7,8,9,10,11,12,1,2,3,4,5}

// If found, return index of the number; if not, reutrn -1

// Require log(N)

int SearchShiftedArray(unsigned int ShiftedArray[], unsigned int start, unsigned int end, unsigned int target)

{

// Start meets end

if( start == end && ShiftedArray[start] != target) {

return -1;

}

unsigned int middle = start + (end - start)/2;

if(target == ShiftedArray[middle])

{

return middle;

} else if(ShiftedArray[middle] < ShiftedArray[start]) { // Right half is sorted linearly

if((target > ShiftedArray[middle]) && (target <= ShiftedArray[end])) {

return _BinarySearch(ShiftedArray, middle + 1, end, target);

} else {

return SearchShiftedArray(ShiftedArray, start, middle-1, target);

}

} else { // Left half is sorted linearly

if((target >= ShiftedArray[start]) && (target < ShiftedArray[middle])) {

return _BinarySearch(ShiftedArray, start, middle - 1, target);

} else {

return SearchShiftedArray(ShiftedArray, middle + 1, end, target);

}

Test cases

Positive: {6,7,8,9,10,11,12,1,2,3,4,5}, target = 3, target = 8

Negative: {6,7,8,9,10,11,12,1,2,3,4,5}, target = 0, target = 13

Boundary: {6,7,8,9,10,11,12,1,2,3,4,5}, target = 6, target = 5

Exceptional: {…max}, target = max

One more interesting thing is the statement that “only about 10 percent of the professional programmers implemented binary search correctly.” Do you know why? Check this.

To Next Cuil

MSDNArchive — Mon, 02 Feb 2009 11:02:00 GMT

Cuil, another so-called Google killer, is at its last gasp. I just knew it. I am not predicting present. Cuil is not the first one, and apparently not the last. For upcoming cuils, here are my words.

Brand. Brand. Brand.

For many people, word of Google has close sentimental connection with bunch of splendid words such as cool, innovation, unselfish, impartial, revolution, and powerful, etc… With brand, Google claims that “People don’t work at Google for the money. They work at Google because they want to change the world!”. With brand, debut of Google’s every new service always arouses buzzes, but seldom notices that Live also has compelling equivalence. With brand, people think only Google can provide best results, but often they can’t tell who is search provider when presented anonymous results set. It is very interesting to take a look at curve of Cuil’s daily unique visitors:

At launch momentum, people rushed to see what this Google killer looks like because of Google’s brand. Ridiculous? Not actually. It is everyone’s inherent attributes as people love to check out events of small probability such as Shoes thrown at Bush, one crazy million-dollar idea. As part of branding strategy, naming is essential. Cuil might not a good name actually. Let me share a story of mine. Back to several years ago, a group of my friends decided to build a website aimed to provide 3^rd service for franchising, called JiaMeng in Chinese. The guys with solid academic management background came with the domain name of 51franchise.com. It turned out a real trouble – hard to explain to customers, not localized. Even ordinary college students don’t know the word franchise, not to mention clients with much less schooling. So, ditu.live.com for Chinese is much better than chinamap.live.com if you take a look at average education level of internet users. All in all, BRAND works like religion, and it takes lifetime to build.

prouducts" border=1 alt="Brand -> prouducts" src="http://lh6.ggpht.com/_A9xfBBUVmv4/SaoUf6_cqWI/AAAAAAAABg4/mtfAty4KaQk/s400/GmailBrand.JPG" width=400 height=149 mce_src="http://lh6.ggpht.com/_A9xfBBUVmv4/SaoUf6_cqWI/AAAAAAAABg4/mtfAty4KaQk/s400/GmailBrand.JPG">

"A Google approach to email" - see how brand helps product marketing.

Infrastructure

GFS. BigTable. MapReduce. They can be competitive advantages. With these put in place, Google can roll out new internet services faster, cheaper, and at scale at few others can compete with. They are designed solely for Internet services. Users quit quickly after dissatisfied performance experience in Cuil. Microsoft software is mainly for an enterprise, supporting 100K concurrent users is “good enough”, but it is far more perfect in internet scenario.

Understand/Repsect Customers

There is no one-size-fits-all solution given the growingly diversified market. Of course you can educate customers, but never expect to change their inherent attributes coming from culture/history/economic development level. If you doubt this claim, check out this article: Search site moves at the speed of China, which reports, “But appreciating such cultural differences is what Baidu.com Inc.’s chief financial officer, Shawn Wang, says gives the Chinese search giant unique insight into the country’s 1.3 billion people as it competes with American rivals such as Google Inc. and Yahoo Inc.” As a result:

Culture

Per Wikipedia, culture means the set of shared attitudes, values, goals, and practices that characterizes an institution, organization or group. Google’s business is built on top of internet, so its organization/knowledge base is built for the internet, just like Microsoft is built for software, mainly enterprise software. I met strong feature PM with deep knowledge needed for enterprise software, say reporting, admin UI, DB admin UI, and information work flow. They understand their customers so much after years of interactions with them. It takes time to accumulate. Top-down hierarchy, heavyweight development process, years of in-house development can hardly catch up with the pace of internet evolution. The same thing is applied to Google – I am equally not optimistic if Google step into enterprise software because of the same reason – culture, enterprise’s DNA.

Web Competition Strategy

What is Cuil’s selling point? (1) Fancy UI. UI is critical for adoption and usage, but it hardly provides a moat. This is provided by two case studies of Apple computer of the nineties and the “X window” system on *nix OS. Both these systems with more attractive UI couldn’t beat windows OS with lower cost and rich applications available. (2) More relevant result. This is an ambiguous area which lacks of widely accepted measure criteria. (3) Cheaper solution. There is a question of sunk cost, of course you can claim you are 1/10 cheaper once reaching Google’s current scale. None of these is compelling from users’ point of view. Why do users bother to go to your site instead? One of the significant differences between web service (say, search) and traditional software business(say, DB) is purchasing decision making process. DB vendors can send to salesmen to target customers’ office and argue the deal. Only quite a few key persons have the final call. They are more analytical, love data. As comparison, everyone can be customers of search, we are more emotional. If I don’t miss anything, looks like the best strategy to monetize Cuil is to be acquired by Google.

No chance to win in search?

Definitely No. But you are doomed to fail if following essential parts are missed:

Remember brand. Remember “winners take all”.
Build your DNA towards internet. DNA = SUM(people, team arch, process, knowledge, …)
Put infrastructure in place. This is the way to help turn your idea into profitable traffic. Not scale-up, scale-out instead.
One thumb rule to compete with dominant market leader

Avoid playing games whose rules are set by opponents. You can hardly win. In this case, better search engine defined by Google are faster, relevant results, simple UI, magic algorithm, PB of data, … Let us think of solving same problems with different approaches. Why search? Help explore and share information. If someone tries to solve this problem by following Yahoo’s tail light to build yet another portal, he has little change to take off. Another example is download - P2P technology solved the download problem without adding more expensive servers/bandwidth.
Attack opponents’ weak points. Google is designed to search everything, but it may not be good at all vertical industries, say shopping. Nibble at its market share if we can’t win in head-to-head way.

Before rolling up sleeves, why we have to win? Why not step away and go find next big thing? Let Google be Google.

Principles for Building Secure Database Applications in Action

MSDNArchive — Thu, 29 Jan 2009 18:10:00 GMT

What I am talking about in this post might be well known to many people(too simple, sometimes naive?), but often most basic things make a difference. OK, get down to business. Thumbs rules for DB security might be:

Define your security boundary(or attack surface)
All input is evil! Evaluate them with whitelist
Don't store blank password, even hard-coded in the source
Put DB in a dedicated server and access it with accounts with least privilege
Put connection string in registry and read it out from code
Use stored procedure
The attacker is told nothing
Save your resources
Specify least assembly permission requirements with attributes

FxCop is obviously a “must-have” for .NET developer, but we have to eliminate complaints one by one. Instead of remembering all “bad behavior” in various tutorials, why not make them our built-in features towards great developers? (if you are still developers, why not much better? ) Let us put most significant principles into simple sample lines of code. Pay special attention to highlighted words.

// <THIS IS UPDATED ON 2/5/2009 PER FEEDBACKS>

using System;

using System.Data;

using System.Data.SqlTypes;

using System.Data.SqlClient;

using System.Security.Principal;

using System.Security.Permissions;

using System.Text.RegularExpressions;

using System.Threading;

using System.Web;

using Microsoft.Win32;

namespace Sample

{

public class SecureDBAppSample

{

[SqlClientPermissionAttribute(SecurityAction.PermitOnly,

AllowBlankPassword = false)] // (1) Blank password is never allowed

[RegistryPermissionAttribute(SecurityAction.PermitOnly,

Read = @"HKEY_LOCAL_MACHINE\SOFTWARE\MyApp")] // (2) Can read only one specific registry key

static string GetName(string Id)

{

string Status = "Name Unknown";

try

{

// (3) Check for valid shipping ID with white list

// 4-10 digist only, anything else is bad. In most production environment,

// inputs check should be done in attack boundary instead. Of course we can check

// it here for defensive programming efforts

Regex r = new Regex(@"^\d{4,10}$");

if (!r.Match(Id).Success)

{

throw new Exception("Invalid ID");

}

// (8) Shut down connection--even on failure.

using (SqlConnection sqlConn = new SqlConnection(ConnectionString))

{

//Add shipping ID parameter.

// (4) Use a store procedure to hide the application business logic

// in case the code is compromised

string str = "sp_GetName";

// (8) Release resources--even on failure.

using (SqlCommand cmd = new SqlCommand(str, sqlConn))

{

cmd.CommandType = CommandType.StoredProcedure;

// (5) Use parameters, instead of string concatentation to build the query

// (6) Force the input to be 64 bits integer

cmd.Parameters.Add("@ID", Convert.ToInt64(Id));

cmd.Connection.Open();

Status = cmd.ExecuteScalar().ToString();

}

catch (Exception e)

{

// TODO: For better debugging purpose, we need log the exception with

// something like Logger.Log(e);

// (7) On error, the attacker is told nothing

if (HttpContext.Current.Request.UserHostAddress == "127.0.0.1")

{

Status = e.ToString();

}

else

{

Status = "Error Processing Request";

}

return Status;

}

//Get connection string.

internal static string ConnectionString

{

get

{

// (9) Store connection string in registry key intead of xml files

return (string)Registry

.LocalMachine

.OpenSubKey(@"SOFTWARE\MyApp\")

.GetValue("ConnectionString");

}

The data in registry key is the connection string.

Data Source=MyDb008; // (10) DB is on remote server.

// Compromised web service does not lead to SQL data access automatically

Integrated Security=SSPI;// (11) Use Windows authentication

Initial Catalog=client

In stead of storing plain text, we can encrypt above connetion string. Keep in mind that I don’t say that they are necessarily the best choice at all times, but many times they are.

Reference: Write Secure Code

Happy New Year of the OX!

MSDNArchive — Wed, 28 Jan 2009 07:17:00 GMT

Hi, my dear friends, Happy New Chinese Year! First of all, a small quiz to you:

Can you guess what it means? Let me explain it a bit. The pic is actually a Chinese word 福 written in a piece of square-shaped paper, and then rotate 180 degree. 福means happiness, “stand-upside-down” and “come” have the same pronunciation in Chinese. So by putting them together, the pic means my best wishes to you. Do you get it? J

During the holiday, I ran into a wiki page saying that the world cattle population is estimated to be about 995,838,000 head. How do they get that number? That is next challenge to you.

I will post my answer when I get one moment.

An AD System to Pay Content Generators

MSDNArchive — Tue, 20 Jan 2009 18:21:00 GMT

Back to not too long ago, I had a half-completed advertising idea related to social shopping. Now I post it here to collect more feedbacks. I call it HappyDog. (Just a name, not related to that DogFood widely used within Microsoft J)

Problems

As everybody knows, ‘YOU’ is named Time’s person of 2006 for the growth and influence of user-generated content on the internet. Why? In Wikipedia’s words:

“… chose the millions of anonymous contributors of user-generated content to Wikipedia, YouTube, MySpace, Facebook, Digg, Second Life, the Linux Operating System, and other providers, as Person of the Year, personified simply as You.”

But on the other hand, if we carefully think of people’s motives of generating contents we can easily find that people do this mostly out of curiosity, self-achievement or volunteerism. Problem #1: Per basic economy principles, these efforts can hardly stand long. Content quality is another downside in such circumstances.

Another awkward problem around current most successful business model, online ad, is the fact that people get tired of spammed AD when they use certain online service for free. Problem #2: But interesting enough, people often have trouble in making the right purchasing decision given even exposed to so much AD, probably because: (1) AD timing is not good, (2) they don't trust the publisher, (3) AD is not carefully targeted. This looks pretty twisted, doesn’t?

At a glance

Taking above problems into consideration, HappyDog is an innovative advertising system which provides following unique values:

For general content generator – get cash paid by sharing your experience with the world, even if you does not own a website.
For shopper – make your shopping decision more smartly
For business, either online or offline – market your product more directly to the most potential customers

Let us take a look at below diagram first:

By content generators, it could be anyone(professors, housewives, kids,...) who contribute anything(answers, e-books, songs, tutorials, ...) in any format(wiki, blog post, video,...), including but not limited to wiki & QA.

When people produce contents, HappyDog helps insert contextual AD into the content. The big difference from existing advertizing system is that content generators gain real revenue in HappyDog system. The revenue coming from advertisers would be divided into several parts: 60% to content generator, 30% to site owner, 10% to HD. HD takes the smallest piece of pie, but apparently we will gain most amount of them, because there is only single HD, 1*M site owners, 1*M*N content generators. In addtion to real $s, the incentives here can also be points/happiness, etc. The point here is to encourage ones by certain ways. But the reason why we highlight monetary incentives is the belief that this makes something long-awaited possible. Let us think about this: Sites like Wikipedia, IMDB, or Amazon, have tons of high quality content that have been contributed freely, but why only them? Because They are cheap, handy offerings to the community. How about an book written online, sold online? Few will do this free mainly because of its costly nature. HD makes it possible though. Another interesting thing is how wikipedia-clones are going in China. They never take off. It might be cultural reasons related to general finance status.

Monetary incentives can help imporve quality of content because one gets financial penalty due to "thumb down" when we put a rating system in place.

HD helps people do better purchasing decision in that HD AD is more personal, and people are encouraged to advertise the products they sincerely love and have first hand experience on. We will address this in detail next. One will be concerned to play dirty because one also has to care about its repuation/credit disclosed by rating system.

Demo

Take Yahoo answers as example, an working page from publisher side would be:

P0 features in 1^st Milestone

In order to pay content generator effectively, following features are essential in M0:

(1) Users have the rights to select their preferred AD - When you answer questions in forums; you actually care your reputation in same way you deliver a public speech. So you also care the AD itself along with your content. Another reason supporting users selectable AD is that people might like to show certain products more persuasively via real personal experience. In short, we prefer “recommend your favorites to friends” scenario.

(2) PPA(Pay Per Action) as major pricing model along with PPC – Advantages of PPA: no click fraud, advertisers’ preferences.

(3) AD type – signature text, picture, inner-text popup, mini-cast

(4) Support offline mode – Although internet users reach 0.8 billion statistically, but not every business has a website. This is especially true when people surf the net mainly for entertainment instead of business. HappyDog aims to benefit this kind of business via so called offline mode. We will take about this later.

(5) An open platform – Open API to foster a strong ecosystem around HappyDog.

How It Works

The conceptual architect for HappyDog might be looking like follows:

In real implementation, HappyDog doesn’t depend on any specific technology platform. Take general open source platform as example, related technology could be:

• Client

– HTML

– Use Javascript/Flash extensively

• Servlet container

– Tomcat/Jboss

– Spring, OR structs

• JAS(Java Application Server)

– Jboss

– Hibernate

• File Server

– Store static data, say photos, html, pic, js

• DB

– MySQL

Its key use cases would be:

Buyers can be publishers or unregistered users. By online, it means the transactions whose completion can be confirmed online, say online order, user registration, complete one survey, software download, etc. By contrast, offline means the remaining trade types, such as haircare, restaurant, face-to-face trading.

PPA Implementation

PPA, as one of the core features of HappyDog System, comes with two flavors: online and offline, as illustrated in following sequence diagrams.

Online-PPA

The steps here are:

(1) User browses and click the publisher site link somehow

(2) Browser sends request to publisher

(3) Publisher response the corresponding content along with Javascript inside

(4) Browser starts rendering the page, and then the Javascript in Browser call the HD to get AD

(5) HD returns content with targeted AD

(6) Brower completes rendering

(7) User continues browsing and click one of our AD

(8) Brower follow the link and send the request to HD along with the needed parameters

(9) HD do several things here:

a) Write “who is publisher, when, advertiser, campaign, etc” into the DB and return with a TRANSACTION_ID

b) Write TRANSACTION_ID into user’s browser cookie

c) Redirect user to advertiser website

(10)Advertiser returns the commercial pages to user browser

(11)Just show it

(12)User is interested in something in advertiser’s web site and fills in form(sales order, lead, signup, etc) and submit

(13)Submit user inputs to advertiser

(14)Advertiser will do below things:

a) Do anything necessary to close the deal

b) Read user’s browser to get TRANSACTION_ID

c) Call beacon code the confirm the transaction with HD with TRANSACTION_ID as parameter

(15)Charge advertiser and share revenue with publisher

To take part in AD promotion of this advertiser, user must give his comments about this shopping experience.

Offline-PPA

Offline mode is bit more complex.

(1) User browses and clicks the publisher site link somehow

(2) Browser sends request to publisher

(3) Publisher response the corresponding content along with Javascript inside

(4) Browser starts rendering the page, and then the Javascript in Browser call the HD to get AD

(5) HD returns content with targeted AD

(6) Brower completes rendering

(7) User continues browsing and click one of our AD

(8) Brower follow the link and send the request to HD along with the needed parameters

(9) HD do several things here:

a) Write “who is publisher, when, advertiser, campaign, etc” into the DB and return with a TRANSACTION_ID

b) Give user the advertiser’s profile and coupon which contains TRANSACTION_ID, advertiser name, promotion campaign.

(10)Just show it

(11)User is interested in advertiser’s promotion program and decide to print the coupon out

(12)Go to HD

(13)Record this as a successful transaction in DB and wait for confirmation(TTL is 2weeks)

(14)User go to the advertiser with coupon to enjoy the service or product(say haircare, dinner, etc), and go back with CONFIRMATION_CODE

(15)Provide service and a CONFIRMATION_CODE

(16)User logon to HD, submit TRANSACTION_ID and CONFIRMATION_CODE

(17)After validation, give user further kickback

A series of CONFIRMATION_CODE are issued to advertiser by HD while he/she enrolls the program. Additionally, in step(15), alternatively advertiser could logon to HD and confirm the transaction.

Open Questions

There are also several problems deserving consideration at current stage:

1. How to get advertisers? Alimama? Google AdSense is such a close system, hard to extend.

2. Looks like current publishing place are not ready for HappyDog. They often filter AD content out. How to solve this?

3. How to share revenue in Wiki scenario? Say 50 people contribute to a wiki page, how to distribute revenue among them?

4. Is the money too little to stimulate enthusiasm among high quality content contributors who are usually payed very well in regular job?

5. How to protect content, for example 1^st tutorial about jailbreaking iPhone? Share revenue?

This post is mainly for feedback collection purpose. Feel free to Pai Zhuan. :-)

Words from potential ex-customers

MSDNArchive — Tue, 30 Dec 2008 14:41:00 GMT

It is only customers who pay dollars as your revenue and enable your read this post comfortably at your office. Almost every company considers “Customer-oriented” as one of their key values although they can be expressed in various ways. It is “Passion for customers, partners, and technology” in Microsoft. This post is about two real personal experiences around customers, or ex-customers.

#1: My mother’s mobile phone

My over-55-year-old mother is of the old school, short-sighted. She has nearly zero knowledge about computer, menu, input method or any that sort of high-tech things. One day she realizes needs of contacting us from time to time in her way and decides to grab a mobile phone. Typical user scenarios of her interest can be summarized in descending order as follows:

P0:

Power On/Off – for power saving
Receive a phone call
Call her contacts(<10)
Set/Close alarm – She need prepare breakfast for family every morning
Handle recharging easily – when to charge, when to start/complete

P1(Features):

Larger UI elements – It is not convenient for her to bring glasses always
Louder sound – Her hearing is aging

And then I go ahead to bring her one of my used feature-rich phones with enough confidence. After helping input her contacts with concerns whether she can ever figure out where to start, I start demo-ing how to use the phone. Problems arise from the very beginning. To power-on, the phone provides two approaches:

(1) Hit a green button, wait for a second, and then hit another button to confirm

(2) Alternatively, hold the green button for 3 seconds

My mother finds she is in trouble telling “hit” vs. “hold”, and it is especially hard for her to judge when the startup itself is completed, or whether she hit the button heavily enough. She also complains that the button is too many for her and keyboard is totally useless for her. UI controls are over-crowded to hardly figure out what is going on in the screen… Similar things happen in other scenarios. This makes me think several things seriously.

(1) Is feature-rich always good thing for customers? People(especially software makers) tend to add more and more features to their products version by version, and then charge customers more because of this. But in reality, this may result in confusion or even troubles sometimes.

(2) Are customers kidnapped? Feature designers always think their product should be used in this way, or in that way. But they can never imagine that for everyone. Providing enough customization-ability is probably a good practice. Customers should have the call on their click length.

(3) The elderly mobile market is ignored? I search for a phone dedicated to the elderly, but to my surprise there is no such type. Because kids and women have much bigger spending power? Can Windows Mobile easily hit this market niche?

#2: Medical checking report

A friend of mine was sent to the hospital after two days of Diarrhea. The doctor made a chemical examination of his excrements with the suspicious of certain virus. When we got the report, one highlighted message shows:

Rotavirus --> Negative

I am totally confused. Negative? What should I do about it? And then I had to turn to a doctor in the front desk and had the conversation as follows.

Me: “Execute me, could you let me know what it means by ‘Negative’?”

Doctor: “It means no reaction for this virus.”

Me: “But what does ‘no reaction’ mean here?”

Doctor: “There is no such virus found in the examination. In other words, your friend looks OK from the report.”

Software industry is not alone around such kind of customer interactions. This reminds me of two thumb rules:

(1) Don’t assume customers are as knowledgeable you/your study/your expectation. They can be lack of most basic things in your own area.

(2) User customers words. Customers care about their own problems more, and don’t care how you help solve them that much.

Multi-threading your UI

MSDNArchive — Mon, 15 Dec 2008 13:19:00 GMT

Basically, window app UI, either WPF or traditional WinForm, is single threaded, which means only one thing can happen in the UI at any given time. To be specific, it is not generally possible to create an object on one thread, and access it from another. In almost all cases this will result in an InvalidOperationException stating that “The calling thread cannot access this object because a different thread owns it.” On the other hand, we can’t do everything in the UI for many reasons such as performance requirement, UX responsiveness needs or testability concerns. One typical scenario justifying this is as follows. You are connecting to a remote server in background thread, and you might also want users to know the connectivity status after a while. So this leads to the general question: How to mul-thread UI? In this post, I will talk about both WPF and Win Form because Win Form has not gone yet. J

Technique #1: Dispatch a delegate on UI thread

Use Dispatcher.Invoke, we can execute the specified delegate on UI thread. The sample code would be:

/// WPF

private void OnConnectivityChanged(object sender, ConnStatusInfoEventArgs statusInformation)

{

this.Dispatcher.Invoke(DispatcherPriority.Normal, (Action)(() =>

{

this.MessageTextBlock.Text = statusInformation.msg;

}));

}

this is the form object or control object owned by the UI thread. In this event handler, we simply call the Invoke method. The first parameter is priority; the second one should a delegate. In our case we cast a statement lambda to Action delegate type. Other than invoke(), you can also use methods such as BeginInvoke() in certain situations. And the equivalent code for WinForm would be:

/// WinForm

private void OnConnectivityChanged(object sender, ConnStatusInfoEventArgs statusInformation)

{

this.Invoke(new System.Windows.Forms.MethodInvoker(delegate()

{

this.MessageTextBlock.Text = statusInformation.msg;

}));

}

We are using an anonymous delegate in above implementation, and you can see the code is pretty similar.

Technique #2: Update the UI while application is not busy

WinForm is still core part of many Windows application UI, especially when people design software years ago. For example, to extend the Outlook 2007 functionalities by writing your own form region in an add-in, you have to follow WinForm principles. To update your own form region in main Outlook UI thread, one of simple ways is to update it when application is idle:

/// WinForm

System.Windows.Forms.Application.Idle += new EventHandler(UpdateMyUI_OnApplicationIdle);

In WPF world, this can be achieved by using DispatcherPriority.ApplicationIdle as dispatch priority parameter in technique #1.

Technique #3: Background Worker

Background worker is the build-in .NET mechanism to multi-thread UI, but the reasons why I list it as #3 are that developers complain that it is not that good to fit all scenarios as opposed to your own thread. If you take a look at exposed events by background worker, you will know why. You can find sample code here if you are still interested in.

Technique #4: Freezable (WPF specific)

In WPF, freezable objects can be frozen, at which point they become read-only and can be used on any thread at any time. These are used across all graphics primitive resources (brushes, pens, etc).

if (myBrush.CanFreeze)

{

// Make the brush unmodifiable

myBrush.Freeze();

// From now on, this frozen freezable object(brush) can be shared across threads

// DoOtherStuff();

}

You can also create your own freezable classes.

Technique #5: HostVisual (WPF specific)

If you doesn’t require interactivity(e.g., keyboard inputs), HostVisual could be another option WPF provides. The element tree owned by the worker threads are rendered into their own composition target (called VisualTarget), and the results are composed into the HostVisual owned by the UI thread. The excellent code sample I found is here.

Getting Object’s All Properties at Runtime

MSDNArchive — Thu, 20 Nov 2008 17:19:00 GMT

We often need to know a given object’s all properties at run time, for example, tracking a data container object’s properties changes across different components. Here is a simple helper class you can use to do this magic.

namespace Helper

{

using System;

using System.Collections.Generic;

using System.Diagnostics;

public static class ObjPropertiesPrinter<objType>

{

public static void OutPut(objType objInstance)

{

Type myObjectType = typeof(objType);

// Get public properties via reflection

System.Reflection.PropertyInfo[] propInfo =

myObjectType.GetProperties();

// Output properties

foreach (System.Reflection.PropertyInfo info in propInfo)

{

Debug.WriteLine(info.Name + ": " + info.GetValue(objInstance, null).ToString());

}

Once this class is added to your project, it can be used in this way:

…

Helper.ObjPropertiesPrinter<MyClass>.OutPut(oneInstanceOfMyClass);

…

Similarly, you can also get other info via reflection such as fields, events, etc.

Programmatically Adding a Column to Your Outlook 2007 Inbox Table View

MSDNArchive — Tue, 04 Nov 2008 10:32:00 GMT

In my previous post, I mentioned that I will write an article about adding a new column to your inbox. Here you go!

What we will achieve

Quite simple. Take a look at below screenshot and you will notice that a new column, SubjectLength, is added to the inbox. Its values are the length of individual email’s subject.

How table view is displayed

This backs to the traditional (but useful) discussion of data and view. In view side, we need add a new column to table view via ViewFields.Insert method. In data side, we can easily add a new user property via MailItem.PropertyAccessor. Now the question is how to correlate view with data? The trick here is MAPI property namespace. They both can be referenced this way. OK, now let us talk about details in code.

Step 1: Add new user property to the inbox folder

Before we add new property to the view or items inside a folder, we have to add that property to the folder first. (Assume theFolder is inbox folder we get before hand)

private void AddUserPropertyToFolder(Outlook.Folder theFolder, string userPropertyName)

{

if (null == theFolder || string.IsNullOrEmpty(userPropertyName))

{

return;

}

// If the user property is not there, create it.

Outlook.UserDefinedProperty nameLengthProperty = null;

try

{

nameLengthProperty = theFolder.UserDefinedProperties[userPropertyName];

}

catch (Exception ex)

{

Debug.WriteLine("The UserProperty is not there." + ex.Message + ex.StackTrace);

}

if (nameLengthProperty == null)

{

try

{

theFolder.UserDefinedProperties.Add(userPropertyName,

Outlook.OlUserPropertyType.olText,

Type.Missing,

Type.Missing);

}

catch (System.Exception exc)

{

Debug.WriteLine("Error in creating User Property:" + exc.Message + exc.StackTrace);

}

Step 2: Add a new field to the inbox table view

Since the inbox folder gets a new user property, we can go ahead to add new field to the inbox table view.

private void AddOneColumnToView(Outlook.Folder theFolder, string userPropertyName)

{

const int newColumnIndex = 1;

if (null == theFolder || string.IsNullOrEmpty(userPropertyName))

{

return;

}

// Update the current view

// IMPORTANT: Actually modifying current view is not a encouraged practice

// In production code, you'd better clone a new view and change that new one

try

{

Outlook.TableView currentView = theFolder.CurrentView as Outlook.TableView;

currentView.ViewFields.Insert(userPropertyName, newColumnIndex);

currentView.Save();

currentView.Apply();

}

catch (Exception exc)

{

Debug.WriteLine("Fail to update view: " + exc.Message + exc.StackTrace);

}

Step 3: Get the prop of newly added column

As we talked about early, MAPI property is the way to connect view and data. So we need get the prop first. If you print the View.XML out, you will find the XML schema is pretty straightforward like below:

<?xml version="1.0" encoding="utf-8"?>

<heading>SubjectLength</heading>

<prop>http://schemas.microsoft.com/mapi/string/{00020329-0000-0000-C000-000000000046}/SubjectLength</prop>

<type>string</type>

</column>

<heading>Subject</heading>

<prop>urn:schemas:httpmail:subject</prop>

<type>string</type>

</column>

<prop>urn:schemas:httpmail:date</prop>

<type>datetime</type>

</column>

</view>

If this is case, below LINQ query code can get highlighted prop ID out. (Yeah, you are right that we need .NET3.5 to run the code.)

private string GetColumnProp(Outlook.Folder theFolder, string userPropertyName)

{

if (null == theFolder || string.IsNullOrEmpty(userPropertyName))

{

return null;

}

// Get LINQ XML elements

XElement folderViewXElement = XElement.Parse(theFolder.CurrentView.XML);

var props = from Columns in folderViewXElement.Elements("column")

where (string)Columns.Element("heading") == userPropertyName

select Columns.Element("prop");

// Return the first element which should be the only one

foreach (var prop in props)

{

return prop.Value;

}

return null;

}

Step 4: Create user property in items with the prop ID

private void AddUserPropertyValuesToItems(Outlook.Folder theFolder, string propName)

{

if (null == theFolder || string.IsNullOrEmpty(propName))

{

return;

}

// Create user property on all mails

Outlook.Items items = theFolder.Items;

string columnProp = this.GetColumnProp(theFolder, propName);

if (null != columnProp)

{

foreach (Outlook.MailItem theMail in items)

{

try

{

Outlook.PropertyAccessor oPA = theMail.PropertyAccessor;

oPA.SetProperty(columnProp, theMail.Subject.Length.ToString());

theMail.Save();

Debug.WriteLine("Mail property is set: " + theMail.Subject);

}

catch (Exception ex)

{

Debug.WriteLine("Set mail property goes wrong." + ex.Message + ex.StackTrace);

}

Finally, put above code into the addin entry point which is ThisAddIn_Startup by default, it will do the magic for you. That is it. Solution file can be found at:

http://www.codeplex.com/bali/SourceControl/changeset/view/10689

Outlook 2007 Views from Programmers' Perspective

MSDNArchive — Sun, 02 Nov 2008 17:38:00 GMT

In this post, I’d like to talk about certain basics for Outlook 2007 starter programmers. Outlook 2007 is shipped with enhanced programmability in various ways. Looking at Outlook Object Model Reference, new users are often confused by terms such as view, reading pane, inspector, Ribbon, etc. Below two pictures illustrate often used view terms here.

Pic 1 Outlook 2007 Main View

Pic 2 Outlook 2007 Inspector Window

Outlook.Application is typical start point for accessing most of objects. Most common ways of customizing Outlook are as followings:

Add a new toolbar in command bars [Sample]

Add a new Form Region in reading pane and inspector. Even, you can also replace the whole form. [Sample]

Add new functionalities to Ribbon. [Sample]

Add a new column in the table view. [Check my blog post here.]

Building Objects from XML of eBay API with LINQ

MSDNArchive — Tue, 21 Oct 2008 18:49:00 GMT

In this post, I’d like to illustrate the power of LINQ with code sample. I query the web service published by eBay. Refer to this article about eBay API concepts.

EbayItem.cs

namespace LINQ2EbayResponse

{

class EbayItem

{

public string ItemID { get; set; }

public string Title { get; set; }

}

Program.cs

namespace LINQ2EbayResponse

{

using System;

using System.Collections.Generic;

using System.Linq;

using System.Net;

using System.Xml.Linq;

class Demo

{

static void Main(string[] args)

{

const string myAppID = @"Microsof-e148-4fe7-840c-38a19f6d59ba";

const string queryString = @"ipod";

Demo demo = new Demo();

demo.MakePOXCall(myAppID, queryString);

Console.WriteLine("End.");

}

private void MakePOXCall(string appID, string queryString)

{

// Make the call to eBay & Retrieve response XML to objects

string eBayUrl = string.Format(@"http://open.api.ebay.com/shopping?appid={0}&version=517&siteid=0&callname=FindItems&QueryKeywords={1}&responseencoding=xml&callback=false", appID, queryString);

WebClient ebayService = new WebClient();

string ebayResponse = ebayService.DownloadString(new Uri(eBayUrl));

XNamespace ebayNamespace = "urn:ebay:apis:eBLBaseComponents";

XElement xmlItems = XElement.Parse(ebayResponse);

var items = from Item in xmlItems.Elements(ebayNamespace + "Item")

select new EbayItem

{

ItemID = (string)Item.Element(ebayNamespace + "ItemID"),

Title = (string)Item.Element(ebayNamespace + "Title")

};

foreach (EbayItem item in items)

{

Console.WriteLine("ItemID: " + item.ItemID + ", Title: " + item.Title);

}

Done! in-memory objects are built. You can also follow ScottGu’s blog to wire up this code with Silverlight UI.

What Facebook means for the enterprise

MSDNArchive — Tue, 14 Oct 2008 18:39:00 GMT

It is Facebook’s heyday probably. Every tiny breath of Facebook will attract lots of discussion and review among major media and tech blog. It is successful to some extent if you don’t care its current sheet too much. But can we mirror Facebook’s popularity to Enterprise?

Let’s start with my experiences with Facebook. I’ve got my Facebook account for about 2 years and deployed my first FB app FunnyQ with FBML months ago. Through Cubics advertising network, this single app rewarded me real dollars whose number is same with one famous film actor. :-) I also have accounts in other popular SNS such as MySpace, Hi5, or Xiaonei, 51 in China. Certain living chatting channel and public forum are also in my radar. And if you'd like more info about my FB app experience, take a look at: http://blogs.msdn.com/bali_msft/pages/my-fb-app-development-experience.aspx.

At initial stage, I will logon at least once every day, but now almost once every two weeks. Most of my friends have the similar dropping visit frequency experiences. From the conversation we had, the primary reason is that Facebook is mainly about casual life around you. But it is another story if you work for companies. We are hired to get things done. Can Facebook help you fix bugs automatically? If not, get out of it if you are in a hurry, boy.

But it is true that Facebook is indeed a penetrating and thought-provoking thing to enterprise software. I’d like to talk about this in several ways in the post:

Goal/what you care. Primary values FB brings are keeping track of emotional connection with persons people care about in a graceful way. What do we care in the company? Maybe job commitments, career development, project, work life balance, etc. Main differences here are flexibility – You can allocate your time at will in spare time, but you often have to work hard to meet certain deadline at your office. Do we really know what we care?
Lightweight One-Stop solution. You know everything about your friends by opening Facebook. But what are tools helping us stay connected with things we care about? You may argue emails. Do you notice people start tired of email overload along with its abuse?
Programmability/Extensibility. Probably Facebook is the most successful end-to-end web programming platform, even better than Salesforce and AWS. Anybody can provide additional functionality to everybody under the framework. Looks like there is long journey before Sharepoint achieves that level, not to mention OWA.
Interactive. Who will play games as childish as Parking War without interactivity with friends? Does interactivity mean much for enterprises?
Credit system. Do something to earn certain points, and not necessarily real dollars. Virtual currency is a good enough way to encourage participation. This interesting phenomenon could be explained by Abraham Maslow’s Hierarchy of Needs. Large enterprises have experts distributed in various areas, but it is not easy to access them. How to get them found/involved?
Trusted relationships. People thought trusts coming along with SNS could be great foundation for business, such as better targeted AD. But the disappointing performance of Facebook Marketplace indicates something different. Looks like people love getting to know friends in bars, yet still tends to get down to business in meeting room. Trusts also exist among co-works. Is there any other better way to leverage them?
Collaboration. Best collaboration examples for me are Wikipedia and source control system. Two key factor: (1) Clear goal, (2) established rules/framework. Can FB help collaboration given the facts that enterprises often need complete certain clear tasks?
Communication. There are various ways to exchange information in Facebook – Wall, feeds, apps, event, video, photo and of course traditional email counterpart, message. All of these are simply there to help you achieve the goal of visiting FB – stay connected with Friends. If the goal is clear in business environment, can we design other rule changing ways to improve our communication?

What Facebook means for the enterprise on earth? Answers may vary from person to person. Here is one of them.

;)

"Search engine cannot make any money"

MSDNArchive — Mon, 06 Oct 2008 18:43:00 GMT

Some of my best friends complaint that my blog was over technical for them. With customer-oriented spirit and inspired by Socct Berkun's the Myths of Innovation, I composed this eye-catching-titled post talking about how to innovate and influence people to adopt your ideas. J

Our world becomes as it is today pretty much because many innovative people influence the whole world by invent things such as telephone, airplane, energy and internet in which way you read this post. But what are similarities they share besides smart, hardworking, etc?

Question asker – Why is it working that way? How to solve this problem? What are concerning customers?
Don’t follow the rules – Don’t do things as told. Not always be student.
Outsider – Take the road less taken. Do things that no one else things of. (不走寻常路，in Chinese). Think from origin.

There were many ideas in people’s mind, but few of them became real innovation. So what is the flow of turning ideas into innovations? Typically the flow would be:

Idea -> Pitch -> Proof of concept -> Prototypes -> Plan -> Acceptance of risk -> Commitment -> Execution -> Innovation!

This flow almost works for everybody. You can be in normal corporations where you report to your manager, or you are genius college student gaining certain cool ideas.

Woo-hoo, it is a great flow, man. But wait a minute,

Where do ideas come from? – I feel like it may come from two ways. 1) You can observe one problem and go back to your house do your homework, finally find out the solution. We can list many supportive examples such as light bulb by Thomas Edison. 2) People may run into something unique, and then start seeking for applying field reversely. For example, low tack adhesive makes post-it possible. So the point here is to observe anything around us and to keep asking questions such as “how to make this better?”, “Is that possible?”, etc. And ideas are not necessarily as big as changing the whole industry, they can be extremely simple but solve problems in graceful way. Also, combination of ideas may also lead to cool ideas. For example, YouTube + HotOrNot = video version of HotOrNot.
Why pitch matters? – Pitch here means convincing somebody, your friends, colleagues, spouse or your managers. This is the first skill you need to have, definitely. This can help you dogfood your ideas among ordinary people to see whether it really meets customers’ needs. Another reason to pitch is that you need their support (sometimes financially) before you go any further. Often you have to answer questions such as “who are your customers?” “how much money can you make/save” just like a mini B-plan. Sometimes pitching comes after POC/prototyping, that is true.
“I am too busy to have time for proof of concept/prototypes…” - Then you won’t be an innovator. It is simple. We are sleeping 1/3 of our time. Everybody have to meet his/her job commitments another 1/3 time per day. But what he does in remaining time sharps what his life would be in longer term. You would be more luck if your cooperation encourages you invest ~10% of working time to pet project. J I assume you know the importance of prototyping, so you have to do experiments to further prove your ideas.
From “plan” down to “execution”, looks pretty familiar… - Actually maybe not. Mistakes can be made. But so what? At least we know this way does not work with these people this time at this place.
What comes after innovation? - Commodification! Sell it out to your manager or even to the general public. The more, the better. At that time, everything else(e.g., $$$) will follow you automatically.

Stay hungry, stay foolish. If somebody with powerful looking tells you something like “search engine cannot make any money”, think about why and try other possibilities instead of immediate drop.

Adding column to existing table and populate with default value

MSDNArchive — Sun, 05 Oct 2008 17:36:00 GMT

From time to time, we run into the scenarios of adding new columns to existing table with millions of records to meet emerging business needs. And these new columns often need initialized with default value. In this post, I’d like to illustrate my solution for such problems. Let us assume,

Database: TestDB

Existing table: TestTable

Existing columns: ID, C1, C2

And need to add,

Column name: “NewColumn”

Default value=”0”

SQLType=”INT”

Most simple solution would be:

USE TestDB;

-- Add 'NewColumn' column and poplulate this column as 0

ALTER TABLE TestTable ADD NewColumn INT;

UPDATE TestTable SET NewColumn = 0;

Often things are not that simple. We often get other questions regarding this problem. Hereby I list them as FAQs.

Q1: My existing table is very complicated with lots of foreign keys and indexes. Does this solution work for that?

A1: Maybe not. I strongly recommend you solve this by creating a new table with the correct order, and copy all the records from the existing one to the new one.

Q2: Your solution will put newly-inserted columns to the last. Can we put them to certain specific places? I am asking because our business rule needs keep audit column in the end.

A2: Unfortunately, there is no quick way to add them in a specific place in the column order in T-SQL. One thing worth mentioning is that column order does not matter because you can select the columns in any order you want.

Q3: Your solution looks fine for me, but I get hundreds of tables to do the same thing. Can I put this into a stored procedure?

A3: Yes and No. Alter table can’t take dynamic parameters. So it is impossible to create SP such as:

-- INCORRECT SAMPLE. DO NOT TAKE IT.

CREATE PROCEDURE sp_addcolumn

@tablename varchar(50) = 0,

@columnname varchar(50) = 0,

@datatype varchar (50) = 0

ALTER TABLE @tablename

ADD COLUMN @columnname @datatype

END

However we can do it in another way like this:

-- Correct sample

CREATE PROCEDURE sp_addcolumn

@tableName VARCHAR(50) = 0,

@colName VARCHAR(50) = 0,

@dataType VARCHAR (50) = 0

DECLARE @tsql VARCHAR (200)

SET @tsql = 'ALTER TABLE ' + @tableName + ' ADD ' + @colName + ' ' + @dataType

EXEC(@tsql)

SET @tsql = 'UPDATE ' + @tableName + ' SET ' + @colName + '= 0'

EXEC(@tsql)

-- The way to use the SP is as followings:

-- EXEC sp_addcolumn 'TestTable', 'NewColumn', 'INT'

Selecting median of two sorted arrays

MSDNArchive — Mon, 22 Sep 2008 12:46:00 GMT

In this post, I’d like to discuss one interesting algorithm problem which took me quite a while to find an ideal solution. The problem is as followings:

Array A and B are sorted with length of m and n respectively. Try to select median of the two arrays within O(log(m+n)) time. For example,

A = { 1, 4, 6, 10, 18 }

B = { 1, 7, 13, 45, 58, 69, 100, 180, 300 }

Answer: 13

Step1: The lengths are equal. Let us start with a special scenario of the problem. The solution is as following.

Examine the middle element of each array, and throw out the lower half of the array with the smaller element (since all those must be less than ½ the numbers) and throw out the upper half of the array with the larger element (since all those must be greater than ½ the numbers). Now both arrays are still the same size. Repeat until you have two elements left. This is your median. Each step, you eliminate half of the numbers, so it should have a runtime of O(logn).

Step2: What if the lengths are not equal?

The smartest solution I’ve found is padding. It is best to think of median as a special case of the select problem, where given a set and an index k, you need to find the kth smallest element. Median is simply Select(n/2) when n is even, and fully defined by Select(\floor(n/2)) and Select (\Ceiling(n/2)) when n is odd. Now there is a simple O(log n) algorithm for the Select problem on the union of two equal sorted arrays, that throws out the appropriate halves of the two lists recursively.

If you want a simpler reduction to equal sized lists, pad the shorter list with an equal number of positive and negative infinities. If you need an odd number of pads (ie: when the source lists have an odd total), do it with positive infinity and (since the set size was odd) return the lesser of the two retuned values.

Bill Li

Move

My PM talk in SJTU

If I were designing a new email service, I would…

Entities

Problems - an abused tool in workplace

At a glance

Demo/UI Mock-up

Key features in Pmail

Building Global Development Team

Pros and Cons

How-to

#3, Hulu, Why?

Lead Without Authority

My AD Fun Experiment

P2P Backup System w/o SPOF for Work Group

My P2P Backup System

Memory Leaks Demo & Detection in .NET Application

Designing Your Own Recent Posts Widget for MSDN Blog

Searching For a Number in Shifted Sorted Array within O(log(n)) time

To Next Cuil

Brand. Brand. Brand.

Infrastructure

Understand/Repsect Customers

Culture

Web Competition Strategy

No chance to win in search?

Principles for Building Secure Database Applications in Action

Happy New Year of the OX!

An AD System to Pay Content Generators

Problems

At a glance

Demo

P0 features in 1st Milestone

How It Works

PPA Implementation

Online-PPA

Offline-PPA

Open Questions

Words from potential ex-customers

Multi-threading your UI

Getting Object’s All Properties at Runtime

Programmatically Adding a Column to Your Outlook 2007 Inbox Table View

Outlook 2007 Views from Programmers' Perspective

Building Objects from XML of eBay API with LINQ

What Facebook means for the enterprise

"Search engine cannot make any money"

Adding column to existing table and populate with default value

Selecting median of two sorted arrays

P0 features in 1^st Milestone