Spout's from wherever

Applications, Infrastructure, and Systems - Part I

2006-05-17T17:12:00Z

I’ve been in the consulting business more than a decade. I’ve designed some really cool applications. I’ve designed the infrastructure for some big applications. What amazes me the most is how infrequently people design systems.

Case in point, I’m working on an enterprise document management system for my current customer. I was brought in to the project very late, with only a handful of iterations left, and most of the application and infrastructure design completed. My job was to come up with the deployment plan to take the app from dev, into the test environment, and finally into production (I know, it’s not Agile, but it’s how the customer does things). Everything was all well and good until I started looking at the disaster recovery plan. It was obvious that the application architects and the infrastructure architects were not in the same room when designing the DR plan. Interestingly, neither side did anything wrong. In fact, both the app and the infrastructure were designed with best practices in mind. The only problem was that they wouldn’t work together.

Part of the disaster recovery plan called for replicating the SQL databases from the production environment to the disaster recovery environment using SAN replication. Good idea. Keep the replication down at the hardware level, and keep the DR boxes cold until needed. The app doesn’t need 5 9s uptime. It’s ok if it takes an hour or two bring up the app in DR. The infrastructure architects assumed that the app would just have to change the connection string to point to the new SQL servers in DR in case of a catastrophic failure of the production environment. Sounds logical, but they didn’t tell the application architect.

The application architect wanted a central repository for all the configuration information. Since this is a distributed app with a load-balanced front end, putting the configuration information in a SQL DB made sense. Each of the front-end boxes just needs a connection string to the common configuration SQL DB. Once again, good idea.

Where the whole thing breaks down, is that the app stores other SQL connection strings in the configuration DB. These connection strings may be to other DBs on the same server. Now think about this for a minute. SAN replication is replicating a DB that contains SQL connection strings. In a disaster situation, the SQL server that the connection strings in the table point to may not be on-line anymore. Can you see a problem with that?

Tomorrow, I’ll talk about possible solutions to the above problem, and how we solved it for the customer.

Cheers,

m²

The hills are alive with the sound Newark

2005-01-04T20:08:00Z

First, please accept my apology for not posting in a while. I won't go into the details, but it will suffice to say I've been on the road and very busy.

Second, I'd like to wish everyone a happy New Year. May 2005 be better than 2004.

My new customer takes me to New Jersey every week. The project is fun. I'm getting back to my roots as a hardcore coder. C++ is my weapon of choice. Enough about technology for now. Each Monday and Thursday as I pass through Concourse C at the Newark airport I can't help but to smile. You see, Bob is invariably there, and he's singing. He's not singing any tune in particular; he's just welcoming people, wishing them a good day, providing directions to the Air Train, and helping hapless travelers find their gate. It's definitely a high point in my day. So for you, Bob, on behalf of the traveling public passing through Newark Liberty Airport Concourse C, I say thank you and keep up the good work.

Cheers,
m²

Another week done

2004-02-27T20:27:00Z

Another week on the road is over. It feels nice to be home, and the cats seem quite happy to see me, too.

One topic that keeps popping up at my current customer is administration of Code Access Security polices. It seems there is an abundance of information on how to sign code, how to assert permissions, and how to use the tools provided by the .Net Framework to create CAS policies. I can't find a single paper that talks about creating an enterprise level (not the enterprise level in .Net) CAS policy; how an organization should go about defining global organization code groups and permissions; how to deploy the policy to all of the machines, and how to classify internally developed apps into the code groups.

Would anyone else find this information helpful, too?

Cheers,
m²

Hotel Woes

2004-02-18T20:40:00Z

It seems the more nefarious elements of our digital world I mentioned in my last post took up residence in my hotel last night. I'm still not convinced they are evil, they may merely be clueless. Whatever they are, they are running a non-authorized DHCP server that is completely disrupting service. I have sneeking suspicion the level 1 and level 2 tech support people aren't necessarily helping the situation.

My room has been down for 12 hours and counting. Sometimes the port appears to be disconnected, other times I can't get a DHCP address. Even the static IPs the helpdesk gave me didn't work.

Ah, life on the road. Gotta love it.

Cheers,
m²

Patch Time

2004-02-18T20:33:00Z

Well, that certainly didn't take long. It took hackers only one week from patch to exploit for the ASN.1 vulnerability. It seems the more nefarious elements of our digital world are getting more proficient at reverse-engineering security patches. This unfortunate situation only emphasizes the need for a good patch management policy.

You must have a patch deployment plan in your organization. Whether your organization is 1 to 100,000 computers, you should have a patch management policy. The plan should include testing patch installs on your standard machine image, regression testing of key applications with the patch, understanding how how back-out the patch, deploying the patch, auditing the deployment process, and doing patch inventory on your machines. You gotta do it. Ain't no way 'round it. Period. Check out NTBugTraq for a good list of patch management tools.

I know this sounds like substantial amount of work, but it really could be as simple as backing up your box, installing the patch, and testing your favorite apps.

One other very important part of having a good patch management policy is following your patch management policy. It's like we say in diving: Plan your dive, dive your plan. You could have a very safe dive plan, but if you don't follow it you could get bent, or worse. You can also have the best patch management plan ever devised, but if you don't follow it, it's worthless.

Every major attack against the Windows OS was against a vulnerability that had a patch available. 331 days from patch to Nimda. 180 days from patch to SQL Slammer. 151 days from patch to Nachi. 25 days from patch to Blaster. Time is getting shorter. How many until the next one (and you know there will be a next one)?

I've patched my boxen; all 15 of them within 2 days of the patch release. Are yours patched?

Cheers,
m²