Wednesday, March 14, 2007

Two of the coolest videos ever

The first is an extraordinary CGI visualization of the internal biomolecular activities of a white blood cell, entitled "The Inner Life of the Cell":

The depictions of the transcription of mRNA into proteins and of vesicles "walking" along microtubules via motor proteins are particularly fascinating. This video has garnered a great deal of interest of late, both in the educational community, and in Hollywood. Have a gander at the Wired article.

The second is a startling bit of micro-cinematography of the real thing, made possible by the use of fluorescent-tagged proteins:

Monday, March 12, 2007

How Microsoft Botched DST 2007

Well, March 11 has come and gone. The mini-Y2K has passed. As expected, nothing major has gone wrong, but there are many minor glitches, and people will be missing a lot of meetings for the next few weeks. After a huge push by IT folk (I've personally put in about 100 hours on DST remediation), we've passed the hump, but Microsoft has a lot of seriously peeved customers. The question is, do they realize just how annoyed we are?

Design Flaws

Before I go into the problems with Microsoft's handling (or mis-handling) of the DST transition, I suppose I should touch upon some underlying design flaws in Microsoft's products which contributed to the issue, and, admittedly, limited Microsoft's ability to respond to the issue in a timely manner.
  • Windows Boxes Tick Local Time - Due to the desktop-centric origins of Microsoft's offerings, the system clocks on Windows boxes tick local time (a design flaw shared by OpenVMS), instead of ticking UTC. Systems which tick UTC instead of local time, such as Unix and related operating systems (including Mac OS X), only have to worry about DST issues in the presentation layer.
  • Exchange Calendar Data Stored in UTC - Ordinarily, storing Calendar data in UTC would be a good thing, except that combining this with the aforementioned fact that Windows systems tick local time makes rebasing (correcting Calendar items for changes in DST rules) problematic. This is because differing DST rules may be in effect between the time that the Calendar item is created and the time it is displayed. When the item is created, local time is translated to UTC. When the item is subsequently displayed, UTC is translated to local time, potentially (as in this case) under different DST rules. Remember, when we shift in and out of DST, we don't want the local time of an appointment to change, just the underlying UTC time. If a meeting was at 10:00am before, it should still be at 10:00am after DST kicks in. So why bother with the translation to UTC?
  • Multiple Timezone Data Fields Associated With Exchange Mailboxes - Exchange Calendar objects have timezone data associated with them. Specifically, there are four timezone fields: OWA timezone, CDO timezone, Outlook timezone, and recurring meeting timezone. ( Please note that if rebasing is performed after the Exchange server is patched, the OWA timezone data is not correctly updated. Unfortunately, this guidance was not provided until fairly late in the process. More on that later.) So, why four fields (not all of which are properly readable by the rebasing tools)? Only a handful of people in Redmond likely know why.
  • Windows Stores Only One Set Of DST Rules - Most other operating systems are capable of simultaneously storing multiple sets of DST rules, including specifications of when given rules take effect. This is why most Unix vendors were able to release DST updates shortly after the energy legislation mandating this change was passed in 2005. However, Windows can only store one set of DST rules at any given time, so Microsoft had to wait until after DST ended in the fall of 2006 to release DST patches containing the 2007 rules.
Microsoft's Response: Belated and Incoherrant

With these fundamental design flaws out of the way, we can now go on to discuss Microsoft's tardy and uncoordinated response to this issue.

I've already touched upon the reason why Microsoft was unable to release the OS-level DST patch until November of 2006 (well behind other OS vendors), but that is no excuse for the fact that the Exchange patch was not available until January, nor were the rebasing tools available until February, and, as mentioned earlier, the guidance for using them kept changing right on through into March. All of these tools should have been fully completed, debugged, and documented in time for release in November, instead of forcing IT departments all across the nation to go into a last minute scramble to prepare for March 11.

This lack of coordination and preparation raises some troubling questions: were the OS and Exchange engineers even coordinating their efforts? Why were the tools not all released at the same time? Why were they not properly tested, leading to changes in vendor guidance for their deployment at the eleventh hour (after it was too late for most of us)?

Speaking of testing, the Exchange Calendar Update Tool is extremely buggy. Not only does it not reliably correct Calendar items (a weakness shared by the Outlook Update Tool for which it serves as a wrapper), but it does not properly extract all four timezone fields for all users. I wasted nearly two weeks on the phone with Microsoft PSS trying to resolve the 0x80004005 errors that the tool was generating whenever I tried to extract timezone data for the users on my Exchange server, only to eventually learn that the solution was to only attempt to extract timezone info for recurring meetings. In other words, when running msextmz.exe, the msextmz.ini file should always have ExportTimezones set to zero or commented out. Just use the ReadCalendarTimezones value for running in export mode. In other words, don't bother trying to use the tool to correct anything other than recurring meetings.

To top everything off, when the tool runs in update mode, it spawns a new Outlook instance for each mailbox being updated, instead of simply using the underlying MAPI libraries. This to me demonstrates the last-minute approach that was taken in putting this tool together.

A Bad Idea Implemented Badly

Of all of the brilliant ideas that Ben Franklin had, Daylight Saving Time, well, wasn't one of them. Sure, it made sense in a world lit only by fire where candles were precious comodities, but it makes little sense in our modern world of 24 hour lifestyles, not to mention the energy lost having to run air conditioning longer in the hot south.

Congress may have had good intentions when they extended DST in the U.S. Energy Policy Act of 2005, but any energy savings that may have been realized have been more than offset by lost productivity. Businesses, especially their IT departments, have had to put countless projects on hold to address this situation, and Microsoft's late and lackluster efforts in this area have certainly not helped matters.

Note: This article is written primarily from the perpective of an IT administrator. For an end-user viewpoint, take a look at .

Friday, March 2, 2007

The Impact of 2007 DST Changes on Kerberos/AD Logins

Here is a thought experiment I came up with to evaluate what will happen when users try to perform Active Directory logins after March 11 from client systems that have not been patched with the new Daylight Saving Time rules...

There are two scenarios to consider:

the local time on the unpatched client remains unadjusted, or the unpatched client is manually adjusted to correct time (but still with incorrect DST status). The latter should fail Kerberos, while the former will succeed.

Please double-check my logic to see if I have thought this through properly.

Scenario 1 - local time unadjusted:

User tries to login at 8:00am CDT (13:00 UTC). However, the unpatched client machine, even though it is set to automatically adjust for DST, is unaware of the new DST rules. It operates as if CST is still in effect, so the local system clock reads 7:00am.

When the user attempts the login, the computers at each end will translate their local clocks to UTC for the Kerberos processing, using whatever DST and time zone rules they think are in effect.

* The patched DC, with the new DST rules in place, will correctly add 5 hours to the value in its local clock, 8:00am, and come up with 13:00 UTC to use in Kerberos negotiations.

* The unpatched client, operating as if CST is still in effect, will incorrectly add 6 hours to the local clock value. But the local clock is an hour behind, reading 7:00am, so a UTC value of 13:00 will be calculated. The UTC values on each end match, and Kerberos is happy time-wise.

Scenario 2 - local time manually adjusted

All steps above are the same except for the last one.

The unpatched client, operating as if CST is still in effect, will incorrectly add 6 hours to the local clock value. But the local clock has been adjusted to read 8:00am, so the calculated UTC value will be 14:00.

Since this value is an hour off from the UTC calculated on the patched DC, Kerberos login will fail.

The moral of the story is this. An unpatched client can still do a Kerberos login, but only if the local clock has not been adjusted to show the correct time.

This adjustment would have to be done manually. If the machine is pulling time via NTP (which is passed in UTC), incorrect DST info will result in the local clock still being an hour behind.