CrashTestChix

I was recently asked to obtain some Lotus product-related merchandise for some giveaways at a technology fair and subsequent presentations.  I checked out the IBM Logo Merchandise Store website and was disappointed to find no specific items with Lotus family product names.  Every thing was either IBM logo related or featured the Lotus Software logo only.  I was looking for items that said "Lotus Notes" or "Lotus Domino" at least.  I do remember previous Lotuspheres where specific product lines - like Sametime and Quickplace (Quikr) - either had giveaways or items for sale.  I have a fanny pack with the old Sametime logo, and I've been asked at recent Lotuspheres - "where did you get that?"  Many of us who work with Lotus products would like something more than a coffee mug that says IBM to take back to the office to share with staff members or customers.  Would it be possible to feature some product branded items at the IBM Logo Merchandise Store, and most certainly at Lotusphere?  And oh don't forget to add the yellow boxing gloves to the list!

At last week's CanSecWest conference, Microsoft engineers presented a new tool to assist developers with the detection of exploitable security flaws in their software. The program is an extension for the Windows program - Windbg. 

http://www.infosecurity-us.com/news/090319_MicrsoftExploitability.shtml

The extension can be downloaded here (Beta):
http://www.codeplex.com/msecdbg

What does the phrase “production environment” mean to you?  A definition on an IBM website states:

“Normal operating environment. Processes and tasks associated with this environment are integral and critical to the organization's productivity and operational stability.”

I am bringing this up only because once I again I find myself in the situation where I am being asked by a support organization to reapply fixes to a production environment, in order to cause a crash and capture debug information.  In other words - CRASH A PRODUCTION ENVIRONMENT.

This of course then leads to the addition of more fixes, and more debug to cause additional crashes in order to capture more data and more debug and more CRASHES ON A LIVE PRODUCTION ENVIRONMENT.

I fully understand that each installation of software is unique.  If configured in different ways, or not provided with enough resources or bound by unknown constraints, that software may react in different ways.  And I know that it is difficult to trap and capture that uniqueness in such a way to provide adequate information to developers who may be writing code to fix problems.

HOWEVER, given the amounts of money that we all pay software vendors for licenses and support shouldn’t the following be applicable?

• That software is tested before it goes out the door…especially if it is portrayed as a “fix.”

• That vendors have their own test environments for those software and hardware platforms that they claim to support.

• That vendors understand the timing of production maintenance cycles and do not imply that customers are holding up the troubleshooting process because they could not bring down their production environment on a moment’s notice.

And we as customers will try to remain calm when trying to express the nature of our technical issues, whilst our users are clamoring for answers and fixes, and management is asking why debug must be added to a production environment.

Might it not be worth examining the existing diagnostic/crash process if it is not capturing as much data as required versus the mechanism of apply fixes, debug, crash, fix, etc.?

This is my "mini" rant for the week.  For those of us who provide user support, how often do we get the email or call from a user that says "this x process is taking FOREVER."  As we know - Notes is NOT (unfortunately ) a network operating system, a disk operating system, or a workstation operating system, and somehow cannot always be responsible for perceived slowness.  And what does a user mean by "forever"?  Does that refer to 30 minutes, 10 minutes, 5 minutes, 2 seconds or 1 nanosecond?   And how do you as a support person keep your wits about you when you realize that they didn't have their PC plugged into the network, or that their inbox contained 40,000 documents, or that the server had crashed during the "forever" bit, or that they were using a dial up connection, or that they were attempting to send 200 Mb WAV files to 300 of their best friends? 

I like to blame some of the time perception on the microwave and the internet.  I am old enough to remember the BEFORE time when neither existed (yes you youngins out there -  there was a time when you had to boil water on the stove top and there was no Google).  And when you researched a technical problem you had to wait for the vendor to provide a manual (usually 30 2-inch ring binders of material).  And a whole meal could not be prepared in less than 5 minutes!  So I think our perception of time in general has become skewed.

I will admit that I too have a short attention span when it comes to resolving server issues.  I want a server task to shutdown IMMEDIATELY when I say quit!  And that a compact on a 5 GB database should take less than a minute, and that ALL server restarts should take under two minutes!  Silly me!  But I do realize that that is not always the case, and that words like "slow" and "forever" should be replaced with "challenged", "teetering", "crawling" or some other descriptor that does not indicate that you will be chained to the desk viewing a blank screen for the rest of eternity!  So if you have any favorites - please share!

Here are a few tidbits of "wisdom" gleaned from our current Notes 8.5 upgrade:

• databases that have been around migrated from previous versions (prior to 7.0.3), with private folders, or complex folder structures - may take longer on a convert - especially if you are converting the designs up from dwa7.ntf to mail85.ntf versus upgrading from mail8.ntf.  Check your resource utilization while the convert process is running to see if the process is affecting overall server availability.
  
• remember to change the mail85.ntf template to enable the compression features that are available beginning with 8.0.2.
  
• for Notes 8.5 installs on Vista workstations -- make sure that UAC has been turned off.  There are PMRs open for Vista/Notes 8.5 - especially with regards to the 8.5 client NSDing on first start after the client install.  The client will start in safe mode without networking, but with networking it will not load.  Support has relayed that there may be some Windows files at play that are either missing or not at the version expected. 
  
• for those workstations that continue to prompt for a software upgrade or a design refresh after the refresh has been confirmed on the server (smart upgrade tracking database) or in design properties of the database - a  nonsupported workaround to make the messages go away involves editing the notes.ini and removing any of the SmartUpgrade related parms (begin with SU).  As always make a backup of the notes.ini before attempting this.
  
• users do miss the red unread marks, and will find the workaround for editing the jar file - so be prepared for questions or issues if that fails.
  
• if a user is prompted with the message - no matching smart upgrade kit found - most likely the user is attempt to install a client kit versus an all client kit.  a nonsupported work around for this is to change the installtype in the notes.ini from installtype=2 (for all clients) to installtype=6; and then have the user attempt to run the smart upgrade again.
  
• Sametime preferences have changed significantly in the Notes 8.5 client, so review those settings before deploying.

So far training issues have been minimal, but we have not yet upgraded those power calendar users yet, and I expect questions may arise with some of those folks.  All and all the project is moving along!

If you are planning to use the SmartUpgrade process and Policies to assign a SmartUpgrade Desktop policy, note that with 8.5 when clients are at 8.0.1 or higher you can assign the user or groups directly to the Policy settings document from the People & Groups Tab in the Admin client.  However, if you are in a mixed environment where your servers may not all be at 8.5 or clients are not at 8.0.1, you will need to assign the policy to the Person document for users and groups, in order for the SmartUpgrade process to recognize that individual users or group members require an upgrade.

Okay, I'll admit I'm not the biggest sports fan around.  But these are exciting times.  My employer - Virginia Commonwealth University - is going to the Big Dance - the NCAA Basketball Tournament.  Both the men's and women's teams are participating -- the women's team for the first time ever! This is a huge event for a school that is often overshadowed in sports events by the University of Virginia and Virginia Tech as we don't have a men's collegiate football team (more on that in a bit).

ESPN's Andy Katz has reported that President Obama has picked VCU over UCLA in the first round! Go Rams!

In December 2008, Bill Cosby visited our Richmond campus to share in the university's 40th anniversary.  During the course of the evening he wore a shirt that said "VCU Football - Undefeated Since 1968".  So maybe Bill knows something we don't know! The hope is that we'll have a football team soon!  So here is a little March Madness for you hard core college basketball fans out there...enjoy the tournament!

See the Wall Street Journal Alert

Article contends that talks are still in early stages for IBM to acquire Sun $6.5 billion in cash.

At Lotusphere 2008, I had an opportunity to visit the Client Reference Lounge and share our story regarding the use of IBM/Lotus technology.  This put into motion the preparation of a public case study, which has recently been published on IBM's Higher Education site.

I would recommend this to all customers.  If you have a story to share, stop by the Client Reference Lounge next year.  Not only do they want to hear about your experiences, but they have comfy chairs and great snacks all day!

Here is some info about our Notes 8.5 client SmartUpgrade deployment. 

First, we used the InstallShield Tuner to build a “stand alone” 8.5 install package for new client installations and for home client installs.  We also built a package that is customized for the corporate configuration that maps the Notes install directory to c:\Notes, since this is where historically we have installed Notes over the years.

We removed the Client Single Login Feature and Lotus Symphony (sadly) as install choices.

We include a modified pernames.ntf that already includes connection documents for all Domino servers within our organization.

Modifications were made to the “plugin_customization.ini” to include the following parameters:

• com.ibm.rcp.ui/USE_LARGE_ICONS=true

• com.ibm.rcp.ui/DOCKED_LAUNCHER=1

• com.ibm.rcp.ui.browser.launcher/USE_EMBEDDED_BROWSER=false

Activities, Sametime Integration and Composite Application Editor – were added via the setup.ini 'CmdLine' as follows to add these features automatically during install without user intervention.

• CmdLine= TRANSFORMS=LotusNotes85.mst SELECTINSTALLFEATURES=Activities,SametimeUI,CAE PROGDIR=C:\Notes DATADIR=C:\Notes\data /qb+

We created a Desktop Policy settings document with specific items relating to the SmartUpgrade process – including referring to the mail85.ntf template and also allowing the upgrade of custom folders.  We also have set up the SmartUpgrade mail-in database on several servers in order to distribute the upgrade load and also limited the number of concurrent downloads that can occur on any one particular server.

Over the past two weeks, we began deploying the Notes 8.5 standard client to a group of early adopters.   The group included those technical staff outside of the normal email support team, who sometimes do not share the same enthusiasm for Notes (yeah believe it or not!), as well as group of general users.  None of the group had been given any preview information prior to the session about new features - so we had not hyped them up with our preconceived notions about what we thought they would like or should like about the new client.  We had them go through the SmartUpgrade process - directly from 7.0.3.

To our relief and great delight, the sessions and feedback were overwhelmingly positive.  Those items receiving a thumbs up included -- the type ahead addressing with contact choices; inline spell checking; calendar overlay; vertical preview pane, and iNotes lite and ultralite.  They raved and raved about Sametime sidebar --you would have thought it was the Notes "candy bar", and they loved the screen sharing feature.  And there was squealing - yes squealing -- about widgets in the sidebar! 

We are using SmartUpgrade to push out an install kit attachment to install 8.5 over 7.0.3, as we do not have a standard file share in our environment.  So far, users are seeing install times of less than 15 minutes, with most experiencing a 10 minute install time.  We are working with support on an issue with the SmartUpgrade policy as it relates to the template upgrade.

Our remaining groups will receive advance notice of course, and we have ramped up lots of training and press for the remaining staff.  I am sure that the technology curmudgeons will find something to be vocal about, but all and all I am ready to push the deployment forward.  Well done to Mary Beth Raven, her staff, and the client development team.

I am a Lotus customer, Lotus administrator, Lotus application designer and email system manager.  I love Lotusphere.  I get excited about the trip and each year prior to departure, I prepare what I affectionately call “The List.” “The List” includes wishes, open PMRs, suggestions for new features, and general rants.  I always make time in the ever hectic schedule for face time with the developers at Lotusphere to discuss “The List.”  I have to admit the developers are always patient and seem to be attentive.  I am in awe of the work they do.  And after spending a week in the glorious Florida sun in the rarified air of Disney, I am calm and reminded of all the reasons why I continue to work with email and specifically Lotus products.

But about this time every year, I go through what I have come to call the Post Lotusphere Blues. Each March after several weeks of server crashes, hangs, fix packs or hot fixes that cause more crashes, I am over the glow and enthusiasm that I had just a few weeks previously.  I am depressed, I am frustrated, and I am angry. 

I support Lotus Domino on Solaris with a mixture of Lotus Notes clients, iNotes users, and IMAP users on PCs and Macs.  According to Lotus documentation Solaris 10 is a supported operating system for Domino and IMAP is a supported server task/protocol.  However, our primary server stability issue has been with IMAP, and we have run through hoops with Lotus when it comes to Solaris support.  Comments from actual support calls have included “you have to call another number – we don’t support that operating system,” “we don’t have a Solaris test server available to test the hot fix,” “what is IMAP?” and probably my personal favorite -- “we can’t test everything.”  And if Lotus Support asks one of my team again for a copy of our names.nsf ..I might have to drive to Westford to have a few words.  On a positive note, and yes there are positive notes here – the Lotus messaging team has been admirable in their efforts to address IMAP reliability.  But in my opinion the fixes have been Band-Aids on a portion of the server that in reality needs a major overhaul.  IMAP is not going away.  More and more cell phones use IMAP as their mechanism for connecting to mail systems. And while some surveys may show that home users use web clients for email access – why should IMAP users be neglected or why should an IMAP client cause a Domino server to crash?  So maybe it is time to add more staff to focus on this particular service/reliability area or come out and say that Lotus no longer supports either Solaris or IMAP.

As a customer, I am also frustrated by the disconnect that seems to take place between what we as customers want/need and what IBM  thinks we want/need.  I do not know why this is taking place – it seems that in certain areas – especially within the Lotus Notes client space (8.0x and 8.5 especially) great strides have been made to ask customers – “what do you want, what do you need, how do you use these features?”  But on the server side – even though some incredible features like DAOS and the ID Vault have been added – where has any value been added with essentials like archiving?  I heard improvements with regards to local archiving; but with e-discovery high on everyone’s list – why would anyone want a user to store their archive data locally?  Come on now!  And what about attachment limits?  A number of customers and business partners have been clamoring for some time for tools to manage attachment sizes from the client side rather than the server side. 

And don’t get me started on marketing.  When was the last time you actually saw a Lotus ad in any media?   I’m an Apple, you’re a Lotus?

But, in the same breath, I would argue with anyone about the fact that I believe Lotus Notes/Domino is a better product that anything in the market.  Maybe that’s the dilemma, and part of my discontent.  Or the general frustration that maybe the “yellow collective” is trying to voice at this time?  That we know Notes/Domino is a great product, but that it is being held back by itself or its own company?  April and sunnier weather will come soon, and I’ll spend less time at my desk, so the Post Lotusphere Blues will ease for me.  But what can we do about them as a community at large?

I am not a big fan of Daylight Saving Time.  Not only am I one of those who needs the sun to remind me to get up in the morning, but as a technology person, DST has presented certain challenges over the years.

So this morning I logged on to my Facebook account, posted a comment only to realize that Facebook was not on DST, but an extra hour ahead.  I checked my  laptop to make sure I was on the right time zone.  I was, so I posted a bug notice.  Sooner or later I'm sure that the Facebook techies will get it sorted out.  I feel for those who will be working on the challenge as I remember having to work on calendar events for those individuals who had not kept up with Microsoft patches and wondered why their Notes calendar reflected meetings in Rangoon time, as well as deploying Blackberry DST patches and checking every server after the big time revision.

So I'm feeling a little depressed about losing two extra hours of daylight every morning...at least according to Facebook!

Mattel has released a Totally Tattooed Barbie...will this be the next have to buy Christmas toy for some?

Over the weekend I moved a Sametime 8.0.2 server from Domino authentication to LDAP authentication.  I was fortunate in that I was able to retain awareness after the migration in the Notes client, iNotes, and Quickr.  But there were some problems with the buddy lists however, as I had expected. 

I found that I could fix a number of buddy list issues with the Export Sametime Contact List feature in the embedded Sametime client.  I'm currently using the Notes 8.5 client connecting to a Sametime 8.0.2 server.  From the tool bar in the Sametime side bar, select the Click for a Menu of Options button, then select Export the Sametime Contact List, To MultiServer Community File.  This allows you to save your buddy list to a file locally in XML format.  You can then edit this file with an XML editor.  In this particular instance the id referenced in the buddy list was not in the format now used for authentication.  Once I changed the id entries, I went back to the Sametime tool bar, selected the Click for a Menu of Options button and then Import Sametime Contact List, From File, From Multi Server Community File, and then selected the file I had edited.  I selected the option to replace.  Once this was done, the awareness in the buddy list was restored.

One of tools that I consider a genuine "Swiss Army Knife" when it comes to connecting disparate data sources and directories is Tivoli Directory Integrator.  Here are a few links to point you in the right direction if you are interested in delving deeper into learning more about TDI.

One of the first places to visit is the IBM Tivoli Directory Integrator Users Group.  This users group includes sources for TDI training, glossary of TDI terms and concepts, Javascript tutorials, tips and hints, as well as step by step examples.

Integrating Lotus Notes/Domino with Tivoli Directory Integrator, authored in 2008 by Eddie Hartman, a member of the TDI Development Team.  This wiki includes a sample CVS file to use to step through the exercises outlined in the wiki.  The wiki also contains chapters on using TDI with Domino databases – other than the Domino Directory, and documentation regarding the different types of Lotus Notes/Domino connectors that ship with the Tivoli Directory Integrator.

And here's a recent posting on developerWorks about using TDI with Lotus Connections.