Chad Scott

I ran into an issue this morning where new and modified program documents are not recognized by the Domino job scheduler. If I added, deleted, or modified a program document, I saw no change in the output from "show sched" on the console, and the program documents never ran. It turns out that this is behavior introduced by the set of NAMELookup fixes documented in Technote 1254752. The various NAMELookup flags are enabled using the notes.ini parameter DEBUG_ENABLE_UPDATE_FIX, which has the following default/maximum value by release:

• 6.5.5 FP2 - 7.0.2: 0/8191
• 7.0.3: 0/16383
• 8.0 and 8.0.1: 8191/16383
• 8.0.2: 8191/32767
• 8.5: 32767/32767

These values enable flags that turn off unnecessary view rebuilds by functional area. For the program document issue, that flag is decimal value two, which means that if you are running any release of 8.x without the notes.ini parameter set or 6.5.5 FP2 or later with the notes.ini parameter set to a value that includes this flag, you will experience this issue. To work around the problem, you can remove the flag, which means subtracting two from the value of DEBUG_ENABLE_UPDATE_FIX in notes.ini. If the parameter is not there, you should set it to the maximum for your release minus two.

I've worked in Domino support for nearly five years, and in that time I've learned that the various monitoring tools included with the product tend to be drastically underutilized by customers. I think it's largely a problem of quantity; there are so many tools that it can be hard to know where to begin. As such, I'm going to cover a multipart introduction to Domino monitoring to help administrators get started using this powerful suite of features.

We'll begin by looking at statistics, monitoring's most basic building block. Domino captures a wide range of statistics by default, and these can be viewed in the Administration client on the Server -> Statistics tab or by running the console command "show stat". You can drill down on individual statistics through the Admin client or run a modified version of the console command to retrieve specific statistics (e.g. 'show stat Server.AvailabilityIndex'). These are great tools for getting a quick snapshot of a particular statistic, but most of the value lies in tracking statistics over time, and it certainly isn't practical to refresh the Admin client or run the console command repeatedly.

To solve this problem, you can create a Statistic Collection document that, not surprisingly, collects statistics. Once a statistic collection document is in place, snapshots of statistics from your servers will be saved to statrep.nsf, giving you an archive of prior statistics. A word of warning: statrep.nsf can grow very large if you let it. The database has a default document purge interval of 14 days, and you should think long and hard before changing it. Note also that statrep.nsf is not a replica throughout the domain. If you decide to make statrep.nsf replicate to all servers, make sure you plan accordingly. If you have a lot of servers, it's a near guarantee that this database will be huge.

With a Statistic Collection document in place, you will be able to leverage additional monitoring features that will make your life much easier. And really, isn't that the goal of every administrator? I'll cover one of those additional features--statistic charting--in Part 2.

Several years ago I got a call about a problem with some relatively unburdened servers that had low Server Availability Indexes (SAI). At the time, there really was no simple documentation on how to set this appropriately to yield the desired SAI, so a common approach was to keep setting the Server_Transinfo_Range notes.ini parameter to successively high values until the problem of low SAI went away. This is not a good approach, however, because setting Server_Transinfo_Range too high will result in a server that never thinks it is busy, even when it should.

Here are the instructions we've had for years for tuning SAI:

1. During a period of heavy usage, monitor the expansion factor on your server. You can use the console command "show stat server.expansionfactor" to do this. You can also monitor performance statistics during these periods. Record enough values for the expansion factor during heavy usage so that you can determine the expansion factor value for a fully loaded server.

2. Determine a value for SERVER_TRANSINFO_RANGE so that 2 raised to the power of that value results in the expansion factor value you chose in step 1.

While accurate, many people find these instructions a bit confusing, so I created a table that you can use to make this very easy. Just track the Server.ExpansionFactor statistic for periods of heavy usage and determine what you want the SAI to read for those periods. Find the closest ExpansionFactor in the table and trace left to your desired SAI. If it is not there, find the two numbers closest to your target. Then trace up to the corresponding value you need to set for Server_Transinfo_Range. Hopefully this will make tuning an appropriate SAI much easier for you.

Changing the system clock is one of the most damaging things you can do in a Notes/Domino environment. In particular, setting the clock forward in time and then back to the current time is a scenario that seems to play out much more than it should. Doing this on a Notes or Domino machine can have many serious ramifications that can have lengthy clean-up procedures.

I understand there may be times where you want to test the correct function of some task or agent at some point in the future, perhaps as part of planning for a specific project or business cycle. If this type of testing must be done, make sure it's in a completely disposable environment. I don't even recommend your normal test environment unless you plan on rebuilding it afterward. This is a perfect scenario for a client/server running in a virtual machine. Make a snapshot, do your testing, and then delete it.