Yellow is the New Blog

In anticipation of the release of Domino 8.5, I've been familiarizing myself with Dojo, which thus far has mostly consisted of identifying syntactical differences between it and other frameworks I'm already familiar with. My conclusion is that it's not all that different; one of the two most noticeable differences is the inclusion of additional parameters in the actual markup to indicate to Dojo how certain tags should be handled. In this example, for instance, the author adds to a div a parameter of "dojoType" with a value of "dijit.form.DropDownButton": this tells Dojo that, upon load, it should render something that looks and behaves like a drop-down button where that div is placed, using the innerHTML of the first span it contains as the label for the button. That's fine. I like that: you can determine from the markup how the page should "look" to the user. Granted, I'd rather see them take a semantic approach, but meh.

The other difference, however, is simultaneously cool and troubling. Every "official" example or demo I've seen includes just the base dojo.js file in the markup, then in a custom script loads in everything else the page will need via the dojo.require() syntax. In the previously linked example, for instance, there are function calls similar to the following:

dojo.require("dijit.Dialog";

Why? Because the base Dojo file only includes core functions - such as XHR, array extensions, and the like... as well as a capacity for dynamically loading other script files. Rather than load the entire framework and only use a small fraction of its capability on any given page, dojo.require() allows you to tell the browser precisely what you need and, therefore, only load that. This should enhance performance by minimizing the amount of data the browser needs to download in order to provide the functionality your page requires...

...in theory. In practice, not necessarily. Some have compared this dynamic loading to Java's import statement, and there are certain similarities. When compiling a Java application, import tells the compiler which classes to load, and excludes everything else. Dojo also uses namespacing to provide clarity regarding what is being included. But the difference is that Java is compiled, and JavaScript is interpreted: when you click Save in Eclipse, therefore, it "builds" everything: the resulting class file is aware of everything it needs to be in order for your application to run. A web page, on the other hand, must download each file individually. As I'm sure I've mentioned before (ah, yes, here it is), a powerful way to improve the performance of any web page is to minimize the number of HTTP requests; although Yahoo's list of recommendations keeps growing, that's still their number one guideline. If, for example, you have 10 script files that are 20 KB each, and one 200 KB file that combines all of them, the browser will load the 200 KB file faster than it would load the 10 separate files, even though the same amount of data is exchanged. That's the primary reason I wrote JSFactory and still use it nearly every day (especially lately). Dojo, on the other hand, takes the opposite approach: I added the Dialog example to an application I'm working on (and no other Dojo functionality), and the result was 35 separate HTTP requests, ranging from 3 KB to a mere 84 bytes. I kid you not: one file it loaded was common.js:

({"buttonOk":"OK","buttonCancel":"Cancel","buttonSave":"Save","itemClose":"Close"})

That's the entire script file. That's ridiculous. I'm logged into Domino, so the LtpaToken cookie sent back to the server via a request header allowing it to decide whether or not I have access to that file is larger than the content sent back once it determined that I did have access to it. Dojo has a reputation for being slow, but it's mostly undeserved: I've found that, once the page has loaded, dijit responsiveness is roughly equivalent to widgets in other frameworks. But this dynamic loading that (I assume) is intended to streamline initial page load, in fact, has the opposite effect, and by the time the user is allowed to interact with the page, the impression of "slow" has already been established.

So what's a developer to do? Well, it's easy, actually... again, in theory. Dojo includes a custom build packaging system. So all you have to do is have a copy of the Dojo library on your local computer, write a build profile that designates which modules you would otherwise be importing via dojo.require() statements (heck, you can even include your own custom script files), and then run a shell script (or a batch file, for the majority of you who would be running this from Windows... though the tutorial assumes Linux and mentions the Windows option as an afterthought), and voila: you have a single (combined, compressed, and otherwise optimized) script file that you can then reference in your web page... assuming you've placed it somewhere on the server (either by adding/updating a file resource in a Domino database or moving it directly into the server's folder structure). Sounds a lot like JSFactory, doesn't it (other than automating the step where it updates your Domino file resource)?

But there are a couple minor caveats. First, you must have a copy of the Dojo source (which you can download here), not the built version that's automatically included on the server with Domino 8.5; second, any time the mixture of modules you need (or the content of any custom script) changes, you have to run the build again, and update the server's copy of the file. In other words, this can become very tedious.

Here's what I'd recommend:

• Use Aptana (or your favorite JavaScript IDE; mine is Aptana for various reasons, one of which is that it supports auto-suggest for Dojo and several other frameworks) to write all custom JavaScript code that doesn't need to be generated dynamically. This keeps your source in a folder the Dojo build script can reference.
  
• Create a build profile that includes a reference to all of the Dojo modules that you're using - plus your custom code - which you'll update whenever that module list changes. For bonus points, write an Aptana script that monitors all files in your project for save events and updates the dependency list automatically based on which dojo.require() statements it finds.
  
• If you're running Windows, place a .bat file for each project on your desktop that cd's to the Dojo build scripts folder and runs the build batch using your profile for the project. If you're using the Desktop toolbar, you can then update the build from anywhere with two clicks: one to view your desktop as a menu, and one to launch the build process.
• Create a project profile in JSFactory that includes just the build output file as the source. There's no need to specify a minified output, since Dojo will already have minified the source, but if you've configured your server to support GZIP, you can enable that option. Then with a single click in your Notes 8 sidebar, you can update the desired file resource element with the contents of your latest build.
  

At my earliest convenience (read: February) I'm going to update JSFactory to automate the middle steps, so your process would be: write the code in an IDE, click "Build Selected" in JSFactory... done. And just a reminder: if you've already bought JSFactory, you own it; assuming you kept the download link, it'll automatically point to the new version once that's released.

If you haven't already seen Chris Blatnick's recent post about universal toolbars, now would be a good time to take a gander. Similarly, if you aren't familiar with Nathan's embedded frameset technique, you may want to check that out as well.

I've been working on an application that, like many Notes apps, contains about 20 views. Okay, point taken: unlike most Notes apps, it only  contains about 20 views. But rather than using the traditional hierarchical outline, I decided to try breaking the interface up into visual sections: the outline on the left only contained 5 links, each to an "area" of the application. Each link would load a form that contains a tabbed-table, with each tab containing an embedded view. I demo'ed this interface to the client, and they liked it - it gives them a bit more visual context than the usual "cram a link to every view into the same outline" approach. Particularly because I can toss into the upper right corner of each "area" form a 32x32 version of the 16x16 icon for the outline entry that loads that area. Ties the whole interface together rather nicely, I feel.

This was all well and good until a last-minute requirement implied a need for the last tab of the last area to contain a frameset rather than just a view. No problem, I thought, I'll just embed it using Nathan's approach... I know that works. But once I had, I encountered a limitation I hadn't seen before: since this application also functions as a mail-in database, the reason I wanted a frameset was to allow the users to view the application's Inbox, but also create custom folders for organizing messages the application receives. Despite setting the correct target frame in every location I could think of (frame properties for the folder outline frame, outline properties for the embedded folder outline, even the outline entries themselves), no matter what, clicking on a folder name would launch a new tab instead of replacing the current folder with the new folder in the correct frame.

I ended up using an entirely different approach. This area of the application is now a frameset instead of a form. The top frame contains the form with the tabbed table, but the tabs no longer contain the corresponding embedded views and framesets... just one field each, named predictably: "launchFrameset_X" ( where "X" is the suffix of the frameset associated with the tab; i.e. "frameset-Mail" is associated with the tab containing the field "launchFrameset_Mail" ). The bottom frame contains the "default" view - the one associated with the first tab. Finally (as far as interface is concerned), the frameset is borderless, and the top frame's height is set to a fixed pixel count to obscure everything below the top border of the tabbed table; the result is that it looks identical to how it did before, except that now the default view not only fills the space horizontally, but can fill the available vertical space as well. So far, so good? To get a feel for what this looks like, here's a screenshot of a separate example database I created (which I call "PIM Slap"... I know, always pushing the boundaries of good taste) that uses this technique (click to embiggen):



So we end up with an interface that crams all of our PIM (plus the Notes Welcome page and an internal frameset stolen from Blatnick) into a single tab-driven context. This whole mess can then be nested into a larger frameset - in the case of the application I was previously describing, a frameset like this one is launched via an outline entry associated with its contents.

Finally, to explain how switching tabs actually changes the content of another frame. It's actually fairly straightforward, but I'd never seen this approach used before, so it was a pleasant surprise when the idea dawned on me: when clicking a tab in a form, if that tab contains at least one editable field, the first one Notes "finds" (i.e. not hidden or disabled) gets auto-focused... onFocus is a field event... we can bind to field events. Each tab's field's onFocus event calls a LotusScript function defined in the globals of the form:

Function switchFrame ()
    Dim targetFrameset As String
    
    If (formLoaded) Then
        Let targetFrameset = "frameset-" & Strright(workspace.CurrentDocument.CurrentField, "launchFrame_")
        Call workspace.SetTargetFrame("frame-Target")
        Call workspace.OpenFrameSet(targetFrameset)
    End If
End Function

NOTE: formLoaded is just a Boolean we toggle to True in the postOpen of the form; it seems that, as the form is loading, the first tab is already selected, so its field gets the focus, which triggers the event, but the target frame hasn't been rendered yet, so Notes doesn't know it exists. As a result, we can't allow this event to target the other frame until we know the form has already fully loaded... which is why we load the default view in the bottom frame; we can thus skip loading it via the tab field for now.

In summary, when any tab is clicked, the form auto-focuses the field that tab contains, its onFocus event is fired, which determines which field has focus, what its name is, and therefore what content to load into the bottom frame. As long as all frame names are unique, this allows for fairly deep nesting without any unpleasant side effects. Nothing too revolutionary, but again, I'd never seen this approach used before, so I wanted to share in case anyone else finds this useful.

The example database is available for download.

A brand new addition to the BleedYellow portfolio is a "labs" version of a microblogging application for Domino called "Squawk". Although it has several integration points with Lotus Connections (for example, you do not need to register separately for this application - you can use your existing BleedYellow credentials - and your profile picture in Squawk is whatever picture, if any, you've uploaded to your BleedYellow profile), the application itself runs entirely on Domino.

Originally envisioned purely as a proof of concept demonstrating that Domino applications can scale massively if architected well, we've decided to host it on an ongoing basis as a BleedYellow service. And, when the time comes, if you'd like to purchase a copy to run in your own intra/extranet (with or without Connections integration)... I suspect we can arrange that.

Squawk is still in the alpha stages, but we wanted to share it with you in its infancy to make you a part of its development. Here's a brief summary of what currently works and what will at a later date:

• Post a Squawk: like most micro-blogging applications, each post is limited to 140 characters.
  
• Community Squawks: this is a global transcript of the last 20 Squawks from anyone in the BleedYellow community.
• My Flock: search for people in the community; each search result will include the individual's name and profile picture (the Connections default image if they haven't uploaded one yet), and a subscribe/unsubscribe link based on whether or not they're already in your Flock. An icon for each member of your Flock will appear in a block on the right; hover over their icon to see their name, click their icon to view their latest Squawks.
• Various links (Profile, Settings, Home, etc.) are either just placeholders or, for the time being, unnecessary: for example, the Home link is a bit superfluous, because at the moment, Squawk is a single-page Ajax application.
  
• My Replies: we haven't wired these in yet, but at the moment our thinking is that @shortname will denote a Squawk as a reply to the individual with the matching shortname. We'll also be adding a reply icon that will automatically insert the shortname for you. But we're also going to be adding nicknames, so if you want to type @trip, for example, to reply to me, you can tell Squawk to treat that as a nickname for me. If you've defined a nickname for a member of your Flock, clicking the reply icon will insert the nickname you use for them instead of the default.
  
• In a related feature, at some point we'll also be allowing you to create groups from your Flock and direct a Squawk just to a specific group or individual: for example, "hello world #test !dxl" would tag the Squawk as "test" and it would only display to members of my "dxl" group. We're also going to allow you to load a default set of groups based on which BleedYellow communities you're a member of and who else is a member of each.
• Speaking of tagging, tags aren't clickable yet but of course will be later... in the meantime (and going forward), if you tag a Squawk with #bug or #feature, we'll use that feedback to improve this application.

So head on over to squawk.bleedyellow.com and check it out.

P.S. Interesting bit of trivia: Squawk instantiates a single view handle when the Community Squawks transcript is requested; searching for people to add to your Flock naturally performs a full-text search... every other function in Squawk ignores indexes entirely.

A deeply ingrained habit of mine is using ampersand to concatenate strings in LotusScript (when I'm not bypassing operators entirely, such as using NotesStream or StringBuffer to optimize massive concatenations) - so deeply ingrained that when NTF asked me today why I don't use plus instead, for a moment I couldn't remember my original reason for switching... but after mulling it over briefly I remembered that I hate type coercion with a passion. It bugs me because it tries to be too smart and often fails. Just like extended syntax. For example:

"100" + "200" = "100200"
This may seem obvious: we're concatenating 2 strings, so it takes the value of one and appends the other. But...
100 + "200" = 300
The LotusScript interpreter decides that, since it's starting with a numeric value, you're obviously trying to add another numeric value. Since in this case the string can be converted to a number, it does so and adds the operands instead of concatenating them as strings. So, logically:
"100" + "200" = ...300?
That's right: plus is primarily an addition operator, so if there's any possible way it can remain one, it will. In fact:
100 + "two hundred" = type mismatch - at compile time if it knows the latter is a string, at runtime if it's a variant... reason #372 why variants should be used sparingly and with caution. Speaking of which, I think it was Nathan who once told me, "Variant should have been called Deviant... 'cause it'll accept any object". Yeah, I know, ew. But you gotta admit, that's darned funny.

So how did he convince me plus is better than ampersand despite my pedantic aversion to coercion?

1. (Author's note: this was actually a third realization, which came after the discussion.) In LotusScript string concatenation, some coercion is unavoidable. Since ampersand's only valid use is as a concatenation operator, any scalar entering the operation is treated as a string whether you wanted it to be or not. So unless everything's a string to begin with, it's coercing anyway, it's just safer. But if we're subscribing to a "flexible input, strict output" philosophy, we're already making sure we know what data type we're dealing with and handling it appropriately, so ampersand doesn't really gain us much... it's only when we're being lazy about this that ampersand saves us from returning unexpected results.
   
2. Whenever possible, it's helpful to use conventions that are optional in one language but required in another. In Formula, but - more importantly, over the long haul - also in JavaScript and Java, we don't have a choice: we have to concatenate with plus; might as well do it in LotusScript too so that we're spending more time thinking about how to implement the actual functionality at hand and less time trying to remember what operators to use in the current language... same reason I always surround my conditional expressions in parentheses: I know LotusScript doesn't force me to, but JavaScript and Java do, so why erode a good habit just because LotusScript would allow me to be lazier than that?
3. In XML, an ampersand must be entity-encoded; a plus can be left alone. This was the "you had me at hello" argument. For the foreseeable future, any convention that allows my code to be handled more easily within an XML context is likely to appeal to me, even if it forces me to abandon a long-held alternate approach.

Anything you can think of that I'm overlooking? If not, I'm switching to plus effective immediately. That is, as soon as I get back to writing LotusScript... it's been nothin' but Java for me all week.

This entry is related, but varies slightly from, Bob Balfe's article on debugging Eclipse plugins in Notes 8. My current hobby project is written entirely in Java, but it's not an Eclipse plugin - at least, not yet... once the application's API is finalized, the code I'm writing now will find its way into a plugin, a web service, a server-side JavaScript library for use in one or more XPages, and so on. But before I get carried away with any of that, I first need to get the core functions working.

I'm still primarily a LotusScript developer, so I'm accustomed to relying heavily on the LotusScript debugger to tell me why my code isn't behaving precisely the way I'd imagined it in my dreams... just kidding, I don't dream in LotusScript. Okay, there was that one time, but trust me... you don't want me to tell that story. Anyway, I've long since adjusted to adding semicolons at the end of each line, curly braces to open and close conditional blocks, and the like... JavaScript's optional preference for same has, over the course of several years, gradually prepared me for Java's non-negotiable stance on such things. But not having as informative of a debugger was really starting to bug me (rimshot). Besides, my testing process had grown rather cumbersome:

1. Write the code in Eclipse (so I can actually get typeahead on both imported and custom classes, and easily integrate with SVN - not only so I can roll back to a previous version if I completely mangle something, but also to take advantage of Nathan's frequent midnight mindstorms).
2. Export to a JAR.
3. Open the test agent in Designer.
4. Delete the JAR from the project and save it (since there's no "refresh" option to force the design element to notice that the local file has changed... I was going to write a separate agent to recreate the test agent using DXL by Base64-encoding the JAR and... yeah, ew.), wait for Designer to squawk at me that it could no longer compile, click Yes to save it anyway, add the JAR back in, save again.
5. Run the agent.
6. Check the three databases that should have new / modified documents if the code worked, and verify that all items were written correctly.
7. Open / switch to the Java Debug Console to sift through ridiculously verbose logging and stack traces for some hint of why one item value had been set incorrectly.
8. Return to step 1, hopefully now aware of which precise line of code needs to be updated - and how - in order to fix the bug. Repeat as long as necessary.
   

As my friend Sean Kelly would put it, "pain... you know it isn't easy". Here's my new process:

1. Write the code in Eclipse.
2. Open the class file that defines the unit test I want to run.
3. Click the cute little bug icon in the Eclipse toolbar to launch debugger, then step into and over until it crashes or completes.
4. Generally speaking, at this point I don't even need to switch back to Notes, because if it didn't work, I saw it go wrong during the debug run, so I know exactly where to look, even if I wasn't logging a durned thing. Return to step 1, repeat as long as necessary.
   

What changed? Surprisingly little:

• I had to add the Notes program file ( on this laptop, that's just "C:\Notes" ) to the PATH system environment variable. And, though I didn't realize this until later, I had to reboot for that to take effect.
• While normally I'd just include a standard JRE in my project build path, I changed this project's build path to point to an "alternate JRE": namely, the one included in Notes 8. To specify the Notes JRE as an alternate in Eclipse, select Window > Preferences, then navigate to Java > Installed JREs. Click Add, choose "Standard VM" as the type ( and click Next ), then give it a name ( like "Notes 8" ), and set the JRE home to "[Notes Program Folder]\framework\rcp\eclipse\plugins\[OS-specific folder]\jre", then click Finish. In my case, the [OS-specific folder] is "com.ibm.rcp.j2se.win32.x86_1.6.0.20080709-200808010926".
• Here's the crucial part: the class has to be runnable, so it needs to be treated like a command-line Java class - namely, it needs a static void "main" method - and it needs to start and stop a Notes thread ( I'd also recommend syncing your Notes and OS passwords so that you can skip being prompted for your password when the thread is started ).
  

For example:

public static void main(String[] args) {
    NotesThread.sinitThread();
    // insert here the code that runs your unit test
    NotesThread.stermThread();
}

That's all it takes. I now have a class for each of my unit tests. So I have one that creates a document, another that formats a summary of a document collection as JSON, and so on. Whenever I make any code changes, I run the unit tests for verifying any functionality that is impacted by the change I just made... so, without ever leaving Eclipse, I know right away whether or not I've just mangled something... and if so, what needs to change.

In total, I submitted (or am listed as a speaker on) 5 abstracts for Lotusphere this year:

• "DXL in the Wild": a BOF co-moderated with Mac Guidera
• "Dispelling the Myth: techniques for architecting massively scalable Domino applications"
• 3 joint sessions with Chris Toohey:
• "Integrate with Anything!"
• "Fixing it with Brokers!"
• "XPage Yourself!: High Performance Domino Application Development"

If you'd like to see any (or, perhaps, all?) of the BP sessions I submitted, please use the links above to sound off over at ideaJam.

I was contacted last week about a comment I had left on Bruce's site stating that I had code for assigning a database a new replica ID. The comment in question was an allusion to some code I'd originally found on the Breaking Par site that uses some Notes API calls to overwrite a database's replica ID. Last night I threw together a utility that makes use of this technique to provide two quick operations:

• Link Replicas: select a source (the database whose replica ID you want to retain) and a target (the database that you want to change to be a replica of the source) and click "Establish Link". The target database will be made a replica of the source database (and any additional replicas of the source).
• Unlink Replica: select a target database that you no longer want to be a replica and click "Unlink Replica". The target database will be assigned a new replica ID, effectively making it a copy of any databases it was previously a replica of.

Since the theme to all of this is replication, I called this utility Nexus, and you can download it here. You might notice the interface has a hint of an LCARS feel to it... the Sci Fi channel was running a TNG marathon yesterday, so Laura and I chose to relax (I got home at 3 AM Saturday morning from a business trip, we went to a family reunion in Gumlog, GA Saturday night, and spent part of Sunday and Monday at various state parks, so we were both a bit wiped) by sitting on the couch for six hours straight watching Star Trek. And then I wrote some code. So yeah, I'm a pretty content little geek.

For the purposes of this discussion, I'm temporarily stepping out of my typical role of "code monkey" and switching to my alternate (but equally important) role as system administrator. Having seen several different approaches to provisioning server connections in Notes, I'm curious how you approach it. Here are a couple examples of what I've seen in different environments:

• During workstation setup, the DNS address of the user's home/mail server is entered, establishing a connection to that server. From then on, attempts to access other servers rely on the ability of Notes to automatically request the destination server's address from the user's home server; as long as that server is accessible, the user doesn't need connections to any other servers, because their home server will tell them how to get there. NOTE: this only works if the home server can determine the destination server's address... if the user needs to get to servers not listed in the primary Domino directory (for example, servers in isolated test or development environments), this feature does them no good.
  
• During workstation setup, whoever is configuring Notes (or walking the user or a delegate through the process) manually defines connection documents to every server the user could ever possibly need to access. This precludes the need for the home server to be contacted to request an address the first time (per session) a new server is accessed, but if any server's network address changes, in addition to updating it in the Domino directory, the change must also somehow be pushed to all the users.
• When a user needs to access a server they don't have a connection for, an administrator sends them an email that includes a button that, when clicked by the user, programmatically creates the connection document. Typically, then, the admin doesn't know the user doesn't already have a connection until the user has already tried to connect and failed.
  
• When a user needs to access a server they don't have a connection for, they are instructed to enter the server's network address - instead of its name - in the "Open Application" ("Open Database", in versions prior to 8) dialog and click "Open", which automatically creates a connection document if they don't already have one. Hence, users don't need to be sent a new button each time they identify a new server they need to connect to; they just need to know its network address.
  
• Domino servers that are only used behind a firewall are named such that internal DNS (or WINS, or another equivalent) allows the Notes client to locate the server by its CN without even requiring a connection document. While this might lead to either Domino servers with ridiculous names the users have little hope of remembering (i.e. TNGATLDM37 might be the 37th Lotus Domino mail server added to a facility in Gatlinburg, Tennessee) in order to conform to an existing DNS naming convention, or require a compromise from the network staff (perhaps in the form of an alias mapping a "friendly" Domino name to a conforming name), this is my personal favorite. The users don't get the impression that Domino - or the network - is unreliable because they can't get to a server that they've never told Notes how to find, Notes doesn't have to query their mail server for a destination address every time they want to open an application on a server that they access infrequently, and neither admin nor user has to spend time configuring connection documents unless they're a remote user... and, while remote and offline access has long been a key strength of Notes/Domino, in the majority of organizations (in my experience, at least), the majority of users need neither.

What's your take on this? Are there other approaches you take to ensuring your users can find Domino servers? Are there specific reasons why you prefer one approach over another?

Yahoo! has done extensive research on how to optimize web site performance, and for some time now has maintained a list of best practices that summarize the results of that research, along with detailed descriptions of why each guideline enhances site performance. Two of their guidelines that go hand in hand are as follows:

• Put stylesheets at the top: the closer to the top of the HEAD your stylesheets are, the more progressively the page will render. Strictly speaking, this doesn't allow the page to load more rapidly, but the progressive rendering speeds up the display of the page once it has loaded.
• Put scripts at the bottom: this may seem counter-intuitive, especially if you're used to cramming all of your script into the JSHeader object in a Domino design element (.....please don't) instead of defining it in libraries or file resources that are then loaded via script tags, but script should be as close to the bottom of the BODY as possible.

Check out Yahoo!'s list for more detail on why they recommend these (and other practices) for maximizing site performance. Thus far I haven't experienced anything in my own development that contradicts their recommendations.