starjuice - Home

Improve readability with 100E2R

2008-06-15T15:45:00Z

You know those things you stumble upon online, that inspire you to rethink the way you handle an aspect of your work?

I've struck upon one such inspiration nugget in 100E2R, The 100% Easy-2-Read Standard. It's a set of 5 complaints and 5 rules for web publishers. The page on which it's presented also includes an interesting list of referrers and some thought provoking commentary.

It promotes the design choices that I now realize underpin the web content that jumps out at me and inspires me to produce more beautiful applications.

So if you've also been scratching your head and trying to figure out why you can't get "that web 2.0 look" right, even though you're using the Trebuchet MS font and the grid implementation du jour, be sure to feast your mind on The 100% Easy-2-Read Standard.

I recommend reading the comments as well. Both the criticism and affirmation will get you thinking (for yourself). The list of referrers provides some great terrain for exploration.

Model, view or controller?

2008-06-14T19:48:00Z

This came up on the rubyonrails-talk mailing list again, and it took me too long to find my answer from last time in the archives, so here it is.

Sometimes it's hard to decide which layer of an MVC application to put code in.

It took me a long time to settle on a simple decision process that works for me, especially in the J2EE environment where it's not as easy to change your mind.

I default to trying to push code down into the model. However, I ask myself these two questions:

Does this code describe anything other than behavior of this model?
Does this code know about CGI or HTML?

If the answer to either of those questions is yes, I look more seriously at putting the code in the controller or an helper. If the answer is "yes and no", I'm probably trying to do too much at once, and need to break it up across the layers.

In summary, favor the model, but not at the cost of layering violations.

The attitude that this comes from is that your domain model (your collection of "models") is the system. The controller is only there to mediate a view into it. This attitude works well for the kinds of applications Rails is suited to.

Here are two more considerations:

Code that you put in the model is easy to use from the Rails console and runner. Code that's in the controller is more of a pain in the arse to get to.
Model instances can call each others' methods, but they can't call methods on controllers without jumping through hoops.

Say no to ProFTPD

2008-01-15T08:06:00Z

It looks like people are still recommending ProFTPD. I wish they wouldn't. This isn't news. It's an opinion I find myself repeating frequently, and now I'll be able to cut'n'paste or provide a URL as appropriate.

Search the National Vulnerability Database for vulnerabilities disclosed against vsftpd and ProFTPD during 2004, 2005 and 2007. You could search all of time, but that would give you a skewed view, because ProFTPD is older than vsftpd, and because you need to allow for software becoming more secure over time.

You'll find 8 vulnerabilities in ProFTPD. One of them is an authentication bypass vulnerability. The rest are stack and buffer overflows that allow denial of service attacks and remote code execution.

You won't find any vulnerabilities for vsftpd.

I used to buy the argument that ProFTPD is more feature rich. But recently, I set up vsftpd for a customer, providing chrooted sessions for virtual users. It was a lot less hassle than it is with ProFTPD.

Interestingly, vsftpd is (at the time of writing) the FTP server software used by kernel.org, the Linux kernel distribution site. From the perspective of a black hat, there aren't many non-commercial sites that would be more useful to break into, so their threat exposure is probably high.

Say no to FTP. If you can't, say no to ProFTPD.

Pinning down Etch backports

2007-09-30T08:05:00Z

For a while, I've wanted kde-3.5.7 on my Debian 4.0 (Etch) laptop. It's available in the etch-backports archive from backports.org. But the etch-backports archive contains a lot of other backported packages, and I want to stick to Etch with the exception of KDE.

You may ask why I don't just accept all packages from the etch-backports archive. There are two reasons: stability and security.

With respect to stability, I reckon that the packages in the etch archive have probably been exposed to the largest number of users. Assuming that a wider user base leads to more bug reports and fixes, these packages would be the most stable. That assumption doesn't always hold, but it's a good generalisation, as generalisation's go.

An enormous amount of effort goes into producing a stable Debian release, and I want to maximise my leverage of that effort. As I become more of a manager, my interest in a bleeding edge laptop wanes.

As for security, the Debian Security Team FAQ suggests that the testing archive can receive security updates less frequently than the etch archive. Since the general backports.org policy is to backport only from the testing archive, I figure I'd wait longer for security updates from the etch-backports archive.

So for the most part, I want packages from the etch archive. For KDE, a suite of hundreds of packages, I want what's in the etch-backports archive.

The solution, as presented by the instructions at backports.org, is to use APT preferences to pin specific packages to the etch-backports archive. Unfortunately, the format of the APT preferences file requires that you pin every single package of the KDE suite. With a full KDE installation comprising over 200 packages.

This seemed like it would be a lot of work, especially since I couldn't find the list of packages for the KDE suite in a single list. Rather, I'd have to navigate an extensive dependency tree, ignoring dependencies that aren't part of the KDE suite.

However, I noticed that all the packages from the KDE suite contain the same text in their long descriptions:

This package is part of KDE

How fortunate. Especially since the dctrl-tools package provides a way to search for packages matching against any header in the package control file.

So after following the instructions at backports.org to add the etch-backports archive to my APT sources and update my package index files from their sources, here's what I did to tell APT to accept KDE packages, but nothing else, from etch-backports:

aptitude install dctrl-tools

grep-aptavail -ns Package -F Description \
       "This package is part of KDE" | sort | \
   for pkg in `cat`; do
       echo
       echo Package: $pkg
       echo Pin: release a=etch-backports
       echo Pin-Priority: 999
   done >> /etc/apt/preferences

aptitude dist-upgrade

The grep-aptavail options instruct it to print out the value of the Package header (but not the name of the header) for all packages available to APT with a Description header that contains "This package is part of KDE". In short, it prints a list of all packages in the KDE suite. The output is then sorted, transformed into a set of APT preference records, and appended to the APT preferences.

Now I can rest assured that any packages I install from the KDE suite will be installed from the etch-backports archive, but other packages available in etch-backports will not be installed unless I deliberately pin them too.

The same approach works for OpenOffice.org. In fact, it's likely to work for any suite of software where the package maintainers have done something sensible with the package descriptions. All packages in the OpenOffice.org suite contain this text in their descriptions:

OpenOffice.org is a full-featured office productivity suite

Now you can have the toys that the cool kids have, without being cool.

Aptana takes over RadRails development

2007-03-11T08:04:00Z

For those of you who've been alarmed by the apparent stalling of the RadRails project, there is hope. I don't usually post news items, but I haven't seen any English coverage of this at RubyCorner, and I know some of you have been worrying that your favourite Ruby and/or Rails IDE is abandonware.

Fear not! Aptana is taking over development of RadRails, after Kyle Shank recently announced that he and his business partner just don't have time for it right now.

For details, see Kyle Shank's announcement, which includes links to a CRN article on how the deal came together and a discussion forum at Aptana's site.

The fundamental flaw in PC security

2007-03-08T08:03:00Z

Heard in the antispam community (and reproduced with permission):

The fundamental flaw in the idea of DRM is that it's not possible to simultaneously show something to someone, and not show it to them.

Is the fundamental flaw in PC security that it's not possible to simultaneously allow users to execute arbitrary code (or make arbitrary network connections, or whatever) and not allow them to?

Huey Callison

Astute, I thought. I've known for ages that the RIAA is ripped off repeatedly by crackpots who claim to have the final ultimate solution to digital rights management, because of this problem. But it had never occurred to me that PC security is the same class of problem.

Until users can't execute arbitrary code on their own personal computers, the security of those computers is a measures / counter-measures game at best.

This is worth bringing up next time someone in the board room laughs at you for suggesting you deliver application functionality via the web.

Surviving the expat upgrade on Gentoo

2007-02-28T08:00:00Z

Moving to ~x86 on Gentoo Linux requires an upgrade to dev-libs/expat-2.0.0 from dev-libs/expat-1.95.8.

Unfortunately, the expat maintainers broke its ABI between 1.95.8 and 2.0.0 without bumping the library version number. This means the Gentoo ebuild maintainer can't easily slot the versions for happy coexistence.

Whether you've already upgraded dev-libs/expat and are now stuck, or you're still using a version earlier than 2.0.0 and are about to upgrade, this will get you over the hump:

cd /usr/lib
emerge -1 '<expat-2'
cp libexpat.so.1.5.0 libexpat-1.95.x.so.1.5.0
emerge -1 '>=expat-2'
ln -fs libexpat-1.95.x.so.1.5.0 libexpat.so.0

Then, one day when your initial ~x86 upgrade is complete, you can go back and remove libexpat-1.95.x.so.1.5.0 and libexpat.so.0 from /usr/lib once you've used revdep-rebuild --library=libexpat.so.0 -pv to verify that the pre-release library has no remaining dependencies.

You can't have the spam you don't pay for

2007-02-27T07:48:00Z

Love him or hate him, John Levine sure knows how to illustrate a point.

Recently, John was participating in a conversation in which someone asserted that if a recipient wants to receive all the spam sent her way, her email provider should allow for that.

Here's John's response, which beautifully nails down the overwhelming influence of cost on email providers' spam filtering policies:

Hi. I belong to a club of people who are trying to prove the existence of God by finding patterns in random data. To do this we need to mail around 100GB files of white noise. They never show up, and when I call my ISP to complain, all I get is gobbledygook about limits to attached sighs or something like that.

If I'm paying for the account, they should be allowing it thru.

Sure, baby, as long as you don't birth a cow when you receive your next bill.

Saving Uploaded Images to the Database

2006-08-12T17:05:00Z

Judging from the questions people ask on the #rubyonrails channel, it seems that it's still a common practice to store uploaded images in the database.

At first glance, it seems sensible. You're storing all the metadata about the image in the database; its filename, its accessibility, its dimensions, when was it uploaded. Why not put everything about the image (including the bits of the file itself) in one place?

The approach even makes sense when you start to think about backups. By storing the images in the database, you can rest assured that your database backups will include the images, and that the images and metadata will be consistent in those backups.

It's an idea that I've found attractive myself. But I discussed this with Rob, and he came up with some really solid arguments against it:

The images consume storage space in the database. Some databases, such as Oracle, require preallocation of physical storage into table spaces, so you want to reduce unnecessary disk consumption.
Connection hogging on the database. One of the most hotly contested resources in an RDBMS is availability of network connections. When your application has to retrieve images from the database, you tie up precious database connections for longer.
The database has little or no value to add to images. With non-binary columns, your data pays for its space in the database with its usefulness in queries, joins and transforms. Binary columns don't pay for themselves in this way. If the image is going to be manipulated after the initial store, the manipulation will happen outside the database.
The operating system's filesystem is highly optimised for caching and delivery of files. In addition, your application server (Rails) or application server proxy (Apache) may also offer caching benefits.

Once you've considered this, the only benefits of storing images in the database are that they're centrally accessible, covered by your database backups, and easy to code for. But these benefits are weak in the face of the arguments above.

As far as central accessibility is concerned, all modern operating systems support some kind of network filesystem. The central accessibility goal is reached simply by exporting the image store as a network filesystem like CIFS or NFS.

When it comes to backups, production servers should almost certainly have a backup regimen that includes more than just the database. Since you should already have at least some paths in your filesystem backed up regularly, it's almost no additional effort to include images in the filesystem backup regimen.

The last remaining benefit is ease of coding. And yes, I imagine that in some application frameworks, it requires more work to store images outside the database. However, put this aside for a moment, and look at everything else we've looked at so far. If an RDMS is a bad fit for image storage, why isn't your application framework making it easy for you to store your images outside the database?

For Ruby on Rails developers, there are at least two rock-solid APIs for attaching images (and other binary files) to your models in a way that's so transparent, you couldn't tell from the code that the images are not stored in the database. For Singles Everywhere (a free online singles / dating site I recently completed for a customer), I used the file_column plugin, which was a pleasure to work with. There's also techno weenie's Acts as attachment plugin.

I suppose there may be cases where it makes more sense to put the images in the database. But I recommend that your default preference be for external storage, and that you insist on strong motivation that outweighs the concerns outlined above.

The Baby in the Java Bath

2006-08-11T12:06:00Z

A few years ago, a really bright developer persuaded me to take Java seriously. I did, and I fell in love.

Looking back, though, I see that I didn't fall in love with Java. I fell in love with object orientation, with design patterns, with agile development and with test-driven development.

As a procedural C and Perl programmer, I hadn't been exposed to these things. I hadn't recognized that I needed them. I certainly knew something was wrong. I just could never get to the end of a software project and feel I'd done a solid job, and that extending the software would be a pleasure.

In the Java world, these things are mandatory. You simply can't survive without them. So you can't work with enterprise Java very long before being exposed to them. And for that, I'm truly grateful, both to Rob, who encouraged me to explore Java, and to the Java community.

Nevertheless, the enterprise Java environment left me unfulfilled, and I now realize that we were never lovers, Java and I. I don't like its rigidity, and am horrified by the complexity of the platform environment mandated by that rigidity.

Last year, I got into Rails with a vengeance. My time in enterprise Java land stood me in great stead. I understood the MVC framework, felt comfortable with most of the patterns codified into Rails, and appreciated why Rails wanted certain things done certain ways.

As a software developer, it feels like I've found my one true love. Ruby is a phenomenally expressive language. Its writability is one thing, but its readability is outstanding. It's beautifully expressive, and manages to provide powerful flexibility without the need to pay for that power when doing simple things.

For me, here's the proof that I've found home. I was a licensed user of IntelliJ IDEA, and was convinced that I would never be able to live without an IDE again, because of its powerful support for refactoring and code analysis. This, after being a die-hard "Vi should be enough" C and Perl hacker.

The dynamism of Ruby is such that I believe the same level of IDE support isn't and simply never will be available for Ruby. But here's the kicker. It doesn't matter. I don't miss my IDE, because Ruby and Rails is so clean, so consistent, so elegant that I just don't need IDE assistance to be productive.

Having successfully deployed Singles Everywhere to an highly available, scalable cluster, I don't buy the FUD that Rails doesn't scale. It does, and a damn-site more elegantly than JBOSS, Tomcat and friends. The thing with Rails is, you don't pay for scalability until it's time to scale.

Singles Everywhere has some surprisingly complex background processing to deal with. When I set up the cron jobs to handle it, the last lingering glow of my infatuation with J2EE flickered out for good as I realized how ridiculous J2EE has become.

You develop a we application. You have some processes that need to run outside request scope. So you set up an application container, add Quartz, set up all sorts of contexts and... Hold on a minute. This is running on a Unix server! How did we get so far abstracted from the usefulness of the operating system that we lost access to the system scheduler? And why isn't it a simple job to fire up a process that runs in the context of the application from that scheduler?

At that point, Java finally went cold for me. I'm done. Thanks for the beautiful babies, but from now on, we'll bath where we don't have to assemble and excite the H2O molecules ourselves.

When to Use Libraries: The Pragmatic Answer

2003-09-23T16:31:00Z

Tom Klaasen recently suggested using the IOUtil library to ease the work Java developers must do to create a String that contains the entire content of an InputStream. His suggestion looks set to spark some lively discussion.

I believe the discussion is a subtype of a more general argument, one that I don't feel deserves the time it enjoys.

First, there are times when it makes sense to store the entire output of a stream in a String. One very good example is the case where you'd like to execute fairly complex regular expressions over the entire String.

Once you've accepted that there are times when this is a necessary operation, the rest of the argument boils down to whether or not to use APIs. Not a particularly fresh debate, but one we don't seem to have been able to put behind us yet, sadly.

I think it's simple; if an API meets your needs and pays for itself, use it. If not, roll your own.

In this particular example, the conversion requires 7 lines of code:

private String getInputStreamAsString(PlainTextInputStream urlInput)
       throws IOException
{
   // The size of the buffer depends on your application.  Either
   // size it to the typical smallest input size or generalize
   // this method to take the size as an argument.
   //
   int readSize = 512;
   byte[] buffer = new byte[readSize];

   int bytesRead;
   StringBuffer urlContent = new StringBuffer(readSize);
   while ((bytesRead = urlInput.read(buffer)) != -1) {
	urlContent.append(new String(buffer, 0, bytesRead));
   }
   String templateText = urlContent.toString();

   return templateText;
}

I'd say that an API would have to be solving more problems than just this one to pay for itself.

The code fragment above comes from a component I wrote, which has to grab and manipulate content from a remote web page. At first, I home-rolled my 7 lines of code and I believe that I made the right decision.

Then, I realized that getInputStreamAsString() was a necessary artifact of my decision to fetch web page content using the despicable java.net.URL and its nefarious henchmen.

When I decided to use Jakarta Commons httpclient instead, I bade a none-too-fond farewell to not only this little home-rolled carrot, but also a good 20 lines of ugly code surrounding the creation and interaction with the URL connection. I believe that this was also the right decision, and the change was painless because I originally encapsulated the dirty work properly.

My point is that it comes down to whether software pays for itself, and that whether or not it does so may well change through the lifecycle of the system. The important thing is to design your software in a way that allows you to change your mind later.

I think a lot of people don't get this because they have the time to fixate on tiny issues of principle. As you get older, you realize how short life is. That process sharpens your sensitivity to the importance of pay-off.

This reminds me, yet again, that I simply must write a review of Martin Fowler's book, "Refactoring: Improving the design of existing code", soon. Stay tuned.

J2EE Service Wrapper Pattern Candidate

2003-09-09T13:58:00Z

Recently, I was tasked with converting a J2EE application's EJB references from remotes to locals, as part of the run-up to initial launch deployment on a single VM. The application features 24 entity beans and 50 session beans, all hand-coded without the use of code generation tools like xdoclet. The exercise made for one of the worst weeks of my working career.

It wasn't the pain of bringing unmaintained local interfaces up to speed with changes made only to the remote interfaces and beans. Nor was it the gruelling effort of tidying up imports and catch clauses for unthrown exceptions after the scripted regular expression edits on client code. It wasn't even the changes that couldn't be tested at compile time, like replacing casts. No, all of this was bearable.

The real soul sucker was knowing that I wasn't making it any easier to handle the undoing of the changes one day when the application outgrows a single VM.

This seems to be a problem that isn't addressed by a published J2EE design pattern. We have the Service Locator pattern, which encapsulates the details of acquiring a reference to a service in a distributed environment. What's missing is a Service Wrapper, that encapsulates the locality of a service.

I think this is a problem to which we should apply the OO design mantra, "Encapsulate the concept that varies." What varies between the use of local and remote references to EJB homes and EJB objects?

The methods on an EJB remote interface must throw javax.rmi.RemoteException, but the methods on an EJB local interface must not throw it. This means that you cannot derive both interfaces from a common ancestor, by which clients refer to either one. So the client is exposed to the locality of EJB object references. When you change from using local interfaces to remote interfaces (or vice versa), you have to add (or remove) exception handling from the client code.
Certain methods on EJB local homes must return EJB local interfaces, while those on EJB remote homes must return EJB remote interfaces. Since client code must know the locality these returned interfaces (as explained above), it must know the locality of the EJB homes as well.

Were it not for these differences, client code would be able to interact with remote and local interfaces interchangeably without modification. While type-based conditional behavior could be introduced into the client, this would result in serious code duplication and degradation of business focus in the client code. So that's not the answer. But these differences can be encapsulated, and I'm suggesting that, in sufficiently large systems, they should be.

Right, enough nancying about.

What locality-related implementation specifics are we trying to isolate client code from?

Variant return types. We can address variant return types by introducing wrappers around locality-biased types and returning those wrappers instead.
variant exception handling. We can address variant exception throwing by handling the deviant exception, RemoteException, in the wrappers.

This kind of wrapping is a realization of the Proxy pattern. What proxying wrappers must be introduced to guarantee isolation of client code from locality-related implementation specifics?

ObjectWrapper. This class encapsulates the locality of an EJB interface. It contains either a remote or local EJB interface, and proxies all method requests to that interface. The implementation of this wrapper is necessarily ugly, but protects client code from type-based conditional behavior. In particular, when a remote EJB interface is used, the wrapper must rethrow RemoteException as an application exception.
HomeWrapper. This class encapsulates the locality of an EJB home. It contains either a remote or local EJB home, and proxies all method requests to that home. The implementation of this wrapper is also ugly. Not only must it rethrow RemoteException, but it must also wrap in ObjectWrappers, any EJB interfaces returned by the methods on the contained EJB home, and then return those ObjectWrappers instead.

What's are our points of entry?

We use the Service Locator as a single point of entry into this layer of encapsulation.
The Service Locator is adjusted to return HomeWrappers instead of EJB remote or local homes. Whether the EJB home contained within any given HomeWrapper should be remote or local can be hardcoded into the Service Locator, or determined from a locality map of arbitrary complexity (e.g. properties file, LDAP tree).
The HomeWrappers return ObjectWrappers instead of EJB remote or local interfaces. The encapsulated locality of the HomeWrapper determines the locality of the EJB interface to be burried within an ObjectWrapper.
Both wrappers isolate client code from RemoteException.
Client code is adjusted to work with the wrappers instead of remote and/or local homes.
The only parts of the system to which EJB remote or local homes and EJB remote or local interfaces are exposed, are the Service Locator and the Service Wrapper layer, consisting of the HomeWrappers and ObjectWrappers.

Note that two very different animals are being proxied here; EJB homes and EJB interfaces. But for each one, the same concept is being encapsulated: locality. So even though I'm proposing the introduction of a layer of indirection over two things, they form a single conceptual layer, and together comprise the Service Wrapper. Indeed, introducing one wrapper without the other doesn't solve the problem at all.

A potential point of controversy is the encapsulation of exception handling. I don't see this as a problem for two reasons:

Based on my project exposure and the code excerpts I've seen from other projects and literature, it seems that most EJB applications don't respond to RemoteException with any degree of intelligence. They either allow the exception to percolate up to the client unmolested, or wrap and rethrow them in a manner that contributes nothing to the cause of failure recovery. At best, they wrap and rethrow in a way that makes it easier for developers to see the context in which failure occurred.
Encapsulation of exception handling within the Service Wrapper layer can actually have failure recovery benefits. This is because the layer introduces a convenient place to put failure recovery code that would otherwise need to be scattered throughout client code. For example, the Service Wrapper layer could respond to transient network partitioning by implementing a retry algorithm, which could completely isolate clients from a significant subset of such errors, such as rebooted routing devices and remote application servers.

The class diagram below shows the introduction of one HomeWrapper and one ObjectWrapper for every EJB located through the Service Locator. This significantly increases the amount of grunt work required to write participating EJBs by hand. However, code generation tools such as xdoclet have gained tremendous support, and it wouldn't take much effort to reduce the labor to a few xdoclet tags. In fact, in the project that sparked off this line of thinking, I'm waiting for The Great XDocletization subproject to complete before applying this approach. It's just too much work to hand-code for over 70 EJBs.

In the interests of simplicity, I've only shown delegation of a single home method, create(). The important features are:

ServiceLocator creates instances of HomeWrapper, populating either localHome or remoteHome and setting isLocal accordingly.
HomeWrapper delegates to localHome or remoteHome based on the value of isLocal. This is ugly, and may benefit from application of the State pattern.
HomeWrapper creates instances of ObjectWrapper, populating either localObject or remoteObject and setting isLocal accordingly.
ObjectWrapper delegates to localObject or remoteObject based on the value of isLocal. This is ugly, and may benefit from application of the State pattern.

Here's a sequence diagram showing a simple interaction, in which the client requests a HomeWrapper, uses it to create a new EJB object, and accesses that object through an ObjectWrapper.

This example is simple. Wrapped finders are a little less simple, because they can return collections of EJB interfaces. In such cases, the proxying method on the HomeWrapper must return a new collection containing each EJB interface wrapped in an ObjectWrapper. Note that the code fragments in the class diagram only hint at conditional handling of locality; they don't show wrappers encapsulating variant return types.

The introduction of a Service Wrapper layer is a non-trivial exercise for large, existing applications. I expect that it's infeasible for most such applications whose EJBs are hand-coded. Mind you, if I'd thought this through properly before I did my painful remote-to-local sweep, I'd have done this as a precursor to the sweep.

Like many J2EE solutions, this is a solution to a complex problem and comes with its own complications. As always, a decision must be made as to whether the application is likely to scale to the point where this solution's additional baggage begins to pay for itself. If that point is likely to be reached, then the Service Wrapper layer is something you want in place long before you get there; it's an expensive retrofit.

I'm very interested in hearing feedback on this idea. A colleague suggested using Dynamic Proxy Classes to reduce the amount of code required, and I'll certainly be looking into that. When I've applied this solution to the project that gave birth to it, I plan to publish a follow-up, complete with doclets and sample code, a reference to the wheel I've reinvented or an admission of abject failure. I'll include any interesting feedback in that follow-up. If you send me stuff but don't want your name associated with, please mention that explicitly. I tend to assume that people want credit where it's due.

Friends Let Friends Write Singletons

2003-09-08T15:55:00Z

Norman Richards recently posted a piece called Friends don't let friends write singletons, in which he expresses contempt for the Singleton pattern. At the end of what I feel is a fairly unbalanced treatment, Norman concludes with "Avoiding singletons isn't always easy."

I'd go so far as to agree that the Singleton pattern is abused. But I suggest the reason Norman has noticed that it can be hard to avoid using the pattern, is that there are situations in which it makes a lot of sense.

Fortunately, he also said: "I want to again ask why one would ever use a singleton." I like a man who expresses willingness to be proven wrong. It's a trait I'm still cultivating. So this one goes out to Norman.

I have a user activity notification component for a J2EE application. It uses MDBs to service notification requests posted to a JMS Topic. The MDBs can send notification messages by email, in which case they're delivered via an SMTP connection pool. All well and good.

But I want runtime control of the size of the connection pool. I also want to be able to query the state of the MDBs and pause and restart them. I use JMX for this. The thing is, my JMX MBean needs to communicate with two things:

The SMTP connection pool. This is facilitated by making the SMTP connection pool a singleton.
The MDBs. This is facilitated by introducing an MDB Observer (see the Observer pattern), with which the MDBs register themselves as Subjects. The MDBs keep the Observer up to date with their status (for monitoring), and check with the Observer before doing certain kinds of work (for control).

Both of these solutions involve the Singleton pattern, and both would be a complete ball-ache to implement without it.

So just give me a second to get a decent grip on the baby, then you can do what you like with the bath water. :-)

FreeBSD Decaf

2003-09-04T15:50:00Z

I've received some horrified responses to my post on J2EE Applications on Wintendo Servers. Why on earth is a FreeBSD committer not fighting tooth and claw to put FreeBSD on a production host instead of Wintendo 2000 Swerver?

Quite simply, FreeBSD is only available in decaf; it lacks the kind of hard-hitting Java support that fuses your vertebrae, staples your eyelids to your face and performs real-time edge sharpening on your visual input.

This applies to both branches of FreeBSD (4.8-STABLE and 5.1-CURRENT), but for different reasons:

The stable branch of FreeBSD emulates threads in user space. Only one process (or emulated thread within a process) is allowed to "descend into" the kernel (technically "execute in privileged mode") at any given time. This restriction is enforced by a single, global lock affectionately referred to as The Big Giant Lock (as opposed to The Little Giant Lock, pulling faces at us through the looking glass).
The development branch of FreeBSD uses fine-grained locking to mediate concurrent access to sensitive kernel data, so it's possible for multiple threads to execute in privileged mode at the same time. That's great. Unfortunately, the process of locking down the various kernel subsystems with fine-grained locks and removing them from under Giant represents an enormous amount of work, and that work isn't finished. The end result is that the development branch is significantly slower than the stable branch.

The bottom line is the same for both branches; highly threaded applications just won't scale on FreeBSD right now. J2EE applications are almost always highly-threaded for any serious production deployment. FreeBSD is fine for a Java developer; that's where I do all my development and testing work, including testing JBoss and Tomcat deployment and execution. But production? Forget about it.

And I'm not just passing on hearsay. I actually tried FreeBSD as an alternative to Wintendo for this deployment. Using the native 1.4.1p3 JDK on FreeBSD 4.8-STABLE, instantiating 2,000 concurrent Tomcat processors would result in an indefinite CPU spin. On 5.1-CURRENT, that number of processors worked, but far too slowly.

It's important to keep in mind that what I've said above has a limited shelf-life. Sooner or later, a FreeBSD release will come along that will invalidate everything I've said above. Even so, I still wouldn't have deployed on FreeBSD if we were already there. Nor on Linux, which already is there.

Why? Pragmatism, something that's seriously lacking in many Unix circles.

The technical lead on the project is most familiar with the Wintendo environment. He has a strong background in Microsoft SQL. While the SQL code used in the project is relatively RDBMS-agnostic, he recognizes that in the early days of production deployment, he's going to be fiddling a lot. He'll be fiddling with the database and he'll be fiddling with application configuration files.

Will it eventually make sense to deploy on a serious operating system? Definitely. But it doesn't make sense right now. In the throes of the product launch, most of the problems encountered are likely to be in the application itself. So what the technical lead most wants is to be freed up to focus on the application. The best way to facilitate this is to host the fledgling application on a familiar platform. As the focus of attention shifts down through the software stack, it will make more and more sense to consider moving to a serious platform, albeit unfamiliar.

Those who know me, know that I'm a vocal Microsoft detractor. Not only do I believe that the corporation produces shit software, I believe they no longer have a grip on any key area of the industry and won't even constitute a blip on the radar in a decade. But I respect the decision of the technical lead. It's a pragmatic one, and it makes.

For now...

Defrosting Stateless Session Beans with MBean Singletons

2003-09-03T10:46:00Z

Stateless session beans are often used as facades for complex or expensive operations in J2EE applications. They're relatively lightweight and benefit from pooling.

One of the problems you can run into with these babies is that facades that perform expensive operations at the beginning of their lifecycle may be very slow for the unlucky consumers whose requests give birth to them. The problem is that the J2EE specification doesn't offer you much in the way of standard ways to control the lifecycle of stateless session beans. This is a good thing, because it keeps their implementation and deployment simple. Of all the components of the EJB specification, stateless session beans are the ones that the smallest number of developers take exception to.

So then how do you get them to perform these expensive startup operations at deployment time? You don't. You can't. Not portably, and hopefully not at all. Instead, a little lateral thinking can result in a graceful solution, and the solution has become something of an idiom in the J2EE world.

Someone recently asked this question on the JBoss users' mailing list, and several people spewed the cliche: use an MBean to initiate the expensive operations in a Singleton at deploy time. Concise, elegant and totally incomprehensible to the poor bugger asking the question.

Here's a prettified version of the answer I sent him. May it serve you well.

Adjust your stateless session beans to offload the expensive lookup operations to a Singleton. For example, they could invoke a method called getStuff() on the Singleton.
Now factor the expensive operations out of the Singleton's getStuff() into getExpensiveStuff(). In getExpensiveStuff(), store the results in instance variables. Then adjust getExpensiveStuff() so that it uses the stored results when possible, only performing the expensive operations when necessary.
In the Singleton's constructor, invoke getExpensiveStuff().
Now for the MBean. An MBean is a "managed bean", part of the Java Management Extensions (JMX) specification.

Create an MBean, called ExpensiveStuffWarmup (implementing interface ExpensiveStuffWarmupMBean), for example. Its constructor should instantiate the Singleton, indirectly invoking the Singleton's getExpensiveStuff() method. For convenience, you may want to do this with a warmStuffUp() method exported as part of the management interface, so that you can invoke it via JMX (using JBoss' jmx-console, for example).

Chapter 2 of the O'Reilly book "Java Management Extensions" provides an excellent introduction to writing your first MBean. I downloaded chapter 2 of this book from somewhere for free a couple of months ago, but can't remember where. I've been sloppy with my notes lately. :-(

You can view the book online if you have an O'Reilly Safari subscription. If you don't have a subscription, you may be able to view the book anyway with a 14 day trial.
Lastly, create a service deployment descriptor for your MBean and arrange for it to be deployed by your application server. For JBoss, this means creating a JBoss service descriptor file called warmup-service.xml, for example, and placing it in the jboss/server/default/deploy directory.

The service descriptor file is simple. The trick is in classloading. To keep things simple, I'd put the JAR file containing the Singleton class in jboss/server/default/lib directory, so that it's available to both the MBean and the stateless session bean, which should be deployed somewhere under jboss/server/default/deploy.

The service descriptor might look like this:
```
<?xml version="1.0" encoding="UTF-8"?>
<server>
   <mbean code="com.example.ExpensiveStuffWarmup" name="example:service=exWarmup">
       
   </mbean>
</server>
```

This may seem like a lot of work, but keep in mind that you're doing optimization work here. With J2EE, optimization is usually something that focuses outside the business logic domain and its only inside the business logic domain that J2EE is supposed to save you time. Har har, did I pull the one with bells on?