Contents: How e-mail works James design objectives Installing and configuring James Testing James with JavaMail Matchers Mailets Additional features What's next? Resources About the author Rate this article Related content: Fundamentals of JavaMail API Working with James, Part 2 Subscriptions: dW newsletters dW Subscription (CDs and downloads)

Claude Duguay (mailto:claude.duguay@verizon.net?cc=&subject=An introduction to Apache's James enterprise e-mail server)
Chief Architect, Arcessa, Inc.
10 June 2003

This article is the first in a two-part series on the Java Apache Mail Enterprise Server, also known as James. It lays a foundation for understanding James and for developing server-side e-mail applications. The article provides a high-level overview, briefly touches on the Apache group's design objectives, and describes how to install and configure a workable development environment. You can also take a brief tour of the features supported by James. You'll find descriptions for both the matcher and the mailet implementations that come with James, and a comparison of the existing functionality with that found in traditional e-mail servers.

The Java Apache Mail Enterprise Server -- generally referred to as James -- is a portable, secure, and 100% Pure Java enterprise mail server built by the Apache group. But it has the potential to be much more than that, thanks to its pluggable protocol architecture and a mailet infrastructure that does for e-mail what servlets do for Web servers. E-mail servers have been around since the early days of DARPA funding for what would eventually become the Internet, but James offers new possibilities for what's often been dubbed the Internet's first killer application.

This is the first of two articles that explore James. It provides an overview of James and a high-level orientation to let us explore its possibilities. In the second article, we'll implement a mailet application that manages unavailability messages. As you'll see, it is surprisingly easy to write applications for James. E-mail is used around the world by millions of people every day, so the possibilities go well beyond the introductory treatment that this series provides. It's my hope, however, that this foundation will serve you well and help you start imagining the possibilities.

How e-mail works
E-mail is simple, in principle. You construct a message with one or more recipient addresses using a mail user agent (MUA). MUAs come in many forms and include text-based, Web-based, and GUI applications; Microsoft Outlook and Netscape Messenger fall into the last category. Each e-mail client is configured to send mail to a mail transfer agent (MTA) and can be used to poll an MTA to fetch e-mail messages sent to the user's address. To do this, you need an e-mail account on a mail server (technically the MTA) and you can, using standard Internet protocols, either work with the e-mail offline (using POP3) or leave the e-mail on the server (using IMAP). The protocol used to send mail both from the client to the MTA and between MTAs is SMTP (Simple Mail Transfer Protocol).

Building a James application Part 2 of this series builds on your knowledge of the basic James infrastructure to implement a practical application.

What really happens between MTAs is only slightly more interesting. E-mail servers rely heavily on DNS and e-mail-specific records called mail transfer (or MX) records. MX records are slightly different from the DNS records used to resolve URLs, containing some additional priority information used to route mail more effectively. I won't delve into those details here, but it's important to understand that DNS is key to routing e-mail successfully and efficiently. James is an MTA, while the JavaMail API provides a framework for an MUA. In this article, we'll use JavaMail to set up a test for our James installation. In the second article in this series, we'll use the James Mailet API to show how you can develop your own James applications.

James design objectives
James was designed to accommodate certain objectives. For example, it is written entirely in the Java language to maximize portability. It was written to be secure and provides a number of features that both protect the server environment itself and provide secure services. James functions as a multithreaded application that takes advantage of many of the benefits available in the Avalon framework. (Avalon is an Apache Jakarta project that features the Phoenix high-performance server infrastructure. Understanding Avalon is useful but not necessary to developing James applications. See the Resources section for more on Avalon.)

James provides a comprehensive set of services, including many that are usually available only in high-end or well-established e-mail servers. These services are primarily implemented using the Matcher and Mailet APIs, which work together to provide e-mail detection and processing capabilities. James supports the standard e-mail protocols (SMTP, POP3, IMAP), along with a few others, using a loosely coupled plug-in design that keeps the messaging framework abstracted from the protocols. This is a powerful idea that may enable James to act as more of a general messaging server in the future or to support alternative messaging protocols such as instant messaging.

The final and most interesting objective delivered by the James design group is the notion of mailets, which provide a component life cycle and container solution for developing e-mail applications. To be sure, it's always been possible to use other MTAs, such as Sendmail, to do this, given that any program can be called and data piped through executables to do the job, but James provides a common, simple API for accomplishing these goals and makes the work easy, thanks to the objects available for manipulation. We'll take a closer look at both the Matcher and Mailet APIs in this article.

Installing and configuring James
James is available from its home page at the Apache foundation Web site (see Resources for a link). You should download the latest production release to work with; at the time of this writing, that version is 2.1.2. You can find the download area by selecting Downloads > Binaries in the left navigation area of the James home page. From there, scroll down to the Release Builds section and select one of the James 2.1 links. Select either james-2.1.2.tar.gz or james-2.1.2.zip, depending on your preference.

We'll also be using the JavaMail API to test our application, so you'll need to download that as well (see Resources). The version currently available is 1.3 and the file is called javamail-1_3.zip. While you're on the main JavaMail page, you'll notice a link to the JavaBeans Activation Framework (JAF), which the JavaMail API requires. (See Resources for a direct link to the JAF.) The current JAF release is 1.0.2 and the file is called jaf-1_0_2.zip. Once you have all these files, you can set up your system to work with James.

We'll set up a directory structure with all the elements we need for development. In production, the setup has to be quite different, arranged based on considerations of both security and functionality. For our purposes, for example, we can work on localhost, but that's not a viable option when working with a real e-mail server deployment. There's plenty of documentation on configuring James as a primary MTA or with Sendmail, and plenty of help on the mailing lists if you need to deploy the server commercially.

With everything unzipped in a James directory, our hierarchy will look like Listing 1. I've removed a few subdirectories under javadoc, src, and the JavaMail webapp demo to keep things more compact and easier to envision.

James
+---jaf-1.0.2
|   +---demo
|   \---docs
|       \---javadocs
+---james-2.1.2
|   +---apps
|   +---bin
|   |   \---lib
|   +---conf
|   +---docs
|   |   +---images
|   |   \---stylesheets
|   +---ext
|   +---lib
|   +---logs
\---javamail-1.3
    +---demo
    |   +---client
    |   +---servlet
    |   \---webapp
    +---docs
    |   \---javadocs
    \---lib

I am assuming that you have version 1.4 of the Java platform set up, independent of the James files. The James configuration notes suggest that some problems have been observed using Java 1.3.0, so you should work with 1.3.1 or higher. In principle, James should work well on any platform that supports a suitable Java 1.4 VM.

Our first step is to start James, because the configuration files are not unpacked until the server has been run once. You'll find a run script (use either run.bat or run.sh, depending on your operating system) in the james-2.1.2/bin directory. When you run that script, the output should look something like Listing 2 (this sample output is from a Windows system):

Using PHOENIX_HOME:   D:\James\james-2.1.2
Using PHOENIX_TMPDIR: D:\James\james-2.1.2\temp
Using JAVA_HOME:      c:\programming\java14

Phoenix 4.0.1

James 2.1.2
Remote Manager Service started plain:4555
POP3 Service started plain:110
SMTP Service started plain:25
NNTP Service started plain:119
Fetch POP Disabled

You can use Ctrl+C to exit the program, and you'll get a message from the Phoenix container that it's exiting when you do so. Strictly speaking, the proper way to have James exit is to use the remote management interface. I've used Ctrl+C in my own development with no negative repercussions. In a deployment environment, however, you should always use the shutdown command.

After shutting down James for the first time, you'll find a file in the james-2.1-2/apps/james/SAR-INF folder called config.xml, which you should take a look at. The first thing you would normally change is the administrator account, which is set to root, with the password root, by default. We'll leave it alone for development but it would be unwise to leave it configured this way in a production system for obvious reasons. The next thing to change is usually the DNS server address, which is necessary if James is to operate as a complete e-mail server. We'll leave this alone as well, given that all our tests will be run from localhost, but again this is an important setting that you should be aware of. The rest of the default settings are fine, given our development objectives, though it's important to understand the configuration file. You can find more information in the documentation provided in the james-2.1.2/docs directory.

Let's add a few users before moving on. To do that, telnet to localhost on port 4555 with the command telnet localhost 4555. You can log in with the root user name and password. After the login, we'll add a few users. The adduser command expects a username and password combination. We'll add users named red, green, and blue for this project, each with the same password as the user name. (I'm sure you know that this is a bad idea when creating real users, but it'll make it easy to configure our test cases.) After adding users, you can use the listusers command to verify the entries and then exit the remote manager by typing quit. The whole session should look like Listing 3. I've highlighted text that you would enter yourself.

JAMES Remote Administration Tool 2.1.2
Please enter your login and password
Login id:
root
Password:
root
Welcome root. HELP for a list of commands
adduser red red
User red added
adduser green green
User green added
adduser blue blue
User blue added
listusers
Existing accounts 3
user: blue
user: green
user: red
quit
Bye

Now we're all set to go with a running James server. As you can see, deploying James and using the remote manager to set things up is relatively straightforward. Obviously, you would need to change several configuration parameters if you wanted the mail server to be secure, but that's not a very complicated process. The real key to using mail servers has more to do with setting up DNS correctly in multiuser and multiserver environments. That's beyond the scope of this article, but isn't that complicated a process either.

Testing James with JavaMail
To make sure our setup is functional, we'll write a quick pair of classes that will send messages and list inbox content, simulating the base functionality of a typical e-mail client. We'll use two classes because the MailClient class, shown in Listing 4, can be reused for testing more complex behavior, which we'll do when we develop our James application in the second article in this series.

import java.io.*;
import java.util.*;
import javax.mail.*;
import javax.mail.internet.*;

public class MailClient
  extends Authenticator
{
  public static final int SHOW_MESSAGES = 1;
  public static final int CLEAR_MESSAGES = 2;
  public static final int SHOW_AND_CLEAR =
    SHOW_MESSAGES + CLEAR_MESSAGES;
  
  protected String from;
  protected Session session;
  protected PasswordAuthentication authentication;
  
  public MailClient(String user, String host)
  {
    this(user, host, false);
  }
  
  public MailClient(String user, String host, boolean debug)
  {
    from = user + '@' + host;
    authentication = new PasswordAuthentication(user, user);
    Properties props = new Properties();
    props.put("mail.user", user);
    props.put("mail.host", host);
    props.put("mail.debug", debug ? "true" : "false");
    props.put("mail.store.protocol", "pop3");
    props.put("mail.transport.protocol", "smtp");
    session = Session.getInstance(props, this);
  }
  
  public PasswordAuthentication getPasswordAuthentication()
  {
    return authentication;
  }
  
  public void sendMessage(
    String to, String subject, String content)
      throws MessagingException
  {
    System.out.println("SENDING message from " + from + " to " + to);
    System.out.println();
    MimeMessage msg = new MimeMessage(session);
    msg.addRecipients(Message.RecipientType.TO, to);
    msg.setSubject(subject);
    msg.setText(content);
    Transport.send(msg);
  }
  
  public void checkInbox(int mode)
    throws MessagingException, IOException
  {
    if (mode == 0) return;
    boolean show = (mode & SHOW_MESSAGES) > 0;
    boolean clear = (mode & CLEAR_MESSAGES) > 0;
    String action =
      (show ? "Show" : "") +
      (show && clear ? " and " : "") +
      (clear ? "Clear" : "");
    System.out.println(action + " INBOX for " + from);
    Store store = session.getStore();
    store.connect();
    Folder root = store.getDefaultFolder();
    Folder inbox = root.getFolder("inbox");
    inbox.open(Folder.READ_WRITE);
    Message[] msgs = inbox.getMessages();
    if (msgs.length == 0 && show)
    {
      System.out.println("No messages in inbox");
    }
    for (int i = 0; i < msgs.length; i++)
    {
      MimeMessage msg = (MimeMessage)msgs[i];
      if (show)
      {
        System.out.println("    From: " + msg.getFrom()[0]);
        System.out.println(" Subject: " + msg.getSubject());
        System.out.println(" Content: " + msg.getContent());
      }
      if (clear)
      {
        msg.setFlag(Flags.Flag.DELETED, true);
      }
    }
    inbox.close(true);
    store.close();
    System.out.println();
  }
}

The MailClient class is primarily designed to let us send messages and to display or delete the list of messages available on the server for a given user. I've declared some useful constants that let us SHOW_MESSAGES, CLEAR_MESSAGES, or both. The MailClient class also implements the Authenticator interface to make it easy to manage the logon process when retrieving e-mail.

I've created two constructors, one of which sets the JavaMail debugging flag explicitly. This prints client/server protocol interactions to the console so that you can see what's going on. The shorter constructor leaves this flag off. The other two arguments are the user name and host. By implication, the e-mail address can be derived from the user and host. We create a PasswordAuthentication object that can be returned by the getPasswordAuthentication() method specified in the Authenticator interface.

The rest of the constructor code sets up the JavaMail properties to reflect the specified user and host, and explicitly specifies the protocols we intend to use. Once we have the Properties object populated, we can call the static Session method getInstance() to get a valid Session reference, which we store in a local variable. Once the constructor has been used with a specified user, we are ready to send and retrieve e-mail for that user on the specified e-mail host.

The sendMessage() method is straightforward as well. It builds a MimeMessage with the specified recipient, subject, and text content, and then sends it using the JavaMail static send() method in the Transport class. To make it easy to see what's going on, we also print a message to the console.

The checkInbox() method does more work because it needs to list messages and, optionally, erase them. It's also possible to just erase messages without looking at them, depending on the flags you use for the mode argument. To actually get the messages, we need to get a reference to the Store through our session object, connect to the server, and then open the inbox folder. After we have a reference to the folder, we can iterate through the messages and show or erase them.

Now that we have our reusable MailClient code, we're ready to write a quick test for the James server on localhost. The JamesConfigTest class in Listing 5 does this by creating three MailClient instances, each associated with one of our users (red, green, and blue). Before you run this code, make sure that those users are valid on the e-mail server.

public class JamesConfigTest
{
  public static void main(String[] args)
    throws Exception
  {
    // CREATE CLIENT INSTANCES
    MailClient redClient = new MailClient("red", "localhost");
    MailClient greenClient = new MailClient("green", "localhost");
    MailClient blueClient = new MailClient("blue", "localhost");
    
    // CLEAR EVERYBODY'S INBOX
    redClient.checkInbox(MailClient.CLEAR_MESSAGES);
    greenClient.checkInbox(MailClient.CLEAR_MESSAGES);
    blueClient.checkInbox(MailClient.CLEAR_MESSAGES);
    Thread.sleep(500); // Let the server catch up
    
    // SEND A COUPLE OF MESSAGES TO BLUE (FROM RED AND GREEN)
    redClient.sendMessage(
      "blue@localhost",
      "Testing blue from red",
      "This is a test message");
    greenClient.sendMessage(
      "blue@localhost",
      "Testing blue from green",
      "This is a test message");
    Thread.sleep(500); // Let the server catch up
    
    // LIST MESSAGES FOR BLUE (EXPECT MESSAGES FROM RED AND GREEN)
    blueClient.checkInbox(MailClient.SHOW_AND_CLEAR);
  }
}

After creating the three MailClient instances, the JamesConfigTest code simply clears each mailbox using the checkInbox() method with the CLEAR_MESSAGES mode, and waits a half second to make sure that the server has processed the deletions. We next send a message to blue from both red and green, and then check for messages to the blue account. When you run JamesConfigTest, you should see output that looks like Listing 6:

Clear INBOX for red@localhost

Clear INBOX for green@localhost

Clear INBOX for blue@localhost

SENDING message from red@localhost to blue@localhost

SENDING message from green@localhost to blue@localhost

Show and Clear INBOX for blue@localhost
    From: green@localhost
 Subject: Testing blue from green
 Content: This is a test message

    From: red@localhost
 Subject: Testing blue from red
 Content: This is a test message

This proves that our James setup is functional; you'll need to have your system set up in this way before proceeding to development. We'll hold off on that until the second part of this series, however. In the remainder of this article, we'll examine the Matcher and Mailet APIs and the prewritten matchers and mailets provided as part of the James distribution. We'll also take a quick look at some additional features supported by James.

Matchers
James comes with a number of standard matchers. Each of these implements the Matcher API, illustrated in Listing 7, and provides functionality that is common to existing MTAs, as well as other useful extensions. The interface is fairly simple; it includes a couple of life-cycle methods, init() and destroy(), along with a pair of bookkeeping methods, getMatcherInfo() and getMatcherConfig(), and the main method, match(), which operates on a Mail object. The Mail reference provides access to container state, the mail message, and metadata for processing.

public interface Matcher
{
  void init(MatcherConfig config);
  void destroy();
  String getMatcherInfo();
  MatcherConfig getMatcherConfig();
  Collection match(Mail mail);
}

A matcher is responsible for recognizing a group of recipients and returns a collection of String objects that represent the recipients to be processed by a mailet. By combining matcher recognition and mailet processing, you can develop complex applications that operate on e-mail messages.

The matchers provided as part of the James distribution enable you to do several things without having to develop your own matcher implementations. It's important to know what these are before deciding to develop your own matchers; in many cases, the job you're contemplating may already be done for you. You can see a list of these prewritten matchers in Table 1:

Table 1. Prewritten James matchers

Matcher	Description
All	Matches all e-mails being processed and returns all recipients
HasHeader	Matches a specified header, if present
HasAttachment	Matches if the message is a multipart message
SubjectStartsWith	Matches messages whose subjects start with the specified subject text
SubjectIs	Matches messages that have a specific subject
HostIs	Matches messages with a specified host
HostIsLocal	Matches messages originating from localhost
UserIs	Matches a specified user
SenderIs	Matches a specified sender
SenderInFakeDomain	Matches senders whose host address cannot be resolved
SizeGreaterThan	Matches messages whose sizes are greater than a specified limit
Recipients	Matches messages with recipients from a specified list
RecipientsLocal	Matches messages with local recipients
IsSingleRecipient	Matches messages with only one recipient
RemoteAddrInNetwork	Matches messages from a specified list of IP addresses, domains, and so on
RemoteAddrNotInNetwork	Matches messages not from a specified list of IP addresses, domains, and so on
RelayLimit	Matches messages relayed through more than a specified number of servers
InSpammerBlackList	Matches addresses to a list provided by mail-abuse.org
NESSpamCheck	Matches spam using a method derived from a Netscape Mail Server
HasHabeasWarrantMark	Matches mail with a Habeas Warrant
FetchedFrom	Matches the X-fetched-from header used by FetchPOP
CommandForListserv	Matches commands for the list server

As you can see from this list, you can accomplish many tasks without writing any new code, including granular operations like matching headers, subjects, or recipients, as well as high-level operations like detecting spam and processing list server commands.

Mailets
Many of James' features are implemented through the Mailet API illustrated in Listing 8, which will seem oddly familiar to developers accustomed to using the Servlet API. Like the Matcher API, the Mailet interface supports two lifecycle methods to provide initialization (the init() method) and shutdown (the destroy() method). Two additional methods return information. The first, getMailetInfo(), returns a String object that should contain information like the author, version, and copyright associated with the mailet. The second, getMailetConfig(), is useful for returning the current mailet configuration information. The init() method takes a MailetConfig object as an argument, so this is normally the object returned by the getMailetConfig() method, though it may have been modified.

public interface Mailet
{
  void init(MailetConfig config);
  void destroy();
  String getMailetInfo();
  MailetConfig getMailetConfig(); 
  void service(Mail mail);
}

Main processing is done in the services() method, which takes a Mail object argument. This object provides additional access to container state, the mail message, and metadata for processing.

To give you an idea of the features that are supported by James and the kinds of mailet applications that already exist, Table 2 provides a list of the mailet implementations available in James right out of the box:

Table 2. Prewritten James mailets

Mailet	Description
Null	Ends processing for the e-mail message
AddHeader	Adds a text header to the message content
AddFooter	Adds a text footer to the message content
Forward	Forwards the message to a list of recipients
Redirect	Provides configurable redirection services
ToProcessor	Redirects e-mail processing to a specified processor
ToRepository	Puts a copy of the message in the specified directory
NotifySender	Forwards the message as an attachment to the original sender
NotifyPostmaster	Forwards the message as an attachment to the postmaster
RemoteDelivery	Manages SMTP host deliveries
LocalDelivery	Delivers messages to local mailboxes
JDBCAlias	Does alias translation using a JDBC data source
JDBCVirtualUserTable	Does more complex alias translation using a JDBC data source
UseHeaderRecipient	Regenerates the mail recipients from the message header
ServerTime	Sends a server time-stamp message
PostmasterAlias	Redirects messages for postmaster@<domain> to an individual's address
AddHabeasWarrantMark	Adds a Habeas Warrant mark to the message
AvalonListserv	Provides basic list server functionality
AvalonListservManager	Processes list server management commands

As you can see from this list, there are several features already available in James thanks to the Mailet API, including complex list server support, aliasing, and storage and routing capabilities.

Additional features
James supports many other capabilities that are beyond the scope of this series; however, we'll briefly mention them here so that you can have a better idea of what James is actually capable of. The first of these is NNTP support, which allows James to act as a Usenet server. James also implements the FetchPOP protocol to support mail-based remote management features. The RemoteManager and SpoolManager provide abstractions that allow multiple types of storage and management support to exist. For development purposes, it's sufficient to rely on the filesystem-based SpoolManager, for example, though both partially and fully database-centric solutions are also provided.

James provides interfaces and services to allow users to be managed effectively, and mailing list support is available out of the box. In fact, the mailing list feature is one of the most used services provided by James and is often one of the primary reasons administrators choose James as an e-mail solution.

What's next?
The James infrastructure is designed for flexibility and easy application development. E-mail application possibilities are limited only by your imagination. In the follow-up installment of this series, we'll develop a simple application that allows users to send e-mail messages to a specific James address to enable vacation message-type features. Users can draft an e-mail that will be sent to all incoming senders until the user sends a cancellation message to another specified James address to turn it off. This solution mimics a mechanism that is often implemented by e-mail client software, but that is generally limited to a single geographical location because of it: if your mail client is turned off, the feature does not work. By implementing this functionality on the e-mail server, we can still check our mail from any location without restraint, and we can easily change our "away" message to a more appropriate response if our plans change.

Resources

• Download the source code used in this article.
  
  
• Download the James packages at the Apache James home page.
  
  
• Learn more about Avalon at the Avalon project home page.
  
  
• To run the example code in this article, you'll need to download the JavaMail API and the JavaBeans Activation Framework.
  
  
• Learn more about JavaMail in the tutorial "Fundamentals of JavaMail API" (developerWorks, August 2001).
  
  
• Find hundreds of other Java technology-related resources on the developerWorks Java technology zone.
  
  

What do you think of this document? Killer! (5) Good stuff (4) So-so; not bad (3) Needs work (2) Lame! (1) Comments?

developerWorks > Java technology