fetchmail acts as a gateway between an external message store such as an IMAP or POP3 server and James. Mail is fetched from the external message store and injected into the James input spool.
fetchmail is useful when delivery via standard SMTP is not an option, as a means of consolidating mail delivered to several external accounts into a single James account, or to apply the mail processing capabilities of James to mail stored in an external message store.
fetchmail has several configuration options that control the fetching and filtering of mail injected into the James input spool. Once there, James' flexible mail processing engine can be used to further process the mail, just as if it had been delivered via standard SMTP.
How fetchmail Works
fetchmail Configuration Parameters
fetchmail Examples
fetchmail Caveats
Mail is delivered by periodically running fetch tasks that read messages from an external message store and injects them into the James input spool. Fetch tasks run concurrently.
A set of filters applies to each fetch task. Each filter provides the ability to reject a message that matches the filter criteria. Rejected messages are not injected into the James input spool; they are either marked as seen or deleted. When a filter is configured to accept a message that matches its criteria, messages are marked with a MailAttribute. This MailAttribute can be detected within the James matcher/mailet chain, allowing further processing as required.
Each fetch task is associated with a single host server. Accounts are defined to the fetch task for each mailbox on the server from which mail is to be fetched. Accounts run consecutively.
Optionally, the fetch task can be configured with an <alllocal> Account that generates an Account entry for each user defined in the James user repository. This removes the requirement to manually add or remove Account entries to the fetchmail configuration each time a James user is added or removed. Currently this is only useful if the server supports virtual mailboxes that allow the same password to apply to all users within a domain.
Accounts can be configured to deliver all mail for an Account to a specified recipient or to deduce the intended recipient from the mail headers.
Accounts are normally configured to deliver all mail for an Account to a specified recipient, ignoring the recipient in the mail headers. This works well in the majority of cases where a mailbox is guaranteed to contain mail for a sole mailbox recipient.
Accounts are configured to deduce the intended recipient from the mail headers when a mailbox contains mail for several users, typically all users in a domain. Used alone, this is not foolproof as there are circumstances when a single unique recipient cannot be deduced from the mail headers alone. Used in conjunction with an appropriately configured <alllocal> account, it is always possible to deduce the intended recipient when the recipient is a James user.
The fetchmail configuration parameters are part of the James configuration,
whose base file is config.xml. For clarity and flexibility, the
fetchmail configuration parameters are stored in the file
james-fetchmail.xml, which is referenced within
config.xml.
The configuration parameters are described below.
The configuration block delimited by the fetchmail tag controls fetchmail.
The tag has these attributes:
The tag has these child tags (minimum cardinality, maximum cardinality):
<fetchmail enabled="true"> ... </fetchmail>
The fetch tag defines a fetch task to be run periodically. Fetch tasks run concurrently.
The tag has these attributes:
The tag has these child tags (minimum cardinality, maximum cardinality):
<fetch name="mydomain.com"> ... </fetch>
<accounts> ... </accounts>
wibble@mydomain.com, flobble@mydomain.com.
The tag has these attributes:
org.apache.james.fetchmail.isBlacklistedRecipient
added to the mail.
<blacklist
reject="true"
leaveonserver="true"
markseen="true">
wibble@mydomain.com, flobble@mydomain.com
</blacklist>
From: header of a mail that has a valid user part
but is missing the domain part.
If not specified, the default behaviour is to append the canonical host name
of the James server.
The tag value is the name of the server to append. The name must be a server
declared in the servernames tag of the James
block in the configuration or the name localhost.
<defaultdomain>
mydomain.com
</defaultdomain>
<fetchall>false</fetchall>
<fetched leaveonserver="true" markseen="true"/>
The host tag declares the IP address of the external server from which mail is fetched.
The tag value is the DNS name or IP address literal of the external server.
<host>pop3.server.com</host>
The interval tag declares the period between invocations of the fetch tasks. If a fetch task is still active from a previous invocation when the period expires, the new invocation is skipped over.
The tag value is an integer representing the number of milliseconds to elapse between invocations of the fetch tasks.
<interval>60000</interval>
The javaMailFolderName tag declares the name of the root folder on the external server from which mail is fetched.
The tag value is the cAsE-sEnSiTiVe name of the root folder on the external
server from which mail is fetched. For POP3 servers this is always
INBOX.
<javaMailFolderName>INBOX</javaMailFolderName>
The javaMailProperties tag declares the properties to be
applied to the JavaMail Session used by the fetch task. These override the
properties answered by System.getProperties(). Many JavaMail
properties are specific to the JavaMail Provider selected by the
javaMailProviderName tag.
Relying on the default values selected by the Provider can be inappropriate. For instance, the default connection and I/O timeout values of infinite for the default IMAP and POP3 Providers is rarely what is required. Consult the documentation of the Provider for details and options.
Documentation for the default Provider for IMAP is located here.
Documentation for the default Provider for POP3 is located here.
Details of how to change a Provider are located here.
The tag has these child tags (minimum cardinality, maximum cardinality):
<javaMailProperties> ... </javaMailProperties>
The javaMailProviderName tag selects the JavaMail protocol Provider used to interact with the external server.
The tag value is the name of a JavaMail supported protocol, such as
pop3 or imap. The name is used to select the default
Provider for the protocol.
<javaMailProviderName>pop3</javaMailProviderName>
The maxmessagesize tag declares the maximum permitted message size for messages injected into the James input spool and what happens to fetched messages that exceed this size.
The tag has these attributes:
org.apache.james.fetchmail.isMaxMessageSizeExceeded added prior to
injection into the James input spool, (see below for the location of an example).
<maxmessagesize
limit="4096"
reject="false"
leaveonserver="false"
markseen="false"/>
An example configuration using James mailet processing to bounce fetched
messages that exceed the maximum permitted size can be found in the file
$PHOENIX_HOME/apps/james/conf/samples/fetchmail/maxMessageSize.xml.
The recipientnotfound tag declares what happens to mail for which a sole intended recipient cannot be found when attempting to determine the recipient from the mail headers.
In configurations with more than one account per fetch task, processing of matched mail can be deferred to the next run of the fetch task. This gives other accounts that may be able to determine a sole intended recipient an opportunity to do so before recipientnotfound processing is invoked.
The tag has these attributes:
org.apache.james.fetchmail.isRecipientNotFound added to the
mail.
<recipientnotfound
defer="true"
reject="true"
leaveonserver="true"
markseen="true"/>
The recursesubfolders tag declares if mail should be fetched from sub-folders of the root folder, or just the root folder.
The tag value is a boolean. If true, mail is fetched from the root folder and its subfolders. If false, mail is fetched from just the root folder.
<recursesubfolders>false</recursesubfolders>
The remoteReceivedHeader tag declares the zero based index of the RFC2822 compliant RECEIVED header used to determine the address and host name of the remote MTA that sent a fetched message and what happens to messages when the specified header is invalid.
Typically, the first (index = 0) RECEIVED header is for the local MTA that delivered the message to the message store and the second (index = 1) RECEIVED header is for the remote MTA that delivered the message to the local MTA. When this configuration applies, the remoteReceivedHeaderIndex should be set to 1.
To verify the correct setting, examine the RECEIVED headers for messages delivered to the configured message store and locate the first one containing a remote domain in the'from' field. Remembering that zero based indexing is used, if this the second header, use an index of 1, if this is the third header, use an index of 2, and so forth.
Matchers such as InSpammerBlacklist use the remote address and/or remote host
name to identify illegitimate remote MTAs. If you do not use such matchers, the
remoteReceivedHeaderIndex tag may be omitted or the default
index value of -1 can be specified. This causes the remote address to be set to
127.0.0.1 and the remote host name to be set to
localhost. Matchers almost always considered these values to be
legitimate.
The tag has these attributes:
org.apache.james.fetchmail.isInvalidReceivedHeader
added to the mail, the remote address set to 127.0.0.1 and the remote
host name set to localhost.
<remoteReceivedHeader
index="1"
reject="true"
leaveonserver="true"
markseen="true"/>
An example configuration using James mailet processing to notify the postmaster
of fetched messages that contain an invalid Received header can be found in the file
$PHOENIX_HOME/apps/james/conf/samples/fetchmail/remoteReceivedHeader.xml.
The remoterecipient tag declares what happens to mail for which the domain part of the recipient is remote. A domain is remote if it is not a server declared in the servernames tag of the James block in the configuration.
The tag has these attributes:
org.apache.james.fetchmail.isRemoteRecipient added to the mail.
<remoterecipient
reject="true"
leaveonserver="true"
markseen="true"/>
The undeliverable tag declares what happens to mail that cannot be delivered.
The tag has these attributes:
<undeliverable
leaveonserver="true"
markseen="true"/>
The userundefined tag declares what happens to mail for which the recipient is not defined as a James user.
The tag has these attributes:
org.apache.james.fetchmail.isUserUndefined added to the mail.
<userundefined
reject="true"
leaveonserver="true"
markseen="true"/>
The account tag declares an account on the external server from which mail should be fetched.
The tag has these attributes:
<account
user="myaccount"
password="mypassword"
recipient="user@localhost"
ignorercpt-header="true"/>
The alllocal tag declares the parameters to be applied to dynamic accounts. The set of dynamic accounts is refreshed each time the fetch task runs by combining the alllocal tag attributes with each of the currently defined James users to create an account for every James user.
The tag has these attributes:
<alllocal
userprefix=""
usersuffix="@external.domain.com"
password="mypassword"
recipientprefix=""
recipientsuffix="@mydomain.com"
ignorercpt-header="true"/>
The property tag declares a name/value pair.
The tag has these attributes:
<property
name="mail.pop3.connectiontimeout"
value="180000"/>
Full sources to the examples discussed below can be found in the directory
$PHOENIX_HOME/apps/james/conf/samples/fetchmail.
When all mail for an account is to be delivered to a single user, configure each account to ignore the recipient in the mail headers and deliver to the specified recipient. The accounts block looks like this:
<accounts>
<account
user="user1@external.domain.com"
password="password1"
recipient="user1@localhost"
ignorercpt-header="true"/>
<account
user="user2@external.domain.com"
password="password2"
recipient="user2@localhost"
ignorercpt-header="true"/>
<account
user="user3@external.domain.com"
password="password3"
recipient="user3@localhost"
ignorercpt-header="true"/>
</accounts>
When an account contains mail to be delivered to many users, configure each account to determine the recipient from the mail headers and deliver to that user. The accounts block looks like this:
<accounts>
<account
user="global@external.domain.com"
password="password"
recipient="fetchmail@localhost"
ignorercpt-header="false"/>
</accounts>
The recipientnotfound tag is used to declare what happens when the recipient cannot be determined from the mail headers. In the example below, mail is injected into the spool using the recipient declared in the account tag:
<recipientnotfound
defer="false"
reject="false"
leaveonserver="false"
markseen="false"/>
When an external server supports virtual mailboxes, fetchmail's dynamic account facility can be used. This greatly simplifies user configuration as the fetchmail accounts for users are automatically synchronized with those defined in the James user repository. This guarantees that mail for all local users will be fetched and delivered.
Currently, there is a limitation that all virtual accounts and the global account must share the same password.
The alllocal tag declares the parameters for the dynamic
accounts. The accounts block below will deliver mail for
user1@external.domain.com to user1@localhost,
user2@external.domain.com to user2@localhost,
userZ@external.domain.com to userZ@localhost etc.:
<accounts>
<alllocal
userprefix=""
usersuffix="@external.domain.com"
password="mypassword"
recipientprefix=""
recipientsuffix="@localhost"
ignorercpt-header="true"/>
</accounts>
The One Account, One User - Dynamic example guarantees delivery of mail for all local users, but leaves other mail on the external server unprocessed. The One Account, Many Users example processes all mail on the external server, but cannot guarantee delivery to the intended recipient. By combining the two, it is possible to guarantee the delivery of mail for all local users and process all mail.
In the snippet below, the alllocal tag declares dynamic accounts for all local users and the account tag configures an account to fetch all mail.
The recipientnotfound tag rejects mail for which a recipient cannot be determined. By the time this processing is activated, the dynamic accounts will have processed mail for all local users, so the mail can only be mail for non-local users or newly arrived mail for local users. It is not possible to know which, but we want to leave mail for local users to be dealt with by the dynamic accounts. The next time the dynamic accounts run any newly arrived mail for local users will be processed. The remainder will be for non-local users and can now be safely dealt with.
The <recipientnotfound defer="true" attribute
enables deferal of the processing of messages for which the recipient cannot be
determined to the next iteration of the fetch task, and is used here. The
relevant tags are:
<accounts>
<alllocal
userprefix=""
usersuffix="@external.domain.com"
password="mypassword"
recipientprefix=""
recipientsuffix="@localhost"
ignorercpt-header="true"/>
<account
user="global@external.domain.com"
password="password"
recipient="fetchmail@localhost"
ignorercpt-header="false"/>
</accounts>
<recipientnotfound
defer="true"
reject="true"
leaveonserver="true"
markseen="true"/>
These are some things to be aware of when using fetchmail:
FetchedFrom
matcher to detect mail injected by fetchPOP. This will not work with fetchmail.
Compared to fetchPOP, there are far fewer occasions when mail injected by
fetchmail requires special processing. When it does, use the HasMailAttribute
matcher to match the attribute named
org.apache.james.fetchmail.taskName to detect all mail injected by
fetchmail. To detect mail injected by a specific fetch task, use one of the
HasMailAttributeWithValue matchers to match on the attribute name and the
attribute value. The attribute value is the name of the fetch task that
injected the mail.
markseen="true" will most likely have no effect and
therefore, the fetchall tag will be inoperative. In this
situation, the only way to avoid repeatedly fetching the same mail is to delete
it from the server using leaveonserver="false"/>.