Consult fetchmail-template.xml in GIT to get some examples and hints.
Fetchmail acts as a gateway between an external message store such as an IMAP or POP3 server and James. Mail is fetched from the external message store and injected into the James input spool.
Fetchmail is useful when delivery via standard SMTP is not an option, as a means of consolidating mail delivered to several external accounts into a single James account, or to apply the mail processing capabilities of James to mail stored in an external message store.
Fetchmail has several configuration options that control the fetching and filtering of mail injected into the James input spool. Once there, James' flexible mail processing engine can be used to further process the mail, just as if it had been delivered via standard SMTP.
How Fetchmail Works
Fetchmail Configuration Parameters
Fetchmail Examples
Fetchmail Caveats
Fetchmail Gmail
Mail is delivered by periodically running fetch tasks that read messages from an external message store and injects them into the James input spool. Fetch tasks run concurrently.
A set of filters applies to each fetch task. Each filter provides the ability to reject a message that matches the filter criteria. Rejected messages are not injected into the James input spool; they are either marked as seen or deleted. When a filter is configured to accept a message that matches its criteria, messages are marked with a MailAttribute. This MailAttribute can be detected within the James matcher/mailet chain, allowing further processing as required.
Each fetch task is associated with a single host server. Accounts are defined to the fetch task for each mailbox on the server from which mail is to be fetched. Accounts run consecutively.
Optionally, the fetch task can be configured with an <alllocal> Account that generates an Account entry for each user defined in the James user repository. This removes the requirement to manually add or remove Account entries to the Fetchmail configuration each time a James user is added or removed. Currently this is only useful if the server supports virtual mailboxes that allow the same password to apply to all users within a domain.
Accounts can be configured to deliver all mail for an Account to a specified recipient or to deduce the intended recipient from the mail headers.
Accounts are normally configured to deliver all mail for an Account to a specified recipient, ignoring the recipient in the mail headers. This works well in the majority of cases where a mailbox is guaranteed to contain mail for a sole mailbox recipient.
Accounts are configured to deduce the intended recipient from the mail headers when a mailbox contains mail for several users, typically all users in a domain. Used alone, this is not foolproof as there are circumstances when a single unique recipient cannot be deduced from the mail headers alone. Used in conjunction with an appropriately configured <alllocal> account, it is always possible to deduce the intended recipient when the recipient is a James user.
The Fetchmail configuration parameters are part of the James configuration, whose base file is fetchmail.xml. For clarity and flexibility, the Fetchmail configuration parameters are stored in the file james-Fetchmail.xml, which is referenced within fetchmail.xml.
The configuration parameters are described below.
The configuration block delimited by the Fetchmail tag controls Fetchmail.
The tag has these attributes:
The tag has these child tags (minimum cardinality, maximum cardinality):
<Fetchmail enabled="true">
...
</Fetchmail>
The fetch tag defines a fetch task to be run periodically. Fetch tasks run concurrently.
The tag has these attributes:
The tag has these child tags (minimum cardinality, maximum cardinality):
<fetch name="mydomain.com">
...
</fetch>
The accounts tag declares the accounts from which mail will be fetched by the fetch task. Accounts run concurrently.
The tag has these child tags (minimum cardinality, maximum cardinality):
<accounts>
...
</accounts>
The blacklist tag declares a list of recipient addresses for whom mail will be rejected and what happens to the rejected mail.
The tag value is a tab, comma or space delimited list of recipient addresses, eg: wibble@mydomain.com, flobble@mydomain.com.
The tag has these attributes:
<blacklist
reject="true"
leaveonserver="true"
markseen="true">
wibble@mydomain.com, flobble@mydomain.com
</blacklist>
The defaultdomain tag declares the domain name to be appended to the From: header of a mail that has a valid user part but is missing the domain part.
If not specified, the default behaviour is to append the canonical host name of the James server.
The tag value is the name of the server to append. The name must be a server declared in the servernames tag of the James block in the configuration or the name localhost.
<defaultdomain>
mydomain.com
</defaultdomain>
The fetchall tag declares if all mail should be fetched from the server, or just unseen mail.
The tag value is a boolean. If true, all mail is fetched. If false, only unseen mail is fetched.
<fetchall>false</fetchall>
The fetched tag declares what will happen to mail on the external server that is successfully injected into the James input spool.
The tag has these attributes:
<fetched leaveonserver="true" markseen="true"/>
The host tag declares the IP address of the external server from which mail is fetched.
The tag value is the DNS name or IP address literal of the external server.
<host>pop3.server.com</host>
The interval tag declares the period between invocations of the fetch tasks. If a fetch task is still active from a previous invocation when the period expires, the new invocation is skipped over.
The tag value is an integer representing the number of milliseconds to elapse between invocations of the fetch tasks.
<interval>60000</interval>
The javaMailFolderName tag declares the name of the root folder on the external server from which mail is fetched.
The tag value is the cAsE-sEnSiTiVe name of the root folder on the external server from which mail is fetched. For POP3 servers this is always INBOX.
<javaMailFolderName>INBOX</javaMailFolderName>
The javaMailProperties tag declares the properties to be applied to the JavaMail Session used by the fetch task. These override the properties answered by System.getProperties(). Many JavaMail properties are specific to the JavaMail Provider selected by the javaMailProviderName tag.
Relying on the default values selected by the Provider can be inappropriate. For instance, the default connection and I/O timeout values of infinite for the default IMAP and POP3 Providers is rarely what is required. Consult the documentation of the Provider for details and options.
Documentation for the default Provider for IMAP is located here.
Documentation for the default Provider for POP3 is located here.
Details of how to change a Provider are located here.
The tag has these child tags (minimum cardinality, maximum cardinality):
<javaMailProperties>
...
</javaMailProperties>
The javaMailProviderName tag selects the JavaMail protocol Provider used to interact with the external server.
The tag value is the name of a JavaMail supported protocol, such as pop3 or imap. The name is used to select the default Provider for the protocol.
<javaMailProviderName>pop3</javaMailProviderName>
The maxmessagesize tag declares the maximum permitted message size for messages injected into the James input spool and what happens to fetched messages that exceed this size.
The tag has these attributes:
<maxmessagesize
limit="4096"
reject="false"
leaveonserver="false"
markseen="false"/>
An example configuration using James mailet processing to bounce fetched messages that exceed the maximum permitted size can be found in the file $PHOENIX_HOME/apps/james/conf/samples/Fetchmail/maxMessageSize.xml.
The recipientnotfound tag declares what happens to mail for which a sole intended recipient cannot be found when attempting to determine the recipient from the mail headers.
In configurations with more than one account per fetch task, processing of matched mail can be deferred to the next run of the fetch task. This gives other accounts that may be able to determine a sole intended recipient an opportunity to do so before recipientnotfound processing is invoked.
The tag has these attributes:
<recipientnotfound
defer="true"
reject="true"
leaveonserver="true"
markseen="true"/>
The recursesubfolders tag declares if mail should be fetched from sub-folders of the root folder, or just the root folder.
The tag value is a boolean. If true, mail is fetched from the root folder and its subfolders. If false, mail is fetched from just the root folder.
<recursesubfolders>false</recursesubfolders>
The remoteReceivedHeader tag declares the zero based index of the RFC2822 compliant RECEIVED header used to determine the address and host name of the remote MTA that sent a fetched message and what happens to messages when the specified header is invalid.
Typically, the first (index = 0) RECEIVED header is for the local MTA that delivered the message to the message store and the second (index = 1) RECEIVED header is for the remote MTA that delivered the message to the local MTA. When this configuration applies, the remoteReceivedHeaderIndex should be set to 1.
To verify the correct setting, examine the RECEIVED headers for messages delivered to the configured message store and locate the first one containing a remote domain in the'from' field. Remembering that zero based indexing is used, if this the second header, use an index of 1, if this is the third header, use an index of 2, and so forth.
Matchers such as InSpammerBlacklist use the remote address and/or remote host name to identify illegitimate remote MTAs. If you do not use such matchers, the remoteReceivedHeaderIndex tag may be omitted or the default index value of -1 can be specified. This causes the remote address to be set to 127.0.0.1 and the remote host name to be set to localhost. Matchers almost always considered these values to be legitimate.
The tag has these attributes:
<remoteReceivedHeader
index="1"
reject="true"
leaveonserver="true"
markseen="true"/>
An example configuration using James mailet processing to notify the postmaster of fetched messages that contain an invalid Received header can be found in the file $PHOENIX_HOME/apps/james/conf/samples/Fetchmail/remoteReceivedHeader.xml.
The remoterecipient tag declares what happens to mail for which the domain part of the recipient is remote. A domain is remote if it is not a server declared in the servernames tag of the James block in the configuration.
The tag has these attributes:
<remoterecipient
reject="true"
leaveonserver="true"
markseen="true"/>
The undeliverable tag declares what happens to mail that cannot be delivered.
The tag has these attributes:
<undeliverable
leaveonserver="true"
markseen="true"/>
The userundefined tag declares what happens to mail for which the recipient is not defined as a James user.
The tag has these attributes:
<userundefined
reject="true"
leaveonserver="true"
markseen="true"/>
The account tag declares an account on the external server from which mail should be fetched.
The tag has these attributes:
<account
user="myaccount"
password="mypassword"
recipient="user@localhost"
ignorercpt-header="true"/>
The alllocal tag declares the parameters to be applied to dynamic accounts. The set of dynamic accounts is refreshed each time the fetch task runs by combining the alllocal tag attributes with each of the currently defined James users to create an account for every James user.
The tag has these attributes:
<alllocal
userprefix=""
usersuffix="@external.domain.com"
password="mypassword"
recipientprefix=""
recipientsuffix="@mydomain.com"
ignorercpt-header="true"/>
Full sources to the examples discussed below can be found in the directory $PHOENIX_HOME/apps/james/conf/samples/Fetchmail.
When all mail for an account is to be delivered to a single user, configure each account to ignore the recipient in the mail headers and deliver to the specified recipient. The accounts block looks like this:
<accounts>
<account
user="user1@external.domain.com"
password="password1"
recipient="user1@localhost"
ignorercpt-header="true"/>
<account
user="user2@external.domain.com"
password="password2"
recipient="user2@localhost"
ignorercpt-header="true"/>
<account
user="user3@external.domain.com"
password="password3"
recipient="user3@localhost"
ignorercpt-header="true"/>
</accounts>
When an account contains mail to be delivered to many users, configure each account to determine the recipient from the mail headers and deliver to that user. The accounts block looks like this:
<accounts>
<account
user="global@external.domain.com"
password="password"
recipient="Fetchmail@localhost"
ignorercpt-header="false"/>
</accounts>
The recipientnotfound tag is used to declare what happens when the recipient cannot be determined from the mail headers. In the example below, mail is injected into the spool using the recipient declared in the account tag:
<recipientnotfound
defer="false"
reject="false"
leaveonserver="false"
markseen="false"/>
When an external server supports virtual mailboxes, Fetchmail's dynamic account facility can be used. This greatly simplifies user configuration as the Fetchmail accounts for users are automatically synchronized with those defined in the James user repository. This guarantees that mail for all local users will be fetched and delivered.
Currently, there is a limitation that all virtual accounts and the global account must share the same password.
The alllocal tag declares the parameters for the dynamic accounts. The accounts block below will deliver mail for user1@external.domain.com to user1@localhost, user2@external.domain.com to user2@localhost, userZ@external.domain.com to userZ@localhost etc.:
<accounts>
<alllocal
userprefix=""
usersuffix="@external.domain.com"
password="mypassword"
recipientprefix=""
recipientsuffix="@localhost"
ignorercpt-header="true"/>
</accounts>
The One Account, One User - Dynamic example guarantees delivery of mail for all local users, but leaves other mail on the external server unprocessed. The One Account, Many Users example processes all mail on the external server, but cannot guarantee delivery to the intended recipient. By combining the two, it is possible to guarantee the delivery of mail for all local users and process all mail.
In the snippet below, the alllocal tag declares dynamic accounts for all local users and the account tag configures an account to fetch all mail.
The recipientnotfound tag rejects mail for which a recipient cannot be determined. By the time this processing is activated, the dynamic accounts will have processed mail for all local users, so the mail can only be mail for non-local users or newly arrived mail for local users. It is not possible to know which, but we want to leave mail for local users to be dealt with by the dynamic accounts. The next time the dynamic accounts run any newly arrived mail for local users will be processed. The remainder will be for non-local users and can now be safely dealt with.
The <recipientnotfound defer="true" attribute enables deferal of the processing of messages for which the recipient cannot be determined to the next iteration of the fetch task, and is used here. The relevant tags are:
<accounts>
<alllocal
userprefix=""
usersuffix="@external.domain.com"
password="mypassword"
recipientprefix=""
recipientsuffix="@localhost"
ignorercpt-header="true"/>
<account
user="global@external.domain.com"
password="password"
recipient="Fetchmail@localhost"
ignorercpt-header="false"/>
</accounts>
<recipientnotfound
defer="true"
reject="true"
leaveonserver="true"
markseen="true"/>
These are some things to be aware of when using Fetchmail:
Use this configuration to fetch Gmail via POP3. Please don't forget to enable POP3 in Gmail (read how-to).
<fetchmail enabled="true">
<fetch name="gmail.com">
<accounts>
<account user="YOUR_USER@gmail.com" password="YOUR_PASSWORD" recipient="YOUR_EMAIL_ADDRESS_IN_JAMES" ignorercpt-header="true"/>
</accounts>
<host>pop.gmail.com</host>
<interval>600000</interval>
<javaMailProviderName>pop3</javaMailProviderName>
<javaMailFolderName>INBOX</javaMailFolderName>
<javaMailProperties>
<property name="mail.pop3.ssl.enable" value="true"/>
</javaMailProperties>
<fetchall>false</fetchall>
<recursesubfolders>false</recursesubfolders>
<defaultdomain>localhost</defaultdomain>
<fetched leaveonserver="true" markseen="true"/>
<remotereceivedheader index="1" reject="true" leaveonserver="true" markseen="false"/>
<maxmessagesize limit="0" reject="false" leaveonserver="true" markseen="false"/>
<undeliverable leaveonserver="true" markseen="false"/>
<recipientnotfound defer="true" reject="true" leaveonserver="true" markseen="false"/>
<blacklist reject="true" leaveonserver="true" markseen="false"> wibble@localhost, flobble@localhost</blacklist>
<userundefined reject="true" leaveonserver="true" markseen="false" />
<remoterecipient reject="true" leaveonserver="true" markseen="false" />
</fetch>
</fetchmail>
Use this configuration to fetch Gmail via IMAP4. Please don't forget to enable IMAP4 in Gmail (read how-to).
<fetchmail enabled="true">
<fetch name="gmail.com">
<accounts>
<account user="YOUR_USER@gmail.com" password="YOUR_PASSWORD" recipient="YOUR_EMAIL_ADDRESS_IN_JAMES" ignorercpt-header="true"/>
</accounts>
<host>imap.gmail.com</host>
<interval>600000</interval>
<javaMailProviderName>imap</javaMailProviderName>
<javaMailFolderName>INBOX</javaMailFolderName>
<javaMailProperties>
<property name="mail.imap.ssl.enable" value="true"/>
</javaMailProperties>
<fetchall>false</fetchall>
<recursesubfolders>false</recursesubfolders>
<defaultdomain>localhost</defaultdomain>
<fetched leaveonserver="true" markseen="true"/>
<remotereceivedheader index="1" reject="true" leaveonserver="true" markseen="false"/>
<maxmessagesize limit="0" reject="false" leaveonserver="true" markseen="false"/>
<undeliverable leaveonserver="true" markseen="false"/>
<recipientnotfound defer="true" reject="true" leaveonserver="true" markseen="false"/>
<blacklist reject="true" leaveonserver="true" markseen="false"> wibble@localhost, flobble@localhost</blacklist>
<userundefined reject="true" leaveonserver="true" markseen="false" />
<remoterecipient reject="true" leaveonserver="true" markseen="false" />
</fetch>
</fetchmail>