Dovecot Sieve plugin
====================

Introduction
------------

Dovecot Sieve is a fully rewritten Sieve implementation for Dovecot v1.2 and
newer. The main reason for rewriting the Sieve engine was to provide more
reliable script execution and to provide better error messages to users and
system administrators. This implementation is part of the Pigeonhole project
[http://pigeonhole.dovecot.org/].

Getting the sources
-------------------

You can get the Dovecot Sieve plugin at this web page
[http://www.rename-it.nl/dovecot/1.2/]. The tarballs for the Sieve releases are
called 'dovecot-1.2-sieve-x.y.z.tar.gz' in which the x.y.z represents the
particular version.

Alternatively, you can obtain the latest version for Dovecot v1.2 from the
Mercurial repository:

---%<-------------------------------------------------------------------------
hg clone http://hg.rename-it.nl/dovecot-1.2-sieve
---%<-------------------------------------------------------------------------

Compiling
---------

Compilation is identical among the CMUSieve and the new Sieve plugin. Use
'--with-dovecot=<path>' to point to 'dovecot-config' file's directory. There
are two possibilities where this could exist:

 1. If you configured Dovecot with '--enable-header-install', you'll have
    'dovecot-config' installed in '$prefix/lib/dovecot/' directory. For
    example:

    ---%<---------------------------------------------------------------------
    ./configure --with-dovecot=/usr/local/lib/dovecot
    make
    sudo make install
    ---%<---------------------------------------------------------------------

 2. Compiled Dovecot sources' root directory. For example:
    ---%<---------------------------------------------------------------------
    ./configure --with-dovecot=../dovecot-1.2.0/
    make
    make install
    ---%<---------------------------------------------------------------------

If you downloaded the sources using Mercurial, you will need to execute
'./autogen.sh' first to build the automake structure in your source tree. This
process requires autotools and libtool to be installed.

Dovecot Sieve includes several command line tools to perform tasks such as
compile, verify and debug Sieve scripts (refer to the README file for more
information). These are built only if you use method 2, because they need to
link Dovecot's .a libraries, which can only be found from Dovecot's source tree
(make install doesn't install them).

Basic Configuration
-------------------

To use Sieve, you will first need to make sure you are using Dovecot's
<deliver> [LDA.txt] for delivering incoming mail to users' mailboxes. Then, you
need to enable the Sieve plugin in your 'dovecot.conf':

---%<-------------------------------------------------------------------------
protocol lda {
..
  # Support for dynamically loadable plugins. mail_plugins is a space separated
  # list of plugins to load.
  mail_plugins = sieve # ... other plugins like quota
}
---%<-------------------------------------------------------------------------

The sieve plugin recognizes the following configuration options in the 'plugin'
section of the config file (default values are shown if applicable):

sieve = ~/.dovecot.sieve :
  The path to the user's main active script.

sieve_global_path :
  A path to a global sieve script file, which gets executed ONLY if user's
  private Sieve script doesn't exist, e.g.'/var/lib/dovecot/default.sieve'. Be
  sure to pre-compile this script manually using the 'sievec' command line
  tool, as explained below.

sieve_global_dir = :
  Directory for :global include scripts for the include extension.

sieve_dir = ~/:
  Directory for :personal include scripts for the include extension.

sieve_extensions = :
  Which Sieve language extensions are available to users. By default, all
  supported extensions are available, except for deprecated extensions or those
  that are still under development. Some system administrators may want to
  disable certain Sieve extensions or enable those that are not available by
  default. Supported extensions are listed on <this page> [LDA.Sieve.txt].
  Normally, all enabled extensions must be listed for this setting, but
  starting with Sieve version 0.1.7, this setting can use '+' and '-' to
  specify differences relative to the default. For example 'sieve_extensions =
  +imapflags' will enable the deprecated imapflags extension in addition to all
  extensions enabled by default.

sieve_subaddress_sep = +:
  The separator that is expected between the :user and :detail address parts
  introduced by the subaddress extension. This may also be a sequence of
  characters (e.g. '--'). The current implementation looks for the separator
  from the left of the localpart and uses the first one encountered. The :user
  part is left of the separator and the :detail part is right.

For example:

---%<-------------------------------------------------------------------------
plugin {
...
   # The location of the user's active script:
   sieve = ~/.dovecot.sieve

   # If the user has no personal active script (i.e. if the file
   # indicated in sieve= does not exist), use this one:
   sieve_global_path = /var/lib/dovecot/sieve/default.sieve

   # The include extension fetches the :personal scripts from this
   # directory. When ManageSieve is used, this is also where scripts
   # are uploaded.
   sieve_dir = ~/sieve

   # The include extension fetches the :global scripts from this
   # directory.
   sieve_global_dir = /var/lib/dovecot/sieve/global/
}
---%<-------------------------------------------------------------------------

Per-user Sieve script location
------------------------------

By default, the Dovecot Sieve plugin looks for the user's Sieve script file in
the user's home directory ('~/.dovecot.sieve'). This requires that the <home
directory> [VirtualUsers.txt] is set for the user.

If you want to store the script elsewhere, you can override the default using
the 'sieve' setting, which specifies the path to the user's script file. This
can be done in two ways:

 1. Define the 'sieve' setting in the plugin section of 'dovecot.conf'.
 2. Return 'sieve' extra field from <userdb extra fields>
    [UserDatabase.ExtraFields.txt].

For example, to use a Sieve script file named '<username>.sieve' in
'/var/sieve-scripts', use:

---%<-------------------------------------------------------------------------
plugin {
...

 sieve = /var/sieve-scripts/%u.sieve
}
---%<-------------------------------------------------------------------------

You may use templates like %u, as shown in the example. See all <variables>
[Variables.txt].

A relative path (or just a filename) will be interpreted to point under the
user's home directory.

Executing Multiple Scripts Sequentially
---------------------------------------

The Dovecot Sieve plugin allows executing multiple Sieve scripts sequentially.
The extra scripts can be executed before and after the user's private script.
For example, this allows executing global Sieve policies before the user's
script. This is not possible using the 'sieve_global_path' setting, because
that is only used when the user's private script does not exist. The following
settings in the 'plugin' section of the Dovecot config file control the
execution sequence:

sieve_before = :
  Path to a script file or a directory containing script files that need to be
  executed before the user's script. If the path points to a directory, all the
  Sieve scripts contained therein (with the proper '.sieve' extension) are
  executed. The order of execution is determined by the file names, using a
  normal 8bit per-character comparison.

sieve_after = :
  Identical to 'sieve_before', only the specified scripts are executed after
  the user's script (only when keep is still in effect!).

The script execution ends when the currently executing script in the sequence
does not yield a "keep" result: when the script terminates, the next script is
only executed if an implicit or explicit "keep" is in effect. Thus, to end all
script execution, a script must not execute keep and it must cancel the
implicit keep, e.g. by executing "'discard; stop;'". This means that the
command "'keep;'" has different semantics when used in a sequence of scripts.
For normal Sieve execution, "'keep;'" is equivalent to "'fileinto "INBOX";'",
because both cause the message to be stored in INBOX. However, in sequential
script execution, it only controls whether the next script is executed. Storing
the message into INBOX (the default folder) is not done until the last script
in the sequence executes (implicit) keep. To force storing the message into
INBOX earlier in the sequence, the fileinto command can be used (with "':copy'"
or together with "'keep;'").

Apart from the keep action, all actions triggered in a script in the sequence
are executed before continuing to the next script. This means that when a
script in the sequence encounters an error, actions from earlier executed
scripts are not affected. The sequence is broken however, meaning that the
script execution of the offending script is aborted and no further scripts are
executed. An implicit keep is executed in stead.

Just as for executing a single script the normal way, the Dovecot Sieve plugin
takes care never to duplicate deliveries, forwards or responses. When vacation
actions are executed multiple times in different scripts, the usual error is
not triggered: the subsequent duplicate vacation actions are simply discarded.

For example:

---%<-------------------------------------------------------------------------
plugin {
...
   # Scripts executed before the user's script.
   #   E.g. handling messages marked as dangerous
   sieve_before = /var/lib/dovecot/sieve/discard-virusses.sieve

   # Scripts executed after the user's script (if keep is still in effect)
   #   E.g. default mail filing rules.
   sieve_after = /var/lib/dovecot/sieve/after.d/
}
---%<-------------------------------------------------------------------------

*IMPORTANT*: Be sure to manually pre-compile the scripts specified by
'sieve_before' and 'sieve_after' using the 'sievec' tool, as explained below.

Script Compiling
----------------

When the Sieve plugin executes a script for the first time (or after it has
been changed), it's compiled into into a binary form. Dovecot Sieve
implementation uses the '.svbin' extension to store compiled Sieve scripts
(e.g.'.dovecot.svbin'). To store the binary, the plugin needs write access in
the directory in which the script is located.

A problem occurs when a global script is encountered by the plugin. For
security reasons, global script directories are not supposed to be writable by
the user. Therefore, the plugin cannot store the binary when the script is
first compiled. Note that this doesn't mean that the old compiled version of
the script is used when the binary cannot be written: it compiles and uses the
current script version. The only real problem is that the plugin will not be
able to update the binary on disk, meaning that the global script needs to be
recompiled each time it needs to be executed, i.e. for every incoming message,
which is inefficient.

To mitigate this problem, the administrator must manually pre-compile global
scripts using the 'sievec' command line tool. For example:

---%<-------------------------------------------------------------------------
sievec /var/lib/dovecot/sieve/global/
---%<-------------------------------------------------------------------------

This is necessary for scripts listed in the 'sieve_global_path', 'sieve_before'
and 'sieve_after' settings. For global scripts that are only included in other
scripts using the include extension, this step is not necessary, since included
scripts are incorporated into the binary produced for the main script.

When manually compiling scripts with 'sievec', if those scripts use the
'include' sieve extension and your 'sieve_dir' is not the /sieve/ subfolder of
the directory of the main file, you can specify it by defining the 'SIEVE_DIR'
environment variable (e.g 'SIEVE_DIR=~/.sieve sievec .dovecot.sieve' )

Script Compiling with Sieve Plugins
-----------------------------------

Warnings, because at this time:

 * Sieve plugins require pigeonhole 0.1.15
 * sieve_extdata is the only plugin available, from
   http://hg.rename-it.nl/pigeonhole-sieve-extdata/

Plugins that require information stored in the global dovecot config will warn
or error when compiled (sievec), dumped (sieved), or tested (sieve-test)
because these tools have no access to dovecot.conf data when running.

There are two ways to provide this information:

(1) by setting an environment variable:

---%<-------------------------------------------------------------------------
bash$ export SIEVE_EXTDATA_DICT_URI=file:/etc/dovecot/somefile.dict
bash$ /usr/local/bin/sievec -d -P sieve_extdata frop.sieve
---%<-------------------------------------------------------------------------

(2) by runing from dovecot with --exec-mail:

---%<-------------------------------------------------------------------------
/usr/sbin/dovecot --exec-mail ext /usr/bin/sievec -P sieve_extdata frop.sieve
---%<-------------------------------------------------------------------------

See also:

 1. http://www.dovecot.org/list/dovecot/2010-January/045811.html
 2. http://www.dovecot.org/list/dovecot/2010-January/045870.html

Script Testing with Sieve Extdata Plugin
----------------------------------------

This plugin probably needs its own wiki page.

extdata allows a sieve script to lookup information from a datasource external
to the script.  The Pigeonhole implementation makes use of Dovecot's dict
mechanism in a read-only manner (scripts cannot update dict data sources!).

Requires:

 1. Dovecot config: sieve_extdata_dict_uri in plugins{}, or use environment
    variable SIEVE_EXTDATA_DICT_URI
 2. a valid dict URI.  (eg: file:/path/to/file, or sql:/path/to/dict-map-file)
 3. a "username", normally the uid of the recipient mailbox who's sieve script
    is executing, or use environment variable USER

for testing we use the sieve-test binary, first supply config & user to test
against:

---%<-------------------------------------------------------------------------
bash$ export SIEVE_EXTDATA_DICT_URI=file:/etc/dovecot/somefile.dict
bash$ export USER=test@domain1.test
---%<-------------------------------------------------------------------------

then run sieve-test with the extension, eg:

---%<-------------------------------------------------------------------------
/usr/local/bin/sieve-test -c -e -f sendingperson@domain2.test \
        -l maildir:/services/vmail/mail/1000000 \
        -r test@domain1.test -P sieve_extdata \
        /services/vmail/home/1000000/extdata-vacation.sieve /tmp/mail-file
---%<-------------------------------------------------------------------------

where extdata-vacation.sieve is something like this:

---%<-------------------------------------------------------------------------
require ["variables", "vacation", "vnd.dovecot.extdata"];

vacation :days 30 :subject "${extdata.vacation_subject}"
"${extdata.vacation_message}";
keep;
---%<-------------------------------------------------------------------------

where "priv/vacation_subject" & "priv/cacation_message" would be looked up in
the dict for user "test@domain1.test"

see <Quota.Dict.txt> relevant to Dovecot 1.2+ for more information on the use
of the Dict mechanism.

Compile and Runtime Logging
---------------------------

Log messages produced during compilation or runtime are written to two
locations:

 * A log file is written in the same directory as the user's main private
   script (as specified by the 'sieve' setting). This log file bears the name
   of that script file appended with ".log", e.g.'.dovecot.sieve.log'. If there
   are errors or warnings in the script, the messages are appended to that log
   file until it eventually grows too large (>10 kB currently). When that
   happens, the old log file is moved to a ".log.0" file and an empty log file
   is started. Informational messages are not written to this log file and the
   log file is not created until messages are actually logged, i.e. when an
   error or warning is produced.
 * Messages that could be of interest to the system administrator are also
   written to the Dovecot logging facility (usually syslog). This includes
   informational messages that indicate what actions are executed on incoming
   messages. Compile errors encountered in the user's private script are not
   logged here.

Migration from CMUSieve
-----------------------

For the most part, migration from CMUSieve to the new Sieve plugin is just a
matter of changing the used plugin name from *cmusieve* to *sieve* in the
'mail_plugins' option in the 'protocol lda' section of the config file (as
explained <above> [LDA.Sieve.Dovecot.txt]). However, there are a few important
differences:

 * The *imapflags* extension is now called *imap4flags*. The CMUSieve
   implementation is based on an old draft specification that is not completely
   compatible. Particularly, the *mark* and *unmark* commands were removed from
   the new specification. For backwards compatibility, support for the old
   imapflags extension can be enabled using the 'sieve_extensions' setting (as
   explained <above> [LDA.Sieve.Dovecot.txt]). This is disabled by default.
 * The *notify* extension is now called *enotify*. The CMUSieve implementation
   is based on an old draft specification that is not completely compatible.
   Particularly, the *denotify* command and *$text$* substitutions were removed
   from the new specification. For backwards compatibility, support for the old
   imapflags extension can be enabled using the 'sieve_extensions' setting (as
   explained <above> [LDA.Sieve.Dovecot.txt]). This is disabled by default.
 * The *include* extension now requires your script file names to end with
   ".sieve" :'include :personal "myscript";' won't work unless you rename
   "myscript" to "myscript.sieve"
 * Be sure to use *UTF8* for the mailbox argument of the *fileinto* command.
   Older CMUSieve installations used modified UTF7 (as IMAP does) for the
   mailbox parameter. If not adjusted, the new Sieve plugin will use the wrong
   folder name for storing the message.

(This file was created from the wiki on 2011-05-11 04:42)
