active queue and courier maildrop

6 msg	How to reinstall postfix
9 msg	content filter - Altermime
14 msg	Service with perl in argv
16 msg	Postfix frontend on existing MySQL DB (routing ...
3 msg	postfix and mysql
5 msg	postfix do not send the raiables to maildrop
2 msg	Masquerading
1 msg	Relay and Domain masquerading

active queue and courier maildrop
\ John Seil (27 Jun 2007)
. \ Victor Duchovni (27 Jun 2007)
. . \ John Seil (28 Jun 2007)
. . . \ Victor Duchovni (30 Jun 2007)

5 msg	SASL2 - loose error message
5 msg	Setting up an intranet mail server
2 msg	simple smtp outbound how-to
1 msg	How to determine relay domains through check_po...
3 msg	message size limit per recipient
33 msg	Empty pflogsumm Reports
7 msg	Send On Behalf Of A User?
6 msg	multiple content_filters
4 msg	Blocked by policy mailin.rzone.de
7 msg	Build with support for both cyrus-sasl and dove...
14 msg	content filter help

Subject:	active queue and courier maildrop
Group:	Postfix-users
From:	John Seil
Date:	27 Jun 2007

Hello,

I have a Postfix (version 2.3.4-RC2) installation here with a
single purpose: accept incoming mail, and then deliver it to
virtual mailboxes using courier maildrop.

Maildrop requires that deliveries be destined for a single recipient
at a time, and so I have set:
virtual_destination_recipient_limit = 1

As per the postfix docs, setting *_destination_recipient_limit to 1
changes the meaning of concurrency from "per-domain" to
"per-recipient" (though it is never made clear if "per-recipient"
means just the "user" part before the @ sign, or the entire fqdn email
address). As I don't want any single mailbox to dominate the available
64 maildrop delivery slots (not to mention fights over mailbox locks
in the content filtering engine), I only want 1 message delivering at
a time to any given user, so I have set:
virtual_destination_concurrency_limit = 1

Mail delivers successfully one message at a time to any given
recipient, and delivers in parallel to different recipients. Things
are working as expected.

--

Setting up for the problem:

I provision two accounts in the mail system:
bob
alice

Now if I use 'postal' to simulate a mail bomb/DoS attack against a
single user, bob, I can exceed the maildrop delivery rate
and the Active queue will fill, followed by the Incoming queue
once the Active queue is maxed, as expected.

On the delivery side, only 1 message at a time can be delivered to
bob, per concurrency_limit above, as expected.

--

The Problem:

The problem comes in after I've slammed postfix with 60,000 messages
for bob. I have 20,000 msgs in Active, and 40,000 msgs in
Incoming all destined for bob, with no other mail in the
queue. It's slowly delivering the mail to the bob mailbox.

Now I send one single message to alice, which ends up in
the Incoming queue.

This message to alice does not deliver for quite a long time.
Postfix continues pulling in messages for bob from Incoming
to Active, even though it can only deliver one message at a time to
bob, not to mention that it already has an Active queue
of 19,999 msgs already destined for bob.

alice is effectively underserved/DoS'd from receiving
email. Eventually the email to alice will be picked up
and delivered, but it's often only after 10k-30k more msgs for
bob have been delivered, which can take a long time.

Because only one maildrop can run at a time for bob, there
are plenty of resources left to run ~60 more parallel maildrops,
which could all be busy delivering mail to other users such as
alice, if only there were Active slots available for them.

--

Resolutions:

What I would like is for Postfix to realize that it can only drain
messages to any single recipient at a rate of 1 at a time, and so
not fill up the Active queue with 20,000 messages for the same
recipient. Is there some way to limit the number of Active slots
allocated to any single recipient to, say, 2 or 3?

I tried tweaking the preemption delivery_slot_[cost|loan|discount]
settings, restarting postfix and re-doing the test from scratch,
but this didn't seem to make a difference. The mail to alice
is not selected before the other messages in Incoming all destined
to bob.

In general, mail to any one recipient (bob) should not be
able to lock out mail for all other recipients (alice, etc..)
via this using-up-all-active-queue-slots mechanism.

Any suggestions or links to pertinent documentation would be greatly
appreaciated.

-js