maildirmake and maildrop quota size problems

Hi folks,

first of all: We use maildrop-2.0.1 at the moment.

We have the following Problems:

we have one archive mailbox where all non-spam emails are cc'd to with
maildrop. A nightly running script creates weekly subfolders and moves the
emails in INBOX into the corresponding subfolder.
This mailbox may reach sizes up to 60 GB for two month!
Normally, it's size is about 10 GB for two month.

Now I wanted to increase the maildir-quota for this mailbox to 60 GB using
the maildirmake -q command, but it seems, that maildirmake stucks or runs
forever or at least for several hours when called on the mailbox with a
size of actually 4.7 GB. It doesn't matter if I remove the maildirsize file
before or not. Until a value of ~2GB everything seems to work correctly.

Inserting the new quota value directly into maildirsize does also not work!
When maildrop then cc's an email to the archive account, maildroprc is
exited with an error message like "timeout: over quota" directly!
The cc command in maildroprc is set in exception { ... }, but when maildrop
directly exits, the mail may be really lost when this error is not
discovered before the mail is deleted from the queue because the maildroprc
script never reaches the point when the email should be delivered to it's
final destination!

My first question is: Is there a general limit for maildir-quota or a limit
for maildirmake -q ?

Second one: Is the "exception {}" statement broken?

Many thanks in advance for every hint which will possibly lead me to a
solution,

yours Henri

| Daten- u. Netzwerkdesign für Internet und Intranet
| Installation & Pflege von Unternehmens-EDV-Strukturen
| Besuchen Sie das virtuelle Koblenz in Second Life:
| http://www.byteconcepts.com/content/...es.php/tPath/4

Henri Schomäcker writes:

> Inserting the new quota value directly into maildirsize does also not work!
> When maildrop then cc's an email to the archive account, maildroprc is
> exited with an error message like "timeout: over quota" directly!
> The cc command in maildroprc is set in exception { ... }, but when maildrop
> directly exits, the mail may be really lost when this error is not
> discovered before the mail is deleted from the queue because the maildroprc
> script never reaches the point when the email should be delivered to it's
> final destination!
>
> My first question is: Is there a general limit for maildir-quota or a limit
> for maildirmake -q ?

If you're on a 32 bit platform, your total mailbox size is generally limited
to a 32 bit value, which is 2gb. If on your specific platform you have
optional compiler flags for large file support, if you recompile everything
you should be able to calculate quotas that exceed >2gb.

> Second one: Is the "exception {}" statement broken?

No, it's not, but you may not be using it correctly. If you trap an
exception, and don't do anything else but terminate your filtering recipe,
the mail delivery will terminate with the normal 0 exit code, and your mail
server will, rightfully, presume that the message has been succesfully
delivered.

So, a mere exception{} merely lets you resume after fatal errors that would
normally terminate filtering. It's up to you to detect this condition, and
communicate it properly, to the mail server, if you so wish.


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEABECAAYFAkisnC8ACgkQx9p3GYHlUOKJZgCdFvHR4RRsYT 7OYTnlL4R/P3zH
Cn8An3Z1Ms21LphXyz+QjMhZ65DHTRGp
=9VOJ
-----END PGP SIGNATURE-----

Sam wrote:
>
> If you're on a 32 bit platform, your total mailbox size is generally
> limited to a 32 bit value, which is 2gb. If on your specific platform you
> have optional compiler flags for large file support, if you recompile
> everything you should be able to calculate quotas that exceed >2gb.
>
>> Second one: Is the "exception {}" statement broken?
>
> No, it's not, but you may not be using it correctly. If you trap an
> exception, and don't do anything else but terminate your filtering recipe,
> the mail delivery will terminate with the normal 0 exit code, and your
> mail server will, rightfully, presume that the message has been
> succesfully delivered.
>
> So, a mere exception{} merely lets you resume after fatal errors that
> would normally terminate filtering. It's up to you to detect this
> condition, and communicate it properly, to the mail server, if you so
> wish.

First of all, many thanks, Sam these are very good hints!

I entered a logline into the script code which tells me the maildirquota...
log "MAILDIRQUOTA is set to: $MAILDIRQUOTA"
....and really uses the correct value out of the database until 2 GB.
Great, but it's really strange: With every email I send to the archive
mailbox, maildrop changes the quota entry in maildirsize to 100 MB or other
values around 100 MB.
So when I use an IMAP client on that mailbox, it's always over
maildir-quota, althought when delivering, the 1.5 GB quota set in the DB is
correctly recognizede an respected.
Could you also help me with an explanation why maildrop does reset the
maildirquota with every mail it receives to a phantasy value?

At the moment, I'm trying to figure out how to get LFS (no, not Linux from
scratch, but Large-File-Support) with kernel gentoo-2.6.14-r5 on, like, you
see, a gentoo system.
It's strange because I'm sure it worked before. The system of the customer
did not change in any way.

But what I do not really understand are your hints for the script.
Please take a look at this "part" of code, in my eyes, when I take a look at
the example code, should be correct, or isn't it?

8<---------8<---------8<---------8<---------8<---------8<---------
if (!/^X-Spam-Level: \*\*\*\*\*/ && !/^X-Bogosity: *Yes/
&& !/^X-Amavis-Alert: *INFECTED/ && !/^Subject: *Eingang VOICE/ )
{
`reformail -D 8000 $ARCHIVE_DUPLICATES_CACHE`
if ( $RETURNCODE != 0 )
{
log "Delivering carbon-copy to ${ARCHIVE_HOME}maildir/"
exception {
cc "${ARCHIVE_HOME}maildir/"
}
}
else
{
log "Email already once delivered to the email archive, skipping elivery!"
}
}
# log "Searching for this email in the duplicates cache $DUPLICATES_CACHE
first..."
`reformail -D 8000 $DUPLICATES_CACHE`
if ( $RETURNCODE == 0 )
{
log "Email from $FROM to $LOGNAME already delivered, skipping delivery!"
exit
}

log "FINAL-DESTINATION - Finally delivering to $DEFAULT"
xfilter "reformail -A'Delivered-To: $LOGNAME'"
exception {
to "$DEFAULT"
}

log "Did NOT deliver to final recipient $LOGNAME"
8<---------8<---------8<---------8<---------8<---------8<---------

Many thanks again,
great to have you reading this,
yours Henri

Henri Schomäcker writes:

> Great, but it's really strange: With every email I send to the archive
> mailbox, maildrop changes the quota entry in maildirsize to 100 MB or other
> values around 100 MB.

If your existing quota exceeds 2gb, and you're on a 32bit platform, the
quota calculation overflows, and the results are undefined.

> But what I do not really understand are your hints for the script.
> Please take a look at this "part" of code, in my eyes, when I take a look at
> the example code, should be correct, or isn't it?
>
> 8<---------8<---------8<---------8<---------8<---------8<---------
> if (!/^X-Spam-Level: \*\*\*\*\*/ && !/^X-Bogosity: *Yes/
> && !/^X-Amavis-Alert: *INFECTED/ && !/^Subject: *Eingang VOICE/ )
> {
> `reformail -D 8000 $ARCHIVE_DUPLICATES_CACHE`
> if ( $RETURNCODE != 0 )
> {
> log "Delivering carbon-copy to ${ARCHIVE_HOME}maildir/"
> exception {
> cc "${ARCHIVE_HOME}maildir/"
> }

There's your problem right there. exception {} traps the delivery error,
and, as documented, execution continues after the exception {} block. Just
like after a successful delivery. So, at this point, you have nothing to
indicate whether the message was successfully delivered, or not.

You probably want to do something like:

DELIVERED=0
exception {
cc [ .. ]
DELIVERED=1
}

If exception traps the fatal error delivering the message, DELIVERED never
gets set to 1, so after the exception block you can test the variable to
determine if the message was delivered.


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEABECAAYFAkiungkACgkQx9p3GYHlUOIZUACeI8sWP7I+92 sRD/DzmfKq5ZXD
O/gAnRLKRUtRVCrR24P9ZW0mxNXrBNhA
=cpKd
-----END PGP SIGNATURE-----

Sam wrote:

>> But what I do not really understand are your hints for the script.
>> Please take a look at this "part" of code, in my eyes, when I take a look
>> at the example code, should be correct, or isn't it?
>>
>> 8<---------8<---------8<---------8<---------8<---------8<---------
>> if (!/^X-Spam-Level: \*\*\*\*\*/ && !/^X-Bogosity: *Yes/
>> && !/^X-Amavis-Alert: *INFECTED/ && !/^Subject: *Eingang VOICE/ )
>> {
>> `reformail -D 8000 $ARCHIVE_DUPLICATES_CACHE`
>> if ( $RETURNCODE != 0 )
>> {
>> log "Delivering carbon-copy to ${ARCHIVE_HOME}maildir/"
>> exception {
>> cc "${ARCHIVE_HOME}maildir/"
>> }
>
> There's your problem right there. exception {} traps the delivery error,
> and, as documented, execution continues after the exception {} block. Just
> like after a successful delivery. So, at this point, you have nothing to
> indicate whether the message was successfully delivered, or not.
>
> You probably want to do something like:
>
> DELIVERED=0
> exception {
> cc [ .. ]
> DELIVERED=1
> }
>
> If exception traps the fatal error delivering the message, DELIVERED never
> gets set to 1, so after the exception block you can test the variable to
> determine if the message was delivered.

Oh, you missunderstood me:
What you wrote is correct and that's actually what I really do, but the
point is, that execution gets stuck inside the exception block!
It does NOT continue afterwards! - That's the reason why I asked if the
exception code may be broken.
It seems that inside the exception block, while cc-ing, calculating the
quota does take forever and a timeout is reached, but then, in my opinion,
it should be like what the exception block is for, continue afterwards,
like you also wrote.

But anyway, many thanks for your help Sam,
yours Henri