LDAP routing + LDAP virtusertable : SendMail

Good day folks,

I'm looking to migrate our existing sendmail servers to using a common
LDAP backend for configuration. I've been working with the LDAP
routing part of this for a while now and am quite happy with the
results; however I've come to an issue I'm unable to unravel when it
comes to adding virtusertables to this mix.

First of all, I'd like to explain why I'm keen on using both
features. Even though LDAP routing can do 90% of what we currently
use our virtusertables for, it would seem that it doesn't support full
domain mapping ala "@foo.com -> %1@bar.org"... which is why I think we
need the virtusertable part of the equation. We also have multiple
internal IMAP servers, for which the "mailHost" feature of LDAP
routing is seemingly ideal.

So I have both LDAP routing and LDAP based virtusertable working quite
happily, however when combining the two it would seem there's an issue
with the order of operations. From what I can tell, the virtusertable
is being consulted after the LDAP routing is performed, thus the
correct mailHost for a given address is being established (or more
specifically, failing to be established) based on the original
recipient address rather than the result of the virtusertable mapping.

Another stumbling point it creates is for rejecting bad recipient
addresses. While testing LDAP routing, I found that it's bounce
option worked very well, however because it's being evaluated prior to
the virtusertable it will now bounce any address for which a
virtusertable mapping should have occurred.

There is of course the possibility of creating a mailLocalAddress
entry for each user at each domain and abandoning the virtusertable
altogether, but I can't help but feel that that is a somewhat dirty
solution and one which will incur additional administrative overhead
in the future.

If anyone has any suggestions or references as to how I might pull
this off... or ideas of alternate implementations, they'd be much
appreciated!

Cheers,

Mike.

===== Sanitized sendmail.mc file follows. =====
[mbowie@ml1 /etc/mail]$ cat ml1*.mc | grep -v \#
divert(-1)


divert(0)
VERSIONID(`$FreeBSD: src/etc/sendmail/freebsd.mc,v 1.30.6.1 2006/04/13
04:00:23 gshapiro Exp $')
OSTYPE(freebsd6)
DOMAIN(generic)

dnl No ident lookup for you! (Doing so reduces our connection times)
define(`confTO_IDENT', `0')

dnl Set our standard LDAP configuration
define(`confLDAP_CLUSTER', `foo')
define(`confLDAP_DEFAULT_SPEC',`-w3 -H ldap://ldap.company.com -p 389 -
b "ou=foo,dc=company,dc=com" -r search')

dnl LDAP Routing. We include the "bound parameter to reject unknown
addresses
dnl FEATURE(`ldap_routing', , , bounce)
FEATURE(`ldap_routing')
LDAPROUTE_DOMAIN_FILE(`@ldap:-k (&(objectClass=sendmailMTAClass)
(sendmailMTAClassName=w)(sendmailMTACluster=foo)) -v
sendmailMTAClassValue')

dnl Enable rewrite rules
FEATURE(`virtusertable', `LDAP')
VIRTUSER_DOMAIN_FILE(`@ldap:-k (&(objectClass=sendmailMTAClass)
(sendmailMTAClassName=w)(sendmailMTACluster=foo)) -v
sendmailMTAClassValue')

dnl Allow us to globally block stuff if we so desire
FEATURE(`access_db', `LDAP')
FEATURE(blacklist_recipients)

dnl Enable for both IPv4 and IPv6 (optional)
DAEMON_OPTIONS(`Name=IPv4, Family=inet')
dnl DAEMON_OPTIONS(`Name=IPv6, Family=inet6, Modifiers=O')

define(`confBIND_OPTS', `WorkAroundBrokenAAAA')
define(`confNO_RCPT_ACTION', `add-to-undisclosed')
define(`confPRIVACY_FLAGS', `authwarnings,noexpn,novrfy')

MAILER(local)
MAILER(smtp)

mbowie <ebuzmo@gmail.com> wrote:

> Good day folks,
>
> I'm looking to migrate our existing sendmail servers to using a common
> LDAP backend for configuration. I've been working with the LDAP
> routing part of this for a while now and am quite happy with the
> results; however I've come to an issue I'm unable to unravel when it
> comes to adding virtusertables to this mix.
>
> First of all, I'd like to explain why I'm keen on using both
> features. Even though LDAP routing can do 90% of what we currently
> use our virtusertables for, it would seem that it doesn't support full
> domain mapping ala "@foo.com -> %1@bar.org"... which is why I think we
> need the virtusertable part of the equation. We also have multiple
> internal IMAP servers, for which the "mailHost" feature of LDAP
> routing is seemingly ideal.
>
> So I have both LDAP routing and LDAP based virtusertable working quite
> happily, however when combining the two it would seem there's an issue
> with the order of operations. From what I can tell, the virtusertable
> is being consulted after the LDAP routing is performed, thus the
> correct mailHost for a given address is being established (or more
> specifically, failing to be established) based on the original
> recipient address rather than the result of the virtusertable mapping.
>
> Another stumbling point it creates is for rejecting bad recipient
> addresses. While testing LDAP routing, I found that it's bounce
> option worked very well, however because it's being evaluated prior to
> the virtusertable it will now bounce any address for which a
> virtusertable mapping should have occurred.
>
> There is of course the possibility of creating a mailLocalAddress
> entry for each user at each domain and abandoning the virtusertable
> altogether, but I can't help but feel that that is a somewhat dirty
> solution and one which will incur additional administrative overhead
> in the future.
>
> If anyone has any suggestions or references as to how I might pull
> this off... or ideas of alternate implementations, they'd be much
> appreciated!

Would you like to *test* cf/m4/proto.m4 patch to implement second pair of
ldapmra and ldapmh lookups if both queries in the first pair return
"no match"?

pair 1: ldapmra(user@domain) & ldapmh(user@domain)
pair 2: %1=user : ldapmra(@domain) & ldapmh(@domain)

It would require patching one file, generating new sendmail.cf but no
recompilation of sendmail binaries.

--
[pl>en Andrew] Andrzej Adam Filip : anfi@onet.eu : anfi@xl.wp.pl
Cats are intended to teach us that not everything in nature has a function.
-- Garrison Keillor

On Oct 16, 1:32 pm, Andrzej Adam Filip <a...@onet.eu> wrote:
> mbowie <ebu...@gmail.com> wrote:
> > So I have both LDAP routing and LDAP based virtusertable working quite
> > happily, however when combining the two it would seem there's an issue
> > with the order of operations.  From what I can tell, the virtusertable
> > is being consulted after the LDAP routing is performed, thus the
> > correct mailHost for a given address is being established (or more
> > specifically, failing to be established) based on the original
> > recipient address rather than the result of the virtusertable mapping.
>
> > Another stumbling point it creates is for rejecting bad recipient
> > addresses.  While testing LDAP routing, I found that it's bounce
> > option worked very well, however because it's being evaluated prior to
> > the virtusertable it will now bounce any address for which a
> > virtusertable mapping should have occurred.
>
> Would you like to *test* cf/m4/proto.m4 patch to implement second pair of
> ldapmra and ldapmh lookups if both queries in the first pair return
> "no match"?
>
> pair 1: ldapmra(user@domain) & ldapmh(user@domain)
> pair 2: %1=user : ldapmra(@domain) & ldapmh(@domain)
>
> It would require patching one file, generating new sendmail.cf but no
> recompilation of sendmail binaries.
>
> --
> [pl>en Andrew] Andrzej Adam Filip : a...@onet.eu : a...@xl.wp.pl
> Cats are intended to teach us that not everything in nature has a function.
>   -- Garrison Keillor

My apologies, it would appear I inadvertently replied via e-mail. (I
blame "Google groups".)

I'd be more than happy to experiment with a patch, although I
personally don't speak m4, so will need some guidance or at least a
starting point.

Sincerest thanks,

Mike.

mbowie <ebuzmo@gmail.com> wrote:

> On Oct 16, 1:32 pm, Andrzej Adam Filip <a...@onet.eu> wrote:
>> mbowie <ebu...@gmail.com> wrote:
>> > So I have both LDAP routing and LDAP based virtusertable working quite
>> > happily, however when combining the two it would seem there's an issue
>> > with the order of operations.  From what I can tell, the virtusertable
>> > is being consulted after the LDAP routing is performed, thus the
>> > correct mailHost for a given address is being established (or more
>> > specifically, failing to be established) based on the original
>> > recipient address rather than the result of the virtusertable mapping.
>>
>> > Another stumbling point it creates is for rejecting bad recipient
>> > addresses.  While testing LDAP routing, I found that it's bounce
>> > option worked very well, however because it's being evaluated prior to
>> > the virtusertable it will now bounce any address for which a
>> > virtusertable mapping should have occurred.
>>
>> Would you like to *test* cf/m4/proto.m4 patch to implement second pair of
>> ldapmra and ldapmh lookups if both queries in the first pair return
>> "no match"?
>>
>> pair 1: ldapmra(user@domain) & ldapmh(user@domain)
>> pair 2: %1=user : ldapmra(@domain) & ldapmh(@domain)
>>
>> It would require patching one file, generating new sendmail.cf but no
>> recompilation of sendmail binaries.
>
> My apologies, it would appear I inadvertently replied via e-mail. (I
> blame "Google groups".)
>
> I'd be more than happy to experiment with a patch, although I
> personally don't speak m4, so will need some guidance or at least a
> starting point.
>
> Sincerest thanks,
>
> Mike.

1) At the end of message please find "two liner" patch of
cf/m4/proto.m4 file as text and uuencoded attachment.
It should generate zero or one extra line in sendmail.cf.

It should generate nothing in sendmail.cf unless you use the
following two lines in your *.mc file:
define(`_LDAP_ROUTING_DOMFALL_')
FEATURE(`ldap_routing')

*REMEMBER*: "domain fall-back" lookups are done only if *both*
"per user" lookups return nothing.

2) Please *DO TEST IT* initially using

echo '3,0 user@example.com' | sendmail -C sendmail-test.cf -d60.5 -bt

3) Let me know if it works as you wanted
I would like to get one "positive test" before publishing it at
http://open-sendmail.sourceforge.net/

Feel free to submit the patch to sendmail.org

---------------------------------------------------------------------

*** proto.m4 2008/10/17 16:25:27 1.1
--- proto.m4 2008/10/17 16:41:34
***************
*** 1461,1466 ****
--- 1461,1468 ----
SLDAPExpand
# do the LDAP lookups
R<$+><$+><$*> $: <$(ldapmra $2 $: $)> <$(ldapmh $2 $: $)> <$1> <$2> <$3>
+ ifdef(`_LDAP_ROUTING_DOMFALL_', `dnl
+ R<><><$+><$+@$+><$*> $: <$(ldapmra @$3 $@ $2 $: $)> <$(ldapmh @$3 $@ $2 $: $)> <$1> <$2@$3> <$4>',`')

# look for temporary failures and...
R<$* <TMPF>> <$*> <$+> <$+> <$*> $: $&{opMode} $| TMPF <$&{addr_type}> $| $3

---------------------------------------------------------------------




--
[pl>en Andrew] Andrzej Adam Filip : anfi@onet.eu : anfi@xl.wp.pl
To follow foolish precedents, and wink
With both our eyes, is easier than to think.
-- William Cowper

On Oct 17, 9:48 am, Andrzej Adam Filip <a...@onet.eu> wrote:
>
> 1) At the end of message please find "two liner" patch of
>    cf/m4/proto.m4 file as text and uuencoded attachment.
>    It should generate zero or one extra line in sendmail.cf.
>
>    It should generate nothing in sendmail.cf unless you use the
>    following two lines in your *.mc file:
>      define(`_LDAP_ROUTING_DOMFALL_')
>      FEATURE(`ldap_routing')
>
>    *REMEMBER*: "domain fall-back" lookups are done only if *both*
>    "per user" lookups return nothing.
>
> 2) Please *DO TEST IT* initially using
>
> echo '3,0 u...@example.com' | sendmail -C sendmail-test.cf -d60.5 -bt
>
> 3) Let me know if it works as you wanted
>    I would like to get one "positive test" before publishing it at
>    http://open-sendmail.sourceforge.net/
>
>    Feel free to submit the patch to sendmail.org
>
> [pl>en Andrew] Andrzej Adam Filip : a...@onet.eu : a...@xl.wp.pl
> To follow foolish precedents, and wink
> With both our eyes, is easier than to think.
>   -- William Cowper

Good morning Andrzej,

That's superb... and much appreciated! After a quick application of
the patch and running my expect test script on the test server, it
would appear that it is indeed re-running the LDAP routes as expected.

One issue that this doesn't resolve however, is the use of the
"bounce" flag in LDAP routing, since the bounce occurs before the
virtusertable can be consulted. I have another issue to address right
now, but once I'm done, I may have a look and see if I can find a way
to "unload" the bounce mechanism the first time the LDAP route is
checked, if DOMFAIL is defined. I'll certainly let you know the
outcome of my "hacking". ;-)

Many thanks!

Mike.

mbowie <ebuzmo@gmail.com> wrote:

> On Oct 17, 9:48 am, Andrzej Adam Filip <a...@onet.eu> wrote:
>>
>> 1) At the end of message please find "two liner" patch of
>>    cf/m4/proto.m4 file as text and uuencoded attachment.
>>    It should generate zero or one extra line in sendmail.cf.
>>
>>    It should generate nothing in sendmail.cf unless you use the
>>    following two lines in your *.mc file:
>>      define(`_LDAP_ROUTING_DOMFALL_')
>>      FEATURE(`ldap_routing')
>>
>>    *REMEMBER*: "domain fall-back" lookups are done only if *both*
>>    "per user" lookups return nothing.
>>
>> 2) Please *DO TEST IT* initially using
>>
>> echo '3,0 u...@example.com' | sendmail -C sendmail-test.cf -d60.5 -bt
>>
>> 3) Let me know if it works as you wanted
>>    I would like to get one "positive test" before publishing it at
>>    http://open-sendmail.sourceforge.net/
>>
>>    Feel free to submit the patch to sendmail.org
>
> Good morning Andrzej,
>
> That's superb... and much appreciated! After a quick application of
> the patch and running my expect test script on the test server, it
> would appear that it is indeed re-running the LDAP routes as expected.
>
> One issue that this doesn't resolve however, is the use of the
> "bounce" flag in LDAP routing, since the bounce occurs before the
> virtusertable can be consulted.

Why do you need virtusertable with (experimental support of)
"per domain fallbacks" in ldap_routing?

> I have another issue to address right now, but once I'm done, I may
> have a look and see if I can find a way to "unload" the bounce
> mechanism the first time the LDAP route is checked, if DOMFAIL is
> defined.

You can use "domain fallback" lookups to make sure "bounce mechanism"
will not be triggered for the domain.

> I'll certainly let you know the outcome of my "hacking". ;-)
>
> Many thanks!
>
> Mike.

P.S.
I have scheduled publishing of the patch at open-sendmail for Sunday.

--
[pl>en Andrew] Andrzej Adam Filip : anfi@onet.eu : anfi@xl.wp.pl
Call on God, but row away from the rocks.
-- Indian proverb

On Oct 17, 10:44 am, Andrzej Adam Filip <a...@onet.eu> wrote:
> Why do you need virtusertable with (experimental support of)
> "per domain fallbacks" in ldap_routing?
>
> > I have another issue to address right now, but once I'm done, I may
> > have a look and see if I can find a way to "unload" the bounce
> > mechanism the first time the LDAP route is checked, if DOMFAIL is
> > defined.  
>
> You can use "domain fallback" lookups to make sure "bounce mechanism"
> will not be triggered for the domain.
>
> P.S.
> I have scheduled publishing of the patch at open-sendmail for Sunday.
>
> --
> [pl>en Andrew] Andrzej Adam Filip : a...@onet.eu : a...@xl.wp.pl
> Call on God, but row away from the rocks.
>   -- Indian proverb

Good afternoon Andrzej,

I'm not sure that I understand your reply.

I take your first question to be "why do I need this functionality in
the first place", which I believe is explained in my initial post.

Regarding the "bounce mechanism", I'm counting on the bounce feature
in LDAP routing to prevent acceptance of messages for invalid
recipients. With the patch as it stands, LDAP routing will deny
acceptance of the message on the "first pass", even though it would
have passed on the second try, after being rewritten by the
virtusertable.

My apologies if I've misunderstood your statements.

Thanks again for your assistance,

Mike.

mbowie <ebuzmo@gmail.com> wrote:

> On Oct 17, 10:44 am, Andrzej Adam Filip <a...@onet.eu> wrote:
>> Why do you need virtusertable with (experimental support of)
>> "per domain fallbacks" in ldap_routing?
>>
>> > I have another issue to address right now, but once I'm done, I may
>> > have a look and see if I can find a way to "unload" the bounce
>> > mechanism the first time the LDAP route is checked, if DOMFAIL is
>> > defined.  
>>
>> You can use "domain fallback" lookups to make sure "bounce mechanism"
>> will not be triggered for the domain.
>>
>> P.S.
>> I have scheduled publishing of the patch at open-sendmail for Sunday.
>
> Good afternoon Andrzej,
>
> I'm not sure that I understand your reply.
>
> I take your first question to be "why do I need this functionality in
> the first place", which I believe is explained in my initial post.
>
> Regarding the "bounce mechanism", I'm counting on the bounce feature
> in LDAP routing to prevent acceptance of messages for invalid
> recipients. With the patch as it stands, LDAP routing will deny
> acceptance of the message on the "first pass", even though it would
> have passed on the second try, after being rewritten by the
> virtusertable.
>
> My apologies if I've misunderstood your statements.

Could you post chain of ldap_routing and virtusertable rewrites you want
to achieve?

--
[pl>en Andrew] Andrzej Adam Filip : anfi@onet.eu : anfi@xl.wp.pl
Education is what survives when what has been learnt has been forgotten.
-- B. F. Skinner

On Oct 18, 9:57 am, Andrzej Adam Filip <a...@onet.eu> wrote:
> mbowie <ebu...@gmail.com> wrote:
> > On Oct 17, 10:44 am, Andrzej Adam Filip <a...@onet.eu> wrote:
> >> Why do you need virtusertable with (experimental support of)
> >> "per domain fallbacks" in ldap_routing?
>
> >> > I have another issue to address right now, but once I'm done, I may
> >> > have a look and see if I can find a way to "unload" the bounce
> >> > mechanism the first time the LDAP route is checked, if DOMFAIL is
> >> > defined.  
>
> >> You can use "domain fallback" lookups to make sure "bounce mechanism"
> >> will not be triggered for the domain.
>
> >> P.S.
> >> I have scheduled publishing of the patch at open-sendmail for Sunday.
>
> > Good afternoon Andrzej,
>
> > I'm not sure that I understand your reply.
>
> > I take your first question to be "why do I need this functionality in
> > the first place", which I believe is explained in my initial post.
>
> > Regarding the "bounce mechanism", I'm counting on the bounce feature
> > in LDAP routing to prevent acceptance of messages for invalid
> > recipients.  With the patch as it stands, LDAP routing will deny
> > acceptance of the message on the "first pass", even though it would
> > have passed on the second try, after being rewritten by the
> > virtusertable.
>
> > My apologies if I've misunderstood your statements.
>
> Could you post chain of ldap_routing and virtusertable rewrites you want
> to achieve?
>
> --
> [pl>en Andrew] Andrzej Adam Filip : a...@onet.eu : a...@xl.wp.pl
> Education is what survives when what has been learnt has been forgotten.
>   -- B. F. Skinner

Andrzej,

Absolutely... I probably should have included that in the original
post.

For example:
- domain-a.com is a virtuser rewrite to domain-b.com... as in "@domain-
a.com %1@domain-b.com".
- One of the mailboxes @domain-b.com has a mailRoutingAddress of
user@, mailLocalAddress entries for info@ and support@ and it also has
a mailHost entry for "mail0.isp".
- Mail coming in for info@domain-a.com will need to be subject to the
virtusertable mapping before the LDAP routing otherwise (a) LDAP
routing will bounce the message since it doesn't exist as a
mailLocalAddress and (b) the mailHost will not be matched, so the
message cannot be passed to the delivery server.
- So: (info@domain-a.com + virtusertable) = info@domain-b.com + LDAP
routing = user@domain-b.com via mail0.isp
- Also: (foo@domain-a.com + virtusertable) = foo@domain-b.com + LDAP
routing = message not accepted, as foo@domain-b.com has no
mailLocalAddress entry

- Another user @domain-b.com may have a regular mailRoutingAddress of
other@domain-b.com, with the mailLocalAddress of the same value but a
mailHost of "mail1.isp".
- Thus: (other@domain-a.com + virtusertable) = other@domain-b.com +
LDAP routing = other@domain-b.com via mail1.isp
- Also: (other@domain-b.com + virtusertable) = other@domain-b.com +
LDAP routing = other@domain-b.com via mail1.isp

I hope that makes some sense.

Thanks for sticking with me on this!

Mike.

On Oct 18, 1:28 pm, mbowie <ebu...@gmail.com> wrote:
> On Oct 18, 9:57 am, Andrzej Adam Filip <a...@onet.eu> wrote:
>
>
>
> > mbowie <ebu...@gmail.com> wrote:
> > > On Oct 17, 10:44 am, Andrzej Adam Filip <a...@onet.eu> wrote:
> > >> Why do you need virtusertable with (experimental support of)
> > >> "per domain fallbacks" in ldap_routing?
>
> > >> > I have another issue to address right now, but once I'm done, I may
> > >> > have a look and see if I can find a way to "unload" the bounce
> > >> > mechanism the first time the LDAP route is checked, if DOMFAIL is
> > >> > defined.  
>
> > >> You can use "domain fallback" lookups to make sure "bounce mechanism"
> > >> will not be triggered for the domain.
>
> > >> P.S.
> > >> I have scheduled publishing of the patch at open-sendmail for Sunday..
>
> > > Good afternoon Andrzej,
>
> > > I'm not sure that I understand your reply.
>
> > > I take your first question to be "why do I need this functionality in
> > > the first place", which I believe is explained in my initial post.
>
> > > Regarding the "bounce mechanism", I'm counting on the bounce feature
> > > in LDAP routing to prevent acceptance of messages for invalid
> > > recipients.  With the patch as it stands, LDAP routing will deny
> > > acceptance of the message on the "first pass", even though it would
> > > have passed on the second try, after being rewritten by the
> > > virtusertable.
>
> > > My apologies if I've misunderstood your statements.
>
> > Could you post chain of ldap_routing and virtusertable rewrites you want
> > to achieve?
>
> > --
> > [pl>en Andrew] Andrzej Adam Filip : a...@onet.eu : a...@xl.wp.pl
> > Education is what survives when what has been learnt has been forgotten..
> >   -- B. F. Skinner
>
> Andrzej,
>
> Absolutely... I probably should have included that in the original
> post.
>
> For example:
> - domain-a.com is a virtuser rewrite to domain-b.com... as in "@domain-
> a.com %...@domain-b.com".
> - One of the mailboxes @domain-b.com has a mailRoutingAddress of
> user@, mailLocalAddress entries for info@ and support@ and it also has
> a mailHost entry for "mail0.isp".
> - Mail coming in for i...@domain-a.com will need to be subject to the
> virtusertable mapping before the LDAP routing otherwise (a) LDAP
> routing will bounce the message since it doesn't exist as a
> mailLocalAddress and (b) the mailHost will not be matched, so the
> message cannot be passed to the delivery server.
> - So: (i...@domain-a.com + virtusertable) = i...@domain-b.com + LDAP
> routing = u...@domain-b.com via mail0.isp
> - Also: (f...@domain-a.com + virtusertable) = f...@domain-b.com + LDAP
> routing = message not accepted, as f...@domain-b.com has no
> mailLocalAddress entry
>
> - Another user @domain-b.com may have a regular mailRoutingAddress of
> ot...@domain-b.com, with the mailLocalAddress of the same value but a
> mailHost of "mail1.isp".
> - Thus: (ot...@domain-a.com + virtusertable) = ot...@domain-b.com +
> LDAP routing = ot...@domain-b.com via mail1.isp
> - Also: (ot...@domain-b.com + virtusertable) = ot...@domain-b.com +
> LDAP routing = ot...@domain-b.com via mail1.isp
>
> I hope that makes some sense.
>
> Thanks for sticking with me on this!
>
> Mike.

Hello Andrzej,

Do you think my examples there are feasible or should I consider
looking to implement this functionality in a different manner?

Cheers,

Mike.