I have a server running SuSe 9.2 Professional and dhcp-server-3.0.1-3. We are using the LDAP connectivity to store our configuration in an OpenLDAP DSA, so our dhcpd.conf looks like - ddns-update-style ad-hoc; ldap-server "localhost"; ldap-port 389; #ldap-username "uid=DHCPDaemon,ou=System Accounts,o=Morrison Industries,c=US"; #ldap-password "***************" ldap-base-dn "ou=ISCdhcpDaemon,ou=SubSystems,o=Morrison Industries,c=US"; ldap-method dynamic; This service works, but several times a day simply dies with a - Dec 16 18:33:07 littleboy dhcpd: Cannot search for (&(objectClass=dhcpHost)(dhcpHWAddress=ethernet 00:01:e6:90:31:65)) in LDAP tree cn=morrison-primary,ou=Configs,ou=ISCdhcpDaemon,ou=SubSystems,o=Morrison Industries,c=US: Can't contact LDAP server - left in /var/log/messages. Restarting the service works. This DSA is also used by Bind, NSS, Samba PDC, and a variety of other services all of which have no problem maintaing or establishing communication. We've used the LDAP patch for dhcpd for a long time, but just moved this server from RedHat 9 to SuSe 9.2, which is when this problem began to manifest.
On Fri, Dec 17, 2004 at 08:04:29AM -0500, Adam Tauno Williams wrote:
I have a server running SuSe 9.2 Professional and dhcp-server-3.0.1-3. We are using the LDAP connectivity to store our configuration in an OpenLDAP DSA, so our dhcpd.conf looks like -
ddns-update-style ad-hoc; ldap-server "localhost"; ldap-port 389; #ldap-username "uid=DHCPDaemon,ou=System Accounts,o=Morrison Industries,c=US"; #ldap-password "***************" ldap-base-dn "ou=ISCdhcpDaemon,ou=SubSystems,o=Morrison Industries,c=US"; ldap-method dynamic;
This service works, but several times a day simply dies with a -
Dec 16 18:33:07 littleboy dhcpd: Cannot search for (&(objectClass=dhcpHost)(dhcpHWAddress=ethernet 00:01:e6:90:31:65)) in LDAP tree cn=morrison-primary,ou=Configs,ou=ISCdhcpDaemon,ou=SubSystems,o=Morrison Industries,c=US: Can't contact LDAP server
- left in /var/log/messages.
Restarting the service works.
This DSA is also used by Bind, NSS, Samba PDC, and a variety of other services all of which have no problem maintaing or establishing communication.
We've used the LDAP patch for dhcpd for a long time, but just moved this server from RedHat 9 to SuSe 9.2, which is when this problem began to manifest.
Which (version of the) LDAP patch were you using last time it worked? BTW, I updated ftp://ftp.suse.com/pub/people/poeml/dhcp/9.2-i386 lately, you will find packages containing the latest patch there. It would be good to know if that one works better. For further analysis, I recommend to attach strace -tt /root/dhcpd.strace $(pidof dhcpd) to the running server. Peter
This service works, but several times a day simply dies with a - Dec 16 18:33:07 littleboy dhcpd: Cannot search for (&(objectClass=dhcpHost)(dhcpHWAddress=ethernet 00:01:e6:90:31:65)) in LDAP tree cn=morrison-primary,ou=Configs,ou=ISCdhcpDaemon,ou=SubSystems,o=Morrison Industries,c=US: Can't contact LDAP server - left in /var/log/messages. We've used the LDAP patch for dhcpd for a long time, but just moved this server from RedHat 9 to SuSe 9.2, which is when this problem began to manifest. Which (version of the) LDAP patch were you using last time it worked?
Not entirely certain, we had to build custom RPMs with whatever patch matched the version of DHCP included in RH9. This was at least a year ago I think.
BTW, I updated ftp://ftp.suse.com/pub/people/poeml/dhcp/9.2-i386 lately, you will find packages containing the latest patch there. It would be good to know if that one works better.
I've updated to these packages, we'll see how it goes.
For further analysis, I recommend to attach strace -tt /root/dhcpd.strace $(pidof dhcpd) to the running server.
Will try that if it fails.
BTW, I updated ftp://ftp.suse.com/pub/people/poeml/dhcp/9.2-i386 lately, you will find packages containing the latest patch there. It would be good to know if that one works better. I've updated to these packages, we'll see how it goes.
Just did the same thing again - Dec 17 14:13:56 littleboy dhcpd: DHCPACK on 192.168.1.94 to 00:0f:1f:98:35:ce (pc02052) via 192.168.1.19 Dec 17 14:50:41 littleboy dhcpd: Cannot search for (&(objectClass=dhcpHost)(dhcpHWAddress=ethernet 00:d0:b7:c1:19:63)) in LDAP tree cn=morrison-primary,ou=Configs,ou=ISCdhcpDaemon,ou=SubSystems,o=Morrison Industries,c=US: Can't contact LDAP server
For further analysis, I recommend to attach strace -tt /root/dhcpd.strace $(pidof dhcpd) to the running server.
Dec 17 15:29:47 littleboy dhcpd: Cannot search for (&(objectClass=dhcpHost)(dhcpHWAddress=ethernet 00:d0:b7:c1:15:84)) in LDAP tree cn=morrison-primary,ou=Configs,ou=ISCdhcpDaemon,ou=SubSystems,o=Morrison Industries,c=US: Can't contact LDAP server nohup strace -tt -o /root/dhcpd.strace -p 20296 & 15:28:36.492118 gettimeofday({1103315316, 492180}, NULL) = 0 15:28:36.492239 select(10, [4 7 8 9], [], [], {14531, 507820}) = 1 (in [4], left {14525, 617000}) 15:28:42.383155 gettimeofday({1103315322, 383274}, NULL) = 0 15:28:42.383356 recvfrom(4, "E\0\0008\331=\0\0\377\1_\32\300\250\1\23 \300\250\1\t\5"..., 1500, 0, {sa_family=AF_INET, sin_port=htons(32961), sin_addr=inet_addr("192.168.1.19")}, [16]) = 56 15:28:42.383626 gettimeofday({1103315322, 383688}, NULL) = 0 15:28:42.383764 select(10, [4 7 8 9], [], [], {14525, 616312}) = 1 (in [4], left {14525, 39000}) 15:28:42.962272 gettimeofday({1103315322, 962399}, NULL) = 0 15:28:42.962476 recvfrom(4, "E\0\0008\331?\0\0\377\1_\30\300\250\1\23 \300\250\1\t\5"..., 1500, 0, {sa_family=AF_INET, sin_port=htons(32961), sin_addr=inet_addr("192.168.1.19")}, [16]) = 56 15:28:42.962760 gettimeofday({1103315322, 962827}, NULL) = 0 15:28:42.962885 select(10, [4 7 8 9], [], [], {14525, 37173}) = 1 (in [4], left {14524, 364000}) 15:28:43.637871 gettimeofday({1103315323, 637991}, NULL) = 0 15:28:43.638068 recvfrom(4, "E\0\0008\331A\0\0\377\1_\26\300\250\1\23 \300\250\1\t\5"..., 1500, 0, {sa_family=AF_INET, sin_port=htons(32961), sin_addr=inet_addr("192.168.1.19")}, [16]) = 56 15:28:43.638345 gettimeofday({1103315323, 638408}, NULL) = 0 15:28:43.638466 select(10, [4 7 8 9], [], [], {14524, 361592}) = 1 (in [4], left {14523, 708000}) 15:28:44.293475 gettimeofday({1103315324, 293599}, NULL) = 0 15:28:44.293676 recvfrom(4, "E\0\0008\331E\0\0\377\1_\22\300\250\1\23 \300\250\1\t\5"..., 1500, 0, {sa_family=AF_INET, sin_port=htons(32961), sin_addr=inet_addr("192.168.1.19")}, [16]) = 56 15:28:44.293965 gettimeofday({1103315324, 294029}, NULL) = 0 15:28:44.294509 select(10, [4 7 8 9], [], [], {14523, 705971}) = 1 (in [4], left {14522, 499000}) 15:28:45.503263 gettimeofday({1103315325, 503407}, NULL) = 0 15:28:45.503487 recvfrom(4, "E\0\0008\331I\0\0\377\1_\16\300\250\1\23 \300\250\1\t\5"..., 1500, 0, {sa_family=AF_INET, sin_port=htons(32961), sin_addr=inet_addr("192.168.1.19")}, [16]) = 56 15:28:45.503831 gettimeofday({1103315325, 503902}, NULL) = 0 15:28:45.503962 select(10, [4 7 8 9], [], [], {14522, 496098}) = 1 (in [4], left {14501, 488000}) 15:29:06.513786 gettimeofday({1103315346, 513902}, NULL) = 0 15:29:06.513979 recvfrom(4, "E\0\0008\331\226\0\0\377\1^\301\300\250\1 \23\300\250\1"..., 1500, 0, {sa_family=AF_INET, sin_port=htons(32961), sin_addr=inet_addr("192.168.1.19")}, [16]) = 56 15:29:06.514264 gettimeofday({1103315346, 514328}, NULL) = 0 15:29:06.514386 select(10, [4 7 8 9], [], [], {14501, 485672}) = 1 (in [4], left {14494, 760000}) 15:29:13.241537 gettimeofday({1103315353, 241655}, NULL) = 0 15:29:13.241732 recvfrom(4, "E\0\0008\331\264\0\0\377\1^\243\300\250\1 \23\300\250\1"..., 1500, 0, {sa_family=AF_INET, sin_port=htons(32961), sin_addr=inet_addr("192.168.1.19")}, [16]) = 56 15:29:13.242001 gettimeofday({1103315353, 242064}, NULL) = 0 15:29:13.242121 select(10, [4 7 8 9], [], [], {14494, 757936}) = 1 (in [4], left {14494, 59000}) 15:29:13.941815 gettimeofday({1103315353, 941933}, NULL) = 0 15:29:13.942010 recvfrom(4, "E\0\0008\331\267\0\0\377\1^\240\300\250\1 \23\300\250\1"..., 1500, 0, {sa_family=AF_INET, sin_port=htons(32961), sin_addr=inet_addr("192.168.1.19")}, [16]) = 56 15:29:13.942298 gettimeofday({1103315353, 942365}, NULL) = 0 15:29:13.942423 select(10, [4 7 8 9], [], [], {14494, 57635}) = 1 (in [4], left {14490, 54000}) 15:29:17.947485 gettimeofday({1103315357, 947603}, NULL) = 0 15:29:17.947680 recvfrom(4, "E\0\0008\331\311\0\0\377\1^\216\300\250\1 \23\300\250\1"..., 1500, 0, {sa_family=AF_INET, sin_port=htons(32961), sin_addr=inet_addr("192.168.1.19")}, [16]) = 56 15:29:17.947959 gettimeofday({1103315357, 948022}, NULL) = 0 15:29:17.948080 select(10, [4 7 8 9], [], [], {14490, 51978}) = 1 (in [4], left {14489, 460000}) 15:29:18.545015 gettimeofday({1103315358, 545140}, NULL) = 0 15:29:18.545218 recvfrom(4, "E\0\0008\331\314\0\0\377\1^\213\300\250\1 \23\300\250\1"..., 1500, 0, {sa_family=AF_INET, sin_port=htons(32961), sin_addr=inet_addr("192.168.1.19")}, [16]) = 56 15:29:18.545504 gettimeofday({1103315358, 545567}, NULL) = 0 15:29:18.545625 select(10, [4 7 8 9], [], [], {14489, 454433}) = 1 (in [4], left {14487, 925000}) 15:29:20.075972 gettimeofday({1103315360, 76092}, NULL) = 0 15:29:20.076169 recvfrom(4, "E\0\0008\331\322\0\0\377\1^\205\300\250\1 \23\300\250\1"..., 1500, 0, {sa_family=AF_INET, sin_port=htons(32961), sin_addr=inet_addr("192.168.1.19")}, [16]) = 56 15:29:20.076458 gettimeofday({1103315360, 76520}, NULL) = 0 15:29:20.076578 select(10, [4 7 8 9], [], [], {14487, 923480}) = 1 (in [4], left {14487, 291000}) 15:29:20.709787 gettimeofday({1103315360, 709905}, NULL) = 0 15:29:20.709982 recvfrom(4, "E\0\0008\331\325\0\0\377\1^\202\300\250\1 \23\300\250\1"..., 1500, 0, {sa_family=AF_INET, sin_port=htons(32961), sin_addr=inet_addr("192.168.1.19")}, [16]) = 56 15:29:20.710250 gettimeofday({1103315360, 710312}, NULL) = 0 15:29:20.710370 select(10, [4 7 8 9], [], [], {14487, 289688}) = 1 (in [4], left {14485, 762000}) 15:29:22.239269 gettimeofday({1103315362, 239390}, NULL) = 0 15:29:22.239480 recvfrom(4, "E\0\0008\331\335\0\0\377\1^z\300\250\1\23 \300\250\1\t\5"..., 1500, 0, {sa_family=AF_INET, sin_port=htons(32961), sin_addr=inet_addr("192.168.1.19")}, [16]) = 56 15:29:22.239763 gettimeofday({1103315362, 239826}, NULL) = 0 15:29:22.239883 select(10, [4 7 8 9], [], [], {14485, 760174}) = 1 (in [4], left {14475, 368000}) 15:29:32.633504 gettimeofday({1103315372, 633628}, NULL) = 0 15:29:32.633722 recvfrom(4, "E\0\0008\332\4\0\0\377\1^S\300\250\1\23\300 \250\1\t\5\0"..., 1500, 0, {sa_family=AF_INET, sin_port=htons(32961), sin_addr=inet_addr("192.168.1.19")}, [16]) = 56 15:29:32.634002 gettimeofday({1103315372, 634065}, NULL) = 0 15:29:32.634123 select(10, [4 7 8 9], [], [], {14475, 365935}) = 1 (in [4], left {14474, 338000}) 15:29:33.662708 gettimeofday({1103315373, 662834}, NULL) = 0 15:29:33.662911 recvfrom(4, "E\0\0008\332\7\0\0\377\1^P\300\250\1\23\300 \250\1\t\5\0"..., 1500, 0, {sa_family=AF_INET, sin_port=htons(32961), sin_addr=inet_addr("192.168.1.19")}, [16]) = 56 15:29:33.663241 gettimeofday({1103315373, 663304}, NULL) = 0 15:29:33.663363 select(10, [4 7 8 9], [], [], {14474, 336696}) = 1 (in [4], left {14463, 153000}) 15:29:44.849121 gettimeofday({1103315384, 849240}, NULL) = 0 15:29:44.849317 recvfrom(4, "E\0\0008\332-\0\0\377\1^*\300\250\1\23\300 \250\1\t\5\0"..., 1500, 0, {sa_family=AF_INET, sin_port=htons(32961), sin_addr=inet_addr("192.168.1.19")}, [16]) = 56 15:29:44.849587 gettimeofday({1103315384, 849650}, NULL) = 0 15:29:44.849707 select(10, [4 7 8 9], [], [], {14463, 150350}) = 1 (in [8], left {14460, 111000}) 15:29:47.890814 gettimeofday({1103315387, 890947}, NULL) = 0 15:29:47.891030 recvfrom(8, "\1\1\6\1\32\34\337~\0\0\0\0\0\0\0\0\0\0\0\0 \0\0\0\0\300"..., 4096, 0, {sa_family=AF_INET, sin_port=htons(67), sin_addr=inet_addr("192.168.10.19")}, [16]) = 303 15:29:47.891444 time(NULL) = 1103315387 15:29:47.891575 write(5, "0\201\271\2\1\34c\201\263 \4Xcn=morrison-primary,o"..., 188) = 188 15:29:47.891887 select(1024, [5], [], NULL, NULL) = 1 (in [5]) 15:29:47.892311 read(5, "", 8) = 0 15:29:47.892493 time([1103315387]) = 1103315387 15:29:47.892637 open("/etc/localtime", O_RDONLY) = 0 15:29:47.892802 fstat64(0, {st_mode=S_IFREG|0644, st_size=811, ...}) = 0 15:29:47.892996 close(0) = 0 15:29:47.893133 open("/etc/localtime", O_RDONLY) = 0 15:29:47.893263 fstat64(0, {st_mode=S_IFREG|0644, st_size=811, ...}) = 0 15:29:47.893474 close(0) = 0 15:29:47.893622 open("/etc/localtime", O_RDONLY) = 0 15:29:47.893762 fstat64(0, {st_mode=S_IFREG|0644, st_size=811, ...}) = 0 15:29:47.894011 close(0) = 0 15:29:47.894137 rt_sigaction(SIGPIPE, {0x401155c0, [], 0}, {SIG_DFL}, 8) = 0 15:29:47.894320 send(3, "<27>Dec 17 15:29:47 dhcpd: Canno"..., 241, 0) = 241 15:29:47.894699 rt_sigaction(SIGPIPE, {SIG_DFL}, NULL, 8) = 0 15:29:47.894897 write(5, "0\5\2\1\35B\0", 7) = -1 EPIPE (Broken pipe) 15:29:47.895090 --- SIGPIPE (Broken pipe) @ 0 (0) ---
On Fri, Dec 17, 2004 at 03:33:32PM -0500, Adam Tauno Williams wrote:
For further analysis, I recommend to attach strace -tt /root/dhcpd.strace $(pidof dhcpd) to the running server.
Dec 17 15:29:47 littleboy dhcpd: Cannot search for (&(objectClass=dhcpHost)(dhcpHWAddress=ethernet 00:d0:b7:c1:15:84)) in LDAP tree cn=morrison-primary,ou=Configs,ou=ISCdhcpDaemon,ou=SubSystems,o=Morrison Industries,c=US: Can't contact LDAP server
nohup strace -tt -o /root/dhcpd.strace -p 20296 &
[...] Can you provide 'lsof -p $(pidof dhcpd)' output as well please? (Assuming that file descriptors are the same from run to run; better capture lsof output together with fresh strace) Does the dhcp server have a direct TCP connection with the LDAP server? No NAT being done in between? Peter
For further analysis, I recommend to attach strace -tt /root/dhcpd.strace $(pidof dhcpd) to the running server. Dec 17 15:29:47 littleboy dhcpd: Cannot search for (&(objectClass=dhcpHost)(dhcpHWAddress=ethernet 00:d0:b7:c1:15:84)) in LDAP tree cn=morrison-primary,ou=Configs,ou=ISCdhcpDaemon,ou=SubSystems,o=Morrison Industries,c=US: Can't contact LDAP server nohup strace -tt -o /root/dhcpd.strace -p 20296 & Can you provide 'lsof -p $(pidof dhcpd)' output as well please? (Assuming that file descriptors are the same from run to run; better capture lsof output together with fresh strace)
littleboy:~ # lsof -p 26678 COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME dhcpd 26678 dhcpd cwd DIR 253,5 168 35750 /var/lib/dhcp dhcpd 26678 dhcpd rtd DIR 253,5 168 35750 /var/lib/dhcp dhcpd 26678 dhcpd txt REG 253,5 576464 35780 /usr/sbin/dhcpd dhcpd 26678 dhcpd mem REG 253,5 106308 3094 /lib/ld-2.3.3.so dhcpd 26678 dhcpd mem REG 253,5 232916 35274 /usr/lib/libldap-2.2.so.7.0.8 dhcpd 26678 dhcpd mem REG 253,5 57741 35272 /usr/lib/liblber-2.2.so.7.0.8 dhcpd 26678 dhcpd mem REG 253,5 1359489 3120 /lib/tls/libc.so.6 dhcpd 26678 dhcpd mem REG 253,5 73086 3115 /lib/libresolv.so.2 dhcpd 26678 dhcpd mem REG 253,5 94878 67560 /usr/lib/libsasl2.so.2.0.19 dhcpd 26678 dhcpd mem REG 253,5 219497 30507 /usr/lib/libssl.so.0.9.7 dhcpd 26678 dhcpd mem REG 253,5 1135011 30505 /usr/lib/libcrypto.so.0.9.7 dhcpd 26678 dhcpd mem REG 253,5 13072 3103 /lib/libdl.so.2 dhcpd 26678 dhcpd mem REG 253,5 41661 3109 /lib/libnss_files.so.2 dhcpd 26678 dhcpd mem REG 253,5 217016 35690 /var/run/nscd/passwd dhcpd 26678 dhcpd mem REG 253,5 217016 35691 /var/run/nscd/group dhcpd 26678 dhcpd 0u REG 253,5 50893 53281 /var/lib/dhcp/db/dhcpd.leases dhcpd 26678 dhcpd 3u unix 0xef042e20 1684142 socket dhcpd 26678 dhcpd 4u raw 1684148 00000000:0001->00000000:0000 st=07 dhcpd 26678 dhcpd 5u IPv4 1684154 TCP localhost:33983->localhost:ldap (ESTABLISHED) dhcpd 26678 dhcpd 7u IPv4 1684161 UDP *:bootps dhcpd 26678 dhcpd 8u IPv4 1684160 UDP *:bootps dhcpd 26678 dhcpd 9u IPv4 1684162 TCP *:7911 (LISTEN) Will post the tail of 'strace -tt -o /root/dhcpd.strace -p 26678' as soon as it dies. BTW, dhcpd is running as "/usr/sbin/dhcpd -chroot /var/lib/dhcp/ -lf /db/dhcpd.leases -user dhcpd -group nogroup eth0"
Does the dhcp server have a direct TCP connection with the LDAP server? No NAT being done in between?
DHCP server is connecting to "localhost", iptables are empty, all default policies are accept. littleboy:~ # iptables -L Chain INPUT (policy ACCEPT) target prot opt source destination Chain FORWARD (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination I disabled all firewall rules at the beginning of this problem, to assist with troubleshooting.
participants (2)
-
Adam Tauno Williams
-
poeml@cmdline.net