Weird samba issue

We have a number of servers that use samba, and "security = server". I know, unrecommended, but i cant change that for now.

The systems are running RHEL5.4, with samba-3.0.33-3.14.el5.

A few weeks ago we had a DC failure in our secondary site. These issues have started since then. The DC was replaced (or rather rebuilt). This samba system was working for the last few years, and when we upgraded the OS to RHEL5 about two months ago, this still worked fine. It was only after the DC crash that this happened.

Basically, automated builders connect to the samba share (on a VOB server), the samba share connects to the AD password server to authenticate.

We are getting random failures of the connection, which is annoying as these are automated builders. We can replicate it by repeated connections to the samba share, but it is sporadic. Sometimes it will connect, sometimes it wont.

We are not seeing any failures in the AD server. From the samba logs:

check_sam_security: Couldn't find user ' AD_USERNAME ' in passdb.
[2010/06/03 00:15:10, 3] libsmb/cliconnect.c:cli_session_setup(1027)
cli_session_setup: NT1 session setup failed: NT_STATUS_LOGON_FAILURE
[2010/06/03 00:15:10, 3] libsmb/cliconnect.c:cli_session_setup(1027)
cli_session_setup: NT1 session setup failed: NT_STATUS_REQUEST_NOT_ACCEPTED
[2010/06/03 00:15:10, 1] auth/auth_server.c:check_smbserver_security(363)
password server domain.controller rejected the password: NT_STATUS_REQUEST_NOT_ACCEPTED
[2010/06/03 00:15:10, 2] auth/auth.c:check_ntlm_password(319)
check_ntlm_password: Authentication for user [peerreview] -> [AD_USERNAME] FAILED with error NT_STATUS_REQUEST_NOT_ACCEPTED
[2010/06/03 00:15:10, 3] smbd/error.c:error_packet_set(106)
error packet at smbd/sesssetup.c(105) cmd=115 (SMBsesssetupX) NT_STATUS_REQUEST_NOT_ACCEPTED
[2010/06/03 00:15:20, 3] smbd/process.c:process_smb(1069)

here is the smb.conf:

[global]
workgroup = YYY
realm = XXXX.COM
security = SERVER
password server = animal.XXXX.com
log file = /var/log/samba/log.%m
log level = 3
max log size = 50
socket options = TCP_NODELAY SO_RCVBUF=16384 SO_SNDBUF=16384
preferred master = No
dns proxy = No
wins server = 172.16.164.100

#restrict anonymus on DC needs this:
# Winbind config. My additions.
idmap uid = 10000-20000
idmap gid = 10000-20000
winbind separator = /

netbios name = ccvob01blrpr
server string = Samba Server
name resolve order = wins host bcast
encrypt passwords = Yes
add user script = /usr/sbin/useradd -g ireland -s /bin/false %u
local master = No
domain master = No
kernel oplocks = No
create mask = 0755
directory mask = 0775
oplocks = Yes
level2 oplocks = Yes
guest ok = false
follow symlinks = Yes

winbind enum users = yes
winbind enum groups = yes
template homedir = /home/%U
template shell = /bin/bash
# this is the key, otherwise Exim sees Domain+Username and fails
winbind use default domain = yes

We are also sometimes seeing "sambatest" appearing in the AD auth logs, however there is no sambatest user in Ad or the RHEL machines.

This has been wrecking my head for about ten days and i dont have much more ideas of where the errors could be.

B