Between May 20th and May 23rd, we were experiencing some mail delivery delays due to two unconnected reasons.
First, mx5in seemed to have lost it's time sync -- and since everything around here works via Kerberos, and Kerberos requires a relativly synchronized clock to do it's thing, it wasn't able to get priviliges to delivery mail into folks' accounts, and a period over the weekend was queuing mail.
The second reason is a bit more strange -- it appears the the AFS client software on a couple of the mail delivery boxes didn't refresh it's volume location information; as we've been doing quite a few volume moves, it seems for a small subset (probably less that 100) of user volumes, these servers lost track of where they are. We noticed this monday morning, and forcibly refreshed the volume location information on all of our servers which kicked loose the mail that was pending delivery.