32 bit API for sendmsg using AppleTalk fails with invalid argument

17 Oct 2004

      Hello,

i am new to this list and look for some help regarding the sendmsg 
system call. I like to find out why the the sendmsg system call works 
for AppleTalk DDP sockets on Suse 9.1/ia32 and fails on a 64 bit Suse 
9.1 OS using the 32 bit existing binary.

I have tried to investigate where the 32bit API parameters gets 
converted into 64 bit. I believe there could be a binary API bug. Is the 
directory "linux/arch/x86_64/ia32" at good place to look?

It looks like the kernel will do booth system calls 32 bit and 64 bit. 
Has somebody an idea where the 32 syscall parameters are getting 
converted into 64 bit for the sendmsg system call?

I checked already the size of the struct sockaddr_at, it is identical on 
32 and 64 bit systems (16 bytes). At the end I have included a 
pre-processed code sequence where the sendmsg fails with Invalid 
argument error code.

Thanks

Helmut Tschemernjak

------------- snip ---
int DDPWrite(int skt, WDSElement *wdsPtr, AddrBlock dest, int type)
{
         struct iovec iov[16];
         register struct iovec *vp = iov;
         register int iovlen = 0;
         struct msghdr msg;
         uint8 typeCode;

         static struct sockaddr_netat sat = {
                 5
         };

         if (skt < 0 || skt > 256) {
                 _stdfprintf(sfstderr, "DDPWrite: skt out of range\n");
                 return ddpSktErr;
         }
         sat.sat_addr.s_net = dest.aNet;
         sat.sat_addr.s_node = dest.aNode;
         sat.sat_port = dest.aSocket;
         typeCode = type;
         vp->iov_base = (caddr_t)&typeCode;
         vp->iov_len = sizeof(typeCode);
         vp++;
         iovlen++;
         msg.msg_name = (caddr_t)&sat;
         msg.msg_namelen = sizeof(sat);

         while (wdsPtr->entryLength) {
                 vp->iov_base = (caddr_t)wdsPtr->entryPtr;
                 vp->iov_len = wdsPtr->entryLength;
                 vp++;
                 wdsPtr++;
                 iovlen++;
                 if (iovlen > 16)
                         return badBuffNum;
         }
         msg.msg_iov = iov;
         msg.msg_iovlen = iovlen;

         msg.msg_control = 0;
         msg.msg_controllen = 0;

         if (sendmsg(skt2fd[skt], &msg, 0) == -1) {
                 toolStatsPtr->ddpSendErrs++;
                 _stdperror("DDPWrite: sendmsg");
                 return (*__errno_location ()) == 11 ? wouldBlockErr : 
unixErr;
         }
         return noErr;
}