Comment # 26 on bug 1165351 from Thomas Blume

(In reply to Thorsten Kukuk from comment #17)
> have a
> table for all possible file descriptors to lookup the data instead of going
> through a list of existing file descriptors. If you set the limit lower than
> possible, you could get a out of array access.
> The code was written at a time, when 1024 file handles was the maximum
> possible, not for that big numbers as of today.
> 
> I would discuss this on the tirpc mailing list. The whole code needs to be
> rewritten to be less memory consuming.

Looking for some more background I've found this:

https://www.shrubbery.net/solaris9ab/SUNWdev/ONCDG/p27.html

-->
The __rpc_dtbsize() function calls the getrlimit() function to determine the
maximum value that the system can assign to a newly created file descriptor.
The result is cached for efficiency.
--<

If that's the purpose I would assume that:

return (tbsize = (int)rl.rlim_max);

is correct.
So maybe we just need to subtract something from dtbsize in case it has an
insane value?