You have to set up a complete root filesystem with all the needed directories and files (/sbin /etc ...) in the chrooted directory.
Hope, that someone can give a minimum list of that.
You need all libraries including the linker (/lib/ld-linux.so.2) and the ld.so.cache in /etc that are needed/linked into the binaries you want to use. The libs can be seen using `ldd binary-file'.
Additionally, some libraries need files such as /etc/passwd, /etc/group and others as well as the terminfo database under /usr/share/terminfo or the timezone description file. Use strace to find out what's missing if the process won't complain loud enough. It's a nice piece of work, but it's possible. On the other hand, you could install a whole system into a directory (under SuSE, newer versions of yast support this). You could use this directory to host a chroot()ed process.
Be aware that running a process under root privileges renders the whole prison porous since a breakout is trivial once you can use chroot(2) within an already chroot()ed environment.