Spent some time trying to go to the bottom of this. There is some memory management problem caused by realloc of arrays between xterm and fontconfig I think, but I am not seeing the actual problem yet.