%gem_unpack is broken, because it changes the timestamp of the gemfiles. Perhaps this macro should be changed to record a certain date, perhaps the timestamp of its input. Then %gem_build should apply it after %patch. At least this seems to be the pattern.