[Bug 374911] change the way duplicates are handled