[Bug 1177579] wrong clamping of hexadecimal digits in dc (desk calculator)