ctrlPlanner Sign in

Why is my converted tolerance wider than the original?

Last updated 10 August 2026

Because both limits were rounded to nearest. Round lower limits up and upper limits down instead, and the converted band stays inside the original.

Rounding a single measurement to nearest is correct. Rounding a pair of limits to nearest is not, because a limit pair is not two numbers, it is a statement about what will be accepted. Round each end to nearest and about half the time both ends move outward, producing a band that admits parts the original drawing did not.

The worked case above is a real one. The honest conversion and the naive one differ by a thousandth of a millimeter, which sounds like nothing until you notice it is entirely in the direction that passes a part the drawing rejects, on a document somebody is going to inspect against.

Round inward instead: lowers up, uppers down. The cost is that you can occasionally reject a part that was in tolerance by a fraction of the rounding increment, which is the direction an inspection error is allowed to go.

Worked

Given	0.50003 / 0.50203 in
Rounded to nearest, which is wrong	12.701 / 12.752 mm
Rounded inward	12.701 / 12.751 mm

Rounded inward: the lower limit rounds up and the upper rounds down, so the converted band is contained inside the original. Anything that passes on this band would also have passed on the drawing it came from.
Rounding both ends to nearest, which is what a calculator does, would give 12.701 / 12.752 -- a band 0.0010 mm wider than the drawing allows. A part inside that and outside this one would pass on the converted print and fail on the original.

Open this case in the calculator, and change the numbers.

Computed by ctrlPlanner. ASME Y14.5, ISO 286.