Converting a tolerance band, and why surface finish does not convert
Rounding two limits is not like rounding one number
Converting a single measurement and rounding it to nearest is correct. Converting a pair of limits and rounding each end to nearest is not, and the reason is that a limit pair is not two numbers, it is a promise about what will be accepted. Round each end to nearest and roughly half the time you move both ends outward, which produces a band that admits parts the original drawing did not.
The error is small in absolute terms and it is entirely in one direction. It is tolerance nobody granted, appearing on a document somebody is going to inspect against, and it will never be caught by review because the converted numbers look exactly as reasonable as the right ones.
Round inward instead
The rule is that lower limits round up and upper limits round down. Both ends move toward the middle, so the converted band is always contained inside the original. Anything that passes on the converted print would also have passed on the drawing it came from.
The cost is real and worth stating: you can occasionally reject a part that was actually in tolerance, by a fraction of the rounding increment. That is the direction an inspection error is allowed to go. Scrapping a good part is expensive; shipping a bad one to an aerospace customer is a different category of problem.
Worth knowing which way your CAD system and your CMM software round when they convert, because most of them round to nearest and do not tell you.
Surface finish pairs, it does not convert
Ra 63 microinches is Ra 1.6 micrometers. That looks like a conversion and is not one. Both values are entries on the preferred-number series defined by ISO 4288 and ASME B46.1, and each is a standard value in its own right. The ladder pairs them; arithmetic does not produce them.
The habit of dividing by 39.37 survives because of an accident: 63 divided by 39.37 is 1.600, so the one value most people check by hand is the one where the shortcut appears to work. It stops working immediately either side of it. 125 divides to 3.175 where the standard value is 3.2. 32 divides to 0.813 where the standard value is 0.8. 250 divides to 6.35 against a standard 6.3.
A drawing carrying Ra 3.175 is a drawing that has been through a calculator, and it tells the coating or grinding shop reading it that nobody checked the standard. It also cannot be matched against a comparator block, because comparator blocks are made to the ladder.
When the number is not on the ladder at all
Somebody will hand you a 50 microinch requirement sooner or later. There is no partner for it, because 50 is not a preferred number. The right answer is not to snap it silently to 63 or to 32 and carry on; it is to say the value is off-ladder, name the nearest rung, and let the person decide with their customer whether the requirement tightens or opens. Then record which, because that decision is a deviation from the drawing and it needs to exist in writing.
Mixed-unit prints are normal now
A shop supplying both a European prime and a domestic one will hold drawings in both units, and increasingly a single drawing carries a metric basic dimension against inch stock, or an inch print with a metric thread on it. The conversion problem stops being a one-off and becomes something that happens on every job, which is the point at which doing it by hand in a spreadsheet starts producing the quiet errors described above.
Doing it across a whole plan
Converting one dimension takes two minutes. Converting a print with forty characteristics, keeping every band inside its original, every finish on the ladder and every thread callout intact, is where the mistakes get in. ctrlPlanner holds both units on the same plan and applies the same inward rounding everywhere, so the control plan and the AS9102 forms come out in whichever unit the customer asked for without anyone retyping a number.
Common questions
Why is my converted tolerance wider than the original?
Because whatever converted it rounded both limits to nearest. Rounding a lower limit down and an upper limit up adds tolerance the drawing never granted. Round lowers up and uppers down instead, and the converted band stays inside the original.
How many decimals should a converted inch tolerance have in millimeters?
Three is the usual working precision, and four when the inch band is tighter than about half a thousandth. The test is whether the converted band still fits inside the original at the precision you chose; if it does not, you need another decimal place.
Is Ra 63 microinches exactly 1.6 micrometers?
They are paired standard values rather than a conversion of one another. 63 divided by 39.37 happens to come out at 1.600, which is why the shortcut looks right, but 125 divides to 3.175 against a standard 3.2 and 32 divides to 0.813 against a standard 0.8.
What do I do with a finish value that is not on the ladder?
Treat it as a question for the customer rather than a rounding problem. Name the nearest standard value, agree whether the requirement tightens or opens, and record the decision.
Does this convert GD&T tolerances too?
The zone value converts like any other tolerance and the same inward rounding applies. Basic dimensions convert as exact values because they carry no tolerance of their own.
Does anything leave my machine?
No. There is no server component and nothing is transmitted. Close the tab and it is gone.
More about ctrlPlanner · Free allowance calculator · Units and precision on a plan