• 1-888-TEC-EASE
  • Outside USA: 1-716-785-6015
Free Tec-Ease Mug GD&T Textbooks GD&T Training GD&T Public Seminars GD&T Computer Based Training GD&T Reference Material
Don’t Add Tolerance by Rounding When Converting Dimensions and Tolerances
(In accordance with Y14.5M-1994 standard)

PDF of this Tip

A lot of folks these days are converting dimensions and tolerances from millimeters to inches. When doing this, it is very common to round off incorrectly. In fact, many CAD systems do not round according to the standards. The result is that the conversion/rounding process often adds tolerance to dimensions. Here is a better way. First, everyone should know that 1inch exactly equals 25.4 millimeters. If you are using 0.03937-STOP. This conversion value is not exact. When dealing with tight tolerances, such as those encountered with grinding tolerances, you may calculate incorrect values using 0.03937. Don’t multiply by 0.03937; divide by 25.4.

Once you convert, it is important that you round off correctly. Take a dimension and tolerance of 11±0.3mm. The limits of the dimension are 11.3mm and 10.7mm. When these limits are converted to inches, the limits will be .444881… in. and .421259… in. If you round to 3 decimal places for common machine shop work, the limits would be rounded to .445 in. and .421 in., using the methods taught in most schools. These values have been represented graphically. As you can see, the conversion and rounding process has actually added tolerance to the limits. The ANSI/IEEE 268 standard, referred to in Y14.5, tells us that "where the original limits may not be violated" this method is unacceptable. According to the Standard, in such a case you must always round the upper limit down and the lower limit up. In other words, you should round both numbers "in" towards the median or center dimension. This method results in tighter tolerances but guarantees that you never open up a tolerance when converting units.

Of course, if it is designed in mm and you build in mm, this is not an issue. But, if you insist on converting metric values to inches, do it right.

Back to Tips Tip added Sep 2005