NavArchHandbook0 ch

No chapters yet.

Engineering toolsIndex of symbolsGlossaryReferences
NavArchHandbook
NavArchHandbook
↵
BookmarksTools
Home›1. Mathematics & Units›1.9 Significant figures, rounding and reporting of results

Chapter 1 / Section 1.9

Significant figures, rounding and reporting of results

Print this section

Significant figures

The digits of a number that carry information about its precision. 0.70 m has two significant figures (precision ±0.005 m); 0.7 m has one (±0.05 m). In stability work "GM = 0.7 m" and "GM = 0.70 m" are different statements. Trailing zeros after a decimal point are significant; leading zeros are not.

Rules of thumb for calculations

  1. Carry at least one extra significant figure through intermediate steps; round only the reported result.
  2. Multiplication and division: the result has as many significant figures as the least precise input (relative precision).
  3. Addition and subtraction: the result has as many decimal places as the least precise input (absolute precision). KM−KGKM - KGKM−KG with KM = 5.98 and KG = 5.2 gives GM = 0.8, not 0.78.
  4. Constants (π\piπ, ggg, conversion factors) are exact for this purpose; use at least 4 figures.
  5. State the precision of inputs taken from drawings and tables: dimensions to the mm are not known to the mm on a steel ship (building tolerance ± several mm on frame spacing; IACS Rec. 47).

Rounding

Round half away from zero (rule A of ISO 80000-1: 2.45 → 2.5) is the common engineering convention; round half to even (rule B: 2.45 → 2.4, 2.55 → 2.6) avoids bias in long sums and is used by most software (IEEE 754 default). State which convention is used when results sit on a criterion boundary — a GM of 0.3449 m rounded to 0.34 fails a 0.35 m criterion; rounded to 0.345 it is ambiguous. Never round in the direction that helps compliance: compare the unrounded value with the criterion and report the margin.

Reporting precision used in this handbook

See the table. The principle: report to the precision at which the quantity can be measured or built, and one more figure where the number enters a criterion.

Units and symbols in reports (ISO 80000-1)

  • Space between number and unit: 6314 t, 25 kPa, 0.70 m; no space for ° (5°).
  • Decimal marker: point in English reports (ISO allows comma or point; the EU and Croatian convention is the comma — state the convention once in the report).
  • Thin space or comma as thousands separator consistently: 6 314 t or 6,314 t — never both in one document.
  • Unit symbols are not pluralised and take no full stop: 5 t, not 5 ts. or 5 t.
  • Write the quantity symbol in italics (LBPL_{BP}LBP​), units upright (m).
  • Products of units with a centred dot: kN·m, t·m; avoid "kNm".

Presenting a calculation

Inputs (with source and units) → formulas (with references) → results → comparison with allowables (unity check or margin) → conclusion. Every table with results should carry the load case, the units in the header and the source of the allowable. The pass/fail statement must quote both the computed value and the limit.

Formulas

01Unity check (usage factor)
UC=computed valueallowable value≤1.0UC = \frac{\text{computed value}}{\text{allowable value}} \le 1.0UC=allowable valuecomputed value​≤1.0

Result unit -

Report UC to two decimals (e.g. 0.87); the margin is (1 − UC) × 100 %.

02Margin against a criterion
margin=actual−requiredrequired×100 %\text{margin} = \frac{\text{actual} - \text{required}}{\text{required}} \times 100\ \%margin=requiredactual−required​×100 %

Result unit %

Tables

Recommended reporting precision

QuantityReport toExampleReason
Ship dimensions L, B, D0.01 mL_BP = 100.00 mDrawing precision
Draughts, trim0.01 m (1 cm)T_F = 5.55 mDraught marks read to 1 cm
Displacement1 t (0.1 t below 1000 t)Δ = 6314 tTPC ≈ 12 t/cm → 1 t ≈ 1 mm
Volumes1 m³ (0.1 m³ for tanks)∇ = 6160 m³—
KB, KM, KG, GM0.01 m (0.001 m in intermediate steps)GM = 0.70 mGM criteria in cm
GZ0.001 mGZ₃₀ = 0.485 mEnters area integrals
Areas under GZ curve0.0001 m·rad0.1166 m·radCriteria at 0.055 / 0.090 / 0.030
Angles (heel, trim, φf)0.1°φ₀ = 9.6°—
Form coefficients0.001C_B = 0.700—
TPC, MCT1cm0.01 t/cm; 0.1 t·m/cm12.05 t/cm; 59.8 t·m/cm—
Plate thickness, scantlings0.1 mm (requirement); 0.5 mm steps (supply)t_req = 9.3 mm → 9.5 mm (or 10 mm)Rolled plate availability
Section modulus, inertia1 cm³; 1 cm⁴Z = 307 cm³Rule formulas in cm units
Stress1 MPaσ = 99 MPaMaterial scatter ≫ 1 MPa
Forces, moments (structural)0.1 kN; 1 kN·mF = 274.7 kN; M = 2044 kN·m—
Unity checks0.01UC = 0.50—
Mass items in loading table0.1 t with 0.01 m centres12.5 t at Kg 7.20 mSum precision
Source: Handbook convention; consistent with the precision of the examples in Chapters 4–5.

Examples

#01

Rounding at a criterion boundary

Worked example

Problem

A fishing vessel's computed fluid GM is 0.3471 m against a required 0.35 m. The hydrostatic KM is known to ±0.01 m. How should the result be reported?

Solution

Report 'GM_f = 0.347 m < 0.35 m required — does not comply; shortfall 0.003 m (0.9 %)'. Rounding to 0.35 m would disguise a failure. The ±0.01 m uncertainty of KM means the true GM may lie between 0.337 and 0.357 m; a result inside the uncertainty band of the criterion is a failure for approval purposes and must be resolved by design change (e.g. 5 t of low ballast lowers KG by about 0.03 m on a 450 t vessel), not by rounding.

References

ISO 80000-1:2022 Quantities and units — Part 1: GeneralClause Clause 7 (Printing rules for symbols and numbers), Annex B (Rounding of numbers)Recommended
NIST SP 811 (2008)Clause Ch. 7 and Appendix B.7 (Rounding)Recommended
IACS Recommendation No. 47 Shipbuilding and Repair Quality StandardClause Part A (New construction tolerances)Recommended

Shows what precision is physically achievable in steel construction.

Previous · 1.8Statistical methods for measurement and survey dataNext · 2.1Stress, strain and material constants (E, G, ν)

Version 2 · last updated 20 Nov 2026 · Editor

On this page

  • Significant figures
  • Rules of thumb for calculations
  • Rounding
  • Reporting precision used in this handbook
  • Units and symbols in reports (ISO 80000-1)
  • Presenting a calculation
  • Formulas
  • Tables
  • Examples
  • References

Applicable standards

  • ISO 80000-1:2022 Quantities and units — Part 1: General

    Clause 7 (Printing rules for symbols and numbers), Annex B (Rounding of numbers)

  • NIST SP 811 (2008)

    Ch. 7 and Appendix B.7 (Rounding)

  • IACS Recommendation No. 47 Shipbuilding and Repair Quality Standard

    Part A (New construction tolerances)

NavArch Handbook·About & disclaimerFeedbackBookmarks© Jadrija Maritime j.d.o.o.