Back to section

NavArch Handbook

Chapter 1 / Section 1.8

Statistical methods for measurement and survey data

Why

Thickness measurements (UTM) in surveys, inclining-test readings, speed-trial runs, weight audits and tank soundings are all repeated measurements with scatter. The engineer must summarise them, decide which readings are outliers, estimate the uncertainty of derived quantities, and fit straight lines.

Mean, standard deviation, standard error

xˉ=1n∑xi,s=∑(xi−xˉ)2n−1,sxˉ=sn\bar x = \frac{1}{n}\sum x_i, \qquad s = \sqrt{\frac{\sum (x_i - \bar x)^2}{n - 1}}, \qquad s_{\bar x} = \frac{s}{\sqrt n}xˉ=n1​∑xi​,s=n−1∑(xi​−xˉ)2​​,sxˉ​=n​s​

sss (sample standard deviation, divisor n−1n-1n−1) describes the scatter of single readings; sxˉs_{\bar x}sxˉ​ the uncertainty of the mean. For a normal distribution about 68 % of readings lie within ±s\pm s±s, 95 % within ±2s\pm 2s±2s, 99.7 % within ±3s\pm 3s±3s.

Outliers

A reading more than 3s3s3s from the mean (or failing Chauvenet's criterion: probability of occurrence × n < 0.5) is suspect: re-measure or document the reason for discarding it. Never discard inclining-test pendulum readings silently; the IS Code requires the plot of all readings.

Linear regression and R2R^2R2

Slope and intercept as in Section 1.6. Coefficient of determination:

R2=Sxy2Sxx Syy,Sxx=∑(xi−xˉ)2,  Syy=∑(yi−yˉ)2,  Sxy=∑(xi−xˉ)(yi−yˉ)R^2 = \frac{S_{xy}^2}{S_{xx}\,S_{yy}}, \quad S_{xx} = \sum (x_i - \bar x)^2, \; S_{yy} = \sum (y_i - \bar y)^2, \; S_{xy} = \sum (x_i - \bar x)(y_i - \bar y)R2=Sxx​Syy​Sxy2​​,Sxx​=∑(xi​−xˉ)2,Syy​=∑(yi​−yˉ​)2,Sxy​=∑(xi​−xˉ)(yi​−yˉ​)

Standard error of the slope: sb=Syy−b Sxy(n−2) Sxxs_b = \sqrt{\dfrac{S_{yy} - b\,S_{xy}}{(n-2)\,S_{xx}}}sb​=(n−2)Sxx​Syy​−bSxy​​​. For the inclining test, the slope is Δ GM\Delta\,GMΔGM and sb/Δs_b/\Deltasb​/Δ is the uncertainty of GM.

Propagation of uncertainty

For a result y=f(x1,x2,… )y = f(x_1, x_2, \dots)y=f(x1​,x2​,…) with independent uncertainties uiu_iui​:

uy=∑i(∂f∂xi)2ui2u_y = \sqrt{\sum_i \left(\frac{\partial f}{\partial x_i}\right)^2 u_i^2}uy​=i∑​(∂xi​∂f​)2ui2​​

For products and quotients y=x1ax2b⋯y = x_1^{a} x_2^{b}\cdotsy=x1a​x2b​⋯ the relative uncertainties add in quadrature: (uy/y)2=a2(u1/x1)2+b2(u2/x2)2+…(u_y/y)^2 = a^2 (u_1/x_1)^2 + b^2 (u_2/x_2)^2 + \dots(uy​/y)2=a2(u1​/x1​)2+b2(u2​/x2​)2+…. For sums and differences the absolute uncertainties add in quadrature. The difference of two nearly equal quantities (GM=KM−KGGM = KM - KGGM=KM−KG) inherits the absolute uncertainties of both — which is why a ±3 cm uncertainty in KG is large when GM is 0.70 m.

Corrosion statistics (UTM surveys)

Report for each plate group: number of readings, mean thickness, minimum, standard deviation, and the percentage below the renewal thickness. Class rules set the renewal criterion on the minimum reading, not the mean (DNV Rules Pt.7 — survey requirements; IACS UR Z10 series) — statistics describe the condition, the rule decides on the extreme.

Formulas

01Sample mean
xˉ=1n∑i=1nxi\bar x = \frac{1}{n}\sum_{i=1}^{n} x_ixˉ=n1​i=1∑n​xi​
SymbolDescriptionUnit
xix_ixi​Reading ias measured
nnnNumber of readings-

Result unit as measured

02Sample standard deviation
s=1n−1∑i=1n(xi−xˉ)2s = \sqrt{\frac{1}{n-1}\sum_{i=1}^{n}(x_i - \bar x)^2}s=n−11​i=1∑n​(xi​−xˉ)2​

Result unit as measured

03Standard error of the mean
sxˉ=sns_{\bar x} = \frac{s}{\sqrt n}sxˉ​=n​s​

Result unit as measured

04Coefficient of determination
R2=Sxy2Sxx SyyR^2 = \frac{S_{xy}^2}{S_{xx}\,S_{yy}}R2=Sxx​Syy​Sxy2​​
SymbolDescriptionUnit
Sxx,Syy,SxyS_{xx}, S_{yy}, S_{xy}Sxx​,Syy​,Sxy​Corrected sums of squares and products-

Result unit -

05Standard error of the regression slope
sb=Syy−b Sxy(n−2) Sxxs_b = \sqrt{\frac{S_{yy} - b\,S_{xy}}{(n-2)\,S_{xx}}}sb​=(n−2)Sxx​Syy​−bSxy​​​
SymbolDescriptionUnit
bbbFitted slope-

Result unit unit of b

06Propagation of independent uncertainties
uy=∑i(∂f∂xi)2ui2u_y = \sqrt{\sum_i \left(\frac{\partial f}{\partial x_i}\right)^2 u_i^2}uy​=i∑​(∂xi​∂f​)2ui2​​
SymbolDescriptionUnit
uiu_iui​Standard uncertainty of input ias input
uyu_yuy​Standard uncertainty of the resultas result

Result unit as result

07Relative uncertainty of a product of powers
(uyy)2=∑iai2(uixi)2for y=∏ixiai\left(\frac{u_y}{y}\right)^2 = \sum_i a_i^2 \left(\frac{u_i}{x_i}\right)^2 \quad \text{for } y = \prod_i x_i^{a_i}(yuy​​)2=i∑​ai2​(xi​ui​​)2for y=i∏​xiai​​

Result unit -

Tables

Normal distribution: coverage of ± k·s

kFraction of readings within ± k sFraction outside
168.27 %31.7 %
1.64590.0 %10.0 %
1.96095.0 %5.0 %
295.45 %4.55 %
2.57699.0 %1.0 %
399.73 %0.27 %
Source: Standard normal distribution; ISO/IEC Guide 98-3 (GUM) Table G.1.

Typical measurement uncertainties in ship work (indicative)

MeasurementTypical standard uncertaintyComment
Draught reading at marks (calm water)± 5–10 mmLarger with waves; read both sides
Water density by hydrometer± 0.001–0.002 t/m³Temperature correction
Inclining pendulum deflection (3 m pendulum)± 1 mm→ tan φ ± 0.0003
Inclining weight mass (certified)± 0.1–0.5 %Weighbridge certificate
UTM plate thickness± 0.1 mmCalibrated gauge, clean surface
Lightship displacement from inclining± 0.5–1 %Draughts + density + deductions
Lightship KG from inclining± 2–5 cmDominated by KM accuracy and free surfaces
Speed over ground (GNSS, trial run)± 0.05 knAverage of double runs
Source: Indicative values from practice; IS Code 2008 Part B Annex 1 (inclining test accuracy requirements).

Examples

#01

Regression analysis of inclining-test readings

Worked example

Problem

Eight weight shifts on the example vessel (Δ=6314\Delta = 6314Δ=6314 t) gave the heeling moments M [t·m] and mean pendulum readings tan φ: (0, 0.0005), (60, 0.0128), (120, 0.0252), (60, 0.0126), (0, −0.0003), (−60, −0.0121), (−120, −0.0249), (−60, −0.0124). Fit M = a + b tan φ, report R², and derive GM.

Solution

n=8n = 8n=8; ∑x=0.0014\sum x = 0.0014∑x=0.0014, xˉ=0.000175\bar x = 0.000175xˉ=0.000175; ∑y=0\sum y = 0∑y=0, yˉ=0\bar y = 0yˉ​=0.

∑xiyi=0+0.768+3.024+0.756+0+0.726+2.988+0.744=9.006\sum x_i y_i = 0 + 0.768 + 3.024 + 0.756 + 0 + 0.726 + 2.988 + 0.744 = 9.006∑xi​yi​=0+0.768+3.024+0.756+0+0.726+2.988+0.744=9.006; ∑xi2=0.0018782\sum x_i^2 = 0.0018782∑xi2​=0.0018782.

Sxy=9.006−8×0.000175×0=9.006S_{xy} = 9.006 - 8 \times 0.000175 \times 0 = 9.006Sxy​=9.006−8×0.000175×0=9.006; Sxx=0.0018782−8×0.0001752=0.0018779S_{xx} = 0.0018782 - 8 \times 0.000175^2 = 0.0018779Sxx​=0.0018782−8×0.0001752=0.0018779; Syy=∑yi2=4×3600+2×14400=43 200S_{yy} = \sum y_i^2 = 4 \times 3600 + 2 \times 14400 = 43\,200Syy​=∑yi2​=4×3600+2×14400=43200.

b=9.0060.0018779=4796 t⋅m,a=0−4796×0.000175=−0.84 t⋅m≈0b = \frac{9.006}{0.0018779} = 4796\ \text{t·m}, \qquad a = 0 - 4796 \times 0.000175 = -0.84\ \text{t·m} \approx 0b=0.00187799.006​=4796 t⋅m,a=0−4796×0.000175=−0.84 t⋅m≈0 R2=9.00620.0018779×43 200=81.1181.13=0.9998R^2 = \frac{9.006^2}{0.0018779 \times 43\,200} = \frac{81.11}{81.13} = 0.9998R2=0.0018779×432009.0062​=81.1381.11​=0.9998 GM=bΔ=47966314=0.760 mGM = \frac{b}{\Delta} = \frac{4796}{6314} = 0.760\ \text{m}GM=Δb​=63144796​=0.760 m

The intercept is negligible (no initial list or restraint), R² is excellent (no free-surface non-linearity), and GM agrees with Example 5.2. Standard error of the slope: sb=(43 200−4796×9.006)/(6×0.0018779)=(43 200−43 193)/0.011267=621=25s_b = \sqrt{(43\,200 - 4796 \times 9.006)/(6 \times 0.0018779)} = \sqrt{(43\,200 - 43\,193)/0.011267} = \sqrt{621} = 25sb​=(43200−4796×9.006)/(6×0.0018779)​=(43200−43193)/0.011267​=621​=25 t·m, i.e. GM ± 0.004 m from the regression alone (the real uncertainty is dominated by KM and the displacement).

#02

Uncertainty of the displacement from measured dimensions

Worked example

Problem

Δ=ρ L B T CB\Delta = \rho\,L\,B\,T\,C_BΔ=ρLBTCB​ with relative uncertainties: ρ 0.1 %, L 0.1 %, B 0.2 %, T 1.0 % (draught ± 5 cm on 5.5 m), CBC_BCB​ 1.0 % (from the hydrostatics at that draught). Find the relative uncertainty of Δ and the absolute value for Δ = 6314 t.

Solution

All exponents are 1:

uΔΔ=0.12+0.12+0.22+1.02+1.02 %=2.06 %=1.44 %\frac{u_\Delta}{\Delta} = \sqrt{0.1^2 + 0.1^2 + 0.2^2 + 1.0^2 + 1.0^2}\ \% = \sqrt{2.06}\ \% = 1.44\ \%ΔuΔ​​=0.12+0.12+0.22+1.02+1.02​ %=2.06​ %=1.44 %

uΔ=0.0144×6314=91u_\Delta = 0.0144 \times 6314 = 91uΔ​=0.0144×6314=91 t. The draught reading and the hull-form coefficient dominate; improving the length measurement would change nothing. This is the order of accuracy of a draught survey, and the reason deadweight surveys for cargo quantity need careful draught and density readings.

#03

UTM thickness readings on a deck plate

Worked example

Problem

Six readings on a 10 mm deck plate [mm]: 9.8, 9.9, 10.1, 9.7, 10.0, 9.9. Report mean, standard deviation and minimum; compare with a renewal thickness of 8.0 mm (20 % diminution).

Solution

xˉ=59.4/6=9.90\bar x = 59.4/6 = 9.90xˉ=59.4/6=9.90 mm. Deviations: −0.1, 0, +0.2, −0.2, +0.1, 0; ∑d2=0.10\sum d^2 = 0.10∑d2=0.10; s=0.10/5=0.141s = \sqrt{0.10/5} = 0.141s=0.10/5​=0.141 mm. Minimum 9.7 mm. Mean diminution 1.0 %, minimum reading 3.0 % below nominal — far above the 8.0 mm renewal limit. Report: 'n = 6, mean 9.90 mm, s = 0.14 mm, min 9.7 mm, 0 readings below renewal thickness'.

References

ISO/IEC Guide 98-3:2008 (GUM) Uncertainty of measurement — Guide to the expression of uncertainty in measurementClause Ch. 5 (Determining combined standard uncertainty)Recommended
IMO IS Code 2008Clause Part B, Annex 1 (Inclining test — plotting and evaluation of readings)Recommended
IACS UR Z10.1–Z10.5 (Hull surveys) and DNV Rules Pt.7 Ch.1Clause Thickness measurement requirements and renewal criteriaMandatory

Renewal is decided on measured minimum thickness vs rule allowable diminution; confirm the edition in force.

Taylor, J.R., An Introduction to Error Analysis, 2nd ed. (1997)Clause Ch. 3 (Propagation of uncertainties), Ch. 8 (Least-squares fitting)Textbook
NavArch Handbook · Chapter 1 · printed on —