Research database

Project information
validation physical biogeochemical models
Project title
ECOAN WP3-OA8: Validation and comparison of coupled physical-biogeochemical models
Project leader
Phil Wallhead (NIVA) and Morten Skogen (IMR)

Cecilie Hansen (IMR)

Andre Staalstrøm (NIVA)


Ocean Acidification
Funding Source

Framsenter Flagship

Summary of Results

During 2015 we updated the validation data set, adding data from ICES1, updating the compilation from WOD2, and adding data from the "Atlantic Ocean" CARINA dataset3 as well as the the "Irminger Sea" and "Icelandic Sea" CARINA time series datasets4  (previously we considered only the "Arctic Mediterranean Sea" CARINA dataset5).  For convenience, these CARINA datasets were combined and parsed into a single CARINA-combined dataset, setting all negative nutrient concentrations to zero (except where -999 was used as a missing data flag; these were removed).  Analysis of the WOD total alkalinity data revealed some suspicious low values; after communicating with the data providers, we concluded that they had likely arisen from incorrect units conversion.  Also, comparing ICES vs. WOD data for nitrate and silicate revealed large discrepancies for the same profiles.  We concluded that the ICES and WOD datasets should for the moment only be used for hydrography, while the better quality-controlled CARINA datasets can be used for hydrography, nutrients, and carbonate system variables.  Attempts to parse the WOD/ICES hydrographic data were abandoned due to excessive computational effort, so we accept that there will be some duplicated (T, S) data in the combined data collection6.

CO2 system variables such as pH and aragonite saturation state Ω(ar) were calculated using CO2SYS.m7.  For model results, phosphate concentrations were estimated by multiple linear regression on (T, NO3, Si) (fitted to the CARINA data) and pressure was estimated using a quadratic regression function of depth.  We used dissociation constants from (8, 9) and total borate concentrations from (10).  Note that CO2SYS assumes that the calcium ion concentration is proportional to the salinity in calculating the calcite and aragonite saturation states11.

The updated data collection was used to make a simple point-to-point assessment of agreement between the SINMOD hindcast output and biogeochemical data in the study region (see Fig. 2) during years 1984-2009 and over depths 0–200 m (Fig. 1).  SINMOD shows a strong Pearson correlation with temperature (0.91) although the root-mean-square error remains substantial (1.5 degC); this is partly explained by the large temperature contrast between Atlantic and Arctic water and the likelihood of mislocating the position of the Atlantic-Arctic water front, leading to large model-data discrepancies.  The agreement with salinity data is not so good, in large part due to a consistent underestimation of salinity in fresher coastal waters (i.e. an exaggeration of freshening, see Fig. 1).  As expected, the agreement with nutrient and carbonate system data is generally poorer than for (T, S), but relative to other biogeochemical models the overall skill is actually rather impressive (see Fig. 1).  Hindcast skill could be improved using a linear bias correction, hence the blue linear regression lines in Fig. 1 rather than the red 1:1 lines.  However, this would not be appropriate for projections because it would generally amount to an extrapolation beyond the established validity of the linear regression.

Point-to-point skill assessments as in Figure 1 provide useful overall metrics to compare models and assess improvement, but they give relatively little information about where the model is going wrong and why.  We have used a kernel smoothing technique to assess the regional pattern of long-term bias for SINMOD in the study region (see Fig. 2).  Data and model output were averaged over the surface 0-50 m and horizontally smoothed onto the model grid, using an exponential kernel with bandwidth 40 km (Fig. 2, upper panels, same colour scale).  For the data this was done for each season and averaged over seasonal analyses to minimize seasonal sampling bias.  To estimate the bias field ( = model minus data long term averages) we smoothed the model-data residuals to minimize sampling bias due to temporal trends and changing spatial sampling patterns between years.  We also excluded from the analysis any regions where smoothing estimates would be based on less than 5 different years of data for any season (see Fig. 2, bottom right). 

For temperature, the bias field analysis revealed an interesting pattern of excessively warm surface water (positive bias in the SINMOD output) east of Iceland, along the Norwegian coast, and into the central Barents Sea (Fig. 2, bottom left).  This suggests that SINMOD may be transporting too much heat into the Barents Sea via the Norwegian Atlantic current.  There are also strong cold spots of negative bias (SINMOD too cold) off the west and north coast of Iceland and over the Lofoten Basin.  The salinity bias analysis (Fig. 3) reveals a saline (positive) bias east of Iceland and along the Norwegian coast, and fresh biases west/north of Iceland, in the central Barents, in the Skaggerak, and on the Russian coast in the east Barents Sea.  Part of these salinity biases likely reflect an excessive freshening in SINMOD close to rivers and regions of ice melt (see Fig. 1).  The remaining (T, S) biases may reflect faults in the circulation and mixing due to inadequate spatial resolution (20 km); the coarse may not be allowing the warm Atlantic water to "wrap around" the west coast of Iceland, and may be neglecting the contribution of mesoscale eddies to heat and salinity transport off the Norwegian shelf and into the Lofoten Basin.  Similar bias analyses were not possible for other variables due to insufficient data coverage.

We also tested the temporal trends and seasonality in the model output in comparison to the trends and seasonality observed in time series data from the Irminger Sea (IRM), Icelandic Sea (IS), and Ocean Weather Station Mike (OWSM) time series stations for all data within the period 1983–2015 (see Figs. 4, 5). Model and data time series were analysed by fitting a seasonal first-order autoregressive (AR1) model12, estimating trends as linear increases and seasonality as the maximum-minus-minimum seasonal effect.   SINMOD trends mostly agreed with the observed trends, at least in the surface 200 m where spin-up time was sufficient (Fig. 4).  However, the SINMOD does underestimate the warming trends at IRM and OWSM (Fig. 4a,m) and salinity increases at all stations (Fig. 4b,h,n), and overestimates the increasing DIC trend at IRM (Fig. 4c).  Seasonality in the 0–200 m layer is broadly well-reproduced by SINMOD (Fig. 5) with the notable exception of salinity at all stations (Fig. 5b,h,n).  Further analyses (not shown) for the Russian KOLA station in the southeastern Barents Sea suggested that SINMOD was underestimating the warming trend in the surface layer.

We conclude that SINMOD provides a good benchmark in terms of biogeochemical model skill, although there is clear room for improvement, notably with regard to modelled salinity.  More carbonate chemistry data are urgently needed to test the model in the eastern Barents Sea and Kara Sea.  Future work should collate and quality-control in situ biomass and productivity data (including ARCSS-PP) and remote sensing data (including ice cover), and use these to further validate and improve the ECOAN ocean biogeochemical models.


Figure 1.  Point-to-point skill assessment for all data sampled within the study region (see Fig. 2) and from 0–200 m depth and years 1984–2009.  SINMOD output was interpolated in 4D (x,y,z,t) to the data, and skill was computed as RMSE = root-mean-square error, BIAS = mean(model - data), CORR = Pearson correlation.  Red lines are 1:1, blue from linear regression.


Figure 2.  Temperature averaged over the surface 50 m and 25 years (1984-2009) from kernel smoothing (bandwidth 40 km) of the data (top left), of the SINMOD output (top right), and of the (model minus data) residuals to yield an estimated bias field (bottom left).  The residual means coverage, computed as the minimum number of years in any one season contributing to the smoothing bias estimates, is shown in the bottom right subplot.


Figure 3.  As in Fig. 2 but for salinity.


Figure 4.  Long-term trend skill assessment for the years 1983–2015 at three time series stations: Irminger Sea (IRM), Icelandic Sea (IS), and Ocean Weather Station M (OWSM).  Trends were estimated using a seasonal AR1 model12.  Black error bars show data trend ± 95% CIs, red envelopes show SINMOD trend ± 95% CIs, and red crosses show where residual (data-model) trend is significant (p<0.05).


Figure 5.  Seasonality skill assessment for the years 1983–2015 at three time series stations: Irminger Sea (IRM), Icelandic Sea (IS), and Ocean Weather Station M (OWSM).  Seasonality was calculated as the maximum – minimum seasonal effect from fitting a seasonal AR1 model12.  Black error bars show data seasonality ± 95% CIs, red envelopes show model seasonality ± 95% CIs, and red crosses show where residual (data-model) seasonality is significant (p<0.05).



1ICES Dataset on Ocean Hydrography. The International Council for the Exploration of the Sea, Copenhagen. 2013.

2Boyer, T.P. et al. 2013, Sydney Levitus, Ed.; Alexey Mishonov, Technical Ed.; NOAA Atlas NESDIS 72, 209 pp.

3Tanhua, T. et al. 2010. Earth Syst. Sci. Data, 2, 17–34.

4Olafsson et al., 2010. Earth Syst. Sci. Data, 2, 99–104.

5Jutterstrom, S. et al. 2010. Earth Syst. Sci. Data, 2, 71–78.

6Ottersen, 2010. ICES Journal of Marine Science 67: 1525–1537.

7van Heuven, S., Pierrot, D., Rae, J. W. B. et al. 2011. In ORNL/CDIAC-105b. Carbon Dioxide Information Analysis Center, Oak Ridge  National Laboratory, U.S., Department of Energy, Oak Ridge, Tennessee

8Millero, F.J. 2010. Marine and Freshwater Research 61(2) 139-142.

9Dickson, A.G., 1990.  J. Chemical Thermodynamics, 22:113-127.

10Uppstrøm, L.R., 1974. Deep Sea Research I,  21(2) 161-162.

11Riley, J. P. and Tongudai, M., Chemical Geology 2:263-269, 1967.

12Wallhead et al., 2014. Global Biogeochem. Cycles, 28, doi:10.1002/2013GB004797.

For the Management

The project is progressing as planned and is on track to meet all milestones and deliverables.

Published Results/Planned Publications

SINMOD validation results are planned to be published together with the projections (see OA9) during 2016.

Communicated Results

SINMOD validation results were presented in a poster at the FRAM Science Days conference

Interdisciplinary Cooperation

This project has demanded the close collaboration of modellers with observationalists and database managers.

Could results from the project be subject for any commercial utilization

The project is on track and already producing important insights into sources of error and bias in ocean biogeochemical model predictions.