Is Simpler Better? A Visualization-Based Exploration of How Parametric Screening Influences Problem Difficulty and Equifinality in Multiobjective Calibration
Open Access
- Author:
- Urban, Rachel Lorah
- Area of Honors:
- Civil Engineering
- Degree:
- Bachelor of Science
- Document Type:
- Thesis
- Thesis Supervisors:
- Patrick Reed, Thesis Supervisor
Patrick Reed, Thesis Supervisor
Thorsten Wagener, Faculty Reader
Patrick Reed, Thesis Honors Advisor - Keywords:
- multiobjective optimization
evolutionary algorithms
model calibration - Abstract:
- This study uses interactive visualization to investigate the common assumption that parametric screening using sensitivity analysis simplifies hydrologic calibration. Put simply, do we make calibration easier by eliminating model parameters from the optimization problem? Traditional approaches for parametric screening focus on model evaluation metrics that seek to minimize statistical error. We demonstrate in this study that additional hydrology relevant metrics (e.g., water balance) are essential to properly screening parameters and producing search problems that do not degenerate into random walks (a severe case of equifinality). This work highlights that we should move beyond a focus on optimality in a traditional error sense and instead focus on enhancing our evaluative metrics and formulations to include hydrology relevant information. Building on the prior work by van Werkhoven et al. 2009, this study utilizes parameter screening results based on Sobol sensitivity analysis to reduce the size of hydrologic calibration problems for the Sacramento Soil Moisture Accounting model (SAC SMA). Our study was conducted across four hydroclimatically diverse watersheds, and we distinguish various sets of parametric screenings, including a full parameter search, as well as parameter screenings based on 5%, 10%, 20%, and 30% Sobol sensitivity levels. For each Sobol sensitivity level there are two subcases: (1) parameters are screened based on statistical metrics alone, and (2) parameters are screened based on statistical and hydrological metrics. The reduced parameter sets were searched using a multiobjective evolutionary algorithm to determine the tradeoff surfaces of optimal parameter settings. Our results contribute detailed interactive visualizations of the 4-objective tradeoff surfaces for all of the parametric screening cases evaluated. For almost all of problem formulations that result from parametric screening, the combined use of statistical and hydrological metrics for screening outperformed the use of solely statistical measures in nondomination sense (i.e., they were better in all metrics). Our visual analysis shows that we need to move beyond traditional statistical calibration methodologies and more rigorously investigate our problem formulations. Future calibration frameworks need to move our focus from finding optimal solutions to instead defining more informative problem formulations that incorporate hydrologic knowledge.