Output
Output of an OSTRICH analysis
Contents
1 Introduction
Upon completion, OSTRICH will have generated various output files. The name of each file will contain the id of the corresponding processor N, where N=0 is the supervisor/master processor responsible for gathering and reporting the majority of the output. Output files generated by subordinate processors are generally only useful for troubleshooting.
- OstOutputN.txt: The main output file, it contains an optimization (or regression) record along with statistical output (if applicable).
- OstErrorsN.txt: Any errors or warnings encountered by processor N are stored in this file.
- OstModelN.txt: A sequential record of every model run evaluated by processor N is stored in this file.
- OstExeOut.txt: The standard output and standard error of Model runs are redirected to this file. For a given processor, only the output from the most recent evaluation is retained.
- OstStatusN.txt: This file is periodically updated with the current progress (i.e. percent complete and number of model evaluations) of the selected search algorithm.
- OstGcopOut: This file keeps track of the cost and constraint information for each model evaluation when the General-purpose Constrained Optimization Platform (GCOP) is used.
- OstNonDomSolutions: When a multi-objective search is performed, this file is periodically updated with the current list of non-dominated solutions that have been discovered by the selected search algorithm.
The following sections describe selected OSTRICH output files in more detail.
2 OstOutput - The Main Output File
The main output file always contains the following elements (i) a GNU Public License disclaimer; (ii) a summary of the basic configuration variables; (iii) an OSTRICH run record detailing each iteration of the optimization algorithm; and (iv) the resulting optimal parameter set(s) and objective function value. The run record contains the parameter and objective function values at each iteration along with an algorithm-dependent value that indicates progress of the algorithm toward convergence. For most algorithms, the parameter and objective function values for the current best solution are reported.
3 OstOutput - Statistical Output
Various statistical measures may be reported if calibration using the WSSE objective function is being performed. See the Overview for a summary of the supported measures. The statistics that are reported will depend on whether or not they were selected in the input file. The following sections describe the output of these statistics.
3.1 Observation Residuals
Observations residuals are reported automatically at the end of every WSSE calibration. Also included in the observation residual output is the correlation between measured and simulated observations (Ry).
3.2 Error Variance and Standard Error of the Regression
Including the StdDev option in the MathAndStats group will cause the error variance (s2) and standard error of the regression (s) to be reported.
3.3 Parameter Variance-Covariance and Correlation
Including StdErr in the MathAndStats group will cause OSTRICH to output the parameter variance-covariance matrix along with the standard error of each parameter. Furthermore, including CorrCoeff in the MathAndStats group will cause OSTRICH to output the parameter correlation matrix.
3.4 Confidence Intervals
Including Confidence in the MathAndStats groups triggers the reporting of linear confidence intervals (CI) for each parameter. OSTRICH will use the value of the CI_PCT variable as the corresponding confidence level (e.g. CI_PCT = 0.95 selects a 95% confidence interval).
3.5 Model Linearity
Including either Beale or Linssen in the MathAndStats group will trigger reporting of the corresponding non-linearity measures and linearity thresholds.
3.6 Normality of Residuals
Inclusion of the NormPlot variable will cause OSTRICH to report a list of normalized residuals and the corresponding normal probability correlation coefficient (R2N).
3.7 Influential Observations
When either the CooksD or DFBETAS variable (or both) is set, OSTRICH will generate and output the corresponding measures of observation influence, along with an assessment of which observations are influential, based on influence thresholds suggested in the literature.
3.8 Parameter Sensitivities
Including the Sensitivity variable in the MathAndStats group causes OSTRICH to report parameter sensitivity measures. These are measures of local parameter sensitivity and are centered on the optimal parameter set.
3.9 Matrices
OSTRICH can be configured to output matrices used for various statistical calculations by including the Matrices variable in the MathAndStats group.
4 OstError – OSTRICH Error and Warning Messages
During execution, OSTRICH logs errors and warnings to the file ostErrorsN.txt, where N corresponds to the processor number. Some errors are severe and will cause OSTRICH to abort while others serve as warnings that the results may not be optimal even though OSTRICH is able to proceed. Error codes and brief descriptions are given in Table 1.
Table 1: OSTRICH Error Codes
| Error Message | Description |
|---|---|
| NO ERROR | No errors were reported. |
| BAD ARGUMENTS | A variable in the ostIn.txt file was assigned an invalid value. |
| FILE I/O ERROR | A file could not be opened for reading or writing. |
| MODEL EXECUTION ERROR | The model executable reported an error. |
| ARRAY OUT OF BOUNDS | An array index got assigned a value outside the bounds of the array. |
| PARAMETER MISMATCH | The search engine lost track of the number of parameters. |
| SINGULAR MATRIX | Matrix inversion failed because of a singular matrix. |
| GRID SIZE IS TOO LARGE | The exhaustive search grid is too large. |
| INITIAL SA TEMPERATURE | There was a problem with the final simulated annealing temperature. |
| PARAMETER BOUNDS | The upper or lower bound of a parameter value was exceeded. |
| COULDN'T BOUND MINIMUM | A line search failed to bound the minimum. |
| UNKNOWN BOUND CONDITION | A line search lost track of its state. |
| COULDN'T PARSE INPUT | Failed to parse a line of input in the ostIn.txt file. |
| MALLOC/NEW FAILED | Dynamic memory allocation failed. |
| JACOBIAN ERROR | Insensitivity of parameters or simulated observations while computing the Jacobian matrix. |
| USER ABORT | User requested the program abort via OstQuit.txt file. |
| BINARY CODED GA | The range of a parameter can’t be encoded into a 32-bit string. |
| OBSERVATION WEIGHTS | An observation weight less than or equal to zero was detected. |
| INSENSITIVE PARAMETER | A parameter had no influence on any simulated observation. |
| INSENSITIVE OBSERVATION | A simulated observation was insensitive to changes in parameters. |
| SUPERMUSE | A SuperMUSE operation failed or timed out. |
| OVERFLOW (DIV-BY-ZERO?) | A computed value is larger than can be represented by the system. |
| NULL POINTER | A subroutine returned NULL unexpectedly. |
| ALGORITHM STALLED | The selected algorithm appears to have stalled. |
| FILE CLEANUP | This is message alerts users to the deletion of temporary files. |
| NON-UNIQUE PARAMETER NAME | A parameter or tied parameter was assigned a non-unique name or a name that is a sub-string of the name of another parameter. |
| FIXED FORMAT PARAMETERS | A fixed format specification contains bad values. |
| DEGREES OF FREEDOM | A statistic could not be calculated due to too few degrees of freedom. |
5 OstExeOut – Redirected Model Output
OSTRICH redirects the standard error and standard output of each model run to a file named ’OstExeOut.txt’. The output of each new model run will overwrite the output of the previous model run so that OstExeOut.txt always contains the standard error and standard output of the most recent model run.
6 OstModel – Model Run Record
OSTRICH maintains a file named ’OstModelN.txt’, where N is the processor number, containing the parameter set and objective function of each model run. This list of model runs is numbered in increasing sequential order, with the highest value corresponding to the most recent model run.