On the accuracy of covariance matrix: Hessian versus Gauss-Newton methods in atmospheric remote sensing with infrared spectroscopy

Most retrieval schemes use a linear approximation of the radiative transfer function within each iteration as well as for error analysis. Like most standard methods, the improved Hessian method relies on a quadratic form of the cost function and linear approximation in the error analysis. Often, there is no robust criterion in determining step size that can be used to calculate covariance matrix by discrete perturbation of the cost function in the Hessian approach. The Hessian method improved recently, however, overcomes this problem by employing adaptive algorithm which uses small step sizes in steep directions and large step sizes in flat directions of the cost function. The results of retrievals of atmospheric trace gases from simulated limb emission spectra show that Gauss-Newton algorithm and the improved Hessian generally give nearly identical volume mixing ratios and error covariance matrices in the original state vector space. Due to interlevel correlations, however, the agreement in the uncertainities in the original state vector coordinate system is partly lost in a space in which the elements of state vector are independent after orthogonal coordinate transformation. The significant discrepancies between the estimated uncertainities by the two methods are found to be related with elements of state vector that are dominantly controlled by flattest eigenvector directions of the inverse covariance matrix. The improved Hessian method determines the uncertainities in those shallowest directions with better accuracy than Gauss-Newton approach. The performance of the Hessian method is also found to be better in resolving structures related to the shallowest eigenvector directions as revealed by better vertical resolutions in the retrieved profiles of the trace species.