Convergence and Statistical Performance of NLS and SNLS-VP under Non-Normal Errors and Challenging Starting Values

Abstract

 The study examines the convergence and performance of the standard Nonlinear Least Squares (NLS) and the Separable Nonlinear Least Squares with Variable Projection Method (SNLS-VP) under non-normal errors and poor starting values. A Monte Carlo study with 500 replications per simulation scenario was carried out with three levels of starting-value difficulty (easy, medium, and hard), three error distributions (Normal, Student's t with 3 degrees of freedom, and Log-normal), and three sample sizes (n = 50, 100, and 200). We compared the two methods in terms of Bias, Root Mean Squared Error (RMSE), Average Number of Iterations (ANI), and Convergence Rate (CR). Results show that NLS generally yields smaller Bias and RMSE, especially for small sample sizes and under Normal and t3 errors. In contrast, SNLS-VP shows more stable convergence in the considered scenarios, with a 100% convergence rate across all simulation setups. SNLS-VP is also more robust to difficult starting values, whereas NLS is generally faster when it converges but requires better starting values. The differences in statistical performance between the two procedures diminish as the sample size increases, especially under Log-normal errors. In sum, the results suggest that the relative performance of NLS and SNLS-VP depends on sample size, error distribution, and starting-value conditions, indicating a balance between statistical accuracy and computational convergence