Participants
Xumeng Cao
Preliminary Results on Non-Bernoulli Distribution of Perturbations for Simultaneous Perturbation Stochastic Approximation
Simultaneous perturbation stochastic approximation (SPSA) has proven to be an efficient algorithm for recursive optimization. SPSA uses a centered difference approximation to the gradient based on only two function evaluations regardless of the dimension of the problem. Typically, the Bernoulli ±1 distribution is used for perturbation vectors and theory has been established to prove the asymptotic optimality of this distribution. However, efficiency of the Bernoulli distribution may not be guaranteed for small-samples. In this paper, we investigate the performance of segmented uniform distribution for perturbation vectors. For small-samples, we show that the Bernoulli distribution may not be the best for a certain choice of parameters.
Demonstration of Enhanced Monte Carlo Computation of the Fisher Information for Complex Problems
The Fisher information matrix summarizes the amount of information in a set of data relative to the quantities of interest. There are many applications of the information matrix in statistical modeling, system identification and parameter estimation. This short paper reviews a feedback-based method and an independent perturbation approach for computing the information matrix for complex problems, where a closed form of the information matrix is not achievable. We show through numerical examples how these methods improve the accuracy of the estimate of the information matrix compared to the basic resampling-based approach. Some relevant theory is summarized.