D.H. Alexander, J. Novembre, and K. Lange. Fast model-based estimation of ancestry in unrelated individuals. Genome Research, 19:1655–1664, 2009.and we encourage you to read the preprint pdf. If you use ADMIXTURE, please cite this publication in your paper.
The quasi-Newton convergence acceleration method is described in more detail in
H. Zhou, D. H. Alexander, and K. Lange. A quasi-Newton method for accelerating the convergence of iterative optimization algorithms. Statistics and Computing, 2009.for which a preprint is also available.