Lots of background material (eg, on linear algebra) can be found on the Web Textbook here. Login and password information are provided in bspace.
Introduction to Linear Algebra, by Strang. A reference on linear algebra, with emphasis on concepts.
Linear Algebra and Applications, by D. Lay. I haven't fully checked this one, but seems to contain interesting examples.
Matrix Computations by Golub and Van Loan. The reference on numerical linear algebra.
Linear Algebra Close to Earth, by J. Khoury. Nice web-based material, with lots of illustrative examples.
Linear Algebra, by J. Hefferon introduces the concepts and examples.
Course Reader for EE 263 contains S. Boyd's lecture notes for a graduate-level class in linear algebra.
A Tutorial on Principal Components Analysis, by L. Smith.
Convex Optimization, by Boyd and Vandenberghe. A reference on convex optimization, at graduate level.
Wikimization, a wiki devoted to optimization.
Optimization journals.