8
$\begingroup$

I am a computer science research student working in application of Machine Learning to solve Computer Vision problems.

Since, lot of linear algebra(eigenvalues, SVD etc.) comes up when reading Machine Learning/Vision literature, I decided to take a linear algebra course this semester.

Much to my surprise, the course didn't look at all like Gilbert Strang's Applied Linear algebra(on OCW) I had started taking earlier. The course textbook is Linear Algebra by Hoffman and Kunze. We started with concepts of Abstract algebra like groups, fields, rings, isomorphism, quotient groups etc. And then moved on to study "theoretical" linear algebra over finite fields, where we cover proofs for important theorms/lemmas in the following topics:

Vector spaces, linear span, linear independence, existence of basis. Linear transformations. Solutions of linear equations, row reduced echelon form, complete echelon form,rank. Minimal polynomial of a linear transformation. Jordan canonical form. Determinants. Characteristic polynomial, eigenvalues and eigenvectors. Inner product space. Gram Schmidt orthogonalization. Unitary and Hermitian transformations. Diagonalization of Hermitian transformations.

I wanted to understand if there is any significance/application of understanding these proofs in machine learning/computer vision research or should I be better off focusing on the applied Linear Algebra?

  • 0
    I would recommend sticking to Applied form. Engineering in most form doesn't require the kind of rigor provided by H&K or for that matter, any mathematical subject. You need rigor only if you want to go about proving things or reading papers.2012-02-24
  • 0
    @Nunoxic I am sorry to say that not everybody who does Math rigorously do it for _going about proving things or reading papers_. It is mostly fun and widens the scope for creative thoughts.2012-02-24
  • 0
    I'm maybe wrong, but I think [Theoretical Computer Science](http://cstheory.stackexchange.com/) would be a better place for this question, isn't it?2012-02-24
  • 0
    @KannappanSampath, I agree, it does influence the way one thinks but considering the opportunity cost of learning H&K, I doubt its worth taking a course. Maybe, it is worth looking into later (once you already know LA). Plus, something I have realized is, for engineers, the first course should ideally **not** be a rigorous course. It should be applied and later, if you are interested, pick up a concise rigorous text. But that's just my opinion.2012-02-24
  • 0
    I think that's it's always good to understand the fundamentals of linear algebra, even if you will mostly focus on applications, because it enables you to choose the most appropriate algorithm for the job. An example from finance: to compute the volatility of a stock portfolio $w$ from a returns matrix $X$, you might calculate $\sigma=\|Xw\|$. But if $X$ is large, a more numerically stable approach (and also a faster one, if you will have to calculate $\sigma$ multiple times) is to calculate the QR decomposition $X=QR$ and then compute $\sigma=\|Rw\|$.2012-02-24
  • 0
    @MDCCXXIX that is exactly the kind of thing one would expect to learn in an applied linear algebra course (QR decomposition), but I agree with your first sentence. I don't know Strang's course, but I would be surprised if it didn't build a solid foundation, even if it doesn't delve into abstract groups and fields.2012-02-24
  • 6
    @stressed_geek: If you are researching Machine Learning and Computer Vision, presumably you are enrolled in some program and have several teachers and at least one adviser? "What kind of linear algebra -- e.g., which textbook -- would be most useful to me for my work?" is an eminently reasonable question to ask any of these people. And you should ask them: they will know much better than any of us, because they will know you and what you are working on much better than we do.2012-02-24
  • 0
    @PascalQyy The scientific computing stack exchange would be more appropriate, http://scicomp.stackexchange.com/ . The theoretical computer science SE is interested in more abstract things like theory of computability, etc.2012-05-29
  • 0
    Frankly, there is no direct application *per se.* But you'll become better in "applied" and "numerical" linear algebra if you understand the theory behind vector spaces.2012-07-13
  • 0
    Strang's books are great for applied math and machine learning people. However, there is some value in understanding the abstract vector space viewpoint, because many spaces we care about (in engineering applications) are not subspaces of $\mathbb{R}^n$. For example, we care about spaces of matrices, and spaces of symmetric matrices. You can map these spaces to subspaces of $\mathbb{R}^n$, but it's not elegant and (in my opinion) not as clear.2012-11-09

3 Answers 3