INDIRECT is mostly used when you need to build the reference, ie, the row number may be a variable from some other calculation, where as a direct link is when you want to always pull from the same cell.
Technically, another difference is that direct links look at the same cell, but INDIRECT looks at same location. Meaning, if you write this formula:
=B2
And then insert a row above row 2, the formula will automatically change to
=B3
because it's following the same cell. If you had written
=INDIRECT("B2")
and inserted a row, the formula does not change because you are always looking at the location "B2", not a particular cell.