This work concerns results on conditions guaranteeing that certain banded $M$-matrices have banded inverses. As a first goal, a graph theoretic characterization for an off-diagonal entry of the inverse of an $M$-matrix to be positive, is presented. This result, in turn, is used in providing alternative graph theoretic proofs of the following: (1) a characterization for a tridiagonal $M$-matrix to have a tridiagonal inverse. (2) a necessary condition for an $M$-matrix to have a pentadiagonal inverse. The results are illustrated by several numerical examples.