We identify all non-splitting bi-unitary perfect polynomials over the field $\mathbb{F}_4$, which admit at most four irreducible divisors. There is an infinite number of such divisors.