In this paper, we calculate the annihilating polynomial of the colored Jones polynomial for the Hopf link and the Whitehead link and consider the link version of the AJ conjecture. We also give another annihilating polynomial of the colored Jones polynomial for those links.