The essential soul of the Beinoni — his very essence is really his animal soul. It is that that he associates with. On the other hand, each morning, when we awaken, we thank God for bestowing within us a Godly soul.
With the Tzaddikim (the righteous), it is not this way. Their very essence is the Godly soul and the part who they associate with being.
The Alter Rebbe quotes a Midrash teaching us that Hillel the Elder would speak to his students and tell them that he would be going to do a favour for some lowly someone. Here, he was referring to his animal soul, that he was compelled to feed, just so that it could stay alive (so that of course he would be able to live, study Torah and perform Mitzvot in this physical world.)
It is because of this animal soul that the Beinoni is so far from God. Even when praying to God, it is not a true love. It is true for him of course. But when his prayers are over, he returns to his usual self, and the love which he once had (while praying) seems to have disappeared. Unlike the Tzaddik, whose love for God is absolute truth, and which stays with the Tzaddik constantly, whether engaged in prayer or not.