“Know that the greatest of losses is for you to be preoccupied with one who will bring you nothing but a loss in your time with Allah – the Mighty and Majestic – and being cut off from Him, a wasting your time with the person, a weakening of your energy, and the dispersing of your resolve. So, if you are tested with this – and you must be tested with this – deal with him according to how Allah would wish, and be patient with him as much as possible. Get closer to Allah and His Pleasure by way of this person, and make your getting together with him something to benefit from, not something to incur a loss from. Be with him as if you are a man who is on a road who was stopped by another man, who then asks you to take him on your journey. Make sure that you are the one who gives him a ride, and that he is not the one giving you the ride. If he refuses, and there is nothing to gain from travelling with him, do not stop for him, bid him farewell, and do not even turn back to look at him, as he is a highway robber, regardless of who he really is.
So, save your heart, be wary of how you spend your days and nights, and do not let the Sun set before you arrive at your destination.
Reference: Ibn al Qayyim ‘al-Wabil as-Sayyib’; p. 45