According to The Monk series, the fortune abandons people because all the wealth and position in the whole world are not real, thus, it is just a pure illusion. It is stated in the series that, no man must stay keeping hold of his wealth and no one must trust its blind prosperity. Hope this helps!
