Freelancers have this advantage over companies. You can choose who you want to work with on a personal level. Old school business types would tell you its not personal just business, that you should be able to work with anyone.
But when we work from home and get to know our clients first hand and see their software grow up with them, it becomes personal.
Choose your clients well. A few will stick with you for a long time. That one annoying client that pays more but gives you too much stress will take your attention away from your other, probably better clients.
Oh sometimes you get great clients that pay well too! 
Update: similar articles
Well said Dilantha
ok i think saw this on some other site?? was it written by you or did u get it from somewhere??
Joker2 if you Google ‘fire clients’ you will find some similar articles.. I saw them too. This particular post I wrote from my past experience as a freelancer.
I’m updating the post with a few of those articles as well.