Irrational or not, still upset

Mornings are mine, it's the only time I get. Yet my wife and now my adult son, who still hasn't found a job, is now up with me in the mornings. I know this would, under normal circumstances, but silly and unjustified. But it takes away any time, there is NO time for me now. I am upset and angry. It's more about our son than my wife, he seems to spend more time lounging around and enjoying the spoils of my hard work than he does anything else. So, my time is becoming my work, which isn't right, and I can't be who I am there, so not really my time at all. Am I being over-emotional? Perhaps, but I don't think so.