It's always a dig at someone for having interests that are different to the person making the comment. And it's basically a judgement that a person chooses to spend their leisure time on something different to the person who said it.
Someone can spend all of their time collecting stamps, getting drunk, playing computer games, listening to jazz, or watching football games - but if they see someone who instead puts their time into writing sonnets in latin, making intricate replica costumes, or anything else that the first person either wouldn't do, or isn't used to, and out comes an implicit criticism that the other person is somehow _so_ extra-blessed with time that they are able to waste huge amounts of it on such a frivolous activity.
Frankly, the only people who get to complain about how other people's time is spent are people who spend _all_ of their own selflessly helping others. Those people have the moral high-ground when it comes to the use of their time. Everyone can stop being snarky about how other people choose to enjoy themselves.
Original post on Dreamwidth - there are
