I think in your case, it's not likely to have all been the same person, and even if it is, it is not likely that they're doing it out of spite against you especially since it is far more likely that it's just about them having a different taste to you. Usually, people don't get so riled up over something like skin concept that they stalk and downvote others just for the hell of it.
...
Usually the stalking target downvote against a particular person happens in ways you intuitively know is likely from a particular person or at least some guy from their camp of supportors, as the point of the exercise is often to protest against you silently, "both of you know what's going on but no one will say anything" kind of deal.
From what I understand of such behaviour, this really only happens after you having certain hot debates (because internet argument is such a serious business, amiright guys? Huh, what do you mean, "No"?). After certain hot debates, every now and then I'd be downvoted for every post I wrote and almost as soon as I wrote them, even if the post I wrote was clearly inoffensive and can't really be wrong for anything, but ofc the downvote knows no bounds. That is when I know someone is out to get me, and the behaviour usually ceases within a few hours as they got bored of it or simply because they cooled down (but I've got a couple instances where it went on longer, though that mostly happened on OCE boards/forum before it became a ghost-town where due to low user count so if you've wronged someone you kind of wrong them for life, or at least a few days).