If a right can be taken away, it isn't just being revoked or removed, it is saying you didn't have the right in the first place.
You can't take something away if it isn't there to begin with. To use your line of thinking: I have an apple. You take it away. I must not have had the apple in the first place because it got taken away.
There is nothing in the nature of 'rights' that insists they obtain, no matter what. As I said before, rights are something that are granted by default. But a person's actions can render those rights inapplicable to that particular person or situation.
Let's consider a person's right, when they are arrested, to remain silent. They have that right, even if they choose to give up that right. A person can be in the middle of a confession and then decide to invoke their rights. That doesn't mean the right doesn't exist, it just means it's not applicable in that particular case.
No confusion at all, just the simple fact that rights are a fallacy, a figment of our misguided trust in the establishment that lets us believe that we actually have something of our own.
The confusion comes in when you try to apply some sort of metaphysical truths to notions like rights or privileges. You're absolutely correct that they are a 'figment' - they are a creation of a particular society or group of people.
Yes, our rights are infringed upon virtually every day. I imagine one day soon that our right to privacy will be a thing of the past. But that is a feature of the society, not of the concept of a 'right'. This is what I mean by using matter-of-fact cases to try to shed some light on the nature of a particular concept. Sure, these concepts are man-made, and to say they 'exist' (in any philosophically significant sense) would be ridiculous. But they're still concepts with applications that are recognized.
I'm sure for any 'right' we can find an instance where that right has been violated. But that misses the point completely about the distinction between rights and privileges. Here is how (at least it looks like) your argument goes:
1) A person can be stripped of certain (or all) rights.
2) A person can be stripped of certain (or all) privileges.
Therefore, rights are identical to privileges.
We could add a subconclusion that rights, properly understood, are an empty concept - meaning there is nothing to which they properly apply.
But of course, this argument is a non sequitur. You are taking the predication of two instances to say they are the same instance. Here's a counterargument to show the flaw:
1) An apple is red.
2) A tomato is red.
Therefore, apples are identical to tomatoes.
I'm not trying to be difficult, I mean, I do see what you're saying. There's nothing that guarantees that everyone will maintain their rights in any situation. But that's just not an interesting conclusion. And to make the further claim that therefore rights are merely privileges by another name simply does not follow. These are unique concepts with unique applications.