It's not a result of not being on AG that causes the reduction. It's because of one of several reasons that are mostly of your own doing. A) You spammed somewhere and the mods removed your comments/posts hence a reduction in points. B) Someone deleted their profile comments, and yours were amongst them. C) In drastic cases, admins can do a user rollback. D) Glitches, whilst rare, have happened before.
So yeah. Anyway, you have been told about this in a previous thread before, so like, you know, try to remember it?
Then... don't spam? And three AP is like, what, a post and a comment? It's nothing, and if you are not going to change your behaviour anyway, you better get used to it.