Otaku roughly means "In love with fantasy." That encompasses all fantasy, not just animes.
Actually that is not correct either, both because there is fantasy anime so Anime Otaku does not make the most sense. But Otaku is used to refer to people with obsessive hobbies, and that can mean just about anything. Otaku can apply to any interest or hobby, it can used for someone who is really into anything from Anime to Zoology and everything in between.
I was closer than almost everyone I know. "omg i m an otaku oxoxox!" A bit more insulting than usual, but accurate enough.
Well yeah just about everyone I know only associates otaku with anime, but it's not. And yeah it is insulting cause it is the Japanesey version of Nerd/Geek.