The Russians, just like any other of us Slavic morons, are VERY patriotic. You were part of the Russians for 2 centuries if I'm not mistaken. So, the Russians consider you Russia. Same like how the Serbians and Croats and Slovaks consider Russia belonging to the kingdom of Serbs, Croats and Slovaks.
Most of the old Ruskies will be reluctant to take the time to learn Latvian, because, well, they're Ruskies. They just (relatively) recently have gotten permission to actually exit their own country, most of them still don't know how stuff works in other countries.
And don't worry about Russians attacking you. Russians are very peacful, except when it comes to the US.