“Let’s put our heads together and start a new country up,” starts Michael Stipe.
To be a country requires three things:
- people (a permanent population)
- sovereignty (the ability to make and unmake laws)
Otherwise, the song is probably about the river of the title, in Ohio: the tribes of Indians that used to live there and its past state of being polluted.