For rowing, I work out my arms and shoulders for strength but I also work out my legs. Even in a canoe, your legs can get tired from the rowing and I find that exercising them too helps me cope with that.
I guess here is another tip.
I wasn't super clear on abdominal exercises. If you are looking to gain strength in your upper back and legs, you will need stronger abs to keep your body stable in the boat while you pull, since you will not get stronger.
If you don't have access to a gym, here are some good exercises to do in the boat bay/dock/where ever you do abs. http://truecardio.com/workout-routines/It is a small website, but it works. If you need any more help, ask away. Rowing/Fitness is a lifestyle that I truly enjoy.