In my experience, height doesn't really do much, apart from the fact that it is best to have all your crew members at a similar height, or unless you are too tall or too short to use a boat (unlikely). The main thing isn't height, but how you and the rest of your crew row in sync with each other.
Obviously though, if you are rowing in a single then it's just you. I believe that it's better to have longer limbs, because you have longer strokes, but honestly, you can compensate for that with strength.
And it wouldn't matter unless you are a competitor - just enjoy it!