Well, first, Hobart and Union are both in the group I look at each week, especially on a week like this one when you have to blow up quite a bit of the top 25 and look at everyone honestly and then reconstruct it.
Second, I think comparison is a game we have to play in Division III, because we can't watch games on TV or see highlights, and there are just too many games that all kick off at the same time for any one person to see a whole lot of them.
Personally, I weight performance against good teams very heavily. Sometimes a team can impress in a loss to a good team, and lose a little ground (make you wonder if you were ranking them too high) with a poor game or by struggling against an OK or bad team.
Of course, this all fluctuates all season ... the teams that are supposed to be OK or bad or good in Week 2 sometimes aren't by Week 7. So as a voter, you constantly have to re-evaluate, or you could end up being guilty of just going by records, which doesn't always tell you a lot.
In Hobart's case, giving up 34/48 and appearing to struggle against RPI looks better when they come back this week and play good defense again (three non-RPI games, 7 points allowed), and if RPI wins a couple more, we start looking at that as an impressive win, and not an escape. But the same way, F&M looked like a good win at the beginning of the year, and now you can't tell as much because they're 2-2 ... Rochester of course won't work as a comparison for a while because it's Johns Hopkins first game and one of Hobart's last. ... It is tough for a team to play a lot of "not well-respected" opponents as you put it, but if their performance against them is consistent, as they win, allowing for maybe one game where a good team struggles/escapes a close call, they'll usually gain ground in the poll.
I think you can and should compare common opponents, and if possible common opponents from leagues. As far as the LL and CC, their teams are right around the same place as far as consideration for my vote, anyway. I have voted for teams from both leagues most weeks.
I guess I should remember to put that disclaimer that if your team isn't mentioned in the 26-35 group, it could be because they got a top 25 vote.
And of course one of the great things about D3 is that the polls give you an idea of who is how strong, but each league champ (for the most part) gets to get in the playoffs and show what they can do regardless of their ranking.