I much prefer first person perspective. I feel less immersed with the character and story if I am floating observer of the action rather than a participant. (There are some exceptions to this though - playing as Batman works better when you are constantly reminded of the awesome guy you are playing.)
I also get irritated when I have to constantly peer past my character when trying to see the environment, either for the practical needs of playing the game (seeing what I am shooting at) or the aesthetic factors of being able to appreciate the view. The game Papo and Yo was an example of the latter; it had a dynamic FOV that would zoom into your character's back when you stopped to enjoy the lovely scenery. It was very irritating.
It annoys me when I can't look up at where I need to go except at the moment the game decides to take camera control away from me to show me something.
I also dislike having to control my camera. In first person, the camera is linked to where you are aiming and where you are going unless you strafe (which 99.9% of the time is what you want). In third person mode (and especially with a controller), the camera becomes yet another thing that you have to fight against; constantly having to adjust it to keep what you want to see in frame. I imagine people coming from consoles are so used to this that they don't notice any more, but as someone who grew up on the PC with Doom & Quake this is something that always seems to get in my way.
The benefits of third person view is that it better simulates peripheral vision (although it goes too far and shows you behind you too). It also makes up for the lack of awareness of your body, although games like the Thief series and Mirror's Edge show that you can still get away without this. It is also better for those games that zoom back quite a way when a more tactical view is required, but I consider that not so much a third person view as a strategic view.