I haven't watched a Harry Potter film since I saw the last one in theaters, so my opinion is probably invalid. But here's how I see it:
The first two are the best family films. They're not too dark and have the whimsy, and most of the setpieces child readers would want to see from the book, like the Quidditch game, the chess battle and so on. Adults can appreciate the filmmaking craft and the British heavyweights giving some gravitas to the whole affair. Chamber of Secrets is a tad darker, and has some scary and even violent parts (Basilisk fight at the end), but the tone is still decidedly more childlike and whimsical. The series started out as stories for children, and it's completely expected that the first films would reflect that.
In the third film the whole tone changes, and moves more towards a teen audience. By the end the series is bordering on R-rated in terms of subject matter. But none of this was known during filming, and neither were plot specifics really. When the books are only an installment ahead, it's kind of expected to just make a straight adaptation, since the filmmakers couldn't really know which elements or themes would be central to the big picture. The tone changed as the books changed, and that's why comparing the first film to the last is like comparing Disney's Aladdin to Schindler's List.
Since I, and a lot of others on this site, grew up right alongside Harry in the books, I can't help but wonder what the series feels like now to a newcomer who can read the whole saga from start to finish without having to wait 2 years between books. I think I was in third grade when I read the first one, and I can't possibly imagine being able to read Deathly Hallows at that age. It's just chock full of straight up shock material, like when Ron teleports wrong and nearly bleeds out, or Hermione getting tortured, and the amount of characters and plotlines just splooges in the fourth book. Not to mention the tone and themes, which at that point are both dark enough to make the book not fun for third graders, and complex enough to fly completely over their heads.