I think it ultimately amounts to whether you view games as entertainment, or as a form of artistic expression. Although a LOT of games that are not fun are poor, there are notable exceptions.
Case in point, Silent Hill 2, the game is largely uneventful, and consists largely of walking through an empty foggy town. But at the same time, your natural curiosity about the surrounding area, and the tension that it creates are enough to keep you playing. For that matter, a LOT of the horror genre fits into this category.
Also Heavy Rain, the gameplay on the whole was rather bland, but the story and characters were so fascinating you wanted to experience with them how everything played out
I think the answer to your question however comes down to one question:
Are you invested in the story enough to see it through to the end?
If the answer is yes, then the game need not be necessarily fun, so long as it allows you to take part. Also fitting in this area to me are JRPGs, but thats another matter
If the answer is no, that isn't a bad thing. One cannot say they are wholly enraptured by the bird's plight, and their struggle to recover their young in Angry Birds, it just means that the story is merely the vehicle by which the gameplay is driven, and it is what makes gaming special. It becomes about the journey rather than the destination.