Honestly, it's a huge plot hole: "Oh, let's have this girl confess her love to the main character, only to have him go off on his adventures and completely ignore her." Seriously, the series just keeps going as if nothing happend. Hell, I would have killed for a "...I don't have an answer for...