You can uninstall it, I think by pressing triangle while it's highlighted. You can't remove the icon from the XMB, though.
Sony really screwed up with the Playstation Store icon in the XMB. I loved it when you just had to go to it horizontally, and you were there. Then you had to press down...