If you want to feel insanely powerful from the get-go, run to the Graveyard and pick up the Zweihander. Then invest enough points in strength to wield it with one hand and you're basically set all the way to Anor Londo (at least). It basically annihilates anything standing in your way, but you have to time your strikes carefully. After that move to maybe the Giant's Halberd (you'll likely have enough points in Strength at that point to wield it one-handed) or the Gargoyle's Halberd, which you'll get as a 100% drop from the second gargoyle in Anor Londo. If you want a classic sword+shield combination, farm for the Silver Knight Straight sword with the Covetous Serpent Ring in Anor Londo. I've found it to be one of my favorite weapons in the game, with very high damage for a simple longsword, fairly light at 6.0 weight and extremely fast strikes. It does require Twinkling Titanite to upgrade though, so keep that in mind.
The Grass Crest Shield seems to be a popular choice for a shield as it increases Stamina recharge, but it only has 95% physical damage blockage. Combine it with the Mask of the Child drop from Pinwheel (there's a 1/3 chance for it), and you'll basically double your Stamina recharge rate. I would suggest getting the Greatshield of Artorias though.
Also, kill Havel the Rock as soon as possible (most easy if you take the Master Key as your starting gift) and use his ring, considering you're going for a knight build. He's insultingly easy using a simple tactic.