The old Timid Penis is a pretty common young-people problem that frankly television sitcoms and the media in general have done just a HORRIBLE job preparing us for.
Nervousness can very often factor into these things, especially when comdoms are involved. Oh God with the condoms.
Here's some generic advice categorised for your convinience
Substance-Assisted:
-Get drunk or at least merrily tipsy when engaging in amarous hanky-panky. It can take the edge off and get you more in the moment, without worrying and getting anxious. I'm not saying "get drunk every time you want to have sex" of course, as that'd certainly kill the spontaneity of the act, as well as your liver. But it's a good way to get yourself into the rhythm of things until you gain the confidence and knowledge that you can "do it" that's needed.
-Viagra. It sounds extreme but it can be a really great placebo. Pop a pill for the first few weeks until you're comfortable in this new sexual relationship. Once confidence has grown, so will certain other things, without need for the drug

Obviously this is quite extreme and women can be somewhat offended by it, so I'd recommend this approach last.
Behaviour-Assisted:
-Take sex off the table for a bit, but not general hanky panky. Engage in cuddlings and fumblings with your partner, just enjoying every sensation and the general arousal without having to worry about where it will lead. You'll more than likely get hard very easily without the strain of performance anxiety (which is what this sounds like, especially with the condom thing) - but even so keep sex off the table for a couple of weeks. Let yourself become confident that your body will do its job keeping and maintaining an erection. The fear of its fragility is often what causes it. Human brains are bitches.
-If condoms continue to be problematic and not using them isn't an option, try to make the event of putting it on a bit less jarring. Traditionally there's an awkward moment when hanky panky has to stop dead cold for a few seconds while you wrestle that wrapper open, fiddle about the item inside and try to slip everything on. That's a mood dampener no matter where you're at in your sex life.
So, providing your partner is willing, have him or her help you put it on, carrying on with foreplay as you do so. Keep your mind in the moment with the act of putting the condom on becoming a secondary background event. Keeps it from being less jarring.
I have no personal experience with the substance-assisted advice listed here, but I do with the behaviour-assisted

I offer as my credentials the fact that I am a sexually active man who had these sorts of struggles for the first eight months or so of his sex life.
EDIT: Oh God and of COURSE! Most importantly, talk with your partner and let them know of the issues and worries you're having. Unless the relationship is entirely based on sex, it's quite likely they'll be understanding about it all and will be happy to help.
Psychological Erectile Dysfunction tends to be embarassing and shameful for a lot of men (and again I blame goddamned sitcoms for always portraying not-getting-an-erection as a once-in-a-lifetime-yet-incredibly-devastating ailment), but it's a pretty bad idea to not talk to your partner about any of it. Their imaginations will start to wander, after all.
Svenparty said:
Just to clarify, The problem is now solved. I totally overreacted and now feel very dumb.
Thanks very much for the helpful advice, Like many of you partially said: It was all in my head(or rather NOT in my head)
Edit 2: Oh... right...

Nothing to do here *jetpack*