Jealousy is an emotion and are really beyond your ability to control, being chemical reactions in your brain that are triggered when given sufficient stimulus. So you are quite allowed to be jealous. But that's not really your question. Your real question is are you allowed to act on that jealousy.
That's a thornier problem as you and this fellow had an understanding, although you reached this understanding without considering the possibility that you might actually know some of his... penis pies, apparently.
Fact is, no matter what you do, nothing is going to be the same. That is just life. Life is change. Nothing lasts forever. It is best to accept this. Even doing nothing, things are likely to change anyway.
My advice, such as it is, is to go ahead and tell this guy what you are feeling. You might want to couch it in term of you did not expect these emotions and how knowing one of these girls is what's making it weird for you. If nothing else, this will bring it all out into the open so that it can be dealt with directly instead of going on a slow boil for months and erupting when you least expect it.
I do not know what he'll say or do. Nor do I know what will happen. but, trying to hide your feelings is likely to lead to regret. We regret the things we did not do much more than we regret the things we did. At least by taking action we learn something. Doing nothing doesn't teach us anything. That's my take.