No. He was just never raised properly.
Goku is actually fairly intelligent. It's just that he never really visited the real world and knew how it worked. If we go look at Tarzan for example, for all we know, he could very well have an IQ of 120 or above. When you go for a very long period of your childhood without any proper education or learning, it gets harder to absorb knowledge later on no matter what. Stop and think about the fact that Bulma was the first girl Goku ever met for a moment. A kid like him can find almost anything fascinating without being unintelligent. He was never taught logic or culture.
On the other hand he shows significant skills in several areas, including fighting and fighting techniques, hunting, formulating strategies and tactics, tricking his opponents, understanding his environement, and he is pretty good at communicating with other intelligent creatures (ignoring the fact that he doesn't have any manners and tact). All of these things wouldn't be possible for a being of lower intelligence, so in short: No, he is not a retard.