as to how it could happened it your specific study, first check

First check your outliers and then your alpha level. it's the basic difference between a sample vs population. by doing a statistical test, you are drawing two samples from two populations and compare their means in order to determine if the underlying two populations have the same means. if you have the population, then you don't need statistical test and if you detect any difference in two groups it's true difference and the situation you mentioned would not happen. if you draw a sample and assume it's random and further more that the underlying distribution is Gaussian (from your description it looks like a simple T-test). Remember when you did the test you made several assumptions: that the underlying distributions of both experimental group and control group are Gaussian, and the variances of the test vs. control are equal, and the sample sizes are equal. If any of the assumptions are violated, then the test would not be robust (hence results not reliable). If all the assumptions are met, you still have the assumption of random sampling which is a nice assumption but often not the reality. When you do a T-test, you are comparing the mean values of experiment group with control group (where the normal dist assumption comes into play so the mean is an unbiased measure), devided by the pooled standard deviation (where the assumptions of equal variance and equal sample size come into play) and compare that to the t-distribution table. If you get a sample with extreme numbers which are true values, they will significantly skew the mean for that group, which could render a significant difference result even though the true means for the two populations are the same. What you could do is to check and exclude these outliers (or sometimes data entry error) from the analysis. (Outliers are usually defined as those outside two standard deviations from the mean.) Also there is your alpha level. Alpha level is the probability that you decide the chance of difference between the two groups is greater than what would occur by random sampling from two populations. The higher the alpha level (0.01 or 0.001 as compared to low alpha level of 0.1) the less likely that the difference you detected is due to random sampling difference and more likely due to true difference in the means of the two population.