or a world where any person would be equally likely to have biological children and grandchildren.
They would have to have precisely the same number of children (i.e. not die to due accident or stillbirth before reproduction), and reproductive assignment between sexual partners would have to be completely random, not sexually selected as it is today (even in countries where sexual partners are determined by parents, the parents are still performing a kind of sexual selection).
But with sexual reproduction, even in such a completely "fair" and "random" mating situation, genetic drift would still occur because although at a single locus each parent passes on one or other allele with a 50% probability of each, that 50% is a statistical average. Out of a ten thousand reproductions, you might find one reproduction where an allele is over-represented or under-represented. So across the population, genetic change will happen.