The Problem

In the Selfish Gene theory of altruism, parents care for their offspring because they carry parental genes. This theory of altruism is extended to genetically related family groups.

But there's a puzzle here. How does an individual recognize its degree of genetic relatedness to another individual? Do individuals have 'gene sensors', like eyes or noses or ears, which can provide a reading indicating degree of relatedness, or shared gene count, when pointed at or pressed against another individual?

In principle, there should not be a problem for an evolutionary process, that has developed exquisitely elaborate sensors like eyes, to develop a DNA comparator. After all, every cell contains the equipment to cut, splice, and replicate DNA. A DNA comparator cell would simply have to compare sampled DNA with the cell's own DNA. Multiple banks of comparator cells, with each comparator devoted to a region of DNA, should be able to perform the comparison of the entire set of genes in much the same time as an eye takes to process an image. If each comparator simply reported 'Yes, it's the same' or 'No, it's different', an accurate degree of match could be produced simply by dividing the sum of the Yeses by the sum of the Yeses and Noes. On the face of it, it certainly looks easy enough.

Yet it appears that such sensors do not exist. If so, it seems a remarkable oversight by selfish genes that they have not provided their carriers with gene recognition sensors of some sort, particularly if the entire purpose of the selfish gene is to reproduce and multiply in future generations. If life really is all about selfish genes replicating and multiplying, living creatures would have evolved gene sensors or kinship indicators before they even developed eyes or ears. Given such gene sensors, parents would then automatically reject offspring that were not their own. When the cuckoo hatched, the cuckolded birds' gene sensors would instantly inform them that it was genetically unrelated to them, and they would abandon it.

Richard Dawkins raises the matter of kin recognition in The Selfish Gene:

How could wild animals 'know' who their kin are, or, in other words, what behavioural rules could they follow which would have the indirect effect of making them seem to know about kinship? The rule 'be nice to your relatives' begs the question of how relations are to be recognized in practice.
(Richard Dawkins. The Selfish Gene. Ch.6)

Selfish genes are supposed to look after their own. Individuals shouldn't behave altruistically towards genetically unalike individuals. That's fraternising with the enemy. Yet Dawkins produces several examples of creatures looking after unrelated individuals. Most of which he dismisses as 'mistakes' of one sort or other.

...there is at least one well-authenticated story of a drowning human swimmer being rescued by a wild dolphin. This could be regarded as a misfiring of the rule for saving drowning members of the school. The rule's 'definition' of a member of the school who is drowning might be something like: 'A long thing thrashing about and choking near the surface'....

...Of course, the rule misfires in domestic or farm situations when a hen is made to sit on eggs not her own, even turkey or duck eggs. But neither the hen nor her chicks can be expected to realize this. Their behaviour has been shaped under conditions that normally prevail in nature, and in nature strangers are not normally found in your nest...

..There is one example of a mistake which is so extreme that you may prefer to regard it not as a mistake at all, but as evidence against the selfish gene theory. This is the case of bereaved monkey mothers who have been seen to steal a baby from another female, and look after it. I see this as a double mistake, since the adopter not only wastes her own time; she also releases a rival female from the burden of child-rearing, and frees her to have another child more quickly. It seems to me a critical example which deserves some thorough research...

...Adult herring gulls do not recognize their own eggs, and will happily sit on other gull eggs, and even crude wooden dummies if these are substituted by a human experimenter. In nature, egg recognition is not important for gulls, because eggs do not roll far enough to reach the vicinity of a neighbour's nest, some yards away.
(Richard Dawkins. The Selfish Gene. Ch.6)

It is rather hard to see how a dolphin can mistake a human for a member of its school, unless dolphins are almost completely blind. If the who-to-rescue-rule really is as vague as Dawkins suggests, one would expect there to be reports of dolphins attempting to rescue any and every sort of large fish, and possibly even the odd rowing boat.

The wealth of examples and counter-examples serves, however, to cloud the real question: how do the creatures recognize their true kin, the ones they share their genes with? Dawkins makes no suggestion that the creatures possess any 'gene sensors' to tell them. Skipping individual relatedness, he shifts to average relatedness. The creatures on average can be expected to be as altruistic to each other as they are on average related. Discussing how human studies of lion prides by Brian Bertram suggested that males are on average slightly less close than half brothers, and females slightly closer than first cousins, Dawkins then makes the amazing assertion that

the average figures that Bertram estimated are available to the lions themselves in a certain sense.
(Richard Dawkins. The Selfish Gene. Ch.6)

In what possible sense are these statistics available to the lions? Were they able to read Bertram's report? Do lions while away their idle hours calculating their likely relatedness? Somehow or other, quite mysteriously, the lions are assumed to somehow 'know' or be able 'estimate' their relatedness.

So we conclude that the 'true' relatedness may be less important in the evolution of altruism than the best estimate of relatedness that animals can get. This fact is probably a key to understanding why parental care is so much more common and more devoted than brother/sister altruism in nature, and also why animals may value themselves more highly than several brothers. Briefly, what I am saying is that, in addition to the index of relatedness, we should consider something like an index of 'certainty'. Although the parent/child relationship is no closer genetically than the brother/sister relationship, its certainty is greater. It is normally possible to be much more certain who your children are than who your brothers are.
(Richard Dawkins. The Selfish Gene. Ch.6 Original emphases)

Dawkins has raised the question of how animals can 'know' who their kin are, and not answered it. What emerges is that they are assumed to just know, somehow or other. No explanation is offered as to how they know. They just do. It's a Just So story.

Dawkins suggests, in an appendix to The Selfish Gene, that kin recognition through smell may provide the answer. But there are two problems here. The first is that he is referring to kinship studies which involve not only parents guessing who their kin are, but also a set of human observers making parallel guesses. To what extent are human observers, themselves lacking 'gene sensors', likely to be any better at assessing kinship than the animals they study? The second problem is that while some individual may have a characteristic odour, then even if that odour is determined by several genes, recognition of that odour allows recognition of odour-determining genes, and no others. And if odour is determined by non-genetic factors, such as what a creature has eaten, or its state of health, or by a population of parasitic (and totally unrelated) organisms that it happens to carry on its skin or in its gut, then odour will provide no information about kinship whatsoever.

Given that individuals lack 'gene sensors', any estimate they might make of their degree of relatedness to other individuals can only be a wild guess. The only other possibility of assessing kinship is that creatures see which males mate with which females, and which females give birth to which offspring, and then somehow construct family trees. For this to work, the creatures would have to be all but omniscient. In reality, any one individual is unlikely to witness a fraction, and quite possibly none, of the matings and births going on around him.

Certainly brothers and sisters can have no idea whether they actually are related. Equally, a father can have no idea whether his offspring has actually been fathered him or by another male. A mother, it would seem, is in the best possible position to know who her offspring are. But in the case of cuckoos which lay an extra egg in an unrelated bird's clutch, a mother often ends up raising a cuckoo instead of one of her own offspring. In mammals, which give birth to live young, eggs usually can't be swapped - but it is possible for offspring to be swapped at birth, and for mothers to raise children to which they did not actually give birth - as occasionally happens when human babies get mis-tagged in maternity hospitals (the ideal environment for swapping). This leaves a situation where a mother is the only individual who has any chance of actually knowing how she is related to the very small subset of the population which are her children. And even then there is uncertainty.

The Answer

Without 'gene sensors', the creatures can't measure their relatedness. Falling back on individual personal experience, of who mated with who, or who gave birth to what, won't help. So the creatures simply don't know the degree of their relatedness.

This may seem strange to humans, each of whom has a Mom and a Pop, and possibly a Bro and a Sis, and a whole army of aunts and uncles and nephews and cousins. After all, we know who we're related to. Or at least we think we do. But human family trees are built by pooling individual human experience. That woman A married man B on such and such a date, and they had a daughter C and then a son D. Marriages, births, and deaths are formally registered and witnessed. They are remembered by friends and relatives. They are photographed and videoed and reported. But none of this constitutes hard evidence of genetic kinship. Daughter C might easily have been fathered by someone other than father B. Son D might have been swapped at birth for the son of some other mother than A, or secretly adopted. Human family trees provide the best guess of which people are most likely to be genetically related, not whether they actually are. Only humans can do what Richard Dawkins suggests that lions can somehow do - 'estimate' their relatedness. The trap into which genetic theorists fall is to convince themselves that animals somehow must in a certain sense 'know' who they are related to, just like we humans know. But they don't know. And neither, really, do we.

Genetic relatedness can be thrown out the window now. And with good reason - if individuals are unable to assess their degree of genetic relatedness to other individuals, it is quite impossible for them to act towards each other on the basis of genetic relatedness. Thus genetic relatedness is one criterion that living creatures cannot be using in their interactions. If mothers look after children, it cannot be because they are genetically related. This suggests very strongly that the natural world is indifferent to the genetic constitution of its members. Some other explanation is needed for altruism than genetic kinship.

This isn't the place to start developing other ideas, but a quick glance may throw up some possiblities. It may be that parents simply adopt the offspring they happen to find themselves landed with. Ironically, Richard Dawkins may have been nearest the real truth when he wrote:

Cuckoos exploit the rule built into bird parents: 'Be nice to any small bird sitting in the nest that you built'.
(Richard Dawkins. The Selfish Gene. Ch.6)

The actual rule, applicable to any parent, might well be: 'If you're not too busy, be nice to any small, helpless creature.' Such a rule, or something very like it, would explain the ready adoption of the offspring of others, and even the adoption of creatures of quite different species. The critical part of this rule is 'if you're not too busy', for once a parent has adopted one small, helpless creature, it will be too busy thereafter to be nice to other small, helpless creatures. In circumstances where food-gathering is not time-consuming, a parent may be able to look after several offspring. Where food is hard to find, parent birds may refuse to adopt any offspring, or abandon those they have adopted. If the actual genetic parents of a small creature very often raise their own offspring, it would simply be because they were present when another small, helpless creature appeared in the world, and they were thus the most likely to be the first to adopt them.

If so, what Dawkins calls 'mistakes' might actually be the way the natural world works. After all, he is only calling them 'mistakes' because he is assuming that Selfish Genes are out to replicate and multiply themselves, and that any behaviour which doesn't fit this assumption must involve some sort of 'misfiring' of the rules.