This is an old punchline in China for many years and I doubt it comes from English literature. I guess the meaning is similar (last name ~= name of the father)
Edit: obviously I should google dark power first lol.
Also a slight edit, I wrote name initially. Of course in the books it's "face of his father", but it still sounds similar [1]. To admit to forgetting the face of one's father is to be deeply shameful, to accuse someone of it is insinuating they should be ashamed of themselves.
Edit: obviously I should google dark power first lol.