Illustration credit Don Dixon.
Dying Earth settings tend to be dark and cold and maybe (as above) glowing red, due to the failing Sun. Nature's cycles or even physical laws themselves have gone perversely off the rails or ceased completely, like a kind of cosmic menopause. (Whether this extends beyond our provincial corner of the universe, and if so why, is another question.) Interestingly, this derangement of reality is usually not the fault of humans - these aren't ecological morality tales - or at least the exhaustion and littering of the planet is pictured dispassionately, as an aggregate trend that has no moral meaning. (Although not a dying Earth work, Stephenson's Anathem has hints of a long history as resources are scavenged from old ruins, although the tone in describing this activity is again very matter-of-fact.) This creates a setting of an incomprehensibly huge and uncaring universe, a clockwork winding down despite any designs harbored by the characters or their ancestors, and indeed some of the profound questions of the human condition become meaningless against such an unforgiving vast backdrop: lay down and die now, or continue struggling to pass on genes and values and create happiness? It's all going to disappear in a few years, so how can it matter? In reality, we're all going to die, either now or a bit later, and this is the moral choice we all face already. It's just whether or not we already know what's going to kill us. It's not surprising then that Dying Earth works are good vehicles for exploring questions of meaning.
The collapse of reality mirrors or brings about a collapse in human society, where reason falls apart. Often the human race comes into contact with forces or beings we can't understand, either revealing themselves in the twilight of existence, or appearing as the clock strikes midnight. And there's a criticism of the subgenre: some works have struck me as yet another excuse to write fantasy but call it science fiction; the breakdown in nature and society returns us to an era of taverns and swords and magic. This is why Jack Vance's work is not my favorite. It's an easy trick to write a sword and sorcery novel but subvert the simplistic paradigm that science fiction differs from fantasy in that "it's in the future, so it might actually happen" (China Mieville has some interesting things to say about this attitude). But dying Earth writers are not the only ones who have used a faraway setting to throw out all the rules of history and write familiar settings. Another cheating technique science fiction writers use is the intervening apocalypse, to reset society and technology. The most honest and original solution to this problem for my money is Vinge's Slow Zone, but there are lots of cheats people use to make world-building easier.
The horror these authors convey at the extinction of reason is at the core of dying Earth prose, more important I think than the used-up Earth or the cooling Sun, and it's this that strikes a chord of unease in many readers, children of the Enlightenment as we are. The far future setting is just a way to get to a place where the laws of reality are broken, although if you're bold enough as Delany with Dhalgren, you can break them in the modern American Midwest. In fact I think we can throw out some of the traditional entrants in the subgenre and simultaneously reconsider some of strange fiction's early heroes as exploring this at least as thoroughly. Indeed, although the Earth is literally dying at the end of the Time Machine, we can make a pretty good guess at how the giant crab things got there (some iteration of evolution) and why they're struggling to survive (the Sun is burning out due to well-understood but inevitable physics).
Non-Euclidean geometry, by I2ebis.
In contrast, Lovecraft might not have been writing about the far future or the death of Earth as such, but he conveyed something far more unsettling that is at the core of the more disturbing dying Earth works, and it's this: we comfort ourselves by describing rationally the small slice of experience that our limited brains can deliver. Even if reason is not an illusion, then there are "black swans" which we cannot hope to have encountered in our brief existence, but which are no less important for our naivete. Asimov's Nightfall hints at this in the gibbering insanity that heretofore unknown darkness brings. In the real world, there are gamma ray bursts, comet swarms, clouds of debris around the galaxy that we rotate into periodically, supervolcano eruptions, and magnetic solar storms. And these at least are all things that we know exist! To the modern naturalist worldview, confident that we have either already understood everything important, or ultimately will, because we can, this is terrifying. As an aside, I don't personally know any unreconstructed theist who is a fan of strange fiction, and I predict it wouldn't seem that strange; they already think the world is fundamentally incomprehensible.
Of all the early strange fiction writers, Hodgson is the one who did this best. Where Lovecraft often gives us the details of the pantheons he has created, Hodgson leaves us in the same fog that his characters suffer. The House on the Borderlands is better known, but The Night Land is a better example (and here's a great resource on that work). He leaves the a-rational horrors lurking in the shadows, their forms not fully understood, exactly as experienced by his characters. King's The Mist seems like a modern cinematic version of the Night Lands, except set in a familiar place.
A specific manifestation of the horror of the irrational, and one which more extreme horror has begun using in recent decades, is the divorce of experience from matter; that is, the existence of consciousness separate from damage to a body or control of the experience, more plainly, the possibility of hell. Lovecraft warns of something like this happening when Cthulhu awakens. Recently Iain Banks wrote Surface Detail, exploring the morality of simulated hells, but it wouldn't be correct to consider this horror of the irrational because there are still "rules"; the hells are a simulation, and the universe of the Culture Banks has created is eminently rational and I would argue is actually an extension of Enlightenment ideals.
A final common thread about far-future works in general, particularly from the classic period, is that they often explain how they got into our hands, as in Stapledon's The Last and First Men, where the author claims to be a mere telepathic mouthpiece for a far future historian. This is interesting because it's not obvious why these works would feel called upon to defend their authenticity, as compared to other works of science fiction.
It should be pointed out that among Lovecraft's concerns was the creeping perverse derangement of high European, especially English, reason by the infiltration of what he may have called the sinister, dark and Oriental races; more on this here. Surely modern California would have presented a nightmare vision to his sensibilities, one which doesn't seem to bother most people today.