We have two keys on our keyboards designated for deleting text: the backspace key and the delete key (these may even be the same physical key, as they are on my keyboard). That is, of course, unless you are in the “Selected text” mode, in which case any key on your keyboard will delete your selected swathes of text.
This is incredibly poor user interface, and unfortunately shared by every operating system I have ever used, including the brand new iOS. I am curious as to who designated this as a good idea. It's been a while since I've read Jef Raskin's book (Raskin was the creator of most of the original Macintosh user interface, which every modern operating system takes after, to this day), but such modal and destructive behaviour seems unlike the rest of his teachings (I could be mistaken).
I believe this to be poor user experience because Text Selection is marking a range of text for manipulation (like copy, move, style changes, etc.). Of course, you might also like to erase the text, in that case you can use the Delete/Backspace key. But to have any key also function as 'Replace all this text', when you're solely in a text manipulation mode just seems like a destructive action, where it wasn't explicitly commanded by the user (it might be explicit if you've experienced the behaviour before, but consider a novice user). I can't think of a good way to rationalize this to a novice user.
The only argument in favour of this behaviour I've ever heard is “People are used to it already, it would be difficult to change”. I've heard this argument applied to many poor-but-existing interface paradigms and I have to say it's completely bunk. The fact is the number of users accustomed to this paradigm could be shrinking every day (they die), while untrained users are arriving every day (they're born). We shouldn't punish new users just because of what we're used to.