- A young scientist (say, with a post-doc position) submits an article to an Elsevier journal.
- I am a natural choice as referee (i.e. it's easier for me to judge the work than for any other potential referee the editor might think of).
If I refuse to referee the article, the editor may have trouble finding a referee, or the referee may be less qualified. Either will result in a longer delay of the process, and in a more random outcome of the process. Meanwhile, maybe the next job application for the author is only a few months away.
My own judgment is that the author preferably shouldn't have sent this article to an Elsevier journal. But I don't feel so strongly about it that I want this to cause him to have one fewer published article on his CV during his next job hunt.
The other scenario where I would accept to referee is the one pointed out by other commenters: I am aware of a problem with the article (does not cite related results/correct result, but one of the proofs is wrong/incorrect results/...) - it would be a disservice to the community not to point this out to the editor. Which is all a referee report is in such a case.