Ah, those weasel words. Considered by whom? Is there any research done in this area? Finnish is not my native language and after having lived here in Finland for 13 years, I am pretty fluent in it. I haven't found it any more difficult that any other foreign languages I have experience with. Different yes, but more difficult hardly. And for your information Chinese too consider their language to be the most difficult to master and so do Russians. I guess the difficulty of a language is a part of the national pride found in many cultures.
English is not easy at all. A myriad of exceptions, completely weird pronunciation rules (again with tons of exceptions), 15 or so tenses - these factors alone make English far from trivial. The only case for simplicity of English I can think of off the top of my head is its ubiquity. In most Western cultures you cannot avoid English, it is everywhere. And if you encounter a language on a daily basis, of course it makes it easy for you.