I was able to avoid casting completely for over two years while working in Scala. Certainly, the kind of work I did somehow supported this.
Only lately I had to resort to casting sometimes. But this happened only when dealing with rather complex types that abstract over nearly arbitrary computations. At one point you reach the ceiling of what is achievable with some fixed type system—and then you need casting.
I am now using shapeless for meta-programming, but I think this is the point where you yearn for more a expressive (dependently typed) type system than the one offered by Scala (or even Haskell). I hope the next generation of programming languages will make this practical.