I think generally when people say "using Skype," they mean using it for video or voice, as those are the specific functions that set it apart from the countless ways people can send text messages to and from each other.
In practice, what sets Skype apart from other messaging apps is that it uses a Microsoft account, not an AOL.com account like AIM, a Google account like Hangouts, a Facebook account like Facebook Messenger, or a Yahoo account like Yahoo Messenger.
If someone asks you to use your phone, I expect that you'd be put out if they used it to scrape off their boots, even though that could be described as "using the phone."
To me, the difference between Skype (voice) and Skype (text) is more akin to the difference between borrowing a phone to place a voice call and borrowing a phone to send SMS. Both are still "using the phone" for its intended purpose.