"In March 1999, Musk co-founded X.com, an online financial services and e-mail payment company. One year later, the company merged with Confinity, which operated a subsidiary called PayPal. PayPal and X.com each had a person-to-person email-based payment system. The original intent was to merge the two systems, but it never happened.
Musk strongly favored the PayPal brand over the X brand. After initially co-branding PayPal with the X brand, including making X.com a subdomain of PayPal, he moved to officially remove the X.com brand for good. Following this, the board appointed PayPal founder Peter Thiel as interim CEO. PayPal's early growth was due in large part to a successful viral growth campaign created by Musk. In October 2002, PayPal was acquired by eBay for US$1.5 billion in stock, of which $165 million was given to Musk. Before its sale, Musk, the company's largest shareholder, owned 11.7% of PayPal's shares."
TL;DR. Musk didn't create PayPal, he cofounded a company with a competing service that merged with the company that owned PayPal and while at the merged company pushed to use the PayPal platform as it was better. Then he left in 2002.
I really don't think we can lay the lion's share of PayPal's shittyness at his feet based on that. By that logic, Windows 8 should see Bill Gates hanged...