From Arstechica's article:
"Although the company no longer offers unlimited plans to new customers, it allows current unlimited customers to renew their plans and has sold millions of existing unlimited customers new... contracts for data plans that continue to be labeled as 'unlimited,'" the FCC said. "In 2011, AT&T implemented a 'Maximum Bit Rate' policy and capped the maximum data speeds for unlimited customers after they used a set amount of data within a billing cycle. The capped speeds were much slower than the normal network speeds AT&T advertised and significantly impaired the ability of AT&T customers to access the Internet or use data applications for the remainder of the billing cycle."
So as a rough order of magnitude estimate "millions of customers" equates to $100's of millions of revenue a month, over nearly 5 years, so they made roughly billions to 10's of billions of dollars on these accounts over the time period. And that is excluding customers that moved to a different plan as a result of the throttling.
The FCC said it believes millions of customers have been affected by AT&T's throttling, with speed reductions that "imped[ed] their ability to use common data applications such as GPS mapping or streaming video." On average, customers' speeds were slowed for 12 days per monthly billing cycle, the FCC said.
These customers were impacted for about a 1/3 of the time, and if you value the throttled service at half the value of the promised service, that comes to 100s of millions to billions of dollars that they were overcharging. So the fine is on the low end of reasonable.
Note, that the FTC is also investigating this and may require AT&T to refund money to their customers in addition to paying the FCC fine.