In the end, all that matters is goals. In some competitions, "goal saldo", or the total amount of goals scored by a team during the entire season, compared to the number of goals against that team, does count for purposes of tie breaking in ladder rankings.
For example, you get points for a win, a tie and a lose. It is possible for two teams to gain an equal amount of points. The team that then has the best ratio of goals for/goals against is ranked higher.
Yellow and red cards do sometimes carry over to further games, sanction wise. A player that got a red card during a match cannot participate in the next match. In Ghana vs. Uruguay, an Uruguan got a red card and he cannot play the next match versus the Netherlands (which I like, since he's pretty good and I'm Dutch).
During a match, stuff like ball ownership, penalties, corners and shots-at-goal are tracked but this is more for amusement value than for rankings.
So, you're not really missing anything.