Calories matter, but source of Calories matter almost as much. Starch and sugars (Carbs) raise insulin levels. Insulin not only helps are bodies burn glucose, but at high levels of insulin, convert glucose into fat.
Low carb diets take advantage of low insulin levels to keep the body from converting food into fat storage. Other processes help too, such that fat, protein, and fiber take longer to process so keep on feeling satiated longer. One still needs to track food, but it becomes easier.
It is much easier to feel full eating a small salad, an 8oz steak and a side of steamed asparagus then it is to eat a 4oz steak and baked potato even if the two have equal amounts of calories. The former is also more nutritionally complete and keep insulin levels low.