There is a drift, and there are fluctuations.
Regarding the drift: The day length is getting gradually longer by about 1.7 milliseconds every century (+2.3ms due to tidal braking, -0.6ms due to glacial rebound). In about 1820 the day was 86400 seconds; now it is longer than that. In a thousand years, the day will be about 86400.017 seconds, and we will need a leap second every couple of months.
[Note: I am simplifying a little here for the sake of clarity by ignoring the difference between a solar day and the stellar and sidereal days, which are about 4 minutes shorter].
Regarding the fluctuations: There are fluctuations of the earth's angular velocity on many timescales. It fluctuates with weather, with the seasons, and with major events on the surface (e.g. a dam creating a new reservoir) and in the earth's crust (e.g. an earthquake or major volcanic eruption) and deeper interior (e.g. we don't really know). All these events are minor rearrangements of the mass of the earth, which change its moment of inertia. Conservation of angular momentum dictates that the angular velocity must change, and it does. Of course the earth isn't a rigid body and that complicates all this. Learn about Geodesy if you want to know more.
In the 1990s the day length was approximately 86400.003 seconds, so we needed a leap second every year. For poorly-understood reasons (possibly some sort of deep mantle activity), the earth's rotation speeded up around the year 2000, and for a while the day length was about 86400.0004 seconds. Now it is slower again, about 86400.001 seconds. These changes all come under the "fluctuations" heading.
There is an organisation called the IERS - International Earth Rotation and reference Systems Service - which collects measurements of all this stuff to very high accuracy and produces all sorts of reports, bulletins, data sets, etc etc.