Physically the speed of the signals going over the wire are the same, being electrical signals going over the same wires, but the 100 bytes go faster (and have lower latency) on the ADSL link because they get on and off the wire faster.
That's why I said it has little effect. I'm convinced that latency has a large effect on throughput but I am unconvinced that throughput has a large effect on latency. I get 16ms latency on my ADSL connection, which is 1.5Mb/s. I don't know anyone with 20+ Mb/s cable connections that has much lower latency than my connection. On the other hand, if you go to something like a 2400 baud modem then latencies are up noticeably.
So, am I misunderstanding? Gigabit fibre should have significantly better latency than a lowly 1.5Mb/s ADSL link? I am not sure it does (haven't tested one).