If you've experimented with your road bike on a downhill the answer is simple. When you have multiple gears on a bike and are speeding down a hill in the largest gear you have, at some point you reach a neutral state where you can no longer pedal any faster because the gearing is maxed out. In other words the bike is traveling faster than you can pedal.
Now consider the rider on the fixed gear bike, they have only one gear. That gear is what you might call a universal size. Its one that allows them to get started from a stop with minimal effort and also affords a limited amount of top speed on the flats or downhills. Because its a lower gear than the one you would normally ride in on those same flats they can out-accelerate you at lower speeds. That advantage disappears quickly at higher speeds because they will reach that neutral state of pedaling at a lower speed than the rider on the road bike.
Put another way, the advantage the single speed has at lower speeds can be equaled by the rider of the road bike by shifting down to the smaller front chain ring and a larger cog on the cassette. The road bike rider will always be able to out-speed the fixed gear rider at higher speeds.