OK, I finally got some time to look at what actually happens in the code. The problem is caused by a long-standing bug in our code and a particular nesting of spans.
The bug that we have is that if a float happens in the middle of the line, our engine always pushes it down, without attempting to shift it to the left or to the right.
Now the particular span nesting that this example uses has a span with a leading margin that contains a float as a first element. That margin is counted as content, so the float bug kicks in and the float gets pushed down.
A workaround is simply to put margin after on an element after the float, e.g.
<p><span class="drop">I</span><span class="first">n</span> the ancient city of London...