<p> has a default top and bottom margin, <div> does not.
Both are block-display elements, but <p> only permits embedding inline-display elements. You can partially work around this by redefining the display property from inline to block for e.g. <span> elements, but it feels a bit icky...
The only downside I can see is by replacing <p> with <div> is that paragraphs are not separated by a blank space in readers that do not support CSS.