The
PortableApps.com official policy is a good summary of when and why to provide 64-bit portableized applications.
tl;dr
Driver code or anything else that interacts with the OS below the level of the WOW64 emulation, and absolutely must use the same bitness as the OS in order to run in the first place.
OR, anything which can see significant speed improvements while requiring very little additional install size overhead.
In either case, both 32-bit and 64-bit versions are installed, and at runtime it figures out which is the best to use.
Distributing only 64-bit is simply not an option in any way shape or form as long as there are still 32-bit systems out there, of which there are many regardless of what you believe logic should indicate. (Real life is disappointing, what can I say...)
Needless to say, calibre does not fit those requirements.
And while Kovid may not feel bound to (or even have heard of) the PortableApps policy... it is a policy because it makes intuitive sense, and it probably makes intuitive sense to Kovid too.
...
If you are a regular use of portable apps, you will hit the same problem all over the place. It is not a situation unique to calibre.