I'm not aware of any plans to modularize the codebase, but the biggest size cost is really the bundled python, Qt5, and ICU libraries. The whole "calibre" python module is 23 MB out of 170 MB. (The resources data is an additional 40MB, nearly half of which is just localization strings.)
The "ebooks" module containing format reading code is actually the largest code component!
(A couple observations on the connectedness of things: the ebook conversion pipeline is a dependency of the GUI library, because that's a central GUI component, it is a dependency of the viewer, because the viewer reads a ton of formats due to internally converting them all into calibre's common denominator OEB representation, and it is joined at the hip to the news collator because news recipes are just a specialized ebook format source. The server component is an alternative interface to the GUI, so both depend on the database management. The viewer and editor could theoretically be divorced from the library, but all three share a lot of common utility code anyway that isn't worth disentangling. There's very little savings to be had.)
So basically the main difference for users (at least on Linux distros with good packaging) when calibre 4.x comes around, will be that the download is "roughly the same size", but it will depend on qt5-webengine instead of qt5-webkit, and in practice sigil already depended on webengine but now webkit can be uninstalled.
I have actually pondered the thought of one day trying to split out the format code into a standalone module as it's the most obviously useful to other projects. But I'm rather busy at the moment.
Someday...
...
BTW regarding python3, my secret was that I waited for someone else to ask, where "ask" involved submitting a giant Pull Request to make much of the python code syntax-compatible with python3 and get the C extensions to compile. I've then spent a ton of time building on that. So, much of the credit goes to @flaviut on Github... he took the first steps, I followed after and polished it up, taking advantage of the fact that I had a semi-viable codebase to actually try to run.
The moral of the story is more or less "in typical open source fashion, the best way to convince someone to change something is to write the code -- or at least a majority of it -- yourself". So I rarely if ever waste time asking Kovid to change how calibre is internally designed. Instead, I look at things which bother me, I ask "how can I fix this", and then I fix it and ask him if he agrees with my proposed solution. Kovid is a reasonable person and appreciates when people help make his job easier.
This is for example how I got calibre to build with system versions of mathjax, html2text, markdown, and feedparser, and how I got calibre ported *from* an internal fork of beautifulsoup3 to a system version of bog-standard bs4. (All these now use calibre's standard module freezing code instead of trying to merge bundled code into the source tree.) It's how I implemented a series of build system changes that mean I don't need to hack around DESTDIR installs (which every distro reinvents separately, usually incorrectly). It's also how I'm getting calibre slowly but surely ported to python3.
I will actually never understand why I am practically the only person who ever tried that, but it works. Instead, people try to "sway his opinion" by asking him to do a lot of work that doesn't help him, and, usually, by insulting him and (I'm mind boggled, but some people actually feel this way) telling him that the world is a worse place because calibre exists and prevents anyone else from being motivated to "do ebooks right".