Looking to get started helping out with Calibre dev

[Jeremy_D]

Hi all,

TL; DR: I'm looking for suggestions on where my efforts could best be directed to help out, particularly with the py3 port. Also looking for further resources to assist me to get up to speed ASAP.

I've searched the forums here and also had a brief look at existing dev page. I found a few forum threads/posts that seemed relevant and suggest working on py3 bug testing and plugin dev/p3-porting, but ideally I'd like some more specific pointers on where is best to get started. Also clarity on where I'd find any outstanding py3 related bugs (searching the LaunchPad bug-tracker for "python3" and "py3" came up blank...).

My current skills, strengths and limitations/shortcomings

• Somewhat novice python programmer; mostly experienced with python3; including porting (CLI/headless) py2 -> py3 scripts/apps.
• No experience with GUI app dev; extremely limited experience with webUI dev.
• Debian "daily driver", particularly experienced with headless/CLI admin.
• Experienced with git & GitHub.
• Aversion to LaunchPad (I know it's what Calibre uses for bug-tracking and I promise I won't whinge - too much... )
• Keen to learn more/better python.
• Committed to open source.
• Only have access to very limited eReader hardware (i.e. don't imagine I'll be able help much with debugging hardware related issues).

Elaboration/context

Ultimately, I have some wishlist items I'd love to see implemented in the future, but I understand that there is lots of other stuff that is much higher priority (and likely required), particularly getting the py3 port completed and 100% reliable.

Also, before I could even begin to think about my personal desires, I need to get up to speed with the Calibre codebase, hone my python skills a bit and get a better understanding of the coding requirements/desires of the existing devs. So starting small seems like the way to go. Judging from other posts I've read, bug-hunting py3 builds and/or porting plugins to py3 are probably the best place to start?! Is that correct and if so, are there good places to find what specifically is best to work on? E.g. plugins that still need porting, py3 related bugs, etc.

Also is there a place (i.e. somewhere a bit more targeted that these forums) where developers coordinate on what everybody is working on? Ideally I don't want to tread on any body else's toes and would like to maximise the value of my contribution (so don't want to double up on stuff that is already in progress).

FYI - My longer term wishlist

My wishlist items are related to the server side of things. Ultimately, I'd like to see support for multiple GUI clients connecting to a single minimalist headless server backend. As noted above, headless Debian/Linux servers is my thing!

• (optional) MySQL (&/or similar) DB backend support.
• ability to run a simplified standalone headless server (without all the GUI deps) - ideally with webUI.

I understand that these things have been requested multiple times all over the internet and are unrealistic in the short term, but ultimately, that's what I'd love to see (and assist push forward as/when the time is right).

Thanks for reading (if you got this far... )

[kovidgoyal]

There are no known outstanding py3 issues in calibre itself. If you want to help with the py3 port, the place to help is with the plugins. See here https://plugins.calibre-ebook.com/stats.html to identify popular plugins and here to identify ones that still need porting: https://www.mobileread.com/forums/sh...d.php?t=326405

Identify one you would like to port from those two links, then check here for the individual plugins' thread: https://www.mobileread.com/forums/sh...d.php?t=118764

make a post there saying you are porting that plugin and when you are done porting it, post the updated version as a beta for the plugin's users to test.

As for co-ordination with other calibre developers I am pretty much the only developer that contributes to core calibre code in any significant way, so you really need to co-ordinate only with me. If you dont like using the forum you will find my email address at the top of the various source code files in calibre.

[Jeremy_D]

Hi Kovid,

Awesome, thanks for your quick and thorough response.

Ok, I'll have a look at the plugins as you noted and follow your directions re posting on the individual plugin's thread. I assume that I should be doing the initial test of the port against the tip of the py3 branch?

Re further dev work, I really appreciate your offer of direct email communication and I may take you up on that at some point. Although as an open source developer myself, I do lean toward having discussions openly as much as possible as a general rule (for your and the project's benefit).

Do you have your code styling preferences documented anywhere? Do you use some sort of linting? E.g. pycodestyle or similar?

Should plugins be aiming to adhere to those requirements, or is it up to individual plugin devs?

Re my eventual hope of assisting with support for MySQL (and/or whatever SQL engine) as a DB backend, do you have any specific thoughts? I recall reading somewhere that you already had hopes to do something like that? If so have you made any decisions or tentative plans?

It'll be a while before I'll be in a position to look at that too much, and it's possibly wise to wait until the py3 code is rock solid before thinking about it too much (although perhaps it already is?). But I've done a little preliminary research and from where I sit, the way to go would be to implement some abstraction layer (e.g. an ORM) so that any code changes would only have to happen once and after that, it could still default to SQLite, but other SQL DBs would then also be an option; without any further code refactoring. But there are quite a few different options and I'd be interested to hear your thoughts (especially as the primary dev and especially if you've already got ideas).

Regardless, here's some ORMs I found that might be of interest:

• SQLAlchemy - seems to be the most popular; opinions suggest that it's a bit complex initially, but pays off in future flexibility.
• Storm - Also seems to be relatively popular, although not a lot of dev activity. It has been recently been ported to py3 recently (see here & here).
• Pony ORM - A newer python ORM which seems to be getting a bit of traction.
• Django's ORM - Also seems to get a lot of recommendations, apparently can be used by 3rd party apps fairly easily - I also found a nice (albeit a little dated) gist of notes.
• web2py's ORM - Also gets a few recommendation, as per Django ORM.

Although I have no experience with any of them, on face value my inclination would be to go with the herd and use SQLAlchemy.

[kovidgoyal]

definitely stick to the forums if you prefer those, email is for if you dont. No I am not fussy about code styling if you are contributing to existing code, try to more or less match it, if you are contributing in a new file use whatever floats your boat since you will be maintaining it. If I take over maintenance, I might reformat, but i really dont care that much. there are some basic pep8 compliance checks via flake8 but I take care of those at merge time anyway.

plugins are definitely up to their maintainers.

there would be o point using mysql or whatever. The difficulties to making calibre multi-user beyond what it already is are not in the database layer. You already have fully capable multiple gui clients connecting to the server. The guis are just web uis, not native guis. And the server runs perfectly fine headless without any gui dependencies.

[kovidgoyal]

And yes the tip of py3 is recommended if you are testing.

[Jeremy_D]

Ok great, thanks.

Originally Posted by kovidgoyal

there would be o point using mysql or whatever. The difficulties to making calibre multi-user beyond what it already is are not in the database layer. You already have fully capable multiple gui clients connecting to the server. The guis are just web uis, not native guis. And the server runs perfectly fine headless without any gui dependencies.

Hmm, ok. I'll have a closer look then. I must have been doing something wrong? Or perhaps just testing a version that was too old?

I'll focus on plugins for now then and we can discuss other stuff later.

Thanks again.