[Device Plugin] Kobo Touch Extended Driver

[stumped]

so calibre today is offering me 2 plug in updates. should I NOT accept them , given the above ?

[jxall]

Originally Posted by DNSB

Are they there in the .epub prior to sending to your ereader as a kepub?

In my case KTE is set to auto-convert epubs to kepubs as usual.

I have also enabled the option to keep a copy of kepubs in my Calibre library, and as I compare the latest epub and the kepubs from the Calibre viewer I confirm that the epub is flawless and the corresponding kepub has the very same spacing bugs shown on the Kobo device.

[gal33]

Originally Posted by paulfiera

Same here. I have the same book in a Kobo Aura ONE which I imported from Calibre last week. The italics have a space after.

The same book imported from Calibre yesterday to a Libra H20 after updating 3 Kobo plugins have no space after the italics.

Opening the book in Sigil, there is a space after the italics.

Exactly the same for my Kobo Forma.

[PeterT]

Not sure whether it's best for people to report that problem here or to open an issue over at https://github.com/jgoguen/calibre-kobo-driver

[davidfor]

Originally Posted by PeterT

Not sure whether it's best for people to report that problem here or to open an issue over at https://github.com/jgoguen/calibre-kobo-driver

I have a fix and have created a pull request. I'm not doing a beta as it doesn't actually break anything, it's just a nuisance.

[icallaci]

Originally Posted by davidfor

I have a fix and have created a pull request. I'm not doing a beta as it doesn't actually break anything, it's just a nuisance.

Thank you, davidfor.

ETA: Fixed with latest version. Thanks.

[jgoguen]

Version 3.2.11 posted with davidfor's fix.

[Strether]

Originally Posted by davidfor

I have a fix and have created a pull request. I'm not doing a beta as it doesn't actually break anything, it's just a nuisance.

Thank you, David. That appears to have solved the problems I noted. Appreciate the quick work.

Jim

[paulfiera]

davidfor, jgoguen, thanks a lot you both for the quick fix

[jgoguen]

Originally Posted by madhatta

Nevertheless, an unwillingness to support your plugin in the context of calibre as it is used by many people

It's quite unfortunate that you're choosing to interpret putting my family, job, daily needs, and health ahead of this plugin as "an unwillingness to support" it. I'm one person, there are limited hours in a day, and there are some things which must get done before optional things like this plugin can be worked on.


Not only am I one person, I'm one unpaid person. I highly doubt you, or even all the users of all my plugins combined, get enough value out of it to pay me enough to quit my job and dedicate that time to calibre plugin development instead. I'm happy to support what I can, but I will not tolerate someone expecting a level of support that's beyond even most gold-level paid support contracts without first being paid at or above that level. And whether or not you actually expect that, or whether or not you intended to come across that way, that's how I read your comments.


The decision to only support Python 3 once calibre has official Python 3 support is similar. I'm one unpaid person with not a lot of spare time, and supporting Python 2 is a disproportionate amount of extra work. By cutting out Python 2 as early as I can, I'm going to save myself a significant amount of time (relative to how much I have to spend on these plugins).

Originally Posted by madhatta

will simply mean that less people can use your plugin, which is a shame.

It would be, but not for the reason you seem to think. I write open source software for myself. It's nice that other people find value in it, and they're definitely significantly better because of all the people using them and reporting bugs they find, but I wrote this for myself and put it up not knowing if anyone would want to use it. I appreciate contributions from people, I appreciate this community that has built up around it (however big or small that may be, I still don't know how many people have installed my plugins), but if every single person stops using this plugin tomorrow it's not changing how much I can/will work on it. User counts aren't a motivating factor for me.

Originally Posted by madhatta

That doesn't place any obligation on you: as I say, you're doing this out of the goodness of your heart, and how you do it is entirely up to you.

You've gone from "I appreciate this plugin and you don't have to do anything" to "you're unwilling to support what I want and I'm complaining about that" to "I appreciate what you're doing and you don't have to do anything", and pretty quickly you go back to "it's not reasonable to expect people to run calibre the way it's supposed to be run". Please pick one: you appreciate what I'm doing and I can choose how to develop and support the plugins and you're respectfully disagreeing with my decision (which is not at all how this reads), or you're expecting me to support calibre plugins in an environment where calibre itself isn't guaranteed to work properly.

Originally Posted by madhatta

I'm just letting you know that when you make people choose between running your (excellent) plugin, and running the distro version of calibre, some will for very good reasons choose to stick with the distro version of calibre.

Each person needs to decide for themselves what tradeoffs they want to make. Some people will decide the plugin is more important and use the official calibre release, others will decide they only want to use what comes in the official distro repos and stop using the plugin. The calculus will be different for each person, and different people may arrive at the same decision for different reason, but there isn't a single right choice here. In my opinion the better choice is to run the official calibre release, but that was my opinion before I wrote this plugin too and I will not say anyone who chooses differently is making a wrong choice.

Originally Posted by madhatta

Surprised? No; it's free software, I expect it to break. But apparently the developers didn't think that the desire to have it work on my system was an unreasonable one.

I suspect you mean something other than what you said? It seems to me you meant to say something more like "the developers didn't think that the desire to have it work on my system was a reasonable one"?


The developer (remember, just me) doesn't think that the desire to have it work on your system was an unreasonable one. The desire to have things work properly in your environment is reasonable. I do think that what appears to be your expectation that I will make it work in your environment is unreasonable.

Originally Posted by madhatta

I'm sorry if it came across as insulting. It wasn't supposed to be, and I'd tried hard at my end to bend over backwards to make it clear how grateful I am for the existence of the plugin at all.

What made it worse for me is that the flattery combined with what came across as an expectation of doing it your way made the flattery seem like a thinly-veiled attempt to flatter me into changing my opinion. Just because you're grateful, or because you express your gratitude, doesn't mean you can't also come across (not saying you are, just that it's how it comes across to me) as being entitled or inconsiderate of other people's circumstances.

Originally Posted by madhatta

But the model "get your OS from the OS people and your apps from the app people" is a Windows model of computer use, not a Linux model.

Just because Microsoft does something, doesn't mean it's automatically bad. A model is the typical way of doing things, not necessarily the best way or the only right way. Sometimes something comes up which is both valid and outside the model.


It's similar to choosing the best tool for the job. I write documents in Markdown normally, or LaTeX if they're professional/academic documents. When I need to use office formats I prefer to use LibreOffice. But sometimes the best tool for the job means logging in to my Microsoft account or booting up Windows and opening Word or Excel. Or, as much as I don't like to administer Windows systems, if you need to do enterprise identity management for a larger company or you need to manage Windows computers there still isn't a viable alternative to Active Directory.

Originally Posted by madhatta

Most people who use Linux stick fairly rigidly to the software that's distributed under the repository+indices+dependencies model because we know how many things start to break, and how balkanised and unsupportable our systems get, if we don't.

This may well be your experience, but I've never run into this problem. My personal computer, my work computer, and the Linux systems I administer have combinations of distro repo packages, third-party packages, and commercial software with installers of varying quality. It's unfortunate that this has been your experience, and my experience since I started using any kind of UNIX has been the opposite, so I can't easily relate to it.

Originally Posted by madhatta

This doesn't mean "only software distributed by the OS people",

True, because anyone can create a repo and you can install any repo on your systems. But, third-party repos pose a problem where using them contradicts what I understand your concerns about using the official calibre release to be.

Originally Posted by madhatta

but it does mean "only software that's been built specifically for the environment provided by the version of the distro I'm choosing to run".

A repo does not at all imply anything at all about how or where the software was built or tested. Let's say, just as an example, you're running Arch Linux. I can create an Arch repo and I can publish any package to it. As long as I create the right metadata for pacman to read it, any Arch user can add my repo as a system repo and install packages from it.


But people who add my repo have no guarantees that I built on Arch. Maybe I actually run OpenBSD and I built the binaries in a chroot with Arch packages unpacked in it, or I run Fedora and used mock to build the Arch binaries. Similarly there's no guarantee I tested on Arch, binaries I build on my Fedora box will likely run fine on an Arch system but (especially with a rolling release model) it's possible there could be breakages not apparent on Fedora. I can publish the Arch package without actually ever having an Arch system available to me.

Originally Posted by madhatta

If this computer had no job other than to run Calibre, I'd follow your advice in a heartbeat. But it does many hundreds of important things for me other than that, most of which are in fact more important than running Calibre, and I can't endanger them all by starting to install bits of software that are not intended specifically to work with it.

If you use third-party repos run by people you wouldn't trust with root access to your system, I will be skeptical of any claims to concerns about using the official calibre release related to the safety or stability of your computer or what runs on it.


Let's say now you run Fedora and you install my DNF repo. Nothing at all stops me from publishing a package which shadows a package from the upstream repos but includes maliciously modified binaries, or which runs a post-install script that makes malicious changes which now aren't tracked by RPM. When you install a package you run the pre- and post-install scripts as root, allow it to put files anywhere on the system (as long as it's not conflicting with a path owned by another package) owned by any UID (though if that UID isn't in the passwd database you should see a warning about it) with any permission bits, and that package may install anything to any path it wants. I could publish a package with software, then publish an update that includes a post-install script that sets capabilities and adds setuid to some binaries, executes and disowns it, and even overwrites core binaries with malicious copies (RPM only prevents the packaged files from conflicting).


Calibre, on the other hand, can be installed manually or automatically without needing any root privileges. You could use chroots or systemd-nspawn to isolate it from the system to further minimize any potential damage if Kovid ever decides he's going to start removing ~/.local/ everywhere. The risk of calibre is far lower than a third-party repository.



Just like deciding whether to use the official or distro calibre package, each person has to decide for themselves whether to trust any particular third-party repository, and the decision may well be different in different environments (I have third-party repos on my personal laptop I wouldn't put on my work laptop). But if someone is willing to install a third-party repo from someone they wouldn't give root access to and also concerned about the security and stability effects the official calibre package may have on their system, I would strongly encourage them to re-evaluate and make sure they understand what it is each one of those things can do.

Originally Posted by madhatta

I repeat, yet again, that it's absolutely fine if you don't want to support that software model. Your plugin is a marvellous thing, which makes the world a better place. I just wanted to try to shed a little light on why you might get pushback from some Linux users when you say "just upgrade Calibre". It's not because we feel entitled, or wish to insult you, or prize you at nought; it's because of decades of experience that suggests that following that software maintenance model doesn't work well for us.

From the very first day I published this plugin I've said that any bugs must be reproduced on the latest plugin version running on the latest official calibre release. The first post of this thread says that if you report a bug and aren't using the latest version of calibre and the latest version of this plugin you will likely be asked to upgrade. Anyone who doesn't like this model, whether for reasons similar to yours or not, has reasonably had plenty of warning before choosing to install my plugins that my requirements conflict with how they want to administer their systems. My choice to focus my time and energy is valid, so is your and their choice to not necessarily upgrade to the latest version of everything right away. But two valid choices can be mutually exclusive.

Originally Posted by madhatta

And I can only repeat the observation that at least one other Calibre plugin community doesn't feel that way.

That's their choice and I'm happy for them that they have the time and energy to do this. Our circumstances are different, just because one plugin community can do something doesn't mean any other plugin community can.

Originally Posted by madhatta

That doesn't mean you have to feel any particular way at all!

Your previous sentence strongly implies the opposite.

Originally Posted by madhatta

It just means it's unwise to confuse "I choose to do otherwise because it's a better use of my limited and valuable time" with "it's impossible".

Both can be true. I choose to support only the latest plugin and calibre, and I will choose to support only Python 3 as early as I reasonably can, because that's a better use of my limited time. With the obligations I have, the things that simply must happen each day, and my personal circumstances, it's not possible for me to do more than that without negatively impacting the time for things that must happen.

[jxall]

Originally Posted by jgoguen

Version 3.2.11 posted with davidfor's fix.

After updating to this build the very same bug that I experienced with v3.2.10 and Calibre 4.18 is back: newly-synced books can't be found anywhere on the device. Only that this time I am using v3.2.11 with Calibre 4.19.

The processing of the books (that occurs when you eject and unplug the device from the PC) is suspiciously fast (as if just one book has been synced instead of a dozen) and none of the new books is either searchable or listed within the My Books section.

On a sidenote, I can confirm that this build has corrected the spacing issue that I was experiencing with v3.2.10 and Calibre 4.19 (kepubs are now displayed correctly at least on Calibre Viewer - of course I can't speak for the Kobo Clara as I am unable to find the books).

(Many many thanks @jgoguen for your generosity in sharing the fruits of your hard and precious work with us!)

[JSWolf]

Originally Posted by madhatta

But the model "get your OS from the OS people and your apps from the app people" is a Windows model of computer use, not a Linux model. Most people who use Linux stick fairly rigidly to the software that's distributed under the repository+indices+dependencies model because we know how many things start to break, and how balkanised and unsupportable our systems get, if we don't. This doesn't mean "only software distributed by the OS people", but it does mean "only software that's been built specifically for the environment provided by the version of the distro I'm choosing to run".

As you are pointing out, the Windows modem works and the Linux model does not. If you got Calibre from the official Calibre website, you would not be complaining that a plugin doesn't work. The Linux repository is a really bad idea. You get old versions, some version of the Python 3 beta, versions that don't work because they were botched by the repository maintainers. So really, give it up. You are fighting a losing batter. You know that your repository botched Calibre so move on and get the current version. So it's YOUR FAULT that the Kobo Touch Extended Driver does not work on your system. You insist on use a version that you know isn't going to work the way you want Calibre to work.

Give up, this is one fight you will never win with the attitude you have.

[davidfor]

Originally Posted by jxall

After updating to this build the very same bug that I experienced with v3.2.10 and Calibre 4.18 is back: newly-synced books can't be found anywhere on the device. Only that this time I am using v3.2.11 with Calibre 4.19.

The processing of the books (that occurs when you eject and unplug the device from the PC) is suspiciously fast (as if just one book has been synced instead of a dozen) and none of the new books is either searchable or listed within the My Books section.

On a sidenote, I can confirm that this build has corrected the spacing issue that I was experiencing with v3.2.10 and Calibre 4.19 (kepubs are now displayed correctly at least on Calibre Viewer - of course I can't speak for the Kobo Clara as I am unable to find the books).

Are the books definitely on the device? Can you see them on the device before you eject? Are you sending new books, or replacing exiting copies?

The quick import could be because the first book failed to be processed correctly. That blocks the rest from being imported. And, if this was happening before, it might be a leftover from a previously sideloaded batch. You will probably need to remove these books, work out which are triggering the problem, fix it and send them again. Unfortunately, the only way I know to do this, is send the books in small batches until you find the problem book.

[jgoguen]

Once you do find what book is causing the problem, can you use the Scrambled ePub plugin to create a scrambled version of the book, verify the scrambled version also fails, and attach it? It would be interesting to see if perhaps there's some odd thing happening. Also attach a full debug log of sending the book to the Kobo device, in case anything is happening during the conversion process.

[jgoguen]

Originally Posted by paulfiera

BTW, are all those spans absolutely necessary? I can not find any .koboSpan declaration in the CSS. Could the HTML be trimmed?

I don't know if the class attribute is needed (it's present in KePubs published by Kobo) but those spans and their id attributes are what Kobo uses to save your reading position (and probably anything else involving locations inside books).