Comparing e-books: Wow

[Divingduck]

Oh, sorry I was somewhere else and had'n seen your request
The Book as zip protected. You have a PM with PW

[arspr]

WOW. Just no more comments needed.

Nevertheless, after playing a little with this new feature (which really adds a lot of value to checkpoints which I hadn't previously used), and just as brainstorming ideas, I would like to post some possible "improvements" (unless proved absolutely pointless):

Currently we have a list of checkpoints which are just "virtual" copies (containers) of the book in different stages of modification (or not). And the last one (or the only one if no checkpoint has been explicitly created) is just an "unnamed" one.

So imagine you have "1" - "2" - "3" - "Unnamed" as your 4 checkpoints in the editing session. When you "revert" to "2", you are just changing the container you are working with. You just jump from "Unnamed" to "2".

The first problem / possible enhancement is that when you continue working in "2" you lose your original version of "2". And maybe you made that "2" checkpoint because of whatever critical/special/important reason, so you don't want to lose the snapshot that "2" actually is.

So I propose the next modification. When you jump to "2" what you would get would be: "1" - "2" - "unnamed" (copy of "2" and current container) - "3" - "unnamed". In this way you would never lose the original checkpoint state when reverting to it, even if you start editing.

The second enhancement. This proposed scheme has a serious trouble if you start jumping between checkpoints: a lot of "unnamed" ones are going to be created. So what we need is the next procedure: when I jump from one checkpoint which is not named to any other I must check if that origin checkpoint is actually different (or not) from its precedent (and parent) one. If it is the same, then just delete it.

So in the previous example, if I had just made "3" and then I have just jumped from the last "unnamed" one I would have ended with just "1" - "2" - "unnamed" (copy of 2 and current container) - "3". The last one would have been automatically deleted. If I jumped back to "3" without making any changes I would return to the starting situation (an "unnamed"-child would be created from "3" and the "2"'s child would be killed).

Third enhancement. Do not lose "branches". If you jump to "2" from the 4th "unnamed" and you start editing you are actually "branching" your work. "2" is no longer in the same "working path" of "3" and "unnamed" (unless you repeat the exact same edit actions...)

But curiously, if you create a new checkpoint from "2", that very same "2", and "3" and "unnamed" in addition, are deleted, as if trying cut off the branches...

Of course Calibre cannot be a full fledged version control system, (wait, maybe with Kovid it can, but that's another story... ), but nevertheless I think that with the powerful diff tool you offer there's no problem on keeping "pseudo-branches".

If possible I would do the following: inserting checkpoints in the queue.

Remember that I have proposed cloning "2" in order to keep it safe. So when you revert to "2" you would have "1" - "2" - "unnamed" (copy of 2 and current container) - "3" - "unnamed".

When I start working in the clone of "2" and then I create a new "2.1" checkpoint I propose having "1" - "2" - "2.1" - "unnamed" (copy of 2.1 and current container) - "3" - "unnamed", where "2.1" and its "unnamed" is actually a different branch than "3" and its "unnamed".

Fourth enhancement. A nearly must because of branches: timestamps and delete checkpoint action. As the queue of checkpoints can become really "messy" if there is a lot of branching, (although with the diff you can always compare them so it's messy in structure but not in browsing their contents), I really think that offering a timestamp in the checkpoint list viewer can be really helpful as it can guide the user about when and how they have been created.

And of course a "delete" checkpoint action would be really necessary...



What do you think about these ideas?

I was considering even trying to code them but maybe they are just too much for me...

[mrmikel]

Seems like it can be solved by saving a copy at any of these junctures instead of cluttering up the hard drive with 25 different copies. You will have to use the new function to tell apart since they will nearly identical, the older of which having errors which may not be readily apparent.

[kovidgoyal]

@arspr: I really dont want to turn checkpoints into a version control system. If you wish to jump back to a checkpoint and edit while still preserving that checkpoint, simply create a new named checkpoint before you start editing.

[kovidgoyal]

Thanks for the books, as I suspected the error happens when beautifying removes all existing differences between a pair of files in the books.

[arspr]

Probably you are fully right. Simpler is better.

But then I think the third issue should be better handled. I mean if you jump backwards and you start editing, you currently have a version control system running with two different branches. Until you create a new checkpoint, the "obsolete" branch is kept saved.

If possible (and if it has no other undesired side effect I haven't thought of) I think you should automatically remove that "obsolete" branch if any modification, or just any typing, occurs.

[kovidgoyal]

@arspr: yes, you are right in that the trailing checkpoints beyond the current one should be removed when you edit. They are currently removed if you use any of the automated tools, but it should also happen when you edit manually. That will be in the next release.

EDIT: However, when I tried to actually implement this, I found that there is no robust way to implement it. The contents of an editor can change because of many different user actions, it is not possible to keep track of all of them let alone decide which ones actually constitute a change or not. SO I'm afraid the trailing checkpoints will have to remain.

[arspr]

Entering brainstorming mode, outside team, non-expert, ignorant role:

Just in case you were so focused in solving the issue in an orthodox, "correct" way that you didn't think of this possible way. Although it has a probability of 99% of being just plain nonsense.

About killing the trailing obsolete branch of checkpoints: When you perform any automated action you have already set an add-checkpoint action which actually removes that branch. So AFAIK you only need to catch the manual "typing" event, because any automated one is already solved.

And thinking about that, it looks pretty similar to me to whatever mechanism you have implemented to detect the file-has-been-modified event. I mean when you open the book you somehow detect when the user modifies any text file, and then you put the pencil icon on its tab and you activate the save button.

I know it's not exactly the same case but, couldn't you "port" that detecting tool to the checkpoint cleaning?

[eschwartz]

Originally Posted by arspr

Entering brainstorming mode, outside team, non-expert, ignorant role:

Just in case you were so focused in solving the issue in an orthodox, "correct" way that you didn't think of this possible way. Although it has a probability of 99% of being just plain nonsense.

About killing the trailing obsolete branch of checkpoints: When you perform any automated action you have already set an add-checkpoint action which actually removes that branch. So AFAIK you only need to catch the manual "typing" event, because any automated one is already solved.

And thinking about that, it looks pretty similar to me to whatever mechanism you have implemented to detect the file-has-been-modified event. I mean when you open the book you somehow detect when the user modifies any text file, and then you put the pencil icon on its tab and you activate the save button.

I know it's not exactly the same case but, couldn't you "port" that detecting tool to the checkpoint cleaning?

I believe that's what he tried. The changelog for removal of this is:

         if is_modified:
             self.set_modified()
-            self.global_undo.truncate()
-            self.update_global_history_actions()

[arspr]

Originally Posted by eschwartz

I believe that's what he tried. The changelog for removal of this is:

         if is_modified:
             self.set_modified()
-            self.global_undo.truncate()
-            self.update_global_history_actions()

As I said, 99% probability of being a complete nonsense...

[Divingduck]

Kovid,
short feedback. The error form post 13 is fixed with your update form yesterday. Thanks a lot.

[Feldegast]

Originally Posted by kovidgoyal

FYI, you can also run the tool from the command line without need to go through edit book first, like this

calibre-debug.exe --diff file1.epub file2.epub

this didn't work in calibre 1.40, i tried it on multiple files using full paths

calibre-debug --diff "/home/user/Calibre\ Library/Author\ Name/An\ ebook\ (495)/An\ ebook.epub" "/home/user/Calibre\ Library/Author\ Name/An\ ebook\ (260)/An\ ebook.epub"

it shows the compare window with no text in each side and the following error at the commandline

Traceback (most recent call last):
File "site-packages/calibre/gui2/tweak_book/diff/main.py", line 458, in <lambda>
File "site-packages/calibre/gui2/tweak_book/diff/main.py", line 364, in file_diff
File "site-packages/calibre/gui2/tweak_book/diff/main.py", line 124, in file_diff
File "site-packages/calibre/gui2/tweak_book/diff/main.py", line 99, in get_decoded_raw
IOError: [Errno 2] No such file or directory: 'calibre-diff'

PS
would it be possible to run the compare function from the library directly... instead of having to open 1 ebook 1st? eg select 2 files from library (function disabled if more than or less than 2 books selected), right click, select compare (greyed out if more or less than 2 books selected) from menu and then the compare window loads the 2 selected files

[kovidgoyal]

If you are using quotes then you dont use back slashes.

And I dont see any point in diffing two entirely unrelated books. You can run the diff from the GUI to compare EPUB to ORIGINAL_EPUB in a single book record by right clicking on ORIGIGNAL_EPUB in the book details panel.

[Feldegast]

i tried again without quotes and got the same issue

then i tried copying the 2 files to the same directory and ran....
calibre-debug --diff 2.epub 1.epub
still got this error

Traceback (most recent call last):
File "site-packages/calibre/gui2/tweak_book/diff/main.py", line 458, in <lambda>
File "site-packages/calibre/gui2/tweak_book/diff/main.py", line 364, in file_diff
File "site-packages/calibre/gui2/tweak_book/diff/main.py", line 124, in file_diff
File "site-packages/calibre/gui2/tweak_book/diff/main.py", line 99, in get_decoded_raw
IOError: [Errno 2] No such file or directory: 'calibre-diff'



the purpose for providing the right click menu feature in the library is for those people who import book collections and end up with 5 or 6 copies of the same book where each copy is slightly different
being able to quickly compare 2 books to decide which to keep would save a lot of time

i would use the debug method but it looks like it is broken and for comparing lots of books less conveniant

[kovidgoyal]

A diff tool is useful for seeing the changes in the code of a book. If you want to compare two books to see which one to keep, open them in the viewer.