Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 04-30-2011, 06:48 AM   #121
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,223
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
I will definitely add a check for series duplicates, that is an excellent idea. I think that will pickup one of the main frustrations people have where two series with the same number are actually completely different (and you get into numbering issues etc). Ideally you want your series names to be unique.

Missing books in a series? I'm not convinced. It isn't going to be able to tell you that you are missing #3 in a trilogy that you have #1 and #2 for. It will also return false positives in the scenarios where say you have an omnibus edition that covers books 1 & 2, so a supposed "gap" isn't really a gap. Then there is the whole issue of numbering of series indexes for magazines, people who always use partials etc. So I'm not going to include it at this point.

Having a mixture of full and partial indexes is not an "error" for many users. I like a lot of people intermix related short stories for a series by putting them in with a "x.1" index, with the main full books having the normal 1,2,3 index values. So I'm also not convinced on that one.

As for magazine related ones, they fall into the same category as "Missing parts" to me. These are not issues with the quality of the metadata (or conversions) of books in your library, which is what this plugin is aimed at. A lot of these are fairly bespoke to how you are storing and using the series index field. Maybe I can be convinced further down the track to add such checks but for now it is out of scope.

As for title case information, the algorithm is aimed at English names. No there is no customisation capability. It is all based on some open source code that is part of the Calibre library. So I would suggest not using the check (or the Calibre title case function) if it does not give you the results you would like. If you want it enhanced, as it is a Calibre function you should go through the normal feature request for Calibre route.
kiwidude is offline   Reply With Quote
Old 04-30-2011, 07:19 AM   #122
drMerry
Addict
drMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmos
 
drMerry's Avatar
 
Posts: 293
Karma: 21022
Join Date: Mar 2011
Location: NL
Device: Sony PRS-650
Thank for telling reasons to in- or exclude functions.

I can agree with most of them.
But on the missing books I think there is a point that you did not mention.
Of-course there is the false positive problem. But the Missing-part-of-series is not a function you would use daily. But when it IS available, it would be a good tool to find books you still would like to have. (for example if you collect a specific series).

You will not see books that are missing with a higher index of the last book you have, but this is the way it has to be. (You also do not fetch isbn-numbers on picture-only pdf files although they are inside the book).
A work around for collectors knowing there are more books would be to add an empty book with the last series-index.

Another option would be: Series with 1 book. Some Ebooks set the series option (often with name of author). Than you have a series with 1 book (#1) that is not a real series. Would be nice to filter this books also (this kind of series do not have set the series-index so the index is set to the default calibre option)
drMerry is offline   Reply With Quote
Old 04-30-2011, 08:03 AM   #123
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,223
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
Of your original suggestions other than duplicate series, the missing books in a sequence is one that I might eventually consider, though it has two issues I can see. The first is identifying the series increment to know a book is missing. For some people/series they use 0.01, 0.02, 0.03 etc. For "normal" series you have 1,2,3. For magazines you have said you use 2011.01, 2011.02 etc. So how can you differentiate those from someone who has a series of 1, 1.1, 1.2, 2, 3, 5? The second issue is to do with visual presentation of the results but you could kludge something together for that if you can solve the first issue.

As for books with one series, yes that could be useful for people with badly messed metadata I guess. Of course there are genuine reasons for having a book with only one in the series, such as when an author starts a new trilogy or you only own one book in the series and haven't created wishlist items for the rest. However like the "Check title with series" option it is something that just brings the results back and it is up to you to look through and identify genuine series issues versus false positives.

Speaking of which, I suggested on the find duplicates thread an idea I had for the "Check title with series" function which would apply to a Series with 1 book check and a number of other quality check functions. That is the ability to specify an exclusion for a book from a particular check.

For instance I modify author sort on some books to remove co-authors so that both sorting and exporting to my Kindle works "properly" to keep my series together. Those books are repeatedly flagged as having author sort problems, even though I know I am happy with them being so. If I could set an exclusion on those books for author sort then the next time I run that check they will not appear. Likewise for other checks as well. I think this could be fairly useful but is a fair amount of work and would need some menu restructuring, extra dialogs etc. You need to be able to review your exclusions, remove them and so on. It's something i will keep in mind for the future.
kiwidude is offline   Reply With Quote
Old 04-30-2011, 12:41 PM   #124
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 14,129
Karma: 5468860
Join Date: Aug 2009
Location: The (original) Silicon Valley, USA
Device: Galaxy Tab 2, Astak Pocket Pro, K4NT
Quote:
Originally Posted by kiwidude View Post
Of your original suggestions other than duplicate series, the missing books in a sequence is one that I might eventually consider, though it has two issues I can see. The first is identifying the series increment to know a book is missing. For some people/series they use 0.01, 0.02, 0.03 etc. For "normal" series you have 1,2,3. For magazines you have said you use 2011.01, 2011.02 etc. So how can you differentiate those from someone who has a series of 1, 1.1, 1.2, 2, 3, 5? The second issue is to do with visual presentation of the results but you could kludge something together for that if you can solve the first issue.
You might have a Guess function (if ticked)
Look at all the indexes for a given series

All integers = basic numbering (default)
4 digit integers with decimals (cant be 2 digit only, 2010.1 is October) is issue date
What do you do with big gaps?


The scheme I use makes it difficult

123 is an omnibus of vols 1, 2, 3
4 is a normal index
4.n is something(s) that slots in between the volume and next. Shorts, fanfic
0.n has no exact placement/unknown (n for ordering of the un-ordered )
yyyy.ii year.issue (Analog Double (2 month) Issues ruin this)
theducks is offline   Reply With Quote
Old 04-30-2011, 12:48 PM   #125
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,223
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
@theducks - yes, exactly that sort of wide ranging series index usage is why I've got a big question mark over the practicality of implementation. Particularly where you have fractionals, there is just so much variability in how people have chosen to use the index to "bring together" stuff. If you limited it to just (for instance) looking at the integer portion of the number then that might be workable, but of questionable value given all the cases it "wouldn't" match.
kiwidude is offline   Reply With Quote
Old 04-30-2011, 01:11 PM   #126
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 14,129
Karma: 5468860
Join Date: Aug 2009
Location: The (original) Silicon Valley, USA
Device: Galaxy Tab 2, Astak Pocket Pro, K4NT
Quote:
Originally Posted by kiwidude View Post
@theducks - yes, exactly that sort of wide ranging series index usage is why I've got a big question mark over the practicality of implementation. Particularly where you have fractionals, there is just so much variability in how people have chosen to use the index to "bring together" stuff. If you limited it to just (for instance) looking at the integer portion of the number then that might be workable, but of questionable value given all the cases it "wouldn't" match.
Can a PI be made to only function with/when a specific filter is in place? Series:series name(s)

then
radio buttons for the test pattern (avoids pattern errors by users).
spin-dial 'Ignore gaps bigger than"
tick: 'Ignore fractional portion'
theducks is offline   Reply With Quote
Old 04-30-2011, 01:35 PM   #127
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,223
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
The plugin already respects search restrictions. But it just sounds horrifically complicated, requiring intimate knowledge of the particular series you are examinging and how they are numbered. It would be orders of magnitude simpler just to visually scan down your books sorted by the series column
kiwidude is offline   Reply With Quote
Old 04-30-2011, 03:54 PM   #128
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 14,129
Karma: 5468860
Join Date: Aug 2009
Location: The (original) Silicon Valley, USA
Device: Galaxy Tab 2, Astak Pocket Pro, K4NT
Quote:
Originally Posted by kiwidude View Post
The plugin already respects search restrictions. But it just sounds horrifically complicated, requiring intimate knowledge of the particular series you are examinging and how they are numbered. It would be orders of magnitude simpler just to visually scan down your books sorted by the series column
Been my plan all along. It has been working for me.
It was the other users that felt the need.

Now if you really wanted a Project
Find the missing Titles and make an 'Empty Book' record to hole their place
theducks is offline   Reply With Quote
Old 04-30-2011, 06:01 PM   #129
drMerry
Addict
drMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmos
 
drMerry's Avatar
 
Posts: 293
Karma: 21022
Join Date: Mar 2011
Location: NL
Device: Sony PRS-650


The others == drMerry?

Well, there are problems, I knew it when I started this question.
But main problem is the false-positive.
This is why the quality-check is a 'helping' function, not a 'dictator'.
Just like the title in series it will show some info.
I think it is a lot of programming, so only start if you're motivated

I think you have to have one thing clear: It is an option presented to the user. The user is not forced to use it to have the benefit of the other functions.
So I can imagine theducks would not use this function because he has to much 'errors' while he uses the index in an other way than the 'normal index numbering' (what is normal...)

But think of the duplicate finder. If I would ask you at this moment to give me a plugin that could do all functions duplicate finder 1.0.2 does, you would say it would be a hard thing to program.
The way you solved that was for a great part by adding separate functions and let the user combine them. (with or without pre filtering the books to test)
I think that is the way you could work at this moment to.

To make it 'simple' you could start with a one-book option. Just select one book (or in left hand one series) en process this.
Give the user an option-screen to select some details (trial and error, only main-indexing, 2digit sub-indexing, 1digit sub-indexing)
That would give a result for the selected series. You could also add an option like last index: #, equals last index

Just some suggestions, but like I said. It may be a long walk so only start when motivated
drMerry is offline   Reply With Quote
Old 05-01-2011, 10:59 PM   #130
DoctorOhh
US Navy, Retired
DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.
 
DoctorOhh's Avatar
 
Posts: 8,743
Karma: 12488297
Join Date: Feb 2009
Location: North Carolina
Device: Nexus 7
I have been editing my ePubs in Sigil and using the Tweak ePub feature in calibre to remove the margins. I would verify that the margins went away in Calibre's viewer. This worked fine until yesterday when I realized my reader uses ADE to render its ePubs and ADE recognizes and uses the page-template.xpgt file for margins. This means I have a bunch of books that looked margin free in Calibre's viewer but really have a margin in them.

As of Calibre 0.7.53 calibre removes the Adobe page template margins during conversion. My problem is identifying the ePubs that have a page-template.xpgt file.

Can you add identifying which books have a page-template.xpgt file to the QC plugin? Once I have these identified I can reconvert the books with the same settings and have Calibre remove the Adobe page template margins.

Thanks for all your work and considering this request.
DoctorOhh is offline   Reply With Quote
Old 05-02-2011, 05:43 AM   #131
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,223
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
Quote:
Originally Posted by dwanthny View Post
Can you add identifying which books have a page-template.xpgt file to the QC plugin? Once I have these identified I can reconvert the books with the same settings and have Calibre remove the Adobe page template margins.

Thanks for all your work and considering this request.
I can do that. I vaguely recall a thread referring to this issue as these .xpgt files being spawn of the devil from the margin issues they caused. And that recently a Calibre change was made to do that margin removal, am I correct? What scenario do they cause a problem on? I.e. Will a user need to do an ePub to ePub conversion or is it just for converting to other formats like mobi? Is there any reason not to offer an option to just remove these files from the ePub?
kiwidude is offline   Reply With Quote
Old 05-02-2011, 06:45 AM   #132
DoctorOhh
US Navy, Retired
DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.
 
DoctorOhh's Avatar
 
Posts: 8,743
Karma: 12488297
Join Date: Feb 2009
Location: North Carolina
Device: Nexus 7
Quote:
Originally Posted by kiwidude View Post
I can do that. I vaguely recall a thread referring to this issue as these .xpgt files being spawn of the devil from the margin issues they caused. And that recently a Calibre change was made to do that margin removal, am I correct?
You are correct as of Calibre 0.7.53 calibre removes the Adobe page template margins during conversion.

Quote:
Originally Posted by kiwidude View Post
What scenario do they cause a problem on? I.e. Will a user need to do an ePub to ePub conversion or is it just for converting to other formats like mobi?
If the page-template controls the margins, and they often do, then a conversion epub - epub or epub - other will need to occur to remove the margins via calibre.

Quote:
Originally Posted by kiwidude View Post
Is there any reason not to offer an option to just remove these files from the ePub?
Quite a few folks in this thread seem to remove the file and all references to it and are happy with the results. If I had the option to just remove the file I would do it. I have already proofed the files in calibre's viewer which seems to ignore this file.

As of April 1 Calibre handles removing the margins from the page-template during conversion. So identifying the older books that have this page-template is my minimum need. Once identified I'll have no problem running them through a ePub to ePub conversion and let calibre deal with it. But I would not be against having the option to remove the file and any references to the file.
DoctorOhh is offline   Reply With Quote
Old 05-02-2011, 04:12 PM   #133
drMerry
Addict
drMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmosdrMerry has become one with the cosmos
 
drMerry's Avatar
 
Posts: 293
Karma: 21022
Join Date: Mar 2011
Location: NL
Device: Sony PRS-650
Quote:
Originally Posted by drMerry View Post
...
2. Where do you get the 'Title Case Information'?
I have some problems with it, All books having the word macht have to be written as MacHt (capital H). I think I made a mistake somewhere but I do not know where.
I also would like change this behaviour so only the first word and names get capitals, but I think I have to change a regex than.
This problem solved in next release of Calibre...
drMerry is offline   Reply With Quote
Old 05-02-2011, 05:58 PM   #134
jackie_w
Wizard
jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.jackie_w ought to be getting tired of karma fortunes by now.
 
Posts: 2,671
Karma: 3818025
Join Date: Sep 2009
Location: UK
Device: Sony PRS-350, PB360, Kobo Glo/AuraHD/Aura6"
Quote:
Originally Posted by kiwidude View Post
Is there any reason not to offer an option to just remove these files from the ePub?
None at all, but it would be a lot less coding effort to leave the .xpgt in place but "neutralise" its margins and I believe this is what the current calibre convert does.

If you want to surgically remove the .xpgt I think you would need to:
  • Find the .xpgt and delete it.
  • Remove the reference to it in the .opf manifest.
  • Remove the <link> reference to it in every html file.

I guess the decision is yours
jackie_w is offline   Reply With Quote
Old 05-02-2011, 06:02 PM   #135
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,223
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
@jackie - thx for the detail. My only thought is that as mentioned previously on this thread there is a perception rightly or wrongly that it can be "scary" to convert a file. Which if I didn't offer a way to remove it you are still forced to do.

Not that I am committing to do it at this point of course, I'm just intelligence gathering...
kiwidude is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] Search the Internet kiwidude Plugins 204 Yesterday 08:21 PM
[GUI Plugin] Kindle Collections meme Plugins 2054 07-12-2014 04:25 PM
[GUI Plugin] Open With kiwidude Plugins 223 07-09-2014 02:42 AM
[GUI Plugin] Clipboard Search kiwidude Plugins 4 09-11-2011 06:14 AM
[GUI Plugin] Book Sync **Deprecated** kiwidude Plugins 111 06-07-2011 07:47 PM


All times are GMT -4. The time now is 12:33 PM.


MobileRead.com is a privately owned, operated and funded community.