Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 01-14-2011, 10:41 AM   #1
pchrist7
Addict
pchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animals
 
pchrist7's Avatar
 
Posts: 385
Karma: 6514
Join Date: Aug 2010
Location: Denmark
Device: Kindle 3 3G+Wifi, Oasis
User column based on part of "Comments" ?

Hi All
Hope some of you can tell whether this is possible or not.
If yes, then please point me in the right directions.
Have a Kindle 3, I always fetch annotations when using Calibre.
what I'm interested in, is if a book is read or not.
The annotations will append something like:

10-12-2010
Last Page Read: Location 4862 (99%)

to each books comments, if it has been opened on the Kindle.

IS it possible to create a new column, which searches the "Comments" for
"Last Page Read: Location " and then grab the 99% to put into my custom column ?

I just can't get my brain around this one
Thanks in advance,
Have a nice weekend all,
pchrist7 is offline   Reply With Quote
Old 01-14-2011, 11:04 AM   #2
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Sure. Make a custom column based on other columns. The template would be something like:
{comments:re((?i).*last page read: location \((\d+)\).*,\1)}

This will find the last 'Last Page Read' in comments. If you want to find the first such line, then try:
{comments:re((?i).*?last page read: location \((\d+)\).*,\1)}

Last edited by chaley; 01-15-2011 at 03:42 AM.
chaley is offline   Reply With Quote
Old 01-15-2011, 02:21 AM   #3
pchrist7
Addict
pchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animals
 
pchrist7's Avatar
 
Posts: 385
Karma: 6514
Join Date: Aug 2010
Location: Denmark
Device: Kindle 3 3G+Wifi, Oasis
@chaley - wow, that was quick :-)
Thanx for the tips. Will look into it later today.
Cheers,
pchrist7 is offline   Reply With Quote
Old 01-15-2011, 06:28 AM   #4
pchrist7
Addict
pchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animals
 
pchrist7's Avatar
 
Posts: 385
Karma: 6514
Join Date: Aug 2010
Location: Denmark
Device: Kindle 3 3G+Wifi, Oasis
Maybe I'm dumb, but ---

Quote:
Originally Posted by chaley View Post
Sure. Make a custom column based on other columns. The template would be something like:
{comments:re((?i).*last page read: location \((\d+)\).*,\1)}

This will find the last 'Last Page Read' in comments. If you want to find the first such line, then try:
{comments:re((?i).*?last page read: location \((\d+)\).*,\1)}
Hi Chaley
Again thanks for your efforts.
However I can't get this to work.
If I create a custom column pct_read with your examples, ALL the contents of the {comments} are shown. Have tried via google to find more info, but I'm into info overload right now.
Q: (?i) = both upper/lower case matches ?
I Know the first letter in the search string is always upper case.
I "just" want to grab 99% , or even better 99
and then use that for another column where pct_read > 98 = read/yes/whatever
Tried changing \1 to \2 ( and 3, 4 etc ) cause I thought the match would there, ( 2 x .* ) but ..
Too tired right now,
If you have the time, energy etc. would it help if I sent you more info ? If so please tell me what you need.
I'm also confused that any custom columns I create will show in the Books Detail. That was not my intension.
Hope this is trivial to you, and other gurues
One of you just might
Eventually I'll have to understand all this.
Thanks so far.

Edit: This is "only" nice to have .. but it would be neat
Edit2: Could be unrelated, but trying to build a new catalog, I got author_sort errors on the author I used for checking the new columns.
Need sleep

Last edited by pchrist7; 01-15-2011 at 07:13 AM. Reason: catalog problems
pchrist7 is offline   Reply With Quote
Old 01-15-2011, 07:38 AM   #5
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
The problem might be related to the expression failing if there isn't an annotation in the comments.

Try
Code:
{comments:'contains($, '(?i).*?last page read: location \(\d+\)', re($, '(?i).*?last page read: location \((\d+)\).*', '\1'), '')'}
This checks to see of the regular expression
Code:
(?i).*?last page read: location \(\d+\)
matches the comments. If it does, then it calls 're' to slice out the number. If it does not, then it returns '' (the empty string).

The (?i) does indeed mean ignore case.

The string test for 'last page read: location \(\d+\)' must match exactly. No extra spaces, no extra punctuation, no variants in spelling, and parenthesis around the numbers.

Ahh... looking at your original example, I note the % character. I didn't account for that. Try
Code:
{comments:'contains($, '(?i).*?last page read: location \(\d+%\)', re($, '(?i).*?last page read: location \((\d+)%\).*', '\1'), '')'}
chaley is offline   Reply With Quote
Old 01-16-2011, 01:37 AM   #6
pchrist7
Addict
pchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animals
 
pchrist7's Avatar
 
Posts: 385
Karma: 6514
Join Date: Aug 2010
Location: Denmark
Device: Kindle 3 3G+Wifi, Oasis
That's not either I'm afraid

Quote:
Originally Posted by chaley View Post
The problem might be related to the expression failing if there isn't an annotation in the comments.

Try
Code:
{comments:'contains($, '(?i).*?last page read: location \(\d+\)', re($, '(?i).*?last page read: location \((\d+)\).*', '\1'), '')'}
This checks to see of the regular expression
Code:
(?i).*?last page read: location \(\d+\)
matches the comments. If it does, then it calls 're' to slice out the number. If it does not, then it returns '' (the empty string).

The (?i) does indeed mean ignore case.

The string test for 'last page read: location \(\d+\)' must match exactly. No extra spaces, no extra punctuation, no variants in spelling, and parenthesis around the numbers.

Ahh... looking at your original example, I note the % character. I didn't account for that. Try
Code:
{comments:'contains($, '(?i).*?last page read: location \(\d+%\)', re($, '(?i).*?last page read: location \((\d+)%\).*', '\1'), '')'}
Hi chaley
Again thanks for the effort so far !
tried your last code, but column is blank
Testing on a book with just the annotations in comments, the HTML of the comments are as follows:

<div class="user_annotations">
<span style="font-weight:bold">
10-12-2010<br>Last Page Read: Location 4862 (99%)
</span>
<br></div>

This is over my head, I'm afraid.
Hope you're still willing to look at this.
pchrist7 is offline   Reply With Quote
Old 01-16-2011, 03:08 AM   #7
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
This works with that example.
Code:
{comments:'contains($, '(?i).*?last page read: location', re($, '(?is).*last page read: location \d+ \((\d+)%\).*', '\1'), '')'}
The previous one was not checking for the numbers after the word 'location'.
chaley is offline   Reply With Quote
Old 01-16-2011, 03:29 AM   #8
pchrist7
Addict
pchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animals
 
pchrist7's Avatar
 
Posts: 385
Karma: 6514
Join Date: Aug 2010
Location: Denmark
Device: Kindle 3 3G+Wifi, Oasis
Lightbulb Brilliant !

Quote:
Originally Posted by chaley View Post
This works with that example.
Code:
{comments:'contains($, '(?i).*?last page read: location', re($, '(?is).*last page read: location \d+ \((\d+)%\).*', '\1'), '')'}
The previous one was not checking for the numbers after the word 'location'.
Hi Chaley
That was it !
Spotted that myself but just couldn't figure out how to do it
Thank you SO much !

Now all I have to figure out is a new custom column Read, that is set to Yes if #pct_read > 94, otherwise No

Do you have the time/energy for this as well ?

This is very good news to Kindler's as we can use at least some of the annotations !



Just did a restart of Calibre.
I had no idea of the potiential performace impact of custom columns.
I guess all records are checked during startup to calculate the custom columns ?

Last edited by pchrist7; 01-16-2011 at 03:47 AM. Reason: wondering
pchrist7 is offline   Reply With Quote
Old 01-16-2011, 03:52 AM   #9
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Here is a solution. It works on the example you gave me.
Code:
program:
	comments = field('comments');
	contains(	comments,
				'(?i).*?last page read: location', 
				cmp(re(	comments,
						'(?is).*last page read: location \d+ \((\d+)%\).*', 
						'\1'),
					94,
					'No',
					'No',
					'Yes'),
				'')
The right way to solve this problem is to write a user-defined formatter function in python, something that became possible in 0.7.40. The reason: it is much easier to deal with 'unexpected situations' when writing in python. For example, the above code absolutely depends on the characters in the parenthesis being digits. If they are not, then ugly 'exception' messages get printed.
chaley is offline   Reply With Quote
Old 01-16-2011, 04:19 AM   #10
pchrist7
Addict
pchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animals
 
pchrist7's Avatar
 
Posts: 385
Karma: 6514
Join Date: Aug 2010
Location: Denmark
Device: Kindle 3 3G+Wifi, Oasis
More Brilliant !

Quote:
Originally Posted by chaley View Post
Here is a solution. It works on the example you gave me.
Code:
program:
	comments = field('comments');
	contains(	comments,
				'(?i).*?last page read: location', 
				cmp(re(	comments,
						'(?is).*last page read: location \d+ \((\d+)%\).*', 
						'\1'),
					94,
					'No',
					'No',
					'Yes'),
				'')
The right way to solve this problem is to write a user-defined formatter function in python, something that became possible in 0.7.40. The reason: it is much easier to deal with 'unexpected situations' when writing in python. For example, the above code absolutely depends on the characters in the parenthesis being digits. If they are not, then ugly 'exception' messages get printed.
Hi Chaley

"write a user-defined formatter function in python"
is WAY of out my capabilities at the moment.

DID try your code in a new column, AND it so far looks brilliant !
Books with "read" > 94 goes to Yes status, those below 94 goes to No, and books WITHOUT goes to "blank"
SO - last challenge, I hope, is setting books with NO %read as "No" instead of "blank"

Sorry for all the hassle, on a sunday, but I think other Kindlers would like this besides me.
pchrist7 is offline   Reply With Quote
Old 01-16-2011, 04:24 AM   #11
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Change the last line, putting whatever you want between the quotes. That will be shown if there isn't a read %.

For example (and being a little silly)
Code:
program:
	comments = field('comments');
	contains(	comments,
				'(?i).*?last page read: location', 
				cmp(re(	comments,
						'(?is).*last page read: location \d+ \((\d+)%\).*', 
						'\1'),
					94,
					'No',
					'No',
					'Yes'),
				'Unopened')
Will show 'Unopened' for all books without the annotation.
chaley is offline   Reply With Quote
Old 01-16-2011, 04:53 AM   #12
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Here is a better version of the last template program. The original was throwing exceptions that didn't hurt anything but cluttered up the log. This one avoids that by ensuring that only numbers are passed to the cmp function.

Code:
program:
	comments = field('comments');
	percent = re(comments,	'(?is).*last page read: location \d+ \((\d+)%\).*', '\1');
	contains(	comments,
				'(?i).*?last page read: location', 
				cmp(switch(percent, '^\d*$', percent, 0),
					94,
					'No',
					'No',
					'Yes'),
				'Unopened')
chaley is offline   Reply With Quote
Old 01-17-2011, 01:03 AM   #13
pchrist7
Addict
pchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animals
 
pchrist7's Avatar
 
Posts: 385
Karma: 6514
Join Date: Aug 2010
Location: Denmark
Device: Kindle 3 3G+Wifi, Oasis
Thanks Chaley !
works like a charm.
A bit slow, but it works.
pchrist7 is offline   Reply With Quote
Old 01-17-2011, 05:46 AM   #14
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
If you are willing to experiment... Below find a template function that does what you want. It should be orders of magnitude faster than the template program.

You must be running 0.7.40 or newer to use this.

To set it up:
1) go to preferences -> advanced -> template functions.
2) in the Function box type: kindle_read_percent
3) in the arg count box type: 5
4) in the documentation box, paste the following:
Code:
Check if the associated field contains a kindle annotation for percent read. If not, return no_page_read_str. If so, then compare that percent against is_read_pct. If the val is equal or larger, then return is_read_str, otherwise return is_not_read_str.

One usage: {comments:kindle_read_percent(95,Yes,No,Unopened)}
5) In the program code box, paste the following:
Code:
def evaluate(self, formatter, kwargs, mi, locals, val, is_read_pct, is_read_str, 
				is_not_read_str, no_page_read_str):
	try:
		test_val = int(is_read_pct)
	except:
		return 'is_read_pct is not a number'

	import re
	mg = re.match('.*last page read: location \d+ \((\d+)%\)', val, re.I + re.DOTALL);
	if mg is None:
		return no_page_read_str
	try:
		f = int(mg.group(1))
		if f >= test_val:
			return is_read_str
		return is_not_read_str
	except:
		pass
	return no_page_read_str
6) press the Create button, then the Apply button, then the Close button.
7) put into your custom column:
Code:
{comments:kindle_read_percent(95,Yes,No,Unopened)}
You can change the strings and percent, but try it once this way to be sure that it works.
chaley is offline   Reply With Quote
Old 01-17-2011, 06:49 AM   #15
pchrist7
Addict
pchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animals
 
pchrist7's Avatar
 
Posts: 385
Karma: 6514
Join Date: Aug 2010
Location: Denmark
Device: Kindle 3 3G+Wifi, Oasis
Talking Jesus !! THIS rocks !!


Hi Chaley !
This is blindingly fast compared the the other solutions.
1-2 seconds instead of 1-2 minuttes when starting Calibre.
Still have the pctread column while testing, fine tuning.
I'm a VERY happy Kindler right now

Maybe you/we could put this somewhere as a sticky or something ?
Other Kindlers might love this besides me.
Subject could be something like: Automagically mark books read


pchrist7 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
"integrity error column book is not unique" Atiajar Calibre 4 12-21-2010 11:22 PM
Can Comments be "saved to disk" for reading? sailingpeanut Recipes 1 10-10-2010 10:06 AM
(2 books) in "On Device" column Sony Reader phenomshel Calibre 23 09-19-2010 05:43 PM
Bug? "Insert metadata as page at start of book" doesnt encode Comments field properly rollercoaster Calibre 2 04-24-2010 10:40 PM


All times are GMT -4. The time now is 10:38 PM.


MobileRead.com is a privately owned, operated and funded community.