Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 05-16-2010, 10:26 AM   #1
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Beta Beta Test of Major New Features

This is a non-sticky discussion thread for the unofficial beta beta development available from lp:~cbhaley/calibre/charles.custcol

Charles,
Instead of doing this by email or PM, this is easier for me, so I'm moving the discussion here.

Our last round is quoted below:
1) I'll check the attempt to delete and get back to you.
2) I suspected you might not see the oddity of LN, FN sorting
3) Yes, there are often unsuspected benefits of new features.
4) New bug:
a) connect to empty folder, add books
b) disconnect
c) reconnect to same folder. The author is now listed as title and title as author for each book.

Quote:
Originally Posted by chaley
Quote:
Originally Posted by Starson17
That fixed it.
1) I'm still seeing failure to delete. The file is removed from the Reader library, but not from the folder. I saw this intermittently before. Still testing, but so far haven't seen any files actually removed from folder.
I never see this. Is there any chance that something has the file open, preventing its deletion?

It would help if you checked that calibre actually attempts to delete the file. A print statement in devices.usbms.driver at line 205 would do it. Best would be to print the value that unlink returns.
Quote:
2) I suppose you know this, but the author sort order is by actual author, not author_sort. Lani Aames is after Dan Abbot.
Actually, I didn't see it because I keep names in LN, FN order.

Is this new behavior? Did it do something different before? Ahhh, you wouldn't know.

I can correct the ondevice scanner and metadata correcter to recompute author_sort for those books that don't have it, then send the metadata back to the 'device'. Once I do that, I will use author_sort.
Quote:
3) This is a nice unexpected benefit. I have a folder of some ebooks to be added, that are mostly duplicates of ebooks already added. I happened to try that folder for a test. The matching algorithm showed which ones were already in the library. That was very useful.
Interesting use of the feature. My guess is that more like this will show up once people more creative than me get their hands on it.
Starson17 is offline   Reply With Quote
Old 05-16-2010, 10:50 AM   #2
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,691
Karma: 6240117
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Quote:
Originally Posted by Starson17 View Post
2) I suspected you might not see the oddity of LN, FN sorting
I have checked with Kovid, and it is broken. Oddly enough, it has been broken since forever. I will fix it soonly .
Quote:
4) New bug:
a) connect to empty folder, add books
b) disconnect
c) reconnect to same folder. The author is now listed as title and title as author for each book.
I don't see this one either. Books I add show the right info.

As an experiment, disconnect from the folder, go to that folder and delete the file .metadata.calibre, and then reconnect to the folder. The device driver will rebuild the metadata from what is found in the books. Are the results different?
chaley is offline   Reply With Quote
Advert
Old 05-16-2010, 11:17 AM   #3
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,691
Karma: 6240117
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Ahhh -- you were adding a new book, not copying from the library?

I just tried doing that, and it worked for me.

This operation will use the preference settings for adding books. Any chance you have ticked 'read metadata only from file name' and that your regexp is getting the two reversed?

My settings are vanilla: no boxes checked on the add preferences tab, with the standard regexp. I added an epub that had metadata in it, and the metadata was right. I added a text file and got the text file name as title with author unknown.
chaley is offline   Reply With Quote
Old 05-16-2010, 11:22 AM   #4
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
[QUOTE=Starson17;913208]This happens with .txt format files. In one pdf it mangled, in another, it looks correct. Some .lits are OK, others not. ...
Still testing, this is a prelim report.
The data looks correct in the GUI during the initial import. It's wrong during the reconnect. The errors can be seen in the .metadata.calibre cache file.

Last edited by Starson17; 05-16-2010 at 11:53 AM.
Starson17 is offline   Reply With Quote
Old 05-16-2010, 11:31 AM   #5
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by chaley View Post
Ahhh -- you were adding a new book, not copying from the library?
No, in each case I was transferring from the library. See below.

Quote:
I just tried doing that, and it worked for me.
That's why it's beta

Quote:
This operation will use the preference settings for adding books. Any chance you have ticked 'read metadata only from file name' and that your regexp is getting the two reversed?
They were coming from the main library. No options checked on Adding books.

Quote:
My settings are vanilla: no boxes checked on the add preferences tab, with the standard regexp. I added an epub that had metadata in it, and the metadata was right. I added a text file and got the text file name as title with author unknown.
What happened with a Send to Device - Main Memory from main library of a txt file?
Starson17 is offline   Reply With Quote
Advert
Old 05-16-2010, 11:47 AM   #6
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,691
Karma: 6240117
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Quote:
Originally Posted by Starson17 View Post
What happened with a Send to Device - Main Memory from main library of a txt file?
Works for me.

I wonder if the delete problem is not related to this? Clearly something is stopping calibre from certain file manipulations. If the metadata cache is not being written, then it tries to determine the metadata from the file. For text files this is clearly impossible, so it uses the filename pattern stored in preferences.
Code:
    pat = re.compile(prefs.get('filename_pattern'))
    name = name.replace('_', ' ')
    match = pat.search(name)
In other words, it will use the last value you put into the regexp box for adding books. If this regexp does not match the filename pattern used for sending to device (also in preferences), then nothing good will happen.

I will look at the path names for local drives. I can easily imagine that the paths are not being normalized to windows (you run vista?), which might cause trouble.
chaley is offline   Reply With Quote
Old 05-16-2010, 11:57 AM   #7
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Somewhere I lost a reply to you. The author/title munge is visible in the .metadata.cache file. I'm multitasking some jobs for the wife: welding oxygen and goldfish - don't ask.
Starson17 is offline   Reply With Quote
Old 05-16-2010, 12:13 PM   #8
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by Starson17 View Post
The author/title munge is visible in the .metadata.cache file.
This is for the SD card mounted in the SD reader. Initial transfer of ebooks looks perfect in the GUI, on the SD card and in the .metadata.cache (AFAICT). Disconnect the folder and reconnect to it - nothing else. Then open the .metadata.cache file and I see all kinds of munge in author/title. Authors become Unknown or title. Titles messed up, etc.

I suggest we focus on this, then see if the delete is still an issue.
Starson17 is offline   Reply With Quote
Old 05-16-2010, 12:22 PM   #9
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by chaley View Post
If the metadata cache is not being written, then it tries to determine the metadata from the file.
AFAICT, it is correctly writing the metadata.calibre file (I assume that is the cache) on the SD card (and on the HD for that matter - yes I'm on Vista). Then it munges it on a reconnect.

I probably do have a mismatch between the regex used to Add Books and the pattern used to save the book out. Normally I have the option to read metadata only from filename on, and I vary the Add Books import regex. Does that regex come into play during the reconnect to the folder (or during the initial transfer from main library to Device)?

Quote:
In other words, it will use the last value you put into the regexp box for adding books. If this regexp does not match the filename pattern used for sending to device (also in preferences), then nothing good will happen.
Does this happen even with a simple transfer from the main GUI library?
Starson17 is offline   Reply With Quote
Old 05-16-2010, 12:27 PM   #10
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by Starson17 View Post
Does this happen even with a simple transfer from the main GUI library?
It looks like the reconnect is attempting to determine the ebook metadata from the ebooks a second time. It reads the metadata internally from the file, if it can, and if not, uses the regex on the filename. Then it revises the correct metadata in the cache file and ends up with a mess. I can fix this by controlling my settings, but it will be problematical for others.

I'll test to verify. (I think this is what you were saying, but it just sank in.)

Last edited by Starson17; 05-16-2010 at 12:38 PM.
Starson17 is offline   Reply With Quote
Old 05-16-2010, 12:29 PM   #11
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,691
Karma: 6240117
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Yes, that is what I was saying, and yes, it will be problematic. It shouldn't recompute the metadata.

I am now looking at when that happens and why. I have seriously broken the code, unfortunately, so it will be a little while before I get back to you.
chaley is offline   Reply With Quote
Old 05-16-2010, 12:31 PM   #12
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by chaley View Post
Yes, that is what I was saying, and yes, it will be problematic. It shouldn't recompute the metadata.

I am now looking at when that happens and why. I have seriously broken the code, unfortunately, so it will be a little while before I get back to you.
Can you post the default Add Save regex and the Send to Device regex. I can recreate something similar to them, but I'd like to use the defaults, and I don't seem to have them in my notes.
Starson17 is offline   Reply With Quote
Old 05-16-2010, 12:34 PM   #13
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,691
Karma: 6240117
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Under some conditions that I haven't yet figured out, it uses
re.compile(r'^(?P<title>[ \S]+?)[ _]-[ _](?P<author>[ \S]+?)_+\d+'))

The default in preferences is '(?P<title>.+) - (?P<author>[^_]+)'
chaley is offline   Reply With Quote
Old 05-16-2010, 12:55 PM   #14
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by chaley View Post
The default in preferences is
OK, this will confirm that setting regex to defaults mostly fixes things. I say "mostly" due to the oddity of text file metadata. I vaguely recall that calibre attempts to divine the metadata from inside the text file rather than always getting it from the filename. If the "metadata only from filename" option is off, it is getting the metadata of a txt file incorrectly on the reread. Apparently calibre never writes the correct metadata internally into a txt file, so with that option off, on the reread of metadata, it's always wrong. Other formats have correct metadata written internally, or if not, the metadata is picked up from the filename on the reread, so I saw errors only with txt files after the default regexes were set.

Turning on the "metadata only from filename" option fixed the txt file problem.

I'll check the delete issues now that this bug is at least identified.
Starson17 is offline   Reply With Quote
Old 05-16-2010, 01:00 PM   #15
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
There's no confirm dialog on delete.
Starson17 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Copia beta test - Get Your Invite Now! Nate the great Announcements 34 11-18-2010 07:48 PM
Beta Test of new Kobo Driver timlegge Calibre 58 06-22-2010 07:37 AM
Beta test of major new features chaley Calibre 130 06-04-2010 08:52 PM
DR800 V2 Beta Firmware Battery Test CoolDragon iRex 29 05-11-2010 05:37 AM
Mobile Forum Beta Test Here! Alexander Turcic Announcements 12 03-10-2004 05:38 PM


All times are GMT -4. The time now is 11:29 PM.


MobileRead.com is a privately owned, operated and funded community.