Calibre beta available that fully integrates custom fields - Page 7

chaley · 09-26-2010, 10:58 AM

Quote:

Originally Posted by PatNY

Yes, in the standard releases I've used the cover was always copied over to the other library. It's not a matter of the second library simply not showing the cover. The jpeg is actually missing from the book folder in the target library.

I confess, I done did it. Yet another case where a performance improvement caused a functional loss.

Fixed.

chaley · 09-26-2010, 11:03 AM

Quote:

Originally Posted by Starson17

FYI, I'm seeing opf files only in records that have epub formats. Books with only .txt or .pdf don't get an opf, even if metadata changes. (As I've seen others report, all epubs have opfs, even if no change was made to metadata.) I saw no slowdown during the creation of the opf files. I assume this isn't a bug, just the current design.

Hmmm..

I see them everywhere, which is what it is supposed to do. The code doesn't ever look at the formats, so I can't see why the file wouldn't be created. There should be a file 'metadata.opf' in every book's folder.

Could you confirm again that if you touch metadata (say, tags) for a book with only a PDF, no file metadata.opf is created? And no exceptions are raised?

Charles

theducks · 09-26-2010, 11:23 AM

I just spot checked a number of folders, some with only PDF, all seem to have a "metadata.opf all dated 9/24/2010 and timestamped 17:35

Starson17 · 09-26-2010, 11:33 AM

Quote:

Originally Posted by chaley

Hmmm..

I see them everywhere, which is what it is supposed to do. The code doesn't ever look at the formats, so I can't see why the file wouldn't be created. There should be a file 'metadata.opf' in every book's folder.

Could you confirm again that if you touch metadata (say, tags) for a book with only a PDF, no file metadata.opf is created? And no exceptions are raised?

Charles

Tags, ratings, all touched. No metadata.opf, except in epubs. I found no epubs without a metadata.opf, no non-epub with a metadata.opf, but I haven't looked at more than about 20-30 files of each type. I've been testing with small libraries, and only allowed the new code onto my big library about an hour ago. I suppose it could still be working? And putting the touched metadata files into a queue to build a metadata.opf later?

No exception raised. I'll do some more searches and report back.

Starson17 · 09-26-2010, 11:44 AM

Quote:

Originally Posted by Starson17

no non-epub with a metadata.opf

Wait, I found one! It must still be working through the list.
Edit: Yes. it's still working - I just saw one being generated by watching a folder in the library.

chaley · 09-26-2010, 11:58 AM

Quote:

Originally Posted by Starson17

Wait, I found one! It must still be working through the list.
Edit: Yes. it's still working - I just saw one being generated by watching a folder in the library.

The backup uses a FIFO queue, so changes you make won't get backed up until the first pass through the library finished.

That can take a while. On the serializer_trunk version, that is around 60 books per minute. The release code is (I think) around 100 books per minute.

Charles

Starson17 · 09-26-2010, 12:02 PM

Quote:

Originally Posted by chaley

On the serializer_trunk version, that is around 30 books per minute. The release code is (I think) around 100 books per minute.

I'm on the serializer_trunk and your estimate looks to be the right order of magnitude, assuming it's working through by book ID.
Sorry for the false alarm.

chaley · 09-26-2010, 12:29 PM

Quote:

Originally Posted by Sweetpea

I've got a problem with the custom fields:

I've got two books:

Book 1:
Author: Thea Beckman
Title: Kinderen van Moeder Aarde
Series: KvMA
Series Index: 1.0 ...

Book 2:
Author: Het Helse Paradijs
Title: 02
Series: [empty]
Series Index: [empty]
...
I created a new custom field:

{Author} - {Series} {Series_index:0>2s} - {Title}

Book 1 is filled with: Thea Beckman - KvMA 1.0 - Kinderen van Moeder Aarde
Book 2 is filled with: Het Helse Paradijs - 00 - 02

Looking more carefully at this, I contend that both are wrong. The first is wrong because it didn't use series formatting and displayed the float. I fixed this.

The second is wrong because *nothing* should be displayed when the book has no series. The '00' shouldn't be there. I will fix this as well.

Note: if you really want the 00, you use another field and the ifempty function. Create a composite field containing {series_index:0>2s}, lets call it AA. In the original you would use
{Author} - {Series} {AA:ifempty(00)} - {Title}

Sweetpea · 09-26-2010, 12:47 PM

Quote:

Originally Posted by chaley

Looking more carefully at this, I contend that both are wrong. The first is wrong because it didn't use series formatting and displayed the float. I fixed this.

The second is wrong because *nothing* should be displayed when the book has no series. The '00' shouldn't be there. I will fix this as well.

Note: if you really want the 00, you use another field and the ifempty function. Create a composite field containing {series_index:0>2s}, lets call it AA. In the original you would use
{Author} - {Series} {AA:ifempty(00)} - {Title}

Nah, I don't want that emtpy index, and I figured that was a bug as well and that you'd fix it when you'd fix the main issue: the lack of formatting

Edit:

is there a list of "commands" (for lack of a better word) you can use inside the field? (such as the "ifempty" and "0>2s")?

Manichean · 09-26-2010, 01:33 PM

Quote:

Originally Posted by Sweetpea

Nah, I don't want that emtpy index, and I figured that was a bug as well and that you'd fix it when you'd fix the main issue: the lack of formatting

Edit:

is there a list of "commands" (for lack of a better word) you can use inside the field? (such as the "ifempty" and "0>2s")?

See the spoiler text in this post.

Starson17 · 09-26-2010, 01:50 PM

Charles,
This is really low priority, but would it be possible to have a field reference {id} to reference the book ID? I've had a few cases where I want to access the ID.
If you're in that area of the code and feel like adding it, I'd use it. Thanks.

chaley · 09-26-2010, 02:16 PM

Quote:

Originally Posted by Starson17

Charles,
This is really low priority, but would it be possible to have a field reference {id} to reference the book ID? I've had a few cases where I want to access the ID.
If you're in that area of the code and feel like adding it, I'd use it. Thanks.

I actually thought it was already there. The problem was that we called it 'application_id' because that is what it must be for serialization. However, application_id isn't considered a displayable field, so you can't get to it.

I submitted the (one line) change to Kovid to put id into the Metadata using {id}.

Edit: code is in the serializer trunk, and will be in the next beta

Starson17 · 09-26-2010, 02:25 PM

Quote:

Originally Posted by chaley

I submitted the (one line) change to Kovid to put id into the Metadata using {id}.

Thanks!

Sweetpea · 09-26-2010, 02:47 PM

These custom fields got my imagination working...

One problem I have, is that my various readers all use a different way of displaying books.

On my WinCE/WinMobile devices, I use Mobipocket. I find using categories on Mobipocket is not easy if you've more categories than your screen is high (and on my PDA, that means no more than 10 categories!). So, I added my OPF files so that the "dc:title" is filled with (let's phrase it with Calibre names):

{author_sort} - {series} {series_index} - {title}

On my BBMini, I don't care what the metatags are filled with, it won't read them anyway (but show the filename instead).

The Sony does categories a bit more friendly, so I'm looking into that. But, the long titles can become annoying in that case. So, I'd like to make my title more like

{series} {series_index} - {title}

But, I don't want two different source OPF files!

Now, Calibre does make an OPF file next to my book file. Wouldn't it be nice, if I were able to say what fields would be used to fill in those basic OPF fields (author and title, mostly)?

so, something like:

<dc:title>{series} {series_index} - {title}</dc:title>
<dc:creator opf:file-as="{author_sort}" opf:role="aut">{author}</dc:creator>

kacir · 09-26-2010, 03:44 PM

I have installed 0.7.903

I have made backup of my library and tried some things. Let's see if I can break something [insert Mad scientist laughter here]. I tried to import some 3000 books with metadata in filenames. Format like "Jack London=the call of the wild.txt". I have created those files from a folder structure that had books like London_Jack/the_call_of_the_wild.txt" using shell script.
I have clicked on import and then I realized I forgot to set Preferences -> Import/export -> Adding books -> Regular expression.
So I started to click very furiously on the Abort button. Calibre did not stop importing. It displayed "Aborting" in the message and happily continued to import garbage for next hour or so.
I had to delete books and import them again.
The process seems to have finished successfully, but the program displayed curious message that the import process hanged.

I have tried to make a custom column, built from other columns. When I sort this column, it does not sort by alphabet (ignoring case) but "the Unix way" with A, B, C, ... Z, then a, b, c, ... z
Windows users are not used to this. They want to have it sorted regardless of case.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Sony collections and custom fields: how to handle duplicates?	chaley	Calibre	13	09-02-2010 04:11 PM
Amazon integrates Video and Audio with Ebooks	luqmaninbmore	News	22	06-28-2010 06:22 PM
PRC file doesn't fully import into Calibre	MSJim	Kindle Formats	1	06-01-2010 02:55 PM
Calibre custom news feed and python help.	harrynewman	Calibre	4	10-08-2009 09:26 AM
Smashwords integrates with Stanza, optimized for iPhone	Smashwords	News	1	11-29-2008 03:26 PM

09-26-2010, 11:23 AM	#93
theducks Well trained by Cats Posts: 31,098 Karma: 60358908 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	I just spot checked a number of folders, some with only PDF, all seem to have a "metadata.opf all dated 9/24/2010 and timestamped 17:35

09-26-2010, 01:50 PM	#101
Starson17 Wizard Posts: 4,004 Karma: 177841 Join Date: Dec 2009 Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T	Charles, This is really low priority, but would it be possible to have a field reference {id} to reference the book ID? I've had a few cases where I want to access the ID. If you're in that area of the code and feel like adding it, I'd use it. Thanks.

09-26-2010, 02:47 PM	#104
Sweetpea Grand Sorcerer Posts: 9,707 Karma: 32763414 Join Date: Dec 2008 Location: Krewerd Device: Pocketbook Inkpad 4 Color; Samsung Galaxy Tab S6	These custom fields got my imagination working... One problem I have, is that my various readers all use a different way of displaying books. On my WinCE/WinMobile devices, I use Mobipocket. I find using categories on Mobipocket is not easy if you've more categories than your screen is high (and on my PDA, that means no more than 10 categories!). So, I added my OPF files so that the "dc:title" is filled with (let's phrase it with Calibre names): {author_sort} - {series} {series_index} - {title} On my BBMini, I don't care what the metatags are filled with, it won't read them anyway (but show the filename instead). The Sony does categories a bit more friendly, so I'm looking into that. But, the long titles can become annoying in that case. So, I'd like to make my title more like {series} {series_index} - {title} But, I don't want two different source OPF files! Now, Calibre does make an OPF file next to my book file. Wouldn't it be nice, if I were able to say what fields would be used to fill in those basic OPF fields (author and title, mostly)? so, something like: <dc:title>{series} {series_index} - {title}</dc:title> <dc:creator opf:file-as="{author_sort}" opf:role="aut">{author}</dc:creator>

09-26-2010, 03:44 PM	#105
kacir Wizard Posts: 3,463 Karma: 10684861 Join Date: May 2006 Device: PocketBook 360, before it was Sony Reader, cassiopeia A-20	I have installed 0.7.903 I have made backup of my library and tried some things. Let's see if I can break something [insert Mad scientist laughter here]. I tried to import some 3000 books with metadata in filenames. Format like "Jack London=the call of the wild.txt". I have created those files from a folder structure that had books like London_Jack/the_call_of_the_wild.txt" using shell script. I have clicked on import and then I realized I forgot to set Preferences -> Import/export -> Adding books -> Regular expression. So I started to click very furiously on the Abort button. Calibre did not stop importing. It displayed "Aborting" in the message and happily continued to import garbage for next hour or so. I had to delete books and import them again. The process seems to have finished successfully, but the program displayed curious message that the import process hanged. I have tried to make a custom column, built from other columns. When I sort this column, it does not sort by alphabet (ignoring case) but "the Unix way" with A, B, C, ... Z, then a, b, c, ... z Windows users are not used to this. They want to have it sorted regardless of case.