Mass import of books metadata from an ASCII file: HELP NEEDED

LARdT · 07-08-2010, 01:22 PM

Kovid,

As far as I'm a complete newbie in python I would be very grateful if you could give me a big hand here.

I would like to program a "complement" in order to be able to import a lot of records with books data into CALIBRE. To make it easier we will suppose that all the book records are new; so we do not have to control anything. We only need to be able to make the mass input into CALIBRE. We will make refinements later. (As soon as I learn a litlle of Python)

We will suppose we have an ASCII file, comma delimited (or whatever), with the metadata of the different books in it and one directory with the JPG files that will make the covers.

Lets suppose we have the following field structure in the ASCII file:

First part of author's name, Second part of author's name, Book Title, Input date, path to JPG cover file

so it will look something like this:

"Jules", "Verne", "20.000 leagues under the sea", "'08/07/2010", "C:\MYCOVERS\20000leaguescover.JPG"

If the program, were some kind of Basic it would look something like this:

Open the ASCII data file for reading

Read the ASCII file, phrase by phrase (line) using some kind of loop

Using the delimiter character ("comma" in our example) you get the phrase chopped in its different fields.
If it were easier we Could use a FIXED LENGHT fields ASCII file and would do the "chopping" using the lenghts of fields.

We make the correspondence between the pieces of our ASCII chopped line and CALIBRE database fields.

Obviously we have been able to OPEN the CALIBRE DATABASE and we are ready to do a WRITE NEW RECORD command in it.
As CALIBRE seems to use a SERIAL KEY to INDEX the book records we should be able to use it. Asuming all records are new, it would be enough with a numeric variable to do the job, which we will increase by one in every loop.

NumericValue_of_FirstRecord_to_Add%%=100

CREATE NEW BLANK RECORD in DATABASE

KeySerialIndex.CALIBREDATABASE= NumericValue_of_FirstRecord_to_Add%%
Name.CALIBREDATABASE="Jules"
Surname.CALIBREDATABASE="Verne"
...
Cover.CALIBREDATABASE="C:\MYCOVERS\20000leaguescov er.JPG"

STORE NEW RECORD in DATABASE

And that is it. We only have to work the LOOP down, reading the following ASCII lines, chop each line in its fields,
create a new BLANK RECORD in CALIBRE, store the field values in each field and save the new record; until we reach the EOF (END OF FILE) in the ASCII input file.

Lets close the ASCII INPUT FILE and the CALIBRE DATABASE OUTPUT FILE.

Could you please be so kind to write the half a dozen python sentences needed to make the program, just enuogh for me to catch the "programming feeling" for starters. I will need also the database definition of the CALIBRE ebook database, and the commands needed for OPENING, CREATING NEW RECORDS, FIELD VALUE LOADING, STORING NEW RECORDS, CLOSING DATABASE, etc...

I know it is a little to much but I promise to be a good pupil and after having a first workable 0.1 version, I'll work to deliver something more useful to the general public.

Thanks in advance.

kovidgoyal · 07-08-2010, 01:30 PM

Use the calibredb command

LARdT · 07-08-2010, 01:56 PM

Thanks, but I'm afraid I'll need a little more detail to be on track.

Could you please send a link with real calibre python code? Is there any CALIBRE PROGS Library?

May I send the code and ask for corrections?

Thanks; I know this a little out of the line, but I need a fast start, otherwise time will eat me up and the thing will remain undone, which I hate the most.

Starson17 · 07-08-2010, 03:08 PM

I struggled not to post anything impolite ... I really did.

Spoiler:

LARdT · 07-08-2010, 04:05 PM

Don't struggle so much. It's bad for your lower belly.

It is much easier not to prejudice. Do you have the lesser idea what kind of relationship exists between Kovid and me?

Eh?
I'll tell you: nothing more than several e-mails. But as far as they have been enough to convey things such a busssiness proposal (which Kovid declined) and several calls for help (which Kovid gracefully attended) I though that I was entitled to another shot.

It seems I was wrong. Never mind, please feel very welcome. And keep up your good policing. But please, first do some training at home before trying to police and censor the internet. Perhaps you may find more juicy issues to police in your own place. No doubt.

Kovid will turn me down without help if he feels like doing it. Don't you think so?

By the way, I like the button thing. And if you can throw a couple of sentences in Python as requested, I'll be very grateful.

07-08-2010, 01:22 PM	#1
LARdT The LARdT of E-Books Posts: 106 Karma: 46 Join Date: Dec 2009 Device: HANLIN V3ext (Papyre 6.1)	Mass import of books metadata from an ASCII file: HELP NEEDED Kovid, As far as I'm a complete newbie in python I would be very grateful if you could give me a big hand here. I would like to program a "complement" in order to be able to import a lot of records with books data into CALIBRE. To make it easier we will suppose that all the book records are new; so we do not have to control anything. We only need to be able to make the mass input into CALIBRE. We will make refinements later. (As soon as I learn a litlle of Python) We will suppose we have an ASCII file, comma delimited (or whatever), with the metadata of the different books in it and one directory with the JPG files that will make the covers. Lets suppose we have the following field structure in the ASCII file: First part of author's name, Second part of author's name, Book Title, Input date, path to JPG cover file so it will look something like this: "Jules", "Verne", "20.000 leagues under the sea", "'08/07/2010", "C:\MYCOVERS\20000leaguescover.JPG" If the program, were some kind of Basic it would look something like this: Open the ASCII data file for reading Read the ASCII file, phrase by phrase (line) using some kind of loop Using the delimiter character ("comma" in our example) you get the phrase chopped in its different fields. If it were easier we Could use a FIXED LENGHT fields ASCII file and would do the "chopping" using the lenghts of fields. We make the correspondence between the pieces of our ASCII chopped line and CALIBRE database fields. Obviously we have been able to OPEN the CALIBRE DATABASE and we are ready to do a WRITE NEW RECORD command in it. As CALIBRE seems to use a SERIAL KEY to INDEX the book records we should be able to use it. Asuming all records are new, it would be enough with a numeric variable to do the job, which we will increase by one in every loop. NumericValue_of_FirstRecord_to_Add%%=100 CREATE NEW BLANK RECORD in DATABASE KeySerialIndex.CALIBREDATABASE= NumericValue_of_FirstRecord_to_Add%% Name.CALIBREDATABASE="Jules" Surname.CALIBREDATABASE="Verne" ... Cover.CALIBREDATABASE="C:\MYCOVERS\20000leaguescov er.JPG" STORE NEW RECORD in DATABASE And that is it. We only have to work the LOOP down, reading the following ASCII lines, chop each line in its fields, create a new BLANK RECORD in CALIBRE, store the field values in each field and save the new record; until we reach the EOF (END OF FILE) in the ASCII input file. Lets close the ASCII INPUT FILE and the CALIBRE DATABASE OUTPUT FILE. Could you please be so kind to write the half a dozen python sentences needed to make the program, just enuogh for me to catch the "programming feeling" for starters. I will need also the database definition of the CALIBRE ebook database, and the commands needed for OPENING, CREATING NEW RECORDS, FIELD VALUE LOADING, STORING NEW RECORDS, CLOSING DATABASE, etc... I know it is a little to much but I promise to be a good pupil and after having a first workable 0.1 version, I'll work to deliver something more useful to the general public. Thanks in advance.

07-08-2010, 03:08 PM	#4
Starson17 Wizard Posts: 4,004 Karma: 177841 Join Date: Dec 2009 Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T	I struggled not to post anything impolite ... I really did. Spoiler: I don't think I succeeded

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Is it possible to sent books to device with filename in non-ascii characters?	flyisland	Calibre	8	10-16-2010 05:35 AM
Mass Import Explosion	hobolibrary	Calibre	5	06-19-2010 04:45 AM
Import: prioritization of metadata source?	ATimson	Calibre	2	02-28-2010 03:57 PM
Ascii file	ProDigit	Lounge	1	12-25-2008 10:08 PM
Help needed with editing Metadata	rhino27	Calibre	10	06-26-2008 01:59 AM

07-08-2010, 01:30 PM	#2
kovidgoyal creator of calibre Posts: 43,870 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Use the calibredb command

07-08-2010, 01:56 PM	#3
LARdT The LARdT of E-Books Posts: 106 Karma: 46 Join Date: Dec 2009 Device: HANLIN V3ext (Papyre 6.1)	Thanks, but I'm afraid I'll need a little more detail to be on track. Could you please send a link with real calibre python code? Is there any CALIBRE PROGS Library? May I send the code and ask for corrections? Thanks; I know this a little out of the line, but I need a fast start, otherwise time will eat me up and the thing will remain undone, which I hate the most.

07-08-2010, 04:05 PM	#5
LARdT The LARdT of E-Books Posts: 106 Karma: 46 Join Date: Dec 2009 Device: HANLIN V3ext (Papyre 6.1)	Don't struggle so much. It's bad for your lower belly. It is much easier not to prejudice. Do you have the lesser idea what kind of relationship exists between Kovid and me? Eh? I'll tell you: nothing more than several e-mails. But as far as they have been enough to convey things such a busssiness proposal (which Kovid declined) and several calls for help (which Kovid gracefully attended) I though that I was entitled to another shot. It seems I was wrong. Never mind, please feel very welcome. And keep up your good policing. But please, first do some training at home before trying to police and censor the internet. Perhaps you may find more juicy issues to police in your own place. No doubt. Kovid will turn me down without help if he feels like doing it. Don't you think so? By the way, I like the button thing. And if you can throw a couple of sentences in Python as requested, I'll be very grateful.

Advert

Advert