mass-importing ebooks in multiple formats?

[hakan42]

Hi,

I bought about 100 ebooks from readersplant.de, at that time, they were offering the downloads in four formats, PDF, HTML, PDB and TXT.

I would like to keep each book in all four formats, but if I add the whole directory of books to calibre, each file is a new book in my database.

Is there a way to "merge" those books later? I would avoid to repeteadly clicking "add alternative format" on each ebook, since I would have to do that about 300 times (one main format, three alternatives, about 100 ebooks).

At this time, my library (outside of calibre) is organised like

<series title>
\
PDF
HTML
PDB
TXT

and within each of those four directories, all the books of that series, in the appropiate format.


Regards,
Hakan

[kovidgoyal]

if all the different format files are in the same directory you can use the one book per directory import option. Otherwise, you're out of luck

[hakan42]

yes, I found the "calibredb add -1" option just a few minutes after writing my question :-) (by the way, I'm using calibre 0.6.6 on Gentoo Linux)

I reorganised my collection to look like

<series title>
\
\-- <issue number>

and put all the four different files into the <issue number> directory.

Now I'm calling

$ calibredb add -1 -r /home/hakan/Archive/EBooks/Atlan\ 1-50/Atlan\ 03

to import issue 03, but I'm getting the following error message:

The output in the first line

Traceback (most recent call last):
File "/usr/bin/calibredb", line 8, in <module>
load_entry_point('calibre==0.6.6', 'console_scripts', 'calibredb')()
File "/usr/lib/python2.6/site-packages/calibre/library/cli.py", line 550, in main
return command(args[2:], dbpath)
File "/usr/lib/python2.6/site-packages/calibre/library/cli.py", line 341, in command_add
do_add(get_db(dbpath, opts), args[1:], opts.one_book_per_directory, opts.recurse, opts.duplicates)
File "/usr/lib/python2.6/site-packages/calibre/library/cli.py", line 280, in do_add
dir_dups.extend(db.recursive_import(dir, single_book_per_directory=one_book_per_directory))
File "/usr/lib/python2.6/site-packages/calibre/library/database2.py", line 1780, in recursive_import
single_book_per_directory else \
File "/usr/lib/python2.6/site-packages/calibre/library/database2.py", line 1766, in import_book_directory
mi = metadata_from_formats(formats)
File "/usr/lib/python2.6/site-packages/calibre/ebooks/metadata/meta.py", line 34, in metadata_from_formats
mi = metadata_from_filename(list(formats)[0])
File "/usr/lib/python2.6/site-packages/calibre/ebooks/metadata/meta.py", line 120, in metadata_from_filename
name = name.rpartition('.')[0]
AttributeError: 'list' object has no attribute 'rpartition'


The method metadata_from_filename gets passed the following list (I added a quick "print name" to meta.py, line 119:

[u'/home/hakan/Archive/EBooks/readers_planet/Atlan 1-50/Atlan 03/Atlan Nr 3 - Das Psycho-Team.pdf', u'/home/hakan/Archive/EBooks/readers_planet/Atlan 1-50/Atlan 03/Atlan Nr 3 - Das Psycho-Team.txt', u'/home/hakan/Archive/EBooks/readers_planet/Atlan 1-50/Atlan 03/Atlan Nr 3 - Das Psycho-Team.html', u'/home/hakan/Archive/EBooks/readers_planet/Atlan 1-50/Atlan 03/Atlan Nr 3 - Das Psycho-Team.pdb']

[kovidgoyal]

can you put in

print repr(formats)

on line 31

[hakan42]

Done, that gives me

[[u'/home/hakan/Archive/EBooks/readers_planet/Atlan 1-50/Atlan 03/Atlan Nr 3 - Das Psycho-Team.pdf', u'/home/hakan/Archive/EBooks/readers_planet/Atlan 1-50/Atlan 03/Atlan Nr 3 - Das Psycho-Team.txt', u'/home/hakan/Archive/EBooks/readers_planet/Atlan 1-50/Atlan 03/Atlan Nr 3 - Das Psycho-Team.html', u'/home/hakan/Archive/EBooks/readers_planet/Atlan 1-50/Atlan 03/Atlan Nr 3 - Das Psycho-Team.pdb']]

The same list, but this time with double parantheses.

If you want, I could send you the complete directory via email. Attaching it here might be problematic, because those are copyrighted works.

Regards,
Hakan

[kovidgoyal]

I've made a change that will hopefully fix this, try the next release and let me know.

[hakan42]

Hi,

I tried 0.6.7, the error message is the same as with 0.6.6. I added the "print" statements to meta.py, and the input parameters to the methods "metadata_from_formats" and "metadata_from_filename" are the same, too.

Is there something else I could try?


Regards,
Hakan

[[u'/home/hakan/Archive/EBooks/readers_planet/Atlan 1-50/Atlan 03/Atlan Nr 3 - Das Psycho-Team.pdf', u'/home/hakan/Archive/EBooks/readers_planet/Atlan 1-50/Atlan 03/Atlan Nr 3 - Das Psycho-Team.txt', u'/home/hakan/Archive/EBooks/readers_planet/Atlan 1-50/Atlan 03/Atlan Nr 3 - Das Psycho-Team.html', u'/home/hakan/Archive/EBooks/readers_planet/Atlan 1-50/Atlan 03/Atlan Nr 3 - Das Psycho-Team.pdb']]

[u'/home/hakan/Archive/EBooks/readers_planet/Atlan 1-50/Atlan 03/Atlan Nr 3 - Das Psycho-Team.pdf', u'/home/hakan/Archive/EBooks/readers_planet/Atlan 1-50/Atlan 03/Atlan Nr 3 - Das Psycho-Team.txt', u'/home/hakan/Archive/EBooks/readers_planet/Atlan 1-50/Atlan 03/Atlan Nr 3 - Das Psycho-Team.html', u'/home/hakan/Archive/EBooks/readers_planet/Atlan 1-50/Atlan 03/Atlan Nr 3 - Das Psycho-Team.pdb']

Traceback (most recent call last):
File "/usr/bin/calibredb", line 8, in <module>
load_entry_point('calibre==0.6.7', 'console_scripts', 'calibredb')()
File "/usr/lib/python2.6/site-packages/calibre/library/cli.py", line 551, in main
return command(args[2:], dbpath)
File "/usr/lib/python2.6/site-packages/calibre/library/cli.py", line 342, in command_add
do_add(get_db(dbpath, opts), args[1:], opts.one_book_per_directory, opts.recurse, opts.duplicates)
File "/usr/lib/python2.6/site-packages/calibre/library/cli.py", line 281, in do_add
dir_dups.extend(db.recursive_import(dir, single_book_per_directory=one_book_per_directory))
File "/usr/lib/python2.6/site-packages/calibre/library/database2.py", line 1782, in recursive_import
single_book_per_directory else \
File "/usr/lib/python2.6/site-packages/calibre/library/database2.py", line 1768, in import_book_directory
mi = metadata_from_formats(formats)
File "/usr/lib/python2.6/site-packages/calibre/ebooks/metadata/meta.py", line 36, in metadata_from_formats
mi = metadata_from_filename(list(iter(formats))[0])
File "/usr/lib/python2.6/site-packages/calibre/ebooks/metadata/meta.py", line 123, in metadata_from_filename
name = name.rpartition('.')[0]
AttributeError: 'list' object has no attribute 'rpartition'

[kovidgoyal]

Open a ticket for this, I don;t have th time to loko at it right now and I'll forget otherwise.

[hakan42]

Done, Ticket #3197

Meanwhile, I'm doing the import by hand, but this will keep me busy for quite a few weeks

Regards,
Hakan

[Gwen Morse]

Searching and reading the forums turned up this thread. I've imported several thousand ebooks into my install of Calibre. The original collection is best described as a "big old mess". I have many directories with multiple copies of the same book (different editions/conversions/formats). So, I have a huge collection of numbered directories that are only slightly less messy than the original.

I can see there's no way to fix things up as part of the initial import. Can I request the option to be able to merge the directories with different copies of the same book once the install is complete. Maybe an "export" function that physically copies files and writes them to a new directory tree on request. Exporting and then reimporting them a couple times would still be faster than moving every one of these ebooks by hand and getting them in just the right directory for a single import.

[SirBC]

I imported about 800 ebooks on my first run, and after about a week of organizing them the number had dropped to about 300 as I had many books with multiple formats. I remember thinking it would be very helpful if there were a "merge" button, so that you could highlight multiple books that you know are actually the same book, hit the merge button, and Calibre would then merge the separate formats into one entry.

[itimpi]

The thing to do is to raise a ticket requeuting this facility and how you see it is meant to operate.

I can see difficulties in describing how it should operate. There are very likely to be clashes in several aspect of the books metadata. Any replacement system has to be slicker than simply selecting one of the entries, adding the other formats to that entry and then deleting the superfluous ones. If it cannot be made effecient it will not be worth doing.

If was thinking along the line of having a right-click entry for a file which is a "Merge books" so that the next one selected is merged in. Any metadata clashes are automatically resolved in favor of the first entry.

[Gwen Morse]

I submitted a ticket but I have no idea if it will be implemented.

In the meantime, is there a competing product I can use just to get things organized into the correct directories? I believe I can re-import them into calibre once they're all cleaned up and organized, one directory per book.

Gwen

[junkyardwillie]

Hopefully it can be done because I was wishing for the same thing as I try to consolidate my library into Calibre

[creedpatton]

I also second this request for a 'merge' option. My library is a complete mess as is and I can't fathom the thought of doing it manually!