How can I force Calibre to add formats that are added manually to the respective f?

[loviedovie]

Originally Posted by BetterRed

@loviedovie - Maybe - I've never had any dvju format files.

But I suspect the problem is that the file name contains 'truncated' titles as in - "The artistic anatomy of trees, their struc" whereas the title in the metadata is presumably "The artistic anatomy of trees, their structure & treatment in painting".

I'm not sure what constitutes a 'duplicate' in the Add process. What happens if the title & author are not truncated in the file name, e.g - "Some Title - Some Author.djvu"

BR

Yeah that might be it. The title is truncated but not the author. In any case Calibre should be able to detect that the file name is pretty close given that the author is the same.

When I put the full file name , it adds as a duplicate so it works. But the thing is that I did not set these file names my self, Calibre truncated the file names on the file system and the converted djvu are named after whatever pdfs there were in the Calibre database folders.


This would have been a perfect solution for me since I have not done any Python coding for years ;(

[BetterRed]

@loviedovie - calibre munges the book title and author names to accommodate the vagaries of the different operating/file systems it supports. The shortening is done to fit names into Windows 255 (ish) path length limit. IIRC the loss of diacritics and 'conversion' of non latin characters (Greek, Cyrillic etc) to ASCII is done in order to maintain compatibility with some Linux file systems.

You could try something along these lines

• delete all the djvu's in the library book folders;
• save the pdfs to a flat folder with template {title} - {authors} using Save to Disk;
• write a bash script to convert the pdf files to djvu files, with same file names;
• delete the pdfs;
• use the Add books from single directory to add the djvu files, with appropriate Add Books settings.

FWIW - to reduce the incidence of shortened file names, I truncate Titles by putting subtitles and straplines into a custom column, so for that sample previously cited, I would have 'The Artistic Anatomy of Trees' in the Title column and 'Their structure & treatment in painting' in my Strapline column.

BR

[loviedovie]

@BetterRed

Thanks for the recommendations. I will give it a try.

[BetterRed]

Originally Posted by loviedovie

@BetterRed

Thanks for the recommendations. I will give it a try.

@loviedovie I forgot something - you'll probably need to adjust the 'save_template_title_series_sorting' twerk to 'strictly alphabetic'



BR

[loviedovie]

Thanks for the tip.

What should I use for the saving template? "{title}-{authors}" ?

[loviedovie]

Ok this does not work me either ;(

Actual book
title:Greek Vases in the J. Paul Getty Museum, Volume 3, OPA 2
auth:János Gy. Szilágy


saved pdf: "Greek_Vases_in_the_J._Paul_Getty_Museum,_Volume_3 ,_OPA_2-@-Janos_Gy._Szilagy.pdf"
converted djvu: "Greek_Vases_in_the_J._Paul_Getty_Museum,_Volume_3 ,_OPA_2-@-Janos_Gy._Szilagy.djvu"

save format: "{title}-@-{authors}"

add books regex: "(?P<title>.+)-@-(?P<author>[^_]+)"

This is tested properly with the regex in the add books window meaning that all the underscores are repalced with spaces properly and the author is recognized properly.

I wonder if the new template which replaces spaces with underscores is the issue?

[BetterRed]

Originally Posted by loviedovie

Ok this does not work me either ;(

Actual book
title:Greek Vases in the J. Paul Getty Museum, Volume 3, OPA 2
auth:János Gy. Szilágy


saved pdf: "Greek_Vases_in_the_J._Paul_Getty_Museum,_Volume_3 ,_OPA_2-@-Janos_Gy._Szilagy.pdf"
converted djvu: "Greek_Vases_in_the_J._Paul_Getty_Museum,_Volume_3 ,_OPA_2-@-Janos_Gy._Szilagy.djvu"

save format: "{title}-@-{authors}"

add books regex: "(?P<title>.+)-@-(?P<author>[^_]+)"

This is tested properly with the regex in the add books window meaning that all the underscores are repalced with spaces properly and the author is recognized properly.

I wonder if the new template which replaces spaces with underscores is the issue?

@loviedovie - yes, that and the fact that we lost the diacritics on the Authors name.

I created a book with that name and author - good choice btw.

The PDF in the book folder is

Greek Vases in the J. Paul Gett - Janos Gy. Szilagy.pdf

I saved it with these settings



And I got this as the saved file name

Greek Vases in the J. Paul Getty Museum, Volume 3, OPA 2 - János Gy. Szilágy.pdf

I used these Add Books settings



When I added a DOCX with the name

Greek Vases in the J. Paul Getty Museum, Volume 3, OPA 2 - János Gy. Szilágy.doc

I got this



And in the book folder I have

Greek Vases in the J. Paul Gett - Janos Gy. Szilagy.doc
Greek Vases in the J. Paul Gett - Janos Gy. Szilagy.pdf

I think that's what we want - give it a go

BR

[loviedovie]

@BetterRed

Thanks for all your help, this works for me Great way to achieve what I needed. This should be a wiki entry

This is my script to convert to djvu. It can be run in a folder. It is linux only and you need pdf2djvu package

#!/bin/bash

fltyp()
{
IFS=$'\n'
filename_full="$1"

filename=$(basename "$filename_full" .pdf)

export DIR=$(readlink -f $filename_full)
DIR=${DIR%/*}
echo $DIR
pdf2djvu -j 4 --lossy -v "$2" -o $DIR/$filename.djvu

}


export -f fltyp

find -type f -name "*.pdf" -exec bash -c 'fltyp "{}" "{}"' \;

[BetterRed]

Originally Posted by loviedovie

@BetterRed

Thanks for all your help, this works for me Great way to achieve what I needed. This should be a wiki entry

glad we got there, sorry it took so long. My fault, I didn't pay enough attention to the regex, I forgot the tweak, and the unset of ascii-fication setting in Save

BR

[loviedovie]

Originally Posted by BetterRed

glad we got there, sorry it took so long. My fault, I didn't pay enough attention to the regex, I forgot the tweak, and the unset of ascii-fication setting in Save

BR

@BetterRed,

You did great help. I was able to add all the Djvus with minor hickups. Hopefully in the future Calibre could detect these additional formats and add them automatically and also it could provide to Djvu conversion as well.

[kovidgoyal]

If all you want to do is get calibre to recognize djvu files in the library folders that are named exactly the same as the PDF files, you can do that by running Library Maintenance and adding the detected extra formats.

[loviedovie]

Originally Posted by kovidgoyal

If all you want to do is get calibre to recognize djvu files in the library folders that are named exactly the same as the PDF files, you can do that by running Library Maintenance and adding the detected extra formats.

Kovid

I actually tried that as the first method but it did not work. It showed the additional formats but there was no way to select anything to "fix". No checkbox if I remember correctly.

[eschwartz]

... And back to the topic of parsing json in bash, it's pretty much impossible -- unless you use a tool like jshon. Obviously a proper programming language like python will be easier to work with -- but if you really want to do it with bash you can.

I have used jshon before, for parsing calibredb's output.