Downloaded metadata merging

[FlyGoneCrazy]

Hey there!

I'm looking for ideas how to avoid downloaded metadata merging. The case is that in my country there are a lot of books without ISBN, because of historic reasons (Soviet union did not used ISBN).

So my problem is that a lot of books do not crawl ISBN and it seems no ISBN is treated as the same ISBN. So they are merged. Or as I see, they are get rid of at all.

Do you know a method how to avoid merging? I've tried to modify plugin and use unique title and unique author, by adding random string. Unfortunately it doesn't work... Those records that are without ISBN are removed.

Thanks for ideas in advance!


Fly

[kovidgoyal]

If you enable only a single metadata download plugin, that should take care fo it. Preferences->Metadata download

[FlyGoneCrazy]

It doesn't matter if there's only one source selected. Result is following:

Running identify query with parameters: 
{u'title': u'Balta drobul\u0117', u'authors': [u'Antanas \u0160k\u0117ma'], u'identifiers': {u'filename': u'Antanas.Skema.-.Balta.drobule.2005.LT.pdf', u'goodreads': u'22588213'}, u'timeout': 30} 
Using plugins: knygos (1, 0, 0) 
The log from individual plugins is below 

****************************** knygos (1, 0, 0) ****************************** 
Found 4 results 
Downloading from knygos took 5.59059000015 


--- 
Title               : Balta drobulė (Baltos lankos)
Author(s)           : Antanas Škėma (357452875)
Publisher           : Baltos lankos
Languages           : lt
Published           : 2013-07-10T00:00:00+00:00
Identifiers         : knygos:knygos/balta-drobule-mazos-dideles-knygos, isbn:9789955236627

--- 
Title               : Balta drobulė
Author(s)           : Antanas Škėma (865272187)
Publisher           : Žaltvykslė
Languages           : lt
Published           : 2009-07-10T00:00:00+00:00
Identifiers         : knygos:knygos/balta-drobule-9ij, isbn:9799986060825

--- 
Title               : Balta drobulė (Skaitinių serija 28)
Author(s)           : Antanas Škėma (575603359)
Publisher           : Baltos lankos
Languages           : lt
Published           : 1999-07-10T00:00:00+00:00
Identifiers         : knygos:knygos/balta-drobule--skaitiniu-serija-28-, isbn:9789986861980

--- 
Title               : Balta drobulė (1990)
Author(s)           : Antanas Škėma (210775480)
Publisher           : Lietuvos rašytojų sąjungos leidykla
Languages           : lt
Published           : 1990-07-10T00:00:00+00:00
Identifiers         : knygos:knygos/balta-drobule-syhushje
    
Title:Balta drobulė
Authors:[u'Antanas \u0160k\u0117ma']

SEARCH BASED ON TITLE AND AUTHOR:
	Querying: https://www.knygos.lt/lt/paieska?q=Balta%20drobul%C4%97 Antanas%20%C5%A0k%C4%97ma&psl=1
	4 results in 1 pages found
		Unique book #1 URL: 'https://www.knygos.lt/lt/knygos/balta-drobule-mazos-dideles-knygos/'
		Unique book #2 URL: 'https://www.knygos.lt/lt/knygos/balta-drobule--skaitiniu-serija-28-/'
		Unique book #3 URL: 'https://www.knygos.lt/lt/knygos/balta-drobule-syhushje/'
		Unique book #4 URL: 'https://www.knygos.lt/lt/knygos/balta-drobule-9ij/'
SEARCH BASED ON TITLE:
	Querying: https://www.knygos.lt/lt/paieska?q=Balta%20drobul%C4%97&psl=1
	4 results in 1 pages found
SEARCH BASED ON ISBN:
	Querying: https://www.knygos.lt/lt/paieska?q=Balta%20drobul%C4%97&psl=1
	4 results in 1 pages found 

******************************************************************************** 
The identify phase took 5.67 seconds 
The longest time (5.590590) was taken by: knygos 
Merging results from different sources 
We have 3 merged results, merging took: 0.00 seconds

After merging I have see attachment. A book release in 1990 is missing as it doesn't have ISBN (according to the log provided above).

[kovidgoyal]

yes if you have results from the same source some of which have isbns and some do not, then the metadata system will keep the isbn based results and discard the rest. This is not something that can be controlled, short of editing calibre source code.

[FlyGoneCrazy]

Well, my first thought was to generate fake random ISBN to avoid merging. Even though it have several disadvantage - can't merge results from different sources, even though they are the same. But it doesn't work. As I guess ISBN structure is checked...

That's why I've started this thread - maybe someone has some brilliant idea. As I've mentioned previously by adding random string to author (BTW before write they are get rid of by Calibre feature - create rules for author records) helped me to overcome this issue partly, when results with ISBN records were merged due to similarity in authors and titles.

Now I'm a bit stuck...

[kovidgoyal]

well the best solution to prevent merging is to modify the calibre metadata download system. If the identifiers dictionary has a key named 'calibre-no-merge' then the result should not be merged and the key should be discarded.

However, this is not really of interest to me, so patches are welcome.

[FlyGoneCrazy]

I do not want to turn off merging, as it's useful in case ISBN exist. My problem is only merging records without ISBN, or merging records from the same provider.

I'll think about patching calibre, however I'm not so good in this.

Talking about my problem, I've solved it by adding a list of 1000 random ISBNs. In case parsing ends up without ISBN, my plug-in picks one from the list. To be able to identify books with fake ISBN, I'm adding a tag !fake_isbn.

It's not the most elegant solution, but it works just fine for me.

The next thing to do - add option to configuration for turning this function on/off.

Case closed.