[GUI Plugin] Import List - Page 10

davidfor · 09-16-2015, 07:54 PM

Unfortunately, kiwidude doesn't have much time to devote to updating the plugins. If someone wants to fix something, or add function, he will look at them and release them if he feels they are appropriate to the plugin.

DreamStatic · 09-16-2015, 10:00 PM

Thanks for the info, I appreciate it.

yonkyunior · 12-11-2015, 01:28 PM

Sorry to bother anyone with this stupid question,
but i dont have a clue about it, the plugin was successfully installed in my 2.40 Calibre, But i dont recognize it in toolbar, optional toolbar, context menu..
how to use it when i dont see the button and trigger the wizard window?

In Plugins
Uuser Interface Actions Plugins the Import List was there.
but the button or anything to trigger dont showing up.

i checked a dozens of time, is this a bug?

theducks · 12-11-2015, 01:36 PM

Not a bug

You just need to tell Calibre where you want the Icon

Preference: Toolbar: <select>: find PI icon on Left, use middle arrow button to move to right: move up as desired

Repeat for each toolbar to taste

IMHO it does not make much sense for this one on a 'When connected' toolbar)

canpolat · 01-23-2016, 07:14 AM

Earlier today, I added a plugin idea (link), but then when thinking about BetterRed's response and possible solutions, I did some more searching an found this plugin. Let me copy-paste the scenario to save you the click:

I export my Calibre library (not the books, only the data stored at Calibre database) and send it to my friend.
She imports that library as a separate book-less library. Then she starts working on fixing the tags, genres, etc.
Once she is ready with fixing, she exports the library and sends it back to me.
When I get the library and importing the differences to my existing library. The plugin prompts me with changes and I accept them (or deny some of them) and the merging happens.
The books with changed metadata are marked in Calibre so that I can work on them (embed metadata, etc.)

It looks like Import List has most of the underlying functionality implemented so I thought maybe I could suggest an extension. Here is my idea:

When importing from a CSV file, Import List asks which column corresponds to which field and then if the books are not found in the library, the user must select them create empty books for them. If Import List also had an export function where the user can select which fields to export, and create a zip file with a CSV file and the related configuration file in it, this could turn out to be the best (only?) export/import library solution for Calibre.

Export would create these two files (CSV and a configuration which describes which columns of the CSV correspond to which fields in Calibe, and possibly some other options), zip them and possibly change the extension to something specific.
Import would have check boxes to: import and overwrite all local data, import but manually merge every single change (also pick which fields to merge).

What do you think?

BetterRed · 01-23-2016, 08:32 AM

Yes, I thought of Import List and CSV's about the same time as yourself - see other thread.

The author of Import List is inactive, so you probably need to find someone who is willing to tackle changing the plugin - and guaranteeing it will still work for all those who are currently using it.

BR

davidfor · 01-23-2016, 08:35 AM

If you want to export the metadata from the calibre library, you can create a catalog. Right-click on the conversion button and choose the catalog option.

For the import, I'm reasonable certain you have described how the import list works. It can create books if there are no matches, and there is some review function. My use of it has been very basic, so I can't really say exactly how it works.

The best thing you can do is to experiment. Copy your library to another, create a csv file that matches what you want and import it. Once you have played a bit, you can work out if the plugin will work for you, or if it needs changes. And you will be better able to state what the changes needed are.

Having said that, I don't think this is how I would do what you are doing. I would probably just exchange the whole library minus the books. You can easily zip the the library directory but exclude any books. That can be sent to the person doing the maintenance. They can add a tag for the changed books, or you can use the modified date to see what was changed. Merging the changes back is a bit harder. Maybe an export and import at that time is the way to do it. It might be worthwhile looking at DaltonT's plugins. Most of them are about fixing metadata and libraries, so they might help.

canpolat · 01-23-2016, 09:24 AM

BetterRed, davidfor:Thanks for your replies. I started playing with Import List. The problem I face is: it is taking forever to import 4 fields from ~6500 books. It has been running for around an hour now and the import still did not finish. So, I assume it got stuck. Bad luck!

davidfor: About zipping the library directory... BetterRed also suggested that but I thought it would be more difficult for my friend to work that way. But now that both of you are suggesting the same thing (and Import List is stuck), I think I will reevalute this idea. So this is basically zipping and unzipping metadata_db_prefs_backup.json and metadata.db, isn't it?

theducks · 01-23-2016, 09:31 AM

Quote:

Originally Posted by canpolat

BetterRed, davidfor:Thanks for your replies. I started playing with Import List. The problem I face is: it is taking forever to import 4 fields from ~6500 books. It has been running for around an hour now and the import still did not finish. So, I assume it got stuck. Bad luck!

davidfor: About zipping the library directory... BetterRed also suggested that but I thought it would be more difficult for my friend to work that way. But now that both of you are suggesting the same thing (and Import List is stuck), I think I will reevalute this idea. So this is basically zipping and unzipping metadata_db_prefs_backup.json and metadata.db, isn't it?

Prefs_backup is used only to RESTORE a Library structure

canpolat · 01-23-2016, 10:33 AM

It looks like sharing the metadata.db will work. It doesn't seem to have the book covers, but should be good enough. Thank you all for your help.

BetterRed · 01-23-2016, 03:26 PM

Quote:

Originally Posted by canpolat

BetterRed, davidfor:Thanks for your replies. I started playing with Import List. The problem I face is: it is taking forever to import 4 fields from ~6500 books. It has been running for around an hour now and the import still did not finish. So, I assume it got stuck. Bad luck!

I assume you were importing into an empty library cloned from your actual library - yes/no? On what evidence did you make the judgement that Import List PI was 'stuck'?

The process of importing a ~6500 row CSV into an empty database would create thousands of Author and Book folders, ~6500 metdata.opf files in the book folders, and write many more thousands of rows to dozens of tables in the database? Doesn't surprise me that it wasn't finished after an hour.

Quote:

Originally Posted by canpolat

davidfor: About zipping the library directory... BetterRed also suggested that but I thought it would be more difficult for my friend to work that way. But now that both of you are suggesting the same thing (and Import List is stuck), I think I will reevalute this idea. So this is basically zipping and unzipping metadata_db_prefs_backup.json and metadata.db, isn't it?

The unexpressed reasons I suggested that you create a 'book-less' copy of your library (i.e. one with no format files) were

obviously to reduce the amount of data needing to be transferred to your friend, and less obviously
to protect you from breaching the copyright terms of your books, and
to make sure your friend was starting out with a recoverable library. Which means the metadata_db_prefs_backup.json would be included in the archive.

A 'bonus' would be that your friend wouldn't be looking at cover-less (boring) library. And the metdata.opfs, which are a recovery source would be in the book folders. And your friend might also be able to source 'better' covers than you already have.

I urge you to create a subset of your library with a couple of hundred books and experiment with that - in the long run it will save you time.

BR

canpolat · 01-23-2016, 03:51 PM

Quote:

Originally Posted by BetterRed

I assume you were importing into an empty library cloned from your actual library - yes/no? On what evidence did you make the judgement that Import List PI was 'stuck'?

The process of importing a ~6500 row CSV into an empty database would create thousands of Author and Book folders, ~6500 metdata.opf files in the book folders, and write many more thousands of rows to dozens of tables in the database? Doesn't surprise me that it wasn't finished after an hour.

The unexpressed reasons I suggested that you create a 'book-less' copy of your library (i.e. one with no format files) were

obviously to reduce the amount of data needing to be transferred to your friend, and less obviously
to protect you from breaching the copyright terms of your books, and
to make sure your friend was starting out with a recoverable library. Which means the metadata_db_prefs_backup.json would be included in the archive.

A 'bonus' would be that your friend wouldn't be looking at cover-less (boring) library. And the metdata.opfs, which are a recovery source would be in the book folders. And your friend might also be able to source 'better' covers than you already have.

I urge you to create a subset of your library with a couple of hundred books and experiment with that - in the long run it will save you time.

BR

Yes, I was cloning a CSV into an empty library. I don't really have any evidence about Import List being stuck (Calibre was unresponsive for an hour or so before I killed it), but I think I don't really get the way Calibre handles this situation. Now that you explain it, I understand all the folders are necessary even though she will be only working on metadata which is stored in the DB file. That basically tells me, perhaps Calibre is not the tool we are looking for. Or, more precisely, we probably need a tool that can work on the calibre DB file without needing to create all those empty folders (seeing the book covers would be nice, but the toll is too high).

BetterRed · 01-23-2016, 05:05 PM

Quote:

Originally Posted by canpolat

Yes, I was cloning a CSV into an empty library. I don't really have any evidence about Import List being stuck (Calibre was unresponsive for an hour or so before I killed it), but I think I don't really get the way Calibre handles this situation. Now that you explain it, I understand all the folders are necessary even though she will be only working on metadata which is stored in the DB file. That basically tells me, perhaps Calibre is not the tool we are looking for. Or, more precisely, we probably need a tool that can work on the calibre DB file without needing to create all those empty folders (seeing the book covers would be nice, but the toll is too high).

The toll on what?

I just copied 100 hundred books out of one library to an empty one. Each book had 2-4 format files (ePUB, DOCX, PDF etc) the library was 84MB, after removing all the format files it was 5MB - so about 5KB per book, most of which is jpegs which don't compress much. So, I removed the 100 cover.jpg files, the folder was now 1.2MB - which compressed down to 300KB

So again, a toll on what?

None of my 'unexpressed reasons' for recommending you send a book-less library are 'obligatory'.

So yes, your friend could work from a library that started off with only the metadata.db. But as she updates the database, the author and book folders will be created; calibre needs them as the place to write its recovery data - i.e. the metadata.opf file you see in every book folder - which is obligatory. So she will be working with a database that is only partially recoverable.

BR

theducks · 01-23-2016, 06:09 PM

BR's point about folder names changing if either Title or Author get touched.

Depending on your situation and both of your locations:
KISS
Have her remote access your machine and fix things there instead of all the gyrations you might need otherwise (still: BACK UP.BACK UP.BACK UP)
Reasonable performance is had on almost any DSL.

VNC has clients that work across all 3 platforms.

If you have a 'Pro' version of Windows: Remote Desktop connection has clients (free) for many platforms (I have one on my Android Tablet

the screen is TINY, but it is there)

PC and Net (LAN) Security are Trust considerations. (Got a older system? Strip it clean of sensitive stuff, Install Calibre there and sneakernet the Library onto it. )
This solution does not count as a LAN/NAS drive the real work is done on the Host PC, not the remote

yonkyunior · 01-27-2016, 11:17 AM

is it just me or bug?
i am using it to grab goodreads listopia
when i turns to typing the page 2 of the list
I put the ?Page=2 instead ?page=2 in the end of the url,
but the ?Page=2 turns out to grab the page one instead of page 2 of listopia,

i dont know its the capital letters/uppercase matters in the goodreads for result in importlist plugin, but when i want to change it to lowercase letter..
the history in the dropdown change de lowercase to uppercase. and i can't remove the history of the links that have been inputted.

12-11-2015, 01:28 PM	#138
yonkyunior Cultivator Posts: 94 Karma: 216 Join Date: Feb 2015 Device: PRST2	How to trigger it? BUG? Sorry to bother anyone with this stupid question, but i dont have a clue about it, the plugin was successfully installed in my 2.40 Calibre, But i dont recognize it in toolbar, optional toolbar, context menu.. how to use it when i dont see the button and trigger the wizard window? In Plugins Uuser Interface Actions Plugins the Import List was there. but the button or anything to trigger dont showing up. i checked a dozens of time, is this a bug?

01-23-2016, 07:14 AM	#140
canpolat Connoisseur Posts: 92 Karma: 17950 Join Date: Mar 2013 Device: Xodo	Earlier today, I added a plugin idea (link), but then when thinking about BetterRed's response and possible solutions, I did some more searching an found this plugin. Let me copy-paste the scenario to save you the click: I export my Calibre library (not the books, only the data stored at Calibre database) and send it to my friend. She imports that library as a separate book-less library. Then she starts working on fixing the tags, genres, etc. Once she is ready with fixing, she exports the library and sends it back to me. When I get the library and importing the differences to my existing library. The plugin prompts me with changes and I accept them (or deny some of them) and the merging happens. The books with changed metadata are marked in Calibre so that I can work on them (embed metadata, etc.) It looks like Import List has most of the underlying functionality implemented so I thought maybe I could suggest an extension. Here is my idea: When importing from a CSV file, Import List asks which column corresponds to which field and then if the books are not found in the library, the user must select them create empty books for them. If Import List also had an export function where the user can select which fields to export, and create a zip file with a CSV file and the related configuration file in it, this could turn out to be the best (only?) export/import library solution for Calibre. Export would create these two files (CSV and a configuration which describes which columns of the CSV correspond to which fields in Calibe, and possibly some other options), zip them and possibly change the extension to something specific. Import would have check boxes to: import and overwrite all local data, import but manually merge every single change (also pick which fields to merge). What do you think? Last edited by canpolat; 01-23-2016 at 07:40 AM. Reason: Typo

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[GUI Plugin] Reading List	kiwidude	Plugins	1315	04-11-2024 12:37 AM
[GUI Plugin] Manage Sony x50 Reader Book List	kpw	Plugins	170	10-02-2014 08:23 PM
Import List plugin idea thread	kiwidude	Development	45	08-13-2012 06:05 AM
[GUI Plugin] WebOS Kindle-Import	CranstD	Plugins	0	01-24-2012 03:36 PM
[GUI Plugin] Plugin Updater Deprecated	kiwidude	Plugins	159	06-19-2011 12:27 PM

09-16-2015, 07:54 PM	#136
davidfor Grand Sorcerer Posts: 24,907 Karma: 47303748 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	Unfortunately, kiwidude doesn't have much time to devote to updating the plugins. If someone wants to fix something, or add function, he will look at them and release them if he feels they are appropriate to the plugin.

09-16-2015, 10:00 PM	#137
DreamStatic Enthusiast Posts: 31 Karma: 10 Join Date: Aug 2014 Device: iphone, ipad, computer	Thanks for the info, I appreciate it.

12-11-2015, 01:36 PM	#139
theducks Well trained by Cats Posts: 29,782 Karma: 54830978 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	Not a bug You just need to tell Calibre where you want the Icon Preference: Toolbar: <select>: find PI icon on Left, use middle arrow button to move to right: move up as desired Repeat for each toolbar to taste IMHO it does not make much sense for this one on a 'When connected' toolbar)

01-23-2016, 08:32 AM	#141
BetterRed null operator (he/him) Posts: 20,553 Karma: 26954694 Join Date: Mar 2012 Location: Sydney Australia Device: none	Yes, I thought of Import List and CSV's about the same time as yourself - see other thread. The author of Import List is inactive, so you probably need to find someone who is willing to tackle changing the plugin - and guaranteeing it will still work for all those who are currently using it. BR

01-23-2016, 08:35 AM	#142
davidfor Grand Sorcerer Posts: 24,907 Karma: 47303748 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	If you want to export the metadata from the calibre library, you can create a catalog. Right-click on the conversion button and choose the catalog option. For the import, I'm reasonable certain you have described how the import list works. It can create books if there are no matches, and there is some review function. My use of it has been very basic, so I can't really say exactly how it works. The best thing you can do is to experiment. Copy your library to another, create a csv file that matches what you want and import it. Once you have played a bit, you can work out if the plugin will work for you, or if it needs changes. And you will be better able to state what the changes needed are. Having said that, I don't think this is how I would do what you are doing. I would probably just exchange the whole library minus the books. You can easily zip the the library directory but exclude any books. That can be sent to the person doing the maintenance. They can add a tag for the changed books, or you can use the modified date to see what was changed. Merging the changes back is a bit harder. Maybe an export and import at that time is the way to do it. It might be worthwhile looking at DaltonT's plugins. Most of them are about fixing metadata and libraries, so they might help.

01-23-2016, 09:24 AM	#143
canpolat Connoisseur Posts: 92 Karma: 17950 Join Date: Mar 2013 Device: Xodo	BetterRed, davidfor:Thanks for your replies. I started playing with Import List. The problem I face is: it is taking forever to import 4 fields from ~6500 books. It has been running for around an hour now and the import still did not finish. So, I assume it got stuck. Bad luck! davidfor: About zipping the library directory... BetterRed also suggested that but I thought it would be more difficult for my friend to work that way. But now that both of you are suggesting the same thing (and Import List is stuck), I think I will reevalute this idea. So this is basically zipping and unzipping metadata_db_prefs_backup.json and metadata.db, isn't it?

01-23-2016, 10:33 AM	#145
canpolat Connoisseur Posts: 92 Karma: 17950 Join Date: Mar 2013 Device: Xodo	It looks like sharing the metadata.db will work. It doesn't seem to have the book covers, but should be good enough. Thank you all for your help.

01-23-2016, 06:09 PM	#149
theducks Well trained by Cats Posts: 29,782 Karma: 54830978 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	BR's point about folder names changing if either Title or Author get touched. Depending on your situation and both of your locations: KISS Have her remote access your machine and fix things there instead of all the gyrations you might need otherwise (still: BACK UP.BACK UP.BACK UP) Reasonable performance is had on almost any DSL. VNC has clients that work across all 3 platforms. If you have a 'Pro' version of Windows: Remote Desktop connection has clients (free) for many platforms (I have one on my Android Tablet the screen is TINY, but it is there) PC and Net (LAN) Security are Trust considerations. (Got a older system? Strip it clean of sensitive stuff, Install Calibre there and sneakernet the Library onto it. ) This solution does not count as a LAN/NAS drive the real work is done on the Host PC, not the remote

01-27-2016, 11:17 AM	#150
yonkyunior Cultivator Posts: 94 Karma: 216 Join Date: Feb 2015 Device: PRST2	is it just me or bug? i am using it to grab goodreads listopia when i turns to typing the page 2 of the list I put the ?Page=2 instead ?page=2 in the end of the url, but the ?Page=2 turns out to grab the page one instead of page 2 of listopia, i dont know its the capital letters/uppercase matters in the goodreads for result in importlist plugin, but when i want to change it to lowercase letter.. the history in the dropdown change de lowercase to uppercase. and i can't remove the history of the links that have been inputted.

Advert

Advert