Vanishing auto add books during import, books left behind in auto add folder

jataadroid · 01-13-2018, 04:08 PM

Running latest calibre on win 10.

When auto adding books, i find alot of empty folder duplicates. I search for duplicates and find the original book and duplicate empty folder. Out of 8000 books added 3500 were empty folder duplicates...which leads me to my next issue. Out of those 8000 imports, once imported there were 3500 duplicate folder. Once i removed those duplicate my library shrank to 4500......what happened to my books/files then? I had 8000 before, some how 3500 got duplicated, 1000 did just fine, and 3500 just disappeared.

When i go back to the source folder, there are 800 or remaining books that weren't auto added.
The numbers don't add up and im left wondering where my files went.
And why did i have 800 files left that werent imported? They were under 255 char file names, source folder worked fine adding the other books....why not these lol.

I can manually add the books via add books>from a directory and it will import some of them but it still leaves some behind...then theres no way to tell whats already been imported.

Im hoping i didnt lose data through this process. Ive really liked calibre and the community. Hoping to get some help.

Sending from my phone, sorry for the hasty sloppy posting. The numbers were approximations.

DaltonST · 01-13-2018, 04:41 PM

jataadroid:

1. Go to Preferences > Adding Books

2. Click the "Adding Actions" tab, and set the option for Auto-Merging as you prefer.

3. Click the "Automatic Adding" tab, and set the option for Duplicate Checking as you prefer, being consistent with your Auto-Merging preference.

4. Look at your Auto-Add folder, and read the note just above it. Added books are deleted. Refer to #2 and #3, above.

5. Only the ebooks within that folder itself are automatically added. No subordinate (child) folders are processed whatsoever. It is not a recursive addition process. It would be too dangerous; think about how someone might point it to their C:\ drive root directory, or their Calibre Library root directory.

DaltonST

jataadroid · 01-13-2018, 07:17 PM

Thank you for the reply.

Auto merging may be the culprit for the imported files not equalling the files in the original source folder. How does that explain the empty folders and left over files not added? I know its something I'm overlooking but I've tried to troubleshoot where I could and came up with nothing.

There are no sub folder in the source folder.

You've given me some things to look into, so I'll check back with my findings.

DaltonST · 01-13-2018, 07:42 PM

If there are no subfolders, then there cannot be any empty folders except possibly for the single auto-add folder.

Only valid book formats will be added. The unadded books might not be valid books, regardless of the book file extension. Test some of them using a book viewer.

DaltonST

BetterRed · 01-13-2018, 11:45 PM

Quote:

Originally Posted by DaltonST

Only valid book formats will be added.

Uh-huh - there is a check box in the Autoadd options labelled - Automatically add files of all types (not just e-books)

Quote:

By default, calibre will automatically add only known e-book files. This option tells calibre to add all file types, except those that have been explicitly ignored below.

BR

DaltonST · 01-13-2018, 11:59 PM

In my first post I did not instruct her to click that checkbox.

BetterRed · 01-14-2018, 12:15 AM

Quote:

Originally Posted by DaltonST

In my first post I did not instruct her to click that checkbox.

Nor did you instruct the OP not to set it.

It may not matter to the thread starter, but someone reading post #4 six months from now as the result of a search could read your statement as implying Autoadd will only add ebook file types.

BR

jataadroid · 01-14-2018, 12:45 PM

thanks for the replies.

I have looked into it and still cant determine the issue based on what you have said so far.
Auto merge added books is not enabled, auto add is not set to "all types",

I have checked the empty folder with the book viewer and file explorer. There is indeed no file in those library entries.

I failed to backup the originial database so i cant compare unfortunately.
Ive just come to terms with the fact that I cant bulk import, I'll have to break it into smaller chunks.

I'll try to replicate the situation and document it for a bug report.

BetterRed · 01-14-2018, 03:39 PM

Quote:

Originally Posted by jataadroid

Ive just come to terms with the fact that I cant bulk import, I'll have to break it into smaller chunks.

I'll try to replicate the situation and document it for a bug report.

How many books did you have in the Autoadd folder. IIRC when I first built my libraries I added books in batches of around 500, but not from Autoadd.

See Kovid's responses in this thread ==>> more workers to speed up autoadd?

BR

jataadroid · 01-14-2018, 11:43 PM

very valid point. who knows how much I lost but I cant do anything about it now.

thanks for the post reference.

01-13-2018, 04:08 PM	#1
jataadroid Junior Member Posts: 8 Karma: 10 Join Date: Dec 2017 Device: Android Tablet - Moonreader Pro	Vanishing auto add books during import, books left behind in auto add folder Running latest calibre on win 10. When auto adding books, i find alot of empty folder duplicates. I search for duplicates and find the original book and duplicate empty folder. Out of 8000 books added 3500 were empty folder duplicates...which leads me to my next issue. Out of those 8000 imports, once imported there were 3500 duplicate folder. Once i removed those duplicate my library shrank to 4500......what happened to my books/files then? I had 8000 before, some how 3500 got duplicated, 1000 did just fine, and 3500 just disappeared. When i go back to the source folder, there are 800 or remaining books that weren't auto added. The numbers don't add up and im left wondering where my files went. And why did i have 800 files left that werent imported? They were under 255 char file names, source folder worked fine adding the other books....why not these lol. I can manually add the books via add books>from a directory and it will import some of them but it still leaves some behind...then theres no way to tell whats already been imported. Im hoping i didnt lose data through this process. Ive really liked calibre and the community. Hoping to get some help. Sending from my phone, sorry for the hasty sloppy posting. The numbers were approximations.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Auto Add Books	Paltieldav	Library Management	6	03-04-2016 12:53 PM
Auto-add imported folder to collection with same name	truth1ness	Library Management	4	05-10-2015 11:59 PM
Auto-Add Metadata When Adding Books	pghaworth	Calibre	8	11-11-2014 01:59 PM
Is this possible: Auto-add book/magazine then auto e-mail to device?	runningwithbulls	Library Management	5	09-10-2012 12:27 PM
Help with new feature: Auto-Add Folder	kbullkar	Library Management	12	03-30-2012 06:15 AM

01-13-2018, 04:41 PM	#2
DaltonST Deviser Posts: 2,265 Karma: 2090983 Join Date: Aug 2013 Location: Texas Device: none	jataadroid: 1. Go to Preferences > Adding Books 2. Click the "Adding Actions" tab, and set the option for Auto-Merging as you prefer. 3. Click the "Automatic Adding" tab, and set the option for Duplicate Checking as you prefer, being consistent with your Auto-Merging preference. 4. Look at your Auto-Add folder, and read the note just above it. Added books are deleted. Refer to #2 and #3, above. 5. Only the ebooks within that folder itself are automatically added. No subordinate (child) folders are processed whatsoever. It is not a recursive addition process. It would be too dangerous; think about how someone might point it to their C:\ drive root directory, or their Calibre Library root directory. DaltonST

01-13-2018, 07:17 PM	#3
jataadroid Junior Member Posts: 8 Karma: 10 Join Date: Dec 2017 Device: Android Tablet - Moonreader Pro	Thank you for the reply. Auto merging may be the culprit for the imported files not equalling the files in the original source folder. How does that explain the empty folders and left over files not added? I know its something I'm overlooking but I've tried to troubleshoot where I could and came up with nothing. There are no sub folder in the source folder. You've given me some things to look into, so I'll check back with my findings.

01-13-2018, 07:42 PM	#4
DaltonST Deviser Posts: 2,265 Karma: 2090983 Join Date: Aug 2013 Location: Texas Device: none	If there are no subfolders, then there cannot be any empty folders except possibly for the single auto-add folder. Only valid book formats will be added. The unadded books might not be valid books, regardless of the book file extension. Test some of them using a book viewer. DaltonST

01-13-2018, 11:59 PM	#6
DaltonST Deviser Posts: 2,265 Karma: 2090983 Join Date: Aug 2013 Location: Texas Device: none	In my first post I did not instruct her to click that checkbox.

01-14-2018, 12:45 PM	#8
jataadroid Junior Member Posts: 8 Karma: 10 Join Date: Dec 2017 Device: Android Tablet - Moonreader Pro	thanks for the replies. I have looked into it and still cant determine the issue based on what you have said so far. Auto merge added books is not enabled, auto add is not set to "all types", I have checked the empty folder with the book viewer and file explorer. There is indeed no file in those library entries. I failed to backup the originial database so i cant compare unfortunately. Ive just come to terms with the fact that I cant bulk import, I'll have to break it into smaller chunks. I'll try to replicate the situation and document it for a bug report.

01-14-2018, 11:43 PM	#10
jataadroid Junior Member Posts: 8 Karma: 10 Join Date: Dec 2017 Device: Android Tablet - Moonreader Pro	very valid point. who knows how much I lost but I cant do anything about it now. thanks for the post reference.