Maximum number of books? - Page 2

theducks · 06-05-2016, 08:40 PM

Quote:

Originally Posted by gridluck

calibre is stuck at 100%, with 9% of constant CPU usage, using 2.5GB ram, and little disk activity, what is it doing now?

Processing the completed Import books and detect metadata/cover generation phase .
Then it has to update the GUI (list), Then it starts building the recovery file in each title folder (~3 per second. Status is available on the Library Maintenance sub-menu )

A very good reason to add in small batches. A crash loses a lot of work

gridluck · 06-05-2016, 09:02 PM

Quote:

Originally Posted by theducks

Processing the completed Import books and detect metadata/cover generation phase .
Then it has to update the GUI (list), Then it starts building the recovery file in each title folder (~3 per second. Status is available on the Library Maintenance sub-menu )

A very good reason to add in small batches. A crash loses a lot of work

can I restart my pc at this point?

theducks · 06-05-2016, 09:42 PM

Quote:

Originally Posted by gridluck

can I restart my pc at this point?

NO
You will lose all the (effort for) things that have not posted (listed in the GUI) and need to start.again

OTOH it may crash (or not) and you will be in the same boat.

Roll them dice, spin the wheel

gridluck · 06-06-2016, 09:27 AM

Quote:

Originally Posted by theducks

NO
You will lose all the (effort for) things that have not posted (listed in the GUI) and need to start.again

OTOH it may crash (or not) and you will be in the same boat.

Roll them dice, spin the wheel

Ok, after 7+ days I finally got Calibre back with 80k+ duplicates..

so without them the library now is 256k+ books, I went to Library settings and clicked on Check Library, it started vacuuming something, hope it vacuums it well.. the database is 426mb, so far I don't see any issues yet, maybe just a bit slow at startup..

I clicked on search and entered a single word, it took about 30 seconds to look for it to spit out the result. Looking at DB size I believe this is a search on metadata only, are there any options or plugins maybe to enable full text search?

theducks · 06-06-2016, 09:49 AM

You saw my Sig

https://www.mobileread.com/forums/sho...d.php?t=118764

BTW Library maintenance works on what is there (DB or files). Vacuuming cleans out 'Deleted' (no longer visible) DB records. Run that after making lots of changes. AND never use the Filesystem explorer INSIDE a Library folder to make changes

Quality Check PI to help mash tings into a common (Author name) consistency
You have weeks of work ahead. Metadata takes TIME and Elbow Grease to get close to right

then
Find Duplicates PI will be one you need

There are lots of Plugins, because someone found a need to do something

Do your homework in the referenced listing to see what else you need

BetterRed · 06-06-2016, 05:27 PM

Quote:

Originally Posted by gridluck

I clicked on search and entered a single word, it took about 30 seconds to look for it to spit out the result. Looking at DB size I believe this is a search on metadata only, are there any options or plugins maybe to enable full text search?

@gridluck - The Quality Check PI can search EPUB contents, it is slow if you search dozens/hundreds of books.

I use several external index based content searchers in Windows - mainly X1, but also Windows File Explorer Search and File Locator, I then use the Drop Search Results PI (DSR) to Mark the 'books'; from there I can tag them, copy them, save them. put them in a reading list... or what ever. DSR is a boon for research.

The metadata search speed can be improved if you exclude Comments in Preferences->Searching->Columns that non-prefixed searches are limited to:

BR

bashour · 10-14-2017, 01:25 PM

Quote:

Originally Posted by kovidgoyal

My first computer was a ZX Spectrum as well, on which I wrote my first program -- which was to generate multiplication tables, so that I had an excuse not to have to memorize them, it output the tables in color and everything

My first computer as well - I loved that computer and the cassette tapes it used to load programs and the noise they made. Loved it.

Syllius · 07-04-2019, 10:54 PM

Heya, I think this is the right place... I have 500gb left on the SSD with Calibre on it and 100gb on the secondary HD that the books are being imported from. Here's the log. Hoping you can help! Thanks again for everything:

Spoiler:

kovidgoyal · 07-04-2019, 10:56 PM

make sure you have space on the disk with the temporary files directory as well.

Syllius · 07-04-2019, 10:57 PM

I get this error when I try to switch libraries after the failed import:

calibre, version 3.44.0
ERROR: Unhandled exception: <b>SQLError</b>:SQLError: no such savepoint: _apsw-0

calibre 3.44 embedded-python: True is64bit: False
Windows-10-10.0.17134-SP0 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '10', '10.0.17134')
Python 2.7.15+
Windows: ('10', '10.0.17134', 'SP0', u'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: Barnes & Noble (1, 2, 15) && Job Spy (1, 0, 159) && English Noun Frequency (1, 0, 10) && Clean Metadata (0, 0, 6) && Embed Comic Metadata (1, 3, 8) && Find Duplicates (1, 6, 3) && Comicvine (0, 11, 2) && Goodreads (1, 4, 0)
Traceback (most recent call last):
File "site-packages\calibre\gui2\__init__.py", line 479, in dispatch
File "site-packages\calibre\gui2\actions\choose_library.py", line 590, in switch_requested
File "site-packages\calibre\gui2\ui.py", line 725, in library_moved
File "site-packages\calibre\gui2\library\views.py", line 843, in set_database
File "site-packages\calibre\gui2\library\views.py", line 635, in save_state
File "site-packages\calibre\gui2\library\views.py", line 629, in write_state
File "site-packages\calibre\db\cache.py", line 62, in call_func_with_lock
File "site-packages\calibre\db\cache.py", line 616, in set_pref
File "site-packages\calibre\db\backend.py", line 134, in set
File "site-packages\calibre\db\backend.py", line 131, in __setitem__
SQLError: SQLError: no such savepoint: _apsw-0

Syllius · 07-04-2019, 10:58 PM

Quote:

Originally Posted by kovidgoyal

make sure you have space on the disk with the temporary files directory as well.

Is there a way to assign a temporary flash memory drive for it?

Syllius · 07-04-2019, 11:13 PM

Sorry to be a pain. Can't seem to do anything inside the program at the moment. Keep getting SQL error. I freed up more space, even uninstalled a few apps to make room on C:, but this is what I keep getting:

calibre, version 3.44.0
ERROR: Unhandled exception: <b>SQLError</b>:SQLError: no such savepoint: _apsw-0

calibre 3.44 embedded-python: True is64bit: False
Windows-10-10.0.17134-SP0 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '10', '10.0.17134')
Python 2.7.15+
Windows: ('10', '10.0.17134', 'SP0', u'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: Barnes & Noble (1, 2, 15) && Job Spy (1, 0, 159) && English Noun Frequency (1, 0, 10) && Clean Metadata (0, 0, 6) && Embed Comic Metadata (1, 3, 8) && Find Duplicates (1, 6, 3) && Comicvine (0, 11, 2) && Goodreads (1, 4, 0)
Traceback (most recent call last):
File "site-packages\calibre\gui2\actions\delete.py", line 403, in delete_books
File "site-packages\calibre\gui2\actions\delete.py", line 377, in do_library_delete
File "site-packages\calibre\gui2\library\models.py", line 415, in delete_books_by_id
File "site-packages\calibre\db\cache.py", line 62, in call_func_with_lock
File "site-packages\calibre\db\cache.py", line 1641, in remove_books
File "site-packages\calibre\db\backend.py", line 1666, in remove_books
File "site-packages\calibre\db\backend.py", line 886, in executemany
SQLError: SQLError: no such savepoint: _apsw-0

kovidgoyal · 07-04-2019, 11:15 PM

Either your disk does not have space, or it is failing, or something is interfering with calibre. Restart in safe mode and see if calibre works, to investigate the last possibility.

Syllius · 07-04-2019, 11:52 PM

You're a life saver, Sir. So I've verified it's not the drives, Calibre passed the library check in Safe Mode. Before I go adding a bunch more, is there a way to see the utilization on the temp file, maybe through the resource monitor? Also, would there be any way to utilize a secondary disk for the temp file? Thank you again so much for your dedication and your sincerity. I hope someday to contribute as selflessly to the community when I get a better bearing on the application structure.

BetterRed · 07-05-2019, 02:06 AM

Quote:

Originally Posted by Syllius

<snip>

Also, would there be any way to utilize a secondary disk for the temp file?

<snip>

On Windows, Calibre uses the default user temporary file directory: %LOCALAPPDATA%\Temp

To change that see CALIBRE_TEMP_DIR in ==>> Calibre User Manual: Environment variables

BR

07-04-2019, 10:54 PM	#23
Syllius Enthusiast Posts: 25 Karma: 10 Join Date: Jul 2019 Device: None	Heya, I think this is the right place... I have 500gb left on the SSD with Calibre on it and 100gb on the secondary HD that the books are being imported from. Here's the log. Hoping you can help! Thanks again for everything: Spoiler: calibre, version 3.44.0 ERROR: Errors while adding: Failed to add any books, click "Show details" for more information ---------------------------------------------------------------------- Failed to add the book: Book (3) D:\Complete\Comics\Books\Book (3).cbr With error: Traceback (most recent call last): File "site-packages\calibre\gui2\add.py", line 414, in add_book File "site-packages\calibre\db\legacy.py", line 225, in create_book_entry File "site-packages\calibre\db\cache.py", line 62, in call_func_with_lock File "site-packages\calibre\db\cache.py", line 1574, in create_book_entry File "site-packages\calibre\db\backend.py", line 874, in execute File "c:\t\t\apsw-zu_bhl\apsw-3.27.2-r1\src\cursor.c", line 236, in resetcursor FullError: FullError: database or disk is full ---------------------------------------------------------------------- Failed to add the book: Book (2) D:\Complete\Comics\Books\Book (2).cbr With error: Traceback (most recent call last): File "site-packages\calibre\gui2\add.py", line 414, in add_book File "site-packages\calibre\db\legacy.py", line 225, in create_book_entry File "site-packages\calibre\db\cache.py", line 62, in call_func_with_lock File "site-packages\calibre\db\cache.py", line 1574, in create_book_entry File "site-packages\calibre\db\backend.py", line 874, in execute File "c:\t\t\apsw-zu_bhl\apsw-3.27.2-r1\src\cursor.c", line 236, in resetcursor FullError: FullError: database or disk is full ---------------------------------------------------------------------- Failed to add the book: Book (4) D:\Complete\Comics\Books\Book (4).cbr With error: Traceback (most recent call last): File "site-packages\calibre\gui2\add.py", line 414, in add_book File "site-packages\calibre\db\legacy.py", line 225, in create_book_entry File "site-packages\calibre\db\cache.py", line 62, in call_func_with_lock File "site-packages\calibre\db\cache.py", line 1574, in create_book_entry File "site-packages\calibre\db\backend.py", line 874, in execute File "c:\t\t\apsw-zu_bhl\apsw-3.27.2-r1\src\cursor.c", line 236, in resetcursor FullError: FullError: database or disk is full ---------------------------------------------------------------------- Failed to add the book: Book (1) D:\Complete\Comics\Books\Book (1).cbr With error: Traceback (most recent call last): File "site-packages\calibre\gui2\add.py", line 414, in add_book File "site-packages\calibre\db\legacy.py", line 225, in create_book_entry File "site-packages\calibre\db\cache.py", line 62, in call_func_with_lock File "site-packages\calibre\db\cache.py", line 1574, in create_book_entry File "site-packages\calibre\db\backend.py", line 874, in execute File "c:\t\t\apsw-zu_bhl\apsw-3.27.2-r1\src\cursor.c", line 236, in resetcursor FullError: FullError: database or disk is full Last edited by theducks; 07-05-2019 at 09:23 AM. Reason: add spoiler tags

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Total Number of Books in Series	Coleccionista	Calibre	4	07-12-2017 03:24 PM
Maximum number of ebooks in Calibre?	Bullwinkle	Calibre	5	09-20-2010 03:13 PM
The greatest number of books on a liseuse	AprilHare	General Discussions	62	09-16-2010 10:27 AM
Maximum number of books on KOBO?	usern	Kobo Reader	19	05-19-2010 11:08 PM
Limit to number of books in Calibre?	ficbot	Calibre	1	02-26-2010 08:46 AM

06-06-2016, 09:49 AM	#20
theducks Well trained by Cats Posts: 29,795 Karma: 54830978 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	You saw my Sig https://www.mobileread.com/forums/sho...d.php?t=118764 BTW Library maintenance works on what is there (DB or files). Vacuuming cleans out 'Deleted' (no longer visible) DB records. Run that after making lots of changes. AND never use the Filesystem explorer INSIDE a Library folder to make changes Quality Check PI to help mash tings into a common (Author name) consistency You have weeks of work ahead. Metadata takes TIME and Elbow Grease to get close to right then Find Duplicates PI will be one you need There are lots of Plugins, because someone found a need to do something Do your homework in the referenced listing to see what else you need

07-04-2019, 10:56 PM	#24
kovidgoyal creator of calibre Posts: 43,850 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	make sure you have space on the disk with the temporary files directory as well.

07-04-2019, 10:57 PM	#25
Syllius Enthusiast Posts: 25 Karma: 10 Join Date: Jul 2019 Device: None	I get this error when I try to switch libraries after the failed import: calibre, version 3.44.0 ERROR: Unhandled exception: <b>SQLError</b>:SQLError: no such savepoint: _apsw-0 calibre 3.44 embedded-python: True is64bit: False Windows-10-10.0.17134-SP0 Windows ('32bit', 'WindowsPE') 32bit process running on 64bit windows ('Windows', '10', '10.0.17134') Python 2.7.15+ Windows: ('10', '10.0.17134', 'SP0', u'Multiprocessor Free') Interface language: None Successfully initialized third party plugins: Barnes & Noble (1, 2, 15) && Job Spy (1, 0, 159) && English Noun Frequency (1, 0, 10) && Clean Metadata (0, 0, 6) && Embed Comic Metadata (1, 3, 8) && Find Duplicates (1, 6, 3) && Comicvine (0, 11, 2) && Goodreads (1, 4, 0) Traceback (most recent call last): File "site-packages\calibre\gui2\__init__.py", line 479, in dispatch File "site-packages\calibre\gui2\actions\choose_library.py", line 590, in switch_requested File "site-packages\calibre\gui2\ui.py", line 725, in library_moved File "site-packages\calibre\gui2\library\views.py", line 843, in set_database File "site-packages\calibre\gui2\library\views.py", line 635, in save_state File "site-packages\calibre\gui2\library\views.py", line 629, in write_state File "site-packages\calibre\db\cache.py", line 62, in call_func_with_lock File "site-packages\calibre\db\cache.py", line 616, in set_pref File "site-packages\calibre\db\backend.py", line 134, in set File "site-packages\calibre\db\backend.py", line 131, in __setitem__ SQLError: SQLError: no such savepoint: _apsw-0

07-04-2019, 11:13 PM	#27
Syllius Enthusiast Posts: 25 Karma: 10 Join Date: Jul 2019 Device: None	Sorry to be a pain. Can't seem to do anything inside the program at the moment. Keep getting SQL error. I freed up more space, even uninstalled a few apps to make room on C:, but this is what I keep getting: calibre, version 3.44.0 ERROR: Unhandled exception: <b>SQLError</b>:SQLError: no such savepoint: _apsw-0 calibre 3.44 embedded-python: True is64bit: False Windows-10-10.0.17134-SP0 Windows ('32bit', 'WindowsPE') 32bit process running on 64bit windows ('Windows', '10', '10.0.17134') Python 2.7.15+ Windows: ('10', '10.0.17134', 'SP0', u'Multiprocessor Free') Interface language: None Successfully initialized third party plugins: Barnes & Noble (1, 2, 15) && Job Spy (1, 0, 159) && English Noun Frequency (1, 0, 10) && Clean Metadata (0, 0, 6) && Embed Comic Metadata (1, 3, 8) && Find Duplicates (1, 6, 3) && Comicvine (0, 11, 2) && Goodreads (1, 4, 0) Traceback (most recent call last): File "site-packages\calibre\gui2\actions\delete.py", line 403, in delete_books File "site-packages\calibre\gui2\actions\delete.py", line 377, in do_library_delete File "site-packages\calibre\gui2\library\models.py", line 415, in delete_books_by_id File "site-packages\calibre\db\cache.py", line 62, in call_func_with_lock File "site-packages\calibre\db\cache.py", line 1641, in remove_books File "site-packages\calibre\db\backend.py", line 1666, in remove_books File "site-packages\calibre\db\backend.py", line 886, in executemany SQLError: SQLError: no such savepoint: _apsw-0

07-04-2019, 11:15 PM	#28
kovidgoyal creator of calibre Posts: 43,850 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Either your disk does not have space, or it is failing, or something is interfering with calibre. Restart in safe mode and see if calibre works, to investigate the last possibility.

07-04-2019, 11:52 PM	#29
Syllius Enthusiast Posts: 25 Karma: 10 Join Date: Jul 2019 Device: None	You're a life saver, Sir. So I've verified it's not the drives, Calibre passed the library check in Safe Mode. Before I go adding a bunch more, is there a way to see the utilization on the temp file, maybe through the resource monitor? Also, would there be any way to utilize a secondary disk for the temp file? Thank you again so much for your dedication and your sincerity. I hope someday to contribute as selflessly to the community when I get a better bearing on the application structure.