Updating database on Kobo Forma

[maddz]

I've been doing a lot of cleaning up in calibre, and now need to get the cleaned up (and restructured) files onto my Forma. This obvious way to do this is a factory reset and then sideload in batches - which will take ages for around 6,000 books.

Is there a better way of cleaning the database on the Forma, deleting any files not now in the database and only transferring the updated files?

[theducks]

Originally Posted by maddz

I've been doing a lot of cleaning up in calibre, and now need to get the cleaned up (and restructured) files onto my Forma. This obvious way to do this is a factory reset and then sideload in batches - which will take ages for around 6,000 books.

Is there a better way of cleaning the database on the Forma, deleting any files not now in the database and only transferring the updated files?

Kobo Utilities (and extended edition has tools to UPDATE the Kobo metadata, TOC (in the Kobo).
Give that a spin before you take the massive Nuke everything path

Those utilities have way more than I have even tried. Look them over

[maddz]

It's not the metadata/TOC, it's the actual database and what will end up as orphan files as the file structure has changed. My understanding is that although the Forma 'sees' the current version on the Forma, when it syncs with Calibre, the updated file isn't recognised as being on the device.

The trouble is that I've cleaned up every file in Calibre which is why in some ways nuking everything and starting over is potentially a good idea.

[JSWolf]

No need to reset your Forma. Plug your Forum into your computer with calibre running and delete your content using calibre. It's the Library (I think) icon on the toolbar with your Forma plugged in. You can use Kobo Utilities to compress the database on the Forma and then you can copy what you want over to your Forma.

[maddz]

Wonderful. It's going to have to be the nuke option; I deleted everything, got nearly half-way through transferring, now it says the Kobo is full.

Which is complete BS. I've barely got 10GB of files to send. I suspect it thinks there's still large files on the device which is fair comment: I did have large work-related PDFs on it (transferred via calibre) which I don't plan resending; if I need them again, I'll pick them up on the Elipsa.

[JSWolf]

Originally Posted by maddz

Wonderful. It's going to have to be the nuke option; I deleted everything, got nearly half-way through transferring, now it says the Kobo is full.

Which is complete BS. I've barely got 10GB of files to send. I suspect it thinks there's still large files on the device which is fair comment: I did have large work-related PDFs on it (transferred via calibre) which I don't plan resending; if I need them again, I'll pick them up on the Elipsa.

Plug the Kobo into the USB and have a look at what's on it. If it is PDF, they should have shown up when you went into calibre to delete all the content.

[DNSB]

Originally Posted by maddz

Wonderful. It's going to have to be the nuke option; I deleted everything, got nearly half-way through transferring, now it says the Kobo is full.

Is this the Forma with the 8GB of internal storage or the Forma with the 32GB of internal storage?

Are you transferring the files into a subdirectory in the exposed storage of the Forma or in the root directory?

[maddz]

Originally Posted by DNSB

Is this the Forma with the 8GB of internal storage or the Forma with the 32GB of internal storage?

Are you transferring the files into a subdirectory in the exposed storage of the Forma or in the root directory?

32GB and I send to main.

[DNSB]

Originally Posted by maddz

32GB and I send to main.

Not sure if you mean a subdirectory called main or the root directory. If you are sending to the root directory, you can and most likely will, run into the limit on the number of directory entries in the root. This issue is aggravated by long file names chewing up multiple directory entries. A long file name uses 1 directory entry for the short filename plus a directory entry for every 13 characters in the filename rounded up.

What is the Send to Device template set to in your KoboTouch driver configuration?

[maddz]

Books are sent to the top level directory on the Forma, the same level as the hidden folders. Should I be sending to a different directory? I've got 7780 individual books that are sent.

Do you mean the save template in calibre export preferences? It's set to: {author_sort}/{series}/{series}-{series_index:>2s} {#subseries} {#subseries_index:>2s}:{title} - {authors}

The save template in Kobo Utilities for Kobo Touch Extended is {title} - {authors} - {series}

[JSWolf]

Originally Posted by maddz

Books are sent to the top level directory on the Forma, the same level as the hidden folders. Should I be sending to a different directory? I've got 7780 individual books that are sent.

Do you mean the save template in calibre export preferences? It's set to: {author_sort}/{series}/{series}-{series_index:>2s} {#subseries} {#subseries_index:>2s}:{title} - {authors}

The save template in Kobo Utilities for Kobo Touch Extended is {title} - {authors} - {series}

That will eat your available entries in the root directory. Do this...

books/{series}-{series_index:>2s} {#subseries}

A subdirectory such as books can dynamically allocate more space as needed up to 4GB. So you'd have plenty of space for your filenames.

[maddz]

Where? Calibre preferences or KoboTouch Extended? It'll have to wait until the other Forma is back, repaired or not.

[JSWolf]

Originally Posted by maddz

Where? Calibre preferences or KoboTouch Extended? It'll have to wait until the other Forma is back, repaired or not.

In the KoboTouch driver settings.

[DNSB]

Originally Posted by maddz

Books are sent to the top level directory on the Forma, the same level as the hidden folders. Should I be sending to a different directory? I've got 7780 individual books that are sent.

Do you mean the save template in calibre export preferences? It's set to: {author_sort}/{series}/{series}-{series_index:>2s} {#subseries} {#subseries_index:>2s}:{title} - {authors}

The save template in Kobo Utilities for Kobo Touch Extended is {title} - {authors} - {series}

The template I am using for the KoboTouch driver (The KoboTouchExtended driver can no longer be used since the KoboTouch driver was extended to include the conversion to kepub) is:

Ebooks/{author_sort:sublist(0,1,&)}/{title} - {authors:sublist(0,1,&)}

The Ebooks at the start of the string just keeps the book files from being stored in the root directory since that has limited directory storage and with long file names taking up multiple entries, you can easily run out of directory entries before disk space. Subdirectories can extend the directory storage so that is not an issue.

The {author_sort:sublist(0,1,&)} entry stores the book files in subdirectories under Ebooks based on the first entry in the author sort list. The :sublist(0,1,&) is what restricts it to the first entry. This is based on trying to keep the total filepath/filename entries as short as possible since calibre will start truncating to fit if they get too long.

The {title} entry is calibre's title string used as the first part of the filename. I do not include subtitles or "A Novel" or etc. in the title.

The {authors:sublist(0,1,&)} uses the first entry in the author list as part of the filename. Again this is more to keep the total filename/filepath below calibre's clipping threshold.

I don't use anything to do with series in the template since my Kobo ereaders are capable of handling series/series index.

To use the convert to kepub ability, you need to check Use Kobo viewer for EPUB books on the Collections, covers and uploads tab of the KoboTouch driver configuration dialog.

Hopefully, this is some help to you.