Duplicated Collections Again!!! - Page 2

PeterT · 12-12-2014, 06:05 PM

I believe the duplication is NOT related to the f/w but rather to the cloud and the data supplied on a sync.

Josieb1 · 12-12-2014, 06:44 PM

Quote:

Originally Posted by PeterT

I believe the duplication is NOT related to the f/w but rather to the cloud and the data supplied on a sync.

But how do we get rid of the duplications? Most of my collections (not all, but 90%) are duplicated up to four times.

robko · 12-12-2014, 09:36 PM

Quote:

Originally Posted by stoney77

I believe the problem is with the Kobo upgrade and not calibre. Calibre says there will be an upgrade in a few days to work Kobo 3.12.

It's a recurring Kobo server issue that causes the duplicate shelves (not firmware or calibre).
The ONLY reason that calibre won't work with 3.12 currently (perhaps excepting the fact that a plugin doesn't seem to be removing duplicate shelves properly) is because the "supported" firmware version number hasn't been updated in calibre yet. Davidfor tests calibre it all the way thru the firmware beta's and if there was an actual issue he'd have posted it.
As PeterT said, in order to get calibre 2.12 to work with Kobo firmware 3.12 "you just need to select the option Attempt to support newer firmware in the Customize option for the driver.". Trust me, Peter knows what he's talking about.

Josieb1 · 12-12-2014, 11:43 PM

Quote:

Originally Posted by robko

It's a recurring Kobo server issue that causes the duplicate shelves (not firmware or calibre).
The ONLY reason that calibre won't work with 3.12 currently (perhaps excepting the fact that a plugin doesn't seem to be removing duplicate shelves properly) is because the "supported" firmware version number hasn't been updated in calibre yet. Davidfor tests calibre it all the way thru the firmware beta's and if there was an actual issue he'd have posted it.
As PeterT said, in order to get calibre 2.12 to work with Kobo firmware 3.12 "you just need to select the option Attempt to support newer firmware in the Customize option for the driver.". Trust me, Peter knows what he's talking about.

I've already done that, and as Peter has said earlier in this thread the Utilities program is not working properly.

davidfor · 12-13-2014, 02:37 AM

I don't understand what any of you are complaining about. What's wrong with 650 pages of collections?

Well, that explains why the overnight sync was still running when I got up this morning. It it looks like if finished at 9:44am, so it took well over two hours.

For those who haven't been through this before, the problem isn't actually duplicates. For some reason, the Kobo server is recreating deleted shelves. When you create a shelf and sync, it is recorded on the server. When you then delete the shelf and sync, that fact is recorded. If you use calibre to manage the shelves, each time you add and remove books, shelves might get created or deleted. Then a sync to the server will update the shelves there. Unfortunately, somewhere along the way an id is generated. This is used rather than the shelf name for matching. It seems to work well if there is very little change in the shelves. But, when the server messes up, this fails.

I've had a look at the database PeterT send and mine. This time something new has happened. The reason why I said above that the server was recreating the shelves is because the timestamps on the duplicates were different. The creation timestamp of each where different and that agreed with reanimating the deleted shelves. This time, the database shows all the timestamps as "1970-01-01T00:00:00Z". And that explains why the utilities plugin is not fixing it. It marks all except on shelf as deleted and it uses the timestamp, either the oldest or newest depending on the option selected, to work out which to keep. With them all having the same timestamp, that doesn't work. I can do something with the ids to handle this. I should have something out later in the evening.

Josieb1 · 12-13-2014, 03:04 AM

Quote:

Originally Posted by davidfor

I don't understand what any of you are complaining about. What's wrong with 650 pages of collections?

Well, that explains why the overnight sync was still running when I got up this morning. It it looks like if finished at 9:44am, so it took well over two hours.

Thanks for explanation but please don't say you had 650 pages of collections

I would have closed my Kobo in shock and hid in the bottom of the drawer until I'd had a strong drink!

Josieb1 · 12-13-2014, 03:06 AM

Quote:

Originally Posted by davidfor

I can do something with the ids to handle this. I should have something out later in the evening.

Thank you and Peter as well for looking at this

davidfor · 12-13-2014, 06:02 AM

Quote:

Originally Posted by Josieb1

Thanks for explanation but please don't say you had 650 pages of collections

I would have closed my Kobo in shock and hid in the bottom of the drawer until I'd had a strong drink!

When I ran the fix, the results were:

Code:

Kobo Utilities - Duplicate Shelves Fixed:
Update summary:
	Starting number of shelves=5843
	Shelves removed=5115
	Total shelves=728

I'm still at 81 pages of shelves. I normally have about 40 pages, but I haven't cleaned up yet after using an SD card with a pile of test books.

In any case, I have fixed the problem and uploaded the plugin. Calibre should be announcing it in the next hour.

PeterT · 12-13-2014, 06:04 AM

Quote:

Originally Posted by davidfor

Code:

Kobo Utilities - Duplicate Shelves Fixed:
Update summary:
	Starting number of shelves=5843
	Shelves removed=5115
	Total shelves=728

Makes my 442 shelves look like a speck on the ground besides Ayres Rock!

Terisa de morgan · 12-13-2014, 06:08 AM

Quote:

Originally Posted by PeterT

Makes my 442 shelves look like a speck on the ground besides Ayres Rock!

I'll hide ashamed with my less than 100 shelves.

bethany7 · 12-13-2014, 11:27 AM

Quote:

Originally Posted by Terisa de morgan

I'll hide ashamed with my less than 100 shelves.

Even with the duplicate shelves issue which I am also experiencing, I still only have 3 pages of shelves! I only tend to shelve by genre, figuring that I'll be able to work out series on my own. Doesn't always work that way though...

Josieb1 · 12-13-2014, 01:08 PM

Quote:

Originally Posted by bethany7

Even with the duplicate shelves issue which I am also experiencing, I still only have 3 pages of shelves! I only tend to shelve by genre, figuring that I'll be able to work out series on my own. Doesn't always work that way though...

I shelve by genre as well and I have three pages of shelves without the duplications.

Josieb1 · 12-13-2014, 01:10 PM

Quote:

Originally Posted by davidfor

When I ran the fix, the results were:

Code:

Kobo Utilities - Duplicate Shelves Fixed:
Update summary:
	Starting number of shelves=5843
	Shelves removed=5115
	Total shelves=728

I'm still at 81 pages of shelves. I normally have about 40 pages, but I haven't cleaned up yet after using an SD card with a pile of test books.

In any case, I have fixed the problem and uploaded the plugin. Calibre should be announcing it in the next hour.

Do you know if Calibre has a fix yet? Or will it be in the next Calibre version? I.e next Friday's release?

PeterT · 12-13-2014, 04:01 PM

The fix is in the updated version of the Kobo Utilities plugin that are available already; no need for a calibre update.

Josieb1 · 12-13-2014, 04:12 PM

Quote:

Originally Posted by PeterT

The fix is in the updated version of the Kobo Utilities plugin that are available already; no need for a calibre update.

Thank you. I've just realised that. I've been out all day so only just updated Calibre and took a look.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Implemented, plugin, or so? Duplicated	juliusromo	Calibre	2	12-26-2012 06:51 PM
Duplicated Shelves	beedaddy	Devices	4	12-19-2012 05:49 AM
Duplicated TOC	fe0642	Calibre	1	08-09-2011 07:40 PM
Duplicated in Calibre	mdibella	Calibre	7	09-01-2010 09:24 AM
Duplicated Folders	Mick4545	Calibre	2	05-24-2010 12:39 PM

12-12-2014, 06:05 PM	#16
PeterT Grand Sorcerer Posts: 13,527 Karma: 78910202 Join Date: Nov 2007 Location: Toronto Device: Libra H2O, Libra Colour	I believe the duplication is NOT related to the f/w but rather to the cloud and the data supplied on a sync.

12-13-2014, 02:37 AM	#20
davidfor Grand Sorcerer Posts: 24,905 Karma: 47303824 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	I don't understand what any of you are complaining about. What's wrong with 650 pages of collections? Well, that explains why the overnight sync was still running when I got up this morning. It it looks like if finished at 9:44am, so it took well over two hours. For those who haven't been through this before, the problem isn't actually duplicates. For some reason, the Kobo server is recreating deleted shelves. When you create a shelf and sync, it is recorded on the server. When you then delete the shelf and sync, that fact is recorded. If you use calibre to manage the shelves, each time you add and remove books, shelves might get created or deleted. Then a sync to the server will update the shelves there. Unfortunately, somewhere along the way an id is generated. This is used rather than the shelf name for matching. It seems to work well if there is very little change in the shelves. But, when the server messes up, this fails. I've had a look at the database PeterT send and mine. This time something new has happened. The reason why I said above that the server was recreating the shelves is because the timestamps on the duplicates were different. The creation timestamp of each where different and that agreed with reanimating the deleted shelves. This time, the database shows all the timestamps as "1970-01-01T00:00:00Z". And that explains why the utilities plugin is not fixing it. It marks all except on shelf as deleted and it uses the timestamp, either the oldest or newest depending on the option selected, to work out which to keep. With them all having the same timestamp, that doesn't work. I can do something with the ids to handle this. I should have something out later in the evening.

12-13-2014, 04:01 PM	#29
PeterT Grand Sorcerer Posts: 13,527 Karma: 78910202 Join Date: Nov 2007 Location: Toronto Device: Libra H2O, Libra Colour	The fix is in the updated version of the Kobo Utilities plugin that are available already; no need for a calibre update.

Advert

Advert