Content server not serving a cached, not most recent version

stumped · 12-12-2018, 10:09 AM

something in the chain of calibre content server on PC -> calibre companion on tablet -> Moon reader on tablet is serving up a older cached copy and it's driving me mad trying to diagnose track it down
I tweaked a book - remoing some ornamental scene breaks. then I wiped the tablet copy and deleted the CC record, reconnected & redonloaded it - but the ornaments were still there. Yet they are definitely not there in the epub , viewed in calibre editor or sigil back on the PC, and I only have one copy of the book in one format, in the library.
My calibre library is on C drive local directory - no cloud stuff involved.

I have been through this in various ways, checking it really has left the tablet - by using file explorers and trying to open bookmarks to it... also checking tablet file time stamp after re fetching it to see that yes its a brand new copy.

What did just help is that I completely closed calibre and restarted it - then re-acquired the book, and now I get a newer version. which suggests that calibre content server, or windows ??? is to blame for the caching. Not CC or tablet.
how can I trace this better ?
the evidence so far suggests that the content server is holding a temp copy of last book sent or recent books sent and to when it is re-requested it uses its own temp copy and doe not get the actual up to date version from the calibre library . is that possible ?

stumped · 12-12-2018, 10:24 AM

so here is a shorter proof of issue:
tweaked the scene breaks a little more in calibre
then on tablet I Opened CC, deleted the book, connected to content server , downloaded the book , disconnected, read it - and the recent changes where not there i.e. a cached version had downloaded again
went back to PC, closed calibre, reopened it
then on the tablet, opened CC, deleted book, connected to content server, re-downloaded... changes were delivered OK
ps calibre version 3.3.1, i could install 3.5 and re-try but I see no mention ofd content server in changes for that version.
PPS I have had similar issues before , but I blamed myself for getting CC deletes and Moon reader deletes out of sync, i am pretty sure I have now proved its a content server caching issue. which will be frustrating when wanting to view the effect of book edits in a reader app. I want to be able to download a book, view it on device, tweak it in calibre library , delete from device, re-acquire it & recheck it....

gbm · 12-12-2018, 10:56 AM

Quote:

Originally Posted by stumped

so here is a shorter proof of issue:
tweaked the scene breaks a little more in calibre
then on tablet I Opened CC, deleted the book, connected to content server , downloaded the book , disconnected, read it - and the recent changes where not there i.e. a cached version had downloaded again
went back to PC, closed calibre, reopened it
then on the tablet, opened CC, deleted book, connected to content server, re-downloaded... changes were delivered OK
ps calibre version 3.3.1, i could install 3.5 and re-try but I see no mention ofd content server in changes for that version.
PPS I have had similar issues before , but I blamed myself for getting CC deletes and Moon reader deletes out of sync, i am pretty sure I have now proved its a content server caching issue. which will be frustrating when wanting to view the effect of book edits in a reader app. I want to be able to download a book, view it on device, tweak it in calibre library , delete from device, re-acquire it & recheck it....

The only books that the sever cashes are one that you are reading in the server.

Do this:

delete from Moon Reader.
Open CC and delete the book
start calibre content server
re-download the book.

By the way for what you are wanting to do I use CC as a wireless device and use the "connect wireless device/sysnc books" option, works every time even with tens of books to be updated.

bernie

ilovejedd · 12-12-2018, 11:05 AM

I've had similar issues on my iPad but found that the problem was iOS caching.

Steps I took to troubleshoot:

download using Safari on iPad (got old version)
download using Chrome on iPad (got old version)
restart iPad and redownload (got old version)
download using a new install of FirefoxPortable on PC (got new version)
download using my smartphone (got new version)

Alas, short of a nuclear option (clear all cookies and browsing history), I don't know of any fixes. I just downloaded the book from Dropbox instead.

stumped · 12-12-2018, 11:37 AM

ok guys, well stopping and restarting calibre is working a workaround,I guess that closing the main program stops and restarts the content server., and only take s a couple of seconds
and doing that, but doing only that would disprove that windows network caching is to blame ?
it seems poor design that content server cannot be trusted to server the latest version, always ?
and yet I probably could use connect as wireless device, I forget exactly why I chose not to do that, standardly. Probably because I only wanted the one way calibre-> device delivery.
i never read in server.
i connect to server, get book onto device, via calibre companion; disconnect, then read on device.
what I'd like is some definitive explanation of what is happening, is it a feature or a bug, and is it documented somewhere that I have not looked. i TRIED SEARCHING THE FORUM FOR CONTENT SERVER CACHE BUT NO JOY THERE.
e.g. is only the last book served cached, or the last n books, or the last x minutes ???

kovidgoyal · 12-12-2018, 11:58 AM

the server most definitely does not cache books. Simply download books directly from it using a browser to test if that is the case. Remove the middle men. And hit refresh in your browser before re-downloading.

stumped · 12-12-2018, 12:07 PM

If that is true, then stopping, starting calibre before re downloading should not change what I get from content server. But it does. As per my last post.
Deleting, then refetching using calibre companion connect to content server with calibre running gives old version. Deleting then referencing after a stop ,restart of calibre gives new version.
Calibre 3.31 64 bit.
Timestamps on device confirm book was re fetched each time
Does content server work in a way that could cause windows to cache ?
Win 10 64 bit

kovidgoyal · 12-12-2018, 12:11 PM

you are testing with too many intermediaries. Use a browser and download directly.

stumped · 12-12-2018, 12:16 PM

If I have to. Was hoping someone would know what the issue was with using calibre companion to fetch. And a recent post from someone else suggested operating system caching could be to blame.
Even if using a browser works.That is a terrible workaround for not being able to use, trust calibre companion.
I could experiment more e.g. Force exiting or stopping cc. Each time.
To get a book to my tablet I should not need to mess with tablet browsers and file explorer utilities before I can see the latest calibre edits .that's why I bought the recommended app !
I am 100 percent certain that I am removing all traces of the old book from my tablets filing system before I re fetch it.

ilovejedd · 12-12-2018, 01:20 PM

Quote:

Originally Posted by stumped

If I have to. Was hoping someone would know what the issue was with using calibre companion to fetch. And a recent post from someone else suggested operating system caching could be to blame.
Even if using a browser works.That is a terrible workaround for not being able to use, trust calibre companion.
I could experiment more e.g. Force exiting or stopping cc. Each time.
To get a book to my tablet I should not need to mess with tablet browsers and file explorer utilities before I can see the latest calibre edits .that's why I bought the recommended app !
I am 100 percent certain that I am removing all traces of the old book from my tablets filing system before I re fetch it.

Unless you do a complete reset of the tablet or you're rooted and have access and can clear Android's tmp and cache directories, you don't have that guarantee.

My guess, restarting the Calibre server initiated a new session so that managed to fix your caching issue.

On mine, restarting Calibre did nothing (forgot to mention I restarted Calibre multiple times in the process of my troubleshooting). The fact that a fresh FirefoxPortable (Win7) install and my smartphone both got the new version while the iPad continued to receive the old versions points that the caching issue isn't on Calibre nor Windows' side.

stumped · 12-12-2018, 01:31 PM

When I claim book was gone from tablet it was because using a saved bookmark in moon reader confirmed book missing. Also looking at the book folder with Xplorer or with moon file browser showed other books but not the deleted one. So it was gone, as a file within the device books folder. Whether it lives on as cached data in the networking system is a other question and not easy to answer.
I think calibre companion may cache stuff. I saw it had a 2mb cache. Big enough for a book, active..
It my main pc is 9ff for the night how's I am done with testing.
A next test, if I can be bothered , would be to add exit or force stop of cc between delete and re download. There is nothing relevant in cc faq.
I remember why I don't use connect as wireless device. It is because I want to control getting books from the tablet. Not have to physically go to the pc and mess with send to device.
Ps limited tests suggest that getting books from cloud will always fetch latest version. So long as I have cloud synced calibre after editing.

12-12-2018, 10:09 AM	#1
stumped Wizard Posts: 3,305 Karma: 10259306 Join Date: May 2016 Device: kobo forma, Kobo Libra, Huawei media Tab, fire HD10, PW3 HDX8.9,	Content server not serving a cached, not most recent version something in the chain of calibre content server on PC -> calibre companion on tablet -> Moon reader on tablet is serving up a older cached copy and it's driving me mad trying to diagnose track it down I tweaked a book - remoing some ornamental scene breaks. then I wiped the tablet copy and deleted the CC record, reconnected & redonloaded it - but the ornaments were still there. Yet they are definitely not there in the epub , viewed in calibre editor or sigil back on the PC, and I only have one copy of the book in one format, in the library. My calibre library is on C drive local directory - no cloud stuff involved. I have been through this in various ways, checking it really has left the tablet - by using file explorers and trying to open bookmarks to it... also checking tablet file time stamp after re fetching it to see that yes its a brand new copy. What did just help is that I completely closed calibre and restarted it - then re-acquired the book, and now I get a newer version. which suggests that calibre content server, or windows ??? is to blame for the caching. Not CC or tablet. how can I trace this better ? the evidence so far suggests that the content server is holding a temp copy of last book sent or recent books sent and to when it is re-requested it uses its own temp copy and doe not get the actual up to date version from the calibre library . is that possible ? Last edited by stumped; 12-12-2018 at 10:24 AM.

12-12-2018, 10:24 AM	#2
stumped Wizard Posts: 3,305 Karma: 10259306 Join Date: May 2016 Device: kobo forma, Kobo Libra, Huawei media Tab, fire HD10, PW3 HDX8.9,	so here is a shorter proof of issue: tweaked the scene breaks a little more in calibre then on tablet I Opened CC, deleted the book, connected to content server , downloaded the book , disconnected, read it - and the recent changes where not there i.e. a cached version had downloaded again went back to PC, closed calibre, reopened it then on the tablet, opened CC, deleted book, connected to content server, re-downloaded... changes were delivered OK ps calibre version 3.3.1, i could install 3.5 and re-try but I see no mention ofd content server in changes for that version. PPS I have had similar issues before , but I blamed myself for getting CC deletes and Moon reader deletes out of sync, i am pretty sure I have now proved its a content server caching issue. which will be frustrating when wanting to view the effect of book edits in a reader app. I want to be able to download a book, view it on device, tweak it in calibre library , delete from device, re-acquire it & recheck it.... Last edited by stumped; 12-12-2018 at 10:30 AM.

12-12-2018, 11:05 AM	#4
ilovejedd hopeless n00b Posts: 5,111 Karma: 19597086 Join Date: Jan 2009 Location: in the middle of nowhere Device: PW4, PW3, Libra H2O, iPad 10.5, iPad 11, iPad 12.9	I've had similar issues on my iPad but found that the problem was iOS caching. Steps I took to troubleshoot: download using Safari on iPad (got old version) download using Chrome on iPad (got old version) restart iPad and redownload (got old version) download using a new install of FirefoxPortable on PC (got new version) download using my smartphone (got new version) Alas, short of a nuclear option (clear all cookies and browsing history), I don't know of any fixes. I just downloaded the book from Dropbox instead.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
3.0.0 Content Server and iOS version	TomCombs	Devices	15	04-18-2018 11:16 AM
Version 3.0 Content Server Logfiles	Crucis	Calibre	4	06-21-2017 05:04 PM
0.8.26 - Mac version - apps won't connect to Content Server	1611mac	Calibre	2	11-12-2011 10:29 PM
Content server not serving all content.	False_Chicken	Calibre	0	10-19-2011 05:56 PM
Content Server - Don't want mobile version	szsori	Calibre	3	04-05-2011 11:49 AM

12-12-2018, 11:37 AM	#5
stumped Wizard Posts: 3,305 Karma: 10259306 Join Date: May 2016 Device: kobo forma, Kobo Libra, Huawei media Tab, fire HD10, PW3 HDX8.9,	ok guys, well stopping and restarting calibre is working a workaround,I guess that closing the main program stops and restarts the content server., and only take s a couple of seconds and doing that, but doing only that would disprove that windows network caching is to blame ? it seems poor design that content server cannot be trusted to server the latest version, always ? and yet I probably could use connect as wireless device, I forget exactly why I chose not to do that, standardly. Probably because I only wanted the one way calibre-> device delivery. i never read in server. i connect to server, get book onto device, via calibre companion; disconnect, then read on device. what I'd like is some definitive explanation of what is happening, is it a feature or a bug, and is it documented somewhere that I have not looked. i TRIED SEARCHING THE FORUM FOR CONTENT SERVER CACHE BUT NO JOY THERE. e.g. is only the last book served cached, or the last n books, or the last x minutes ???

12-12-2018, 11:58 AM	#6
kovidgoyal creator of calibre Posts: 43,844 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	the server most definitely does not cache books. Simply download books directly from it using a browser to test if that is the case. Remove the middle men. And hit refresh in your browser before re-downloading.

12-12-2018, 12:07 PM	#7
stumped Wizard Posts: 3,305 Karma: 10259306 Join Date: May 2016 Device: kobo forma, Kobo Libra, Huawei media Tab, fire HD10, PW3 HDX8.9,	If that is true, then stopping, starting calibre before re downloading should not change what I get from content server. But it does. As per my last post. Deleting, then refetching using calibre companion connect to content server with calibre running gives old version. Deleting then referencing after a stop ,restart of calibre gives new version. Calibre 3.31 64 bit. Timestamps on device confirm book was re fetched each time Does content server work in a way that could cause windows to cache ? Win 10 64 bit

12-12-2018, 12:11 PM	#8
kovidgoyal creator of calibre Posts: 43,844 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	you are testing with too many intermediaries. Use a browser and download directly.

12-12-2018, 12:16 PM	#9
stumped Wizard Posts: 3,305 Karma: 10259306 Join Date: May 2016 Device: kobo forma, Kobo Libra, Huawei media Tab, fire HD10, PW3 HDX8.9,	If I have to. Was hoping someone would know what the issue was with using calibre companion to fetch. And a recent post from someone else suggested operating system caching could be to blame. Even if using a browser works.That is a terrible workaround for not being able to use, trust calibre companion. I could experiment more e.g. Force exiting or stopping cc. Each time. To get a book to my tablet I should not need to mess with tablet browsers and file explorer utilities before I can see the latest calibre edits .that's why I bought the recommended app ! I am 100 percent certain that I am removing all traces of the old book from my tablets filing system before I re fetch it.

12-12-2018, 01:31 PM	#11
stumped Wizard Posts: 3,305 Karma: 10259306 Join Date: May 2016 Device: kobo forma, Kobo Libra, Huawei media Tab, fire HD10, PW3 HDX8.9,	When I claim book was gone from tablet it was because using a saved bookmark in moon reader confirmed book missing. Also looking at the book folder with Xplorer or with moon file browser showed other books but not the deleted one. So it was gone, as a file within the device books folder. Whether it lives on as cached data in the networking system is a other question and not easy to answer. I think calibre companion may cache stuff. I saw it had a 2mb cache. Big enough for a book, active.. It my main pc is 9ff for the night how's I am done with testing. A next test, if I can be bothered , would be to add exit or force stop of cc between delete and re download. There is nothing relevant in cc faq. I remember why I don't use connect as wireless device. It is because I want to control getting books from the tablet. Not have to physically go to the pc and mess with send to device. Ps limited tests suggest that getting books from cloud will always fetch latest version. So long as I have cloud synced calibre after editing.