very small bug report

[cybmole]

KF HDX.
if I've previously disconnected from content server, and let tablet sleep...

Then when I wake the tablet, launch CC & try to connect the 1st try always ( I think - very small sample size) thinks a while & says nope , cant find a content server.
But 2nd try (always - but with same reservations) works immediately, on the same settings. maybe it has a too aggressive time-out ? ( or it could be that my wifi is still initialising but I don't think that is the case )

if you want me to look more deeply, please advise. otherwise I'm happy to poke it twice! I'm only using CC about once per week.
if you want logs, you'll need to tell me where to find them!

( I confess to finding the calibre cloud app to be simpler for my needs ( + it works when my PC is off) ; it's only drawback is that it sees only my dropbox backup so if that is not updated ( e.g. if i forgot that I'd turned off dropbox sync), I don't find the newly edited book that I'm looking for. I then turn to your CC for a 2nd opinion as that connects direct to my calibre library.

If I leave your CC connected it resumes fine after a KF sleep. but there';s no wat to delete a local copy of a book while connected to the server ? ( so I am disconnecting just to do that )

[chaley]

Originally Posted by cybmole

KF HDX.
if I've previously disconnected from content server, and let tablet sleep...

Then when I wake the tablet, launch CC & try to connect the 1st try always ( I think - very small sample size) thinks a while & says nope , cant find a content server.
But 2nd try (always - but with same reservations) works immediately, on the same settings. maybe it has a too aggressive time-out ? ( or it could be that my wifi is still initialising but I don't think that is the case )

if you want me to look more deeply, please advise. otherwise I'm happy to poke it twice! I'm only using CC about once per week.
if you want logs, you'll need to tell me where to find them!

Have you entered an IP address or URL into CC's settings, or are you using "automatic"? If the latter, then not only must wifi finish initializing on your tablet, calibre must "wake up" on your computer. CC will wait 5 seconds for calibre to respond. In addition, because CC is using a "broadcast" to send its message, the networking subsystem makes no guarantee that the message will actually be delivered.

The best way for me to see what is happening is to have a CC debug log. The log will tell me what CC tried to do and what success it had at the various stages. To send a debug log, go to CC's settings, scroll to the bottom, and choose "Send a debug log to MultiPie" (I am assuming that your tablet knows about your email accounts). You will see two destinations, one at multipie and one at charles dot haleys etc. Erase the multipie address; my son needn't get involved. If you want to look at the log then it is at Android/data/com.multipie.calibreandroid/files/calibre.companion.debug.log.txt.

If I leave your CC connected it resumes fine after a KF sleep. but there';s no wat to delete a local copy of a book while connected to the server ? ( so I am disconnecting just to do that )

I assume that you mean connected to the content server? Regardless of which connection, you are right, but for different reasons. You can't delete books when connected as a wireless device because there is no way to tell the connected calibre that the device book list has changed.

You can't delete books when connected to the content server because it never occurred to us that someone would want to do that. If we add it then we would need to deal with the ambiguity of local delete vs calibre-side delete (which in fact we can't do). A simple "Delete" menu item wouldn't tell the user on which side the delete would happen, which isn't good because it is destructive.

I assume that the issue of deleting while connected to the content server arises because you are using CC only as a transport. Grouping, sorting, etc are of no interest to you because you do all of that in the kobo app. This is probably an edge use case, and I am not sure whether it is good to add options/complexity to accommodate it.

[cybmole]

i have sent a log + a message linking my email to this forum account so you can identify what it is about. I was to dumb for figure how to remove the CC.
I did not peek at the log so I don't know for sure if it covers the issue. anyway we can continue this via email once you get it. no need ot tell the world until it's solved!

PS you mention time to wake up calibre, but if that is running ( idling) on PC then is it already awake enough to see any incoming content server requests, even if its not been asked to do anything for several hours ?

PPS
You can't delete books when connected to the content server because it never occurred to us that someone would want to do that

well there's always one awkward so & so & here he is

why - because all I want to do is bring in book(s) them import into Kobo android app. That app will then offer to delete the book from Sd card, but if I say yes, then CC's own records of what is stored locally get messed up. CC will not know that Kobo app has just deleted a book.
SO its a better work flow ( fro me) to fetch with CC, then import book with Kobo app ( with copies it into kobo'a own database) then say no to kobo app offer to delete original, then do the delete from inside of CC ( for which I have to do a disconnect....)

[theducks]

Cybe

Many times, Calibre has to wake back up (fully) on my W7 Laptop (3G RAM), probably because some part has swapped out to VM or the HD went to sleep from lack of use.

I am not sure how a 'I hear you, please wait' could be built in, in such a way that it would not also swap out or sleep. If CC was using static IP for the server, a ping might be used to detect and then extend the initial timeout. WAPs also time out sessions with no activity and a new key may need to be negotiated (this one caused a lot of grief with a laser Price Gun that we installed).

[chaley]

Originally Posted by cybmole

i have sent a log + a message linking my email to this forum account so you can identify what it is about. I was to dumb for figure how to remove the CC.
I did not peek at the log so I don't know for sure if it covers the issue. anyway we can continue this via email once you get it. no need ot tell the world until it's solved!

The logs indicate that you have put an IP address and port into CC's connection server settings, and that most of the time connections to those values work. More below...

PS you mention time to wake up calibre, but if that is running ( idling) on PC then is it already awake enough to see any incoming content server requests, even if its not been asked to do anything for several hours ?

Ahhh, the wonders of virtual memory on computers.

If a running program hasn't done anything for quite some time then its "working set" (memory required to hold the program code) is usually shoved out onto the hard disk to make room for other programs, or simply to make room for the operating system to do other stuff. If and when the swapped-out program needs to run again, it must reload all that stuff back from the hard disk. This can take a long time, sometimes many seconds.

If the calibre content server is not resident in the computer's memory when CC tries to connect then calibre is reloaded. If the time to re-fetch calibre exceeds CC's timeout (2.5 seconds), then CC will give up and try something else. In this case, something else is to ask the network if calibre is around somewhere (the "broadcast") and wait for a few seconds. For some reason on your network, calibre never responds to a broadcast. The connection fails. However, because calibre has been loaded into memory, the next connection attempt succeeds within that 2.5-second window.

The only thing I can see to do is to increase the timeout, but I am not sure that doing so is a good idea. If I increase the timeout then there will be a higher likelihood that calibre will make it back into memory and be able to respond, but it isn't certain (how long is long enough?). On the other hand, people who are using primary and alternate connection settings will pay the extra time on every connection when the secondary connection is the one to use. Perhaps less important, but people who for whatever reason have put something invalid into CC's settings and are depending on broadcast will also pay the extra time.

PPS
You can't delete books when connected to the content server because it never occurred to us that someone would want to do that

well there's always one awkward so & so & here he is

why - because all I want to do is bring in book(s) them import into Kobo android app. That app will then offer to delete the book from Sd card, but if I say yes, then CC's own records of what is stored locally get messed up. CC will not know that Kobo app has just deleted a book.
SO its a better work flow ( fro me) to fetch with CC, then import book with Kobo app ( with copies it into kobo'a own database) then say no to kobo app offer to delete original, then do the delete from inside of CC ( for which I have to do a disconnect....)

Another workflow would be to allow the kobo app to delete the book, then from time to time run CC's "Find missing books" to remove the left-over CC database entries. Of course, running Find missing does require you to disconnect, but it obviates the need for you to remember which books have been deleted. Find missing is also much faster than deleting books one by one, as it can delete all the missing books in one swoop.

Regardless, I will think about deleting books from the content server's book details screen. The only real objection is the one I raised earlier, and perhaps that could be handled by a somewhat longer "Delete" menu line, something like "Delete CC's copy".

EDIT: if you are running Windows then be sure that your hard disk is not fragmented. I have seen virtual memory performance increase by a factor of ten simply by running defrag.

[chaley]

Originally Posted by theducks

I am not sure how a 'I hear you, please wait' could be built in, in such a way that it would not also swap out or sleep. If CC was using static IP for the server, a ping might be used to detect and then extend the initial timeout. WAPs also time out sessions with no activity and a new key may need to be negotiated (this one caused a lot of grief with a laser Price Gun that we installed).

A ping isn't good enough, because CC must really connect to calibre's content server. Doing a ping would say that the computer is there, but it doesn't say that calibre is there. There is also the problem that a lot of AV tools prevent computers from answering pings.

I suppose that a successful ping could tell CC to increase the timeout, but a ping cannot replace the "real" connection attempt.

[theducks]

Get the Page defrag tool (pagedfrg.exe) from sysinternals (M$).

A normal defrag does not touch the swap file (and some other system files or those still open.)

[cybmole]

thanks for quick & detailed replies.

I am happy to connect twice - it always works 2nd time.

my drives are not fragmented & I have 6gb ram ( well 5 really as there's a 500Mb cache in utorrent if that is active, & another 500 Gb in a Ram drive) but that's not to say the windows does not swap calibre out anyway if it's not seen any use since last PC sleep.

I leave calibre in 1 of 2 working states usually: minimised but still fully open open i.e. there's open tab in task bar, or "closed" but actually just minimised to system tray,; are both the same state, actually for listening purposes

I've used the clean up missing books route & yes that is a valid alternative which I could switch to as my default.

i think "delete local copy" is an unambiguous menu option, or even just "delete local"

would I be right in thinking that CC does not have or need permissions to delete the PC calibre library copy anyway, in a typical setup ?. ( esp in content server client mode )

[cybmole]

Originally Posted by theducks

Get the Page defrag tool (pagedfrg.exe) from sysinternals (M$).

A normal defrag does not touch the swap file (and some other system files or those still open.)

i used to have an expensive paid-for defrag that could do that, but I got fed up of the buy this years greatest & latest version hustles & switched to freebies, first to Jkdefrag & now to auslogics ( which is fastest to date)
not sure if that can check page file so I@~ll try grabbing what you suggest, to satisfy the new curiosity itch

scratch - it's an old program or an old version & does not like me , or does not do win 7 64 - tells me " make sure you are an administrator, error loading pagedefrag driver"

[chaley]

Originally Posted by cybmole

scratch - it's an old program or an old version & does not like me , or does not do win 7 64 - tells me " make sure you are an administrator, error loading pagedefrag driver"

According to sysinternals, it doesn't work on 64-bit windows.

I use the built-in tool. It works well enough. I have it set to run on a schedule, once a week at 2 am.

[cybmole]

i am pretty sure that a fixed size page file does not get fragmented anyway. I fixed mine at 6Gb , same as my RAM, agas ago. only the dynamic option should fragment.

A fast way to clean a page file anyway it is to go to windows advanced settings and remove it completely!, THEN recreate it afresh, rebooting in between steps if needed. Windows should assign a contiguous chunk of space when it re-creates.

that's probably quicker than tracking down some 3rd party program & then debating do you really want to let some relatively unknown software meddle with such a key component!

I did see that piriform, makers of the excellent ccleaner, have something called defraggler, free - with boot time defrag options for system files,but it it s o o o o slow. it estimated that it would need over a day optimising my C drive, after I'd already defragged it with a different product! so there's no way I'd allow that to do a boot time defrag, as It could be Xmas next before its hands the PC back!

we stray somewhat from the original thread topic, I fear...