Loading many pdfs via ssh on paperwhite eventually fails

jonvarteresian · 02-20-2019, 12:05 PM

Hi,

Here is an interesting problem that I hope someone can shed some light on.

I have a kindle paperwhite that I ssh into, upload a pdf and display on the screen via:

lipc-set-prop com.lab126.appmgrd start app://com.lab126.booklet.reader:/mnt/us/documents/doc.pdf

works perfectly up to about 900 cycles. Each time I specify a new file name, and delete the old filename. around 916 cycles. (yes, its that repeatable) it stops working and I get 'cannot open application' errors. the only way out is to reboot the kindle and start again.

i tried sending a

lipc-set-prop com.lab126.appmgrd clearAllContext 1

in the hopes that it was that easy, but same behavior.

any ideas on what i should look at?

thanks!

NiLuJe · 02-20-2019, 12:40 PM

Have you tried forcing a content indexing pass after each new file is ssh'ed in?

knc1 · 02-20-2019, 01:42 PM

When you delete a document, do you also delete the metadata directory that is created when the document is put onto the Kindle?

PS: Model of Kindle and current firmware version under discussion please.
That information is needed by anyone who tries to duplicate your problem.

j.p.s · 02-20-2019, 02:08 PM

Is there some script that does everything removing a document via the kindle GUI does?

If not, and it is possible, such a script might be handy.

knc1 · 02-20-2019, 02:35 PM

Quote:

Originally Posted by j.p.s

Is there some script that does everything removing a document via the kindle GUI does?

If not, and it is possible, such a script might be handy.

It is already posted here, somewhere.
The original is in English, not that thing from China found by someone recently.

jonvarteresian · 02-20-2019, 02:52 PM

interesting. When the 'badness happens' i get one of these messages for each file i transferred/displayed/deleted:

190220:192818 cvm[3869]: E MasterIndex:NoIndexFile

ath=/mnt/us/system/Search Indexes/y872e71d51e338b8bb6a0ef4e5c411b55:

I will add the index refresh and see what happens

jonvarteresian · 02-20-2019, 02:58 PM

yes, i believe i do delete the metadata.

sorry about not including the kindle model. Its a paperwhite, model pq94wif running 5.10.0.2

knc1 · 02-20-2019, 04:36 PM

Quote:

Originally Posted by jonvarteresian

yes, i believe i do delete the metadata.

sorry about not including the kindle model. Its a paperwhite, model pq94wif running 5.10.0.2

You would know it if you did delete the metadata.
That answer sounds like you don't have a clue about it.
Also, the message snippet you posted reads as if you have not deleted the metadata.

Use the first four (or six) characters of the Kindle's serial number and the table you find here:
https://wiki.mobileread.com/wiki/Kindle_Serial_Numbers
To learn what your model is called.
Tell us and correct your side-bar information box also.

jonvarteresian · 02-20-2019, 10:17 PM

hi again,

Kindle is a Kindle PaperWhite 4 (2018) WiFi, 8GB (SN G0000 PP...)

ok, i do delete the metadata (the .sdr folder associated with the file). The problem is that this cycle of uploading via ssh, displaying, and then deleting works up to 900-ish cycles. after that is when i start to get those messages.

once i hit that threshold is when the log includes all those NoIndexFile messages.

I tried this command after loading a new file and deleting the old, but no change:

dbus-send --system /default com.lab126.powerd.resuming int32:1

Now, I just call appmgrd start <fn> to display my image, and when i want a new one I call appmgrd start again with a new filename and delete the old files. I never stop the viewer, could that be the problem? I tried appmgrd stop but it didnt seem to do anything.

I do appreciate the help.

jonvarteresian · 02-20-2019, 10:32 PM

Quote:

Originally Posted by NiLuJe

Have you tried forcing a content indexing pass after each new file is ssh'ed in?

so do i do this re-indexing after i ssh up a new file, or after i remove a file from the kindle? or both?

NiLuJe · 02-20-2019, 11:14 PM

Since I assume you're deleting then sending a new file without anything in between, after the new file gets here *should* be enough?

Otherwise, both times, for completeness' sake.

Unless they're using the same filename, or you're bypassing the fuse layer, or something, in which case, both times, definitely.

knc1 · 02-21-2019, 01:12 AM

Quote:

Originally Posted by jonvarteresian

so do i do this re-indexing after i ssh up a new file, or after i remove a file from the kindle? or both?

Which media are you using for SSH? USB cable or Wifi?

jonvarteresian · 02-21-2019, 08:05 AM

i am using a USB cable for the SSH connection. (touch /mnt/us/ENABLE_USBNET).

Thinking about this more, I am not sure if it is the 900+ unique files that are opened in the viewer, or the fact that i open the viewer 900+ times. I am running a test right now that alternates between 2 files which are SSH'd at the beginning of time, and then just opened. I don't delete those at all. If it still fails at 900+ then it has to do with how i am controlling the viewer.

knc1 · 02-21-2019, 08:18 AM

Quote:

Originally Posted by jonvarteresian

i am using a USB cable for the SSH connection. (touch /mnt/us/ENABLE_USBNET).

And there is your problem, PEBKAC.
It is a well known "gotcha" that the system does not always recognize things that are ssh'd in over the USB cable.

That message when you reach 900+ entries could be worded better, but I think it translates too: "Indexer todo work queue overrun".
So don't do that.
Restart the framework occasionally - say once ever 100 books.

jonvarteresian · 02-21-2019, 10:28 AM

thanks, makes sense.

I also thought that maybe i could just turn off the indexer. I tried this link

https://www.mobileread.com/forums/sh...d.php?t=220735

but it didnt seem to affect the indexer on the PW4. Do you know another way to disable the indexer? I certainly don't need it for my application.

thanks for all the help.

02-20-2019, 12:05 PM	#1
jonvarteresian Member Posts: 15 Karma: 10 Join Date: Feb 2019 Device: PW4 Kindle PaperWhite 4 (2018) WiFi, 8GB	Loading many pdfs via ssh on paperwhite eventually fails Hi, Here is an interesting problem that I hope someone can shed some light on. I have a kindle paperwhite that I ssh into, upload a pdf and display on the screen via: lipc-set-prop com.lab126.appmgrd start app://com.lab126.booklet.reader:/mnt/us/documents/doc.pdf works perfectly up to about 900 cycles. Each time I specify a new file name, and delete the old filename. around 916 cycles. (yes, its that repeatable) it stops working and I get 'cannot open application' errors. the only way out is to reboot the kindle and start again. i tried sending a lipc-set-prop com.lab126.appmgrd clearAllContext 1 in the hopes that it was that easy, but same behavior. any ideas on what i should look at? thanks!

02-20-2019, 12:40 PM	#2
NiLuJe BLAM! Posts: 13,478 Karma: 26012494 Join Date: Jun 2010 Location: Paris, France Device: Kindle 2i, 3g, 4, 5w, PW, PW2, PW5; Kobo H2O, Forma, Elipsa, Sage, C2E	Have you tried forcing a content indexing pass after each new file is ssh'ed in? Last edited by NiLuJe; 02-20-2019 at 12:43 PM.

02-20-2019, 02:52 PM	#6
jonvarteresian Member Posts: 15 Karma: 10 Join Date: Feb 2019 Device: PW4 Kindle PaperWhite 4 (2018) WiFi, 8GB	interesting. When the 'badness happens' i get one of these messages for each file i transferred/displayed/deleted: 190220:192818 cvm[3869]: E MasterIndex:NoIndexFileath=/mnt/us/system/Search Indexes/y872e71d51e338b8bb6a0ef4e5c411b55: I will add the index refresh and see what happens

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Loading Calibre Library folder from SSH server	sajwijnings	Library Management	3	10-04-2018 03:15 PM
Book loading fails odd way	pokkis	Kobo Reader	5	10-19-2016 12:37 PM
ssh over wifi on paperwhite	nasser	Kindle Developer's Corner	9	07-22-2013 02:24 PM
SSH Kindle Paperwhite	kevin67	Amazon Kindle	2	02-16-2013 11:57 AM
Help: OS X + SSH + Paperwhite	Ecaz	Kindle Developer's Corner	4	11-01-2012 09:17 PM

02-20-2019, 01:42 PM	#3
knc1 Going Viral Posts: 17,212 Karma: 18210809 Join Date: Feb 2012 Location: Central Texas Device: No K1, PW2, KV, KOA	When you delete a document, do you also delete the metadata directory that is created when the document is put onto the Kindle? PS: Model of Kindle and current firmware version under discussion please. That information is needed by anyone who tries to duplicate your problem.

02-20-2019, 02:08 PM	#4
j.p.s Grand Sorcerer Posts: 5,285 Karma: 98804578 Join Date: Apr 2011 Device: pb360	Is there some script that does everything removing a document via the kindle GUI does? If not, and it is possible, such a script might be handy.

02-20-2019, 02:58 PM	#7
jonvarteresian Member Posts: 15 Karma: 10 Join Date: Feb 2019 Device: PW4 Kindle PaperWhite 4 (2018) WiFi, 8GB	yes, i believe i do delete the metadata. sorry about not including the kindle model. Its a paperwhite, model pq94wif running 5.10.0.2

02-20-2019, 10:17 PM	#9
jonvarteresian Member Posts: 15 Karma: 10 Join Date: Feb 2019 Device: PW4 Kindle PaperWhite 4 (2018) WiFi, 8GB	hi again, Kindle is a Kindle PaperWhite 4 (2018) WiFi, 8GB (SN G0000 PP...) ok, i do delete the metadata (the .sdr folder associated with the file). The problem is that this cycle of uploading via ssh, displaying, and then deleting works up to 900-ish cycles. after that is when i start to get those messages. once i hit that threshold is when the log includes all those NoIndexFile messages. I tried this command after loading a new file and deleting the old, but no change: dbus-send --system /default com.lab126.powerd.resuming int32:1 Now, I just call appmgrd start <fn> to display my image, and when i want a new one I call appmgrd start again with a new filename and delete the old files. I never stop the viewer, could that be the problem? I tried appmgrd stop but it didnt seem to do anything. I do appreciate the help.

02-20-2019, 11:14 PM	#11
NiLuJe BLAM! Posts: 13,478 Karma: 26012494 Join Date: Jun 2010 Location: Paris, France Device: Kindle 2i, 3g, 4, 5w, PW, PW2, PW5; Kobo H2O, Forma, Elipsa, Sage, C2E	Since I assume you're deleting then sending a new file without anything in between, after the new file gets here should be enough? Otherwise, both times, for completeness' sake. Unless they're using the same filename, or you're bypassing the fuse layer, or something, in which case, both times, definitely.

02-21-2019, 08:05 AM	#13
jonvarteresian Member Posts: 15 Karma: 10 Join Date: Feb 2019 Device: PW4 Kindle PaperWhite 4 (2018) WiFi, 8GB	i am using a USB cable for the SSH connection. (touch /mnt/us/ENABLE_USBNET). Thinking about this more, I am not sure if it is the 900+ unique files that are opened in the viewer, or the fact that i open the viewer 900+ times. I am running a test right now that alternates between 2 files which are SSH'd at the beginning of time, and then just opened. I don't delete those at all. If it still fails at 900+ then it has to do with how i am controlling the viewer.

02-21-2019, 10:28 AM	#15
jonvarteresian Member Posts: 15 Karma: 10 Join Date: Feb 2019 Device: PW4 Kindle PaperWhite 4 (2018) WiFi, 8GB	thanks, makes sense. I also thought that maybe i could just turn off the indexer. I tried this link https://www.mobileread.com/forums/sh...d.php?t=220735 but it didnt seem to affect the indexer on the PW4. Do you know another way to disable the indexer? I certainly don't need it for my application. thanks for all the help.

Advert

Advert