Cannot save books due to memory error

malexherron · 05-13-2020, 03:46 PM

I have been having issues downloading books from Google Drive library. I am able to log in and view new books as they are added to Calibre, however when I download a book I see the following errors in the logs:

Code:

2020-05-10 04:06:43.418:  Cloud downloadFile: url=Gary Gygax/D&D 5e - Baldur's Gate - Descent into Avernus (516)/D&D 5e - Baldur's Gate - Descent into Aver - Gary Gygax.pdf, localLpath=D&D 5e - Baldur's Gate - Descent into Avernus - Gary Gygax.pdf
2020-05-10 04:07:07.454:  google getFile
java.lang.OutOfMemoryError: Failed to allocate a 139927976 byte allocation with 25149440 free bytes and 53MB until OOM, max allowed footprint 170714096, growth limit 201326592
	at java.lang.StringFactory.newStringFromBytes(StringFactory.java:81)
	at java.lang.StringFactory.newStringFromBytes(StringFactory.java:209)
	at okio.Buffer.readString(Buffer.kt:669)
	at okio.Buffer.readString(Buffer.kt:658)
	at okhttp3.logging.HttpLoggingInterceptor.intercept(HttpLoggingInterceptor.kt:264)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:112)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:87)
	at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.kt:184)
	at okhttp3.RealCall.execute(RealCall.kt:66)
	at com.multipie.cclibrary.Cloud.Google.GoogleAPI.getFile(GoogleAPI.java:376)
	at com.multipie.cclibrary.Cloud.BookDownloaderHelpers.downloadFile(BookDownloaderHelpers.java:39)
	at com.multipie.cclibrary.Cloud.DownloadQueue$ServiceThread.run(DownloadQueue.java:123)
2020-05-10 04:07:07.457:  Log flush
2020-05-10 04:07:07.644:  FileManager: deleting book /storage/5DA6-BDE6/CalibreCompanion/D&D 5e - Baldur's Gate - Descent into Avernus - Gary Gygax.pdf, result true
2020-05-10 04:07:07.659:  Download queue: try to download 511 D&D 5e - Lost Mine of Phandelver
2020-05-10 04:07:07.663:  Cloud: download failed: D&D 5e - Baldur's Gate - Descent into Avernus
2020-05-10 04:07:07.740:  checkLpathComponents started D&D 5e - Lost Mine of Phandelver - Gary Gygax w/URI=true
2020-05-10 04:07:07.926:  checkLpathComponents finished D&D 5e - Lost Mine of Phandelver - Gary Gygax
2020-05-10 04:07:07.927:  Cloud downloadFile: url=Gary Gygax/D&D 5e - Lost Mine of Phandelver (511)/D&D 5e - Lost Mine of Phandelver - Gary Gygax.pdf, localLpath=D&D 5e - Lost Mine of Phandelver - Gary Gygax.pdf
2020-05-10 04:07:39.066:  google getFile
java.lang.OutOfMemoryError: Failed to allocate a 152330936 byte allocation with 25165824 free bytes and 40MB until OOM, max allowed footprint 184519848, growth limit 201326592
	at okio.Buffer.readByteArray(Buffer.kt:817)
	at okio.Buffer.readString(Buffer.kt:669)
	at okio.Buffer.readString(Buffer.kt:658)
	at okhttp3.logging.HttpLoggingInterceptor.intercept(HttpLoggingInterceptor.kt:264)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:112)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:87)
	at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.kt:184)
	at okhttp3.RealCall.execute(RealCall.kt:66)
	at com.multipie.cclibrary.Cloud.Google.GoogleAPI.getFile(GoogleAPI.java:376)
	at com.multipie.cclibrary.Cloud.BookDownloaderHelpers.downloadFile(BookDownloaderHelpers.java:39)
	at com.multipie.cclibrary.Cloud.DownloadQueue$ServiceThread.run(DownloadQueue.java:123)
2020-05-10 04:07:39.071:  Log flush

Seems like a memory issue? Is there anything I can do to assist this? I have tried moving the local storage from internal to SD and back again. Any thoughts are appreciated.

PMrev · 05-13-2020, 06:44 PM

Quote:

Originally Posted by malexherron

I have been having issues downloading books from Google Drive library. I am able to log in and view new books as they are added to Calibre, however when I download a book I see the following errors in the logs:

Code:

2020-05-10 04:06:43.418:  Cloud downloadFile: url=Gary Gygax/D&D 5e - Baldur's Gate - Descent into Avernus (516)/D&D 5e - Baldur's Gate - Descent into Aver - Gary Gygax.pdf, localLpath=D&D 5e - Baldur's Gate - Descent into Avernus - Gary Gygax.pdf
2020-05-10 04:07:07.454:  google getFile
java.lang.OutOfMemoryError: Failed to allocate a 139927976 byte allocation with 25149440 free bytes and 53MB until OOM, max allowed footprint 170714096, growth limit 201326592
	at java.lang.StringFactory.newStringFromBytes(StringFactory.java:81)
	at java.lang.StringFactory.newStringFromBytes(StringFactory.java:209)
	at okio.Buffer.readString(Buffer.kt:669)
	at okio.Buffer.readString(Buffer.kt:658)
	at okhttp3.logging.HttpLoggingInterceptor.intercept(HttpLoggingInterceptor.kt:264)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:112)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:87)
	at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.kt:184)
	at okhttp3.RealCall.execute(RealCall.kt:66)
	at com.multipie.cclibrary.Cloud.Google.GoogleAPI.getFile(GoogleAPI.java:376)
	at com.multipie.cclibrary.Cloud.BookDownloaderHelpers.downloadFile(BookDownloaderHelpers.java:39)
	at com.multipie.cclibrary.Cloud.DownloadQueue$ServiceThread.run(DownloadQueue.java:123)
2020-05-10 04:07:07.457:  Log flush
2020-05-10 04:07:07.644:  FileManager: deleting book /storage/5DA6-BDE6/CalibreCompanion/D&D 5e - Baldur's Gate - Descent into Avernus - Gary Gygax.pdf, result true
2020-05-10 04:07:07.659:  Download queue: try to download 511 D&D 5e - Lost Mine of Phandelver
2020-05-10 04:07:07.663:  Cloud: download failed: D&D 5e - Baldur's Gate - Descent into Avernus
2020-05-10 04:07:07.740:  checkLpathComponents started D&D 5e - Lost Mine of Phandelver - Gary Gygax w/URI=true
2020-05-10 04:07:07.926:  checkLpathComponents finished D&D 5e - Lost Mine of Phandelver - Gary Gygax
2020-05-10 04:07:07.927:  Cloud downloadFile: url=Gary Gygax/D&D 5e - Lost Mine of Phandelver (511)/D&D 5e - Lost Mine of Phandelver - Gary Gygax.pdf, localLpath=D&D 5e - Lost Mine of Phandelver - Gary Gygax.pdf
2020-05-10 04:07:39.066:  google getFile
java.lang.OutOfMemoryError: Failed to allocate a 152330936 byte allocation with 25165824 free bytes and 40MB until OOM, max allowed footprint 184519848, growth limit 201326592
	at okio.Buffer.readByteArray(Buffer.kt:817)
	at okio.Buffer.readString(Buffer.kt:669)
	at okio.Buffer.readString(Buffer.kt:658)
	at okhttp3.logging.HttpLoggingInterceptor.intercept(HttpLoggingInterceptor.kt:264)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:112)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:87)
	at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.kt:184)
	at okhttp3.RealCall.execute(RealCall.kt:66)
	at com.multipie.cclibrary.Cloud.Google.GoogleAPI.getFile(GoogleAPI.java:376)
	at com.multipie.cclibrary.Cloud.BookDownloaderHelpers.downloadFile(BookDownloaderHelpers.java:39)
	at com.multipie.cclibrary.Cloud.DownloadQueue$ServiceThread.run(DownloadQueue.java:123)
2020-05-10 04:07:39.071:  Log flush

Seems like a memory issue? Is there anything I can do to assist this? I have tried moving the local storage from internal to SD and back again. Any thoughts are appreciated.

Hey,

Thanks for sending this in, let me take this back to our dev team and I'll get back to you.

Thanks,
Alex

malexherron · 05-14-2020, 03:30 PM

Appreciate the response, I will standby for a resolution. Thanks!

malexherron · 05-19-2020, 12:07 AM

Still occurring in 5.4.4.9

PMrev · 05-19-2020, 01:38 PM

Quote:

Originally Posted by malexherron

Still occurring in 5.4.4.9

Hey,

Still looking into issue.

Sorry for the delay!

Thanks,
Alex

chaley · 05-19-2020, 02:07 PM

Someone might want to look at why the books are 140GB and 152GB long respectively. They are rather (!) large.

stumped · 05-19-2020, 03:28 PM

Quote:

Originally Posted by chaley

Someone might want to look at why the books are 140GB and 152GB long respectively. They are rather (!) large.

Also, checking Amazon etc. , those titles are sold only as hardback books.
Any pdf versions are from not legit sources.

Botched conversions which report invalid file sizes?

JSWolf · 05-19-2020, 03:30 PM

Quote:

Originally Posted by stumped

Also, checking Amazon etc. , those titles are sold only as hardback books.
Any pdf versions are from not legit sources.

Botched conversions which report invalid file sizes?

Given that these books are illegal, I would say DO NOT HELP THE OP WITH ILLEGAL EBOOKS!

stumped · 05-19-2020, 04:04 PM

Quote:

Originally Posted by JSWolf

Given that these books are illegal, I would say DO NOT HELP THE OP WITH ILLEGAL EBOOKS!

I carefully did not say that. Maybe he bought the hardbacks and made the botched pdf versions from those.

A screenshot showing the reported source file sizes of the files that CC cannot download may be a clue
I doubt he has a 146Gb Google drive !

vikarti · 05-20-2020, 04:47 AM

Size alone is not reason not to fix issue.

I do have some earlier (before they went subscription) Meng To (https://designcode.io/) books. 593 and 1361 Mb. Epubs. (epub3 with videos). They are paid-for.
I also do have Tales from Shakespeare (version for testing read-aloud functionality) (As far as I understood - it's public domain book from public domain sources. Check at https://www.readbeyond.it/ebooks.html )

Many raywenderlich.com tutorials in 100-250 Mb range (at least ones I bought from them)(they have both epub3 and PDF versions).
Some graphical stories I got from my subscriptions at https://subscribestar.adult/ are in 350+ Mb range.

If there is a problem with large books - it should be fixed if possible.
It shouldn't matter how exactly this issue was found.

I check OneDrive with Shackespeare's ReadAloud epub book from readbeyond and 1+ Gb one from Meng To. CC crashes.
Crashlog looks different (but contains several
2020-05-20 14:54:19.665: uncaught exception
java.lang.OutOfMemoryError: OutOfMemoryError thrown while trying to throw OutOfMemoryError; no stack trace available)
Full crashlog send using in-app reporting function.
So there is yet another problem with cloud library integration -

.

JSWolf · 05-20-2020, 05:13 AM

One thing I just thought of is to switch to the 6-bit Java. 32-bit Java could be running out of memory given it only has a 3GB limit. If the OS is 32-bit, it's time to update that to 64-bit as 32-bit Java is rather poor these days.

stumped · 05-20-2020, 05:42 AM

Quote:

Originally Posted by JSWolf

One thing I just thought of is to switch to the 6-bit Java. 32-bit Java could be running out of memory given it only has a 3GB limit. If the OS is 32-bit, it's time to update that to 64-bit as 32-bit Java is rather poor these days.

if it is the java code in the CC app that is struggling, very few mobile devices have >3Gb memory.
3GB memory should be plenty for moving a 300Mb book- all it is doing is a file copy

it may help, with screenshots of source folders , to establish at what size the app crashes. I recall moving a 100Mb sized book with an older version, of CC [ from content server ,. not cloud. i have not re-tested that with latest version ]

it may also help to test if the crash is only with cloud solution or if it is also with library on SD card or with calibre content server/wireless device connectivity options

vikarti · 05-20-2020, 06:36 AM

My tablet on which I tested large books with OneDrive have 4 Gb RAM and have 64-bit android.

stumped · 05-20-2020, 06:58 AM

if you can get those books ok via a different app, that will prove it is a CC issue, not a books issue
e.g. with moon reader, try connect directly to calibre OPDS server
another elimination proof would be to simply export/download the books from google drive to device, leaving out CC,

PS that post suggesting books size was 150Gb - is that a miscalulation? what size shows within your google drive

stumped · 05-20-2020, 07:27 AM

i will test. i have, archived, 2 enormous pdf game guides, one is 150Mb the other is 270mb
I will put them both back in to my calibre library then try to get them with CC 4.4.9 via content server. using my lower spec tablet ( fire HD10 with google stuff added... )

... both downloaded OK. just a few seconds per book ( I have fast wifi)

so I wonder if your issue is specific to how google drive works

i have nothing bigger to test with, sorry
don't really want to bother uploading those to my dropbox as I am confident they would download OK from there also

05-20-2020, 04:47 AM	#10
vikarti Groupie Posts: 185 Karma: 1004974 Join Date: May 2013 Location: Solar system, Earth Device: none	Size alone is not reason not to fix issue. I do have some earlier (before they went subscription) Meng To (https://designcode.io/) books. 593 and 1361 Mb. Epubs. (epub3 with videos). They are paid-for. I also do have Tales from Shakespeare (version for testing read-aloud functionality) (As far as I understood - it's public domain book from public domain sources. Check at https://www.readbeyond.it/ebooks.html ) Many raywenderlich.com tutorials in 100-250 Mb range (at least ones I bought from them)(they have both epub3 and PDF versions). Some graphical stories I got from my subscriptions at https://subscribestar.adult/ are in 350+ Mb range. If there is a problem with large books - it should be fixed if possible. It shouldn't matter how exactly this issue was found. I check OneDrive with Shackespeare's ReadAloud epub book from readbeyond and 1+ Gb one from Meng To. CC crashes. Crashlog looks different (but contains several 2020-05-20 14:54:19.665: uncaught exception java.lang.OutOfMemoryError: OutOfMemoryError thrown while trying to throw OutOfMemoryError; no stack trace available) Full crashlog send using in-app reporting function. So there is yet another problem with cloud library integration -. Last edited by vikarti; 05-20-2020 at 05:04 AM. Reason: results of my experiment with OneDrive

05-20-2020, 06:58 AM	#14
stumped Wizard Posts: 3,305 Karma: 10259306 Join Date: May 2016 Device: kobo forma, Kobo Libra, Huawei media Tab, fire HD10, PW3 HDX8.9,	if you can get those books ok via a different app, that will prove it is a CC issue, not a books issue e.g. with moon reader, try connect directly to calibre OPDS server another elimination proof would be to simply export/download the books from google drive to device, leaving out CC, PS that post suggesting books size was 150Gb - is that a miscalulation? what size shows within your google drive Last edited by stumped; 05-20-2020 at 07:02 AM.

05-20-2020, 07:27 AM	#15
stumped Wizard Posts: 3,305 Karma: 10259306 Join Date: May 2016 Device: kobo forma, Kobo Libra, Huawei media Tab, fire HD10, PW3 HDX8.9,	i will test. i have, archived, 2 enormous pdf game guides, one is 150Mb the other is 270mb I will put them both back in to my calibre library then try to get them with CC 4.4.9 via content server. using my lower spec tablet ( fire HD10 with google stuff added... ) ... both downloaded OK. just a few seconds per book ( I have fast wifi) so I wonder if your issue is specific to how google drive works i have nothing bigger to test with, sorry don't really want to bother uploading those to my dropbox as I am confident they would download OK from there also Last edited by stumped; 05-20-2020 at 07:29 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Unable to access purchase history due to a server error	Nascarfan1972	Kobo Reader	1	09-01-2015 12:36 PM
Win 7 backup error due to Calibre	NSILMike	Devices	2	09-17-2014 08:22 AM
PRS-600 Cannot connect due to USB error?	Griezz	Sony Reader	3	08-16-2010 06:05 AM
Effect On Battery Life Due To Ebook Formats And Memory Cards	Solitaire1	Sony Reader	1	11-14-2009 05:08 AM
Operation failed due to an error	loromo	Sony Reader	2	07-22-2008 08:34 PM

05-14-2020, 03:30 PM	#3
malexherron Junior Member Posts: 8 Karma: 10 Join Date: May 2020 Device: Android	Appreciate the response, I will standby for a resolution. Thanks!

05-19-2020, 12:07 AM	#4
malexherron Junior Member Posts: 8 Karma: 10 Join Date: May 2020 Device: Android	Still occurring in 5.4.4.9

05-19-2020, 02:07 PM	#6
chaley Grand Sorcerer Posts: 12,411 Karma: 8012664 Join Date: Jan 2010 Location: Notts, England Device: Kobo Libra 2	Someone might want to look at why the books are 140GB and 152GB long respectively. They are rather (!) large.

05-20-2020, 05:13 AM	#11
JSWolf Resident Curmudgeon Posts: 79,657 Karma: 145864619 Join Date: Nov 2006 Location: Roslindale, Massachusetts Device: Kobo Libra 2, Kobo Aura H2O, PRS-650, PRS-T1, nook STR, PW3	One thing I just thought of is to switch to the 6-bit Java. 32-bit Java could be running out of memory given it only has a 3GB limit. If the OS is 32-bit, it's time to update that to 64-bit as 32-bit Java is rather poor these days.

05-20-2020, 06:36 AM	#13
vikarti Groupie Posts: 185 Karma: 1004974 Join Date: May 2013 Location: Solar system, Earth Device: none	My tablet on which I tested large books with OneDrive have 4 Gb RAM and have 64-bit android.

Advert

Advert