iOS Calibre Companion error wiping the app empty.

[Josieb1]

I've been using Calibre Companion on my iPhone, it's on my iPad as well. I use it to keep metadata via txt files. I had over 4900 files in the app.

Today I've been trying to update it with an extra 32 files, instead of adding the new files the app disconnected and removed all files in the app. I tried to load them all again in tranches of 1000 at a time. It was working until the end of the third tranch when that disconnected and again wiped the app empty.

Any idea what's going on? I was able to update my ipad app without any problems.

[Josieb1]

Actually my iPad app is empty too now. Both apps have nothing.

[chaley]

It sounds like you ran into the same problem reported in this thread. If so then CC V1.1.1 released a day or 2 ago should fix it. Have you received that new version from Apple?

[Josieb1]

Originally Posted by chaley

It sounds like you ran into the same problem reported in this thread. If so then CC V1.1.1 released a day or 2 ago should fix it. Have you received that new version from Apple?

No not yet. Thanks for the link.

[Josieb1]

I downloaded the app again today from the app store and got version 1.1.0

[Steve_Haley]

As Chaley wrote, this is a bug I've been trying to fix for a while. More detail in that other thread. As for the update, it takes a few days for Apple to approve them. I submitted it end of last week but it didn't get approved until the end of the weekend. By the timestamp on your post, it seems like you checked only a couple of hours before it got approved.

[Josieb1]

Originally Posted by Steve_Haley

As Chaley wrote, this is a bug I've been trying to fix for a while. More detail in that other thread. As for the update, it takes a few days for Apple to approve them. I submitted it end of last week but it didn't get approved until the end of the weekend. By the timestamp on your post, it seems like you checked only a couple of hours before it got approved.

The new version does the same thing. I got 3000 books on the app on my iphone and it crashed doing the next 1000. Right now the IOS cc app is unusable for me.

The new version 1.1.1 came through this morning.

[Josieb1]

I'm not bothering to try and use this app any more I'll have to stick with the android version. Its just a little annoying as i need the ios app more.

[chaley]

Originally Posted by Josieb1

The new version does the same thing. I got 3000 books on the app on my iphone and it crashed doing the next 1000. Right now the IOS cc app is unusable for me.

The new version 1.1.1 came through this morning.

The problem seems to be related to iOS memory management, which is why it is so hard to replicate. Free memory is collected periodically, but the period is unpredictable. An app can run out of memory between collections. I mention this because contrary to what I said in your calibre bug report, the fact you are using TXT files might actually be relevant. It will take *much less time to send empty TXT files than to send real books, meaning books will arrive much more quickly meaning that CC will touch the database far more frequently than "normal". If I am right then sending smaller groups and waiting a bit between them might help.

-----

Ahh, I see your latest post. I will leave what I wrote for posterity. I suggest you contact Steven to get a refund.

[Josieb1]

Originally Posted by chaley

Ahh, I see your latest post. I will leave what I wrote for posterity. I suggest you contact Steven to get a refund.

I'm not interested in a refund as I said to Steven ages ago, I live in hope of an ios app that will one day work for me. Ideally I need the app to work on my phone and I cannot imagine a day where I will have an android phone, but who knows what the future holds.

[Josieb1]

Originally Posted by chaley

The problem seems to be related to iOS memory management, which is why it is so hard to replicate. Free memory is collected periodically, but the period is unpredictable. An app can run out of memory between collections. I mention this because contrary to what I said in your calibre bug report, the fact you are using TXT files might actually be relevant. It will take *much less time to send empty TXT files than to send real books, meaning books will arrive much more quickly meaning that CC will touch the database far more frequently than "normal". If I am right then sending smaller groups and waiting a bit between them might help.

[/URL].

I was using my MAC this time which has 8GB of memory, DH's W10 PC has 12GB, I will install the 64bit calibre on his PC, and later this week I will try again using smaller batches and see what happens. As this problem did affect another person it will be interesting to see if the problem is fixed for them.

I don't mind the crashing tbh if only the app wouldn't wipe itself clean when it does.

[Steve_Haley]

The other thing that will help is to fully close the app (double tap home button then swipe it away) after transferring each 1000 books. The app performs some maintenance on the database at start-up which will help; this only takes a few seconds to do.

There are two problems happening at the same time: (1) the app used to run out of RAM during book transfers. For me, when transferring 1000 books, the app used to use about 120mb and now it only uses about 50mb. This problem manifested itself by the app abruptly crashing (returning to the home screen of the iPad without notice) during book transfers. This really should happen a lot less now.

(2) the database file itself could get too large. This one hasn't been fixed quite as nicely, since it requires the app to be restarted, as mentioned above. However, it should still help; for me, my database file reached approximately 800mb after transferring 1000 books, and then after restarting, the new maintenance code reduced it down to 50mb. Different devices have different limits, but it seems that if the app is launched with a database of between 2 and 4gb, the database corrupts and gets reset. So it's a bit of a pain, but as long as you restart the app between sending batches of 1000 books, you should be able to get further.

(And yes, figuring out a better way of handling this is still on my list!)

[Josieb1]

Originally Posted by Steve_Haley

The other thing that will help is to fully close the app (double tap home button then swipe it away) after transferring each 1000 books. The app performs some maintenance on the database at start-up which will help; this only takes a few seconds to do.

There are two problems happening at the same time: (1) the app used to run out of RAM during book transfers. For me, when transferring 1000 books, the app used to use about 120mb and now it only uses about 50mb. This problem manifested itself by the app abruptly crashing (returning to the home screen of the iPad without notice) during book transfers. This really should happen a lot less now.

(2) the database file itself could get too large. This one hasn't been fixed quite as nicely, since it requires the app to be restarted, as mentioned above. However, it should still help; for me, my database file reached approximately 800mb after transferring 1000 books, and then after restarting, the new maintenance code reduced it down to 50mb. Different devices have different limits, but it seems that if the app is launched with a database of between 2 and 4gb, the database corrupts and gets reset. So it's a bit of a pain, but as long as you restart the app between sending batches of 1000 books, you should be able to get further.

(And yes, figuring out a better way of handling this is still on my list!)

Thanks very much for the advice Steve

Hopefully I'll be able to try this at the weekend when I have a full day to play with hubbys pc.

[Josieb1]

Steve_Haley, using your advice about closing the app after each tranch of files added along with my husbands W10 Pc, which is an I7 with 12GB Ram, and Calibre 64Bit I got all my 4900 TXT files loaded on both my iPad and my iPhone. Hopefully I can just add new txt files as I buy new books now.

The memory issues maybe why the iOS app has only ever worked on my husbands laptop as it's the most powerful laptop we have, my MAC only has 8GB and our W7 laptops even less. Plus the W7 laptops don't have Calibre 64Bit.

Thanks again for the advice

[chaley]

Originally Posted by Josieb1

Steve_Haley, using your advice about closing the app after each tranch of files added along with my husbands W10 Pc, which is an I7 with 12GB Ram, and Calibre 64Bit I got all my 4900 TXT files loaded on both my iPad and my iPhone. Hopefully I can just add new txt files as I buy new books now.

The memory issues maybe why the iOS app has only ever worked on my husbands laptop as it's the most powerful laptop we have, my MAC only has 8GB and our W7 laptops even less. Plus the W7 laptops don't have Calibre 64Bit.

I think I confused things. The memory issue is in the phone/tablet, not in calibre. It calibre running out of memory would give different symptoms and a different log.

Good that it is working, and also good that you added confirmation to one of Steven's theories about what must be happening.