Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 08-18-2024, 01:21 PM   #9826
Wolgan
Enthusiast
Wolgan began at the beginning.
 
Wolgan's Avatar
 
Posts: 30
Karma: 18
Join Date: Jun 2017
Device: Kindle Fire
Email ParsingPerformance Bug? Maybe?

I use a lot of email subscriptions to keep up to date on stories, and I've noticed that when I use the Get Story URLs from Email option, it seems to take significantly longer to parse RoyalRoad notifications than any other. It'll speed through 10-15 emails in as many seconds, then sit at a single Royal Road email notification for upwards of 30 seconds.

I had let my notifications build up for quite a while, and had a few hundred for it to go through, and watching as messages ticked over to Read in my mailbox as FFF scanned through them it was very consistent (and fascinating, in a nerdish sort of way). 1-2 seconds per email (at most), until it hits an RR email, then it would hang for a relatively long time.

I'm happy to pack some of the emails up and/or forward to you if it'll help.

Potentially useful info:
  • Using dedicated gmail mailbox for notifications.
  • Incoming stories are tagged with the "Story Updates" label by gmail rules on receipt.
  • Other sites that I get notification for that do not have this issue include: StoriesOnline (via feedrabbit.com rss-to-email service), AO3, fanfiction.net, and hentaifoundry.

Screenshot of my FFF email settings

This is minor, as I usually just work on other stuff while it pulls emails, so low priority/impact (for me anyway).

edit: I dug into the email and I see RR is using custom links that have to be translated/resolved to actual story urls. That would explain the delay and I can't think of any obvious way to get around it. I think I'll start shuffling those off to a different folder/label and manually calling for updates directly from the existing stories in Calibre. I don't have too many of those anyway.

Last edited by Wolgan; 08-18-2024 at 01:35 PM.
Wolgan is offline   Reply With Quote
Old 08-18-2024, 01:40 PM   #9827
citac
Fanatic
citac ought to be getting tired of karma fortunes by now.citac ought to be getting tired of karma fortunes by now.citac ought to be getting tired of karma fortunes by now.citac ought to be getting tired of karma fortunes by now.citac ought to be getting tired of karma fortunes by now.citac ought to be getting tired of karma fortunes by now.citac ought to be getting tired of karma fortunes by now.citac ought to be getting tired of karma fortunes by now.citac ought to be getting tired of karma fortunes by now.citac ought to be getting tired of karma fortunes by now.citac ought to be getting tired of karma fortunes by now.
 
Posts: 556
Karma: 1020204
Join Date: Sep 2008
Location: Bosnia and Herzegovina
Device: Lenovo Yoga Tab 2 (Android)
Quote:
Originally Posted by JimmXinu View Post
If you delete everything in personal.ini, the default example will be copied in. I do not recommend copying defaults.ini.

Each library has it's own FFF settings. Changes in one do not affect any other libraries.

For example, I have a copy of libraries used just for testing different FFF settings.
Oh, that's good to know about both the resetting the personal.ini and different settings for libraries, it makes me feel better about experimenting.

I think I need to go back to the metadata management for fanfic to try and work out what exactly I need to see in my library and what custom columns I need to create.

In preparation, I'm hoping that someone might know how to deal with a situation where an author has several series arcs that all follow each other. When sorting by series, they are not sorted in the arc order, but by series. I would like to see my collection sorted by fandom, author and series, and for the series arcs to be sorted correctly. I don't think that's what the multiseries is for - that's more for one fic belonging to several series, correct? How do I achieve what I need?
citac is offline   Reply With Quote
Advert
Old 08-18-2024, 02:11 PM   #9828
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,556
Karma: 4277329
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
Quote:
Originally Posted by Wolgan View Post
edit: I dug into the email and I see RR is using custom links that have to be translated/resolved to actual story urls. That would explain the delay and I can't think of any obvious way to get around it. I think I'll start shuffling those off to a different folder/label and manually calling for updates directly from the existing stories in Calibre. I don't have too many of those anyway.
Yeah, that's special code for RR. I'm a bit surprised I put that in. Somebody may have contributed it--or nagged especially hard for it.
JimmXinu is offline   Reply With Quote
Old 08-21-2024, 04:24 PM   #9829
Clem2605
Enthusiast
Clem2605 began at the beginning.
 
Posts: 37
Karma: 10
Join Date: May 2020
Device: Kobo H2O, Kobo Libra H2O, Boox Tab Ultra C
Hello!

Recently, the option to 'update Antology epub' doesn't work on my end, including in cases where it worked before. The error message tells me 'Book isn't an FanFicFare Antology or contains book(s) without valid Story URLs.'

My anthologies come exclusively from AO3 series and I know there was an issue recently with the changes AO3 made to paginate those bigger than 20 (?) stories, so I'm guessing the problem comes from here especially since the timeline fit. However, the issue comes with all anthologies I downloaded before you adapted the plugin, not only the bigger ones.

When I re-download the stories, it works, so my guess is that the URLs have changed and manually changing them could resolve the problem, but I don't know where those are.
Where do I need to modify the URLs so the FFF recognizes my books?

Thanks!

Edit: Sorry, I was debugging while writing so my post is a bit weirdly written, since I don't have that many anthologies, I hadn't noticed those downloaded recently work without issue.

Last edited by Clem2605; 08-21-2024 at 04:26 PM.
Clem2605 is offline   Reply With Quote
Old 08-21-2024, 04:30 PM   #9830
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,556
Karma: 4277329
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
To the best of recollection, no, series URLs didn't change.

Are you sure you didn't edit/polish/convert/etc the anthology books in question?

Do you have one (SFW only please) you could post for me to look at?
JimmXinu is offline   Reply With Quote
Advert
Old 08-21-2024, 05:26 PM   #9831
Clem2605
Enthusiast
Clem2605 began at the beginning.
 
Posts: 37
Karma: 10
Join Date: May 2020
Device: Kobo H2O, Kobo Libra H2O, Boox Tab Ultra C
Quote:
Originally Posted by JimmXinu View Post
To the best of recollection, no, series URLs didn't change.

Are you sure you didn't edit/polish/convert/etc the anthology books in question?

Do you have one (SFW only please) you could post for me to look at?
I only edit the metadata of my incomplete fanfictions, and I keep track of the complete ones I've edited, so it seems farfetched, especially since, as far as I can tell, all of my antologies downloaded before june/july are concerned.

I've included one example (it should be sfw), here is the AO3 link, if needed:
https://archiveofourown.org/series/3741151 (removed since I've found the issue)

Edit: Now that I think about it, I do convert from epub to epub when I want to change the default generated cover, I'm gonna try and see if it is the issue

Edit 2: Yes it was! Thank you! But now that we have established that, does that mean that I can't convert the antologies that I may have to update in the future or is there a way to make it work?

Last edited by Clem2605; 08-21-2024 at 05:32 PM.
Clem2605 is offline   Reply With Quote
Old 08-21-2024, 06:36 PM   #9832
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,556
Karma: 4277329
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
Quote:
Originally Posted by Clem2605 View Post
Edit: Now that I think about it, I do convert from epub to epub when I want to change the default generated cover, I'm gonna try and see if it is the issue

Edit 2: Yes it was! Thank you! But now that we have established that, does that mean that I can't convert the antologies that I may have to update in the future or is there a way to make it work?
Correct.

FFF's anthology feature relies on EpubMerge's UnMerge feature, which in turn relies on the epub not changing too much. In this case, not discarding 'unused' metadata files from the individual epubs.

Calibre's Convert, while a great tool, rewrites everything.

But! You can use Calibre's "Polish book" to update the cover with far less processing and impact. My tests show it doesn't break anthologies.
JimmXinu is offline   Reply With Quote
Old 08-22-2024, 10:30 AM   #9833
Clem2605
Enthusiast
Clem2605 began at the beginning.
 
Posts: 37
Karma: 10
Join Date: May 2020
Device: Kobo H2O, Kobo Libra H2O, Boox Tab Ultra C
Thank you!

I had hoped to find a solution that didn't include re-downloading, but it seems this isn't my lucky day...

Have a good day!
Clem2605 is offline   Reply With Quote
Old 08-25-2024, 08:32 PM   #9834
icecold
Enthusiast
icecold began at the beginning.
 
Posts: 39
Karma: 10
Join Date: Mar 2020
Device: Android,
Is there a way to normalize SV/SB urls so that if the thread title changes FFF/Calibre won't think it's a new ebook?
icecold is offline   Reply With Quote
Old 08-25-2024, 09:31 PM   #9835
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,556
Karma: 4277329
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
Not currently.

We have special code in the plugin for fanfiction.net and scribblehub.com to search for existing story URLs ignoring the title part, but no other sites.

For ffnet, that code was added when we started keeping the title part of story URLs instead of discarding it. Much more recently, somebody nagged me about it on scribblehub until I added it. Some author was changing their story title every day or something.

You can work around it by choosing the story in your library and updating it from there instead of by URL (or email).


This isn't something I've seen very much on SB/SV. Are other people seeing it happen a lot? For specific authors/subforums/fandoms, or across board?
JimmXinu is offline   Reply With Quote
Old 08-25-2024, 09:52 PM   #9836
icecold
Enthusiast
icecold began at the beginning.
 
Posts: 39
Karma: 10
Join Date: Mar 2020
Device: Android,
I know of one quest that changes its title every so often when the main character changes. It happened recently to one thread I follow that added Peggy Sue to the title. I want to say that it happens at least once every three months, though I haven't exactly been keeping track of how often it happens. One author I follow just went off hiatus and changed his two stories' titles to reflect that this week.

Edit: I haven't exactly noticed any particular one forum (SB/SV/QQ) doing thread renaming more often than the other forums.

Last edited by icecold; 08-25-2024 at 10:16 PM. Reason: Avoiding double post
icecold is offline   Reply With Quote
Old 08-27-2024, 01:18 AM   #9837
booster-t
Member
booster-t knows what time it isbooster-t knows what time it isbooster-t knows what time it isbooster-t knows what time it isbooster-t knows what time it isbooster-t knows what time it isbooster-t knows what time it isbooster-t knows what time it isbooster-t knows what time it isbooster-t knows what time it isbooster-t knows what time it is
 
booster-t's Avatar
 
Posts: 21
Karma: 2464
Join Date: Dec 2020
Location: Vancouver, Canada
Device: iPad
Here's a new one for me ... all of a sudden, I'm getting this error when trying to download from FFN. —> Browser Cache Failed to Load with error ''utf-8' codec can't decode byte 0x92 in position 0: invalid start byte'. <—

The 0x92 position changes from time to time. I haven't changed anythng since yesterday other than to clear the cache. I'm on a Mac OSX 14.6.1

Stephen
booster-t is offline   Reply With Quote
Old 08-27-2024, 09:28 AM   #9838
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,556
Karma: 4277329
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
It's hard to say from just that. Could be a simple problem, or it could be unfixable.

Can you get a debug log? Restart in debug mode under Calibre's Preferences dropdown, try to download, quit Calibre, save the debug log txt file. Zip the file, post it here.
JimmXinu is offline   Reply With Quote
Old 08-27-2024, 11:43 AM   #9839
booster-t
Member
booster-t knows what time it isbooster-t knows what time it isbooster-t knows what time it isbooster-t knows what time it isbooster-t knows what time it isbooster-t knows what time it isbooster-t knows what time it isbooster-t knows what time it isbooster-t knows what time it isbooster-t knows what time it isbooster-t knows what time it is
 
booster-t's Avatar
 
Posts: 21
Karma: 2464
Join Date: Dec 2020
Location: Vancouver, Canada
Device: iPad
Quote:
Originally Posted by JimmXinu View Post
It's hard to say from just that. Could be a simple problem, or it could be unfixable.

Can you get a debug log? Restart in debug mode under Calibre's Preferences dropdown, try to download, quit Calibre, save the debug log txt file. Zip the file, post it here.
OK Jim ... here's the log.

Also, I tried this with an existing story and got a different offset. But, fichub can download the file and if I rename it to what the file is called in Calibre, it seamlessly appears. Just have to manually change the update date :-)
Attached Files
File Type: zip tmphwhdt0hd.txt.zip (2.7 KB, 67 views)

Last edited by booster-t; 08-27-2024 at 12:06 PM.
booster-t is offline   Reply With Quote
Old 08-27-2024, 01:01 PM   #9840
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,556
Karma: 4277329
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
The log says that a cache file was found, but that it failed to read the headers.

Works fine for me on Win10, so I borrowed a Mac and I do see the same error there.

Honestly, that's good news--troubleshooting a problem I can see is many times easier than one I can't. More to follow.
JimmXinu is offline   Reply With Quote
Reply

Tags
fanfiction

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] KindleUnpack - The Plugin DiapDealer Plugins 494 05-18-2024 02:34 AM
[GUI Plugin] Open With kiwidude Plugins 403 04-01-2024 08:39 AM
[GUI Plugin] Marvin XD Philantrop Plugins 126 01-29-2017 12:48 PM
[GUI Plugin] KiNotes -axel- Plugins 0 07-14-2013 06:39 PM
[GUI Plugin] Plugin Updater **Deprecated** kiwidude Plugins 159 06-19-2011 12:27 PM


All times are GMT -4. The time now is 06:25 AM.


MobileRead.com is a privately owned, operated and funded community.