Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Related Tools

Notices

Reply
 
Thread Tools Search this Thread
Old 10-14-2013, 07:57 PM   #31
cyana
Enthusiast
cyana began at the beginning.
 
cyana's Avatar
 
Posts: 37
Karma: 10
Join Date: Dec 2011
Device: iPad
Thanks for your reply!
I'm running into another problem now - not sure but it seems to have to do with new setting options of 3.2 233 - it appears that no download links are generated in the books pages, also new ebooks are not uploaded anymore. I'm not aware of any setting changes I made, I simply installed 3.2 over 3.1.

I recall that in 3.1 you must not set the option 'Do not generate OPDS downloads' even when 'Do not generate HTML downloads' is unchecked it will lead to this result. However, in 3.2 it seems that when you choose not to generate an OPDS catalog, the 'Do not generate OPDS downloads' - option is checked automatically and unchangeable resulting in missing html-download links (?)

thank you
cyana
cyana is offline   Reply With Quote
Old 10-14-2013, 11:39 PM   #32
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,552
Karma: 950151
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
Quote:
Originally Posted by cyana View Post
Thanks for your reply!
I'm running into another problem now - not sure but it seems to have to do with new setting options of 3.2 233 - it appears that no download links are generated in the books pages, also new ebooks are not uploaded anymore. I'm not aware of any setting changes I made, I simply installed 3.2 over 3.1.
You may have come across a bug where you cannot successfully do more than one generate run without reloading calibre2opds. The second (and subsequent) attempts to generate have the symptoms you describe. It is due to a performance optimisation that can incorrectly cache information between generate runs. You have to exit/restart calibre2opds between the generate runs to avoid this. A note about this has been added to the download links for calibre2opds.

Quote:
I recall that in 3.1 you must not set the option 'Do not generate OPDS downloads' even when 'Do not generate HTML downloads' is unchecked it will lead to this result. However, in 3.2 it seems that when you choose not to generate an OPDS catalog, the 'Do not generate OPDS downloads' - option is checked automatically and unchangeable resulting in missing html-download links (?)
I'll check this out. It sounds as if there is something wrong in the GUI code that tries to track the allowed combinations. I need to look at the internal logic. It may be that there should not be separate entries for OPDS and HTML downloads - just a single one saying whether download links are wanted. Because of the way HTML is generated from the XML that makes up an OPDS catalog (which still happens internally even when you do not want am OPDS catalog at the end) it may not be possible to successfully specify separate download settings.

EDIT: I have checked the code and the structure now DOES allow for HTML downloads to be enabled or disabled when OPDS catalogs are disabled. The option that is not allowed is to have an OPDS catalog with downloads disabled AND an HTML catalog with them enabled. I will update the GUI code so that this is the restriction that is enforced.

Keep the bug reports flowing! The number of people who are actively involved in calibre2opds during development cycles has dropped which means that it seems more of these type of errors get missed during testing. Despite that the program should still be improving I hope .

EDIT: I have raised issues on the Calibre2opds issue register for the points you have raised. Perhaps you might check these to see if I have missed anything or got any of the details wrong?

I should shortly be able to make a build available that although it will be labelled as a 3.3 beta it will primarily be the 3.2 release with reported bugs fixed as although 3.3 development has started the new facilities are not yet ready.

Last edited by itimpi; 10-15-2013 at 02:23 AM.
itimpi is offline   Reply With Quote
Advert
Old 10-15-2013, 10:51 PM   #33
slantybard
my parent's oops...
slantybard ought to be getting tired of karma fortunes by now.slantybard ought to be getting tired of karma fortunes by now.slantybard ought to be getting tired of karma fortunes by now.slantybard ought to be getting tired of karma fortunes by now.slantybard ought to be getting tired of karma fortunes by now.slantybard ought to be getting tired of karma fortunes by now.slantybard ought to be getting tired of karma fortunes by now.slantybard ought to be getting tired of karma fortunes by now.slantybard ought to be getting tired of karma fortunes by now.slantybard ought to be getting tired of karma fortunes by now.slantybard ought to be getting tired of karma fortunes by now.
 
Posts: 463
Karma: 1477570
Join Date: Feb 2009
Device: Vx->Handera->Clie-> Axim->505->650->KPW/Aura ->iOS/L2
FYI the 3.2 - 233 release

Hi itimpi, sorry to bother you but the latest 3.2 233 dmg release on the Mac is broken (I know you built that install for folks like me and I feel bad that it's not working). When I install it into my applications folder and try to load it, the application immediately quits.

My console returns this log when I try and launch COPDS:
2013-10-15 7:35:43.127 PM com.apple.launchd.peruser.501[137]: ([0x0-0x20df0dd].Gui[27235]) Exited with code: 1
slantybard is offline   Reply With Quote
Old 10-17-2013, 01:44 PM   #34
cyana
Enthusiast
cyana began at the beginning.
 
cyana's Avatar
 
Posts: 37
Karma: 10
Join Date: Dec 2011
Device: iPad
Hello itimpi,

thanks for your efforts!

I could now resolve my win7 issues, for all those with the same problem - and these are all who use the WebDAV interface to talk to their cloud/webspace with HTTPS protocol via mounted local drive (as required by calibre2opds for the moment) - it's definitely is a win7 problem, that sometimes was fixed by MS but constantly reappears with new updates. So in one line - you can't get it to work. You need a nice program called NetDrive. It's free for personal use, and available here.

Initially, NetDrive did not work for me, although I entered port 443 into the configuration. You have to go to options, and set the used protocol to HTTPS - a bit odd because it seems to be a global setting, and not dedicated to the site you define for WebDAV access. However, afterwards it works like a charm and you have your cloud as local drive mounted. It's very slow, however this also seems to be a win7 problem.

Calibre2opds is uploading the complete calibre library (2,500 books) and the catalogs at the moment, as all caches were cleared due to the new target folder, and it's busy already since more than 24 hours. The library upload was finished about 10 hours ago, the catalogs are about 15%. 15% in ten hours. I'm off to London on Monday, I only hope the upload is finished till then

Quote:
EDIT: I have checked the code and the structure now DOES allow for HTML downloads to be enabled or disabled when OPDS catalogs are disabled. The option that is not allowed is to have an OPDS catalog with downloads disabled AND an HTML catalog with them enabled. I will update the GUI code so that this is the restriction that is enforced.
That's how it was with 3.1. The problem occurred because you cannot uncheck the 'Do not generate OPDS downloads' box once you decide not to generate opds catalogs; the checkboxes are combined in 3.2 so that when you do not generate OPDS catalogs, the 'Do not generate OPDS downloads' box comes up checked and greyed automatically.

Another bug I came across (actually an old one already from 2.x) - the publication date does not show up in the book page although the box is checked and the date is provided.

And another feature request - a link from the author's name to the author's allbooks-page in every book catalog. At the moment, you have to drill down to the book's page, from where you go back to the author to see all his books.

again, thanks a lot for your great program!
cyana
cyana is offline   Reply With Quote
Old 10-18-2013, 06:07 AM   #35
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,552
Karma: 950151
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
Quote:
Originally Posted by cyana View Post
Calibre2opds is uploading the complete calibre library (2,500 books) and the catalogs at the moment, as all caches were cleared due to the new target folder, and it's busy already since more than 24 hours. The library upload was finished about 10 hours ago, the catalogs are about 15%. 15% in ten hours. I'm off to London on Monday, I only hope the upload is finished till then
Hopefully the options to Minimize Changes will help with keeping down the number of files in the future. I have just implemented a suggestion where if you have both Minimize Changes AND the option to have internal links the counts of books are omitted from the cross-reference link. That means that the book records in the catalog stay unchanged between runs unless you actually change the meta data associated with them. This should make Minimize Changes more efficient. The change will be available in the next build I make available for download (in the next day or so).
Quote:
That's how it was with 3.1. The problem occurred because you cannot uncheck the 'Do not generate OPDS downloads' box once you decide not to generate opds catalogs; the checkboxes are combined in 3.2 so that when you do not generate OPDS catalogs, the 'Do not generate OPDS downloads' box comes up checked and greyed automatically.
The GUI code has now been fixed so this is now possible. The only combination that is now not possible is to have both an OPDS catalog without downloads AND an HTML with downloads. This is now the restriction enforced by the GUI.

Quote:
Another bug I came across (actually an old one already from 2.x) - the publication date does not show up in the book page although the box is checked and the date is provided.
I will check this out. Assuming I can reproduce it, it will be trivial to fix. It has been added as Bug Report to the Calibre2opds issue register.

EDIT:Turns out this bug only occurs if you do not also have the Publication Date as Year option set.

Quote:
And another feature request - a link from the author's name to the author's allbooks-page in every book catalog. At the moment, you have to drill down to the book's page, from where you go back to the author to see all his books.
I like the idea - I will have to see if there is a way of incorporating it that will not break OPDS compatibility for the XML catalogs. I am assuming that the implementation you are thinking of is if in the HTML catalog you have a list of books, then where the top line is a clickable link to the book details, the second line containing authors should be a clickable link to the author? Rather than make the link only to the All-Books for an author I am thinking it would be better going just to the Author details - any thoughts on that? What are you thoughts about multiple authors - should there be separate links for each author. If have raised this a feature request on the Calibre2opds issue register - you can add comments there if you have further ideas.

EDIT:I think the above idea IS achievable enough quite easily. If so then I should extend it also being used in the Book Details pages. It can also be extended for Series/Tags in those pages which would make the explicit Cross-Reference section no longer needed.

Last edited by itimpi; 10-18-2013 at 09:18 AM.
itimpi is offline   Reply With Quote
Advert
Old 10-18-2013, 06:14 AM   #36
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,552
Karma: 950151
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
Quote:
Originally Posted by slantybard View Post
Hi itimpi, sorry to bother you but the latest 3.2 233 dmg release on the Mac is broken (I know you built that install for folks like me and I feel bad that it's not working). When I install it into my applications folder and try to load it, the application immediately quits.

My console returns this log when I try and launch COPDS:
2013-10-15 7:35:43.127 PM com.apple.launchd.peruser.501[137]: ([0x0-0x20df0dd].Gui[27235]) Exited with code: 1
I am not sure if there is anything I can do about this as I do not have a Mac myself. Unfortunately the calibre2opds developers who had access to Macs no longer appear to be active. I can look into whether any of my friends who have macs could help with debugging why it is not working.

I have raised it as a bug report on the Calibre2opds issue register so that it does not get forgotten.
itimpi is offline   Reply With Quote
Old 10-18-2013, 05:43 PM   #37
cyana
Enthusiast
cyana began at the beginning.
 
cyana's Avatar
 
Posts: 37
Karma: 10
Join Date: Dec 2011
Device: iPad
Hello itimpi,

again, thanks a lot for your efforts! Unfortunately, IE 10 crashes when I try to create an account at calibre2opds.myjetbrains.com so I cannot add comments there. However, your implementation of author links would be really great and is exactly what I had in mind. I would not drop the x-refs for tags in the book's page, they are very useful to switch to more books of the given genre.

Meanwhile, the catalog generation is about 30% - my flight to London leaves on Sunday at 1 PM. So I don't think it will make it. What happens to the cache when I cancel the catalog generation? Will it continue at the point where it was interrupted, or will it restart from the ground? If so, a break/continue option would be very helpful - considering that an upload under such conditions might be very likely for win7 users and can take a week, so an unexpected interruption might happen from time to time - however when it is not necessary anymore to create an opds catalog, the upload time will drop significantly. The time consuming part is the sheer number of the files, not their size.

best greetings and thanks!
cyana
cyana is offline   Reply With Quote
Old 10-18-2013, 05:51 PM   #38
cyana
Enthusiast
cyana began at the beginning.
 
cyana's Avatar
 
Posts: 37
Karma: 10
Join Date: Dec 2011
Device: iPad
As for the Mac issue, I have a MacBook here but I'm not experienced with Macs, they are too complicated for me
If someone can tell me how to create a dmg file I'd be happy to help, when I'm back from UK.
cyana is offline   Reply With Quote
Old 10-18-2013, 06:31 PM   #39
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,552
Karma: 950151
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
Quote:
Originally Posted by cyana View Post
again, thanks a lot for your efforts! Unfortunately, IE 10 crashes when I try to create an account at calibre2opds.myjetbrains.com so I cannot add comments there. However, your implementation of author links would be really great and is exactly what I had in mind. I would not drop the x-refs for tags in the book's page, they are very useful to switch to more books of the given genre.
I was just thinking of discontinuing a separate cross reference section on the page as the authors, tags and summaries shown in the book details would become links that are functionally equivalent to the current cross-reference links as long as you want cross-reference links in the first place.

Quote:
Meanwhile, the catalog generation is about 30% - my flight to London leaves on Sunday at 1 PM. So I don't think it will make it. What happens to the cache when I cancel the catalog generation? Will it continue at the point where it was interrupted, or will it restart from the ground? If so, a break/continue option would be very helpful - considering that an upload under such conditions might be very likely for win7 users and can take a week, so an unexpected interruption might happen from time to time - however when it is not necessary anymore to create an opds catalog, the upload time will drop significantly. The time consuming part is the sheer number of the files, not their size.
No - it is not possible to interrupt a generation and resume. There s lots of info cached in RAM so stopping the program loses this.

I must admit that I do not understand why it takes so long to run the generate? I can only assume that the way your system operates is not like most cloud services where the files are held locally - and a background process does the sync to the cloud. Calibre2opds then normally just generates the local copy and lets the sync service run separately at whatever speed it can achieve. It sounds as if you might be trying to write directly to the cloud in real time?

The cache file that is used to help optimise copying is currently only written out at end-of-run. I need to think if there would be any advantage to writing out a partial cache file on an interrupted run. I would have to think about what the effect would be on the next generate run to see if it is likely to speed things up or slow them down.

Last edited by itimpi; 10-18-2013 at 06:37 PM.
itimpi is offline   Reply With Quote
Old 10-18-2013, 08:52 PM   #40
cyana
Enthusiast
cyana began at the beginning.
 
cyana's Avatar
 
Posts: 37
Karma: 10
Join Date: Dec 2011
Device: iPad
Quote:
Originally Posted by itimpi View Post
I must admit that I do not understand why it takes so long to run the generate? I can only assume that the way your system operates is not like most cloud services where the files are held locally - and a background process does the sync to the cloud.
There is a problem with the sync application of my cloud provider. It relies on the WevDAV interface, that seems to be buggy on win7. Anyway, WebDAV also lets you mount a web address as local drive, and this normally would be a great thing. On win7 however, you run into a authentication problem when using a https protocol, were the mutual key gets lost after some minutes and cannot be restored. However, the tool 'NetDrive' can deal with that problem, so I now have a local drive X: that maps directly to my cloud, and I can now enter e.g. X:\eBooks as target folder in calibre2opds.

The bad performance comes from the huge number of files that accumulate in the folders, because they are reread every time when another file is uploaded. So when a folder has 2.000 items and you add the 2001th file, the system first fetches the entire directory, then uploads the file, and so on.
cyana is offline   Reply With Quote
Old 10-20-2013, 11:16 AM   #41
scissors
Addict
scissors ought to be getting tired of karma fortunes by now.scissors ought to be getting tired of karma fortunes by now.scissors ought to be getting tired of karma fortunes by now.scissors ought to be getting tired of karma fortunes by now.scissors ought to be getting tired of karma fortunes by now.scissors ought to be getting tired of karma fortunes by now.scissors ought to be getting tired of karma fortunes by now.scissors ought to be getting tired of karma fortunes by now.scissors ought to be getting tired of karma fortunes by now.scissors ought to be getting tired of karma fortunes by now.scissors ought to be getting tired of karma fortunes by now.
 
Posts: 241
Karma: 1001369
Join Date: Sep 2010
Device: prs300, kindle keyboard 3g
Search facility

Hi Itimpi

The latest 3.2 w32 exe is the first install since 2.3 that actually runs on my system.

Well done

Last edited by scissors; 10-20-2013 at 11:33 AM.
scissors is offline   Reply With Quote
Old 10-20-2013, 05:01 PM   #42
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,552
Karma: 950151
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
Quote:
Originally Posted by scissors View Post
The latest 3.2 w32 exe is the first install since 2.3 that actually runs on my system.
Glad to hear it

It is not obvious, but the internal structure of the code in the 3.2 release is now much cleaner than it used to be. Hopefully this means that there will be less inadvertent errors, and also that development of new capability will be easier.

It is worth pointing out that I have posted a 3.3 beta build which fixes a number of errors reported against the 3.3 release.
itimpi is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Calibre2opds v3 - OPDS catalogs from your Calibre library itimpi Related Tools 31 02-10-2013 04:16 PM
Popular public OPDS catalogs kguil Apple Devices 37 01-01-2013 05:00 PM
OPDS catalogs access knfevg PocketBook 7 12-04-2012 04:55 AM
List of Free Online OPDS Catalogs? Madmick General Discussions 1 01-14-2012 12:09 AM
Call for comments: OPDS Catalogs 0.9 draft (open catalog standard ebooks) zelda_pinwheel News 4 05-14-2010 01:02 PM


All times are GMT -4. The time now is 09:45 AM.


MobileRead.com is a privately owned, operated and funded community.