Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Related Tools

Notices

Reply
 
Thread Tools Search this Thread
Old 11-17-2010, 12:58 PM   #61
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Quote:
Originally Posted by ldolse View Post
Edit - I now see that the single quote is escaped in the actual html generated by the PHP, but when the browser actually goes to it it converts it back to a single quote - pasting the URL into the browser location bar will illustrate it. Anyway either way the book cover isn't found.
Interesting.

Even though I will fix the problem by eliminating the quotes, I still wonder about this. I really can't make it fail with Firefox 3,6,12 or IE 8, with servers on Debian and Windows. I am beginning to think that the problem is at the browser end.

Are you using Safari or Chrome? If you use Firefox, does it also fail?
chaley is offline   Reply With Quote
Old 11-17-2010, 05:14 PM   #62
ldolse
Wizard
ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.
 
Posts: 1,337
Karma: 123455
Join Date: Apr 2009
Location: Malaysia
Device: PRS-650, iPhone
Thanks for digging into it some more.

The covers/books are inaccessible from all browsers, Safari/Safari(ios), Firefox, Chrome, etc. That said, the escaping/unescaping behavior does vary. Chrome and Firefox converts it back to actual single quote, safari leaves it as %27.

I can access the books just fine when I manually go the the actual filesystem path URL with a quote, it's just when the quote is used through php that it breaks - somehow the directory checks done by php fail when there is a single quote included. This causes the generic cover to be used as an alternate and the book itself can't be downloaded at all.

I'm thinking perhaps you can't reproduce because there is something different in 5.3 - it does appear some string handling changes were made, maybe this got fixed in the process.
ldolse is offline   Reply With Quote
Advert
Old 11-17-2010, 11:23 PM   #63
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Quote:
Originally Posted by ldolse View Post
I'm thinking perhaps you can't reproduce because there is something different in 5.3 - it does appear some string handling changes were made, maybe this got fixed in the process.
I run 5.2 on my windows machine for testing.

As I will make the problem go away by getting rid of the title strings (end of this month), I am going to chalk this behavior up to gremlins.
chaley is offline   Reply With Quote
Old 11-27-2010, 04:01 AM   #64
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Release 0.2.0 is available at the usual place. The major change is to switch to using calibre database IDs instead of text for cover and format fetching (along with associated security and performance changes), which is the reason for the change from V0.1 to V0.2.

One note: while testing I discovered that memory used by the PHP function used to resize cover images can explode, sometimes exceeding server limits. If some covers aren't showing, then check the size of the cover image. Experimentation indicates that cover images less than 1000 x 1000 pixels seldom (if ever) exhibit this problem.

Changelog:
Spoiler:
Code:
45 charles <>	2010-11-26 {REL 0.2.0}
      Release

   43 charles <>	2010-11-26
      Add some missing mimetypes (rtf, lit, doc, zip)

   42 charles <>	2010-11-26
      Add 'go to page' capability to default template.

   41 charles <>	2010-11-26
      Change version number.

   40 charles <>	2010-11-26
      Change cover and book format to use the DB ID instead of the path string. Eliminates errors with quotes in file names.
chaley is offline   Reply With Quote
Old 11-27-2010, 07:40 AM   #65
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Quote:
Originally Posted by hakan42 View Post
[...] could you please summarize and share the differences between your work (the PHP scripts) and the Calibre Content Server as it is integrated in calibre itself? I'm thinking in direction of integrating your results into the main server, and here knowing what and why differences there are instead of analyzing your output surely helps
(This conversation started in https://www.mobileread.com/forums/sho...42#post1238342)

I am not quite sure what you are asking for, but I will give it a try.

The built-in calibre server has more functionality than the PHP server. I implemented the functionality subset that I needed, and have added some things asked for by Idolse. For example, the PHP server does not support OPDS, and there are significant differences in search capabilities.

There are massive differences in architecture over and above the PHP/python differences.

- The built-in calibre server is a constantly-running process. It opens the database when it starts and holds it open. The upside is that it can cache some amount of data. The downside is that it must detect database changes and do the correct thing. The PHP server reopens the database on every request. It computes what it needs at that time, produces the output, then shuts down. The upside is that there is no issue with database changes, as it sees the newest automatically. The downside is that it must recompute information that in fact doesn't change often, such as category contents.

- The PHP server runs under an existing web server (apache or IIS), making some things easier and some things harder. For example, running under SSL, multiple users, and security are all taken care of by the containing server. The downside is setup. You must configure a containing server.

- The built-in server uses AJAX. Javascript on the client asks the calibre server for book information as needed, formats that information, then displays it. The PHP server uses a more 'normal' HTML approach, formatting the information on the server, then sending it. Because of this, the PHP server can easily support modifiable page templates. The calibre server requires hacking the javascript. In addition, the PHP server is better for less-intelligent devices (e.g., phones and readers) that can't handle AJAX.

- The PHP server has a smaller footprint than the built-in server, both in terms of installation prerequisites and resident memory.
chaley is offline   Reply With Quote
Advert
Old 11-29-2010, 06:36 AM   #66
ldolse
Wizard
ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.
 
Posts: 1,337
Karma: 123455
Join Date: Apr 2009
Location: Malaysia
Device: PRS-650, iPhone
I got the new php server installed - so far all problems solved, Thanks!
ldolse is offline   Reply With Quote
Old 11-29-2010, 11:11 AM   #67
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,775
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
@chaley: Just FYI, in the new content server code, hacking javascript to change layout isn't needed anymore. It uses templates available in resources/content_server/browse
kovidgoyal is online now   Reply With Quote
Old 12-05-2010, 06:06 PM   #68
hakan42
Zealot
hakan42 is on a distinguished road
 
hakan42's Avatar
 
Posts: 136
Karma: 60
Join Date: Jul 2009
Location: Munich, Germany
Device: Nook Classic rooted; Galaxy S IV with Aldiko, other older devices
@kovid: Could the calibre content server made to run in an read-only mode? My main target is making my collection visible to my mobile devices while being alway in sync with the main code base, but the library itself "belongs" to a seperate unix user.

I tried to get the mod_wsgi version from http://calibre-ebook.com/user_manual/server.html to run, but this fails immediately because the wsgi server tries to write to the calibre db... And for the mod_proxy solution, I am somehow too stupid and tired to really see while the rewrite rules are not executed


Regards,
Hakan
hakan42 is offline   Reply With Quote
Old 12-05-2010, 07:49 PM   #69
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,775
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
the python sqlite module cannot open the db in read only ode, so no.
kovidgoyal is online now   Reply With Quote
Old 12-06-2010, 10:35 AM   #70
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
New Release

Release 0.2.1 is available at the usual place. This release adds sorting when on the books view.
chaley is offline   Reply With Quote
Old 12-08-2010, 10:03 AM   #71
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
New Release

Release 0.2.1 is available at the usual place.

This release adds two significant new features:
- The possibility of a short-form display (with thanks to guz/asam for the idea). If you set $config['use_short_form'] = true, then the standard books display will show a limited number of fields and a truncated comments field. The server adds a link to the book that takes the user to the full display for that book. The fields to show in the books display are listed in the config variable $config['fields_in_short_form']. The number of comment characters to display is set in the config variable $config['short_form_comments_length'].

- Where it makes sense, field items are now search URLs. This means that a user can click on a tag to see all books with that tag. Same thing works for all other text-based fields. The creation of URLs is controlled by the config variable $config['fields_to_make_urls'], which has the same syntax as the fields_to_display config variable.

Finally, some links have been moved around to make the display more compact. In particular, the format links are now under the author.

If you have customized your templates, then you may need to change the book template to take advantage of short form or see the moved links.
chaley is offline   Reply With Quote
Old 12-12-2010, 03:44 PM   #72
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
FYI: I have found a reasonably significant bug in V0.2.1 and all earlier versions. The previous/next page links do not work correctly for book displays based on a category.

I have fixed the problem and will release it at some point. I am not in a hurry, given that no one has reported the bug. The next release (when it happens) will also change the URLs on categories from search URLs to category URLs.
chaley is offline   Reply With Quote
Old 12-12-2010, 10:18 PM   #73
ldolse
Wizard
ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.
 
Posts: 1,337
Karma: 123455
Join Date: Apr 2009
Location: Malaysia
Device: PRS-650, iPhone
Thanks! Just installed it and the new additions seemed to work well, and the 'short_form' option was one I'd considered asking for, so that's great!. One recommended tweak - I accidentally deleted my config file, so had to re-create it from scratch. Images weren't showing because I forgot to set the $config['web_dir'] variable. I just set it to:
Code:
$config['web_dir'] = './';
to fix it -- perhaps ./ should be the default here, since 95% of the time index.php will be launched from the same directory as the rest of the related php files?

One problem I'm having is related to Android/Aldiko - my brother is an Android user. Once Aldiko is installed it associates the 'application/epub+zip' mimetype to itself, and will load the epub once it's downloaded. I have the mimetype correctly configured on my web server, and if I just manually go to the URL of the epub in the browser it downloads and launches Aldiko. However if I try to load the epub using the PHP server the download fails. Android just says the title of the download is <unknown>, and that it was unsuccessful. I haven't tried pcaps or anything like that, but I can give it a shot if need be.
ldolse is offline   Reply With Quote
Old 12-13-2010, 03:40 AM   #74
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Quote:
Originally Posted by ldolse View Post
One recommended tweak - I accidentally deleted my config file, so had to re-create it from scratch. Images weren't showing because I forgot to set the $config['web_dir'] variable. I just set it to:
Code:
$config['web_dir'] = './';
I changed the default to '.'. The trailing slash isn't needed, and probably shouldn't be there, because it will result in path names containing 2 slashes.
Quote:
One problem I'm having is related to Android/Aldiko - my brother is an Android user. Once Aldiko is installed it associates the 'application/epub+zip' mimetype to itself, and will load the epub once it's downloaded. I have the mimetype correctly configured on my web server, and if I just manually go to the URL of the epub in the browser it downloads and launches Aldiko. However if I try to load the epub using the PHP server the download fails. Android just says the title of the download is <unknown>, and that it was unsuccessful. I haven't tried pcaps or anything like that, but I can give it a shot if need be.
My guess is that the browser can't handle parsing the file name component out of a URL containing query strings. Instead, it looks for an extension at the end of the URL, and then gets confused.

I have uploaded a 'release' 0.2.3_beta that embeds the required information into the path instead of using query strings. Let me know if this fixes the problem. If so, I will release it. If not, then we need to look further. Use the link above, as the beta is not listed on my calibre page.

Note that the beta also contains the fixes for prev/next and the change to use categories instead of searches on URLs
chaley is offline   Reply With Quote
Old 12-13-2010, 01:40 PM   #75
ldolse
Wizard
ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.
 
Posts: 1,337
Karma: 123455
Join Date: Apr 2009
Location: Malaysia
Device: PRS-650, iPhone
Quote:
Originally Posted by chaley View Post
I changed the default to '.'. The trailing slash isn't needed, and probably shouldn't be there, because it will result in path names containing 2 slashes.
Huh - on my system it didn't work without the trailing slash, but maybe it's specific to my server. Not a big deal.


Quote:
Originally Posted by chaley View Post
My guess is that the browser can't handle parsing the file name component out of a URL containing query strings. Instead, it looks for an extension at the end of the URL, and then gets confused.

I have uploaded a 'release' 0.2.3_beta that embeds the required information into the path instead of using query strings. Let me know if this fixes the problem. If so, I will release it. If not, then we need to look further. Use the link above, as the beta is not listed on my calibre page.

Note that the beta also contains the fixes for prev/next and the change to use categories instead of searches on URLs
The change doesn't seem to make a difference, I'll do some further research, get some packet captures done and let you know if I can spot the key difference.

The 2.3 beta seems like it may have introduced some other problem - requests to the PHP server are frequently hanging/timing out from the Android browser. Requests to other directories on the same server are fine while getting the timeouts, and access from a desktop to the php server is fine when Android is hanging. Hopefully I can ID the cause when I do the other debugging.
ldolse is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Calibre Command Line usage with PHP artoros Related Tools 13 07-01-2010 09:57 AM
Web Content Server mezme Calibre 7 01-08-2010 09:59 PM
Calibre web server on MAC I cant get to work.. any help appreciated. stustaff Calibre 5 01-06-2010 06:05 PM
Calibre - web server problems eclpmb Calibre 9 11-28-2008 02:04 PM


All times are GMT -4. The time now is 11:22 AM.


MobileRead.com is a privately owned, operated and funded community.