How the Browser downloads files

davidfor · 06-04-2017, 04:36 AM

With the browser not working with the server in the calibre v3 beta, I've been doing some investigation as to why. And asking questions elsewhere to find out what is wrong. The answer turns out to be simple, but it relates to how the browser downloads files. And it probably explains why it doesn't work with some sites.

Firstly, the browser is built around WebKit. The user agent it sends is:

Code:

Mozilla/5.0 (Linux; U; Android 2.0; en-us;) AppleWebKit/538.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/538.1 (Kobo Touch 0370/4.4.9344)

Which appears to be a three or four year old version.

From what I can see, the browser is mainly a wrapper around WebKit. Whatever WebKit will handle, the browser will handle. That works well for most web pages.

For anything that WebKit doesn't handle, the download is passed to nickel and nickel decides what to do with it. So, for downloading a book:

Tap the link and WebKit starts a download from the server.
As WebKit doesn't handle the book formats, it passes the download to nickel.
Nickel terminate the download.
Nickel looks at the URL and headers and decides if it can handle the file.
If it can't handle the file, it does nothing.
If nickel can handle the file, it prompts you to download the file or cancel.
Once you say yes, it downloads the file and tells you when it finishes.

All that works with simpler sites. There is a bit of extra download where WebKit starts the download but nickel restarts it.

The problem with the calibre v3 server is the URL. When nickel takes over the download, it looks at the URL to work out the file name. This worked with the old server as the URL included the file name with the extension. The new calibre server is using an API-like URL that doesn't include the file name. The file name is supplied as a part of the HTTP header in the "Content-Disposition" field. The Kobo browser does not use this header field. This is probably the reason that some other sites don't work.

As I said, the answers came from discussion elsewhere. Including someone who will be known as "some guy". Kobo now know about the problem. Hopefully they will fix it soon. There may also be a way to handle it in the calibre server, but I haven't looked at this as yet.

davidfor · 06-05-2017, 08:06 PM

Just an update. Kovid has made a change to the mobile view to include the file name in any download links in the update server. I have tested this and it is working correctly.

06-04-2017, 04:36 AM	#1
davidfor Grand Sorcerer Posts: 24,906 Karma: 47303748 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	How the Browser downloads files With the browser not working with the server in the calibre v3 beta, I've been doing some investigation as to why. And asking questions elsewhere to find out what is wrong. The answer turns out to be simple, but it relates to how the browser downloads files. And it probably explains why it doesn't work with some sites. Firstly, the browser is built around WebKit. The user agent it sends is: Code: Mozilla/5.0 (Linux; U; Android 2.0; en-us;) AppleWebKit/538.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/538.1 (Kobo Touch 0370/4.4.9344) Which appears to be a three or four year old version. From what I can see, the browser is mainly a wrapper around WebKit. Whatever WebKit will handle, the browser will handle. That works well for most web pages. For anything that WebKit doesn't handle, the download is passed to nickel and nickel decides what to do with it. So, for downloading a book: Tap the link and WebKit starts a download from the server. As WebKit doesn't handle the book formats, it passes the download to nickel. Nickel terminate the download. Nickel looks at the URL and headers and decides if it can handle the file. If it can't handle the file, it does nothing. If nickel can handle the file, it prompts you to download the file or cancel. Once you say yes, it downloads the file and tells you when it finishes. All that works with simpler sites. There is a bit of extra download where WebKit starts the download but nickel restarts it. The problem with the calibre v3 server is the URL. When nickel takes over the download, it looks at the URL to work out the file name. This worked with the old server as the URL included the file name with the extension. The new calibre server is using an API-like URL that doesn't include the file name. The file name is supplied as a part of the HTTP header in the "Content-Disposition" field. The Kobo browser does not use this header field. This is probably the reason that some other sites don't work. As I said, the answers came from discussion elsewhere. Including someone who will be known as "some guy". Kobo now know about the problem. Hopefully they will fix it soon. There may also be a way to handle it in the calibre server, but I haven't looked at this as yet.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
902 qt web browser downloads	mimble	PocketBook	4	05-10-2013 04:48 PM
Cannot rearrange files in Book Browser	BartB	Sigil	3	12-18-2011 11:52 AM
Safari downloads MOBI & EPUB as text files	webfolk	Workshop	3	11-14-2010 03:27 AM
Sony 505 files/downloads into Ebookwise imp?	onerubberducky	Sony Reader	1	03-19-2009 11:48 AM
Kindle Browser and local files	slm	Amazon Kindle	2	06-03-2008 08:29 PM

06-05-2017, 08:06 PM	#2
davidfor Grand Sorcerer Posts: 24,906 Karma: 47303748 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	Just an update. Kovid has made a change to the mobile view to include the file name in any download links in the update server. I have tested this and it is working correctly.

Advert