Kindle and Calibre Content Server

JoeFloyd · 01-25-2010, 07:26 PM

I've discovered how to easily access the Calibre Content Server from the Kindle web browser. This will allow you to search your Calibre library, and download supported formats directly from you home computer to Kindle using the wireless connection.

In effect, your entire Calibre library of supported formats can be accessed directly from your Kindle.

There are few problems that I had to work around.

First, my ISP blocks incoming port 80 traffic. This means I can't run a web server on a standard port.

Second, the Kindle doesn't seem to allow arbitrary port numbers to be entered into the web address of the browser.

Third, the Calibre Content Server doesn't want to use restricted ports (1-1024) on all of the versions of windows that I've tried.

Each of these problems require a workaround.

Start by setting up the Calibre Content Server
1) Assign a port (default is 8080)
2) provide a user name and password

Start the Calibre Content Server and verify that the you can see web page being generated by Calibre.

Using your web browser you can direct it to your local machine running Calibre by entering 127.0.0.1:8080 into the web address. You should be prompted for the user name and password you entered into the Calibre Content Server setup dialog.

OK, now the tricky part is that the Kindle web browser will not allow arbitrary ports to be specified as part of the web address. For instance, if you try to go to a web site with and address of the form.

http://www.google.com:8080

The browser will return an error.

As far as I know, the browser is restricting the web port to the default value of 80.

http://www.google.com:80

This will work.

Now, the interesting thing.

http + ssl is web traffic over a secure socket connection. This is usually directed to port 443

For example, when you use a web address of the form

https://www.google.com

This will connect to port 443 not port 80.

You can also entered a web address of the form;

http://www.google.com:443

What's interesting about this is that the web browser will attempt to connect to the server at port 443, but not over a secure socket!!!

My ISP does not block port 443, so I can use a web address of the form

http://my.ip.address:443

This will connect to my computer running the Calibre Content Server on port 443.

Now, the problem is to somehow get Calibre to respond to requests on port 443. I couldn't figure out how to configure Calibre to allow this, so I went looking for some software to forward connections from one port to another port.

What I found was a windows application called Port Forwarding For Windows.

http://www.quantumg.net/portforward.php

This software will forward incoming connections on a given port to a host(can be the same machine) with a different port.

There isn't much to this software. When you run it, the GUI won't show much of anything. On the application menu, you click Redirection -> Add
This will bring up a dialog that has three boxes.

The first box is the external port that the Kindle will use to connect to this machine. In this case, the port is 443.

The second box is destination. This box should be the IP address of the machine running Calibre. On your home network, the router/gateway may have automatically assigned the IP address of the computer using DHCP. You can find the IP address by opening a cmd window and executing the ipconfig command.

The last box is the port you entered into the Calibre Content Server setup GUI.

When this is configured correctly, you should be able to use a web browser on you local computer to browse the Calibre using the port 443.

127.0.0.1:443

Port Forwarding For Windows GUI will show a number of connections being made while the web browser is connecting to Calibre.

The final step is to open the port 443 on you broadband router/gateway. Each manufacture has a different way of doing this, but in the majority of cases you want to go to the port forwarding setting of the router and add a rule to foward port 443 to the machine running Calibre and the Port Forwarding For Windows application.

Now from the Kindle type the public IP address of your router (this can be found using the router web interface) or setup a free DynDNS account to allow the use of fixed hostname.

http://my.ip.address:443

A dialog will appear on the Kindle asking for the user name a password you entered into the Calibre Content Server setup GUI. Once you enter the correct user name and password, the Calibre web interface will load.

It's extremely slow, but it works. You can download a book by selecting one of the supported formats (txt, mobi, prc, html). The browser will download the file and the next time you go to your home screen the new book will appear at the top of the list.

Hope this works for you.

JoeFloyd · 01-27-2010, 04:29 PM

Hi,

If anyone wants to try from their Kindle to the Calibre Content server, I'm going to open up access to the server for a couple of days.

The only content that is shared are news feeds that Calibre has organized into Kindle periodicals.

Now, the instructions:

On your Kindle bring up the built-in web browser:

Press the Home button to go to the home scree
Press the Menu button to display the main menu
Using the 5-way controller scroll down to Experimental
Using the 5-way controller select Basic Web

Once the basic web application loads
Using the 5-way controller move the cursor highlight up to the web address box
Enter the following into the web address:

testbooks.gotdns.com:443

Using the 5-way controller, move the cursor right until the go to item is highlighted. Select go to.

This is connect to the Calibre Content Server. When prompted, enter username: calibre and password: calibre

If all goes well, the Calibre web page will load on your Kindle.

To download a periodical, select the mobi link next to a periodical.

Pushka · 01-27-2010, 04:35 PM

Does this work for International kindles as we dont get web access from the kindle?

Yep, it works! SO, can I load my own library then?

JoeFloyd · 01-27-2010, 04:40 PM

My guess would be that if the wireless web access is limited in your country, then this would not work.

Running your own Calibre Content Server is one way to make your personal library available to you anywhere you have wireless access.

As far as international users with restricted web access, it could be that the Kindle global ISP is not blocking port 443. That's interesting. If that's the case, then an international user should be able to connect to secure web servers like gmail. https://mail.google.com

Dahaun · 01-29-2010, 11:40 AM

Quote:

Originally Posted by Pushka

Does this work for International kindles as we dont get web access from the kindle?

Yep, it works! SO, can I load my own library then?

For me it is not working. Any ideas what to do?

JoeFloyd · 01-29-2010, 12:55 PM

My ISP updated the external IP address of my router and I needed to propagate the change up to the DynDNS server.

I just tried it from my Kindle, and it works for me.

Give it another try.

Dahaun · 01-29-2010, 01:01 PM

I have international version...

JoeFloyd · 01-29-2010, 01:30 PM

User Pushka reported that he was able to connect using an international Kindle, but I don't know where Pushka is located.

Pushka may have been in one of the countries which do have wireless web access enabled.

Dahaun · 01-31-2010, 08:45 AM

Quote:

Originally Posted by Pushka

Does this work for International kindles as we dont get web access from the kindle?

Yep, it works! SO, can I load my own library then?

Where did you take the test?
I mean country...

ragefather · 02-04-2010, 08:04 PM

This is fantastic. Thanks so much for the in depth walk through. It works great. One problem I noticed with the Kindle is that I wouldn't download a book I had that was 14MB. It kept saying it was already downloaded, but it definitely was not. I imagine there might be a cap on the size the basic browser will DL.

Thanks again!!

JoeFloyd · 02-05-2010, 12:55 AM

Yeah, if the Kindle web browser fails to download a file, the partially downloaded file may still exists in the documents folder of the Kindle. If you try to download a file that has the same name of a file already in the documents folder the browser complains.

The Kindle e-book reader application home screen does not display files that it doesn't recognize as ebooks. I'm guessing that an incomplete file doesn't look like an ebook to the Kindle, so it doesn't display the file in the home screen.

If you connect your Kindle to your computer and look in the documents folder I think there may be an incomplete copy of the file that was causing you problems.

Also, the free web access that Amazon provides does cost Amazon real money for each megabyte transferred over Sprint or AT&T's network. AFAIK, Amazon hasn't set a limit on how much 3G goodness each Kindle is allowed to use, but it's a sure bet that if Amazon sees the wireless bill increase beyond some comfortable threshold the free web access will be taken away.

One other cool feature of Calibre is that it can be setup to e-mail each periodical (news feeds) you want to your kindle e-mail account. This will automatically allow Calibre to deliver a periodical each morning just like Amazon's own newspaper subscription. However, Amazon does charge 15 cents per megabyte for data sent through this service.

thomman · 02-11-2010, 10:36 AM

This is an interesting thread. For people who did access the site, can you please confirm that Amazon does *not* charge the 15c/MB for this download?

JoeFloyd · 02-12-2010, 12:12 PM

For those who live in an area that has full web access, downloading files using the experimental web browser does not incur a fee. So, downloading books from your own personal Calibre Content Server is free.

The terms of service for the Kindle do state that free web access may become a fee based service at some point. But, for the moment, free web access includes free file downloads. The web browser limits which file types can be downloaded. The current list is TXT, MOBI, PRI, and I think HTML. PDF is not a current format that the browser will download.

The Kindle email account associated with your Kindle me@kindle.com can be used by Calibre to automatically send a periodical to your Kindle after it has been generated by Calibre. This service is what has the 15c/MB fee. On the Manage You Kindle page at Amazon there is a section called Your Individual Charges , but it seems that Amazon may not be actually charging for this service just yet.

Based upon what has been posted by Amazon concerning the Kindle Development Kit and application data usage limits. The KDK guidelines state that free apps can use up to 100 KB per month for free. Apps that use more than 100 KB month must be used as part of a monthly subscription. My guess is that once the Kindle Apps starting rolling out, all Kindle data usage may be lumped into the Kindle Apps monthly subscription plan.

jbloudg20 · 03-09-2010, 10:52 AM

I've got this all setup from my computer. My ISP allows port 80, so I'm using that. If I type my external ip from this computer, it works flawlessly.

However, when I type it from my kindle, I get an error:
The website you requested is not responding. Please try again later.

koland · 03-10-2010, 09:46 AM

All Kindle traffic goes thru proxy servers, so Amazon can block any traffic they want to. First, though, I'd go to a friend's house or somewhere you can use another computer that is on a different ISP, so it has to go thru the ISP's firewall. Then see if you can access it from that PC. Unless you have a fixed IP plan, you'll need to figure out a way to dynamically determine your IP address (which changes on each connection). Only after you have it working via pure PC connections would I try to troubleshoot the Kindle connection itself.

01-25-2010, 07:26 PM	#1
JoeFloyd Junior Member Posts: 9 Karma: 112 Join Date: Sep 2009 Device: kindle	Howto: Access Calibre Content Server from Kindle web browser I've discovered how to easily access the Calibre Content Server from the Kindle web browser. This will allow you to search your Calibre library, and download supported formats directly from you home computer to Kindle using the wireless connection. In effect, your entire Calibre library of supported formats can be accessed directly from your Kindle. There are few problems that I had to work around. First, my ISP blocks incoming port 80 traffic. This means I can't run a web server on a standard port. Second, the Kindle doesn't seem to allow arbitrary port numbers to be entered into the web address of the browser. Third, the Calibre Content Server doesn't want to use restricted ports (1-1024) on all of the versions of windows that I've tried. Each of these problems require a workaround. Start by setting up the Calibre Content Server 1) Assign a port (default is 8080) 2) provide a user name and password Start the Calibre Content Server and verify that the you can see web page being generated by Calibre. Using your web browser you can direct it to your local machine running Calibre by entering 127.0.0.1:8080 into the web address. You should be prompted for the user name and password you entered into the Calibre Content Server setup dialog. OK, now the tricky part is that the Kindle web browser will not allow arbitrary ports to be specified as part of the web address. For instance, if you try to go to a web site with and address of the form. http://www.google.com:8080 The browser will return an error. As far as I know, the browser is restricting the web port to the default value of 80. http://www.google.com:80 This will work. Now, the interesting thing. http + ssl is web traffic over a secure socket connection. This is usually directed to port 443 For example, when you use a web address of the form https://www.google.com This will connect to port 443 not port 80. You can also entered a web address of the form; http://www.google.com:443 What's interesting about this is that the web browser will attempt to connect to the server at port 443, but not over a secure socket!!! My ISP does not block port 443, so I can use a web address of the form http://my.ip.address:443 This will connect to my computer running the Calibre Content Server on port 443. Now, the problem is to somehow get Calibre to respond to requests on port 443. I couldn't figure out how to configure Calibre to allow this, so I went looking for some software to forward connections from one port to another port. What I found was a windows application called Port Forwarding For Windows. http://www.quantumg.net/portforward.php This software will forward incoming connections on a given port to a host(can be the same machine) with a different port. There isn't much to this software. When you run it, the GUI won't show much of anything. On the application menu, you click Redirection -> Add This will bring up a dialog that has three boxes. The first box is the external port that the Kindle will use to connect to this machine. In this case, the port is 443. The second box is destination. This box should be the IP address of the machine running Calibre. On your home network, the router/gateway may have automatically assigned the IP address of the computer using DHCP. You can find the IP address by opening a cmd window and executing the ipconfig command. The last box is the port you entered into the Calibre Content Server setup GUI. When this is configured correctly, you should be able to use a web browser on you local computer to browse the Calibre using the port 443. 127.0.0.1:443 Port Forwarding For Windows GUI will show a number of connections being made while the web browser is connecting to Calibre. The final step is to open the port 443 on you broadband router/gateway. Each manufacture has a different way of doing this, but in the majority of cases you want to go to the port forwarding setting of the router and add a rule to foward port 443 to the machine running Calibre and the Port Forwarding For Windows application. Now from the Kindle type the public IP address of your router (this can be found using the router web interface) or setup a free DynDNS account to allow the use of fixed hostname. http://my.ip.address:443 A dialog will appear on the Kindle asking for the user name a password you entered into the Calibre Content Server setup GUI. Once you enter the correct user name and password, the Calibre web interface will load. It's extremely slow, but it works. You can download a book by selecting one of the supported formats (txt, mobi, prc, html). The browser will download the file and the next time you go to your home screen the new book will appear at the top of the list. Hope this works for you. Last edited by JoeFloyd; 01-25-2010 at 07:46 PM. Reason: Update title

01-27-2010, 04:35 PM	#3
Pushka Wizard Posts: 1,119 Karma: 1019140 Join Date: Oct 2009 Location: Australia Device: kindle, Ipad, Iphone, Nexus and PPW	Does this work for International kindles as we dont get web access from the kindle? Yep, it works! SO, can I load my own library then? Last edited by Pushka; 01-27-2010 at 04:41 PM.

01-27-2010, 04:40 PM	#4
JoeFloyd Junior Member Posts: 9 Karma: 112 Join Date: Sep 2009 Device: kindle	My guess would be that if the wireless web access is limited in your country, then this would not work. Running your own Calibre Content Server is one way to make your personal library available to you anywhere you have wireless access. As far as international users with restricted web access, it could be that the Kindle global ISP is not blocking port 443. That's interesting. If that's the case, then an international user should be able to connect to secure web servers like gmail. https://mail.google.com Last edited by JoeFloyd; 01-27-2010 at 05:13 PM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Calibre Content Server	statiiq	Calibre	18	11-04-2016 03:12 PM
Calibre 0.7.19 content server and kindle 3 problem	tommet	Calibre	6	09-18-2010 05:21 PM
Calibre Content Server	TonytheBookworm	Calibre	2	09-07-2010 09:47 PM
content server in calibre	bruceOz	Calibre	1	04-16-2010 04:23 AM
Calibre 0.6.42 Crashing when using content server	John1170	Calibre	1	02-24-2010 12:25 AM

01-27-2010, 04:29 PM	#2
JoeFloyd Junior Member Posts: 9 Karma: 112 Join Date: Sep 2009 Device: kindle	Hi, If anyone wants to try from their Kindle to the Calibre Content server, I'm going to open up access to the server for a couple of days. The only content that is shared are news feeds that Calibre has organized into Kindle periodicals. Now, the instructions: On your Kindle bring up the built-in web browser: Press the Home button to go to the home scree Press the Menu button to display the main menu Using the 5-way controller scroll down to Experimental Using the 5-way controller select Basic Web Once the basic web application loads Using the 5-way controller move the cursor highlight up to the web address box Enter the following into the web address: testbooks.gotdns.com:443 Using the 5-way controller, move the cursor right until the go to item is highlighted. Select go to. This is connect to the Calibre Content Server. When prompted, enter username: calibre and password: calibre If all goes well, the Calibre web page will load on your Kindle. To download a periodical, select the mobi link next to a periodical.

01-29-2010, 12:55 PM	#6
JoeFloyd Junior Member Posts: 9 Karma: 112 Join Date: Sep 2009 Device: kindle	My ISP updated the external IP address of my router and I needed to propagate the change up to the DynDNS server. I just tried it from my Kindle, and it works for me. Give it another try.

01-29-2010, 01:01 PM	#7
Dahaun Member Posts: 17 Karma: 10 Join Date: Jan 2010 Location: Central Europe Device: Kindle DX	I have international version...

01-29-2010, 01:30 PM	#8
JoeFloyd Junior Member Posts: 9 Karma: 112 Join Date: Sep 2009 Device: kindle	User Pushka reported that he was able to connect using an international Kindle, but I don't know where Pushka is located. Pushka may have been in one of the countries which do have wireless web access enabled.

02-04-2010, 08:04 PM	#10
ragefather Junior Member Posts: 1 Karma: 10 Join Date: Feb 2010 Device: Kindle 2	This is fantastic. Thanks so much for the in depth walk through. It works great. One problem I noticed with the Kindle is that I wouldn't download a book I had that was 14MB. It kept saying it was already downloaded, but it definitely was not. I imagine there might be a cap on the size the basic browser will DL. Thanks again!!

02-05-2010, 12:55 AM	#11
JoeFloyd Junior Member Posts: 9 Karma: 112 Join Date: Sep 2009 Device: kindle	Yeah, if the Kindle web browser fails to download a file, the partially downloaded file may still exists in the documents folder of the Kindle. If you try to download a file that has the same name of a file already in the documents folder the browser complains. The Kindle e-book reader application home screen does not display files that it doesn't recognize as ebooks. I'm guessing that an incomplete file doesn't look like an ebook to the Kindle, so it doesn't display the file in the home screen. If you connect your Kindle to your computer and look in the documents folder I think there may be an incomplete copy of the file that was causing you problems. Also, the free web access that Amazon provides does cost Amazon real money for each megabyte transferred over Sprint or AT&T's network. AFAIK, Amazon hasn't set a limit on how much 3G goodness each Kindle is allowed to use, but it's a sure bet that if Amazon sees the wireless bill increase beyond some comfortable threshold the free web access will be taken away. One other cool feature of Calibre is that it can be setup to e-mail each periodical (news feeds) you want to your kindle e-mail account. This will automatically allow Calibre to deliver a periodical each morning just like Amazon's own newspaper subscription. However, Amazon does charge 15 cents per megabyte for data sent through this service.

02-11-2010, 10:36 AM	#12
thomman Member Posts: 14 Karma: 14 Join Date: Aug 2009 Location: USA Device: Kindle 2, ipod touch (less), PC (least)	This is an interesting thread. For people who did access the site, can you please confirm that Amazon does not charge the 15c/MB for this download?

02-12-2010, 12:12 PM	#13
JoeFloyd Junior Member Posts: 9 Karma: 112 Join Date: Sep 2009 Device: kindle	For those who live in an area that has full web access, downloading files using the experimental web browser does not incur a fee. So, downloading books from your own personal Calibre Content Server is free. The terms of service for the Kindle do state that free web access may become a fee based service at some point. But, for the moment, free web access includes free file downloads. The web browser limits which file types can be downloaded. The current list is TXT, MOBI, PRI, and I think HTML. PDF is not a current format that the browser will download. The Kindle email account associated with your Kindle me@kindle.com can be used by Calibre to automatically send a periodical to your Kindle after it has been generated by Calibre. This service is what has the 15c/MB fee. On the Manage You Kindle page at Amazon there is a section called Your Individual Charges , but it seems that Amazon may not be actually charging for this service just yet. Based upon what has been posted by Amazon concerning the Kindle Development Kit and application data usage limits. The KDK guidelines state that free apps can use up to 100 KB per month for free. Apps that use more than 100 KB month must be used as part of a monthly subscription. My guess is that once the Kindle Apps starting rolling out, all Kindle data usage may be lumped into the Kindle Apps monthly subscription plan.

03-09-2010, 10:52 AM	#14
jbloudg20 Connoisseur Posts: 65 Karma: 30 Join Date: Dec 2008 Location: Storrs, CT Device: PRS-900 & K2i	I've got this all setup from my computer. My ISP allows port 80, so I'm using that. If I type my external ip from this computer, it works flawlessly. However, when I type it from my kindle, I get an error: The website you requested is not responding. Please try again later.

03-10-2010, 09:46 AM	#15
koland Grand Sorcerer Posts: 8,560 Karma: 8033155 Join Date: Aug 2008 Location: TN, USA Device: kindle(all), nook, nookcolor, Sony, Kobo, epic, iphone, iPad, pc	All Kindle traffic goes thru proxy servers, so Amazon can block any traffic they want to. First, though, I'd go to a friend's house or somewhere you can use another computer that is on a different ISP, so it has to go thru the ISP's firewall. Then see if you can access it from that PC. Unless you have a fixed IP plan, you'll need to figure out a way to dynamically determine your IP address (which changes on each connection). Only after you have it working via pure PC connections would I try to troubleshoot the Kindle connection itself.

Advert

Advert