With my very limited coding knowledge I fooled around in the code and added some debug statements to the read_available_plugins() function:
Code:
def read_available_plugins():
display_plugins = []
br = browser()
br.set_debug_http(True)
br.set_debug_redirects(True)
br.set_debug_responses(True)
try:
raw = br.open_novisit(MR_INDEX_URL).read()
prints('============ RAW ==========')
prints(raw)
if not raw:
return
except:
traceback.print_exc()
return
raw = raw.decode('utf-8', errors='replace')
prints('============ RAW UTF8 ==========')
prints(raw)
root = html.fromstring(raw)
prints('============= ROOT ============')
prints(root)
list_nodes = root.xpath('//div[@id="post_message_1362767"]/ul/li')
This is the output of the debug statements:
Debug browser object:
Quote:
Restarting with: C:\Program Files (x86)\Calibre2\calibre-debug.exe ['calibre-deb
ug.exe', u'-g']
Starting up...
Started up in 1.37699985504
send: u'GET /forums/showpost.php?p=1362767&postcount=1 HTTP/1.1\r\nAccept-Encodi
ng: identity\r\nHost: www.mobileread.com\r\nConnection: close\r\nUser-Agent: Moz
illa/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.13) Gecko/20101210 Gentoo Firefo
x/3.6.13\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Connection: close
header: Set-Cookie: bblastvisit=1306351703; expires=Thu, 24-May-2012 19:28:23 GM
T; path=/; domain=.mobileread.com
header: Set-Cookie: bblastactivity=0; expires=Thu, 24-May-2012 19:28:23 GMT; pat
h=/; domain=.mobileread.com
header: Expires: 0
header: Cache-Control: private, post-check=0, pre-check=0, max-age=0
header: Pragma: no-cache
header: Content-Type: text/html; charset=ISO-8859-1
header: X-UA-Compatible: IE=7
header: Content-Encoding: gzip
header: Content-Length: 7691
header: Date: Wed, 25 May 2011 19:28:23 GMT
header: Server: Apache
send: u'GET /forums/showpost.php?p=1362767&postcount=1 HTTP/1.1\r\nAccept-Encodi
ng: identity\r\nHost: www.mobileread.com\r\nConnection: close\r\nUser-Agent: Moz
illa/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.13) Gecko/20101210 Gentoo Firefo
x/3.6.13\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Connection: close
header: Set-Cookie: bblastvisit=1306351704; expires=Thu, 24-May-2012 19:28:24 GM
T; path=/; domain=.mobileread.com
header: Set-Cookie: bblastactivity=0; expires=Thu, 24-May-2012 19:28:24 GMT; pat
h=/; domain=.mobileread.com
header: Expires: 0
header: Cache-Control: private, post-check=0, pre-check=0, max-age=0
header: Pragma: no-cache
header: Content-Type: text/html; charset=ISO-8859-1
header: X-UA-Compatible: IE=7
header: Content-Encoding: gzip
header: Content-Length: 7691
header: Date: Wed, 25 May 2011 19:28:24 GMT
header: Server: Apache
send: u'GET /forums/showpost.php?p=1362767&postcount=1 HTTP/1.1\r\nAccept-Encodi
ng: identity\r\nHost: www.mobileread.com\r\nConnection: close\r\nUser-Agent: Moz
illa/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.13) Gecko/20101210 Gentoo Firefo
x/3.6.13\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Connection: close
header: Set-Cookie: bblastvisit=1306351705; expires=Thu, 24-May-2012 19:28:25 GM
T; path=/; domain=.mobileread.com
header: Set-Cookie: bblastactivity=0; expires=Thu, 24-May-2012 19:28:25 GMT; pat
h=/; domain=.mobileread.com
header: Expires: 0
header: Cache-Control: private, post-check=0, pre-check=0, max-age=0
header: Pragma: no-cache
header: Content-Type: text/html; charset=ISO-8859-1
header: X-UA-Compatible: IE=7
header: Content-Encoding: gzip
header: Content-Length: 7691
header: Date: Wed, 25 May 2011 19:28:25 GMT
header: Server: Apache
|
raw output (only the first few lines):
Quote:
============ RAW ==========
]{SH;[ߡGSw b۞%@&L&EVV% gkݲd025UN =qޯ>
#1LG8@X8pËCۋQ"vO(t 9zo k ǹovsƪ
JZxRjo{<ւqjuKx~ܲ4D%C5t=h$S Ok_(LeV.&cie6u
h]q"qCek]YXf ɖu%'7Q%1N s̼&d7_g@8
*d8; Ӭ6gsp z\.gv[1yv'e&& ^a
nx( U3wL;9?~>ƺ_}yCw%5E]tpx~PЬɤc"ŅeeqO
\TA#:$PTT*D ])HPS^/I?8Y ya^4{9dꇗx*lT
jmӦ^4] `Ay%b~m̫V)Mu՜ٳp״oLq/Z"{̷'0ܱ
|
raw utf8 (only the first few lines)
Quote:
============ RAW UTF8 ==========
��]{S�H��;[�ߡGSw �b��۞%@&���L&��E�V
V�%� �gk�����ݲd025U��N =�qޯ>��������#
1LG�8�����@X���8p�ËC��ۋ �Q���"v��O�(t
9zo k��� ǹ���ov��s�ƪ��� JZx��R�j��o
{<���ւqj����uKx~ܲ�4�D��� %C5�t=h$S�O��k�_
��(Le�V.&ci����e��6uh�]��q"��q�Cek��]�Y
�Xf�� ɖu%'7Q�%�1N�� �s̼&d7� ���_�g��@��8
|
root output
Quote:
============= ROOT ============
<Element p at 0x5b63450>
|
Provided that the "prints" statement is not totally wrong for outputting this kind of content it seems that the http connection is all fine and then things starts to get messy.
Hope you can make more out of it. If you provide the instructions I will gladly add some more debug output to trace this.
Best regards,
SoulStyle