problems with login page

MasterPlexus · 11-20-2024, 04:39 PM

I'm facing some issues with the loginpage of my recipe. I assume the Webpage identify mechanize. To check that I would like to see the content of the loginpage wich is loaded. I try to save it, but I did not get it as I don't know how to handle.
my try:

Code:

br.open(loginURL)            
file = open(r"c:/temp/loginpage.txt","w")
file.write(br._html) # or br.content I don't know...
file.close()

Thanks for any help.
BR MP

kovidgoyal · 11-20-2024, 09:10 PM

res = br.open()
html = res.read()

MasterPlexus · 11-21-2024, 04:28 PM

Thanks. I could see now, that Heise has included now a javascript layer between which creates a token or session. Without it it is not possible to reach out to the login.
May I have to play with 'webengine'. are there somewhere examples for logins which uses webengine?

kovidgoyal · 11-21-2024, 09:30 PM

webengine does not support javascript based logins either. The only way to implement JS based logins is to basically reverse engineer the JS and send the corresponding HTTP requests to the server. someday I might add support for js based logins and more generally JS based scraping but not anytime soon.

MasterPlexus · 11-23-2024, 10:58 AM

Hi.
In the docu it is written that webengine uses a chromium engine. Just to confirm, this does not use or use Javascript ? I assume it should use javascript, but of course I had to rebuild the login by myself correct?
BR
MP

kovidgoyal · 11-23-2024, 10:44 PM

It does not use javascript, it uses the chromium *network stack* to defeat bot detection that relies on things like HTTPS handshake signatures.

11-20-2024, 04:39 PM	#1
MasterPlexus Enthusiast Posts: 27 Karma: 10 Join Date: Nov 2024 Device: Tolino 4HD	problems with login page I'm facing some issues with the loginpage of my recipe. I assume the Webpage identify mechanize. To check that I would like to see the content of the loginpage wich is loaded. I try to save it, but I did not get it as I don't know how to handle. my try: Code: br.open(loginURL) file = open(r"c:/temp/loginpage.txt","w") file.write(br._html) # or br.content I don't know... file.close() Thanks for any help. BR MP

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Dedicated login page?	mrecom	Server	2	07-29-2022 03:33 PM
Updated Registration and Login page	WirlyWirly	Feedback	0	11-25-2020 09:29 PM
Kobo Arc login problems	Piers	Kobo Tablets	0	06-01-2017 05:08 AM
full page image problems with iBooks/cover problems in iTunes	iain robinson	ePub	1	06-28-2013 11:10 AM

11-20-2024, 09:10 PM	#2
kovidgoyal creator of calibre Posts: 45,342 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	res = br.open() html = res.read()

11-21-2024, 04:28 PM	#3
MasterPlexus Enthusiast Posts: 27 Karma: 10 Join Date: Nov 2024 Device: Tolino 4HD	Thanks. I could see now, that Heise has included now a javascript layer between which creates a token or session. Without it it is not possible to reach out to the login. May I have to play with 'webengine'. are there somewhere examples for logins which uses webengine?

11-21-2024, 09:30 PM	#4
kovidgoyal creator of calibre Posts: 45,342 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	webengine does not support javascript based logins either. The only way to implement JS based logins is to basically reverse engineer the JS and send the corresponding HTTP requests to the server. someday I might add support for js based logins and more generally JS based scraping but not anytime soon.

11-23-2024, 10:58 AM	#5
MasterPlexus Enthusiast Posts: 27 Karma: 10 Join Date: Nov 2024 Device: Tolino 4HD	Hi. In the docu it is written that webengine uses a chromium engine. Just to confirm, this does not use or use Javascript ? I assume it should use javascript, but of course I had to rebuild the login by myself correct? BR MP

11-23-2024, 10:44 PM	#6
kovidgoyal creator of calibre Posts: 45,342 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	It does not use javascript, it uses the chromium network stack to defeat bot detection that relies on things like HTTPS handshake signatures.

Advert

Advert