View Single Post
Old 01-21-2013, 03:01 PM   #1
indianinva
Junior Member
indianinva began at the beginning.
 
Posts: 1
Karma: 10
Join Date: Jan 2013
Device: Kindle
Receipe request for Calibre

First of all,

Folks at Calibre, thank you so much for this awesome software. Second, I want to request recipe for Proquest. Proquest hosts tons of online sources, some of which are very expensive otherwise, e.g. Wall Street Journal.

Is it possible to obtain wsj from Proquest using Calibre. I have written some code which does that (sort of). But my code uses Selenium (partly since I do not understand the authentication mechanism all that well).

Here is my code

import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys

def getnext():
driver.find_element_by_link_text("Next page").click()
driver.find_element_by_id("mlcbAll").click()
return


url='http://search.proquest.com.proxy.xx.edu/publication/10482'
driver = webdriver.Firefox()
driver.get(url)
driver.find_element_by_id("UserIDinput").clear()
driver.find_element_by_id("UserIDinput").send_keys ("xxx")
driver.find_element_by_css_selector("input[type=\"submit\"]").click()
driver.find_element_by_id("passwordInput").clear()
driver.find_element_by_id("passwordInput").send_ke ys('xxxxxx')
driver.find_element_by_css_selector("input[type=\"submit\"]").click()
driver.find_element_by_link_text("View most recent issue").click()
driver.find_element_by_id("mlcbAll").click()
errorcode=0
while errorcode ==0:
try:
getnext()
except:
errorcode=1
driver.find_element_by_id("saveExportLink").click( )
##Introduce wait here
time.sleep(100)
el = driver.find_element_by_name("exportMode")
for option in el.find_elements_by_tag_name('option'):
if option.text == 'HTML':
option.click()
driver.find_element_by_id("submitButton").click()
indianinva is offline   Reply With Quote