Gamasutra recipes broken

tom_a_sparks · 11-09-2015, 02:39 PM

Gamasutra recipes are broken, I got them working, but because the printable version is no longer work they just dont have the articles

gamasutra_fa.recipe:

Code:

__license__   = 'GPL v3'
__copyright__ = '2010, Darko Miletic <darko.miletic at gmail.com>'
'''
gamasutra.com
'''

import re
from calibre.web.feeds.news import BasicNewsRecipe

class Gamasutra(BasicNewsRecipe):
    title                 = 'Gamasutra Featured articles'
    __author__            = 'Darko Miletic'
    description           = 'The Art and Business of Making Games'
    publisher             = 'Gamasutra'
    category              = 'news, games, IT'
    oldest_article        = 2
    max_articles_per_feed = 200
    no_stylesheets        = True
    encoding              = 'cp1252'
    use_embedded_content  = False
    language              = 'en'
    remove_empty_feeds    = True
    masthead_url          = 'http://www.gamasutra.com/images/gamasutra_logo.gif'
    extra_css             = ' body{font-family: Verdana,Arial,Helvetica,sans-serif } img{margin-bottom: 0.4em} .title{font-size: x-large; font-weight: bold} '

    conversion_options = {
                          'comment'          : description
                        , 'tags'             : category
                        , 'publisher'        : publisher
                        , 'language'         : language
                        , 'linearize_tables' : True
                        }

    feeds = [(u'Feature Articles', u'http://feeds.feedburner.com/GamasutraFeatureArticles')]



    def preprocess_html(self, soup):
        for item in soup.findAll(style=True):
            del item['style']
        return self.adeify_images(soup)

gamasutra_news.recipe

Code:

__license__   = 'GPL v3'
__copyright__ = '2010, Darko Miletic <darko.miletic at gmail.com>'
'''
gamasutra.com
'''

from calibre.web.feeds.news import BasicNewsRecipe

class Gamasutra(BasicNewsRecipe):
    title                 = 'Gamasutra News'
    __author__            = 'Darko Miletic'
    description           = 'The Art and Business of Making Games'
    publisher             = 'Gamasutra'
    category              = 'news, games, IT'
    oldest_article        = 2
    max_articles_per_feed = 200
    no_stylesheets        = True
    encoding              = 'cp1252'
    use_embedded_content  = False
    language              = 'en'
    remove_empty_feeds    = True
    masthead_url          = 'http://www.gamasutra.com/images/gamasutra_logo.gif'
    extra_css             = ' body{font-family: Verdana,Arial,Helvetica,sans-serif } img{margin-bottom: 0.4em} .newsTitle{font-size: xx-large; font-weight: bold} '
    conversion_options = {
                          'comment'          : description
                        , 'tags'             : category
                        , 'publisher'        : publisher
                        , 'language'         : language
                        , 'linearize_tables' : True
                        }


    feeds = [(u'News', u'http://feeds.feedburner.com/GamasutraNews')]



    def preprocess_html(self, soup):
        for item in soup.findAll(style=True):
            del item['style']
        return self.adeify_images(soup)

tom_a_sparks · 11-10-2015, 06:13 PM

UPDATE: the print option dose work but for multipage articles only

tom_a_sparks · 11-10-2015, 07:37 PM

just noticed a hiccup, the multipage article have a ?page=1 at the end of their urls
due to my lack of python programming the string replace command is not working

kovidgoyal · 11-10-2015, 10:25 PM

What replace command, I dont see a replace command in your posted recipes?

tom_a_sparks · 11-11-2015, 01:08 AM

sorry add this

Code:

    def print_version(self, url):
        if (url.find("?page=1") != -1):
            temp_url = url.replace("?page=1", "")

        # Always add "?print=true" to the end of the url.
        print_url = temp_url + "?print=1"

        return print_url

kovidgoyal · 11-11-2015, 01:21 AM

You want

Code:

def print_version(self, url):
   return url.partition('?')[0] + '?print=true'

tom_a_sparks · 11-11-2015, 10:00 AM

no luck this is the url is not get converted

Quote:

http://www.gamasutra.com/view/featur...ign.php?page=1

the ?print=1 is getting add at the end

Quote:

http://www.gamasutra.com/view/featur...page=1?print=1

kovidgoyal · 11-11-2015, 11:38 AM

That's not possible unless the ? is escaped using url encoding in the original url. Stick a print statement inside print_version to see what the original url is.

tom_a_sparks · 11-11-2015, 12:25 PM

I was running an older version and save the edits to the newer file and wounding why nothing was changing, D'oh!

Code:

__license__   = 'GPL v3'
__copyright__ = '2010, Darko Miletic <darko.miletic at gmail.com>'
'''
gamasutra.com
'''
import re
from calibre.web.feeds.news import BasicNewsRecipe

class Gamasutra(BasicNewsRecipe):
    title                 = 'Gamasutra Featured articles'
    __author__            = 'Darko Miletic'
    description           = 'The Art and Business of Making Games'
    publisher             = 'Gamasutra'
    category              = 'news, games, IT'
    oldest_article        = 2
    max_articles_per_feed = 200
    no_stylesheets        = True
    encoding              = 'cp1252'
    use_embedded_content  = False
    language              = 'en'
    remove_empty_feeds    = True
    masthead_url          = 'http://www.gamasutra.com/images/gamasutra_logo.gif'

    conversion_options = {
                          'comment'          : description
                        , 'tags'             : category
                        , 'publisher'        : publisher
                        , 'language'         : language
                        , 'linearize_tables' : True
                        }

    remove_tags_before = dict(name="div",attrs={'class':'page_item'})
    remove_tags       = [
                          dict(name='meta')
                         ,dict(name='link')
                         ,dict(name='hr')
                         ,dict(name='div', attrs={'class':'hide-phone'})
                         ,dict(name='div', attrs={'class':'nav_links'})
                         ,dict(name='div', attrs={'class':'superfooter'})
,dict(name='span', attrs={'class':'comment_text'})
,dict(name='a', attrs={'type':'button'})
                         ]
    remove_attributes = ['width','height','name']


    feeds = [(u'Feature Articles', u'http://feeds.feedburner.com/GamasutraFeatureArticles')]



    def print_version(self, url):
        if (url.find("?page=1") != -1):
            temp_url = url.replace("?page=1", "")
        else:
            temp_url = url
        # Always add "?print=true" to the end of the url.
        print_url = temp_url + "?print=1"

        return print_url

11-10-2015, 06:13 PM	#2
tom_a_sparks Member Posts: 21 Karma: 1010 Join Date: Dec 2011 Device: sony reader PRS-T1, kobo touch	UPDATE: the print option dose work but for multipage articles only Last edited by tom_a_sparks; 11-10-2015 at 07:38 PM. Reason: removed code, and fixed spelling mistake

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Financial Times (UK) and Financial Times (International) recipes are broken	mattyb	Recipes	1	09-05-2014 10:49 AM
Free (iTunes) Tasting Chef's Recipes: Summer 2012 Cookbook [Enhanced Recipes]	ATDrake	Deals and Resources (No Self-Promotion or Affiliate Links)	3	08-15-2012 11:15 PM
Asian Recipes - 50 Tasty & Easy Unique Exotic Recipes (With Images Of Each Dish And C	asiafoodguru	Self-Promotions by Authors and Publishers	1	08-10-2012 05:01 AM
Times Of India, DNA recipes broken?	mihirp	Recipes	1	09-23-2011 03:09 PM
Broken Ipod works Fine! except that its broken	Andybaby	Lounge	1	06-04-2009 02:03 AM

11-10-2015, 07:37 PM	#3
tom_a_sparks Member Posts: 21 Karma: 1010 Join Date: Dec 2011 Device: sony reader PRS-T1, kobo touch	just noticed a hiccup, the multipage article have a ?page=1 at the end of their urls due to my lack of python programming the string replace command is not working

11-10-2015, 10:25 PM	#4
kovidgoyal creator of calibre Posts: 43,892 Karma: 22666668 Join Date: Oct 2006 Location: Mumbai, India Device: Various	What replace command, I dont see a replace command in your posted recipes?

11-11-2015, 01:08 AM	#5
tom_a_sparks Member Posts: 21 Karma: 1010 Join Date: Dec 2011 Device: sony reader PRS-T1, kobo touch	sorry add this Code: def print_version(self, url): if (url.find("?page=1") != -1): temp_url = url.replace("?page=1", "") # Always add "?print=true" to the end of the url. print_url = temp_url + "?print=1" return print_url

11-11-2015, 01:21 AM	#6
kovidgoyal creator of calibre Posts: 43,892 Karma: 22666668 Join Date: Oct 2006 Location: Mumbai, India Device: Various	You want Code: def print_version(self, url): return url.partition('?')[0] + '?print=true'

11-11-2015, 11:38 AM	#8
kovidgoyal creator of calibre Posts: 43,892 Karma: 22666668 Join Date: Oct 2006 Location: Mumbai, India Device: Various	That's not possible unless the ? is escaped using url encoding in the original url. Stick a print statement inside print_version to see what the original url is.