[TonytheBookworm]

Alright, for the life of me I don't know what the deal is with my "test environment". I reported this as a potential bug but Kovid stated that it wasn't that I just needed to add more remove_tags and such. Well Yeah
The issue at hand is still this. If i take and run ebook-convert test1.recipe out_put_dir --test -vv > myrecipe.txt

Yes, I know it only gets too feeds. but the problem is in "most not all but most cases" it cleans the dang junk up. So when i run the test after using remove_tags then I see no junk I automatically assume that it is fine. But that is simply not the case. For instance the popular science code that I submitted for 7.20 I was led to believe it worked fine because the console test looked perfect. yet when i look run the built in recipe in 7.20 i get junk. I ran the recipe without --test and the same exact article(s) that showed perfect when using the --test switch show junk.

So what is the suggestion? To not use --test and run full feed parsing? The test just makes it faster to well "test"
here is the code if your interesting in seeing what I mean. I mean yeah i realize that it needs more remove_tags and keep_only tags but that is not the issue. The issue is why does the same article from the same feed with the same code show two totally different results when i use --test vs not using --test ?

Spoiler:

from calibre.web.feeds.news import BasicNewsRecipe
from calibre.ebooks.BeautifulSoup import BeautifulSoup, re

class AdvancedUserRecipe1282101454(BasicNewsRecipe):
    title = 'Popular Science'
    language = 'en'
    __author__ = 'TonytheBookworm'
    description = 'Popular Science'
    publisher = 'Popular Science'
    category = 'gadgets,science'
    oldest_article = 7 # change this if you want more current articles. I like to go a week in
    max_articles_per_feed = 100
    no_stylesheets = True
    remove_javascript = True
    
    masthead_url = 'http://www.raytheon.com/newsroom/rtnwcm/groups/Public/documents/masthead/rtn08_popscidec_masthead.jpg'
    
    remove_tags = [dict(name='div', attrs={'id':['toolbar','main_supplements']}),
                   dict(name='span', attrs={'class':['comments']}),
                   dict(name='div', attrs={'class':['relatedinfo related-right','node_navigation','content2']}),
                   dict(name='ul', attrs={'class':['item-list clear-block']})]                     
    feeds          = [
                      
                      ('Gadgets', 'http://www.popsci.com/full-feed/gadgets'),
                      ('Cars', 'http://www.popsci.com/full-feed/cars'),
                      ('Science', 'http://www.popsci.com/full-feed/science'),
                      ('Technology', 'http://www.popsci.com/full-feed/technology'),
                      ('DIY', 'http://www.popsci.com/full-feed/diy'),
                      
                    ]

    
 #The following will get read of the Gallery: links when found    
        
    def preprocess_html(self, soup) :
        print 'SOUP IS: ', soup
        weblinks = soup.findAll(['head','h2'])
        if weblinks is not None:
            for link in weblinks:
                if re.search('(Gallery)(:)',str(link)):
                  
                  link.parent.extract()
        return soup
  #-----------------------------------------------------------------

Kovid: I will submit an update to you soon for Popular Science as well. Seems like both of my submissions for 7.20 were no go for launch That's what i get for assuming all was fine when using the --test switch