View Single Post
Old 09-10-2013, 12:03 PM   #1854
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,322
Karma: 3966249
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
Quote:
Originally Posted by FaceDeer View Post
...
So I went into adapter_fimfictionnet.py and figured out how to fix my own problem so it wouldn't be a hassle. I felt kind of embarrassed about how my previous bug report turned out to be a bug with Calibre itself rather than FFDL.
...
Looks like I got to it first--attached is a version that incorporates my version of your changes.

I've included a diff so you can see how I did it. I did use dateutil.parser like you did (stupid 2nd/3rd/4th issue), but I also used the same loop for publishedDate and updatedDate.

Spoiler:
Code:
@@ -21,7 +21,8 @@
 import re
 import urllib2
 import cookielib as cl
-from datetime import datetime
+#from datetime import datetime
+import dateutil.parser as dparser
 import json
 
 from .. import BeautifulSoup as bs
@@ -185,6 +186,7 @@
         hrstr="<hr />"
         descdivstr = '<div class="description">'+descdivstr[descdivstr.index(hrstr)+len(hrstr):]
         self.setDescription(self.url,descdivstr)
-        
+
+        # Can't trust dates from API anymore I'm told.
         # Dates are in Unix time
         # Take the publish date from the first chapter posted
@@ -189,10 +191,25 @@
         # Dates are in Unix time
         # Take the publish date from the first chapter posted
-        rawDatePublished = storyMetadata["chapters"][0]["date_modified"]
-        self.story.setMetadata("datePublished", datetime.fromtimestamp(rawDatePublished))
-        rawDateUpdated = storyMetadata["date_modified"]
-        self.story.setMetadata("dateUpdated", datetime.fromtimestamp(rawDateUpdated))
-
+        # rawDatePublished = storyMetadata["chapters"][0]["date_modified"]
+        # self.story.setMetadata("datePublished", datetime.fromtimestamp(rawDatePublished))
+        # rawDateUpdated = storyMetadata["date_modified"]
+        # self.story.setMetadata("dateUpdated", datetime.fromtimestamp(rawDateUpdated))
+        
+        oldestChapter = None
+        newestChapter = None
+        # Scan all chapters to find the oldest and newest, on
+        # FiMFiction it's possible for authors to insert new chapters
+        # out-of-order or change the dates of earlier ones by editing
+        # them--That WILL break epub update.
+        for chapterDate in soup.findAll('span', {'class':'date'}):
+            chapterDate = dparser.parse(chapterDate.contents[1].strip())
+            if oldestChapter == None or chapterDate < oldestChapter:
+                oldestChapter = chapterDate
+            if newestChapter == None or chapterDate > newestChapter:
+                newestChapter = chapterDate
+        self.story.setMetadata("datePublished", oldestChapter)
+        self.story.setMetadata("dateUpdated", newestChapter)
+        
         chars = soup.find("div", {"class":"inner_data"})
         # fimfic stopped putting the char name on or around the char
         # icon now for some reason.  Pull it from the image name with
@@ -215,7 +232,11 @@
                 if not isinstance(value,basestring):
                     value = unicode(value)
                 self.story.setMetadata(metakey, value)
-        
+
+        rawGroupList = soup.find('ul', {'id':'story_group_list'})
+        if rawGroupList is not None:
+            for groupName in rawGroupList.findAll('a', {'href':re.compile('^/group/')}):
+                self.story.addToList("groups",stripHTML(groupName))
             
     def getChapterText(self, url):
         logger.debug('Getting chapter text from: %s' % url)


UPDATE Sep 14 2013 - Remove obsolete beta versions

Last edited by JimmXinu; 09-14-2013 at 06:10 PM. Reason: Remove obsolete beta versions
JimmXinu is offline