MobileRead Forums - How to handle badly formed xml from web page?

The Goodreads API I use for my Calibre Goodreads sync plugin uses (mainly) xml responses to return the results. However I have found a situation where the xml being returned is "badly formed". It can't be displayed in a web browser, due to the error, nor can it be parsed using ElementTree.

I have traced the problem down to a particular field in the xml which seems to have corrupted content - it is missing the opening <![CDATA[ within the xml text (though it has the closing ]]>).

Spoiler:

Code:

<?xml version="1.0" encoding="UTF-8"?>

<GoodreadsResponse>

        <Request>

                <authentication>true</authentication>

                                <key><![CDATA[UxvtOM3ogQWjfgiCnMleA]]></key>

                    <method><![CDATA[review_list]]></method>

        </Request>

        <reviews start="1" end="1" total="1">

    <review>

  <id>149091755</id>

      <book>

  <id type="integer">43912</id>

  <isbn>0006483569</isbn>

  <isbn13>9780006483564</isbn13>

  <publication_day type="integer">3</publication_day>

  <publication_month type="integer">12</publication_month>

  <publication_year type="integer">2001</publication_year>

  <publisher>Voyager</publisher>

  <text_reviews_count type="integer">2</text_reviews_count>

  <title>

    <![CDATA[Krondor: Tear of the Gods (The Riftwar Legacy, #3)]]>

  </title>

  <image_url>http://photo.goodreads.com/books/1249516742m/43912.jpg</image_url>

  <small_image_url>http://photo.goodreads.com/books/1249516742s/43912.jpg</small_image_url>

  <link>http://www.goodreads.com/book/show/43912.Krondor</link>

  <num_pages>384</num_pages>

  <average_rating>3.64</average_rating>

  <ratings_count>1379</ratings_count>

  <description>

<br/>A DROP IN THE OCEAN?A raid upon the high seas signals an attack of unprecedented magnitude by the 

forces of darkness. For the holiest of holies, the Tear of the Gods has been lost to the Temple of Ishap. 

After a raid planned by Bear, one of the most brutal pirates to sail the Bitter Sea, goes dramatically wrong, 

the colossal gems sink below the waves.So begins a story of the Tear of the Gods, the most powerful artifact 

known to the Temples of Midkemia. For it allows the temples to speak with their gods. Without it, they are 

lost for a decade, until another gem is formed in the distant mountains.Squire James, William, and Jazhara, 

new court magician, must seek out the location of this gem, with Brother Solon, a warrior priest of Ishap, and 

Kendaric, the sole member of the Wreckersâ€™ Guild with the power to raise the ship. They are opposed by 

the minions of Sidi, servant of the Dark God, who seeks to possess the Tear for his own ends, or to destroy 

it, denying it to the forces of light.This third tale in The Riftwar Legacy is a breathless race for a priceless treasure. 

Itâ€™s a race against time, against the myriad sinister and competing evil forces desperate for the all-powerful 

prize, and ultimately against the fundamentals of nature, which in Midkemia can be as formidable as the 

Gods themselves]]>

  </description>

<authors>

    <author>

    <id>8588</id>

        <name><![CDATA[Raymond E. Feist]]></name>

    <image_url><![CDATA[http://photo.goodreads.com/authors/1190654917p5/8588.jpg]]></image_url>

    <small_image_url><![CDATA[http://photo.goodreads.com/authors/1190654917p2/8588.jpg]]></small_image_url>

    <link><![CDATA[http://www.goodreads.com/author/show/8588.Raymond_E_Feist]]></link>

    <average_rating>3.91</average_rating>

    <ratings_count>64963</ratings_count>

    <text_reviews_count>1751</text_reviews_count>

  </author>

  </authors>  <published>2000</published>

</book>

      <rating>0</rating>

  <votes>0</votes>

    <spoiler_flag>false</spoiler_flag>

  <shelves>

            <shelf name="currently-reading" />

      </shelves>

  <recommended_for><![CDATA[]]></recommended_for>

  <recommended_by><![CDATA[]]></recommended_by>

  <started_at>Fri Feb 18 17:38:12 -0800 2011</started_at>

  <read_at></read_at>

  <date_added>Fri Feb 18 17:38:12 -0800 2011</date_added>

  <date_updated>Fri Feb 18 17:38:12 -0800 2011</date_updated>

  <read_count></read_count>

    <body><![CDATA[]]></body>

    <comments_count>0</comments_count>

  <url><![CDATA[http://www.goodreads.com/review/show/149091755]]></url>

  <link><![CDATA[http://www.goodreads.com/review/show/149091755]]></link>

</review>

  </reviews>

</GoodreadsResponse>

I've raised this just now as a bug on the Goodreads API forums, but given they don't seem to be very actively responding to issues I want to try to handle this case myself. That particular description field doesn't happen to be one I need the values of.

Currently I am using ElementTree to load the http content and retrieve elements, but of course it blows up trying to use et.fromstring() when badly formed, as below:

Code:

            root = et.fromstring(content)

            reviews_node = root.find('reviews')

            if reviews_node is not None:

                total = int(reviews_node.attrib.get('total'))

                end = int(reviews_node.attrib.get('end'))

                book_nodes = reviews_node.findall('review/book')

                for book_node in book_nodes:

                    book = {}

                    goodreads_id = book_node.findtext('id')

                    book['goodreads_id'] = goodreads_id

                    isbn = book_node.findtext('isbn13')

                    book['goodreads_isbn'] = isbn

                    (title, series) = self.convert_goodreads_title_with_series(book_node.findtext('title').strip())

                    book['goodreads_title'] = title

                    book['goodreads_series'] = series

                    # Grab the first author only for now

                    book['goodreads_author'] = book_node.findtext('authors/author/name')

Any suggestions as to what I could do if anything?