Actually calibre uses poppler to read metadata, which is yet another library
The reason for that is poppler allows rendering of the PDF, so calibre can render the first page as the cover.
Are you sure your PDF metadata is in the PDFInfo structure and not in the extended XML metadata?