VitalBook format (.vbk)

cyanic · 10-04-2014, 06:11 PM

Hi everyone! I've recently been exposed to VitalSource as a course I'm taking has its primary textbook only available from VitalSource. In any case, I've found Bookshelf to be quite annoying and slow, so I took out some time to figure out the format.

.vbk Format
The .vbk format is, at its core, a container format. It contains all the components of the book, laid out as files in the container. It's sort of like the type of large resource packs you see in games these days. The file is read from the end: there is a "magic number" that identifies the file as in .vbk format, the size of the header, and the header itself. The header is an XML string that contains basic information about the file, such as version, built date, file table info, sanity info, and some metadata. The rest of the data are file data as laid out by the filemap.

There are currently four distinct categories of book formats that can be contained by the .vbk file. They are vitalbook, epubbook, picturebook, and pdfbook.

VitalBook
The vitalbook format (aka DashML) is basically an XML document, stored with VitalSource's proprietary serialization format. The document has a custom XML schema. Images and videos are stored as file entries in the .vbk container. Patent for the serialization format can be found here. Also of interest is this WIPO application, which includes an old copy of the DTD for the XML, starting on page 35. Last known DTD version is 3.2 3.4.

EPUBBook
The epubbook format is just the contents of an EPUB file put inside a .vbk container, along with search indexing data. The files can be extracted and packed to a Zip file for a valid EPUB book.

PictureBook
The picturebook format contains images of each page of a book. Alongside those images are a manifest, text contents of each page, metadata file for position of glyphs (for text selection), linebreaks, and links, and index files. This format may be harder to convert to e-book reader formats, considering they're just images. Probably the best format to convert to is PDF.

PDFBook
The pdfbook format is basically the same as picturebook, except instead of images of pages, there is a single PDF of the book. The text content and page metadata files are still generated and included (for whatever reason). The PDF appears to be the same file as is submitted by the publisher.

DRM
The DRM used in this system is pretty straightforward. The individual streams inside the .vbk are encrypted with AES-256 CBC as necessary (i.e. everything except for cover and cover thumbnail). The keys for the files are stored in a license file (oddly, also a .vbk; its password is encrypted with RSA-2048 OAEP and the encrypted password stored in the .vbk's header). The license file is delivered from VitalSource's server, and contains basic account information (name, email, device ID) and book information (password, print and copy limits, expiration date). The patent for the DRM system can be seen here. (Interesting to note, the system in practice deviates slightly from the patent by using a single, static RSA keypair instead of a unique keypair for each user.) While it appears no .vbk is distributed without encryption, it is entirely possible to read one that is not encrypted (I tried it on Bookshelf; although it'll prevent you from opening the file if it doesn't find its ID (derived from file name...) in the license file, a rename did allow the decrypted book to work just fine).

Next steps
What I'd like is to get VitalBook support in calibre. I've made a utility to decrypt, extract, and convert .vbk files (convert meaning vitalbook to XML, not other types to other formats, though I do have an epubbook to EPUB converter). Unfortunately, it's all written in C#, and I don't know Python, so I can't write a plugin for calibre. Would anyone be interested in bringing support for it to calibre? I'm thinking most basically extracting epubbooks and PDFs for import, and getting metadata from the .vbk header. (Although the metadata doesn't contain publisher name, publication date, or category info.)

Existing converters
Although there are no "offline" converters publicly available right now, there are still ways to convert certain VitalBook variants into other formats.

PDFBook: Use the dumping DLL to dump the embedded PDF file.
EPUBBook: Use Extractor.epub to convert to EPUB.
VitalBook and PictureBook: Use a PDF printer to convert to PDF (example procedure). Note this may be slow, and the resulting pages will include a watermark header and the page quality may be lower.

gonzule · 01-15-2015, 07:59 PM

Hi Cyanic, have you made any progress in converting the .vbk files into another format? epub, pdf, whatever?

cyanic · 01-16-2015, 12:46 AM

I actually already made a program to extract EPUBBook and PDFBook, and to convert PictureBook into PDF. I intend to release them after some cleanup, but they're going to be pretty useless without the decryption components (which I can't include due to forum rules on DRM).

gonzule · 01-16-2015, 06:11 AM

mmmm. i see and you are right about the forum rules. But does it still count as something "wrong" even if i already own the book in VitalSource Bookshelf format, but i want to convert it to be able to read it in my sony reader which doesn't support the VBK format?

dgatwood · 01-31-2015, 12:41 AM

In the U.S., it is legally dubious thanks to the DMCA. It is legal to create it, and it is legal to possess it, but it may or may not be legal to distribute it (as I understand it, hinging in large part on whether you can convince people that such a book constitutes a "computer program", making it qualify for a fairly significant loophole).

What you could do, though, is document how the format works, but leave out the de-DRM portions, and let somebody else outside the U.S. re-reverse-engineer the remaining bits.

mrs1 · 02-20-2015, 03:43 AM

Thank you for your efforts in extracting ebooks from Vitalsource. It really is an annoying format.

I actually already made a program to extract EPUBBook and PDFBook, and to convert PictureBook into PDF. I intend to release them after some cleanup, but they're going to be pretty useless without the decryption components (which I can't include due to forum rules on DRM).

Would you consider posting this program to a file-sharing service?

Thanks!

cyanic · 02-21-2015, 10:58 PM

Quote:

Originally Posted by mrs1

Would you consider posting this program to a file-sharing service?

I could, but it still wouldn't work on books with DRM, which is basically all of them.

eschwartz · 02-23-2015, 01:49 AM

You could blog about it elsewhere and mention and link to the offsite blog with the download. That is how the popular DeDRM tools are handled, and it is in accordance with MR guidelines.

gonzule · 03-04-2015, 06:44 PM

I've continued looking for something similar elsewhere but so far you seem to be theonly one who has made any progress in this. It really is an annoying book format (VBK) as the APP to read it on android is really crappy and there is no way to use it on my eReader.

heidi_helen · 03-19-2015, 09:50 AM

Quote:

Originally Posted by cyanic

I've found Bookshelf to be quite annoying and slow…

Me too.

Quote:

Originally Posted by cyanic

The .vbk format is, at its core, a container format.

I suspected this.

Quote:

Originally Posted by cyanic

There are currently four distinct categories of book formats that can be contained by the .vbk file. They are vitalbook, epubbook, picturebook, and pdfbook.

The one I just purchased contains a pdfbook. I'd really like to be able to extract it so I can open it in iBooks on my iPad. I'm very interested in whatever you're developing.

pdurrant · 03-19-2015, 09:55 AM

I suspect that Apprentice Alf (or Apprentice Harper) would be happy to include the existing code in the DeDRM tools.

cyanic · 03-20-2015, 11:17 AM

Someone will have to port the code to Python first to get it included in DeDRM, and I don't know Python. I've emailed Apprentice Alf about the conversion tool before, but I never got a response.

cyanic · 03-21-2015, 09:24 PM

Quote:

Originally Posted by heidi_helen

The one I just purchased contains a pdfbook. I'd really like to be able to extract it so I can open it in iBooks on my iPad. I'm very interested in whatever you're developing.

If you're using Windows, I actually have a DLL you can use to extract the PDF here.

gonzule · 03-27-2015, 01:11 AM

Quote:

Originally Posted by cyanic

If you're using Windows, I actually have a DLL you can use to extract the PDF here.

got exited there for a moment, but then i realized my vbk book is a epubbook

.

The most interesting thing is that i havent been able to find this book i have in any other format online. No pdf, no epub, no mobi, nothing.

cyanic · 03-27-2015, 04:57 PM

Here's something interesting for everyone:

It's local copies of Firebug Lite and JS Console. It's not only limited to Bookshelf. You can try it in Calibre too.

10-04-2014, 06:11 PM	#1
cyanic File format tinkerer Posts: 19 Karma: 66468 Join Date: Oct 2014 Device: PC	VitalBook format (.vbk) Hi everyone! I've recently been exposed to VitalSource as a course I'm taking has its primary textbook only available from VitalSource. In any case, I've found Bookshelf to be quite annoying and slow, so I took out some time to figure out the format. .vbk Format The .vbk format is, at its core, a container format. It contains all the components of the book, laid out as files in the container. It's sort of like the type of large resource packs you see in games these days. The file is read from the end: there is a "magic number" that identifies the file as in .vbk format, the size of the header, and the header itself. The header is an XML string that contains basic information about the file, such as version, built date, file table info, sanity info, and some metadata. The rest of the data are file data as laid out by the filemap. There are currently four distinct categories of book formats that can be contained by the .vbk file. They are vitalbook, epubbook, picturebook, and pdfbook. VitalBook The vitalbook format (aka DashML) is basically an XML document, stored with VitalSource's proprietary serialization format. The document has a custom XML schema. Images and videos are stored as file entries in the .vbk container. Patent for the serialization format can be found here. Also of interest is this WIPO application, which includes an old copy of the DTD for the XML, starting on page 35. Last known DTD version is 3.2 3.4. EPUBBook The epubbook format is just the contents of an EPUB file put inside a .vbk container, along with search indexing data. The files can be extracted and packed to a Zip file for a valid EPUB book. PictureBook The picturebook format contains images of each page of a book. Alongside those images are a manifest, text contents of each page, metadata file for position of glyphs (for text selection), linebreaks, and links, and index files. This format may be harder to convert to e-book reader formats, considering they're just images. Probably the best format to convert to is PDF. PDFBook The pdfbook format is basically the same as picturebook, except instead of images of pages, there is a single PDF of the book. The text content and page metadata files are still generated and included (for whatever reason). The PDF appears to be the same file as is submitted by the publisher. DRM The DRM used in this system is pretty straightforward. The individual streams inside the .vbk are encrypted with AES-256 CBC as necessary (i.e. everything except for cover and cover thumbnail). The keys for the files are stored in a license file (oddly, also a .vbk; its password is encrypted with RSA-2048 OAEP and the encrypted password stored in the .vbk's header). The license file is delivered from VitalSource's server, and contains basic account information (name, email, device ID) and book information (password, print and copy limits, expiration date). The patent for the DRM system can be seen here. (Interesting to note, the system in practice deviates slightly from the patent by using a single, static RSA keypair instead of a unique keypair for each user.) While it appears no .vbk is distributed without encryption, it is entirely possible to read one that is not encrypted (I tried it on Bookshelf; although it'll prevent you from opening the file if it doesn't find its ID (derived from file name...) in the license file, a rename did allow the decrypted book to work just fine). Next steps What I'd like is to get VitalBook support in calibre. I've made a utility to decrypt, extract, and convert .vbk files (convert meaning vitalbook to XML, not other types to other formats, though I do have an epubbook to EPUB converter). Unfortunately, it's all written in C#, and I don't know Python, so I can't write a plugin for calibre. Would anyone be interested in bringing support for it to calibre? I'm thinking most basically extracting epubbooks and PDFs for import, and getting metadata from the .vbk header. (Although the metadata doesn't contain publisher name, publication date, or category info.) Existing converters Although there are no "offline" converters publicly available right now, there are still ways to convert certain VitalBook variants into other formats. PDFBook: Use the dumping DLL to dump the embedded PDF file. EPUBBook: Use Extractor.epub to convert to EPUB. VitalBook and PictureBook: Use a PDF printer to convert to PDF (example procedure). Note this may be slow, and the resulting pages will include a watermark header and the page quality may be lower. Last edited by cyanic; 03-31-2015 at 10:13 PM. Reason: Added converters section

01-31-2015, 12:41 AM	#5
dgatwood Curmudgeon Posts: 629 Karma: 1623086 Join Date: Jan 2012 Device: iPad, iPhone, Nook Simple Touch	In the U.S., it is legally dubious thanks to the DMCA. It is legal to create it, and it is legal to possess it, but it may or may not be legal to distribute it (as I understand it, hinging in large part on whether you can convince people that such a book constitutes a "computer program", making it qualify for a fairly significant loophole). What you could do, though, is document how the format works, but leave out the de-DRM portions, and let somebody else outside the U.S. re-reverse-engineer the remaining bits. Last edited by dgatwood; 01-31-2015 at 12:43 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
VBK converstion TO other format	NetVicious	Workshop	6	01-27-2012 03:22 AM
from vbk to azw?	barclay76	General Discussions	0	01-02-2012 03:18 PM
Vitalbook and jetBook Lite	DJHARKAVY	Workshop	0	12-27-2009 10:40 AM
.VBK to any other format	alkeshtech	Other formats	0	03-13-2009 10:29 AM
.vbk conversion?	baobab77	Workshop	4	01-07-2009 01:05 AM

01-15-2015, 07:59 PM	#2
gonzule before sleep, read or TV? Posts: 103 Karma: 10 Join Date: Apr 2008 Location: Australia Device: Kobo Libra 2	Hi Cyanic, have you made any progress in converting the .vbk files into another format? epub, pdf, whatever?

01-16-2015, 12:46 AM	#3
cyanic File format tinkerer Posts: 19 Karma: 66468 Join Date: Oct 2014 Device: PC	I actually already made a program to extract EPUBBook and PDFBook, and to convert PictureBook into PDF. I intend to release them after some cleanup, but they're going to be pretty useless without the decryption components (which I can't include due to forum rules on DRM).

01-16-2015, 06:11 AM	#4
gonzule before sleep, read or TV? Posts: 103 Karma: 10 Join Date: Apr 2008 Location: Australia Device: Kobo Libra 2	mmmm. i see and you are right about the forum rules. But does it still count as something "wrong" even if i already own the book in VitalSource Bookshelf format, but i want to convert it to be able to read it in my sony reader which doesn't support the VBK format?

02-20-2015, 03:43 AM	#6
mrs1 Junior Member Posts: 8 Karma: 10 Join Date: Dec 2014 Device: M96	Thank you for your efforts in extracting ebooks from Vitalsource. It really is an annoying format. I actually already made a program to extract EPUBBook and PDFBook, and to convert PictureBook into PDF. I intend to release them after some cleanup, but they're going to be pretty useless without the decryption components (which I can't include due to forum rules on DRM). Would you consider posting this program to a file-sharing service? Thanks!

02-23-2015, 01:49 AM	#8
eschwartz Ex-Helpdesk Junkie Posts: 19,422 Karma: 85397180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	You could blog about it elsewhere and mention and link to the offsite blog with the download. That is how the popular DeDRM tools are handled, and it is in accordance with MR guidelines.

03-04-2015, 06:44 PM	#9
gonzule before sleep, read or TV? Posts: 103 Karma: 10 Join Date: Apr 2008 Location: Australia Device: Kobo Libra 2	I've continued looking for something similar elsewhere but so far you seem to be theonly one who has made any progress in this. It really is an annoying book format (VBK) as the APP to read it on android is really crappy and there is no way to use it on my eReader.

03-19-2015, 09:55 AM	#11
pdurrant The Grand Mouse 高貴的老鼠 Posts: 71,504 Karma: 306214458 Join Date: Jul 2007 Location: Norfolk, England Device: Kindle Voyage	I suspect that Apprentice Alf (or Apprentice Harper) would be happy to include the existing code in the DeDRM tools.

03-20-2015, 11:17 AM	#12
cyanic File format tinkerer Posts: 19 Karma: 66468 Join Date: Oct 2014 Device: PC	Someone will have to port the code to Python first to get it included in DeDRM, and I don't know Python. I've emailed Apprentice Alf about the conversion tool before, but I never got a response.

Advert

Advert