Collection Builder

[pepak]

Command-line utility for generating collections for Sony Reader. The whole point is that while it is easy to upload books to the Reader without having to use any GUI tools, it is much more difficult to actually generate and maintain collections (playlists) - it requires manual editing of a cache file and updating it every time a book changes. This tool lets you maintain a list of files that constitute a collection and generate all the collections by running one command. Source code included.

http://www.pepak.net/sonyreader/colbuild.zip

[pepak]

Aargh. Apparently the Reader uses File Modification date for files stored on SD card (File Creation date for files stored in built-in memory). Will have to fix that and perhaps do some testing with SD card (I onlty tested built-in memory so far).

[mkaluza]

sounds interesting...I'll look into the code when I have some time. How does it work? I mean, how does it define collections? I saw something about a playlist file there - does it mean, you have to define collections manually? Is it possible to make it define collections based on directory structure - dir name=collection name, or does it do it already?

Btw - have you used some windows-specific stuff? I mean- if not, it should be fairly easy to create a linux version

[pepak]

The collections are defined by an external file which looks something like this:

MemoryType=int
Recursive=1
FileNameRegExp=\.(lrf|pdf|txt|rtf|epub)$
Cache=database\cache\media.xml
BooksPath=database\media\books\
BooksBasePath=database/media/books/

[Collection of Something]
something\book1.pdf
something\book2.lrf
something\book3.lrf

[My Favorite Books]
something\book1.pdf
foo\bar.txt
cool.rtf

It could be modified to work directly with directory structure via regexps, probably easily enough. At the moment it seemed more useful to go for a list-based approach, as it allows placing a book into several collections and ordering the collection correctly.

Windows-specific stuff - there is a CreateProcess call to execute Calibre's whatever-META.EXE and capture its output, but I am sure that could be easily rewritten for Linux. The biggest obstacle will be the XML library I used - it seemed like a lot of work just to make it compile under FreePascal.

[pepak]

Collection Builder modified with two major updates:

1) Now it works with books stored on a SD card.

2) It supports "unlisted collections" - basically, this lets you build collections based on file name and/or directory structure, without having to list all books by hand. Still, the old "listed collections" are preferable due to the advantages written above:

Originally Posted by pepak

it allows placing a book into several collections and ordering the collection correctly.

Download link remains the same.

[mkaluza]

Good points, but I think both methods could work together - first create collections based on directory structure and then add things defined in the playlist - this way you just manage the list of "duplicated" books instead of all books, so probably in most cases it would be just "favorites".

As for reordering, this can't be done using filesystem itself, but you can always check for a file named "order" or something (the reader doesn't display unknown extensions, so it wouldn't make any mess) if you really needed to place books in order different to alphabetical. In fact I find one disadvantage in using a list file - if you want the books sorted, you need to keep the list sorted

Similarly - you could check for some file that would tell the program not to create a separate collection based on this directory. And this would cover all the possibilities I think.

Anyway, these are just minor remarks - great thanks for the program and the "filesystem collections" - I really hated creating them in Calibre (I'm a console creature - I prefer to type, than to click )

As for linux build, I thought more about Borland Kylix - as far as I know it's free and it's been made to build Delphi projects on linux. But I could be not completely up to date.

[pepak]

Originally Posted by mkaluza

Good points, but I think both methods could work together - first create collections based on directory structure and then add things defined in the playlist - this way you just manage the list of "duplicated" books instead of all books, so probably in most cases it would be just "favorites".

That's pretty much what the todays version does. Except it does it in the opposite order - first creates the "listed playlists", then creates the filename-based playlists.

In fact I find one disadvantage in using a list file - if you want the books sorted, you need to keep the list sorted

I never considered that a disadvantage - since I spend so much time with command-line, it makes little difference to me if I use dir /b *.lrf >files or dir /b /on *.lrf >files :-)

Anyway, these are just minor remarks - great thanks for the program and the "filesystem collections" - I really hated creating them in Calibre (I'm a console creature - I prefer to type, than to click )

Give it a try and let me know what you think.

As for linux build, I thought more about Borland Kylix - as far as I know it's free and it's been made to build Delphi projects on linux. But I could be not completely up to date.

As I said, the biggest obstacle will probably be the XML library. If I can overcome that, the rest should be reasonably simple (though I won't be able to do it as my experience with Linux is limited - and only user-level at that).

[pepak]

Version 0.04 compiles with FreePascal. If anyone wants to modify it to work under Linux, I would appreciate it. It shouldn't be too difficult, I hope.

Even better, someone who knows Python could rewrite the utillity into Python. That way it could be made a part of Calibre and use its libraries - especially metadata extraction, which is, at the moment, extremely slow.

[gwynevans]

Originally Posted by pepak

Even better, someone who knows Python could rewrite the utillity into Python.

I haven't got time myself, because things are getting busy at work, but I've attached a Python script that I wrote a while back to create collections from file names (of the form 'author [- series N ] - title.lrf') in case you want to try yourself, as it might help getting started.

I'm not sure if it's 100% correct though, as when I just ran it, the Reader showed the "sd" icon by the collections, although they seemed to work OK!

Run with the file to modify as an argument, e.g. "lrfCategorize.py F:\database\cache\media.xml"

[just_me]

Originally Posted by pepak

Aargh. Apparently the Reader uses File Modification date for files stored on SD card (File Creation date for files stored in built-in memory). Will have to fix that and perhaps do some testing with SD card (I onlty tested built-in memory so far).

Hi,

I got interested in the whole collections issue (based on directories and such) and I noticed that apparently my reader uses the file modification date rather than the creation date for the books in the internal memory.

Can someone confirm this? Was this changed in the latest firmware?

[pepak]

With 18040 it works as I described.

[Student1]

Originally Posted by pepak

Command-line utility for generating collections for Sony Reader. The whole point is that while it is easy to upload books to the Reader without having to use any GUI tools, it is much more difficult to actually generate and maintain collections (playlists) - it requires manual editing of a cache file and updating it every time a book changes. This tool lets you maintain a list of files that constitute a collection and generate all the collections by running one command. Source code included.

http://www.pepak.net/sonyreader/colbuild.zip

Very interesting idea, thank you for your efforts!

I wonder is there a way to append collection data or get the list from whats on the sony reader already?

From what i can see you build up your list from a list of titles you enter manually in the ini file, is there anyway to get the information from calibre. For instance upload all the books with calibre and if the collection information fails... like it always does the programm can then rebuild the xml from the calibre metadata info from those books?

[pepak]

Originally Posted by Student1

I wonder is there a way to append collection data or get the list from whats on the sony reader already?

It would be quite easy to import an existing playlist - it's pretty much the opposite process of what ColBuild does. I just never saw the need for it - the regular "list directory contents to file" seemed fast enough to me, especially considering that I will only ever do that once.

As for appending data - the utility does not (or should not, unless I made some error) modify collections already stored in the Reader. It should only overwrite the collections that are defined in its playlist and leave the others alone.

From what i can see you build up your list from a list of titles you enter manually in the ini file,

If you are familiar with regular expressions and keep your books in an organized manner, you can also skip the ini-based playlists completely and go for regular-expression-based playlists instead - see options CollectionRegExp and CollectionRegExpTitle.

is there anyway to get the information from calibre. For instance upload all the books with calibre and if the collection information fails... like it always does the programm can then rebuild the xml from the calibre metadata info from those books?

Theoretically, yes, it is possible. It depends on whether you can export the metadata from Calibre in a suitable format. The big problem is that you would have to find a suitable field for collection information which would appear in all possible formats.

[Student1]

Thanks for the information! In any case it a very smart utility! Ive been struggling with the prs 505 loosing collection after the number of books on the device becomes too big! I think this utility is just the answer to this problem! Thanks!!

[vvaidy]

This is terrific ... but is there documentation on how the collections system works? I gather it is an XML file, but I'm wondering if there is a doc that describes the file, format, etc.

The reason I ask is that I would like to modify one of these two programs to work with some other software I use, and it isnt quite clear to me exactly how Collections are stored on the PRS 505. Thanks!