Register Guidelines E-Books Search Today's Posts Mark Forums Read

 09-15-2008, 05:26 AM #1 pepak Guru   Posts: 601 Karma: 4150 Join Date: Mar 2008 Device: Sony Reader PRS-505 Collection Builder Command-line utility for generating collections for Sony Reader. The whole point is that while it is easy to upload books to the Reader without having to use any GUI tools, it is much more difficult to actually generate and maintain collections (playlists) - it requires manual editing of a cache file and updating it every time a book changes. This tool lets you maintain a list of files that constitute a collection and generate all the collections by running one command. Source code included. http://www.pepak.net/sonyreader/colbuild.zip
 09-15-2008, 11:41 AM #2 pepak Guru   Posts: 601 Karma: 4150 Join Date: Mar 2008 Device: Sony Reader PRS-505 Aargh. Apparently the Reader uses File Modification date for files stored on SD card (File Creation date for files stored in built-in memory). Will have to fix that and perhaps do some testing with SD card (I onlty tested built-in memory so far).
 09-16-2008, 02:40 AM #3 mkaluza Member   Posts: 20 Karma: 27 Join Date: Jul 2008 Device: PRS-505 sounds interesting...I'll look into the code when I have some time. How does it work? I mean, how does it define collections? I saw something about a playlist file there - does it mean, you have to define collections manually? Is it possible to make it define collections based on directory structure - dir name=collection name, or does it do it already? Btw - have you used some windows-specific stuff? I mean- if not, it should be fairly easy to create a linux version
 09-16-2008, 03:50 AM #4 pepak Guru   Posts: 601 Karma: 4150 Join Date: Mar 2008 Device: Sony Reader PRS-505 The collections are defined by an external file which looks something like this: Code: MemoryType=int Recursive=1 FileNameRegExp=\.(lrf|pdf|txt|rtf|epub)\$ Cache=database\cache\media.xml BooksPath=database\media\books\ BooksBasePath=database/media/books/ [Collection of Something] something\book1.pdf something\book2.lrf something\book3.lrf [My Favorite Books] something\book1.pdf foo\bar.txt cool.rtf It could be modified to work directly with directory structure via regexps, probably easily enough. At the moment it seemed more useful to go for a list-based approach, as it allows placing a book into several collections and ordering the collection correctly. Windows-specific stuff - there is a CreateProcess call to execute Calibre's whatever-META.EXE and capture its output, but I am sure that could be easily rewritten for Linux. The biggest obstacle will be the XML library I used - it seemed like a lot of work just to make it compile under FreePascal. Last edited by pepak; 09-16-2008 at 03:52 AM.
09-16-2008, 06:00 AM   #5
pepak
Guru

Posts: 601
Karma: 4150
Join Date: Mar 2008
Collection Builder modified with two major updates:

1) Now it works with books stored on a SD card.

2) It supports "unlisted collections" - basically, this lets you build collections based on file name and/or directory structure, without having to list all books by hand. Still, the old "listed collections" are preferable due to the advantages written above:
Quote:
 Originally Posted by pepak it allows placing a book into several collections and ordering the collection correctly.

 09-16-2008, 02:48 PM #6 mkaluza Member   Posts: 20 Karma: 27 Join Date: Jul 2008 Device: PRS-505 Good points, but I think both methods could work together - first create collections based on directory structure and then add things defined in the playlist - this way you just manage the list of "duplicated" books instead of all books, so probably in most cases it would be just "favorites". As for reordering, this can't be done using filesystem itself, but you can always check for a file named "order" or something (the reader doesn't display unknown extensions, so it wouldn't make any mess) if you really needed to place books in order different to alphabetical. In fact I find one disadvantage in using a list file - if you want the books sorted, you need to keep the list sorted Similarly - you could check for some file that would tell the program not to create a separate collection based on this directory. And this would cover all the possibilities I think. Anyway, these are just minor remarks - great thanks for the program and the "filesystem collections" - I really hated creating them in Calibre (I'm a console creature - I prefer to type, than to click ) As for linux build, I thought more about Borland Kylix - as far as I know it's free and it's been made to build Delphi projects on linux. But I could be not completely up to date.
09-16-2008, 03:06 PM   #7
pepak
Guru

Posts: 601
Karma: 4150
Join Date: Mar 2008
Quote:
 Originally Posted by mkaluza Good points, but I think both methods could work together - first create collections based on directory structure and then add things defined in the playlist - this way you just manage the list of "duplicated" books instead of all books, so probably in most cases it would be just "favorites".
That's pretty much what the todays version does. Except it does it in the opposite order - first creates the "listed playlists", then creates the filename-based playlists.

Quote:
 In fact I find one disadvantage in using a list file - if you want the books sorted, you need to keep the list sorted
I never considered that a disadvantage - since I spend so much time with command-line, it makes little difference to me if I use dir /b *.lrf >files or dir /b /on *.lrf >files :-)

Quote:
 Anyway, these are just minor remarks - great thanks for the program and the "filesystem collections" - I really hated creating them in Calibre (I'm a console creature - I prefer to type, than to click )
Give it a try and let me know what you think.

Quote:
 As for linux build, I thought more about Borland Kylix - as far as I know it's free and it's been made to build Delphi projects on linux. But I could be not completely up to date.
As I said, the biggest obstacle will probably be the XML library. If I can overcome that, the rest should be reasonably simple (though I won't be able to do it as my experience with Linux is limited - and only user-level at that).

 09-17-2008, 04:14 AM #8 pepak Guru   Posts: 601 Karma: 4150 Join Date: Mar 2008 Device: Sony Reader PRS-505 Version 0.04 compiles with FreePascal. If anyone wants to modify it to work under Linux, I would appreciate it. It shouldn't be too difficult, I hope. Even better, someone who knows Python could rewrite the utillity into Python. That way it could be made a part of Calibre and use its libraries - especially metadata extraction, which is, at the moment, extremely slow.
09-17-2008, 07:10 PM   #9
gwynevans
Wizard

Posts: 1,343
Karma: 1065246
Join Date: Nov 2007
Location: UK
Device: iPad 2, Nexus 4 & 7, Kindle Voyage
Quote:
 Originally Posted by pepak Even better, someone who knows Python could rewrite the utillity into Python.
I haven't got time myself, because things are getting busy at work, but I've attached a Python script that I wrote a while back to create collections from file names (of the form 'author [- series N ] - title.lrf') in case you want to try yourself, as it might help getting started.

I'm not sure if it's 100% correct though, as when I just ran it, the Reader showed the "sd" icon by the collections, although they seemed to work OK!

Run with the file to modify as an argument, e.g. "lrfCategorize.py F:\database\cache\media.xml"
Attached Files
 lrfCategorize.zip (1.5 KB, 429 views)

02-05-2009, 03:02 AM   #10
just_me
Member

Posts: 18
Karma: 1334
Join Date: Feb 2009
Device: Nook Simple Touch (rooted)
Quote:
 Originally Posted by pepak Aargh. Apparently the Reader uses File Modification date for files stored on SD card (File Creation date for files stored in built-in memory). Will have to fix that and perhaps do some testing with SD card (I onlty tested built-in memory so far).
Hi,

I got interested in the whole collections issue (based on directories and such) and I noticed that apparently my reader uses the file modification date rather than the creation date for the books in the internal memory.

 02-05-2009, 11:50 AM #11 pepak Guru   Posts: 601 Karma: 4150 Join Date: Mar 2008 Device: Sony Reader PRS-505 With 18040 it works as I described.
04-18-2009, 01:05 AM   #12
Student1
Groupie

Posts: 159
Karma: 170
Join Date: Feb 2009
Device: PRS-505
Quote:
 Originally Posted by pepak Command-line utility for generating collections for Sony Reader. The whole point is that while it is easy to upload books to the Reader without having to use any GUI tools, it is much more difficult to actually generate and maintain collections (playlists) - it requires manual editing of a cache file and updating it every time a book changes. This tool lets you maintain a list of files that constitute a collection and generate all the collections by running one command. Source code included. http://www.pepak.net/sonyreader/colbuild.zip
Very interesting idea, thank you for your efforts!

I wonder is there a way to append collection data or get the list from whats on the sony reader already?

From what i can see you build up your list from a list of titles you enter manually in the ini file, is there anyway to get the information from calibre. For instance upload all the books with calibre and if the collection information fails... like it always does the programm can then rebuild the xml from the calibre metadata info from those books?

04-18-2009, 02:41 AM   #13
pepak
Guru

Posts: 601
Karma: 4150
Join Date: Mar 2008
Quote:
 Originally Posted by Student1 I wonder is there a way to append collection data or get the list from whats on the sony reader already?
It would be quite easy to import an existing playlist - it's pretty much the opposite process of what ColBuild does. I just never saw the need for it - the regular "list directory contents to file" seemed fast enough to me, especially considering that I will only ever do that once.

As for appending data - the utility does not (or should not, unless I made some error) modify collections already stored in the Reader. It should only overwrite the collections that are defined in its playlist and leave the others alone.

Quote:
 From what i can see you build up your list from a list of titles you enter manually in the ini file,
If you are familiar with regular expressions and keep your books in an organized manner, you can also skip the ini-based playlists completely and go for regular-expression-based playlists instead - see options CollectionRegExp and CollectionRegExpTitle.

Quote:
 is there anyway to get the information from calibre. For instance upload all the books with calibre and if the collection information fails... like it always does the programm can then rebuild the xml from the calibre metadata info from those books?
Theoretically, yes, it is possible. It depends on whether you can export the metadata from Calibre in a suitable format. The big problem is that you would have to find a suitable field for collection information which would appear in all possible formats.

 04-19-2009, 12:39 AM #14 Student1 Groupie   Posts: 159 Karma: 170 Join Date: Feb 2009 Device: PRS-505 Thanks for the information! In any case it a very smart utility! Ive been struggling with the prs 505 loosing collection after the number of books on the device becomes too big! I think this utility is just the answer to this problem! Thanks!!
 05-13-2009, 08:12 AM #15 vvaidy Junior Member   Posts: 1 Karma: 10 Join Date: May 2009 Device: Sony PRS 505 This is terrific ... but is there documentation on how the collections system works? I gather it is an XML file, but I'm wondering if there is a doc that describes the file, format, etc. The reason I ask is that I would like to modify one of these two programs to work with some other software I use, and it isnt quite clear to me exactly how Collections are stored on the PRS 505. Thanks! Last edited by vvaidy; 05-13-2009 at 08:18 AM. Reason: added info