Shiny New E-Book Gizmo: The Amazon Kindle


View Full Version : Multi-directory search


daudi
04-15-2008, 09:25 AM
This is a spin-off from something else I'm playing with. The built-in iliad search will only search the current directory, and if I'm not mistaken will only search the title field, not the description.

This little script will allow you to search multiple directories and will grep the manifest file. This means you can put key words in the description field and it will match on those.

You specify where to start the search in the run.sh script in the search folder (for now). I have included some examples. It is currently set up to search the whole USB device, but it would be easy to change to search CF or internal memory.

You could even copy the search folder and have one for USB, one for CF, etc. or even for specific folders on your CF, e.g. /mnt/cf/books and another for /mnt/cf/articles.

There is another script for deleting the results. Don't use the built-in delete because this deletes the original files rather than the symlinks. I think thomas posted a better way to do symlinks and I'll try that later.

You enter the search phrase by changing the FIRST LINE of the description field of the search entry. At the moment this is "albert", because that is what I was searching for when testing this. Use the label function to change the description, then click on the search entry.

So, in summary, these are the steps:


get shell access if you do not already have it
unzip search.zip somewhere, e.g. your PC
(change the STARTDIR in search/run.sh to where you want searches to start from--no longer necessary)
copy the whole thing to somewhere on your iliad (anywhere will do)
navigate to the search folder using the contentlister
change the description field to your search expression
click on the "search" entry in the contentlister
when the search is complete you will see a "results" folder
view your results in the results folder
delete the results using the "delete" entry in the contentlister

I started doing this because I think I will soon have a way of tagging and taking snippets from PDFs and plan to store the tags in a tags file in the appropriate PDF container directory and snippets in a snippets file. I want to be able to search for snippets or tags.

The main limitations at the moment are:

no way to search within search results
you have to navigate to the search folder, you can't just get there quickly from anywhere
inelegant way of getting user input


I've tested this a little on my USB key with about 800Mb of files and it did not fall over. Feedback welcome.

Edit: just made a couple of updates to zip file.
Edit: the limit on the number of results is not working. I'll have to look into this.
Edit: new version---quoted search expression to allow spaces
Edit (2008-05-01): new version that I think fixes the spaces in filename problem and adds a path config tool. Also fixed hit limit.
Edit (2008-05-01): Added snippet search tool

yokos
04-16-2008, 07:13 AM
It seems like spaces are not allowed in directory names.
Search string: Krause
grep: /mnt/cf/pdf/Foundations: No such file or directory
grep: of: No such file or directory
grep: GTK+: No such file or directory
grep: Development/manifest.xml: No such file or directory
[...]
./run.sh: 87: [[: not found

After changing dir's name from "Foundations of GTK+ Development" to "Foundations_of_GTK+_Development" it is working.
:iloveyou:

[EDIT] I had got a space bug in my jukebox mpd scripts, too.

daudi
04-16-2008, 07:45 AM
Thanks! I've now quoted the grep so hopefully fixed the space bug.

yokos
04-16-2008, 08:08 AM
Thanks! I've now quoted the grep so hopefully fixed the space bug.
Not yet. It doesn't work with spaces. :rolleyes:
[very OT] Battery of my iLiad is low & I don't have a power adapter next to mine. :eek:

daudi
04-28-2008, 04:53 AM
Not yet. It doesn't work with spaces. :rolleyes:
[very OT] Battery of my iLiad is low & I don't have a power adapter next to mine. :eek:

OK, I've looked into this a little more. To handle this nicely we need a newer version of busybox. The version on the iliad does not seem to support -print0 for find or -0 for xargs, but the current release does. This current release is version 1.10.1. Adam has released version 1.7.2 (http://www.mobileread.com/forums/showpost.php?p=113952&postcount=5) and I am not sure if that supports these options or not. I'll try to test it this evening. The alternative would be to use temporary files in the script instead. That would be a less elegant solution though.

yokos
04-29-2008, 08:03 AM
OK, I've looked into this a little more. To handle this nicely we need a newer version of busybox. The version on the iliad does not seem to support -print0 for find or -0 for xargs, but the current release does. This current release is version 1.10.1. Adam has released version 1.7.2 (http://www.mobileread.com/forums/showpost.php?p=113952&postcount=5) and I am not sure if that supports these options or not. I'll try to test it this evening. The alternative would be to use temporary files in the script instead. That would be a less elegant solution though.
Oh, interesting.
It should be no deal to update busybox. Termial apps are dream targets to port compared to GUI apps with all the odds.
I think Adam hasn't enabled all features of busybox during configure to make the binary small.

EDIT:
Using busybox:

BusyBox is extremely configurable. This allows you to include only the
components and options you need, thereby reducing binary size. Run 'make
config' or 'make menuconfig' to select the functionality that you wish to
enable. (See 'make help' for more commands.)

daudi
04-29-2008, 08:21 AM
Ahaaa. Very helpful, thanks. I'm not going to be able to look at this again for a few days, but this will save me time mucking around trying to figure out things that might not need figuring out. Thanks.

daudi
05-01-2008, 08:15 AM
It turned out to be rather easy to make this work with the standard busybox so I have decided to do that rather than require people to upgrade their version of busybox. I have tested it and think it is working now with directories with spaces in the path and would be grateful it you could try it again Yokos.

I have also changed the way that search paths are configured. Instead of having to hard code the path in the script, I have now added a config contentlister entry that reads a file called search.rc. This file consists of one line per directory, a semi-colon and a description. The default contents on this file are: /mnt/usb;USB
/mnt/cf;Compact flash
/mnt/free;All internal memories
You can easily add others to particular directories. With this you just click on the entry in the contentlister to cycle through the paths to search. The description field in shown in the contentlister is updated with the description of the current search path.

I've also fixed the number of hits limit code. This is currently set to 40 results. You can change this near the top of the script.

I'm updating the post at the top of this thread with this version.

Dabon
05-02-2008, 10:37 AM
Dear Daudi,

Good day!
Thank you for working so diligently on such great and useful tool.
I have tried to install this tool several times, but unsuccessfully. i am not getting the message " installation successful". I am able to switch the locations for research between internal memory, USB and CF, but the launching of research is unsuccessful, the iliad does not react when I click on search folder...
Any ideas or suggestions on what I could do to get it working?

Thanks a million in advance!

D.

daudi
05-02-2008, 10:50 AM
Dear Daudi,

Good day!
Thank you for working so diligently on such great and useful tool.
I have tried to install this tool several times, but unsuccessfully. i am not getting the message " installation successful". I am able to switch the locations for research between internal memory, USB and CF, but the launching of research is unsuccessful, the iliad does not react when I click on search folder...
Any ideas or suggestions on what I could do to get it working?

Thanks a million in advance!

D.

There's nothing to install so there's no need for an "installation successful" message :)
I suspect it probably is running but using the default search pattern. I think I probably need to re-write the first post or write another one to make things clearer.

A key thing from my first post is:
You enter the search phrase by changing the FIRST LINE of the description field of the search entry. At the moment this is "albert", because that is what I was searching for when testing this. Use the label function to change the description, then click on the search entry.


Have you used this to enter the search phrase? It is a kludge, I know, but seemed to me to be the simplest way of getting simple user input. Once you have entered your search phrase clicking on the search entry should run the search.

WARNING! At the moment the general search searches the entire manifest file. This means that if you are searching for "The Great Escape", for example, do not enter "esc" as the search phrase. "esc" will match "escape" okay, but it will also match "Description" which exists in (almost?) every manifest file.

BTW, I should also explain that the snippet-search relates to this thing (http://www.mobileread.com/forums/showthread.php?t=22804).

If you have entered your search phrase correctly and it is still not working we'll have to delve a little deeper. Let me know how you get on.