SiteScooper with iSiloX

macrotor · 11-26-2002, 12:25 AM

Okay, I just happened to bumble onto a link to your site while looking for Yahoo buddies about iSilo.  Truth be told, I thought iSilo's fame was starting to fade and I wanted to make sure it was still supported.  But, onto the subject.

Some of you have mentioned that SiteScooper could be used with iSilo.  I am one of those people that have taken it to the extreme.  I have a Perl script that runs Sitescooper on a large number of news and comics sites (since it handles cookies quite well).  It exports this all into a directory of html files.  The Perl script then calls iSiloXC to convert these files to my Palm install directory.  Pretty easy since SiteScooper automatically creates an index.html file that links to all created content in the "text" directory.  This Perl script is called from cron to run about 30 minutes before I hotsync each morning.

The result?  It leaves Avantgo in the dust.  It takes a little time to set-up, but it's quite easy to add and edit content once you have it running.  Sitescooper can be found at http://www.sitescooper.org.

My system is MacOSX 10.2.2 although Linux or FreeBSD will work as well.
So, I'm offering my assistance to those who want it.  In return, just make sure you promote iSilo and keep the format alive.  Plucker is the only other automated converter, and it just doesn't do as good of a job as iSilo.  You don't have "files" section like the Yahoo groups do, so I can't post my scripts here.

I'm going to stick my neck out this ONE time.  My email is [url=mailto:james_sorenson@yahoo.com.">james_sorens on@yahoo.com.[/url]  

Oh...just a side note.  I also use iSilo with mhonarc so that I can convert my email within Mail.app...formatting, pictures and all!  Man, I love iSilo!

Alexander Turcic · 11-26-2002, 05:30 AM

Macrotor, that is very nice to hear! I know there is a lot you can do with sitescooper, I just haven't had the time to explore it myself yet.

If you like, I can add a sitescooper section to the mobile section of my site and add your contribution to it (with all due credits). We could add a little step-by-step guide on how to make sitescooper run with iSilo and give a few examples.

macrotor · 11-26-2002, 11:04 AM

A sitescooper topic may not be a bad idea.
I suppose I could just print the text of the PERL script in a separate message.

Since I use a VisorPhone, I've found a way to make iSilo act as a "News Push."  Every threee hours, my PERL script runs sitescooper and iSiloXC to get all my news and email.  It then copies the resulting files to my "Sites" folder on my iDisk so that I can download them from the web.  I use LGet on my Visor to periodically download a fresh set of news and email when I have some time to burn.  This mimics the feature where Avantgo can refresh news wirelessly.

Zire · 10-16-2003, 01:38 PM

I would like to know how to get only the new york times articles into my palm.

stobs · 10-17-2003, 03:18 AM

Hallo,

I would like to have a sitescooper section, too. Alexander, we talked about that a long time ago.

I use iSiloX together with sitescooper, too. Since yesterday with iSiloX 4.0.

You don't need extra software to use iSiloX with sitescooper, just fetch the most actual version:
http://sitescooper.org/download/index.html
Those files under "The Bleeding Edge"
win: http://sitescooper.org/devel/sitescooper-full.zip
mac: http://sitescooper.org/devel/sitescooper-full.MacOS.zip

Besides of a have-to-be-written step-by-step doc for installing sitescooper, you have to:
- Download and Install perl: http://www.perl.com/pub/language/inf...re.html#binary
- download and "install" sitescooper (see above)
- edit the file sitescooper.cf, that's perhaps the most tricky part (not too hard).

after that, run it:
eg:
perl sitescooper.pl -config sitescooper.cf -misilox

-S.

Quote:

Originally Posted by Alexander

Macrotor, that is very nice to hear! I know there is a lot you can do with sitescooper, I just haven't had the time to explore it myself yet.

If you like, I can add a sitescooper section to the mobile section of my site and add your contribution to it (with all due credits). We could add a little step-by-step guide on how to make sitescooper run with iSilo and give a few examples.

Zire · 10-17-2003, 01:52 PM

What about my question?

juco · 10-19-2003, 07:12 AM

Quote:

Originally Posted by stobs

Hallo,
Besides of a have-to-be-written step-by-step doc for installing sitescooper, you have to:
- Download and Install perl: http://www.perl.com/pub/language/inf...re.html#binary
- download and "install" sitescooper (see above)
- edit the file sitescooper.cf, that's perhaps the most tricky part (not too hard).

after that, run it:
eg:
perl sitescooper.pl -config sitescooper.cf -misilox

-S.

Hi all,

I'm using the combination of Sitesooper and Plucker for a long time now and I'm quite happy with that combination. However, since everybody praises iSilo and I'm missing the option of saving viewed articles to memopad, I'm evaluating iSilo right now.

My problem is that I don't get Sitescooper to do the conversion work. I switched the commandline parameter from "-mlucker" to "- misilo", so Sitescooper tried to run "iSiloC32.exe" which wasn't there. I thought that this must have been an earlier version of the iSilo converter, so I edited sitescooper.cf to look for iSiloXC.exe instead.

Now Sitesooper DOES find the conversion program, but the parameters involved seem to be invalid. See my screenshot of the error messages at

<http://www.klein-aber-fein.de/bildarchiv/sonstiges/problem.jpg>

What do I have to do to get the whole thing running? I still have Plucker as a working fallback solution but now I want to kow... ;-)

Thank you for yor support!

Best wishes,
Ralph

Personal Palm Tips at http://www.klein-aber-fein.de/palm/

Alexander Turcic · 10-19-2003, 07:57 AM

@stobs and others:

Would you be interested in a dedicated Sitescooper section here on this forum? I can put it up in no time, but its success would also depend on your help since I am not very experienced with Sitescooper yet.

stobs · 10-19-2003, 11:55 AM

I think we should try it,
I don't know, how many are using sitescooper and will try to support other.

-S.

Quote:

Originally Posted by Alexander

@stobs and others:

Would you be interested in a dedicated Sitescooper section here on this forum? I can put it up in no time, but its success would also depend on your help since I am not very experienced with Sitescooper yet.

emale07 · 11-14-2003, 09:34 PM

Is there someone who would be willing to spell out how to do this with iSiloX and Mac OS X to a complete perl newbie?

I'm serious - talk to me like I'm 6. None of the instruction that I've read anywhere make any sense. I've never scripted ANYTHING before.

emale

11-14-2003, 09:34 PM	#10
emale07 Enthusiast Posts: 33 Karma: 25 Join Date: Jun 2003 Location: Kansas City (USA) Device: Amazon Kindle 2	teach me, O wise ones Is there someone who would be willing to spell out how to do this with iSiloX and Mac OS X to a complete perl newbie? I'm serious - talk to me like I'm 6. None of the instruction that I've read anywhere make any sense. I've never scripted ANYTHING before. emale

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Download websites with links like iSiloX	baboyak	Calibre	2	08-12-2010 11:45 AM
iSiloX Web Channel Collection v1.1.0 for Plucker users	woodswalker	Reading and Management	1	06-08-2003 03:56 AM
Viewing iSiloX pages with Plucker	thenightfly42	Reading and Management	2	03-21-2003 02:38 PM
Sitescooper people?	Alexander Turcic	Lounge	10	01-14-2003 02:20 AM

11-26-2002, 12:25 AM	#1
macrotor Connoisseur Posts: 59 Karma: 2418 Join Date: Nov 2002 Location: Fremont, CA, USA Device: Tungsten\|C with Nokia6200	Okay, I just happened to bumble onto a link to your site while looking for Yahoo buddies about iSilo.  Truth be told, I thought iSilo's fame was starting to fade and I wanted to make sure it was still supported.  But, onto the subject. Some of you have mentioned that SiteScooper could be used with iSilo.  I am one of those people that have taken it to the extreme.  I have a Perl script that runs Sitescooper on a large number of news and comics sites (since it handles cookies quite well).  It exports this all into a directory of html files.  The Perl script then calls iSiloXC to convert these files to my Palm install directory.  Pretty easy since SiteScooper automatically creates an index.html file that links to all created content in the "text" directory.  This Perl script is called from cron to run about 30 minutes before I hotsync each morning. The result?  It leaves Avantgo in the dust.  It takes a little time to set-up, but it's quite easy to add and edit content once you have it running.  Sitescooper can be found at http://www.sitescooper.org. My system is MacOSX 10.2.2 although Linux or FreeBSD will work as well. So, I'm offering my assistance to those who want it.  In return, just make sure you promote iSilo and keep the format alive.  Plucker is the only other automated converter, and it just doesn't do as good of a job as iSilo.  You don't have "files" section like the Yahoo groups do, so I can't post my scripts here. I'm going to stick my neck out this ONE time.  My email is [url=mailto:james_sorenson@yahoo.com.">james_sorens on@yahoo.com.[/url]   Oh...just a side note.  I also use iSilo with mhonarc so that I can convert my email within Mail.app...formatting, pictures and all!  Man, I love iSilo!

11-26-2002, 05:30 AM	#2
Alexander Turcic Fully Converged Posts: 18,163 Karma: 14021202 Join Date: Oct 2002 Location: Switzerland Device: Too many to count here.	Macrotor, that is very nice to hear! I know there is a lot you can do with sitescooper, I just haven't had the time to explore it myself yet. If you like, I can add a sitescooper section to the mobile section of my site and add your contribution to it (with all due credits). We could add a little step-by-step guide on how to make sitescooper run with iSilo and give a few examples.

11-26-2002, 11:04 AM	#3
macrotor Connoisseur Posts: 59 Karma: 2418 Join Date: Nov 2002 Location: Fremont, CA, USA Device: Tungsten\|C with Nokia6200	A sitescooper topic may not be a bad idea. I suppose I could just print the text of the PERL script in a separate message. Since I use a VisorPhone, I've found a way to make iSilo act as a "News Push."  Every threee hours, my PERL script runs sitescooper and iSiloXC to get all my news and email.  It then copies the resulting files to my "Sites" folder on my iDisk so that I can download them from the web.  I use LGet on my Visor to periodically download a fresh set of news and email when I have some time to burn.  This mimics the feature where Avantgo can refresh news wirelessly.

10-16-2003, 01:38 PM	#4
Zire Fanatic Posts: 522 Karma: 14050 Join Date: May 2003 Location: Astoria, NY Device: Zire 71	I would like to know how to get only the new york times articles into my palm.

10-17-2003, 01:52 PM	#6
Zire Fanatic Posts: 522 Karma: 14050 Join Date: May 2003 Location: Astoria, NY Device: Zire 71	What about my question?

10-19-2003, 07:57 AM	#8
Alexander Turcic Fully Converged Posts: 18,163 Karma: 14021202 Join Date: Oct 2002 Location: Switzerland Device: Too many to count here.	@stobs and others: Would you be interested in a dedicated Sitescooper section here on this forum? I can put it up in no time, but its success would also depend on your help since I am not very experienced with Sitescooper yet.