|
BTW, although parse_index is the preferred way to deal with this, you can also try using "recursions" which sets the depth that links will be followed. If your pages are clean, with only links to pages you want, this might do the job.
Also, I should warn you that linklogs where the links are to random pages of interest are poor fodder for recipes. Recipes need some consistency to the pages retrieved if it is going to remove ads, apply formatting, etc. If the links point to just any old random pages, and not to pages with common attributes, the recipe results will be poor.
|