New Google Chrome Browser

[pilotbob]

Originally Posted by axel77

* Well software agents are something that I read about in almost every computing scientific article, but never have seen anything close to it in reality.

I think search agents are certainly/possible the future. However, I don't think Google will become irrelevant. On the contrary... the search agent will need a search engine service backing it. There is no way your local PC using even a broad band connection will have the time/bandwith/space to search all the data on the web... and you don't want all your resources doing that.

What will happen is the search indexes like Googles will need to become even more more annotated with metadata to categorize the data. So searches for stuff will become more focused. Single word searches won't happen anymore. For example, you might put in a word like france and then a suggestion list will pop up with stuff like food, travel, sites, landmarks, history to narrow down the search.

No Google will actually be a provider of search agents which use Goggles search index to mine for what you want. The search agent of course will watch what you do on your pc... It will be like the "awesome bar" in FF3 on steroids suggesting stuff based on your documents, emails, sites you've browsed, forum posts, blogs you follow, etc.

For some real interesting search results take a look at cuil.com... it is a new search engine that does some of that "search agent" type stuff. In FF3 you can't see the text box... but in IE it shows. I'll have to email them about that.

BOb

[axel77]

For me an agent is a (theoretical) piece of software you send away, that does collect/process information somewhere else based on its code, and then comes back with the data. Maybe we misunderstood what we were talking about.

To some very primitive degree of this is the way how gnutella searches work. You send away your query string with your destination IP, and it will spread itself autonomously over the gnutella network, and every node that has a hit, or knows of a hit sends the answer to your IP. However its a very primitive searching technique which a) only searches contents from the title line. b) It takes eons to get a good answer list. I mean just imagine the code of google, they have the whole internet indexed (apart from the invisible deep net) and you hit any query, and get an answer in milli seconds... just imagine what great development there is behind this... If somebody would have asked me 20 years ago, if such is technically possible, I'd have said no... And I did see the pre-altavista days, where webcrawler results came drippling it piece by piece.... unsorted of course...

[Shaggy]

Originally Posted by pilotbob

That is so not true. Yes, you need to store the data that is contained in the index, you do not need to cache the full page/site.

BOb

How do you think they make the index? They can't index a site without downloading/caching it.

[acidzebra]

Originally Posted by Shaggy

How do you think they make the index? They can't index a site without downloading/caching it.

Initially I wanted to say Bob was right, you can download the content, index it, erase the content and keep just the index and the links, but then you could never show that ever-so-handy two phrases excerpt/context surrounding your keyword hit. If the amount of keywords is large enough, you might as well just keep all the content and reference that - a simple lookup in a cached copy is more efficient than storing two lines of text for every keyword.

Of course, you could work without the context of one or two lines and let people guess by the url whether a given search result is good or not but somehow I don't think that will go over well.

So you are correct, without the cache you will not get the context, just a keyword and a link. Which is a horrible way to search the web.

[pilotbob]

Originally Posted by Shaggy

How do you think they make the index? They can't index a site without downloading/caching it.

Right, but they only need to store the index data once the page/site has been indexed. There is no need for them to keep the whole page in their cache... which the do if you have ever view google cached pages.

googleguide http://www.googleguide.com/cached_pages.html talks about the fact that Google will remove the cached pages if requested by the site owner. I assume this is done with the no-cache directive in the header. However, removing it from the cache doesn't remove it from the index. So, they do not "need" to retain the cache once the page has been index to provide search results.

BOb

[Shaggy]

Originally Posted by pilotbob

Right, but they only need to store the index data once the page/site has been indexed. There is no need for them to keep the whole page in their cache... which the do if you have ever view google cached pages.

I understand that, but Taylor's argument is that there should be no caching of his copyrighted material (or profitting from it via advertisements) at all. He's saying he wants zero caching, not just changing how long a cache is kept for. To me, it sounds like he's saying "no caching of copyrighted content, ever" which I was pointing out means "no search engines".

Also, as acidzebra pointed out, not retaining any cache means that you completely lose the excerpt/context and search results just become a list of URLs with no way of knowing any more details without going to every link.

Yes, as you pointed out, there are already ways to get google to not retain a cache as well as to not index your site at all. Taylor doesn't seem to be happy with those solutions though.

[nekokami]

I really don't want to have to play with yet another browser, but I suppose if Google Chrome gains any appreciable market share, I'll have to install it so I can test the website I am responsible for. :sigh:

The Google ethical question is interesting. I've heard Taylor's concerns before, but not so clearly stated as they ended up being in this thread.

Originally Posted by Taylor514ce

Both. The "Cache" and the advertising system are linked. To list specific items of concern to me:

1. Caching content in the first place. I have concerns about privacy, and authorial control.
2. Using the cached content (used for search results) to drive advertising.

Regarding the first point: The data doesn't have to be stored for the indexing to take place, and I appreciate that what Taylor is concerned with here is that Google can continue to offer copies of content he has removed, for example.

Let's suppose that one is responsible for providing critical information, e.g. current status on an epidemic. One would need to be able to control whether "stale" versions of that content are displayed. Google cache could display such stale content.

However, is it reasonable to require content publishers to "opt in" to allowing this data to be cached for viewing, when again, this is the model upon which the web was built? Perhaps the "no-cache" directive should have more strength than a "gentleman's agreement." There might be problems enforcing this internationally, however. It would probably need to be appended to something like the Berne convention.

Regarding the second point: is it reasonable that Google assumes that content publishers want their sites to be listed within a Google search result, regardless of the fact that Google will place advertisements on that page? Is it legal? Is it an ethical use of publicly available, but privately owned data?

I'm not an expert on the law, by any means. But I can readily come up with examples of cases in the physical world in which directory listings of publicly available information are provided either for a fee or via advertising-supported means. For example, most telephone companies in the US provide listings of residential and business phone numbers, and advertising is often sold in the same printed volumes. (In fact, in recent years, the phone company I deal with has taken to charging a fee not to list this data.) Names and addresses are collected by numerous agencies and sold to direct marketers. I don't like this process, but it is apparently legal. Photographs of scenic villages are used to advertise venues located in those villages. I don't believe the owners of the houses and gardens that are included in these photographs even need to be asked if they want their property included in such a photograph, so long as the view is from a public location, such as the street. The distinction, again, seems to be where to draw the line between "public" and "private."

Originally Posted by Taylor514ce

Ideally, I want a web that is driven by content authors. I think the future web will have "search agents" rather than "search engines". Rather than go to a "search engine" which consumes sites and offers results/ads, we'll be able to construct custom search agents and send out our own spiders and bots, which will return results without violating copyright and without weighting the results based on advertising campaigns. Personal sytsems will become powerful enough to create your own individual "search cache", and web sites will cooperate in communicating and updating "subscribers". In essence, "Google" will become irrelevant.

Originally Posted by axel77

To some very primitive degree of this is the way how gnutella searches work. You send away your query string with your destination IP, and it will spread itself autonomously over the gnutella network, and every node that has a hit, or knows of a hit sends the answer to your IP. However its a very primitive searching technique which a) only searches contents from the title line. b) It takes eons to get a good answer list.

This is one problem I can see with this kind of content author-driven search service. Another is that this really only distributes the caching problem even more widely.

It also assumes that users-- owners of private computers-- will be willing to devote their own resources to being part of a search mechanism, rather than putting up with ads (and the bias that comes with them). Based on my observations of seeder vs. leecher ratios on bittorrent networks, I'm not optimistic.

But let's assume that some non-commercial search entity exists and is effective. One would need to be able to determine whether to allow searching of ones content by commercial or non-commercial means, and the present robots.txt and nocache directives would not be sufficient, because they would exclude searches by such non-commercial services. The question might then become: do we need a "non-commercial" directive, or a "commercial-ok" directive? I.e., should allowing commercial search engines to index and/or cache one's site be opt-in, or opt-out?

I don't know the answer to this. I suspect that the vast majority of people who create web content would prefer to be included in both commercial and non-commercial search engines, even if both were viable. So from a usability standpoint, opt-out would make sense. From a legal and ethical standpoint, however, I can see preferring opt-in. Either way, again we would need something stronger than a "gentleman's agreement," which would probably require an international convention.

Well, those are my current thoughts on this complex issue....

[Shaggy]

Originally Posted by nekokami

Let's suppose that one is responsible for providing critical information, e.g. current status on an epidemic. One would need to be able to control whether "stale" versions of that content are displayed. Google cache could display such stale content.

That's what no-cache and robots.txt are for. Every search engine, that I know of, honors them.

Regarding the second point: is it reasonable that Google assumes that content publishers want their sites to be listed within a Google search result, regardless of the fact that Google will place advertisements on that page? Is it legal? Is it an ethical use of publicly available, but privately owned data?

It has already been ruled on in the US. Search engine indexing/caching is considered "fair use" by the courts. I don't know specifically what other countries have ruled, but I have never heard of a country that says it's illegal. I'd be interested in hearing about it, if you know of any.

[Taylor514ce]

Some specific lawsuits I know of: New York Times, which provides certain content to subscribers only. Google became a subscriber, took the content, and cached it. Non-subscribers could then view the content directly from Google's cache. To suggest that this was ethical because the New York Times didn't have a no-cache entry etc. is ludicrous, and Google lost that one. Another was Perfect 10, a "men's magazine", which had photos from their magazine on their site, presumably to entice people to join the site and/or subscribe to the magazine. Google cached the images and made them available via images.google.com, circumventing the publisher's intent. Google lost that one, too.

The problem with robots.txt and no-cache is that there is no penalty if a search engine decides not to honor it. One shoudn't have to say "don't use this, it's mine" because copyright already covers that. The court decision in Nevada that the cache constitutes fair use is wrong, and I'm sure we'll see more specific cases in the future.

Google has taken this rather narrow ruling to mean that they can now cache ANYTHING, and started scanning copyright books from libraries, which caused several more lawsuits.

That's the Google attitude: we will scan, search, index and cache whatever we want, however we want, and you have to sue us if you don't like it.

[Shaggy]

Originally Posted by Taylor514ce

Some specific lawsuits I know of: New York Times, which provides certain content to subscribers only. Google became a subscriber, took the content, and cached it. Non-subscribers could then view the content directly from Google's cache. To suggest that this was ethical because the New York Times didn't have a no-cache entry etc. is ludicrous, and Google lost that one. Another was Perfect 10, a "men's magazine", which had photos from their magazine on their site, presumably to entice people to join the site and/or subscribe to the magazine. Google cached the images and made them available via images.google.com, circumventing the publisher's intent. Google lost that one, too.

Sure, those were specific cases of subscriber content. That's different from caching publicly available content. There was a similar case in Belgium. A content provider had articles available for free on their website for a limited time. Google cached the free articles, which wasn't a problem. However, the company also takes older content off of it's website and places it into a seperate archive. It then charges customers for access to the archive. They sued Google, saying that because the content was no longer publicly available, and only open to subscribers, that Google should not be allowed to offer it for free. The judge agreed.

However, the funny thing is that the judge didn't impose a penalty on Google. Rather, he said that Google should take content down if the original website requests that they do so. Which was actually Google's policy all along.

The problem with robots.txt and no-cache is that there is no penalty if a search engine decides not to honor it.

Do you know of any search engines that don't honor it?

That's the Google attitude: we will scan, search, index and cache whatever we want, however we want, and you have to sue us if you don't like it.

It's not just Google, it's all search engines. That's how they work. If you want to retain privacy over your content, then don't post it on an open web site.

[Taylor514ce]

Originally Posted by Shaggy

If you want to retain privacy over your content, then don't post it on an open web site.

Privacy is one concern, yes. But you've missed my central point, so I'll amend your statement:

"If you want to retain your copyrights, then don't post anything on an open website." I disagree with that in principle, and so do these specific cases. I see a major philosophical difference between "we'll take it down if you ask us to", by which time the damage is done, and "you need my permission before you can have it in the first place", which is how copyright in fact works.

[Shaggy]

Originally Posted by Taylor514ce

"If you want to retain your copyrights, then don't post anything on an open website."

You don't lose your copyright when a search engine caches your content.

I disagree with that in principle, and so do these specific cases.

No, these specific cases were about caching subscriber content. IE content that was not meant to be publicly available.

..."you need my permission before you can have it in the first place", which is how copyright in fact works.

Not when it falls under fair use.

[JSWolf]

[OT]Amalthia, is your avatar from The Last Unicorn? I really like that movie.[/OT]

[Amalthia]

Originally Posted by JSWolf

[OT]Amalthia, is your avatar from The Last Unicorn? I really like that movie.[/OT]

Yep, my avatar is from the Last Unicorn.

I loved the movie, the book, and I have the soundtrack.

[hidari]

http://news.bbc.co.uk/2/low/technology/7597699.stm