Wei-hsing Wang, Ph.D.
1. What is Google anyway?
With so much news recently on Google, no double you will say Google is a search engine. Well, how does search engines work?
There are four parts of a search engine: user interface, proprietary ranking process, database system, and spiders (also called worm or robot). What you see on their web site is just the user interface part. You can enter a keyword or a phase into a box on the web page, and you will get a list of search results on the next page. Behind the scene, the user interface talks with the ranking process and the database to select findings (hits) from the search engine’s collection and present them in their own order to you.
How did the search engine’s findings get into its database? That is the job of a spider, which is a software function like your web browser. They sent out requests to all the web site in the world, (well, not quite, but at least 20%, and it is a huge number of web pages) and put all the pages into the database. Search engines compete with each other in all those four areas. They try to offer more features in user interface so it is easy to use and to narrow down to specific needs, better ranking system so that the results are more relevant, more powerful database and spiders so that they can provide answers in nanoseconds and have the biggest collection of web pages.
A “Directory” is a distant relative of search engines. People, instead of spiders and databases, are finding and organizing web resources. People look at web sites and write comments about them, and categorize the web sites. Users click on categories and narrow down to the specific area of needs and find a list of URLs (links). Some Directories are “closed,” which are designed and maintained by a company. Others are “open,” which are offered by volunteer editors.
Two other members in “searching the web” family are Portals and Meta-tools. In general, a tool let users search several search engines without setting up its own database is a meta-tool. A web site offers both a search engine and a directory (and “much more”) is a portal.
Here are examples of each category:
2. Five Generations in the family
How did those different kinds of tools came to existence? What will they evolve into in the future? Let’s look back to 10 years ago.
Some people may still remember those days before “search engine” became a commonly known term. People were told or bumped into a URL. People used “bookmark” to remember the URLs. Everybody soon found out that a long bookmark list was not easy to manage. Sadly, some people today still struggle to manage their long bookmark file (or something similar).
Then came the big idea, search engine. Search engines caught on quickly. Those free tools help you find anything you want by typing in keywords. People can get to many great web pages effortlessly. The problem is no longer “can I find anything on this subject” but “I found too many pages than I can handle.” The meaning of “too many” quickly grew from thousands into millions of URLs. However, the quality of the URLs is very unreliable. In addition, search engines also have huge blind spots, also called the invisible web.
Many people realized the shortcomings of search engines. They introduce URLs on their own web sites. In many cases, people recommend URLs on their “My favorite web sites” pages. Since a person screens the web sites, the quality is generally better than those URLs selected by search engine software. However, these URL pages age quickly. People could not keep those links up to date. Links can be broken. Web sites’ quality and structure may also change.
Human beings can do a better job than search engine “spider” software. But, this is clearly not a one-person job. Organizations offer the next generation solution: the directories. An organization can watch over an area of web sites, make recommendations with a team of experts. The recommendations can be kept in a database. People can find quality web sites with the help of directories. However, the biggest problem with directories is that they are mostly in the invisible web. People don’t know they exist.
New age Information specialists (some are librarians) are here to help. Librarians are not made equal. While some librarians still will point you to the index cards, some will answer your questions with a Google search, only a few will actually point out the differences between an array of search engines and directories. If you find those new age information specialists, you are in luck. The 5th generation solutions will give you much better answer than the 2nd generation search engines. However, finding them and get them to work with you is not easy. Many people can’t find those experts and are still at the mercy of search engines.
ZoomerOne is the new generation solution that can lead the public to the best web resources. ZoomerOne is a software tool, which gathers best web resources for the subject, the keywords, and the users, based on information specialists’ suggestions. NicheUSA, L.L.C. works closely with information specialists to create and maintain ZoomerOne, which combines the power of search engines, individuals, and directories.
3. Problems and Tips
Many people only use search engines. Many people count on the searching power provided by Google (and the future MSN) for decision-making. Only a small but growing number of us can see beyond Google.
The biggest problem of search engines is lack of quality control. Search engines search everywhere, including garbage dump. You are getting a mixed, polluted list of findings. Your findings can contain the three most unwanted kinds of web sites: those wanting your money, those wanting your information, and those wanting your soul.
Many web sites and web elements are heavily used to sell you products and services, or collect your personal information, with or without your knowledge, for resell or more targeted sale. Other web sites provide users with misleading, poisonous, and awful material, which is taxing the society. As long as those unwanted web sites meet the ranking criteria of a search engine, you will get to see them.
Worse yet, many (if not all) search engines sell list positions for profit, which means the ranking process allow whoever pays the search engine be listed first. Google is probably the most honest one that do not mix paid and fairly ranked links together. However, with the trick of “Google Bombing,” (a method to manipulate search result position in Google search, you can find more information by search Google Bombing in Google) even Google findings cannot be trusted. Experts cannot offer any help besides warning “Buyers Beware!”
A savvy user has to look beyond Google. Actually one has to look beyond all search engines. At the minimum, use some directories. The first thing is shopping around. Take a look of the spectrum of tools and understand their strengths and weaknesses. Search engines are capable of searching a huge number of web sites, perform full text search, provides quantity but not quality. Directories can offer structured, categorized information around a subject with good quality, but are not updated as often as search engines, and not as extensive as search engines. An experienced user generally use search engines and directories like using both hands, passing findings between tools to locate the best web resources.
4. ZoomerOne to Infinity and Beyond
The web is ever changing, so do the tools of using the web resources. Google 2.0 is not the ultimate answer, and MSN 2.0 is not the ultimate answer, either. A combination of tools from different categories could be your best bet. With the tools we mentioned above you can make use of the best web resources much better than just provided by Google. However, if you do not intend to make yourself a professional web searcher, you may need to consult with information specialists from time to time to get updates.
All those general search tools are good for the general, however, you know you are special, and you have special needs. If you can go to a specialist, why do you need the opinion of a generalist? ZoomerOne can offer you the best web resources customized for your knowledge level, subject, and keywords based on information specialist’s knowledge.
Many more search engines and directories will join the market, ZoomerOne product family will grow to take advantages of those new tools like web browsers grew to take advantages of new HTML and multimedia features. A savvy web user can see beyond the hype and get much more from the best web resources than plain Google.