The Invisible Web: Uncovering Information Sources Search Engines Can't See Book

	Figures and Tables	xiii
	Foreword	xv
	Acknowledgments	xix
	Introduction	xxi
	The Invisible Web in a Nutshell	xxii
	What to Expect from This Book	xxiii
	Boldly Go Where No Search Engine Has Gone Before	xxvi
	About www.invisible-web.net	xxix
Chapter 1	The Internet and the Visible Web	1
	How the Internet Came to Be	2
	Early Net Search Tools	3
	Enquire Within Upon Everything	8
	Weaving the Web	10
	Early Web Navigation	12
	The First Search Engines	13
Chapter 2	Information Seeking on the Visible Web	17
	Browsing vs. Searching	19
	Web Directories	22
	How Web Directories Work	22
	Issues with Web Directories	24
	Search Engines	26
	How Search Engines Work	26
	Issues with Search Engines	32
	Search Engines vs. Directories	36
Chapter 3	Specialized and Hybrid Search Tools	37
	Targeted Directories and Focused Crawlers	38
	Targeted Directories	39
	Focused Crawlers	41
	Vertical Portals (Vortals)	43
	How to Find Vortals	44
	Metasearch Engines	44
	Issues with Metasearch Engines	45
	Value-Added Search Services	46
	Alternative Search Tools	48
	Browser Agents	48
	Client-Based Search Tools	50
	Web Rings	51
	Fee-Based Web-Accessible Services	52
	Next Stop: The Invisible Web	52
Chapter 4	The Invisible Web	55
	Invisible Web Defined	56
	Why Search Engines Can't See the Invisible Web	62
	Four Types of Invisibility	70
	The Opaque Web	70
	The Private Web	73
	The Proprietary Web	73
	The Truly Invisible Web	74
Chapter 5	Visible or Invisible?	77
	Navigation vs. Content Sites	78
	Direct vs. Indirect URLs	79
	The URL Test	80
	Specialized vs. Invisible	82
	Visible vs. Invisible	83
	The Library of Congress Web Site: Both Visible and Invisible	86
	The Robots Exclusion Protocol	89
Chapter 6	Using the Invisible Web	91
	Why Use the Invisible Web?	92
	When to Use the Invisible Web	95
	Top 25 Invisible Web Categories	96
	What's NOT on the Web--Visible or Invisible	103
	Spider Traps, Damned Lies, and Other Chicanery	105
	Keeping Current with the Invisible Web	109
	Build Your Own Toolkit	111
Chapter 7	Case Studies	115
Case 1	Historical Stock Quotes	115
Case 2	Patent Information	117
Case 3	Real-Time Tracking	119
Case 4	Locating an Out of Print Book	120
Case 5	Telephone Numbers and Zip Codes	121
Case 6	Finding Online Images	122
Case 7	Investment Research	123
Case 8	The Invisible Web Fails to Deliver!	124
Chapter 8	The Future: Revealing the Invisible Web	127
	Smarter Crawlers	128
	The Promise and Pitfalls of Metadata	129
	Beyond Text	130
	Delving into Databases	130
	Hypertext Query Languages	132
	Real-Time Crawling	132
	Long Live the Invisible Web	133
Chapter 9	The Best of the Invisible Web	135
	Invisible Web Pathfinders	135
	An Invisible Web Directory	137
	Frequently Asked Questions about the Directory	138
	In Summary: The Top 10 Concepts to Understand about the Invisible Web	142
Chapter 10	Art and Architecture	145
	Architecture	146
	Artists	147
	Galleries on the Web	148
	Gateways to Art and Architecture Resources	151
	Reference	152
Chapter 11	Bibliographies and Library Catalogs	153
	Bibliographies	154
	Library Catalogs	160
Chapter 12	Business and Investing	163
	Company Information and Research	164
	Consumer Resources	169
	Economics--United States	169
	Economics--World	173
	Financial Institutions	175
	General Business Resources	176
	Government Contracts	177
	Industry-Specific Resources	178
	Investment Resources	182
	Jobs and Career Information	185
	Lookup Services	187
	Marketing Resources	188
	Pension Resources	189
	Personal Finances	190
	Philanthropy and Non-Profit Resources	190
	Research and Development	192
	Real Estate	193
	Tariffs and Trade	194
	Trade Shows and Conventions	196
Chapter 13	Computers and Internet	199
	Computers and Computing	200
	Internet Resources	203
Chapter 14	Education	207
	Classroom and Teacher Support	208
	Directories and Locators	209
	Financial Information and Scholarships	212
	General Education Resources	213
	Statistics	215
Chapter 15	Entertainment	217
	Amusements	218
	General Entertainment Resources	218
	Movies and Cinema	219
	Music	221
	Performances and Events	224
Chapter 16	Government Information and Data	227
	Directories and Locators	228
	General Government Resources	230
	Government Documents	231
	Government Officials	234
	Government Programs	235
	Politics, Policy, and International Relations	236
	Statistics	238
Chapter 17	Health and Medical Information	241
	Diseases and Conditions	242
	Images	246
	Healthcare and Medical Information	247
	Healthcare Professional Resources	249
	Locators	251
	Nutrition	253
	Patient Information and Consumer Resources	254
	Pharmaceutical Drugs	256
	Research	257
	Workplace Health and Safety	259
Chapter 18	U.S. and World History	261
	United States History	262
	World History	267
Chapter 19	Legal and Criminal Resources	271
	Attorney and Law Firm Locators	272
	Crime and Criminals	273
	Decisions	273
	Documents and Records	274
	General Legal Resources	274
	Intellectual Property	276
	Laws, Codes, and Treaties	278
Chapter 20	News and Current Events	283
	Audio	284
	Directories	284
	News Search Resources	285
	Video	288
Chapter 21	Searching for People	291
	Famous and Historical People	292
	Genealogy Resources	293
	Group and Affiliation Directories	294
	Online White Pages and Lookup Tools	297
	Veterans and Currently Serving Military	299
Chapter 22	Public Records	301
	General Public Records Resources	302
	Location-Specific Public Records	303
Chapter 23	Real-Time Information	311
	Environment	312
	Government	313
	Miscellaneous Tracking	314
	Space and Satellite	314
	Stock Quotes	315
	Transportation	315
	Weather	317
Chapter 24	Reference	319
	Associations	320
	Awards	321
	Books	322
	Calculators	323
	Consumer Resources	324
	Dictionaries, Glossaries, and Translation Resources	325
	Food and Beverages	327
	General Reference Resources	328
	Journals and Periodicals	331
	Library/Online Searching	332
	Locators	333
	Maps and Geography	335
	Sports	338
	Travel	339
	Weather	341
Chapter 25	Science	343
	Agriculture	344
	Biology	345
	Botany	347
	Chemistry	348
	Earth Sciences	350
	Energy	353
	Engineering	355
	Environment	356
	General Science Resources	361
	Mathematics and Physics	362
	Oceanography	362
	Research and Development	363
	Space and Astronomy	365
	Weather and Meteorology	367
Chapter 26	Social Sciences	369
	Anthropology	370
	Archaeology	370
	Demographics	371
	Development Resources	373
	General Resources	374
	Gender Studies and Data	375
	Latin America	376
	Military Resources	377
	Psychology	377
	Research and Development	378
	Religion	378
Chapter 27	Transportation	381
	Air	382
	Automobile	384
	General Transportation Resources	384
	Maritime	386
	Railroad	387
	Glossary	389
	References	397
	About the Authors	401
	Index	403