Every day, a significant amount of my time is spent working on a computer. And, of all the things we use PCs for, storing and retrieving data still forms a critical component of our work, no matter what the nature of work actually is. What amazes me is that, in spite of years of advances in the field of personal computing, no one has bothered to come up with a better way to search-and-retrieve the data we store on our machines !

When you think about it, human beings don’t think in terms of categories and sub-categories. Instead, each thought triggers another one associated with it. That’s the foundation of how we remember every thing we do and are able to recall it at will. But a typical File Management software (like Windows Explorer) confines us to an un-natural way of processing data, requiring us to store every thing with specific filenames in specific locations. The result : When you want something, you can’t always remember where you put it.

To my surprise, I discovered that the exact same problem was stated way back in 1945 ! Vannevar Bush, in an article published in the Atlantic Monthly, spoke of how the growing volume of scientific work is going to need a better of way of storing and retrieving information.

According to Bush :

Our ineptitude in getting at the record is largely caused by the artificiality of systems of indexing. When data of any sort are placed in storage, they are filed alphabetically or numerically, and information is found (when it is) by tracing it down from subclass to subclass. It can be in only one place, unless duplicates are used; one has to have rules as to which path will locate it, and the rules are cumbersome. Having found one item, moreover, one has to emerge from the system and re-enter on a new path.

Of course, at that time, the PC wasn’t invented and Bush envisaged a mechanical machine – something he called a “Memex” – complete with levers and motors, that would hopefully solve the problem. Unfortunately, in spite of processor power (and storage needs) increasing in leaps and bounds, very little progress has been made in this area even today.

I can no longer comprehend a life without Search Engines like Google, and I am pretty sure the “Internet” would not have happened if it weren’t for these indexing utilities to help us sift through the tons and tons of data poured out on the web every day. The information overload we experience on an individual level is not very different in nature…

May be the answer is a personal search engine for our personal computers… a Google-like alternative that would search the entire “database” of files and folders so that we wouldn’t *have* to remember where we put things in the first place ! But such a system would not be able to re-create paths that we followed to record (or search for) specific topics of information.

What is needed is a more flexible way of doing this : A system that allows us to store multiple links to all the data that is stored in one giant location. This data can be either created by us, or sourced from elsewhere as we scoop it along our search on the Internet. We could then look up the same file in many different ways by searching on key words that were included at the time of storage.

Such a system would also allow us to “bind” disparate scraps of information whether they exist as Word documents, Excel sheets, Powerpoint presentations, simple Text files or URLs to websites on the Internet. It will also allow us to look up that “trace” at a future point in time, and recall every thing we did to get there, making it truly meaningful and human-friendly. Wouldn’t it be great if this were possible?

In the meanwhile, I’m going crazy trying to look for that proposal we’d sent to the client last year… Was it in the Pursuits directory or the Proposals one? May be it was in the Clients directory…

I hope some body invents the Memex soon !