Web Bot Technology

SOURCE : http://urbansurvival.com/simplebots.htm

In June 2001 I began to correspond with a reader of my website who said he was willing to share access to a promising new web technology, on the condition that I protect his identity. The person related that he had been a very senior programmer with a software company in the Pacific Northwest (you can guess which company, right?) and besides being a SQL ace, he was also heavily into linguistics and a language called Prolog, which is more like an artificial intelligence language than anything else.

I was skeptical, to be sure, but a few days after we began the email exchange of ideas, he sent me a program he had written that allows a computer to be turned into speed reading tool. It was based on rapidly displaying individual words on a computer screen. He said this was a technology that he had developed and sold for a while on the Internet. He also explained how the development rights to the technology had been sold to a company ( www.ebrainspeed.com ). In essence, after looking up the patent he held for the technology, I was convinced that this fellow was for real and might be on to something with the method of looking for linguistic shift on the Internet as a tool to forecast future events.

He described how technology worked. A system of spiders, agents, and wanderers travel the Internet, much like a search engine robot, and look for particular kinds of words. It targets discussion groups, translation sites, and places were regular people post a lot of text.

When a "target word" was found, or something that was lexically similar, the web bots take a small 2048 byte snip of surrounding text and send it to a central collection point. The collected data at times approached 100 GB sample sizes and we could have used terabytes. The collected data was then filtered, using at least 7-layers of linguistic processing in Prolog, which was then reduced to numbers and then a resultant series of scatter chart plots on multiple layers of Intellicad ( http://www.cadinfo.net/icad/icadhis.htm ). Viewed over a period of time, the scatter chart points tended to coalesce into highly concentrated areas. Each dot on the scatter chart might represent one word or several hundred.

To define meanings, words or groups of words have to be reduced to their essence. You know how lowest common denominators work in fractions, right? Well the process is like looking for least common denominators among groups of words.

The core of the technology therefore is to look at how the scatter chart points cluster - condensing into high "dot density" areas which we call "entities" and then dissolving or diffusing over time as the entities change. Do a drill down into a dot and you get a series of phrases...

Our first published work in the area occurred in early July of 2001 and is available at http://www.urbansurvival.com/tip.htm.

What becomes obvious when reading about the technology is that it sometimes reads a bit like the I Ching (the Chinese Book of Changes) because the technology doesn't come out and say "go look for a terrorist attack over there" What it does is gives phrases that would be associated with how people talk about an event, or more accurately, how they change their speech to reflect their thought processes because of an event (after).

The web bot technology apparently taps in to an area of preconscious awareness. It's here that you run into the ramifications of Dean Radin's work at the Boundary Institute and the work of the Princeton Global Consciousness Project - both of which Art has talked about on his show.

The Global Consciousness Project registered what appears to have been a disturbance in "the force" or the regularly orderly operation of life associated with 9/11: http://www.boundaryinstitute.org/randomness.htm. Supposed "random" numbers generated all over the world appeared to become less random immediately prior to 9/11.

The second point is contained in Dean Radin's paper at http://www.boundaryinstitute.org/articles/timereversed.pdf ("Time-reversed human experience: Experimental evidence and implications"). The mind-bending evidence in Radin's work is that in a laboratory, people begin to react to an event as early as 6-seconds before it takes place. In other words, if you are about to show someone a horribly grotesque picture of something, they will already be physically reacting to it before the picture actually becomes visible. Up to 6-seconds, or so, and in a lab! In quantum terms, Radin's work demonstrates that people are physically able to perceive 6-seconds into the future.

Now let's flip back to September of 2001: So there I was, having just completed a sales & marketing job in San Francisco the previous week, wondering was the ABM Missile test the "world changing event" - or was there something else - and we all know that it was something else - the 9/11 terrorist attack.

The second web bot posting forecast an attack on house or assembly - but again, it was I Ching-like in wording.

Also in this timeframe, we responded to a Defense Department Broad Area Announcement - and tried to distill the concepts we've been discussing into a single PowerPoint slide. The project didn't receive funding, although we recently received a note from a venture capital group associated with government projects asking if our information could be kept on file.

Then we forecast an attack to occur the day of a commemorative event. That was the crash of American 587, and although that may eventually be blamed on excessive movement of the aircraft's vertical stabilizer, what people thought at the time was it was another terrorist attack. http://www.urbansurvival.com/webbot2.htm., There is a fair amount of noise in the outputs because we haven't had the resource to build filtering systems.

We forecast many attributes of the D.C. sniper case. This was significant because not only did we get the Army connection right, but there was also discussion of familial dysfunction. This was at a time when it was thought there was only one person involved - and the web bots got right that there was more than one. In fact, one reader sent in the right Army divisional insignia after reading the output. See http://www.urbansurvival.com/bot4.htm.

More recently, in January of 2003 the web bots were going on and on about a "maritime disaster" - which didn't make any sense to us, until the Space Shuttle Columbia disaster hit. Columbia wasn't a gem of the ocean, it was a space ship. We think of web bot outputs like holding up a sheet of paper with several hundred pinholes in it - and trying to guess what's on the other side by looking through pinholes, but just for an instant.

The most recent run we did was posted in July 2003, and was a mission-specific run designed to assess where the next terrorist attack on the U.S. might come from. As you read the following, please appreciate that the words in black were published about 50 days before the northeast power outage and that when the outage occurred, again everyone thought it was a terrorist attack: The blue text is the closeness of fit after the fact:

The [Wahabi] entity is seen as preparing an attack on a site with aspects of (South) [Niagara power flows where?] with specific references to a place where the (wood) in the (earth) {grows} {upward [perfect fit with the Falls] }. Near by will be an (energy) {plant} [bingo!] filled with {tremendous} (effort/work) [Yup!]. The target chosen for the entity has associations with a (vertical wall/ascent) [more Falls refs] near which is a (low land) or (low lying area) [ the water flows this way] from which there rises (power) [no doubt about it] and (influence/confluence) [like the confluence of the disparate pieces of the river at the Falls?] . This latter may suggest a site where a power plant sits at the base of a cliff or naturally occurring wall [ You're kidding, can we be any more precise than this?} that has the power lines rising to it. Further, there is a form of a (departure) {to the south} close by which suggests some form of transportation hub [the river was a transportation artery historically] with a main focus to the south which lies close to the chosen target [which we assume to be NYC] . Other details suggest that near to the chosen target, is a place where (heaps) of {small things} (rise upward) to some height. The number of both aspects and attributes which related to wood, and woods, and copse, suggest that there is some form of natural woodland setting near to the target. [Yes, it's a generally woody area!] This woodland is near an area where (rivers) {congregate}[Direct hit!] and the (earth) {waters} (flow together) which suggests an estuarine environment [Clear enough?]. There are also references indicating a (waste) {flow} or disposal is nearby [Turbine waste gates?] . This (waste) aspect has the attributes of {dismal} and {abysmal} as well as attributes indicating a {great} (stink/smell) [At about 10PM EDT CBC News reported cities were dumping untreated sewage into rivers in the affected area due to the power outage!].

On Saturday morning after the event, at the suggestion of several readers who had read the entire web bot work, here are some additional points of fit (again in blue) with the black text being the July early July output.

Further, there are references to (empty city) nearby the target. [One reader suggested that an electrical switchyard, with rows and rows of transformers, looks like an "empty city".] Perhaps an area of industrial park which is expected to be deserted at the time of the attack, or could even reference abandoned buildings, though the attributes are not so clear as to be able to make a definitive interpretation. [Who would have interpreted a switch yard 2 months ago?] However, that said, there are a number of words referring to (brakes), and (brake noise) and (dead man switch) and (switches/changes) [Now we're back on track, except maybe we should have gotten "breaker" rather than "brakes" - still, close enough, don't you think?] which would indicate a near by train yard. [Oh yeah, they have cranes on tracks to move the heavy gear around generating stations.] As the aspects bring up noise of the (brakes) as being able to be heard from the site of the attack, this would suggest that a busy train yard is close enough to contribute to the majority of the on-site noise. [Or you can hear the falls as they "break" over the top...] There are also some small number of references which can be interpreted towards petroleum refineries.

In addition to the (empty city) we find a number of aspects relating to (royalty) such as (king) and (courtier) which have attributes which doubly modify both these aspects as well as a primary aspect of (Mount) as in mountain or large hill. [ No question here, Niagara is the "king of falls" and it's a named place with a named "hill". I need to also mention that King's Point Ontario is very nearby.] Being linked to the aspects of (royalty) suggests that there is a named hill involved with the site which has a local connotation with (kings). There are also indicators that a (community) {project} is nearby which is associated with (social status). [This is really curious - as most people go to Niagara Falls to celebrate their honeymoons - a big change in "social status", right?] This (community) {project} has an association with (rhythmic motion) and a {centered} (wall). [Or, centered around the falls, as it turns out.] This latter could be referencing a tidal or sea wall, though this is merely a guess at this stage. [Amazing call, huh?] There are some references suggesting that the chosen target site is very noisy. [Yeah I'd call the site noisy too...]

Further, the entity suggests that there are (steps) involved with the attack, though these are not steps as in stairs but rather are to be interpreted as a many small movements or actions which will be necessary for the attack to be successful. This further suggests that the [Wahabi] entity is planning a significant action which will involve some level of coordination with many parts. [Time will tell on this part.] The other references within this data set further amplify this idea in that the aspects return to the idea of (step by step), (small incremental movements) {bring success}. Oddly, a large component of these aspects include other aspects relating to (drunkenness) or (inebriated). These are modified and amplified by references to (not sober), (stay sober), and (advancing intoxication). This data set includes attributes indicating some need to avoid speed or haste in that the attributes include {step by step}, {steady progress}, (do not) {overleap}.

A series of curious references to (blind) and (darkness) also are part of the attack plan as seen by the model. [Darkness? Like the power going out???!!!] These references also include aspects of (blindly deluded), (tricked), (exhaustion), (blind impulse), (fear of darkness), (leap) {into the dark} and other aspects relating to lack of sight. [Again, absolutely correct: on Thursday evening NYC was basically blinded, tricked and exhausted by the event!]

The sample size for this run was unusually small - and that may have helped get the fit so good. It's hard telling.

Where does the technology stand today? Both Cliff and I are not able to pursue the project further because of personal and economic reasons. The personal reason is that to sit down and work through even the distilled data is a terribly taxing process on Cliff. Picture working on word riddles for a week of 14-hour days... it's very difficult to do. Especially when there's no "right answer" therefore everything has to be checked and rechecked.

Then there's the cost. Bot runs are one-off programming events. So you can spend a month or so building the mission and deploying it. Once deployed you need serious bandwidth - T-1 or better to collect the data and then you need lots of server space to run the Prolog on. This type of processing is not easily adaptable to shared computing like SETI - and developing that would take time anyway.

We also know that the Chinese have a similar project - because we have swept their source code during our runs. The difference is they are doing larger data samples.

We apologize that our data is not as neat and as well preserved as one might hope. Part of that is because I've been through two computers since we started this, and Cliff has been through a couple of ISP's...some of the early files were lost, although most of the outputs were logged in to a dot mil computer so timestamps are available - and some of the information was sent upstream within the military, although we were probably looked at as a couple of software "wingnuts".

Do we have a theory of how it works? Sort of. It's like changes in language precede large emotional events. The larger the emotional impact of an event, the more advance notice the bots seem to give.

If you picture some pin holes in a piece of paper - and you imagine being able to look through each pinhole for a fraction of a second - with the objective of seeing the future on the other side of the paper - that's where the web bots are today.

Ever since Plato's Allegory of the Cave, people have sensed that odd things go on at the archetype level of consciousness. The web bots are an attempt to use the high data density of the internet to sample language and seek linguistic shifts that we believe may precede events. The initial results suggest that language shifts on a macro level begin to occur 45 to 90 days before society-changing events. We believe we've demonstrated, most recently with the Northeast Power Outage forecast, that changes in language do indeed precede events - on a far grander scale than Dean Radin's work first suggested - and these language changes are available by sampling routine internet traffic.