The Wikipedia Affair

There has been a great deal of discussion on the Internet about research into the usefulness/effectiveness of various NLP techniques.  Unfortunately most of the people doing the arguing seem to have a preset agenda and therefore tend to present a very distorted view of the situation.  For example, at the time when I wrote the first version of this FAQ the relevant page on Wikipedia (which was riddled with inaccuracies from start to finish), cited various references to studies which supposedly "proved" that "NLP doesn't work" or some similar claim.  Which seems to have led some NLPers to become a little concerned about whether NLP ought to be able to provide valid research in support of its various models and techniques
Although I wish those people well in their efforts, I strongly suspect that NLP, like psychology in general, is simply not a subject which can be rigorously tested along the lines of something as restrictive as the "scientific method.".

(For what it may be worth, it turned out that at least some of the "people" responsible for the nonsense on Wikipedia were, in fact, a single person using a dozen or more "sock puppets" (aliases).
How ironic, then, that the self-appointented "guardians" of Wikipedia were actually defending the sock puppets against a number of people who were trying to correct the deliberate misinformation!

Another Perspective

Taking a more objective view of the situation, so far as that is possible, the situation is actually split three ways:

1. There is a large body of second rate research which suffers from a lack of knowledge on the part of the experimenters and/or poor design of the experimental process and/or misinterpretation of the results.  Further research carried out since I wrote the first half of this bullet point has revealed that this applies to at least one of the best known summaries of the research carried out in the late 1970s and early 1980s, along with many of the research projects covered by the review.
    
2. A second kind of "experimental evidence" consists of studies which appear to have been carried out competently, but which nevertheless "fail to support" the claims made for various NLP techniques.
    
3. Lastly, there is some research on certain NLP techniques which has produced positive results; plus there are various "mainstream" studies which have no direct link to NLP but which seem to support the validity of certain concepts in NLP.
    
4. Most worrying of all is the discovery that certain very senior "scientists" have written widely quoted criticisms of NLP which, to anyone who actually bothers to read them, reveal that the writers haven't done any actual research at all, and have only a vague and distorted notion of what NLP and its associated techniques are about.

To put a little more flesh on those bones:

1. Much of the "scientific" research carried out on NLP techniques has been produced by undergraduates going for their M.A. or M.Sc. or at best (judging by the length of the theses, post-graduate researchers going for a Ph.D. - not by fully qualified, experienced researchers.  And even where the research has been the work of people who are, on paper at least, more qualified, this is unfortunately no guarantee that their experimental design and execution is valid.  Indeed, the evidence shows that in a number of cases the researchers simply haven't carried out the kind of basic groundwork needed to pave the way for an effective study.
    
   For example, the "eye accessing cues" model has been the subject of a great number os studies, yet most of the ones I've been able to get details of repeat a basic, crucial error.  That is to say, NLP-related techniques require that we calibrate a person's eye movements as they naturally occur in order to track which rep system(s) a person is currently using.
   The experimenters, on the other hand, routinely try to push their subjects into exhibiting relevant eye movements by asking a (usually disjointed) series of questions which seem, to the experimenters, to be appropriate for the purpose.

   In practise, the experimenters would be better off filming or videoing the subjects in conversation or being interviewed about some interest they have, and then reviewing the recording to see if what they say ties up with their eye movements.
   It's a simple enough format, and one you can try for yourself whilst watching an interview or chat show on television.
    
   For what appears to be a competently executed undergraduate study - where the results are invalidated because of the experimenter's lack of understanding of the technique he is investigating, see the Dillingham experiment which was carried out at the University of Central Florida.
    

For a detail examination of a study by a post-graduate researcher, which has been frequently quoted, and just as frequently misunderstood, see the Hogan experiment.
 
2. Whilst I appreciate the positive intentions of those people who cite various studies which have "failed to support" some NLP technique or other, the fact is that their conclusions are invariably downright inaccurate.
    
   
   • Firstly because so many critics seem unable to recognise that:
      
       (a) NLP itself is nothing more than a single modeling technique, and
      
       (b) even if it were possible to totally invalidate any particular NLP-related model (the "eye accessing cues" model, for example), this would in no way invalidate the field of NLP as a whole BECAUSE the various models and techniques are inter-related, but not inter-dependent).
      
     
   • Secondly, because genuine scientific investigation is not about proving that something is "true" or "false", but about establishing the statistical probability that the hypothesis under investigation is true or false.  That is to say, on the one hand it is certainly possible to accumulate so much evidence against a certain proposition that it would take a quite outlandish leap of faith to still suppose it was true.  On the other hand, no matter how negative the results of a given study, they can never completely invalidate the model or technique being tested - they can only show that it is highly unlikely that it is valid.
     The statement that a particular experiment, or set of experiments, "failed to support" a particular hypothesis is a far more accurate and honest description than claiming that the hypothesis has been "disproved".
      
     
   • The claim that "there is no scientific evidence for NLP" is frequently (usually?) meant to imply that some NLP technique or application has not been validated using the scientific method (see box below).  Whilst this may well be true, it presupposes that the dominant philosophy in modern science - materialist reductionism - (which asserts that nothing exists unless it can be reduced to basic atomic building blocks) is unassailably correct.  Only trouble is, we cannot establish the probability of the accuracy of the materialist reductionist philosophy using the scientific method.
     Oops!
      
     
   • Having said that, it is equally difficult to positively "prove" a particular hypothesis in psychology, and it would therefore be more accurate to say "this experiment supports the hypothesis" rather than "this experiment has proved the hypothesis to be true."
      
3. In the third group we have work such as the many studies by Professor Elizabeth Loftus on the plasticity of memory.  Professor Loftus makes no mention of NLP in those studies which I have read, and quite possibly has no idea that NLP even exists.  Nevertheless, her studies of "false memory syndrome" and how it occurs demonstrate that memories - one's internally stored personal history - can indeed be "rewritten" (or "reprogrammed") simply by bringing memories into conscious awareness, and that we already do this, to a greater or lesser extent, as part of everyday life.
    
   Even more direct support for NLP models and techniques can be found in psychological studies involving the so-called "Chameleon Effect", which relates to both non-verbal "pacing" (to use the NLP term): see AI Seduces Stanford Students; and in relation to both verbal and non-verbal pacing, matching predicates. and David Groves' "Clean Language": see Copycat waitresses get bigger tips.
    
   This kind of evidence does not prove that specific NLP-related techniques are valid, but they do establish an empirical basis for further investigation.

NLP-specific Research

And how this relates to NLP:

• The scientific method is applicable to situations where the elements involved are expected to behave consistently.  Hold a magnet over a pile of iron filings and if the magnet is close enought, some filings from the top of the pile move upwards and become "attached" to the surface of the magnet.  If this doesn't happen then something is definitely wrong.  Either the "iron" filings aren't really of a metal that responds to magnetism, or the magnet isn't really a magnet, or the magnet is still too far away from the filings to have an effect.
  What certainly won't be true is that the magnet isn't working because it had a row at home last night, or because it just can't be bothered, etc.
   
  
• In any kind of psychological testing, on the other hand, all sorts of variables come into play.
  Nowdays psychologists try to work with just two variables, known as the independent variable (IV) and the dependent variable (DV), the point being to see what happens to the DV when the experimenter tweaks the IV.
  For example, in another of Loftus' experiments the IV was a set of verbs decribing the meeting of two cars - "hit", "crash", "collide", etc.   The DV was the estimates of the speed the cars were travelling at when they came together, as given by five groups of subjects.  The nature of the experiment was to test whether the verb used to describe the event made any difference to how fast the subjects thought the cars were moving.
   
  However, in addition to the IV and DV there are also other variables which aren't under the experimenter's control, though they may affect the outcome of the experiment.  These are known as "confounding" variables (CVs).  It is this last group of variables which make life so difficult when trying to carry out genuinely "scientific" tests on human beings.
   
  
• Even if we could get stable results, testing psychological models isn't as straightforward as authentic scientific modelling.  When iron filings are "attracted" to a magnet the reason is basically because the magnet creates an "field" which influences alignment of the ferous filings.  (That's a pretty crude explanation, but it encompasses the basics.)
   
  When we hypnotize someone, on the other hand, even if we only induce a light trance, we really have no idea what actually happens.  We might say the person's focus of attention becomes constricted, but how are we actually doing that?  And why does that make them more suggestable?
  And applying that directly to NLP, does the "fast phobia technique" cause a phobia to disappear, or does it simply help a person to tap into their own resources?  Whatever that means.

Having said all that, some research has already been carried out on the use of NLP by people who genuinely understand NLP.  Some years back Richard Bolstad, a well-known NLP trainer and author from New Zealand, compiled a list of "successful" research into the use of various NLP-related techniques, and his results can be seen here:
 
http://www.stant-1.demon.co.uk/artcl007.htm
 
A similar list (which incorporates Bolstad's review) can be found here:
 
http://www.eanlpt.org/research.html
 
In both cases it would probably be most accurate to say that the findings demonstrate some support for the "usefulness" of certain NLP-associated techniques.

More recently the team at the Austrialian NLP training company Inspiritive have initiated an ongoing programme of research into the various NLP models and techniques following the principles of "best practice" for this kind of activity.
See this page on their website for details.

In the UK, The Society of Medical NLP are involved in two separate in-depth studies in the UK regarding the use of NLP in medical contexts.  The results of those studies will be released sometime in 2007.
UPDATE:   Garner Thomson and Dr Khalid Khan issued their first book Magic in Practice: Introducing Medical NLP, foreword by Richard Bandler, in January 2008.

And as of June, 2006. the University of Surrey, in the UK, has announced a project similar to the one in Australia, to be led by Paul Tosey, senior lecturer at the university, and Jane Mathison, who became only the second person in the world to be awarded a Ph.D based on a thesis which focused (positively) on NLP.
See: http://www.nlpconference.co.uk/Folder.2004-03-13.8939184105/nlpresearchproject

In the UK again, a team of NLPers carried out an educational project in association with Durham local authority to improve teaching methods and student performance.  A detailed full color report in pdf format is available here: Durham NLP Project 2006