DiSC is a good and valid tool for understanding and adapting work-related behaviors
The DiSC construct was originally developed by William Marston, who proposed ideas about emotions that deviated from conventional wisdom about emotions gathered by biologists and psychologists at that time. Today, his views on emotions and his claim of the existence of a psychon are simply ignored.
Why did he use the ‘four primary colors’ in his model? Did he believe in synesthesia, the phenomenon where stimulation of one sensory pathway (e.g. color vision) leads to automatic experiences in other sensory pathways? And why do his color choices deviate so much from other models that use colors, such as Discovery Insights (based on Jung)? If we use Ockham’s razor, the simplest explanation is that the use of colors in any of these models or questionnaires is probably just BS.
Key words
Wonder Woman, dominance-submission, inducement-compliance, (primary) colors, yellow = cowardly
What is the DiSC about?
It is an assessment tool based on the ideas of William Moulton Marston (1893-1947). Walter Clarke was allegedly the first person to construct a questionnaire based on the ideas of Marston. He used a checklist of adjectives and asked people to rate themselves. Later the test was used in a forced choice (ipsative) format.
Today, most proponents of the DiSC model claim it captures four constructs:
● Dominance (D)
● Influence (I) (the original name for this construct was Inducement)
● Steadiness (S) (the original name for this construct was Submission)
● Conscientiousness (C) (the original name for this construct was Compliance)
There are several questionnaires that allegedly measure the DiSC.
● DiSC Classic
● DiSC Indra (Indra stands for IN-Depth Relationship Assessment)
● DiSC Persolog
● Puzzle DISC
● Everything DiSC
● Everything DiSC for Managers
● Everything DiSC Work of Leaders
● Everything DiSC 363 for Leaders
● The Thomas Test
● Performax Personal Profile System
● DISCUS
● PAL-Topas
Some of these instruments, such as the former DiSC Indra and the Everything Disc, have divided the four quadrants into up to 18 subscales.
I won’t discuss each questionnaire separately, however. By now, the reader will know that before we look at the psychometric properties, we must first look at the theoretical soundness.
■ Executive Summary Theory
Anyone with good observation skills can see that human interactions sometimes consist of dominant and submissive behaviors. Inducement and compliance are human behaviors too. But is that enough to capture human personality or behavior and frame it within a four- quadrant model? William Marston considers emotions such as anger, fear, and sadness to be irrelevant (he referred to them as Victorian artifacts). His claim that emotions follow rather than precede behavior is entirely false. This completely denies findings in both the biological and psychological sciences. The idea that ‘primary emotions’ can be linked to ‘primary colors’ is also a confabulation. There is no logical reasoning nor empirical findings to support this claim. The DiSC Indra refers to Jungian Typology and, as can be read elsewhere, Jung’s theory is entirely impossible and has been refuted. Some proponents claim that the interpersonal circumplex is one of the theoretical foundations, but this is preposterous, as the DiSC was first described in 1928, whereas the interpersonal circumplex was described in 1957. You simply cannot base your theory on a theory that was developed some 30 years later.
William Marston is not a well-respected psychologist, but a well-respected cartoonist (Wonder Woman). In fact, his work in the field of psychology has never been taken seriously as demonstrated by the total absence of references to his work in the scientific literature.
Empirical Data
There is absolutely no evidence that the theory is valid. Quite the contrary, current theory and evidence demonstrate that emotions precede behavior (though our behavioral impulses, as a result of our emotions, can also be inhibited or regulated by another part of our brain, an area in the frontal lobe).
The fact that one can perform a statistical analysis pointing to four factors doesn’t prove the existence of an underlying four-quadrant structure in the brain. Moreover, the statistical methods employed are flawed, not in the least because techniques are used on ipsative or ipsatized scores, creating artificial independencies and dichotomies.
The theoretical/empirical grid
Psychometric properties
William Clarke was probably the first person to build a DiSC questionnaire based on Marston’s model, though different providers have their own version of the DiSC questionnaire. This means I had to look at the available data from many sources. The Buros Institute gave a damning judgment of the DiSC Indra. The major problem, however, is the use of ipsative (DISCUS) or artificially ipsatized scores (Everything DiSC). For the Everything DiSC questionnaires, a kind of ipsatization procedure is used by transforming normative scores (which allow comparisons between individuals) into ipsatized scores. This is done by calculating a mean and a standard deviation for each respondent, and then the mean is subtracted for each item and the difference is divided by the standard deviation. The researchers I consulted about this procedure are unison: this is not a permissible procedure , contrary to what the authors claim (p. 32 of the Wiley ‘Everything DiSC manual). Researchers have demonstrated why covariance techniques such as MultiDimensional Scaling cannot be used on artificially ipsatized scores—but this is exactly what the authors of the manual did. The psychometric methodologies are very obscure and heavily criticized in the literature.
I also found evidence that the DiSC persolog and the Everything DiSC do not measure the same ‘C’ construct, although they are based on the same theory.
Ultimately, the question remains: what does the DiSC really measure? For example, the authors of the Everything DiSC manual can’t seem to make up their minds: does it capture tendencies, priorities, personality, interpersonal dynamics, or behavior? Given that both the Everything DiSC and the DISCUS measures show positive correlations with two of the 5 or 6 factor model of personality—namely agreeableness and extraversion—we could be very lenient and say that they capture, to some extent, 2 of the 6 personality traits.
Conclusion
It is quite strange that people—especially HR people—don’t take the time to investigate the central ideas behind the DiSC, such as the primary emotions and primary colors. The total absence in peer-reviewed academic literature is a strong indication that the underlying theory is wrong and a dead end. The body of experts doesn’t support the theory at all. The only way to refute this fact is to believe in a global conspiracy against Marston’s idea. As far as understanding interpersonal relationships, psychological research has produced much better theories that are supported by evidence, such as the 5FM and 6FM (personality traits), and the interpersonal circumplex (interpersonal motives and interpersonal behavior). Why wouldn’t you use valid measures if they are available? These measures are constructed to allow comparisons between individuals, something which is no longer possible if ipsative questions (e.g. DISCUS) or artificial transformations into ipsative scores are used (e.g. Everything DiSC).
Moral Assessment
I find it immoral to teach people an entirely wrong theory. What the questionnaires actually measure is far from clear. If the intention is to measure personality, then the 5FM or 6FM questionnaires should be used instead. If the intention is to measure interpersonal behavior, then the well-established questionnaires based on the interpersonal circumplex should be used. They are more reliable and accurate, and thus allow us to act with more care and prudence towards employees.
Discussion
■ Theoretical soundness
The foundations for all the DiSC models and questionnaires harken back to the book Emotions of Normal People by psychologist William Moulton Marston (1928). The book is free of copyright and can be consulted or downloaded at the following link: https://archive. org/details/emotionsofnormal032195mbp
Contrary to the consensus of my Champions League of biologists and psychologists, Marston did not accept that anger, fear, sadness, and joy should be considered basic (or primary) emotions. He believed ‘fear’ and ‘anger’ reflected the terminology used in the ‘Victorian era’ (1928, p. 3), and that that these emotions were mere responses to a stimulus that was much more ‘elementary.’ His first proposition was that there were ‘four basic stimuli’ and ‘four basic types of emotional response’ (1927b). His second proposition was that ‘primary colors’ were associated with these basic types of responses: The ‘superior’ dominance-submission axis was related to blue (dominance) and yellow (submission). The ‘inferior’ compliance-inducement axis was related to green (compliance) and red (inducement). He also thought women preferred yellow and men preferred a ‘bright, saturated blue.’ He considered this preference to be ‘obvious,’ as males were much more dominant and submission would make them ‘yellow’ cowards—whereas women who displayed submission were merely considered weak as “women live by emotional standards peculiar unto themselves” (1927b, p. 16).
Marston was clear on the origins of his color theory: he referred to the Hering Colour Theory, which ‘supposes’ the existence of four primary colors. He simply accepted that supposition without evidence, and supposed himself that the colors could not be a real physical “primaries,” though he accepted that they were “psychological primaries”: “It is clear, of course, that these four hues must be psychological and not physical primaries” (p 4). He pointed to the existence of four color zones in our retina. He also proposed that a unit
called a psychon47 would produce energy, resulting in “the conscious energy constituting colour sensations.” His 1927(b) paper is built entirely upon assumptions and speculation. To illustrate this, the word ‘might’ is used 42 times in 29 pages.
The combination of his first and second propositions resulted in the following four stimuli and responses (pages references for the 1928 book):
● Dominance (blue): he considered this the stimulus that gives rise to a ‘rage’ response. This rage response releases additional ‘motor energy’ resulting in dominant and even aggressive behavior. He considered this behavior an ‘obvious’ response to “overcome or to dominate environmental opposition ” (p. 347) . In short, “ a dominant stimulus brings forth a dominant response”48 (p. 351). He believed the ‘dominance emotion’ is the product of past difficulties and is a conditioned response (p. 131). He never referred to the modern concept of emotions (anger, fear) nor accepted that they could be the product of an innate human nature—in his view, every response had to be learned.
● Inducement (red): Marston views inducement as another ‘love’ reaction to the same stimulus that leads to submission. But inducement is a ‘more active’ love reaction, for example, a child that holds out its hands to be tickled by its mother. He considered inducement to be a primary emotional response type that seeks to induce submissive responses from another individual.
● Submission (yellow): Marston considered this a ‘passive love’ reaction. This leads to “a diminution of the tonic motor discharge”—i.e. the muscles relax (p. 356). In case you hadn’t noticed the pattern yet, Marston concludes that “a simple submissive stimulus evokes a basic emotional submission response” (p. 385). Marston believed that women are aware of the power of submission, whereas men think of it as weak.
● Compliance (green):49 Marston relates this to a ‘fear’ response that results in behavior such as crying or urinating (children) or fleeing from the scene (adults). He defined compliance as “control of tonic motor discharge reinforcement by a phasic reflex” (p. 350). Of course, Marston again believes that a compliant stimulus will lead to a compliant response.
The four initials of these terms form the acronym DISC. However, the careful reader will notice that the current assessment tools no longer use Marston’s initial terms. In the ‘Everything DiSC’ technical manual we can find no explanation as to why, with only the following stated: “Marston identified what he called four ‘primary emotions’ and associated behavioral responses, which today we know as Dominance (D), Influence (i), Steadiness (S), and Conscientiousness (C)” (p. 1). I can only speculate about their reasoning, but most likely the old descriptions and original explanations were not very attractive to a business environment—the ‘love’ reactions of inducement and submission were unlikely to have been accepted by managers… Nevertheless, the descriptions of influence and steadiness are still very similar to the descriptions Marston used, meaning only superficial cosmetic changes have been made. The description of conscientiousness is different, however. The manual describes conscientiousness as “private, analytical, and logical” (p. 2). We also read
47 He thought psychons were “junctional tissues or connective nerve filaments.”
48 Marston used some circular reasoning. His 1927 paper surely cannot be considered scientific, as it was not based on empirical research but wild speculation. He often referred to the work of Watson and then writes of himself “I have reached the conclusion…”
49 This is very strange since, in his 1927 paper, Marston writes that “first and foremost, a very strong association between ‘nature’ and the colour green” exists and that therefore it seemed plausible to him that green provokes “a naïve emotional feeling of ease, relaxation, harmony and mild pervasive pleasantness and well-being associated with green sensation.” Therefore, he thought it possible that ‘there exists some innate, psychoneural connection between green sensation and compliance emotion” (p. 23) The description of the fear response is in contrast to this feeling of relaxation.
that conscientiousness is defined differently from the mainstream definition used in the 5FM and 6FM (see elsewhere). If we perform an analysis of the adjectives, it seems the concept of conscientiousness refers to introversion: private, quiet, reserved, interpersonally restrained (pages 2 and 7), mixed with aspects of reflection and logic. This is a classical confusion: people would like to believe that introversion equals reflection and logic. But this is incorrect and a conflation of two distinct concepts. Let’s see the definitions used by the HEXACO questionnaire (a 6FM measure):
● The dimension of introversion-extraversion: Extraversion can be defined as the “tendencies to become engaged in social endeavours (such as socializing, leading or entertaining).” Introverted people or people with low scores “Consider themselves unpopular, feel awkward when the centre of social attention, are indifferent to social activities, and feel less lively and optimistic than others do.” Extraverted individuals or people with high scores “Feel positively about themselves, feel confident when leading or addressing groups of people, enjoy social gatherings and interactions, and experience positive feelings of enthusiasm and energy” (taken from the sample report of the HPS360, 2015, p. 11). No trace of reflection or logic is mentioned.
● The dimension of conscientiousness: this is defined as “ tendencies to become engaged in task-related endeavors (such as working, planning, and organizing).” People with low scores “ tend to be unconcerned with orderly surroundings or schedules, avoid difficult tasks or challenging goals, are satisfied with work that contains some errors, and make decisions on impulse or with little reflection,” whereas people with high scores “Organise time and physical surroundings, work in a disciplined way toward goals, strive for accuracy and perfection in their tasks, and deliberate carefully when making decisions” (taken from the sample report 2015, p. 34)
The only instance in which the Everything Disc manual refers to the modern concept of conscientiousness is when it refers to systematic and accurate. According to the manual, conscientiousness encompasses three conceptual elements: being analytical, being precise, and being private. It is clear they are mixing up constructs, something that is unacceptable in modern psychology. A construct should be distinct from others to allow for accurate description and measurement. The authors of the manual themselves acknowledge conscientiousness is defined in an entirely different way than the current (accepted) definitions as used in the 5FM and the 6FM.
In the DISCUS questionnaire, the “C” doesn’t represent Compliance nor Conscientiousness, but ‘Carefulness,’ which is described as “withdrawn, controlling and conscientious.” Once again, same mix-up of unrelated concepts.
Some distributors also refer to other theoretical underpinnings. For example, the Everything Disc Manual also refers to the interpersonal circumplex research tradition (pp. 2 and 8-10). This is strange, given that Marston died in 1947 and the first publications about the interpersonal circumplex didn’t appear until 1954 (Guttman, 1954). Timothy Leary—the psychologist who is considered to be the founding father of the interpersonal circumplex tradition—published his first book on the research of the Kaiser Foundation Psychology Research Project in 1957. He made no reference whatsoever to William Marston. I contacted the members of the SITAR organization (the Society for Interpersonal Theory and Research) and received confirmation there is no reference to Marston. In fact, he is never referenced in relation to the circumplex. So, we can rule out that his ‘two-dimensional model’ is the same as a circumplex, and the DISC cannot have a common basis with the interpersonal circumplex. The comparison with the well-accepted interpersonal circumplex is unwarranted as no proof is offered that the DiSC questionnaires are based on the two interpersonal dimensions of agency and communion. To substantiate this claim, they would have
to comply with the current standards of an interpersonal circumplex. The DiSC Indra, the predecessor of the Everything DiSC, also referred to Carl Gustav Jung’s type theory. This theory is discussed elsewhere in this book. Whenever models refer to theories that are not in line with each other, critical questions should be raised. A questionnaire must measure one theoretical construct, not three, as the Everything Disc claims to do (Marston’s theory about basic emotions, the Interpersonal Circumplex, and Jungian typology).
The authors of the manual also make other comparisons at face validity without offering proof that their assumptions are valid (pp. 2-3). They compare the two main dimensions of the DISC to several other constructs. For example, the authors of the manual say the vertical axis represents fast-paced (top) versus moderate-paced (bottom)50 and can be compared to the constructs of surgency ( N orma, 1963) and potency ( G oldberg, 1981) . The horizontal axis ranges from skeptical (left) to accepting (right) and can be compared to the construct of agreeableness (McCrae & Costa, 2010) , thus referring to one of the five or six trait domains of personality (see the chapter on the 5FM and 6FM). The authors of the Everything DiSC manual report no experiments or other empirical research to back up their assumptions.
The curious case of… Wonder Woman’s lie detector
At the origins of the DISC (or DiSC, whatever51) is Marston. William Marston received his PhD in psychology in 1921 (long before psychology made any efforts to rely on the scientific methods we know today) and is often credited as the creator of the systolic blood pressure test, which was one of the components of a ‘lie detector.’ The lie detector is a highly untrustworthy tool for detecting the truth, as prominent scholars in modern psychology have demonstrated.52 However, he abandoned the field of psychology in 1941 when he channeled his obsession with women into a famous comic book character known as Wonder Woman . 53 He expressed his obsession with bondage themes by frequently illustrating Wonder Woman and the other characters in tied up positions. Michael Schein reminds us that Marston “wrote extensively on the role polyamory and bondage would play in the liberation of humankind.”54 Jill Lepore, a professor of American History of Harvard University even wrote a book on Marston, confirming his obsession with polyamory and bondage. 55 Marston also gave wonder Woman a golden lasso that produced truth just like the polygraph (the lie detector).
Marston seems to have been obsessed with sexual intercourse and this is reflected in the two axes of the DISC. He devoted large sections of his book to writing about sexual interactions between married men and women. He claimed that human sexual relationships could be understood through the two pairs of ‘basic emotions’: the dominance-submission axis and the compliance-inducement axis. He thought that women were submissive most of the time during sexual intercourse and believed that this was problematic. He cited a certain Dr. H.W. Long (1919) who advised women to be ‘on top’ of their husbands to “initiate all the movements of both parties” (1928, p. 335). He also refers to behaviorist John Watson who interpreted that stroking of the skin, tickling, gently rocking, and patting resulted in ‘love reactions in an infant.’ Marston quickly concluded (without evidence) that these responses
50 As the manual does not specify what “fast-paced” and “moderate paced” refer to, I am afraid I can’t clarify this.
51 Why it is written with a small ‘i’ instead of a capital letter, as with the others, is a mystery to me. Maybe it seems more exotic?
52 You can read more in the book 50 Great Myths of Popular Psychology by Lilienfeld, Lynn, Ruscio, and Beyerstein, four expert scholars in psychology.
53 Some sources claim his wife Elizabeth came up with the idea to create a female character.
54 “Positive Psychology is Garbage (And Why You Should Follow Its Founder’s Lead,” Michael Schein on Forbes.com, July 5, 2018.
55 The Secret History of Wonder Woman, Jill Lepore, 2015, pp. 119-120.
could be categorized into two classes: reactions of submission (voluntary yielding) and reactions of inducement, or active solicitation or stimulation.
We can also say that he had a strange view of the mechanism of fear. He again referred to John Watson, who became infamous for his (unethical) experiments on a small child. The case is known as the “ Little Albert experiment ,” a single case experiment that has many methodological flaws by modern standards. Watson claimed he had conditioned the oneyear old ‘Albert’ by producing a loud sound when the child played with a rat. One conclusion was that older children rapidly crawl away from an unpleasant stimulus, such as an extremely loud sound. Marston, however, had an alternative interpretation: he believed this was evidence of ‘compliance’ rather than a fear response followed by avoidance behavior..56 He also believed fear mustn’t necessarily be felt when being chased by a bear. He was convinced that it isn’t the bear chasing you that causes fear, but the realization that you cannot run fast enough. So, according to Marston, the bear is not an unconditioned stimulus (modern biologists and psychologists consider predator animals to be unconditioned stimuli: we have an innate fear of them) but a conditioned stimulus that is ‘strong enough’ to create a “discharge into appropriate running-away muscles” . He also believed that dominance can only be combated by dominance, so if the person fleeing the bear can release dominant energy, he or she can add this dominant energy to the ‘running reactions.’ Of course, this defies all current knowledge of the maximum running speed of humans and bears, which is limited by our biological makeup. Marston’s confusion of cause (stimulus), path (e.g. the stimulus travels from the thalamus to the amygdala and the visual cortex), and effect (avoidance behavior) can only be seen as a hilarious mistake nowadays. Fleeing behavior preceding the fear response is simply impossible.57 There is a large consensus that we have many emotions. Anger, fear, sadness, and joy surely are some of them. One of the most cited references is the work of Paul Ekman, who researched emotions and their accompanying facial expressions across multiple cultures. This resulted in the idea that there is a common human nature for emotions (what is now known as the Darwin-Tomkins set of nine affects58) and their facial expressions. They are present at birth in all cultures:
● interest-excitement
● enjoyment-joy
● surprise-startle
● distress-anguish
● anger-rage
● fear-terror
● shame-humiliation
● disgust
● dissmell
In the theoretical considerations, I will explain how Marston offered an alternative explanation for our emotions and how he linked his alternative emotions to basic colors. He was also the ‘inventor’ of the circular representation of the DiSC, including the colors (1928, p. 104). He also made an effort to give the field of psychology its own ‘elementary matter unit,’ much like protons in physics, atoms and molecules in chemistry, and neurons in neurol-
56 Not only was this already the view at the time, it is still considered to be the case by virtually all biologists and psychologists. Fear provokes freezing, avoidance, and fleeing of fighting responses in both humans and non-human animals.
57 An excellent book on the subject matter is The Emotional Brain: The Mysterious Underpinnings of Emotional Life by brain researcher Joseph Ledoux.
58 Link: http://www.tomkins.org/what-tomkins-said/introduction/nine-affects-present-at-birthcombine-to-form-emotion-mood-and-personality/
ogy. In that vein, he introduced the psychon as the elementary unit in psychology. He proposed that the psychon produced energy and formed the basis for our consciousness. Indeed, he speculated that the principal function of the psychon was to produce consciousness (in Emotions of Normal People, 1928, p. 52).
What else is wrong with this stuf?
It is entirely unclear what the DiSC actually seeks to measure. The authors make several, often incompatible claims. For each claim, there is a more valid and reliable alternative. For example, Everything DiSC claims to measure an individual’s tendencies and priorities. On the same page, we also read that the “ultimate goal of Everything DiSC is to take wisdom about interpersonal dynamics ” ( E verything D i SC M anual, p. 1) . But on page 99, we read that the ‘Work’ tool can help people better understand their personality and their colleagues and provide proscriptions for improving their relationships. The manual promises the same advantages for sales people, managers, and leaders.
For example, on many occasions the authors of Everything DiSC claim that their model is a circumplex, or even an interpersonal circumplex. But where researchers in the true interpersonal research tradition show the relative proportion of each behavior, and ‘summarize’ the ‘interpersonal style’ by calculating a vector, the Everything DiSC profiles do not show the relative proportion of the different behaviors, rather plotting the ‘DiSC style’ with a dot and a shaded area. In my opinion, this results in an artificial representation of people’s style.
Figures above III.3: the DiSC style represented with a dot and shading (graph on the right) is used “to help participants understand where they might have difficulty stretching” (note: the non-shaded, white area). Adapted from the original by the author. The original cannot be reproduced under the principle of fair use due to the creative work that was put into the model by the authors. Original drawings and color scheme can be found in the brochure “How My Graph Became a Dot.” In this representation, the colors deviate from Marston’s color scheme.
Figure III.4 above right: the true interpersonal circumplex tradition shows the actual relative use of each interpersonal behavior scale. Most of the time, the raw scores on each scale are displayed, as is the case for this questionnaire developed by Danny Rouckhout and Rik Schacht (current name of the questionnaire: CBS360). The arrow is a vector summarizing the tendency relating to the vertical axis (agency) and the horizontal axis (communion). The vector gives an indication of the ‘average’ interpersonal style, in this case ‘friendly and rather assertive.’
However, the claim that the interpersonal circumplex was one of the theoretical foundations is preposterous, as the DiSC was first described in 1928, whereas the interpersonal circumplex wasn’t first described by Guttman until 1954. You simply cannot base your theory on a theory that was developed 26 years later.
Why do people believe DiSC can ofer them valuable insights?
I guess we have to look at the usual suspects: The combination of our credulous human nature and our somewhat lazy tendency to believe authority arguments (‘Dr. William Marston, a psychologist’), especially if we are not trained psychologists or aren’t trained in critical thinking. Combine that with nice colors (probably no one suspects they might have a ‘deeper’ meaning) that help remind us of the model and intense marketing efforts by a large company like Wiley,59 and you will understand the renewed popularity of this model.
What does my Champions League of experts say?
There isn’t an ounce of doubt: they don’t agree with Marston’s theory about basic emotions, nor do they believe in psychons as elementary units. The idea that ‘every response’ is learned and not due to our human nature has been debunked many times, including by Steven Pinker in his book The Blank Slate (2002) or David Buss in his handbook Evolutionary Psychology (2005).
What does the majority of the feld of experts think?
The exact same thing as the champion’s league. They don’t take it seriously.
The theoretical score: -2, given the theory isn’t taken seriously by the overwhelming majority of scholars. The theory starkly contradicts the current theory about emotions, in both the fields of biology and psychology. In the academic literature, Marston’s ideas have never been adopted, as the lack of citation of Marston’s work demonstrates. Regarding personality traits, there are much better models to capture them, such as the 5FM or the 6FM.
59 A big publishing company with headquarters in Hoboken, New Jersey, USA
■ Empirical evidence
What is the level of evidence?
Marston’s theory about emotions is in total contradiction to current thinking about emotions. Therefore, it came as no surprise to me that the most important psychological database, APA’s PsycNet,60 features only two peer-reviewed articles referring to his work on DISC. One is a critical historical review by Geoffrey Bunn (1997), who doesn’t take Marston’s work seriously; the second is a Spanish article reviewing the life and work of Marston.
When I consulted Google Scholar, 61 the seminal article has been cited 11 times. Only 3 peer-reviewed articles have referred to the paper. Kimball Young (1927) only refers to Marston as an author who developed an alternative theory on emotions, while Malan et al. (2014) only describes how the DISC is sometimes used in hospitality environments (without any reference to reliability or validity). The third article is Geoffrey Bunn’s (1997) So, it seems there is a deafening silence with regard to Marston’s work on emotions, which is evidence that his work has never been taken seriously by academic scholars. It should be seen as a surprise that the DISC model has been adopted by commercial firms and found its way into corporate life.
Needless to say, there is no evidence that ‘primary emotions’ are linked to ‘primary colors.’ Marston even believed that these primary colors could explain the origins of culture too. For example, he believed that yellow was the primary color of the Chinese and that their culture was built on the ‘emotional response of submission.’
Moreover, most DISC models don’t use the original colors Marston proposed. For example, the Wiley Everything Disc uses green instead of blue for dominance, yellow instead of green for conscientiousness/compliance and blue instead of yellow for stability/submission, though it maintains red for influence/inducement. Regarding the terminology, as mentioned before, it seems that the labels were merely renamed because the original names were inconvenient for business purposes. I can only guess that is also why the original reference to love and sexual relationships has been omitted, and why the colors are no longer used or explained according to Marston’s version.
However, the most important argument is that the consensus among several research fields (biology, clinical psychology, and I/O psychology) reveals that emotions such as fear, anger, and sadness are not considered Victorian artifacts, but are accepted to be cognitions that lead to typical (behavioral) responses.
Regarding the other claims about the links to the interpersonal circumplex or Jungian Typology, it seems that some distributors want to cast such a wide theoretical net for the sole purpose of impressing us with name dropping. If Jungian Typology really is related to the DiSC, the only difference in the conclusion would be that two pseudoscientific theories are used.
How likely is it this theory will ever prove to be valid?
Not a chance. Modern science is incremental, not disruptive, as some people want to make us believe. The knowledge of emotions is vast and solid. The people from Everything DiSC write they “maintain some of the core principles” that Marston used, but they fail to mention which ones. They also say their model incorporates “many additions and changes that are
60 Search date: April 13, 2016.
61 Search date: April 13, 2016
informed by advances in psychological measurement and theory” (manual, p. 185), again without pointing out which ones. This vague description cannot hide the absence of a good description of the theoretical constructs, its problematic structural validity, or the lack of clarity regarding what they are actually trying to measure (preferences, personality, interpersonal behavior… they claim it all, even though these theoretical constructs differ greatly).
The empirical score -2, considering the evidence contradicts Marston’s specific view of ‘basic emotions’ and his denial of fear, anger, and sadness as true basic emotions. Marston launched hypotheses about human consciousness and ‘basic emotions’ that have never been taken seriously and thus have not been investigated further. Nor has his idea linking ‘primary colors’ to ‘primary emotions’ been taken seriously. Some of the (many) DiSC questionnaires show quite acceptable psychometric properties (a good circular MDS structure, good internal consistencies, and good test-retest reliabilities). This is again a demonstration that it is possible to build a questionnaire with good psychometric properties even if the theory is wrong. That’s why psychometric properties tell us (almost) nothing about the theoretical soundness and the evidence for the theory.
Psychometric properties
Again, the most important remark I wish to repeat is that we shouldn’t even have to look at psychometric properties if we know the theory is a dead end or wrong. This is especially true if the evidence for the theory is also negative. But I’ve still made the effort to peruse the very extensive manuals62 and do my work thoroughly. If you wish to stop reading here, I fully understand… I must admit you won’t miss anything, other than maybe learning something about the flawed use of statistics.
In assessing the psychometric properties, I couldn’t rely on peer-reviewed scientific literature because, as I mentioned before, there is a total absence of published, independent peer research in the most important databases. I have already made this same point, but I want to repeat my warning: the high number of retractions of academic papers reported on a website like RetractionWatch is a clear indication of how much malpractice exists, even among academic researchers. Quite often data is simply fabricated. Only because of the peer-review process and elevated competition resulting in criticism are such problems revealed (although sometimes after many years). When these public publications and peer reviews are lacking, we have no clue as to how reliable commercial data and reports may be and whether or not they are fabricated. There is simply no way to check their claims. In light of the many lies I have discovered and the research on people’s tendency to be dishonest,63I am always very skeptical of in-house manuals…
In this case, the only independent reviews I could find were two reviews by the Buros Institute Reviews . And the reviews are simply negative. 4 reviewers 64 have reviewed the DiSC Indra model, which was an instrument allegedly developed and based on Marston’s theory, the subsequent model developed by Walter Clarke (1956), and the Personal Profile System created by John Geier (1979). The Buros reviews are critical of several aspects:
62 For example, the Everything Disc Manual is 247 pages long—the limited electronic research manual contains 51 pages.
63 See for example in Dan Ariely’s book The (Honest) Truth about Dishonesty.
64 C W Conoley, Professor of Educational Psychology; L Castillo, Assistant Professor of Educational Psychology, Texas A&M University; G Denzine, Associate Professor of Educational Psychology, Northern Arizona University; and F.T L Leong, Professor of Psychology, University of Tennessee (Note: positions at the time of publication).
“the technical manual lacks details regarding the criteria the test developers used in their decision-making processes in terms of item and scale development”
“no empirical investigations are cited on the relations between personality measures and DiSC Indra profiles.”
“Overall, the technical manual provides incomplete and superficial treatment of important test development information.”
“Moreover, I found no published articles on the psychometric properties of the DiSC Indra in any professional journals.”
“No reliability data are provided in test-retest form. Of great concern is the fact that researchers using the DiSC Indra would not have the ability to compute reliability coefficients for their samples because the raw data are not available.”
“Although the authors claim the DiSC Indra is to help individuals in any type of relationship to reduce conflict and increase effectiveness, and for understanding about their relationship dynamics, no studies providing evidence of predictive validity are reported.”
“So, the current DiSC Indra model is a measure of interpersonal behavior that is actually quite similar to Timothy Leary’s (1957) work in terms of the well-known interpersonal circle.” .../…” it does not provide any predictive validity data or any data suggesting how the DiSC Indra provides incremental validity or added usefulness above and beyond the measures of the interpersonal circumplex to which it is highly similar in theory and structure.”
“It actually overlaps so much with the interpersonal circumplex that there are questions about whether there is a need for a separate instrument to measure these interpersonal dimensions.”
It should be noted that the Everything DiSC measures are the next generation of the DiSC Indra. Wiley has bought Inscape Publishing and has now created a separate website for the “Everything DiSC” ‘brand.’ The first version of the Everything DiSC was created in 2008 and consisted of 79 adjectives. The Everything DiSC manual even mentions that some research was conducted on samples of the DiSC Indra data (p. 19). This means we must keep the criticism of the Buros Institute in the back of our mind. For the current Everything DiSC questionnaires, we must rely on the extra information provided by the distributors themselves. In the Everything DiSC Manual, we can read: “The Everything DiSC assessment uses the circle, or circumplex, as illustrated in Figure 1.1, as an intuitive way to represent this model” (p. 2) The problem is that a true circumplex is a statistical model (like a simplex and radex) that must comply with several statistical criteria, especially in the case of an interpersonal circumplex (stating something about the relationship between people, hence inter-personal). Fabrigar, Visser & Browne (1997) outlined the criteria for a true circumplex (see next paragraph). Describing the DiSC as an ‘intuitive way’ is not very scientific, is it?
A circumplex or not? Please make up your mind. The authors of the Everything Disc Manual claim that their DiSC questionnaires are interpersonal in nature. Moreover, the authors state: “The circumplex structure of the assessment conforms well to expectations, as assessed by multidimensional scaling,65 scale intercorrela-
65 For more explanation of MDS or Multidimensional Scaling, see the chapter on the interpersonal circumplex in Part V
65 For more explanation of MDS or Multidimensional Scaling, see the chapter on the interpersonal circumplex in Part V
tions, and factor analysis” (p. 23 of the electronic manual). This would imply that the DISC is a direct competitor of the well-established interpersonal circumplex research tradition, on which many valid, reliable, and peer-reviewed questionnaires are based.66 Although Figure 2.1 in the Everything Disc manual displays a nice circular structure for the DiSC Indra questionnaire (p. 19), I could not find conclusive evidence that it complies with the criteria for an interpersonal circumplex:
● In an interpersonal circumplex, the two main dimensions must be identifiable as agency and communion. These two main dimensions need to be perpendicular (90° angle)—I found no evidence in the manual that the two main dimensions of the DiSC model are agency and communion.
● The scales should have equal spacing. Mathematically, the midpoint of each of these octants must be at 45 degrees of the midpoint of the adjacent octants. From a scientific point of view, all eight vectors in the interpersonal circumplex must be as close as possible to 45 degrees from each other (this seems to be OK if we look at the MDS plot in Figure 4.2 on page 43, but it is unclear from which questionnaire the adjectives stem (see below for the 363 Leader Model).
● The scales should have equal vector lengths measured from the center of the circle (for the Everything DiSC Assessment67 this is OK, but not for the 363 Leader model).
● Opposite scales (or octants, or poles) in the circle must show a negative correlation, but adjacent scales need to show a positive correlation (the correlations must become smaller the further the scales are from each other in the circle).
66 For an overview, see the chapter on the interpersonal circumplex in Part V 67 I guess this is their basic questionnaire version. daring stubborn blunt impatient frank bold frm direct decisive pioneering expressive energetic spontaneous high-spirited talkative outgong lively enthusiastic open warm optimistic sociable playful cheerful generous receptive tactful accepting obliging serene calm gentle patient lenient humble compliant conforming modest soft-spoken self-controlled restrained careful cautious quiet silent methodical unexpressive private solitary prudent systematic unemotional perfectionistic reserved introverted eventempered agreeable trusting neighborly easy-going light-hearted accommodating full-of-live animated enterprising compelling dynamic persistent assertive stern resolute strict restless forceful critical analytical serious matter-of-fact cynical skeptical demanding aggressive dominant outspoken adventurous competitive questioning
Figure 2.1 (p. 19) in the Everything DiSC manual shows a good circular structure of adjectives, although it should be noted that data from three different questionnaires are used (DiSC Classic, DiSC Indra, and Everything DiSC). Such a mixture of data was already criticized in the Buros Institute review. Mixing data is not acceptable, particularly if the same scoring format is not used in each questionnaire.
Figure 4.2 as published on page 43 in the Everything DiSC manual. Note that the spacing among scales is almost 45° and show almost equal vector lengths from the midpoint of the circle.
So perhaps it is a circumplex based on another pair of axes? Or maybe it doesn’t use the two interpersonal dimensions of agency and communion? On page 10, the authors explain that the “dominance” position would be in the exact same position as “arrogance” is in the upper-left quadrant of the interpersonal circumplex (see figure below), thus forming the orthogonal axis known as “ arrogance-agreeableness ” which links to the 5FM model’s agreeableness dimension. Because they claim the DiSC has the nature of a circumplex as well, it must consist of two perpendicular axes. They identify the first as the dominancesubmission axis, which is the same as arrogance-agreeableness.
68 Because of the first requirement of a circumplex—the two axes must be perpendicular to each other—it logically follows that the other axis must be the extraversion-introversion axis, or in DISC terms, the influence-conscientiousness axis. If the dimensions are truly the same, then these dimensions are merely two dimensions from the 5FM and the 6FM. So, either the DiSC only measures two traits of the 5FM or 6FM (extraversion and agreeableness), and has given a problematic label to at least conscientiousness/compliance, or it should be viewed as an alternative circumplex, but not an interpersonal circumplex. In either case, the DISC should comply with the standards of a true circumplex, and this is nowhere reported in the manual. Face validity is just not enough proof.
Figure III.5 : The two interpersonal dimensions from the 5FM, extraversion and agreeableness, ‘lie’ on the two diagonal or orthogonal axes of the circumplex. Source: Rouckhout & Schacht (2008) N-IAS Manual, p. 17. Note: the original Dutch words used in the research for this figure have been translated to English for the reader’s reference. For English readers: this has been independently replicated in research conducted by Costa & McCrae (2011, p. 93)
68 Indeed, they use ‘submission’ (the old denomination from Marston) instead of ‘steadiness’!
Strange internal consistency fgures
The authors report good internal consistency (Cronbach’s Alpha ranging from .79 to .90) and good test-retest reliabilities (.86 to .88) for the Everything DiSC Assessment, although the respondents took the test for the second time after only two weeks. The web center for Social Research Methods (www.socialresearchmethods.net) states: “The amount of time allowed between measures is critical. We know that if we measure the same thing twice that the correlation between the two observations will depend in part by how much time elapses between the two measurement occasions. The shorter the time gap, the higher the correlation; the longer the time gap, the lower the correlation.”69
However, a psychometrician from the University of Antwerp drew my attention to something strange. It seems that the good internal consistency of the scales is not always the case for all respondents. On average, the internal consistency is problematic in 24% of the respondents (they are euphemistically labelled as ‘unclear’ respondents). From viewing a more detailed table from another brief research report, we learn that none of the 8 scales reached the minimum standard70of .70 Cronbach’s alpha (table 3, p. 11) with those ‘unclear’ respondents, as they were called. The lowest scale (D) has a Cronbach’s alpha of only .34, while the highest only reaches .64 (S and SC) in those respondents. Of course, .34 is unacceptably low as there is a broad consensus71 that alphas below 0.50 are unacceptable (e.g. George & Mallery, 1997; Taylor & Kuyatt, 1994). We should already be throwing the questionnaire in the trash can.
To solve this problem, the authors of the manual report that they automatically increase the number of items. If the software notices that some respondents don’t respond ‘consistently,’ their adaptive testing system adds extra items: “extra, adaptive questions that some respondents receive if there is a large amount of variation within their responses.” To reach alpha levels above the .70 minimum standard, we read that on average 24% of the respondents receive a variable number of extra items on a given scale. For the C-scale however, 35% of the respondents needed 8 more items (19 instead of 11). And for the CD-scale, an increase from 12 to 22 items resulted in only .74. It is neither possible nor good practice to add items ad infinitum: how can they ever compare all the people if they have to calculate norms? It is simply impossible to calculate norms for all possible variants..72
The biggest problem, however, is that they artificially increase the reliability by increasing the number of items in the scales. To reach acceptable levels of internal consistency, they add 5 items in 6 scales, and even a staggering 8 items (the scale goes from 11 to 19 items) and 10 items (the scales goes from 12 to 22 items) (reported in Table 3 on page 12 in the 2013 Research Report for Adaptive Testing Assessment). The authors of the manual state that the internal reliability is increased, but they don’t attribute this to the simple lengthening of the scales. They are quite confident that they actually improved the true reliability of the scale by do-
69 Consulted on February 19, 2019.
70 The authors themselves accept this standard: “Alpha values above .70 are generally considered acceptable and satisfactory.”
71 In most scientific domains (thus also in psychology), the table produced by Wikipedia is generally accepted: https://en.wikipedia.org/wiki/Cronbach%27s_alpha. Whether you do a search on Google scholar, or via PsycNet (American Psychological association), most papers refer to this consensus, which is a convention, not an empirical fact.
72 Together with Prof. Dr. Reinout de Vries, we calculated how many norms one would need to allow people not to respond to items in an 8-scale questionnaire. In this case the following formula would result in 324, 518,553,658,427,000,000,000,000,000,000 different norms: (2^k1-1)*(2^k2-1)*(2^k3-1)*…*(2^k8-1). a^b means a to the power of b; k1 to k8 = the number of octants—8 in this case. k1-1 = the number of items in a scale. Even if 4 items are omitted, you would need 7,420,490 different norms. Who wants to do the math for adding items?
ing this: “By comparing these two columns, we can see the internal consistency is much higher for these unclear respondents when they receive the extra items. In essence, these extra items are used to further gauge the target trait when the normal assessment has produced unclear or variable results” (2013, p. 11). Bad practice, bad science.
The authors don’t seem to realize that they are contradicting themselves: Page 21 gives a set of 7 recommendations, the last of which reads: “Scale length should be kept to a minimum. Scales should contain items necessary to demonstrate evidence of strong reliability and validity, but additional items should be used sparingly.” Indeed, there is a large consensus among psychometricians that you should have a sufficient number of questions, but you shouldn’t exaggerate either. If you add items that are very similar to the other items, you end up increasing your mathematical internal consistency or internal reliability (Cronbach’s alpha). Using the Spearman-Brown formula, you can calculate how much you will increase your statistical reliability. But such an increase in internal consistency by adding similar items doesn’t increase your true reliability (as defined in the dictionary). Our conclusion must be that this is an artificial increase that cannot hide the problems with the ‘normal’ version.
A true interpersonal circumplex or not?
I took a look at another of the questionnaires—the Everything DiSC 363 for Leaders Model. It is clear that William Marston did not conceptualize the DiSC with the purpose of assessing leaders, nor did Walter Clarke (who created the first questionnaire based on Marston’s theory).
Page 134 of the Everything Disc Manual presents a nice circular structure for the Everything DISC 363 for Leaders Model (hereafter ED363). Unfortunately (but not unexpectedly), I asked Wiley for permission to use this figure but was denied.73 The question is whether the nice circular structure is artificial or not. To begin with, the labels on the outskirts of the circle in the figure are called ‘approaches.’ What an ‘approach’ means is not further specified in the manual. The labels inside the circle are called ‘practices’—the term ‘practice’ is again not further defined. In the electronic manual, we read that the authors consider the DISC to reflect ‘how people approach their work.’ As I said, the authors go to great lengths to compare the ED363 with the interpersonal circumplex, maybe because this model is very popular in some countries (e.g. Australia). These are the facts:
● In the development stage of the questionnaire, people answered a 7-point Likert score, which is a normative score.
● Each ‘approach’ contains three practices.
● Each of the 24 practices are measured with three items.
● The manual reports very high internal reliability coefficients, ranging from 0.93 to 0.96 (all raters).
● The manual reports several MDS solutions, one of the techniques for checking a circumplex structure.
So, the question is: is the DiSC a true interpersonal circumplex? Let’s use the criteria from the true circumplex tradition to check this. Bolded text represents the true circumplex criterions:
73 There are, however, several web sites where you can find the graph yourself: www.corexcel.com/pdf/ disc-363-leaders-validation-report.pdf ; http://www.mypotential.ie/introducing-everything-disc-363-for-leaders/ ; www.mypotential.ie/ introducing-everything-disc-363-for-leaders/ and https://a360c.com/everything-disc-363-for-leaders/
1. Can we identify the two axes of agency and communion in the model? The manual doesn’t report whether this was checked. To detect these two factors, a factor analysis can be conducted to gain an initial idea as to whether the content relates to agency and communion. Next we must conduct a convergent validity study with a well-established measure (such as the IAS-R by Wiggins). The interpersonal circumplex convention dictates that Agency is posited at the 90° angle (the vertical axis). This model doesn’t show where agency is positioned, so we can only try to evaluate this at face validity: I cannot discern agency in this model, as only very active behaviors (promoting the self, competitive, results oriented) should be found at one pole of the agency dimension, while very passive interpersonal features and behavior should appear at the other pole. For example, ‘being humble’ in a true interpersonal circumplex is in the fourth quadrant, very close to the bottom of the vertical axis. In this model, ‘showing modesty’ is very close to that position, but so is ‘maintaining composure,’ an emotional feature that figures in another octant (3) in the interpersonal circumplex of emotions (e.g. Russel, 1997; Altarriba et al., 2003). Other discrepancies arise when I analyze the content of this model with the content of the styles in the CLS360, the peer-reviewed tool that measures the Leadership Circumplex.74 In the CLS360, the items in the inspirational style are related to goal setting, enthusiasm, courage, and offering clear explanations, which are similar to two practices of the ‘energizing approach’ in the DISC: ‘showing enthusiasm’ and ‘rallying people to achieve goals.’ But inspirational style also covers ‘practices’ such as ‘communicating with clarity,’ located at the opposite pole of ‘energizing’ in the ED363, something that is simply impossible in a true interpersonal circumplex. In the CLS360, ‘showing confidence’ is also included in the inspirational style (octant 1). In relation to communion, I find the same problems: in the interpersonal circumplex, communion is represented as the horizontal axis: the right side expresses social, warm behavior such as giving help and advice and listening (octant 2), as well as empathy and providing autonomy (octant 3). However, if we consider that the orthogonal axis between inclusive and affirming represents the right social side of communion, then we should find asocial and aggressive behavior on the other side. Instead, we find ‘practices’ such as focusing on results, setting high expectations, and taking charge, which are all agentic and not representative of the opposite pole of communion.
2. Are agency and communion perpendicular in the model ? As I explained in point one, no they are not. The authors have made no effort to demonstrate this.
3. Are the adjacent octants positively correlated? Based on Table 6.24 (p. 143) pro-vided in the manual, 6 out of the 8 octants of the DiSC show high correlations: they range between 0.78 and 0.85. This is unusually high, pointing to a big overlap. In a true circumplex, the target loading 75 of an adjacent scale is 0.707. On the other hand, the correlation between commanding and resolute is below that mark (0.56) and the correlation between energizing and affirming is far too low (0.34).
4. Are the opposite octants negatively correlated ? Mathematically, opposite poles should show a negative correlation close to -1.0. The coefficients presented in Table
74 See Part IV for a brief discussion, or my extensive book on the Leadership Circumplex: Around Leadership. Bridging the Scientist-Practitioner Gap. Disclosure statement: I contributed to the development of this questionnaire by helping write the items and commenting on the research conducted by Marleen Redeker.
75 This is the ‘predicted’ correlation. Predicted is a misleading word, as this is a strictly mathematical calculation. In a true circumplex, octant 1 should mathematically show a correlation of 0.707 with octants 2 and 8. Then the magnitude of the loadings should gradually decrease until they reach a negative loading of -1.0 at the opposite pole. Therefore, octant 1 should not relate to octant 3 (0.0), should negatively relate (-.0707) to octant 4, and should exhibit the biggest negative relation with octant 5 (-1.0).
6.24 show us this is absolutely not the case and is very problematic, as the following table shows (remember the target loading is -1.0):
5. Do the scales have the same vector length from the center of the circle? Based on the ‘approach’ scales, this is relatively true, as the scales almost show a nice circular structure (Figure 6.14). Based on the Practice scales, however, this is problematic. I thought that this MDS plot was missing for the practice scale, so I asked Danny Rouckhout of Antwerp University to run the MDS solution to test the structure. Here is the result:
Common Space
Figure III.6: the MDS solution that Danny Rouckhout ran on the normative data presented in Table 6.27 of the Everything DISC 363 for Leaders. As you can see, this shows a problematic structure: the nice circular structure is nowhere to be found in a two-dimensional solution. The image looks rather like a Zeppelin. FO = finding opportunities scale/practice; SB = stretching the boundaries; PBA = promoting bold action; SE = showing enthusiasm; BPN = building professional networks; RPAE = rallying people to achieve goals; BA = being approachable; AC = acknowledging contributions; CPE = creating a positive environment; SOI = staying open to input; SD = showing diplomacy; FD = facilitating dialogue; MC = maintaining composure; SM = showing modesty; BF = being fair-minded; CC = communicating with clarity; PDA = promoting disciplined analysis; PSS = providing a sense of stability; SHE = setting high expectations; SPAP = speaking up about problems; IM = improving methods; SC = showing confidence; TC = taking charge; FR = focusing on results.
Upon further inspection, I realized that our MDS solution was the same as Figure 6.15 in the Everything Disc Manual (although it had rotated, which doesn’t change the structure as rotation is arbitrary). Take a look for yourself at the figure. Is this really a circle? Do the dots appear at the same distance from the center of the circle? Are they equally spaced? 76 The answer is loud and clear: no. Despite that, the authors of the Everything Disc Manual claim it is indeed a circumplex (p. 142) : “ Third, clusters are spaced at roughly equal distances around the circle, although not perfectly equidistant.”
76 If each practice really followed the model as proposed in the ED363, the mathematically calculated distance from one to another would be 15 degrees. A circle has 360 degrees and there are 24 practices, so the calculation is as follows: 360 ÷ 24 = 15.
In my view, the conclusion is inevitable: this is not a true interpersonal circumplex It is not even a circumplex because opposite scales are not negatively correlated around -1.0.
The authors include a table (6.27) where they also show “ipsatized scales.” Let’s now look at what happens if you use the “ipsatized scales” from their Table 6.27.
Figure III.7: the MDS solution Danny Rouckhout produced using the ipsatized intercorrelations presented in Table 6.27 from the Everything DISC Manual. As you can see, this shows a nice circular structure in a two-dimensional solution. The Legend for these codes is the same as for the previous figure.
Isn’t it strange that the authors themselves didn’t produce this nicer circular structure? It would surely have convinced people who are only impressed by the sheer volume of the manual, tables, and graphs. How did Danny perform this ‘magical’ transformation from a blurry, non-circular picture to an almost perfect circumplex structure? He used the ipsatized intercorrelations from Table 6.27. How did this ipsatization occur? The following is what the manual mentions on page 142: “ To control for overall leadership ability, scale scores77 were ipsatized for each leader. That is, each leader’s total average rating was calculated along with his or her total standard deviation78 across all items. The total mean was subtracted from individual item means, and then this number was divided by the total standard deviation. Ipsatized scale means were then calculated by summing ipsatized item scores for each scale. As a result, each leader’s mean ipsatized scale score across all scales was zero.”
Nick Brown pointed out to me that if you make the effort to look closely at the two figures, you will notice a change in the order, or distances, of the ‘practices.’ For example, SHE and SC are furthest to the right in dimension 1. In the nice circular plot, FR and TC are farther to
77 This is the procedure used for the 8 ‘approach’ or style scales. The procedure to ipsatize the ‘practices’ subscales is not explained.
78 The standard deviation is a measure indicating how spread out the bell-curve of a normal distribution is. If the bell-curve is narrow and steep, then the standard deviation is small; the more spread out the data— i.e. the bell-curve is wide and gradually increases and decreases, like the image we conjure when thinking of a bell-curve or a Gaussian distribution—the larger the standard deviation. For example, if you have a measure where the midpoint of the distribution is 70, and the standard deviation is 3, then 68% of the data will fall within the range of 67 (70-3) and 73 (70+3).
the right than SHE. We can see the same on the left side: in the first plot, MC is by far furthest to the left in dimension 1. In the circular plot 2, practices BA and FD have about the same position (which represents a very large shift for FD, as you can see), and two items (SOI and SD) are now even further to the left. This means that whatever process was performed ultimately changed the order of the items in a nonlinear way.
Is this kind of artificial ipsatization79 an acceptable procedure?
Assuming the authors were truthful in their description of the procedure on page 142, the question is, did they perform an acceptable procedure, or did they allow themselves too much freedom in manipulating the data? Let me first refer back to the Belbin chapter, where I explained the main problems with ipsative scores. Ipsative scores are either obtained by using rank-ordering or forced-choice items. They result in a dependence between the different scale scores, as the total sum of scores is constrained. The sum of all scores of a person is always constant, which is not the case in a normative scoring format (you can give higher or lower scores for each item, often ranging between five or seven points, resulting in a variability of scores and different sums of all scores).
In this case, normative scores (a 7-point Likert score) are transformed into ipsative scores afterwards. To make things even more complex, it seems that a number of procedures exist to do this.
On page 32 of the manual, the procedure I described above is outlined, but the authors claim that “this type of transformation is common in instruments that measure circumplex properties because all items in a circumplex assessment are balanced by conceptually opposite items (Locke, 2000, Solds, Dudbman, Demby, & Merry, 1993; Wiggins, Steigert, & Gaelick, 1981).” At first sight I was impressed, because Locke and Wiggins in particular are well-known experts on the interpersonal circumplex. When I consulted the literature, I found that some experts in the interpersonal circumplex (e.g. Gurtman, 1994; Horowitz et al., 1988; Tracey, 2000) accept a procedure that they labeled ‘ipsatization’ only if there is a common, general, or overall factor. Such a general factor that is shared by all four poles80 can result in a shift in correlations. So, what those researchers do is ‘ignore’ the overall factor, and only plot the scales of the other two factors. I contacted another important contributor to the circumplex tradition, Kenneth Locke, who confirmed that “ within the interpersonal circumplex tradition, ‘ipsatizing’ does refer to centering a respondent’s scores around his/ her average score.” He also added that it does not refer to dividing by the standard deviation, as in the procedure described for the ED363. He referred me to articles by DeYoung (2013), Erickson (2009), Fournier (2009), Gurtman (2009), Ojanen (2005), Russel (2007), Wright et al. (2009; 2013), and Wu (2015) describing the procedure to omit the general factor. I contacted SITAR members with the same question and I received additional reactions from Thane Erickson, Terence Tracey and Aidan Wright. Thane Erickson agreed that there is no obvious reason to divide items by the SD. Meanwhile, Terence Tracey’s papers demonstrate that you should carefully investigate alternative explanations for the existence of a general factor: it could reflect (1) real substance (e.g. a distress factor in the IIP), (2) mere nuisance (e.g. a pattern of scoring that reflects liking everything or disliking everything), or (3) bias (e.g. the general factor reflects systematic variance in the scores which are unrelated to the construct) (Tracey, 2012; Tracey et al., 1996).
Indeed, the procedure these researchers use is called centering or standardizing around the personal mean. If the existence of an additional, general (response elevation) factor is
79 Closs (1996) calls the procedure to transform non-ipsative Likert-type scores into ipsative scores “artificial ipsatization of scores.”
80 The two opposite poles of agency and the two opposite poles of communion.
demonstrated and explainable, then this procedure is accepted. A general (response elevation) factor means that each item loads on this factor. This is often the case with questionnaires that deal with emotions (such as the circumplex questionnaires of emotions or the Inventory of Interpersonal Problems). Scales measuring anxiety or depression, for example, typically have this general factor. Most of the time researchers create a general scale for the general factor and a sub-scale for the other factors. So, let’s take a look at two conditions: first, was the procedure described by the DiSC manual authors the same as the procedure allowed by interpersonal circumplex researchers and, second, do the authors demonstrate and explain the general factor?
Let’s address the second question first: is the general factor demonstrated and explained? The authors of the Everything Disc manual give the following as their reason for ipsatizing the scores: “the ipsatized means remove the variance due to a general factor (presumably overall leadership ability).” Did you notice the word “presumably”? And, why would it reflect ‘overall leadership ability’? It might as well be a general factor of social desirability. Of course, it isn’t unthinkable that they may have just decided to ipsatize because the MDS of the ipsatized intercorrelations display a nicer circular structure. I tried to solicit more information from Mark Scullard, who answered “there are substantial correlations between all of the Approach Scales and the questions ‘He/she is a good leader’ and ‘He/she is respected as a good leader’. When a first factor is exacted from unrotated factor analysis of the Practice Scales, this factor correlates .88 and .87 with the questions above, respectively. The other factors show correlations well below .10.”81 I was not convinced by this reasoning, but when I requested additional information about why he made the presumption regarding ‘leadership ability’ based on these two questions, and asked for the ‘good reasons,’ I received the following answer two days later: “Because of his schedule and his leading role in our product development team, Mark is unfortunately unable to provide additional responses, or he would not have time to focus on his priorities!”
Of course, this brief response is not very helpful. Especially given that the two questions above are virtually identical. No other hypotheses were tested (e.g. it is not a general factor, the ‘general factor’ reflects nuisance or bias, etc.) but ‘overall leadership’ ability is assumed. Thus, the general factor is neither convincingly demonstrated nor explained.
Regarding the ‘allowed procedure,’ it is obvious the authors of the E verything Disc manual deviated from this procedure this is what they have written: “The total mean was subtracted from individual item means, and then this number was divided by the total standard deviation. Ipsatized scale means were then calculated by summing ipsatized item scores for each scale.” Kenneth Locke expressed the following concerns in relation to the procedure in personal correspondence with me which he has allowed me to reproduce here (March 26, 2018):
“One concern is that they did not simply ipsatize (by subtracting the respondent’s mean score), they standardized the scores within person (by also dividing the score the standard deviation of the respondent’s score across items). I cannot think of a good reason do that second step (dividing by the standard deviation), because it completely levels theoretically meaningful differences between respondents who show very tiny differences across the different styles versus respondents who show very large differences across the different styles. For example, imagine Leader A reports about the same level of every style, and reports a tiny 0.2 difference between “showing modesty” and “showing confidence”. Meanwhile, Leader B reports very different levels of different styles and reports a large
81 Personal correspondence on April 17, 2018.
2.0 (i.e., 10 times larger) difference between “showing modesty” and “showing confidence”. After standardizing the scores within person, these two leaders—who are likely to look very different in reality—may end up with identical scores.”
“But the broader concern is, as you point out, it is unclear what model the scales map onto. The purpose of organizing scales into a circumplex is that the circumplex shows how every scale is meaningful related to every other scale and to theoretically meaningful axes (such as agency and communion), and allows you to combine octants into quadrants or bipolar scales or a single overall vector. When I look at the Everything DISC model, I am completely confused as to what the combinations of octants would tell us. For example, “showing modesty” and “promoting disciplined analysis” are one octant apart, but it’s unclear what they share in common; likewise, “improving methods” and “being approachable” are opposites, but it’s unclear why. Thus, it is unclear what the respondent’s single overall vector would be summarizing about that person.”
It is therefore safe to conclude that the Everything Disc manual authors deviated from this accepted procedure of centering around the mean. But did they lie, or did they misread the literature? Remember the criticism of many academics I referred to in the introductory chapter: psychologists often give themselves a lot of freedom to choose statistical techniques. In this case the authors ‘presumed,’ ‘suspected,’ or ‘assumed’ a general factor, and therefore used a different ipsatization procedure. The result is a blurry picture transformed into a clear circular structure with a lot of nonlinear changes to the positions of the original raw nominal scores. How could such a change in position occur? I contacted Julia Moeller from Yale University, as she had written a rather technical paper on the effects of ipsatization. She wrote to me82 stating that ipsatization doesn’t always makes sense for getting rid of the ‘common general factor.’ If people genuinely have broad interests, ipsatizing their scores on a vocational interest questionnaire doesn’t make sense: “It makes people who are interested in everything look exactly the same as people interested in nothing.” In the case of the DiSC, by removing the assumed general factor of leadership ability, both able and unable leaders might as well get the same score. She explained to me what changes through ipsatization:
1. “The first information lost through ipsatization is whether an item was affirmed or rejected by an individual. The absolute level (response scale) gets lost. Where typically the response scale midpoint is the original line where ‘high/affirming’ and ‘low/rejecting’ scores can be distinguished, ipsatization creates a new baseline: the person’s mean score across all items 83
2. Dividing ipsatized scores by the intra-individual standard deviation84 also changes the distances between items. For example, imagine a set of 10 items/responses, a re-sponse scale ranging from 1 (not at all) to 5 (very much), a person’s mean score of 3, and a standard deviation of 1.58113883 (the person’s 10 responses to could have been 1,5,2,4,1,5,2,4,3,3). In this example, the difference between the first response (“1”) and the second response (“5”) is 4 units, while the difference between the eighth response (“4”) and the ninth response (“3”) is 1 unit. However, after subtracting the person’s mean score from each item and dividing it by the person’s standard deviation, the dif-ference between the first response (“-1.341640786”) and the second response (“1.341640786”) is -2.683281573, while the difference between the eighth response (“0.670820393”) and the ninth response (“0”) is 0.670820393.
82 Personal correspondence on May 12, 2018.
83 This is the first calculation made by the creators of the ED363
84 This is the second calculation made for the ED363. This resulted in a shift in the ‘order’ or ‘distances’ as Nick Brown had already noticed.
3. The logic you describe entangles intra-and inter-individual criteria/reference frames of ranking scores on a high-low logic. The ipsatization/intra-individual standardization defines a score to be high if it is above the person’s mean score. The inter-individual correlation defines a score to be high if it is higher than other people’s scores in that sample. The combination of both approaches is really difficult to interpret, because it entangles both reference frames for deciding what is a “high score”. Neither the ipsatization nor the correlation reveal whether an item scored “high” in terms of the original response scale / item affirmation, which is often forgotten.
4. The covariance matrix may also change, see Cornwell and Dunlap, 1994; Closs, 1996; and Chan, 2003.”
(Bold emphasis my own: this is to highlight each time a piece of the procedure was used by the authors of ED363)
She also confirmed the following problems that the researchers I consulted and I suspected:
● “I agree that the “second step (division by the standard deviation) usually is not part of ipsatization and I don’t see a particular reason to do it.”
● “Ipsatization changes the covariance matrix in a way that makes the data unsuitable for correlational techniques like exploratory and confirmatory factor analysis, structural equation modeling, and multivariate techniques like multiple regression and multivariate analysis of variance ( C ornwell and D unlap, 199 4 ; C loss, 199 6 ; C han, 200 3 ) . Multidimensional scaling would be one of those covariance-based techniques that might be affected by ipsatization.” (Bold emphasis my own)
I eagerly consulted the papers Dr. Moeller referred to: Closs used a national sample of 2,808 college students questioned about occupational interests, and found several interesting things: a non-ipsative ‘like-dislike’ matrix showed that two interests were positively correlated (+0.28), whereas the ipsative matrix showed a negative correlation (-0.44), and of course “both cannot be true” (p. 43). He also noted that the ipsative matrix gives “rise to spurious bipolar factors.” The procedure used by Scullard and Baum might result in an artificial bipolar structure of the transformed scores in the ‘ipsatized’ version of the ED363. (Artificial) bipolar scores are likely to result in a bipolar structure, a typical aspect of a circumplex.
Julia Moeller also referred me to Gabriel Nagy, a measurement specialist from the Leibniz Institute for Science and Mathematics Education in Kiel, Germany. Dr. Nagy was so kind as to look at the problem and also confirmed that dividing each ipsatized score by a person’s standard deviation could produce additional artifacts.85 He stated it is very difficult to assess the complex effects of the procedure used by Scullard and Baum. He consulted the academic literature on the subject and referred me to an article by Bohrnstedt and Goldberger (1969) published in the Journal of the American Statistical Association. The articles provide several equations. Equation 13 can be used and simplified if some additional assumptions are made:86
85 E-mail correspondence from June 13, 2018.
86 These assumptions are: (1) the reciprocal of the individuals’ SD (1/S) is normally distributed; (2) the ipsative scale scores X and Y have, together with 1/S, a multivariate normal distribution; (3) the reciprocal of the individuals’ SD is uncorrelated with X and Y
COV(X/S, Y/S) = E(X)E(Y)VAR(1/S) + COV(X,Y)E(1/S^2)87
In this formula, COV(X/S, Y/S) is the expression of the covariance between two ipsative scores: X/S and Y/S refer to ipsative scores X and Y divided by S, the individual Standard Deviation, and E = the expectation (or the mean).
He wrote to me that “this equation shows that the within-person standardization alters the relationships of ipsative variables (i.e., COV(X,Y)) in a complex way, because COV(X/S, Y/S) depends on the variables’ means (i.e., E(X)E(Y)), whereas covariances or correlations between X and Y do not.” He thinks that “the dependencies of correlations of the means could be one reason why the procedure changed the order of the items in a nonlinear way.” He concluded that the evaluation of an instrument’s structure should not depend on the mean structure as was done in the context of the ED363. But he is not entirely sure of the ipsatization process on the MDS result, as “traditional MDS builds upon a (double-)centered matrix of dissimilarities that has been found to be often very close to the correlation matrix of ipsative scores.” Nevertheless, he ‘completely’ agrees with the other researchers who looked at this matter at my request: ipsatization poses several problems as it “reduces the rank of the correlation matrixes of variables.”
Gabriel Nagy also referred me to another article (sigh) written by Mark Davison for Psychological Bulletin in 1985. One interesting observation in this paper is that “the general factor plays a prominent role in several major theories of human abilities” (p. 103) and he refers to the example of mental testing (e.g. GMA or general mental ability testing, resulting in an IQ score). But he contrasts that with areas of psychology based on rating scales (as is the case with the DiSC too), where he points out that, in this case, “the general factor has sometimes been dismissed as a theoretically uninteresting, response level factor” (p. 103). He calls attention to the fact that one must investigate what the general factor and person means really mean: they could merely reflect response tendencies. He writes that ‘yeasaying’ in self-ratings and halo in employee evaluations result in certain artifacts that “manifest themselves in person means or in the general component” (p. 102). Scullard and Baum did not provide evidence for their assumption that the general factor could be more than just a response tendency. I also remind you that Mark Scullard did not want to further reply to my questions. What is clear however, once again, is that their procedure deviates from the procedure described in the Davison 1985 paper as well. But even simply removing the general factor is a controversial procedure, as Davison points out.
I know this is all very technical, and it goes way beyond my understanding of mathematics and statistics, but that is why I contacted these specialists. They all agree on one aspect: they consider division by the mean, as applied by Mark Scullard and Dabney Baum, to be problematic. The question is: did Scullard and Baum fully grasp the consequences of their mathematical choice? Or did they tinker with the equation until it resulted in a nice circular structure? Their refusal to provide further answers to my questions raises some serious doubts.
Besides this technical aspect of whether the procedure is allowed or not, the major problem with this discussion is the following: when people take a personality test or questionnaire,
87 George Bohrnstedt was so kind as to e-mail me this article. In this article, the non-simplified equation (13) reads as follows: C(xy, uv) = E(x)E(u)C(y, v) + E(x)E(v)C(y, u) + E(y)E(u)C(x, v) + E(y)E(v)C(x, u) + C(x, u)C(y, v) + C(x, v)C(y, u). Gabriel Nagy explained in an e-mail how the formula can be rearranged and simplified and why ‘u’ was replaced by ‘y’ (simply because most people think of x and y and not of x and u when thinking about two variables). I think it is already complicated enough, so I will not reproduce all the steps that lead to the simpler equation.
they wonder what their relative position is within a population 88 In order to know that, their scores are compared to the norms of a population (e.g. a norm group of leaders)— which constitutes a so-called inter-individual comparison. Norms can only be legitimately used with normative scores, not with ipsative or ipsatized scores. Remember, ipsative stems from the Latin ipse, which means he or himself. Ipsative scores for an attribute only refer to his/her scores on other attributes, whereas normative (x-point) scores allow for comparisons with other people. Several authors convincingly demonstrated that using ipsative scores to make comparisons with other individuals creates problems. Closs showed that an ipsative preference score for a ‘type 2 interest’ suggested that a person had a higher interest than 98% of the population, whereas a Like-Dislike score (non-ipsatized) score suggested the contrary: the person would have less interest than 99% of the population. These opposite conclusions show how problematic ipsatization can be. As Closs states: “Normative interpretation of scores from ipsative measures can grossly distort the person’s true standing on the factors measured” (p. 45). He further writes about how harmful this can be to a person’s career. I concur: the main reason I wrote this book was to prevent harm being done to individuals. In the case of the ED363, some individuals might grossly underestimate their leadership qualities, whereas others might grossly overestimate them.
Our own research (Marleen Redeker et al., 2012)89 demonstrated that it is possible to obtain a true interpersonal circumplex using the raw scores from a 5-point Likert scale. If the items of a questionnaire reflect the true underlying dimensions of an interpersonal circumplex— namely agency and communion—there is no need to use procedures such as ipsatization or centering around the mean—unless there is good evidence that a general factor exists. But there is certainly no need for the procedure used by the developers of the ED363.
To summarize the problems with the statistical techniques used for the ED363: (1) the general factor is not proven beyond any doubt, as other explanations are possible; (2) it is not clear what the item content is or whether the items even make sense after this general factor is removed; (3) the authors did not use the accepted method (centering around the mean) within the rich interpersonal research tradition, but divided the centered score by the standard deviation; and (4) ipsatized scores should not be used for covariance-based techniques such as MDS. Apart from these rather technical reasons, (5) ipsative scores can never be used to compare one individual to another.
As you can see, a problem (the problematic theory underlying the DiSC) rarely stands alone.
More problems to come…
The predictive validity for this theory has not been researched thoroughly either. For example, on page 151 of the manual, we read that the leaders were rated on “ three global leadership effectiveness variables.” But, each of these three variables were measured with… only one question each ! 90 Moreover, these three questions were asked at the same time when the raters filled out the DiSC 363. This is simply incredible. Had they never heard of collecting data at the same time from the same source, halo effects, or the fact that measurements with only three questions are unreliable? It seems not, given that they produced 2 tables to ‘prove’ the ‘predictive’ correlations with leadership effectiveness.
88 In the case of recruitment as well, that is of course what recruiters are looking for.
89 I was involved in this research, but only Danny Rouckhout and Marleen Redeker conducted the statistical analyses.
90 “xxx is a good leader”; “xxx is respected in the organization as a good leader” and “I enjoy working with xxx. ”
The manual reveals some other facts that should at least raise a few eyebrows:
● To assess the criterion validity of the Everything DiSC Assessment version, it was compared to the scores by raters who had completed one of their other tools—the Everything DiSC 363. The problem is that the average intercorrelation of ipsative scales equals -1/(m-1). The sum of all intercorrelations with another instrument will also tend to zero.
● Several of the MDS maps they did produce (remember they did not produce them every time) show problematic structures: they do not demonstrate that the theorized circumplex structure complies with the statistical requirements. This is the case for the MDS Map for Customers ( E verything D i SC S ales—Figure 6.5 on page 108) , the MDS Map for Salespeople (Figure 6.6 on page 111), or the MDS Map of the Management Priorities (Figure 6.11 on page 119)
● It is clearly stated that all norms are based on a U.S. adult population only. How representative that may be for other populations is unclear.
● The comparison with the NEO-PI-3 (p. 199) reveals other interesting points:
o The DiSC Assessment shows correlations between two subscales and the facet of assertiveness (part of the Extraversion dimension) in the NEO-PI-3. The D shows a correlation of 0.55 (D)ISC and Di has a correlation of 0.68. The opposite scales of D and DI, namely S and SC show negative correlations of respectively -0.30 and -0.75 with assertiveness. But there is almost no correlation with the other five facets of Extraversion. This leads to my conclusion that this scale is only measuring a small aspect (one facet, assertiveness) of the much broader construct of extraversion as captured in the 5FM or 6FM questionnaires. I tried to get an answer as to whether this table was based on their 7-point scale scores, or the ipsatized score (or centered scores). They declined to answer me, as I mentioned before.
o 6 out of the 8 subscales of the DiSC show moderate to high correlations with the full dimension of extraversion—which offers evidence that these constructs are not distinctive enough and show unacceptable overlap:
Di: .45
i: .69
iS: .52
SC: -.57
C: -.63
CD: -.34
The ‘C’ in DiSC stands for Conscientiousness (or the original compliance) and should not show such high correlations with extraversion.
o The only other important correlations are to be found between the NEO-PI-3’s Agreeableness and 5 scales (again too many) of the DiSC:
Di: -.40
iS: .52
S: -.67
CD: -.48 (this is the most worrisome correlation)
D: -.58
What about the DiSC persolog?
As for the DiSC persolog, no independent reviews are available. The providers claim their model measures 4 behavioral prototypes. Once again (sigh), this differs from other explanations offered by Wiley: interpersonal dynamics, personality, preferences, or priorities. The manual is far less detailed than the everything DiSC Manual, so taking their claims for granted requires an even bigger leap of faith (which I don’t recommend, as you will know by now). They just say that the internal consistency lies between 0.71 and 0.81 and test reliability between 0.72 and 0.85. No tables are provided.
Both the DiSC persolog and the Everything DiSC claim to have a circular or even circumplex structure. Moreover, both refer to interpersonal behavior. Therefore, it would be most evident to have a convergent validity comparison with other measures, like the circumplex questionnaires. As I already wrote, the authors of the Everything DiSC claim there is “substantial overlap with the Interpersonal Circumplex theory” (manual, p. 187) without offering evidence that it is a true circumplex that respects modern psychometric standards. Instead, the two DiSC competitors have run convergent validity comparisons with the NEO-PI91 instruments, one of the best 5FM questionnaires out there. This is quite strange because this questionnaire measures 5 personality traits, of which two are truly interpersonal in nature (extraversion and agreeableness). Moreover, as the Everything Disc manual clearly states, its construct of conscientiousness cannot be compared to the NEO-PI-3 construct of conscientiousness. Their own results indeed confirm this (the facets correlate between -.15 to .18, which is very low, demonstrating that their definition and operationalization of conscientiousness is different from the NEO-PI-3 definition). But the DiSC persolog shows high correlations between their operationalization of conscientiousness (0.64—which is considered very high).
So, it seems that the DiSC persolog and the Everything DiSC do not measure the same ‘C’ construct! In the DiSC persolog, the domain dominance shows high correlations with 4 dimensions of the NEO-PI-R: neuroticism, extraversion, agreeableness, and conscientiousness. In the NEO-PI-R, these 4 dimensions are almost independent from each other. This reveals that the construct of dominance in the DiSC persolog is not ‘pure’ and is insufficiently delineated. In other words, it is not a distinguishable construct that is independent of others. Several sub-facets of the NEO-PI-R dimension of Extraversion show high correlations with both influence and extraversion from the DiSC persolog. The DiSC persolog also shows high correlations with facets of neuroticism. These are all indications that the DiSC instrument fails to capture the well-known personality traits. And yet the developers of the DiSC persolog don’t seem to realize the problems the many correlations display: on the contrary, they almost proudly highlight the high correlations with the facets of the NEOPI-R.
Psychometric properties of DISCUS
Let me briefly discuss one independent study conducted on the DISCUS by Monica Martinussen and her colleagues (2001). The DISCUS is clearly posited as a personality measure by the publisher (Roodt, 1997). 103 undergraduate students completed92 three questionnaires. The researchers compared the results of a 5FM measure (5PFa, Engvik, 1993) with both a normative and an ipsative version of the DISCUS. Below are the troublesome findings:
● The correlations with the 5FM measure 5PFa were quite different in magnitude forthe two DISCUS versions. For example, the ipsative version showed 5 significant negative correlations with the 5FM, whereas the normative had only 1 significant negative correlation.
● Using only four scales is problematic when ipsative scoring is used (as already pointed out by Bartram in 1996) as the problems with ipsative scales increase as the number of scales is reduced.
● The sum of correlations between the four ipsative DISCUS dimensions and external variables from the 5FM were close to zero. Indeed, the problem is that the average intercorrelation of ipsative scales equals -1/(m-1). The sum of all intercorrelations with another instrument will also tend to zero. Danny Rouckhout (again, thank you!)
91 The Everything DiSC is compared to the NEO-PI-3 and the DiSC persolog with the former version, the NEOPI-R
92 For the ipsative DISCUS, 99 students completed the questionnaire, instead of 103.
calculated whether this would be true here as well: and it is indeed: so convergent or divergent validity is an artifact. This constitutes no evidence for criterion validity.
● The four dimensions of the DISCUS represent combinations of the 5FM traits, rather than representing independent traits (or whatever it is they are trying to measure). The DISCUS seemed to capture only the interpersonal traits (Agreeableness and Extraversion), much like the Everything DISC 363 for leaders.
● They warn that the DISCUS should not be used for personnel selection (although the DISCUS manual promotes such use).
Problems, problems, and more problems. It comes as no surprise to me, as I think I have quite convincingly demonstrated that the underlying theory is entirely wrong.
Conclusion
The theory was never accepted in mainstream psychology and surely not by my Champions League of psychologists. The efforts to cast a wide net so that the DiSC could encompass or span multiple theories, such as pseudoscientific Jungian Typology or the well-established Interpersonal Circumplex, is preposterous and can be entirely refuted.
Problems never comes alone. I smelled a rat, and I believe I have convincingly demonstrated that several of the statistical techniques used are not allowed or are highly problematic. Eminent psychometricians have been warning against these problems since 1954 (Guilford). Ipsatized data cannot safely be used to compare individuals to other individuals. The Everything Disc for Leaders 363, for example, isn’t capable of telling a leader his or her relative position on the scales compared to other leaders.
There are much better instruments to help find out about your personality or about interpersonal (behavioral) style. For more information on these, refer to the chapters on personality traits which describe the 5FM and 6FM, and to the chapter on the interpersonal circumplex. As the authors of the Everything Disc manual themselves state (p. 10) : “ The Interpersonal Circumplex has proven itself over time to be a very robust model of personality.” Remember the reviewers of the Buros Institute have pointed this out as well. I concur, which is why I strongly recommend against using any DiSC model, as the theory on which they are based is entirely wrong and even discriminatory (e.g. towards women). The DiSC model used by DiSC Indra and later by Everything Disc is based on adjectives. Adjectives have been used to adequately describe personality in five or six trait dimensions (the 5FM and the 6FM), and adequate measures have been developed (e.g. the NEO-PI-R and the HEXACO). What’s more, interpersonal adjectives have been used to develop the interpersonal circumplex and the adequate measures based on them (such as the IAS-R by Jerry Wiggins). So, there are plenty of valid and reliable alternatives if you know what you want to measure and why.
I would like to thank Danny Rouckhout, Kenneth Locke, Thane Erickson, Terence Tracey, Julia Moeller, Gabriel Nagy, and Nick Brown for their contributions regarding the evaluation of the statistical techniques.
■ Original sources consulted
Two reviews by the Buros Institute (url: http://www.unl.edu/buros/bimm/index.html)
The Everything DiSC Manual by Wiley (written by Mark Scullard and Dabney Baum)
The DISC Persolog Manual by persolog GMbH in Germany (written by Renate Wittmann)
Altarriba, J., Basnight, D. M. & Canary, T. M. (2003). Emotion representation and perception across cultures. In: Lonner, W. J., Dinnel, D. L., Hayes, S. A. & Sattler, D. N. (eds.), Online readings in psychology and culture (Unit 4, Chapter 5), http://www.wwu.edu/~culture), Center for Cross-Cultural Research, Western Washington University, Bellingham, Washington USA.
Bartram, D. (1996). The relationship between ipsatized and normative measures of personality. Journal of Occupational and Organizational Psychology, 69(1), 25–39.
Bohrnstedt, G. W., & Goldberger, A. S. (1969). On the exact covariance of products of random variables. Journal of the American Statistical Association, 64(328), 1439–1442.
Bunn, G. C. (1997). The lie detector, Wonder Woman and liberty: the life and work of William Moulton. History of the Human Sciences, 10(1).
Chan, W. (2003). Analyzing ipsative data in psychological research. Behaviormetrika, 30(1), 99–121.
Closs, S. J. (1996). On the factoring and interpretation of ipsative data. Journal of Occupational and Organizational Psychology, 69(1), 41–47.
Costa, P. T., & McCrae, R. R. (2011). The five-factor model, five-factor theory, and interpersonal psychology. Handbook of interpersonal psychology: Theory, research, assessment, and therapeutic interventions, 91–104.
Davison, M. L. (1985). Multidimensional scaling versus components analysis of test intercorrelations. Psychological Bulletin, 97(1), 94–105.
DeYoung, C. G., Weisberg, Y. J., Quilty, L. C., & Peterson, J. B. (2013). Unifying the aspects of the Big Five, the interpersonal circumplex, and trait affiliation. Journal of personality, 81(5), 465–475.
Erickson, T. M., Newman, M. G., & Pincus, A. L. (2009). Predicting unpredictability: Do measures of interpersonal rigidity/flexibility and distress predict intraindividual variability in social perceptions and behavior? Journal of Personality and Social Psychology, 97(5), 893–912.
Fabrigar, L. R., Visser, P. S. & Browne, M. W. (1997). Conceptual and methodological issues in testing the circumplex structure of data in personality and social psychology. Personality and Social Psychology Review, 1, 184–203.
Fournier, M. A., Moskowitz, D. S., & Zuroff, D. C. (2009). The interpersonal signature. Journal of Research in Personality, 43(2), 155–162.
George, D., & Mallery, P. (2003). SPSS for Windows step by step: A simple guide and reference. 11.0 update (4th ed.). Boston: Allyn & Bacon.
Gurtman, M. B. (1994). The circumplex as a tool for studying normal and abnormal personality: A methodological primer. In S. Strack (Ed)., Differentiating normal and abnormal personality (pp. 243–263). New York: Springer Publishing Co.
Gurtman, M. B., & Lee, D. L. (2009). Sex differences in interpersonal problems: A circumplex analysis. Psychological Assessment, 21(4), 515–527.
Guttman, L. (1954). A new approach to factor analysis: The radix. Mathematical thinking in the social sciences. New York: Columbia University Press, 258–348.
Horowitz, L. M., Rosenberg, S. E., Baer, B. A., Ureño, G., & Villaseñor, V. S. (1988). Inventory of interpersonal problems: psychometric properties and clinical applications. Journal of consulting and clinical psychology, 56(6), 885–892.
Malan, G. G., Cobanoglu, C., Waldo, R. D., & Yang, W. (2014). Managing with style: An analysis of work styles of hotel managers. Journal of Human Resources in Hospitality & Tourism, 13(2), 190–209.
Marston, W. (1927a). Primary emotions. Psychological Review, 34(5), 336–363.
Marston, W. (1927b). Primary colors and primary emotions. Psyche, 30, 4–33. (article is no longer available – I received a print copy).
Marston, W. (1928). Emotions of Normal People. New York: Harcourt, Brace and Company.
Martinussen, M., Richardsen, A. M., & Vårum, H. W. (2001). Validation of an ipsative personality measure (DISCUS). Scandinavian journal of psychology, 42(5), 411–416.
Moeller, J. (2015). A word on standardization in longitudinal studies: don’t. Frontiers in psychology, 6, 1389.
Ojanen, T., Grönroos, M., & Salmivalli, C. (2005). An interpersonal circumplex model of children’s social goals: Links with peer-reported behavior and sociometric status. Developmental psychology, 41(5), 699–710.
Redeker, M., de Vries, R.E., Rouckhout, D., Vermeren, P. & de Fruyt, F. (2012). Integrating leadership: The leadership circumplex. European Journal of Work and Organizational Psychology. Published online first: http://dx.doi.org/10.1080/1359432X.2012.738671 (free download).
Roodt, K. (1997). A reliability and Validity study on the DISCUS Personality Profiling System (Axiom). Online publication: https://www.axiomsoftware.com/disc/validity/disc-validity-and-reliability-the-roodtreport.php (downloaded on March 22, 2018).
Russell, J. A. (1997). How shall an emotion be called? In R. Plutchik & H. R. Conte (Eds.), Circumplex models of personality and emotions (pp. 205–220). Washington, DC, US: American Psychological Association. Taylor, B. N., & Kuyatt, C. E. (1994). Guidelines for evaluating and expressing the uncertainty of NIST measurement results.
Tracey, T. J. (2000). Analysis of circumplex models. In Handbook of applied multivariate statistics and mathematical modeling (pp. 641–664).
Tracey, T. J. (2012). Problems with single interest scales: Implications of the general factor. Journal of Vocational Behavior, 81(3), 378–384.
Tracey, T. J., Rounds, J., & Gurtman, M. (1996). Examination of the general factor with the interpersonal circumplex structure: Application to the Inventory of Interpersonal Problems. Multivariate Behavioral Research, 31(4), 441–466.
Wright, A. G., Hallquist, M. N., Morse, J. Q., Scott, L. N., Stepp, S. D., Nolf, K. A., & Pilkonis, P. A. (2013). Clarifying interpersonal heterogeneity in borderline personality disorder using latent mixture modeling. Journal of personality disorders, 27(2), 125–143.
Wright, A. G., Pincus, A. L., Conroy, D. E., & Elliot, A. J. (2009). The pathoplastic relationship between interpersonal problems and fear of failure. Journal of Personality, 77(4), 997–1024.
Wu, L. Z., Roche, M. J., Dowgwillo, E. A., Wang, S., & Pincus, A. L. (2015). A Chinese translation of the Inventory of Interpersonal Problems–Short Circumplex. Journal of personality assessment, 97(2), 153–162.
Young, K. (1927). The field of social psychology. Psychological Bulletin, 24(12), 661–691
■ Other References (abstracts, summaries, excerpts, or reviews)
Clarke, W. V. (1956). The construction of an industrial selection personality test. The Journal of Psychology, 41, 379–394.
Marston, W. M. (1917). Systolic blood pressure symptoms of deception. Journal of Experimental Psychology, 2(2), 117.