Why Psychological Evaluations are Bogus

This page is http://www.thelizlibrary.org/therapeutic-jurisprudence/custody-evaluator-testing/forensic-mmpi2.html

A Grammatical Analysis of the MMPI-2

"I can't say": truthful, thoughtful responding in the
forensic context will (appropriately) invalidate the test.
By Elizabeth J. Kates, Esq.
This is not a radical article. It might seem that way at first, but upon reflection, the position should seem quite tame. Lawyers' standard advice to their litigation clients is to "answer questions truthfully". To think before speaking. Perhaps to respond succintly. But in any case, to respond "truthfully". Only rarely does the advice deviate from this, e.g. for fifth amendment purposes, or for questions that violate attorney-client privilege, in which case the client is told not to respond at all. In preparing clients for deposition and court testimony, it also is common to review the kinds of questions a client might get from opposing counsel, and among other things, to caution the client to recognize "traps", that is, questions that appear to be demanding a simple yes or no, blanket agreement or denial, but which in fact just cannot be answered in that manner if they are to be answered truthfully (or at least without being misleading). But what happens when clients are ordered to take forensic examinations in the context of litigation? The same kind of counseling is not done. Should it be? This article argues that the answer to that question is "yes".
Psychological tests given in the family law forensic context are inherently invalid. They do not measure what they purport to measure (anything at all pertaining to parenting capacity). They cannot discern past facts. They have no accurate predictive ability. They in fact rarely shed any light on the legal issues to be decided, but conversely, often enable forensic evaluators to mischaracterize the parties and justify, under the pretext of adding in "objective science", their biased opinions. The problem likely is present in other forensic litigation contexts, but is particularly bad in the area of family law. That there is little interrater reliability in psychological diagnosing is well-known. That the introduction of psychological forensics in a child custody case creates burdensome expense, complicates cases, and creates new problems is well known. Thatparenting evaluators frequently get it wrong is well-known. That no psychological tests can discern who is or is not a "good parent" is well-known. But that child custody evaluators actually have little or no wisdom to add in a family law case for some reason is not also well-known. And it should be.
Isn't avoiding these kinds of distortions, errors, and malfeasances exactly why clients are prepared for testimony and told when they should not respond at all? Of course. So why -- contrary to all other aspects of the litigation, and arguably contrary to multiple constitutional and statutory rights that clients have (first, fourth, fifth amendment, privacy, etc.), they are not prepared adequately for psychological tests? Because psychologists make a lot of noise about "coaching" and "invalidating the tests"? Because the psychological trade promotion industry has sold the public, including legislatures and judges, one heck of a bill of goodsregarding their so-called "expertise", which in fact amounts to nothing much at all in the area of parenting? Because the courts have become inundated with more and more unnecessary "expert testimony" in denigration of traditional rules of evidence and due process?
Psychological tests bolster the aura of expertise and the opinions uttered as "evidence" in court by forensics, creating a veneer of pseudo-objectivity and scientific validity.
The first line of defense, of course, is to object to having any client take these tests. Case law in multiple states indicates irreparable harm from them when mental health is not "at issue". That justifies an interlocutory appeal. But there's not always time or money for that. And some judges would forego ordering custody evaluations altogether if they merely duplicated what the judges recognize to be ordinary evidence. The idea of some kind of scientific or medical "assessment" that can ascertain "facts" that would be beyond the ability of the judge to discern (like a medical lab test) feels key. So if avoiding this snake oil isn't possible, then the client at least should be well-prepared.
The granddaddy of psychological personality tests is the MMPI, or MMPI-2 in its revised version. It is one of the oldest of the psychological tests, and the one that has been most studied, and also the psychological test most often administered by psychologists for use in court. So this first article on psychological test taking in the forensic litigation context will address the MMPI-2.
The MMPI-2 is what is called a "forced choice" test. The primary (now revised) version of the test has 567 true/false questions. (There is a short form, as well as an adolescent version, and also ultra-dubious versions translated into different languages.) The test taker is told that that he or she must answer "true" or "false" in response to the various statements. The problem with doing so is that most of the questions contain undefined words that could have multiple meanings, or are vague, or have compound parts, or do not permit exceptions, or contain false assumptions, or otherwise cannot -- if one actually were to think about them before responding -- be answered simplistically "true" or "false". For example, "Answer true or false: I have stopped beating my children." (The actual test questions are not given in this article, but they readily can be found in numerous relatively inexpensive books available to the public, e.g. mmpi-2 administration, mmpi-2 practitioner's guide, mmpi in court, etc.)
The instructions that litigants typically are given when they are sitting for the test do not clearly let them know that they have a third choice, rather than rushing through and guessing "true" or "false". That choice is "can't say". This is the choice of not responding to the question and leaving the answer blank. Not in a "non-cooperative" way, of course. But whenever the question cannot be truthfully and accurately answered either "true" or "false". Telling the client not to respond hastily, not to guess, and not to affirm or deny statements that cannot be affirmed or denied, is the same advice you would give the client preparing to sit for a deposition. Although your advising your client about how to take the test is virtually certain to irritate the psychologist administering the test, the overriding context here is in fact court. Legal advice prevails regarding what is or is not in the client's interests, or the goals. The court case is important. A forensic psychological test is not otherwise of any benefit whatever to the client.
In the context of litigation, every individual who is court-ordered to respond to a question, which includes to take a psychological test has the absolute right to respond only truthfully. That means that the litigant has the absolute right to select the "can't say" choice if there is any legitimate reason the litigant "can't say". If too many litigants did this, of course, that rapidly would make this and other psychological tests pretty useless in the courtroom context. And psychologists don't want that to happen because, like voo-doo, these tests help to validate the opinions of the forensic evaluator, and make those opinions appear to be important, grounded in mysterious and difficult scientific analysis, objective and authoritative. But what is good for the psychological industry is neither the lawyer's nor client's concern.
Litigants are told by psychologist test administrators to try to answer all the questions, and if they are unsure, to answer based on whether the statement seems "more true" or "more false" for them. That's absurd, but it does encourage the litigant to go ahead and pick the "less wrong" choice. Typically, the litigant is nervous, and fearful of not seeming cooperative, and so, unless someone thinks beforehand to tell the litigant he or she does not have to do this, the litigant in fact will. And risk getting into hot water. It's the lawyer's job to make sure that the client understands the difference between "not cooperating" and being a patsy. Note that the length of this particular test also aids this kind of "cooperation" by virtually guaranteeing that the test taker must read quickly, and answer quickly without giving much thought to the questions. This is the opposite of how litigants are instructed to respond to questions in depositions and in the courtroom.
What lawyers must understand, and what they must convey to their clients is: whether or not later down the road a client may or may not be cross-examined on an "admission against interest" made with respect to any specific test item, in effect, the client who is under compulsion to sit for a psychological examination nevertheless is in court. Everything said can and may be held against him. There is a cumulative effect in the responses to each item. Clients also should be told that this test is not in fact timed, to refuse to be rushed, that they are permitted to take all the time they need, and that if they grow tired, they are permitted to ask for a recess and to finish the test (and/or the rest of the tests they are expected to take) at another time.
At first look, the questions themselves don't seem all that difficult to answer. But the sheer number of questions discourages the litigant from thinking too much about any one question. The test is designed to get the subject to rapidly answer based on feelings, and on first impressions, rather than thinkng. The test contains "traps" such as the consistency items in which the same question essentially is asked again at a later point in the test (if it's answered "inconsistently", that's a ka-ching in the "lie" scale.) Therefore psychologist does not want the subject to pause and take much time before answering each question. Or to go back and forth to check previous responses. Thinking before responding to each item also would make the test unwieldy and unduly expensive from an administration standpoint, of course, but mostly, it is thinking that would throw a monkey wrench into how the test works.
The psychology industry, and particularly the test publisher, similarly do not want subjects to see the test questions in advance because this would encourage thinking instead of impressionistic responding. It is not because anyone is likely to easily remember all the questions or be able to "cheat" the test by selecting some kind of memorized well-adjusted pattern of true and false answers. It is because the test taker would have contemplation time. The test taker would realize how ridiculous most of the statements are, and not respond using gut-level "feelings" and guessing, but with the brain engaged. Truthfulness is not desired, nor is accuracy. Consider: this is the opposite of how lawyers would counsel their clients to respond to questions about themselves and their circumstances in any other context pertaining to their case! And why should it be different here? Because there's something valid and important that the psychologist is going to ascertain about a reasonably normal, healthy (albeit stressed) family law litigant by having him or her take a psychological test? Actually, no.
Even when the litigant is not court-ordered to take the test, however, it is the individual's absolute constitutional right to give only honest, thoughtful, truthful responses to any question asked. There is no penalty for "think time". No individual can be forced to speak falsely. But if, in fact, the litigant thinks, and thinks appropriately, about the various test questions, few of them will be able to be answered. Moreover, if there are too many "can't say" answers (sometimes as few as 30 will do it, although 70 is a sure bet) that will invalidate the test. It simply won't be able to be used at all. No inference of any kind can be made from an invalid test. Given that psychology really does not have the ability to do what psychologists pretend it can do, and given that so many custody evaluators simply spout horseshit, backed up by the implied objective validation of psych test magic, and aided by the handy phrases and suggestions given in the computer printouts of the results of these things, it may very well be preferable for the test to be completely invalidated than for the litigant to risk being misjudged, wrongly assessed, or incorrectly labeled. The psychology industry actively fights to keep this information from getting out. The test results are not in fact based on truthful answers but ways of sloppy thinking, and are dependent upon the subject not considering that the questions themselves frequently beg for an answer of "can't say". In fact, numerous articles about these tests, so-called "research" write-ups, repeat completely unverified assumptions, such as the following, in which the psychology industry fools itself:

"...an estimated administration time of between 1 and 2 hours for the majority of cases. In patients with severe psychopathology this administration time may extend to between 3 and 4 hours...Greene (1997) has estimated the expectable range of omissions at between 1 and 15 for normal subjects and between 0- 20 for psychopathological patients. In general, the administration protocol is considered to be invalid if the respondent leaves 30 or more items unanswered in the first 370; if these omissions occur after item 370, clinical interpretation can go ahead for the basic clinical scales and validity scales, but not for the rest of the scales. Excessive omission of items is usually considered to be related to patterns of defensiveness, indecision, carelessness, fatigue or inability to read and understand the items (Butcher & Williams, 1992; Graham, 1993)." ASSESSMENT OF RESPONSE DISTORTION IN MMPI-2, Héctor González Ordi and Iciar Iruarrizaga Díez, Papeles del Psicólogo, 2005. Vol. 26, pp. 129-137 [emphasis added]

No research confirms these assumptions. Psychopathy, defensiveness, indecision, carelessness, fatique or inability to read and understand might cut it as hypotheses if the test taker is voluntarily taking the test in the hope for and belief in therapeutic treatment. But this list omits the primary reason for "can't say" responses in the forensic setting. And that is: that the subject truthfully and accurately just can't say.
Litigants are entitled to know that in litigation, the same rules apply to answering any question to an evaluator as apply to testimony in a deposition or in court. They are entitled to know that no one has the right to force them to respond with a guess, to endorse a false belief, to answer what they cannot honestly say they know (particularly if a question is phrased in the absolute), or to respond true or false to a compound question, or one that has implicit assumptions or undefined terms, or to respond quickly and inaccurately to any question that actually requires some amount of contemplation, reflection or recall of history.
Prior to taking the test, too, every litigant should be given a little grammar refresher, e.g. the difference in the meaning of, for example, "I worry" versus "I have been worrying lately" versus "On occasions in the (recent/far) past I have worried". Every litigant should be reminded that all of us feel and behave differently in different kinds of settings and among different groups of people and at different times. Every litigant should be advised about the concept of the relative ("it depends") versus the absolute ("always" and "never"), as well as the dangers of answering without thinking, based on impressions or "feelings" (feelings aren't "thinking", and a "feeling" is not accurately a thought or a "belief".) It is easier and cheaper to prevent problems than to fix them after they have happened. In the forensic context, not having a "valid" MMPI-2 will avoid a good amount of discovery time, deposition time, cross-examination time, and countering-evidence time, including forensic consultant time, and all the attendant expense. Real evidence will have to make the day. Is this radical? Not in the least. It's no more and no less than every lawyer attempts to achieve for every other circumstance in which his or her client talks about the client's case... article continued below

The explanation of how to take forced-choice psychology tests should include a discussion that addresses (same as with regard to answering other kinds of questions in the case) examples such as the following: Do you know how intensely or not the majority of other people in the world feel emotions or pain? If not, then you can't say anything about how you measure up compared to those other people, can you.  Do you have personal knowledge about how often, or when, or under what circumstances any other person in the world might tell a lie? If not, then you can't say anything about that subject, can you.  Can you define the word "unfair"? Does it mean different things to different people? Could it mean different things in different contexts? If the word is not defined, then you can't answer a question regarding what may or may not be "unfair" that other people do, could you. Don't answer a question or respond to any statement unless you are absolutely certain you know what every word in that sentence means.  Are you a sociological researcher? And do you also have the memory of a savant? If not, then you cannot answer any questions that ask you about what "most people" do, can you? Of course not. And certainly you have no personal knowledge about this either. (And, by the way, how much is "many"? "Many" compared to what?)  Define the word "swear" for me. If I asked you whether you ever felt like "swearing", would you know whether that meant "take an oath" or -- in its informal context -- "utter an obscenity"? If the term were not defined, could you truthfully answer? No. You couldn't say. You would need a definition. You would need to know what the questioner meant by the word. You shouldn't assume you know. And, of course, if you are the sort of person who just f**king talks the way you want to when you want to without having "feelings" about word usage, you couldn't respond to this item either, could you.  Suppose I asked you whether you felt like "smashing things". That sounds violent, doesn't it. It sounds like "smashing" could mean "breaking". But if you were a potter, you would smash clay, and if you were a cook you might smash potatoes, and if you were a tennis player, it would imply something very different, altogether, wouldn't it. You couldn't truthfully say whether or not you ever felt like "smashing" anything unless you knew for sure what I was referring to, could you. (By the way, how often is "sometimes"?) So unless you sometimes (routinely, on an ongoing periodic basis) have the feeling of wanting to hit, break, and destroy anything within reach, indiscriminately, you couldn't answer this question, could you. It would not be truthful to admit to wanting to smash "things" if you are thinking only of specific things -- such as dishes, or clay, or a tennis ball.  Do you have personal knowledge and can you answer whether or not, and how often or when any other person in the world might do any particular thing? If not, then you can't say anything about that subject, can you. There are billions of people in this world, in all kinds of circumstances and different cultures. You don't have personal knowledge of what "most" people or even "few" people actually do, do you. And you can't really say anything about yourself in relation to them either. Do you have personal knowledge of what most people think? Didn't think so. (By the way, what percent of people, in the context of everyone in the world, would be "few"? How about "many"?)  Do you have personal knowledge and can you answer whether or not, and how often or when any other person in the world might have a particular emotional need, physical need, financial need, or motive? How about "all people"? If not, then you can't say anything about those subjects, can you as far as "most people" or "few people". You don't have personal knowledge of what "most people" or "few people" actually need, want, or think, do you. That means you don't know anything about yourself in relation to them, do you.  Define "family". Is your ex-spouse your "family"? Your children? Are the parents and siblings you lived with during your childhood currently your "family"? How about your cousins? Great aunts and uncles? Step-family members? Sorority sisters? Army buddies? Current boyfriend/girlfriend? Deceased grandparents? Nuclear family? or extended family? Current family? or former family? Does "family" mean different things to different people? If I asked you about things you do (present tense) with family members, you would need to know what I meant by "family" to answer the question, wouldn't you?   Quickly now -- you have 20 seconds -- give me a list of every party you've ever been to. You need more time? Don't answer the question. How about a list of every disagreement you've had with anyone. Can't say? Okay. How about every time you've ever spoken in public? No? Can't say. If you can't remember all of the events, significant or insignificant, then you can't say anything about what may or may not have happened at most of them, or few of them, or "in general" can you? Have you ever been asked for examples of things and then only later thought of more? Or after contemplating something, such as, oh let's say, volunteer work to put on a resume, didn't think of an example until later? Can you remember all of the times any particular thing may or may not have happened to you when you were in school? Would that be elementary school? High school? College? Grad school? Night school? How could you quantify or estimate how frequently or not something may or may not have happened if you cannot instantly recall all the events? You can't. You can't say.   What's the definition of "good"? Is what is "good" to you always the same as what is "good" to everyone else? How do you know? You don't. Can you measure "goodness"? Is there any time you can think of when "good" would be a sort of negative description? How about when something that is truly excellent is called merely "good"? What's the definition of "bad"? Does that ever mean "good"? How about the word "important"? Could the same thing be be important or not to different people depending on whose point of view we're considering it from? Let's talk about value judgments, beliefs, unverifiable opinions versus facts, and feelings versus thinking...  Let's talk a little about grammar. Suppose I said "I have a car." That's the present tense, isn't it. That's not the same as saying "I have had a car", is it. How about "I had a car." Or "I used to have a car." Or "Once I had a car." If I asked you whether you "have" arguments with family members, that's the present tense. Are you having an argument right this very minute with a family member? No. So in this case, the present tense means some period of time other than right now, this very minute. It's the "continuous present tense." The continuous present tense, as opposed to the present that means "right now" implies some regular occurrance during some extended period of time that is considered to be the present. What period of time would be covered by "the present time" when we are not talking about something that is continuous, but happens periodically, starts and stops? How far back would that go? It's unclear, isn't it.  If you were asked a question about something you "have" and it's not something you "have right now at this very minute, or else "have" continuously (versus "continually"), you would need to know what period of time were covered in order to answer accurately, wouldn't you. By the way, if something has happened in the past, do you have any proof of whether it will happen again? Do you have a crystal ball? So if something is an event that recurred in the past, as opposed to something continuous, like "having a car", you don't know whether it can be described as something continuing right now, can you. You can't. You can't say, because you don't know whether it will happen ever again.  Let's discuss "accuracy" in responding to questions. If I asked you whether "most people" in the world love their spouses, would you be able to answer truthfully from personal knowledge that the majority of people in the entire world who have spouses love them? No. You really have no idea, and I really have no idea. We also first might want to define what is meant by "love" too. Just to be clear.  Let's review "compound sentences". Compound sentences are two complete thoughts -- what could be two separate sentences -- combined into one. If you were to be asked a compound question in court, on cross-examination I would object. I would object because two separate sentences, or questions in that case, require two separate answers. Here's an example of two sentences: (1) "I don't understand". (2) "I have been angry." Suppose that you don'tunderstand (so if I asked you true or false, "I don't understand", you would say "true"). And suppose that you have not been angry (so if I asked you true or false, "I have been angry", you would say "false".) Answer this compound question true or false: "I don't understand why I have been angry." You can't say, because one part is true and one part is false. Which part of the sentence is more important? You don't know. You can't say.  Now suppose I added even more into that sentence. Suppose I told you to respond true or false: "OFTEN I don't understand why I have been angry." Now we have not two, but three parts. How much is "often"? Doesn't it depend on "compared to what"? Often compared to who? What is "often"? In whose judgment? By whose perspective?  Suppose I said "sometimes". That's a better word than "often" in that there's more flexibility, but now consider a fourth problem with the compound sentence. The first part of the compound sentence is in the present tense. "I don't understand." It implies some ongoing, recurring, regular occurrance -- or else some continuous state -- over some period of time that includes "the present". What period of time would that be? This week? This year? The past five years? Your whole lifetime? It's impossible to respond accurately, isn't it. You could only answer "true" or false" to a statement about a present period of time if the person making the statement clarified the period in question, what they meant.  True or false: "Getting yelled at hurts my feelings". Suppose a three-year-old yelled at you. Would that hurt your feelings? Probably not. Could you say that "as a general rule" it's "true" that getting yelled at hurts your feelings? Perhaps. It wouldn't be completely accurate, would it. Even if you think that's somewhat accurate, are you sure that you are able to remember all of the times someone yelled at you, and whether your feelings were hurt? Wouldn't it depend upon why you were being yelled at? The statement doesn't include the conditional words "as a general rule". It's phrased as an absolute. Suppose I added in there a little more. True or false: "Getting yelled at hurts my feelings horribly". Now could you answer true? Could you honestly answer false either? Do you remember whether your feelings were hurt a little or a lot? It's not worded well, so as worded, you really can't say.  True or false: "Sometimes I feel stupid". Would you say that's true? Something everyone would endorse? Do you feel stupid regularly over some period including the present time? Now it's not so clear, is it. But that's what the present continuous tense implies. You regularly feel stupid, even if it's not "often". How often is "sometimes"? A few times a week? A few times a month? How often is "often"? Compared to what? Think carefully about the statement. It does not say, which would, perhaps be more accurate "There have been some occasions in the past when I have felt stupid." If it did say that, then perhaps you could respond "true". Perhaps everyone could. Assuming they remember. If they don't remember, or if they have to think about whether they felt "stupid" or whether it was some other kind of feeling, such as merely "ignorant", or "clumsy", or the definition of the term, why then perhaps they just can't say.  Now suppose I added even more into that sentence. "Sometimes I CERTAINLY feel stupid." You're feeling stupid, and you're also completely certain that you feel stupid. What is that word "certainly" adding? Even if you have felt stupid, did you really think you were stupid? Can you say? (By the way, do you actually remember?) Watch adjectives. Let's take another example. What's the difference between someone being "cheerful" and someone being "unusually cheerful"? How does the adjective "unusually" change the word "cheerful"? Does it mean "very cheerful"? Or does it imply a cheerfulness that is not ordinary for the otherwise grumpy person? It's unclear, isn't it. By the way, who defines "unusually"? Would that be "unusually" from your subjective perspective, or "unusually" from someone else's perspective? Or from some objective standard of what's "usual" or not for you or for other people? If you don't know, you can't say.  Let's consider compound sentences again. Here are some examples of a sentence "one". "It makes me angry."'It makes me impatient." "It makes me nervous." It makes me happy." "It bothers me". "I believe." Now let's add a sentence "two". Mix and match, and make one compound sentence. "People interrupt me." "I am being followed." "Others are doing that." "I am shy." "I am sad." "I had an argument." For example "It makes me nervous WHEN others are doing that." Or "It bothers me when people interrupt me." By the way, how often do people interrupt you? Can you remember all the notable as well as unimportant and unnoticed events? (Do your kids interrupt you? The telephone? Your boss at work? Does it make a difference? Does it depend?) The varieties are infinite. You can't answer a compound sentence with unconditional parts "true" or "false" unless both parts of the sentence are always "true" or always "false", and even then it could be ambiguous...   By the way you have a first amendment right not to answer any questions about your religious beliefs because they're not at issue in this case. But it's up to you... [Ditto, if applicable, private protected medical information, sexual functioning, sleep habits, appetite and bodily functions.] Etc...

The scores on the MMPI-2 are based on studies done on various demographic populations of people to see how these groups on average answered various questions "true" or "false". The scores provide a way to compare the test taker's answers with those of various groups of persons -- psychopaths, depressives, housewives, airline pilots, liars, fakers, persons considered to be mentally normal, persons considered to have mental problems, and so forth. The 567 questions are grouped (with overlap -- many questions show up in multiple categories, or "scales") to show patterns of responding that are common with this or that group of people who share some common trait or traits. It's profiling, or stereotyping.
Thus the answers given by a test taker stereotype the subject as being similar to various arbitrary groupings of people who selected similar patterns of responses in various groups of questions. One issue is that this is indeed a stereotype. Even assuming that the people in the stereotype group who share a common characteristic ALL responded in a certain way ("all" never happens), that still does not mean that some other, as-yet unidentified group of persons would not also respond in this way. Thus, the MMPI-2 has, for example, an identified group of questions that together constitute the K scale. Subtle liars tend to answer "false" to many of the questions that supposedly more honest persons would endorse as "true", and also people with higher educations from higher socio-economic backgrounds (especially those who are pilots with Air Force backgrounds and possibly government security clearances applying for jobs) also tend to answer the same way. For all we know, people from certain towns in upstate New York with graduate degrees, an interest in carpentry, musical ability and curly red hair or going through a divorce might also tend to answer the same way. But no one knows. That latter group of people hasn't been researched.
How people are grouped into categories is arbitrary. Sex? Religion? Geography? Family background characteristics? Assumed culture? Some psychological trait? And so forth. The possibilities and categories and subcategories and breakdowns are endless. Also fairly arbitrary, and subject to research bias, is the assumption made as to the identified trait shared in common among members of the group that supposedly is so important that it's moving the results, so important that it warrants joining possibly very different people together in a group as if they were very similar. That identified trait may or may not be what has created the similar responses. Mistakes are made, and often. This is one reason repeat studies sometimes do not replicate the findings of prior studies. This kind of assumptive categorization (stereotyping) usually occurs before any research does, and before any attempted replicating research. So not only is the categorization arbitrary, but when it includes personality traits or mental disorders, it's not even a sure thing that all of those in the group that was studied in fact had the characteristic in common.
The point is that the test came first, the categorization next, and the research afterward, in a kind of researchical hunt-and-peck ("let's see if the people in this newly-created category respond to specific test items similarly...") "Research" (basically hunting for patterns of responses, throwing out ideas for groupings that don't yield results and following up with more research on groupings that do) has gone on for decades. The groupings of people with various characteristics are arbitrary. Some are not even real, such as categories of people who are told to fake feeling depressed, or to lie about this or that to see if they can fool the researchers. Often the categories that presume to group people by a defining trait miss what is more important that the members of the group also might have in common. For example, if people with borderline personality disorder (BPDs) from the midwest tend to respond to certain questions a certain way, would BPDs from Alabama respond the same way? Who knows. Would Hispanic BPDs respond the same way as Scientologist BPDs? Who knows. (By the way, what's an "Hispanic"? Does everyone who is labeled "Hispanic" share the same cultural, physical, biological, familial, socio-economic and educational characteristics? Not even close. It's not a category of people who share real traits, but a political category.) Would green-eyed BPDs tend to respond the same way as brown-eyed BPDs? Who knows. And by, the way, how do we know that they really were BPDs? Faith. Faith that the researchers isolated the defining trait, faith that if the trait were something fuzzy such as a mental disorder rather than curly red hair, that they diagnosed it correctly, faith in the DSM that there even is such a disorder... Faith. Religion. Voo-doo...
What this all means is that, like any profiling or stereotype, what groups of people as a whole tend to do means nothing as far as the truth, the facts, or the reality, when applied to any given individual -- especially in a court case. It's an hypothesis about "maybe". (This is a huge flaw in the field of applied psychology generally, why psychology is not science.) The MMPI-2 scales are generalizations or likelihoods about people (with more or less accuracy) of the sort usually not otherwise admissible in a court of law as "evidence" of anything about the specific person in question. Making things worse, the test is now roughly seven decades old, and the 567 statements are written with much odd, archaic, provincial, and outdated -- and occasionally offensive -- wording and idiom usage. The language is solidly midwest, middleclass, "white folks" circa World War II. (Does anyone have "spells" any more? Do "girls" take the MMPI-2?)
A litigant's unusual or unique traits or circumstances could very well make that litigant appear to be similar to people grouped by some negative attribute. A person with unusually virtuous character might answer similarly to groups of people who are lying or "faking good" or being overly "defensive". A person with real medical problems could match groups of people who are hypochondriacs, lying or malingering or "faking bad" (not even getting into the HIPAA and medical privacy violations here of the medically-related test questions where that's not at issue in a court case.) A person who is unusually smart or creative could end up appearing to be just like a deviant. A police officer may come off aggressive and paranoid. If a subject's first language is not English, and he or she struggles with the meanings of deliberately equivocal words, the results are further distorted. A litigant under stress in a pending court case who inadvisedly leans his or her forced choice guesses (because the individual questions, if not all of the "scales", are fairly transparent) to those that he thinks will not harm him in court if isolated items were plucked out by opposing counsel on cross-examination (always a risk) will appear to be a liar, a "catch-22". The better response to any ambiguous item, which also will avoid the admission against interest problem, is "can't say".
The clinical psychologist is supposed to use test results from a person seeking therapy to generate hypotheses regarding what might be problems requiring therapy. The forensic evaluator is supposed to do likewise, and verify the test hypotheses against additional tests and a gathering of non-test facts. It doesn't happen that way. [note] The computer printouts of hypotheses for these tests [see sample report] are like astrology readings, and can be applied any which way anyone wants to apply them, positively or negatively. The evaluator can dismiss an "hypothesis" generated by the test based on the evaluator's belief that it doesn't apply ("Hispanic men tend to have this scale elevated; it doesn't mean anything"), or, with the application of cognitive bias, discover that in fact the seemingly nice, normal, well-behaved litigant is a secret nutjob with emotional issues ("She answered similarly to people who expend a lot of energy to keep from showing their anger..." [5 pages later] "Her obvious anger"...") Like an astrology reading, it can always be applied.
People are told they should respond either "true" or "false" because "there are no right or wrong answers." But that's not actually the truth. Although no individual answer by itself may be "wrong", various answers are grouped together to make sub-scores. Together these groups of answers contribute to what is or is not -- as judged by the test assessor -- to be the equivalent of a good or bad, or, essentially right or wrong, score for the collective group. And this is without even considering the problem of an evaluator who is deliberately or subconsciously biasing his or her evaluation.
The sub-groups, or "scales", purport to tell the psychologist such things as whether the subject appears similar to or maybe is a hypochondriac, depressed, hysterical or an attention freak, psychopathic, overly masculine or feminine, paranoid, self-critical, anxious, perfectionistic, confused or schizophrenic, manic or grandiose, or introverted or extroverted. There are many other scales, with scores based on collections of answers to selected items that have been compared to the average answers of various demographic groups. Over the years, more and more scales have been concocted. For example, some scales purport to illuminate such things as whether the subject has addictive tendencies, or is repressed or "overly controlled", hostile, a liar, or giving correct answers based on whether or not he or she is consistent in answering. There are multiple scales, and subscales, disputed and controversial scales, and newly invented scales being beta-tested. (Anyone wanting to know more about these details can refer to any of the many available books and articles on the subject. This article, however, is addressing something the rest don't.)
Additionally confounding the validity (or "reliability") problem with this and other psychological tests is that the same person could answer the questions very differently at another time, in another place, in another mood, or depending on the purpose for which the person is taking the test. That's most likely because people are guessing at what the statements mean and arbitrarily choosing "true" or "false" when, if honest, they should be skipping the question as a "can't say". That's also because today, having taken a walk through a spring garden on the way to the psych's office, Joe is feeling chipper and thinks he might like the work of a florist, whereas three months ago, when it was snowing out, this possibility didn't occur to him and he sort of thought he liked mechanic's magazines. (He no longer likes mechanics magazines, having been sued for nonpayment of an inadvertent renewed subscription by one of them.) Susie of course, couldn't say whether she "feels" that she "might like" the work of a nurse because "I think that I would like to be a nurse" is nonsensical, given that she actually has an R.N. degree, has worked for five years as a nurse, and is heading into med school. She also doesn't have a clue what a "mechanic's magazine" is.
The many questions on the lengthy test and the relatively limited time apparently allotted to respond encourage the spontaneity of giving "true" or "false" first impression responses that are essentially thoughtless. People who want to be cooperative will respond in ways that they would not respond if they gave more thought. People who have gone through school taking bubble tests with objectively right and wrong answers also get used to choosing what they think might be the "best guess" depending on the way they are leaning. They will, in the rush, seize upon words or part of the sentence, which, save for the limited response choice, is really not much different from guessing ink blots. It would be kind of like showing someone aRorschach inkblot and saying "This card depicts a butterfly. True or false?" None of this is a good thing if this test is being given involuntarily to assess a litigant's personality and mental functioning, and much is at stake depending on the evaluation of the answers, whether that be a court case or employment. It is possible that the litigant could guess his or her way into a "profile" that is similar to the "profiles" of answers given by people with personalities that are nothing at all similar to the way the litigant usually is.
The psychologist is supposed to use the test taker's scores, along with other information known about the subject, to generate hypotheses about the subject and the subject's personality. That might work for someone seeking therapy (or it might not). But for someone taking the test because he or she has been court-ordered to have a forensic psychological evaluation, the test administrator is not someone who (let's be real, okay?) knows anything much about the person (the conflicting allegations and hearsay statements received from both parties are very common and not established facts or evidence, especially when those allegations have yet to be tried in court. Neither are court pleadings. Neither do a few hours of talking with and observing the litigants in an artificial setting give more than a smidgen of information about how those people act "in real life".)
All the forensic evaluator has is a menu of test result options from which to cherry pick hypotheses, which easily can be manipulated any which way the evaluator wants them to be -- and most easily in the very common event in which the psychologist refuses timely and adequately to disclose the testing materials and data in discovery). [Ever see a detailed computerized astrological summary? We all know that Taureans are stubborn, Leos are leaders, and everyone is going to have a significant family event occur soon, and some kind of negative work issue... Know what "cold reading" is? Forensic psychology is an interesting variant, but with less potential for fact-checking. Yes, yes... the practitioners are serious and have studied for years in accredited schools. Here's another religion field you might want to believe in -- astrological psychology.]
Honest, thoughtful answers by most clients will mean that the client is far more likely than not to be unable to answer 70, 100, or even 300 or more out of the 567 questions. [The author of this article is unable to answer in excess of 400 of the MMPI-2 questions. Perhaps she can't read, or is a psychotic, or is just unfortunately tainted with that malady Pearson desperately wants to prevent, near memorization of the test questions, as well as which ones are on a number of scales. Or maybe Libras just have these issues...]
Lawyers' standard advice to test takers is to "answer honestly" because lawyers have been assured by psychologists that collecting groups of the client's answers from items that most people "endorse", in other words, say "true" to, supposedly will give the psychologist insight into whether the client is a liar. (Never mind that credibility is supposed to be within the province of the trier of fact, and that stuff like polygraph tests are banned from court evidence.) A client in a court case does not want to appear to be prone to lying. And the client's lawyer, snookered by decades of psychology trade promotion, so-called scholarly articles from every side, pro and con, about strengths and weaknesses of psychological tests, and so forth, may not have given much thought to how the client should approach the situation in order to avoid looking that way.
It's not that hard, though. The client is not clearly told that "can't say" might be the most honest answer. So here goes: the correct answer very well may be "can't say". The lay public's confidence in these tests, as well as that of many mental health professionals, is based on myths deliberately promulgated by the psychology trade promotion groups and test developers and publishers. The industry that deliversquack therapies of all kinds, also has created beliefs among the public that, by using these tests, psychologists somehow are able to discern things about the personality and mental workings of an individual, in a manner similar to the way physicians might do (objective) diagnostic testing for diseases. Well they can't. This fraud is aided by the terrorism of the so-called lie detection and malingering scales as well as the marketing propaganda and reams of literature instructing, pontificating and debating over the obscure and difficult fine points of protocol and research. For the most part, this conveys false messages. The tests are nonsense on many levels. And the MMPI-2 is supposedly the least flawed and nonsensical of them all.
If you're a lawyer, stop asking psychologists how to prep clients for these tests and start giving your clients legal advice. If you're a judge, stop wasting the time and resources of the court system, the litigants and the taxpayer on psychological forensics. Stop it. Stop it now.

-- liz
This webpage is: http://www.thelizlibrary.org/therapeutic-jurisprudence/custody-evaluator-testing/forensic-mmpi2.html
Discovery Issues:http://www.thelizlibrary.org/therapeutic-jurisprudence/custody-evaluator-testing/index.html