How the Conveniently Measurable Becomes Truth

Some scientific fields including life and social sciences make use of surrogate measurements or surrogate test environments to make sweeping assertions and incremental steps forward in the field. Easily measured phenomena is used in place of more representative ones to indicate a target event. This type of analysis actually exists within a larger class of fallacious, inductive approaches to experimentation. In less admirable situations, these may be wielded to push a pet theory or wrangle unwarranted funding or clout. In even more extreme cases, such approaches are taken to accelerate drugs through clinical trials with less than desirable outcomes.

An example of a surrogate observation: I see smoke coming through a chimney; therefore, someone is cooking. Then again, the house may be burning down. Then again again, the dryer exhaust may be ported to the chimney. Then again, the chimney in question may silhouette an adjacent house’s chimney which is emitting the smoke. You see the issue.

Note, I do not completely condemn such observations in discourse or private research, as it can be useful and informative in a pinch, particularly with bleeding edge research. Although, even in those contexts, actively conceding the shortcomings should be acknowledged and sustained. Further, it can be argued that every measurement is a surrogate or inductive measurement, but that is a different discussion and is not productive here. Their use is mostly problematic in public and potentially massively influential channels, or when those wielding it stand to incur substantial financial gains, as it might be easier to ignore poor experimental design in such situations. I also do not mean to target one field or entity for being more or less cavalier with such measurements; I am addressing it more generally as a pervasive phenomenon in human thought that is more detrimental in some domains more than others.

While many erroneous conclusions drawn from surrogate observations are harmless, unlimited use in science – particularly in clinical/pre-clinical domains – can have disastrous outcomes[1]. Irresponsible and frankly lazy invocation[2] of such tools can be found in the annuls of sociology and psychology. This type of analysis can take many subtle forms with varying degrees of validity, some more warranted than others. Of all technical disciplines, sociology seems to be one of the largest practitioners of surrogate measurements; further, it is commonly used in combination with plausibility, dogma, and narrative in place of logical or evidence-based formulations.

Even big dumb, unverifiable concepts like “law of least mental effort” are common in associated literature; what could that law mean, given our impoverished conception of the human brain? Such seemingly colloquial concepts are first engendered through intuition, accepted in key circles, then “confirmed” in biased, black box studies which are indiscernible from rudimentary polls. Some are not even formally “confirmed”; nonetheless, they permeate the field and become a priori truth, perpetuated through incestuous, referential affirmation in literature. I present this as empirical information, first and second hand. An example is comprehensively demonstrative:

“The ‘law of least mental effort’ clearly has intuitive appeal, in part from the strong analogical relationship between mental and physical effort (for discussion, see Eisenberger, 1992). It also makes sense from a normative perspective, since a bias against mental effort would steer cognition toward more efficient tasks (see Botvinick, 2007), and might preserve limited cognitive resources (see Muraven & Baumeister, 2000). Remarkably however, despite its widespread application, the ‘law of least mental effort’ appears never to have been subjected to a direct experimental test.”[2]

Over time, surrogate measurements become gold standard and dogma, with frequency increasing proportionally with field abstractness. Psychology bad, sociology worse. The caveat disclaimer that previously accompanied them fades. Some or all experts in the field will undoubtedly understand the shortcomings of the approach; however, this is of no concern. The public message and impact will have been administered, in some cases, shifting public sentiment and policy, in other cases, directly damaging individuals’ health. The course of the field is always affected for reasons of precedence and intrigue. If nothing else, outcomes serve as a distraction from rigorously uncovered truth.

These types of measurements have no place at the table with the scientific method, at least not in its ideal form (although this form may be rare). Solutions exist; simple statistical approaches can be used to determine how closely the observed variable represents the target event, what sample size is needed, etc. However, this can only get a researcher to “these data are meaningful”, that’s it. It does not afford the researcher free-form narrative describing causation. Such statistical approaches are even uncommon in studies and are replaced with norms such as experiments always being carried out in triplicate rather than representative sample sizes. Such practices become more apparent in hard problems such as human consciousness and behavior, where metaphysics and storytelling operates under the guise of science.

Acknowledging surrogate observations as an issue is not merely quibbling and is not just propped up by scarce anecdotes; rather, it exists as a more fundamental issue of reasoning and thought, crudely derivable without examples, if you will afford me the time.

There exists a slight of hand, a subtle logical slip in the application of surrogate observations. Let’s entertain the following problem:

  • I would like to know “X”
  • “Y” unequivocally informs me of “X”
  • “Y” is not conveniently observable for reasons of cost, time, or shortcomings of current technologies
  • “Z” is observable
  • “Z” has been shown to correlate/covary with “X”
  • “Z” is a convenient measurement

CONCLUSION: We will measure “Z” to get information about “X”.

This decision is almost presupposed, obvious, and irrefutable in some situations. After “Z” is chosen by me, a precedence is set and others can now freely observe “Z” without much thought or ridicule. This all because measuring “Y” is impractical and not productive, as it were. “Y” disappears from the set of options for characterizing “X”. Over time, within the field, “Y” becomes “X”.

With this thought in mind , a cursory survey of vaguely technical fields may frighten you.

My recommendation, in the face of sparse data, impoverished models, or poor measurement technologies, make no assertions or musings. Acknowledge their shortcomings and develop better ones. To do any different is pure mysticism.

And you thought this was a godless era.

  1. Echt DS, Liebson PR, Mitchell LB, Peters JW, Oblas-Manno D, Barker AH, et al. Mortality and morbidity in patients receiving encainide, flecainide, or placebo. The Cardiac Arrhythmia Suppression Trial. N Engl J Med. 1991;324:781–788.
  2. Kool W, McGuire JT, Rosen ZB, Botvinick MM. Decision Making and the Avoidance of Cognitive Demand. J Exp Psychol Gen. 2010 Nov; 139(4): 665–682.

ConMari

A review of “The life changing magic of tidying up”

Summary: All physical or psychological problems originate from belongings or organization of those belongings. Aspire to a superficial lifestyle created by ads and focus your whole being around that. Obsess over organizing, selecting, and reorganizing you possessions. Distract or deflect from issues in your life by focusing on this process. Do this religiously; rather, do this as a religion.

The title implies an arrogance, like a motivational speaker is about to tell me with certainty how I’ve been living my life all wrong. At first sight, the book looks way too thick for what the title betrays as the topic under consideration. The title length is likely foreshadowing the endless equivocation within, I appreciate the title to the extent that it serves as a warning sign. If American capitalism and advertising haven’t already immersed me in distracting trivialities enough, this book surely will. Before I crack the spine of a book surely entrenched in idolizing belongings, I reflect on how many terrible tragedies are probably occurring in the world.

Each section taken individually warrants almost the same list of criticisms as the others, so this should be a terse review. The author heavily uses “my clients” phraseology as a mechanism for corroborating and legitimizing the content. Rather than constructing logical, convincing assertions, she simply leverages ethos persuasion listing innumerable unnamed affirming past clients. This in combination with the deterministic section titles, “Storage Experts Are Hoarders” cultivate an air of arrogant certitude of which is sustained throughout the book.

The undertone of many sentences speak blatantly as if to say, “I already have all this worked out; I know this; my clients know it. Why don’t you?”. The style is akin to infomercials, inundated with oversimplifications, unverifiable testimonies, and romanticized results. It is in the style of a deluded, self-declared prophet of trivialities, here to speak contemptuously and profoundly to the laymen that have not yet been enlightened. This becomes clear once you parse the vague idiomatic language, anecdote, and endless contradictions. “My clients often want me to teach them what to put where. Believe me, I can relate, but unfortunately, this is not the real issue”. And just you wait, the source and resolution of all your pain will soon be revealed.

The section continues with a criticism of apparently sinister and subversive storage product ad campaigns with their catchy simple phrases, promising you salvation from your mess. This is of course revealed to be a lie. The author unknowingly described a perfect analogy for this book. A cursory search reveals the author’s company simply named KonMari- rather ConMari- as it literally sells storage boxes. How can such egregious contradiction be sustained and permitted. Put plainly, she vehemently denounces storage container companies, then starts a storage container company. Rest assured, they’re only $89. The nice site has a humanizing mission statement that ham-fistedly attempts to obscure the underlying motive. A perfect testament to the contradiction and arrogance embraced by the author.

Certain sections that betray a particularly dogmatic tone are bolded, although none have a hint of profundity and are the literary equivalent of taking a distasteful family vacation photo: self-important, abundant, imposing, useless, and the outcome of which is really just paper garbage except to the individuals involved.

The author makes heavy use of personal anecdote, chronicling the journey from folly to… different folly. She asks the reader to stop obsessing over abundant or redundant belongings, which, initially, sounds admirable. However, this is immediately followed by a deep fetishization of few belongings, a plan for enshrining your things. This so portrays the deeply rooted nefarious facets of capitalism underlying much of the book. The stories successfully humanize her and appeal to an imagined audience, what a dark imagination she must have. These stories attempt to add authenticity and act as stand-ins for genuine deduction, secretly leading the reader down the same fallacious, inductive path as the author. One sentence could have really replaced all the equivocation: “I remember when I was dumb, depressed, and disorganized like you, but I figured it out and so can you if you follow me”. Such anecdotes need not necessarily be true to serve the purpose, which is to provide a seemingly innocuous mechanism to bully the reader into believing the profundity and validity of her statements. It really is one long infomercial; see if these statements sound familiar:

  • “Once you experienced the powerful impact of a perfectly ordered space, you, too, will never return to clutter. Yes, I mean you!”
  • “Wouldn’t you like to live this way, too?”

It becomes clear early that the book’s basis, the proselytized personal lessons learned and respective repercussions if not learned, originate from a conflation of personal preference and objective physical laws. The author generalizes subjective lessons to everyone and declares them self-evident laws. Something of an attempted pseudo-psychological brainwash. If I find chewing a straw to be stress relieving, should I be so sure of the inherent efficacy of this behavior that I should fill up 200+ pages with descriptions of my straw chewing endeavors. Nope.

I was going to do a full review, but the book turned out to be cyclic and unsubstantial. If it wasn’t for the 6 million copies sold, I might have even felt immediate pity. However, the burden of responsibility of criticizing such a nonsensical, self-important, and arrogant work falls on us. And anyway, I feel more pity for our society that it allowed such contrived, misleading garbage to rise to such ubiquity. Are we so lost that we are desperate enough to look for answers in vague, equivocating quibbling about the ordering and organization of our trinkets?  This book is such a high-order artifact of the current state of capitalism in America. It is a bible for the church of Things and owned things. Listen, and you will be saved from your manic-depression, alienation, failure, and all other psychological stresses; I promise. This sort of pseudo psychological diagnosis is dangerous and misleading; it is implied throughout the book:

  • “Putting their house in order positively affects all other aspects of their lives, including work and family”
  • “… the relief won’t last because you haven’t addressed the true cause of your anxiety.”
  • “Visible mess helps distract us from the true source of the disorder”
  • “… I read books on psychology”
  • “This deeply affects your mind…”
  • “I too once lacked confidence. What saved me was tidying”

Thesis: Do not enshrine belongings. Face personal problems with clarity, honesty, and vigilance not distraction. Find your way to psychological contentment through valid avenues, not by following a cult that tells you to clean your room. If you reach contentment by chewing a straw, understand it is not for everyone.