Re: <nettime> rage against the machine

[tbyfield <tbyfield@panix.com>]

On 29 Mar 2019, at 6:32, William Waites wrote:

> It seems to me it is a question of where you draw the system boundary. 
> If the
> system is an aeroplane that is flying, then the recording device is 
> not part of
> the control loop and it is not a cybernetic tool in that context. If 
> the system
> is the one that adjusts and optimises designs according to successes 
> and
> failures, then the recording device definitely is part of the control 
> loop and
> it is a cybernetic tool.
This is where 'classical' cybernetics drew the line. Second-order 
cybernetics, which came later (late '60s through the mid/late '70s) and 
focused on the 'observing systems' rather than the 'observed systems,' 
drew that line differently. I don't have a solid enough grasp of the 
work of people like Heinz von Foerster and Gordon Pask to say with any 
certainty how and where they'd draw it, but in general their approach 
was more discursive and less, in a word, macho. So they'd be less 
interested in the isolated 'technical' performance of a single plane or 
a single flight and more interested in how people made sense of those 
technical systems — for example, through the larger regulatory 
framework that Scot spoke of: regular reviews of the data generated and 
recorded during every flight. Scot's note was a helpful reminder that 
the purpose of a black box is just to duplicate and store a subset of 
flight data in case every other source of info is destroyed. In that 
view, it doesn't matter so much that the black box itself is input-only, 
because it's just one component in a tangle of dynamic systems — 
involving humans and machines — that 'optimize' the flight at every 
level, from immediate micro-decisions by the flight staff to 
after-the-fact macro-analyses by the corporation, its vendors, 
regulatory agencies, etc. The only reason we hear about (or even know 
of) black boxes is that they fit neatly into larger cultural narratives 
that rely on 'events' — i.e., crashes. But we don't hear about these 
countless other devices and procedures when things go right. Instead, 
they just 'work' and disappear into the mysterious 'system.'
(As a side note, this brings us back to why Felix's overview of how 
different regimes contend with complexity is so stunning — 
'complexity' is a product of specific forms of human activity, not some 
mysterious natural force:
	https://nettime.org/Lists-Archives/nettime-l-1903/msg00127.html

His message reminds me very much of what I love about Marshall Sahlins's 
work and, in a different way, of Moishe Postone's _Time, Labor, and 
Social Domination_: basically, 'complexity' is immanent.)
But back to my point: Morlock's original take about the Boeing 737 
crashes and how this thread unfolded, or at least to one of the areas 
where Brian and I seemed to part ways. It's easy to lose sight of the 
larger dimensions and implications of these human–machine assemblages. 
For example, media coverage very quickly focuses on detailed specialist 
subjects, like the design of the MCAS system that's failed on 737s; 
then, a few days later, it suddenly leaps to a totally different order 
and focuses on regulatory issues, like the US FAA's growing reliance on 
self-regulation by vendors. We've grown accustomed to this kind of 
non-narrative trajectory from countless fiascos; and we know what 
sometimes comes next, 'investigative journalism,' that is, journalism 
that delves into the gruesome technical details and argues, in essence, 
that these technical details are metonyms for larger problems, and that 
we can use them as opportunities for social action and reform of 'the 
system.'
This journalistic template has a history. I know the US, other nettimers 
will know how it played out in other regions and countries. A good, if 
slightly arbitrary place to start is Rachel Carson's 1962 book _Silent 
Spring_ and Ralph Nader's 1965 book _Unsafe at Any Speed_. (It isn't an 
accident that Carson's work opened up onto environmental concerns, 
whereas Nader's was more geeky in its focus on technology and policy: 
there's an intense gender bias in how journalism identifies 'issues.') 
From there, the bulk of ~investigative journalism shifted to militarism 
(i.e., Vietnam: defoliants like Agent Orange, illegal bombing 
campaigns), political corruption (Watergate), intelligence (mid-'70s: 
the Pike and Church committees looking into CIA abuses etc), nuclear 
power (Three Mile Island), military procurement, policy and finance 
(HUD, the S&Ls, etc), etc, etc. I've left out lots of stuff, but that's 
the basic drift, although these decades also saw an immense rise of 
investigative focus on environmental issues. Whether the results of all 
that environmental work have been satisfying I'll leave as an exercise 
for the reader.
That template goes a long way toward explaining how and why journalistic 
coverage of 'tech' is so ineffectual now. It can't get its arms around 
*the* two big issues: the extent to which the US has become a laboratory 
for national-scale experiments in cognitive and behavioral studies, and 
the pathological political forms that 'innovation' is enabling around 
the world. The US has ironclad regulations and norms about experimenting 
on human subjects, which are enforced with brutal mania in academia. 
But, somehow, we haven't been able to apply them to pretty much 
everything Silicon Valley does. Instead, we get ridiculous kerfuffles 
about Facebook experimenting with making people 'sad' or the tangle 
around Cambridge Analytica, which is both real and borderline-paranoiac. 
The blurriness of that boundary is a by-product of, if you like, the 
micro-epistemological divide that separates general journalism and 
investigative journalism. We're terrible at 'scaling' this kind of 
analysis or down: either from subtract to concrete, by saying 'WTF is 
going on?!' and channeling it into broad, effective limitations on what 
infotech companies can do, or from concrete to abstract, by catching 
companies like FB doing dodgy stuff then hauling them in front of 
Congress and asking 'where else does this approach apply?' (Europe has 
been much better at this, but the cost of doing so is other train wrecks 
like the fiasco with Articles 11 and 13.)
That was precisely the divide that started this thread, when Brian 
attacked Morlock over whether the MCAS system was a discrete 
implementation of AI. Brian was right, but my point was that it doesn't 
matter because Morlock's broader point was right and (imo) matters much 
more. Does the MCAS mechanism in Boeing's 7373 implement AI properly 
speaking? Who cares? Are Boeing and all the other aircraft manufacturers 
drowning in misplaced faith in machine 'intelligence' in every aspect of 
their operations? Yes. And does that misplaced faith extend far beyond 
individual companies? Yes. And the result is a systemic failure to think 
critically about where this misplaced faith is leading. The
standard technophilic response is to universalize ideas like 
'technology,' 'innovation,' and 'complexity,' and to argue that they're 
inexorably built into the very fabric of the universe and therefore 
historically inevitable. But none of that is true: what *is* true, as 
Felix argued, is that 'complexity' is an externality of human activity, 
and that we seem to be doing a crap job of imagining political economies 
that can strike a balance between our discoveries, on the one hand, and 
human dignity, on the other.
We need some sort of global Pigouvian tax on complexity: a way to siphon 
off the profits generated by messing around the edges of complexity and 
invest it in, for lack of a better word, simplicity. If we last long 
enough, we might even get it; but, like Morlock, I fear it'll take a 
true global catastrophe for people to realize that. Why? Because the 
best 'actually existing' institutions we have now — representative 
government, the media, extra- and supra-national entities like 
coordinating bodies, and (LOLZ) stuff like nettime — all get lost when 
they try to cross the divide between general and specialized forms of 
knowledge.
And that, BTW, is why second-order cybernetics is so relevant: it was 
interested in observing systems rather than observed systems. It's also 
why, wording aside, Morlock's first impulse was right: addressing this 
problem will require an end to corporate obfuscation of liability and 
identifying exactly who, within these faceless edifices, is making 
specific choices that betray the public trust. He doesn't seem to think 
we can manage it; I think we can. But squabbling over the fact that he 
said 'burning people at the stake' won't help — which is, again, why I 
asked:
> And that begs an important question that leftoids aren't prepared to 
> answer because, in a nutshell, they're allergic to power: what *would* 
> be appropriate punishments for people who, under color of corporate 
> activity, engage in indiscriminate abuses of public trust.
Cheers,
Ted

#  distributed via <nettime>: no commercial use without permission
#  <nettime>  is a moderated mailing list for net criticism,
#  collaborative text filtering and cultural politics of the nets
#  more info: http://mx.kein.org/mailman/listinfo/nettime-l
#  archive: http://www.nettime.org contact: nettime@kein.org
#  @nettime_bot tweets mail w/ sender unless #ANON is in Subject: