Jessica Kerr, aka @jessitron 
, explains how modern software teams must collaborate not only with each other, but with their own tools of automation. We must build systems that are good team players. She introduces the line of representation. Presents Four prerequisites (the pillars of joint activity) and four precautions (ironies of automation). article
_jessitron's presentation clarifies a lot of background knowledge that surrounds what I presented at my poster session in the New Relic Product Offsite._
We are part of a sociotechnical system. Code, running software, & tools are all on the team impacting lives with software. All of us learn and change every day.
**Joint Activity:** Basic Compact, Mutually Predictable, Mutually Directable, Common Ground
**Basic Compact:** mutual purpose until further notice
A counterexample from aviation: there’s a story of a near miss of an airplane crash. There was a fuel leak, and the crew just happened to notice that the left fuel tank level was dropping while the engines were pulling from the right. They go look out the window, and ope! there goes the fuel! Then they observe that the rudder is way to the right, because the fuel levels threw the plane off balance, and the autopilot was just doing its business, compensating, compensating, turning more and more to the right, not telling anyone that it was near the edge of what it could do.
The lesson is: don't try harder! Talk to your team first.
The Basic Compact means, don’t swallow errors! Errors are data. Maybe don’t surface them to users, but get that information to the humans on the team. That’s how we learn.
Collaboration is not about being perfect. We work with each other within the team, so that the rest of the work can rely on the team.
**Mutually Predictable:** signal what you're doing and why.
In cars that have driver assist, there’s a thing that when you get too close to the car in front of you, it hits the brakes. Which is good -- unless you’re trying to pass that car. So you need to use your turn signal. This tells the driver-assist that you’re about to change lanes, so it can choose not to brake. It can predict you, and act in cooperation.
In any machine learning software, I want it to report on what rules fired to trigger the result. The software needs to be accountable: able to explain why it acted as it did. That will let us reach mutual predictability.
**Mutually Directable**
There’s a funny property of negotiations: the person with the least flexibility has more power. If you’re a vendor, and I’m buying software from you, and I’m a manager who can authorize up to $20k/year, then I have more power. You have more responsibility — you have a number to meet this quarter. If you don’t sell it to me for $20k, then maybe we’ll have another meeting next week... maybe. I have less flexibility, and therefore more power.
When the “computer says NO”, the computer is in charge.
Within our teams, keep the human in charge. Humans are maximally flexible, so that means the automation needs ultimate flexibility too: meaning, it can get out of the way and let the human do something else.
**Common Ground:** shared context and mental models
Common ground is essential and expensive. It is also mathematically impossible in the presence of uncertain message passing.
Counterexample: a nurse opens a patient record and clicks through six “Alert!” dialogs to get to the screen she needs. When “these two medicines are dangerous when combined” is at the same level as “this room is out of bedsheets,” then too much information is no information.
Stop pretending we can defeat complexity, and start balancing it. “Increased complexity can be balanced with increased feedback.” (Woods, Cook)
Let’s rename "error messages" to "help and guidance" messages, and teach people how the system works.
**Ironies of Automation:** smarter automation is harder to operate; exceptions are the rule; automating easy stuff makes hard stuff harder; less flexible party has more power
Your people won’t be cheaper -- they’ll be more valuable. Don’t train people to follow procedures; computers can do that. Training for procedure is like programming except way more expensive. Train for understanding, to help with problem solving.
When software is inflexible, it has all the power, and that is not what we want. So make the best path obvious and easy, but never mandatory.
When we increase the flows of learning, we keep getting better. There is no done; Our system is never going to be perfect. Your coworkers are never perfect. Your automations are never perfect. So make them collaborative.