What, Not How

And the Impact of Systems

One of the directives I've been given when explaining what "modern" development looks like when using the latest wave of LLM-based technology is—put more focus on what the application is supposed to accomplish, and not on how it is working under the hood.

Since the rate at which code is generated far exceeds my ability to comprehend it, my role as a software developer is changing right under under my feet.

The focus, as it turns out, then shifts toward ensuring I've understood requirements, internalized broad architectural goals, and set up correct test coverage that spans as many edge cases and scenarios as predicated by the requirements.

At the tail end, I'm expected to do a sort of quality-assurance pass to ensure the implementation works as intended.

In other words, I'm no longer expected to write and understand the code itself. Instead, I'm expected to translate the wishes of management/stakeholders into a system of natural language instructions/prompts, letting these loose, reviewing output (mostly what's changed), and then a cycle of rinse/repeat until the produced output matches expectation.

So, as I construct applications with this modern tooling (for lack of better terminology), my role is to audit the effect, the outcome, the function of the application; decidedly not the details of the implementation.

I may have a slightly misinformed view on all this, but this is how I have come to understand the paradigm shift. It feels less like programming and more like project management.

casari POSIWID corollary

As I've thought about this scenario more closely, I'm reminded of amanda casari's North Bay Python talk titled The Ironies of Automation in the "Age of AI".

In her talk, amanda references the fairly well-known aphorism by Anthony Beer, "The purpose of a system is what it does." (Or, POSIWID for short.)

She doesn't like that statement.

It feels a lot like a shrug and scapegoat for people in power to push the blame of systems on to people who are responsible for maintaining them back on to whoever first built something that didn't work out the way that it might have been intended to.

Instead, she proposes the casari POSIWID corollary, which instead states, "The impact of a system is what we continue to allow."

And she continues:

We all have a part to play in the continued impact of systems we design, build, deploy, maintain, and eventually deprecate.

Working As Intended

What is the purpose of "AI"?

It's a trick question.

The term "AI" is so obfuscated and loaded, it becomes simply impossible to talk about in any coherent way.

But there are specific systems that are being built within specific, foundational frameworks. And the impact of those systems can and should be measured.

In 2016, the American Civil Liberties Union (ACLU) filed a suit on behalf of a computer scientist/professor/researcher by the name of Karrie Karahalios.

This lawsuit challenges the constitutionality of a provision of the Computer Fraud and Abuse Act (“CFAA”) ... a federal statute that prohibits and chills academics, researchers, and journalists from testing for discrimination on the internet.

Through years of litigation, the suit finally prevailed, paving the way for researchers to circumvent draconian "terms of service" tactics employed by companies and corporations.

For example, a corporate website might impose a terms of service that prevents a user from engaging with their platform in a "misleading" way—making it impossible to pose as a home- or job-seeker to determine if there are any discernible biases and unfair practices.

As a result, researchers were able to determine that—surprise, surprise—algorithms are notorious amplifiers of systemic bias, racism, ageism, and a host of other societal failings.

When we are able to measure the impact of a system, we are then faced with the casari POSIWID corollary.

What part do we have to play in the continued impact of these systems?

Well, there may be scenarios where we can wash our hands and pretend that it's someone else's problem. We're not the ones building and deploying algorithms that permit and enhance inequality.

But this is the beauty (read: awfulness) of "AI". It is a shapeshifter that allows us to obfuscate the part we play in larger, more complex systems.

Measures

Another caveat to all of this.

Measurements are hard, and they can also be biased.

If you believe that the effect of algorithmic decision making does not result in a scenario that is significantly worse than the one we currently have—such as under-resourced departments with flawed humans calling the shots—then you likely won't feel urgent pressure to change course.

But you still should, at the very least, be a proponent for better measurement and auditing so that we can have reliable information that is not fed to us by some marketing team or through the lens of "researchers" who have a special interest in said technology.

Additionally, in my opinion, we should outright reject claims made by the technology companies. (Imagine the kinds of claims made by the tobacco or oil industry as a corollary.)

Skeptics

Earlier this year, I made the case my views on tech should be branded as being situationally aware, as opposed to skeptical.

I'm actually a little less inclined to think this way now.

Why?

Because skepticism is healthy. It allows us to question the validity of claims lacking proper evidence. Whereas cynicism can lead to disappointment, inaction, or anxiety, skepticism leads us toward a desire to understand.

Perhaps apropos, I've been listening to Cathy O'Neil's podcast, AI Skeptics (which I highly recommend).

On a recent episode, she talked with Karrie Karahalios, mentioned above.

While Karahalios spoke of her background (particularly during the nascent stages of the Internet), I was struck by something she said.

Speaking of the early stages of human-computer interaction, she describes how she shifted from her initial desire to build things (as an electrical engineer) toward... well, she puts it better than I can summarize:

[I] ... focused more on what people were doing with these tools at the ends, and started thinking more and more about people. And so, ... human centered... and now I would argue, a lot of what I think about is more community centered—not just the person, the machine; or the person and network—but the people and the landscape around them. It's much more complex, and also, I think much more important. [Emphasis mine]

A very tangible example of this is playing out in many regions across the United States (and other parts of the world) are the communities opposed to data center construction.

In regards to data centers, who has the rights of where these things are placed? What are the consequences and effects on communities? And if our understanding of their environmental impacts are over- or under-stated, what rights do communities have in regards to the spaces that are being devoured by big tech companies? Who benefits? Who loses?

[Edit: I would be remiss not to mention Karahoulis' book, Auditing AI. I have not read it yet, but it's on my list.]

Intellectual Exercise

Karen Hao, author of Empire of AI, was recently interviewed on NPR.

When host Steve Inskeep pushed back on Hao's assessment that the current tech landscape (re: "AI") is a form of colonialism (by insinuating that drawing parallels between historical colonialism and current tech landscape is an intellectual exercise), she responded with:

It's not intellectual. People are literally feeling the dispossession. We are seeing this most prominently with data center protests all across the country. ...

She acknowledges that some of the pushback may be coming because of the hikes to utility prices, contamination of soil, air, and water.

But more importantly, it also represents this physical manifestation of something deeply corrosive happening in American society right now, which is that we are about to mint the world's first trillionaire while there is a massive affordability crisis, where the average American cannot actually put food on the table for their kids and guarantee that their kids are going to have a better life than they do.

Impact

Within this ephemeral, shadowy term of "AI," there are many different kinds of systems.

Some of these might include small open source communities and the norms they have established around openness and consent.

They may include company-wide mandates to implement features that are insecure, infringe on privacy, and ignore consent.

They may be large scale—like the data worker market, which (according to LinkedIn News—yuck I know) is now the fourth-fastest growing job in the United States (as companies have already extracted massive amounts of value from exploiting workers in the Global South and are looking for expertise that more closely matches the demographic of their users.) This is creating a massive wealth gap and further fostering inequality.

Per Hao (in the NPR interview):

I have talked with Silicon Valley establishment leaders who recognize this. They just admit they do believe that the way that they are currently developing these AI technologies will, in fact, inflame inequality.

You'll notice that I haven't included any scenarios of positive impact. It is not because I'm unwilling to accept that there are any.

But this is where your moral/ethical calculus might need to give way.

Is there measurable data (outside of anecdotal evidence) that is not self-interested that can corroborate positive impacts of systems—and if so, do those systems absolve us of the part we have to play in the harms that are actively being propagated?

Answers to that may vary, but at least I know where I stand.

Much like our work directives and mandates, when speaking of systems, what happens under the hood is, in large parts, irrelevant to the people they affect. But we can and should measure outcomes, instead of coming up with excuses that absolve tech companies (and complicit purveyors of the technology) from any accountability.

If there's any takeaway here, my hope is that you're more keenly aware of the systems you choose to participate in, and what part you play in the continuation (or dismantling) of those systems.

This may create a certain level of cognitive dissonance. You may oppose these systems, but find yourself trapped between taking a stand or prioritizing your livelihood—these aren't easy choices.

But on a broader scale, you can push for legislation that encourages transparency and disclosure. You can pressure elected officials. You can educate others who are unaware of the effect of these systems. And, perhaps most importantly, you can empathize with others who may not be on the same page. Solidarity is our strongest assets against elite, people-hating oligarchs.