Here's a question I keep circling back to: what happens the day a machine has to make a judgment call, not just a calculation? Not "what's the fastest route" but "is this the right thing to do." We're closer to that day than most of us admit, and I don't think we've built the muscle for it yet.
AI stopped being a novelty a while ago. It's woven into how we search, write, drive, diagnose, and decide — quietly, without ceremony. The harder question isn't whether AI is capable. It's whether it's aligned with anything we'd recognize as a moral compass. So let me offer you a thought experiment, not a finished blueprint: a "Master Control Algorithm" (MCA), a layer that checks every consequential AI decision against a set of core human values before that decision becomes an action. And beyond any single system, imagine something more ambitious still — a shared reference point many AI systems could consult, a common ethical dictionary for machines. I'll call it a Central AI Values Server, or CAIVS.
Neither of these exists, and I'm not aware of anyone seriously building them at this scale. But thinking them through is useful, because it forces a question every AI developer eventually has to answer: whose values, exactly, are we encoding — and how? For the grounding, we've written elsewhere about what core values actually are and why they matter. Here, I want to push into the mechanics.
The idea of a Master Control Algorithm
Every AI system already makes value-laden decisions, whether or not anyone designed it to. A content recommender decides what's worth your attention. A hiring tool decides who looks qualified. A self-driving car, in a split second, decides whose safety matters more. None of these are neutral. The question is whether the values behind them were chosen deliberately or simply inherited from whatever data happened to be lying around.
What the MCA would actually do
Picture it less like a rulebook and more like a compass a ship's captain checks before altering course. The MCA wouldn't make decisions itself. It would sit between the AI's raw output and the real world, evaluating a proposed action against a defined set of values and either waving it through or sending it back. A filter, not a brain.
It would need to bend, not just hold
A rigid ethical checklist ages badly — think of how quickly "acceptable workplace conduct" has shifted even within a single career. So the MCA, to be worth anything, would need to learn and adjust as norms shift, without losing its footing on the values that don't move: things like honesty, harm reduction, basic dignity. Holding both at once — firm on fundamentals, flexible on application — is the hard design problem here, not a footnote.
Why this can't be an engineering project alone
This is where I'd push back on how most AI ethics gets discussed, as if it's a technical problem with a technical solution. It isn't. Encoding "don't cause harm" into working software needs philosophy to define harm, psychology to understand how humans actually weigh competing goods, and sociology to account for cultures disagreeing on the answer. A team of engineers alone will build something that looks rigorous and behaves strangely the first time it meets a situation nobody anticipated.
A useful, imperfect analogy
Think of the MCA as a constitution rather than a statute book. A constitution states principles — equality, due process — and leaves courts to work out what they mean case by case, generation after generation. Statutes try to specify everything in advance and go brittle the moment reality doesn't match the wording. An MCA built like a constitution, with judgment built in rather than engineered out, would age far better than one built like a rulebook.
Translating intuition into logic
Most of what we know about right and wrong, we know as a feeling before we know it as a principle. You sense something is off before you can say why. AI has no access to that felt sense — only whatever proxy we manage to encode. The real work of an MCA is translation: taking something as slippery as moral intuition and rendering it into something a system can weigh against alternatives, without losing what made the intuition trustworthy in the first place.
Choosing the values worth encoding
None of this matters if we can't agree on what the values actually are. Some are broad enough that almost nobody argues with them — "do no harm" is a good example. Others need to be specific enough to draw an actual line in an actual situation, which is where the disagreement starts. We've explored the values already shaping today's AI systems elsewhere. Here, I want to look at what a more deliberate, future-facing set might include.
Transparency
Where it already shows up: Serious AI labs increasingly try to make their systems explainable — not just outputs, but training data, methodology, and known limitations. OpenAI's charter doesn't name transparency as one of its four core principles, but it shows up as a supporting commitment throughout — the pledge to publish safety research, for instance, and to be transparent about any changes to the charter's own principles down the line.
Why it will matter more, not less: As these systems get harder to fully understand even for their own creators, being able to say plainly how a decision was reached becomes the thing trust is built on. Without it, users and regulators are just asked to take AI's word for it.
Fairness and non-discrimination
Where it already shows up: Careful teams reduce bias through diverse training data, deliberate design choices, and regular auditing. It's slow, unglamorous work, and easy to skip.
Why it will matter more, not less: AI now touches decisions about who gets a loan, an interview, a diagnosis. Bias baked into a system doesn't stay contained to one bad outcome — it replicates, at scale, faster than any single biased human could.
Beneficence — actively doing good
Where it already shows up: In healthcare especially, AI is oriented toward a clear good: better diagnostics, better treatment matching, faster triage.
Why it will matter more, not less: As AI's reach extends well past healthcare, "do no harm" quietly stops being enough. The more embedded these systems become, the more we should expect them to contribute something, not merely avoid making things worse.
Privacy preservation
Where it already shows up: Techniques like differential privacy let models learn from data patterns without exposing the individuals inside that data.
Why it will matter more, not less: Every new use case for AI seems to want more data, not less. The tension between what a system could know about you and what it should be allowed to know isn't going away — it's a live conflict between safety and freedom, and it needs deliberate answers, not default ones.
Continuous learning and adaptation
Where it already shows up: Most modern AI systems are, by design, built to keep learning from new data rather than freezing at deployment.
Why it will matter more, not less: A system that can't adapt becomes actively misaligned over time, not just outdated — because the world it was trained on keeps moving without it.
Accountability and responsibility
Where it already shows up: Regulation and industry guidelines are slowly catching up, pushing developers and deployers to own the consequences of what they build and release.
Why it will matter more, not less: Mistakes are inevitable at this scale — the question is whether there's a clear mechanism to catch them and correct course. And as transhumanism gains ground and the line between human augmentation and machine autonomy continues to blur, "who's accountable here" will get harder to answer, not easier — the augmented human, or the machine?
Collaboration and harmony
Where this is heading: As AI moves further into shared workspaces and households, it should be built to complement human judgment rather than compete with or quietly override it.
Why it will matter more, not less: The most interesting gains tend to show up at the boundary between human and machine capability, not on either side alone — but only if the collaboration is designed deliberately, rather than assumed.
Choosing values isn't really a rule-setting exercise. It's a statement about what kind of future we're trying to build, made concrete one design decision at a time.
How something like this could actually be built
This part is for readers who want the mechanics. If you're here for the concept rather than the architecture, skip ahead to the next section — nothing downstream depends on this.
Building an MCA means treating ethical evaluation as an architectural layer, not an afterthought bolted on at the end. Here's roughly how the pieces would need to fit together.
A layered structure
- Input layer — where raw data, sensor readings, and user commands first arrive.
- Pre-processing layer — where that input is cleaned, filtered, and shaped into something usable, with noise stripped out.
- MCA layer — the evaluation core. Every processed decision passes through here and gets weighed against the defined values before it's allowed to proceed.
- Execution layer — where cleared actions actually run, whether that's a machine learning model, a control algorithm, or physical robotics.
Ethical branching in the decision tree
Inside the MCA layer, you'd expect something like a decision tree where each branch carries its own ethical weighting depending on context — a medical decision evaluated differently from a logistics one, say, even if the underlying model is the same.
A feedback loop that actually gets used
Every decision the MCA approves or blocks needs to be logged and reviewed, not just recorded and forgotten. That history is what lets the system — and the humans overseeing it — refine the values framework as real cases surface edge conditions nobody thought to specify in advance.
An escape hatch for the genuinely hard cases
Some decisions won't have a clean answer inside the system itself. For those, the MCA could reach out to an external reference — a values database, published guidelines, or a shared resource like the CAIVS concept below — for a second opinion rather than guessing.
A way to override it, carefully
In genuine emergencies — a safety-critical or medical scenario where seconds matter — there has to be a way to override the MCA, automatically under tightly defined conditions or through direct human intervention. But every override needs logging just as rigorous as every approval, so it can be reviewed afterward rather than quietly forgotten.
It has to be fast
None of this matters if it can't run in real time. An ethical layer that adds noticeable lag will get bypassed or quietly disabled the first time it's inconvenient. Distributed processing and edge computing aren't optional extras — they're what keeps the safeguard from becoming the thing people route around.
Keeping it current
A values framework that doesn't update is really just a fossil wearing a values framework's clothes. As norms shift and technology moves, the MCA would need scheduled review, and possibly machine-learning-assisted refinement, to stay genuinely current rather than merely present.
Someone has to own that review, and this is where the thought experiment gets uncomfortable rather than merely technical. A council of experts, presumably. But appointed by whom? An independent global body, a not-for-profit, a democratically elected panel with representation across nations? I don't have a clean answer, and I'd be suspicious of anyone who claims they do.
Where this runs into trouble
No framework like this arrives without friction. Genuinely hard cases will produce gray areas where the "right" call isn't obvious even to the humans who designed the system — and in those moments, the MCA would need to escalate rather than guess.
If you've spent much time with tools like ChatGPT, you may have noticed a pause before it answers something sensitive or ethically loaded, as if some review is happening in the background. Whether that's true or just a perception, it points to something real: that kind of checkpoint could become a deliberate safeguard rather than an incidental byproduct.
But that raises an uncomfortable question of its own: who validates the people doing the validating? We've already seen, in other contexts, how content moderation and "fact-checking" functions can drift into gatekeeping — with material dismissed at the time later turning out to have real merit. If a values layer for AI is going to lean on human review at all, that review needs its own accountability, or it just relocates the trust problem.
Which is really the argument for transparency as a non-negotiable, load-bearing value in any future Master Control Algorithm — not a nice-to-have, but the thing that keeps the rest of the system honest.
Where value-aligned robotics might be heading
The MCA isn't a finished answer. It's one way of thinking about a problem we're going to be living with for a long time regardless of which specific architecture wins out. What I'd defend is the underlying instinct: that values need to be a deliberate, engineered layer in these systems, not an emergent accident of whatever data they happened to be trained on.
A shared reference point: the Central AI Values Server
Here's the harder version of the same problem. Even if one company builds a thoughtful values layer into its own AI, that says nothing about the thousands of other systems being built alongside it, each with its own assumptions baked in. One possible answer — speculative, and I mean that — is a Central AI Values Server: a shared hub any AI system could query via an API, checking proposed decisions against a common, agreed-upon set of values.
Agreeing on a short list
The idea would only work with real restraint — perhaps three to five values that enough of humanity could actually stand behind: something like fairness, empathy, transparency. You'll notice those overlap with the list above; that's not an accident. But getting from "these sound right to me" to genuine cross-cultural consensus is a different order of difficulty entirely, and I don't think it's one we should pretend is close to solved.
Checking decisions as they happen
Once those values were established, the server would function in something close to real time: an AI system proposes an action, the server checks it against the shared values, and returns an approval or a rejection.
Built to move with us
Like the MCA, this can't be static. As the values themselves shift over time — and they will — the server would need updating in step, or it becomes exactly the kind of fossil we warned against above.
The obvious vulnerability
Centralizing something this consequential creates an obvious target. A single point of failure that every AI system on earth consults becomes a single point of catastrophic failure if it's compromised, manipulated, or simply wrong. Security and integrity here aren't details to solve later — they're the whole game.
Why this is genuinely hard to build
Getting governments, companies, and technologists to agree on anything at this scale, across legal systems and cultural traditions that don't share the same starting assumptions, is a monumental undertaking before you've written a line of code. And the real-time, global-scale technical demands sitting underneath it are not small either.
A direction worth naming, even if we can't build it yet
I don't think the CAIVS gets built anytime soon, if ever, in the form described here. But naming the direction still matters. A world where AI systems share some common ethical reference point, rather than each inventing its own from scratch, is a meaningfully different — and better — world than the one we're currently building toward by default.
Where this leaves us
The real question was never how intelligent our machines can become. It's whether we can get them to reflect the values we actually hold, rather than the ones we happened to leave lying around in the data. The Master Control Algorithm and the Central AI Values Server are both thought experiments, not blueprints — ways of making a slippery problem concrete enough to argue about.
And it genuinely isn't a problem for technologists or ethicists to solve on our behalf. It needs the widest possible range of voices, doubts included, because the doubts are often where the real insight lives. Our shared values are the only reliable compass we have for a future this uncertain — not because they're perfect, but because they're ours, worked out together rather than handed down.
Consider this an invitation more than a conclusion — to keep asking these questions rather than assuming someone else already has the answer. The goal isn't machines that merely compute. It's machines, and the people building them, that hold something like conscience in common.
Discover your own core values
Take the free, research-backed Values App assessment — nine minutes, max.
Discover your values