Computers are not the only type of machine that presents this problem. Organizations, including corporations, are also data-processing machines. Data comes in — the availability and price of supply, customer demand, rules, economic outlook — and the company uses the data to figure out the most efficient way to create goods and services.

Like any good AI, companies have feedback loops and cycles of self-improvement. While sales figures, balance sheets and bonuses all provide interim feedback, there is one feedback cycle that dominates: stock value. High share price equals success. A low share price is a sign of failure or (it will be argued) fails to fully reflect value. CEOs never say, “Yes, our share price fell, but that is because we have taken a number of steps to help employees, customers and the community and to reduce our carbon footprint, and those steps lowered our financial return, but we thought that was the right balance.” Why can’t companies proudly trade off share value for human rights or environmental sustainability?

Capital markets expect corporations to create shareholder value in the first instance — at least over the long term. Of course, it is often the case that share value coincides with social and environmental value — like a company that develops an innovative source of renewable energy. So corporations often “do well by doing good.” But here is the rub: If the First Law of Corporate Process is “create shareholder value,” then corporations will not do good if they cannot do well by doing so.

Moreover, they will do “bad” if it is legal and increases shareholder return; it’s called “market equilibrium.”

And bad can be quite profitable, because the benefits of irresponsible choices accrue entirely to the company involved, while social and environmental costs are spread throughout the economy. This misalignment incentivizes contribution to climate risk, inequality and corruption — issues creating suffering and instability around the globe, and ultimately harming ordinary investors, who diversify their portfolios and depend on rising markets for their retirement and other long-term goals.

Thus, one corporation’s choice to use a cheap but dirty fuel will minimally affect it from an environmental perspective, but it will receive all the cost savings, allowing it to undercut competitors and increase returns to its shareholders. This race to the bottom is repeated daily in the shareholder-first paradigm, where competition among companies is allowed to trump concern for the systems in which those companies are embedded.

We need to reprogram our financial system by establishing a new first principle: The First Law of Corporate Dynamics should be to preserve the systems in which our economy is embedded. Only if that condition is met should companies begin to deliver returns to shareholders.

The shareholder value principle evolved because business must attract private capital in order to produce the goods and services that we rely upon. But subjugating that shareholder value to an imperative to do no harm would add needed common sense to the corporate operating system. This will benefit most investors, who are diversified across the markets and who bear the externalized environmental and social costs as shareholders in other companies and as human beings who live in the world.

But if shareholder returns continue to be the first priority, corporations will continue to seek advantage over competitors in the battle for customers and capital, whatever the cost, and will continue to fight against laws and regulations that protect our planet and our future whenever it is profitable to do so. Unlike the invasion of paper clips, the threats from the shareholder-first machine are real, present and getting worse: If we are going to address climate risk, growing inequality and systemic instability, we must change the corporate operating system.