All individuals are incentivized to do the wrong thing. CEO's are incentivized to sell data to make money. Engineers are incentivized to create bad software via making the people who pay them happier. Users are incentivized to give up their data in exchange for a free service. Politicians are incentivized by political donations and getting information they aren't constitutionally privileged to get.
Doing the ethical thing requires making less money (or losing money) for nearly all parties involved. Doing the right thing requires sacrifice.
In a happy world, the CEO has long term vision and sees the long term cost of loss of trust. The engineers see the ethical problem or betraying their peers and use their pocket veto to do the right thing. The user should be willing to pay a reasonable cost to receive the service they use. Politicians should see that the individual incentives harm the whole and create regulations that disincentivize the poor behavior.
Non-rhetorically: How do we ensure as a society that we live in the latter, and not the former?
I think the only way is to raise a generation of people who see data aggregation/brokerage and user-device-hijacking as immoral, much like how we raised people staring last century to view eugenics as immoral. The weird approach that Stallman has always taken is the only one that can win, as crazy as it seems.
We need religious fervor. We need to decry bundling spyware and "analytics" with free alarm clock apps as evil. Finally, we need "know-it-when-I-see-it" type Software Decency laws that we can leverage to fine evildoers into oblivion (of course, those will follow automatically if we succeed in moralizing the issue).
Twitter and tech companies are not the first industry to have ethical problems like this. This is a problem we've solved many times in the past with strict regulations and laws. Finding ethical people is expensive, but writing laws is cheap.
If actual data privacy laws existed in the US, this situation would never have happened. In the linked twitter thread, he says that "legal" said it was ok. That right there is the safety valve that we can control to keep corporations in check.
Why doesn't my local supermarket price gouge us when there's a hurricane about to hit? That's an obvious way to increase profits. In fact, if it weren't illegal to do that, I'd argue that any CEO who didn't do that for "ethical reasons" should be fired and possibly even sued by shareholders.
That’s what governments and laws are for. Expecting companies to work for the greater good is naive at best; and we shouldn’t really get upset when they don’t. That’s not their motivation, as you’ve highlighted.
Strong legislation and independent legislators are what’s needed
While I agree completely, it seems that legislative ability is captured by the upper class, reinforcing the cycle of self-enrichment at the cost of global good.
I guess the root question is: how should middle class people wage class warfare?
>Non-rhetorically: How do we ensure as a society that we live in the latter, and not the former?
Incentive alignment. Nothing short of hardcore government regulation of personal data, and the remuneration for (opt-in) usage of said data, will change anything.
Doing the ethical thing requires making less money (or losing money) for nearly all parties involved. Doing the right thing requires sacrifice.
In a happy world, the CEO has long term vision and sees the long term cost of loss of trust. The engineers see the ethical problem or betraying their peers and use their pocket veto to do the right thing. The user should be willing to pay a reasonable cost to receive the service they use. Politicians should see that the individual incentives harm the whole and create regulations that disincentivize the poor behavior.
Non-rhetorically: How do we ensure as a society that we live in the latter, and not the former?