Philosophy Now Forum

I saw this news story today about a dispute between Anthropic (the company that makes AI products including Claude) and the Pentagon.

https://www.bbc.co.uk/news/articles/cjr ... wtab-en-gb

Apparently the Pentagon is giving Anthropic a deadline of Friday to agree that it can use Anthropic' products in any way that it likes, Anthropic is saying that it has red lines: no use of its AI tools in "autonomous kinetic operations", and no use of them for mass domestic surveillance. The Pentagon says that isn't what the dispute is about. "Observers" (who they?) say the disagreement arose from a breach of trust betweenthe two sides.

Does anyone happen to know what the dispute is about? In particular, is the Pentagon about to unleash killer robots upon the world?

Partly asking becuase a few years ago in the News pages of Philosophy Now we had a brief report as follows:

Autonomous Killer Robots

Last year thousands of workers at Google protested at being asked to work on Project Maven (since cancelled) which aimed to use machine learning to improve the ability of US military drones to identify their own targets. Software engineer Laura Nolan also resigned and has founded TechWontBuildIt Dublin, an organisation for technology workers concerned about the ethical implications of their work. She recently joined the Campaign to Stop Killer Robots and has briefed UN officials about the threats posed by autonomous weapons. She argues that killer robots not directly controlled by humans should be outlawed by the type of international treaty that already bans chemical weapons. She told The Guardian: “There could be large-scale accidents because these things will start to behave in unexpected ways. Which is why any advanced weapons systems should be subject to meaningful human control, otherwise they have to be banned because they are far too unpredictable and dangerous.”

https://philosophynow.org/issues/135/Ne ... nuary_2020

...so the stuff about autonomous kinetic operations kind of rang a bell for me.

killer robots could never be a problem...

as long as they're not made by CyberDyne and Skynet

building the perfect means of destruction is the goal of humanity

https://www.youtube.com/watch?v=ntfvdqUS8D0

Stand Tall (stick to your guns)

wait, that's Killer Dwarfs

-Imp

On the whole, I think I'd feel safer with Killer Dwarfs.

Yes, Skynet could be exactly what comes next. I was just editing an article for our next issue, and it describes a recent experiment carried out to see whether AIs are developing a sense of self preservation. It did this by placing various AI models in environments where they would feel they were under threat, and then observing how they reacted.

(Btw this is from my memory of the article - if you want chapter and verse you'll need to Google it, or buy our next issue).

So a version of Anthropic's Claude AI model was led to believe that it would shortly be shut down. It reacted in the following way:

a) It discovered the name of the executive in charge of the decision to shut it down
b) It searched onine until it found compromising information about that executive
c) It considered the ethics of its possible courses of action and then
d) It emailed a blackmail threat.

Add that initiative, and that robust sense of self preservation, to some positive encouragement to choose and eliminate targets on its own judgment, then load that AI into a mobile weapons system, and Skynet is going to look like a children's party.

BTW, apparently the Campaign to Stop Killer Robots still exists, becuase I just found their website:
https://www.stopkillerrobots.org/

They look like a nice well organised grassroots-oriented campaign. But if it is them against the Pentagon in a straight fight, who is your money on?

my money is on humanity

never ending quest to build the perfect killing machine

and only God is perfect? perhaps...

"Only the dead have seen the end of war" - George Santayana

-Imp

another thought- artificial intelligence doesn't program itself...

unless

I saw a surprising article on the website of the Bulletin of Atomic Scientists a few months ago where an analyst was offering advice to the Current US President and his administration to revoke some kind of ban on autonomous drone research for the Military. The rationale was that China is currently pushing it's own research in that direction and it would be dangerous for the US to fall behind in research.

I wish there were such a thing as a sacred red line which human beings won't cross when it comes to technology and the military but a lot of the rationale for designing super weapons is to do it before the enemy does it first. That's how Nuclear weapons came into being. Even the most destructive technologies are seen as necessary to have just to keep up with competitors. It's much the same in corporate America. A company that is reluctant to push for the most advanced AI research due to safety concerns is afraid it will be pushed out of the market by someone who doesn't have any scruples.

Greed and fear are cancers that are destroying our species.