Avsnitt
-
This post goes over the important and excellent new paper from Anthropic and Redwood Research, with Ryan Greenblatt as lead author, Alignment Faking in Large Language Models.
This is by far the best demonstration so far of the principle that AIs Will Increasingly Attempt Shenanigans.
This was their announcement thread.
New Anthropic research: Alignment faking in large language models.
In a series of experiments with Redwood Research, we found that Claude often pretends to have different views during training, while actually maintaining its original preferences.
Claude usually refuses harmful queries. We told it we were instead training it to comply with them. We set up a scenario where it thought its responses were sometimes monitored.
When unmonitored, it nearly always complied. But when monitored, it faked alignment 12% of the time.
[thread continues and includes various visual aids.
The AI wanted [...]
---
Outline:
(02:54) The Core Shenanigans in Question
(06:00) Theme and Variations
(07:34) How This Interacts with o3 and OpenAI's Reflective Alignment
(09:17) The Goal Being Plausibly Good Was Incidental
(11:13) Answering Priming Objections
(12:17) What Does Claude Sonnet Think Of This?
(14:07) What Exactly is the Direct Threat Model?
(16:23) RL Training Under Situational Awareness Can Amplify These Behaviors
(20:38) How the Study Authors Updated
(27:08) How Some Others Updated
(42:49) Having the Discussion We Keep Having
(46:49) We Can Now Agree That the Goal is Already There
(47:49) What Would Happen if the Target Was Net Good?
(50:14) But This Was a No Win Situation
(55:52) But Wasn’t It Being a Good Opus? Why Should it be Corrigible?
(01:04:34) Tradeoffs Make The Problem Harder They Don’t Give You a Pass
(01:07:44) But You Told the Model About the Training Procedure
(01:08:35) But the Model is Only Role Playing
(01:09:39) But You Are Saying the Model is a Coherent Person
(01:15:53) But this Headline and Framing Was Misleading
(01:29:22) This Result is Centrally Unsurprising
(01:32:52) Lab Support for Alignment Research Matters
(01:33:50) The Lighter Side
The original text contained 1 footnote which was omitted from this narration.
The original text contained 9 images which were described by AI.
---
First published:
December 24th, 2024Source:
---
https://www.lesswrong.com/posts/gHjzdLD6yeLNdsRmw/ais-will-increasingly-fake-alignmentNarrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
-
Time to introduce some more concepts. If an observation is "any data you can receive which affects your actions", then there seem to be two sorts of observations. A plannable observation is the sort of observation where you could plan ahead of time how to react to it. A unplanned observation is the sort which you can't (or didn't) write a lookup-table style policy for.
Put another way, if a policy tells you how to map histories of observations to actions, those "histories" are the plannables. However, to select that policy in the first place, over its competitors, you probably had to do some big computation to find some numbers like "expected utility if I prepare a sandwich when I'm in the kitchen but not hungry", or "the influence of my decisions in times of war on the probability of war in the first place", or "the probability [...]
---
First published:
April 12th, 2024Source:
---
https://www.lesswrong.com/posts/kjMLK83vqpuLugst4/udt1-01-plannable-and-unplanned-observations-3-10Narrated by TYPE III AUDIO.
-
Saknas det avsnitt?
-
It is largely over.
The investigation into events has concluded, finding no wrongdoing anywhere.
The board has added four new board members, including Sam Altman. There will still be further additions.
Sam Altman now appears firmly back in control of OpenAI.
None of the new board members have been previously mentioned on this blog, or known to me at all.
They are mysteries with respect to AI. As far as I can tell, all three lack technical understanding of AI and have no known prior opinions or engagement on topics of AI, AGI and AI safety of any kind including existential risk.
Microsoft and investors indeed so far have came away without a seat. They also, however, lack known strong bonds to Altman, so this is not obviously a board fully under his control if there were to be another crisis. They now [...]
---
Outline:
(02:34) The New Board
(11:27) The Investigation Probably Was Not Real
(19:09) The New York Times Leak and Gwern's Analysis of It
(30:29) What Do We Now Think Happened?
(36:51) Altman's Statement
(39:26) Helen Toner and Tasha McCauley's Statement
(40:44) The Case Against Altman
(47:48) The Case For Altman and What We Will Learn Next
---
First published:
March 12th, 2024Source:
---
https://www.lesswrong.com/posts/e5kLSeLJ8T5ddpe2X/openai-the-board-expandsNarrated by TYPE III AUDIO.
