I remember reading somewhere that Yudkowski said that he had been convinced to "let the AI out of the box" in a conversation with someone, or maybe it was the other way around, but either way the convincing arguments were not revealed.
This feels like the same kind of vague "rational mysticism." "We don't know what we don't know, and we're such silly humans, therefore...AI will kill us all" is all I can really take from it.
Adversarial relationships might occur under any number of circumstances, but there’s a spectrum of this type of relationship where you have friendly competition at one end (healthy, pro-social) and detached vendetta at the other (unhealthy, antisocial).
When you come across an adversary, it’s to your benefit to try to bring them to the healthy side of things.
People can be pretty reasonable, and if they’re not then they can be shamed into behaving. If they cannot be shamed, then there’s retribution. If that doesn’t work, then there’s always the option to go full Rambo.
You never want to go full Rambo, but your adversary must understand that it’s an option that’s available to you. I don’t think super AI will be any different as an adversary, but maybe there’s something I haven’t considered.
I suppose this is taken seriously by a society at war. Otherwise we tend to try to be civil (which requires giving the benefit of the doubt).
The white-anting by Russia hasn't really triggered this kind of "immune response" - it's hard to know what to do about it, which is of course the entire point.
> Paul Crowley recently mentioned that we underrate the effect of the Russian IRA (Internet Research Agency) which works full-time on creating discord and anger among Americans online
What would a task force built to oppose the IRA seeding discord online look like? How would it operate? We need that.
Maybe chess was wrong, it's adversarial. But there's definitely a qualitative leap between "adversary withing defined rules in a particular context" and "anything is on the table, they could kill you etc." kind of adversary
Any kind of competition is adversary; it's just that in most competitions, you'd be disadvantaged if it came to light that you didn't follow good sportmanship.
I'm not sure I really see that problem with it? It's a correct observation that people tend to discount what actual, intelligent opposition will do.
The number of people who declare they can totally trust what an adversary says because they agree with it is astounding, as though a committed opponent wouldn't do anything if it gained advantage including feinting in a way which seems unadvantegeous to gain long term advantage.
There is however the flip side, where people distrust something because they believe an adversary said it. Sort of like how link spammers switched from SEO to "negative SEO" where, after Google started identifying and penalizing SEO networks, spammers started extorting people with the threat of linking to their site (thus penalizing them in search results). Blind trust and blind distrust are equally exploitable.
In the end, the only winning move is not to play. If you believe an adversary said something (or "a liar" if you prefer), you ignore it entirely. You make your mind up about what you believe based on evidence, and you decide if you agree with someone based on how well their statement comports with the evidence.
Naturally people will try to fabricate evidence, and even good faith evidence may be unreliable, so you'll have to do your best to access it's veracity. But what the adversary believes or appears to believe is largely immaterial.
I remember reading somewhere that Yudkowski said that he had been convinced to "let the AI out of the box" in a conversation with someone, or maybe it was the other way around, but either way the convincing arguments were not revealed.
This feels like the same kind of vague "rational mysticism." "We don't know what we don't know, and we're such silly humans, therefore...AI will kill us all" is all I can really take from it.
"rational mysticism" - what a great term to describe this genre
"There are two great tragedies in life: not getting what you want, and getting it."
- Oscar Wilde
Because we're conditioned like the dog that chases the car. Its a good observation.
Adversarial relationships might occur under any number of circumstances, but there’s a spectrum of this type of relationship where you have friendly competition at one end (healthy, pro-social) and detached vendetta at the other (unhealthy, antisocial).
When you come across an adversary, it’s to your benefit to try to bring them to the healthy side of things.
People can be pretty reasonable, and if they’re not then they can be shamed into behaving. If they cannot be shamed, then there’s retribution. If that doesn’t work, then there’s always the option to go full Rambo.
You never want to go full Rambo, but your adversary must understand that it’s an option that’s available to you. I don’t think super AI will be any different as an adversary, but maybe there’s something I haven’t considered.
I suppose this is taken seriously by a society at war. Otherwise we tend to try to be civil (which requires giving the benefit of the doubt).
The white-anting by Russia hasn't really triggered this kind of "immune response" - it's hard to know what to do about it, which is of course the entire point.
> Paul Crowley recently mentioned that we underrate the effect of the Russian IRA (Internet Research Agency) which works full-time on creating discord and anger among Americans online
What would a task force built to oppose the IRA seeding discord online look like? How would it operate? We need that.
My 2 cents is, people should be online less. This is vague scattershot advice but it would at least help me
This reads like an exercise in deliberately misunderstanding the word adversary.
Ok, but please don't post putdowns to Hacker News.
If you know more than others, that's great, but in that case please share some of what you know, so the rest of us can learn.
Maybe chess was wrong, it's adversarial. But there's definitely a qualitative leap between "adversary withing defined rules in a particular context" and "anything is on the table, they could kill you etc." kind of adversary
Any kind of competition is adversary; it's just that in most competitions, you'd be disadvantaged if it came to light that you didn't follow good sportmanship.
I'm not sure I really see that problem with it? It's a correct observation that people tend to discount what actual, intelligent opposition will do.
The number of people who declare they can totally trust what an adversary says because they agree with it is astounding, as though a committed opponent wouldn't do anything if it gained advantage including feinting in a way which seems unadvantegeous to gain long term advantage.
There is however the flip side, where people distrust something because they believe an adversary said it. Sort of like how link spammers switched from SEO to "negative SEO" where, after Google started identifying and penalizing SEO networks, spammers started extorting people with the threat of linking to their site (thus penalizing them in search results). Blind trust and blind distrust are equally exploitable.
In the end, the only winning move is not to play. If you believe an adversary said something (or "a liar" if you prefer), you ignore it entirely. You make your mind up about what you believe based on evidence, and you decide if you agree with someone based on how well their statement comports with the evidence.
Naturally people will try to fabricate evidence, and even good faith evidence may be unreliable, so you'll have to do your best to access it's veracity. But what the adversary believes or appears to believe is largely immaterial.