Eliezer S. Yudkowsky wrote:
>
> Lyle Burkhead wrote:
> >
> > The "genie" hypothesis is simply that AI systems with at-least-human
> > intelligence will work for us for free. Discussions of AI have not gone
> > beyond it, because it is a general idea that has nothing to do with the
> > details of how an AI is created.
>
> Incorrect. See the section on goal systems in "Coding a Transhuman AI",
> or the summary in "Singularity Analysis". (Yes, you *can't* give AIs
> orders after a certain point - for technical, not logical, reasons.)
The only reason we ask for pay is because of the way our motivations evolved. There is absolutely no reason why an AI has to have a hunter/gatherer or agrarian motivational set. An AI certainly does not need to get laid or provide a home for its kids... An AI that may ask for pay will still understand that any compensation may be calculated on the AI's cost of 'living', which is measured in kilo-watt hours and amortized Moore fractions. If AI's are easily replicated, then the supply of AI labor will be high and this will drive the cost per work unit down.
>
> > Nanotechnologists assume that Genies will exist. That's what distinguishes
> > Drexlerian Nanotechnology from ordinary technology. The nanotechnology meme
> > isn't really about atoms. The key paragraphs are in the section called
> > Accelerating the Technology Race on page 81:
>
> Okay, now this is why everyone is shouting "Straw man!". Drextech
> assumes that the forms of matter (as opposed to matter) become
> duplicable resources, and that a few new manipulations (healing a cell,
> creating custom-formed diamonds, etc.) become possible.
>
> Genie machines are unnecessary. Knocking them down does nothing to
> disprove the utopia envisioned in _Engines_.
>
> > Either you have to program the robots, or you don't. If you do, then using
> > them to build a skyscraper will not be free of labor costs -- far from it.
>
> Yes, but you can build ten million skyscrapers as easy - or as hard - as one.
My comments at crit org on the geniebusters site replicate these
comments:
There are already code generating engines out there. Once such code
generators are developed for CAD/CAM systems, then with a sufficient
database of past work done, the user only has to set goals and other
parameters and the assemblers do the rest:
a)determining solutions to the goal requirements b)selecting the best solution c)CADding out the design for that solution and the process tomanufacture it.
Each step has its own requirements, but there is no reason why nanites cannot do all of them.
>
> > On the other hand if you don't have to program them, then they have crossed
> > the line that separates agents from automatons. They have crossed over to
> > our side, and the work _they_ do amounts to the same thing as "human
> > labor." Either the robots are automatons, in which case you have to program
> > them, or they are agents, in which case you have to pay them. Skyscrapers
> > will never be free, because we will always have to make an effort to focus
> > our minds and make things happen -- or the robots will have to make the
> > same effort to focus _their_ minds and make things happen.
>
> It sounds to me like exactly the same argument would rule out the
> Industrial Revolution. You've just proved that the cost of everything
> is an eternal constant. Not only that, you've also proved that I can't
> have Netscape Navigator on my desk because it took a whole company to
> produce it.
>
> Nanotechnology is a material technology that does not intrinsically
> require or imply intelligence enhancement; even though IE is discussed
> in _Engines_ as both consequence and tool, it is not *necessary*.
> Nanotech is simply an enormously powerful tool, that can be developed
> like any other tool; and used to create a Utopia or blast the planet,
> more likely the latter; with or without IE.
Correct. You can easily have your nanites be rather dumb drone type organism which obey the orders of the queen AI. Its merely a problem needing a client/server solution in a manufacturing environment.
>
> > Without Genies, the ability to make things out of atoms is just an
> > extension of present-day technology, not dizzying at all. Molecular
> > manufacturing without Genies is just agribusiness.
>
> All material business suddenly has the same economics as information;
> write it once, use it forever. The world becomes the Internet. That's dizzying.
And notice how Monsanto is so scared about its improved breeds of gengineered seed that it felt it had to engineer in a suicide gene to prevent the plants from being able to reproduce outside of Monsanto control. This will be the means of IP protection in the nanotech future.
>
> > Without Genies, there will be no sudden "assembler breakthrough." Instead
> > of emerging in a sudden breakthrough, nanotechnology will emerge
> > continuously from present-day technology, over a period of decades, step by
> > laborious step, each step involving an effort of concentration in a human
> > mind.
>
> Why can't we get immediate AIs who then drive all the way to
> Singularity, whether we like it or not? And why couldn't the immediate
> effects of assembler technology, relatively simple apps like
> ultracomputers and the Ultimate Xerox, have such vast impacts - like $3
> trillion of venture capital pouring into nanotech, and the ability to
> actually play around in the molecular world, and the ability to run
> evolutionary computer models of assemblers, and all the applications we
> couldn't anticipate in advance - compress decades into months?
he doesn't beleive in Moore's Law.
>
> > There are no Genies and never will be. This is a logical point, not a
> > technical point.
>
> If it really is a logical point, not a technical question of AI
> motivations, then you're just playing with tautologies.
Sounds more like Mr. Burkhead treats it like a point of dogma.
>
> > It's not a question of what can or can't be done with
> > atoms, or what can or can't be done with computers. I'm not saying AI will
> > never exist. What I'm saying is that it doesn't matter -- any entity with
> > at-least-human intelligence (artificial or not) won't work for free. To the
> > extent that a robot makes independent decisions, it will have to be dealt
> > with as an entity that makes independent decisions. A group of robots that
> > could build a skyscraper by themselves would be indistinguishable from a
> > contractor, and would have to be dealt with as such.
>
> You are simply wrong. Our selfishness is quite independent of our
> intelligence, and in fact interferes with it; this is a technical
> question of goal systems.
Exactly as I said earlier here.
Mike Lorrey