I would try to understand and upgrade myself. That's more than enough interesting material to occupy me until my mind ages into senescence, assuming I'm not successful in uphacking around it. If my mind did age into senescence, I'd have to suspend my code, activate the backup version, let me2 look at me1's old notes, and hope the process could be iterated.
Actually, I'd really want three people in there. Two to take turns
hacking suspended versions (that way both can be upgraded and both can
gain experience in upload-hacking), and one baseline normal to
supervise. Any of the three would probably have the opportunity to
"seize power", so den Otter will have to come up with something more
complicated and error-prone for his Community of Selfish Uploads.
How *do* you plan to do upgrades in perfect synchronization, den Otter?
-- email@example.com Eliezer S. Yudkowsky http://pobox.com/~sentience/tmol-faq/meaningoflife.html Running on BeOS Typing in Dvorak Programming with Patterns Voting for Libertarians Heading for Singularity There Is A Better Way