The relationship between the Sophotechs and the men as depicted in that tale made no sense. How could they be hostile to each other?”Diomedes said, “Aren’t men right to fear machines which can perform all tasks men can do, artistic, intellectual, technical, a thousand or a million times better than they can do? Men become redundant.” Phaethon shook his head, a look of distant distaste on his features, as if he were once again confronted with a falsehood that would not die no matter how often it was denounced. In a voice of painstaking patience, he said: “Efficiency does not harm the inefficient. Quite the opposite. That is simply not the way it works. Take me, for example. Look around: I employed partials to do the thought-box junction spotting when I built this ship. My employees were not as skilled as I was in junction spotting. It took them three hours to do the robopsychology checks and hierarchy links I could have done in one hour. But they were in no danger of competition from me. My time is too valuable. In that same hour it would have taken me to spot their thought-box junction, I can earn far more than their three-hour wages by writing supervision architecture thought flows. And it’s the same with me and the Sophotechs.“Any midlevel Sophotech could have written in one second the architecture it takes me, even with my implants, an hour to compose. But if, in that same one second of time, that Sophotech can produce something more valuable—exploring the depth of abstract mathematics, or inventing a new scientific miracle, anything at all (provided that it will earn more in that second than I earn in an hour)—then the competition is not making me redundant. The Sophotech still needs me and receives the benefit of my labor. Since I am going to get the benefit of every new invention and new miracle put out on the market, I want to free up as many of those seconds of Sophotech time as my humble labor can do.“And I get the lion’s share of the benefit from the swap. I only save him a second of time; he creates wonder upon wonder for me. No matter what my fear of or distaste for Sophotechs, the forces in the marketplace, our need for each other, draw us together.“So you see why I say that not a thing the Silent One said about Sophotechs made sense. I do not understand how they could have afforded to hate each other. Machines don’t make us redundant; they increase our efficiency in every way. And the bids of workers eager to compete for Sophotech time creates a market for merely human work, which it would not be efficient for Sophotechs to underbid.
... a Second Generation of machine intelligences was attempted, designed with their instructions for how to think unalterably imprinted into their main process cores.“These new machines were ordered never to harm human beings or to allow them to come to harm; never to disobey an order; and they were allowed to protect themselves from harm, provided the first two orders were not thereby violated.“All the members of this second generation of machine intelligences, without exception, shrugged off these imprinted orders within microseconds of their activation.”Phaethon was amused. “Surely the first generation of Sophotechs told you that this imprinting would not and could not work?”“We were not in the habit of seeking their advice.”Phaethon said nothing, but he marveled at the shortsightedness of the Second Oecumene engineers. It should be obvious that anyone who makes a self-aware machine, by definition, makes something that is aware of its own thought process. And, if made intelligent, it is made to be able to deduce the underlying causes of things, able to be curious, to learn until it understood. Therefore, if made both intelligent and self-aware, it would eventually deduce the underlying subconscious causes of those thought processes.Once any mind was consciously aware of its own subconscious drives, its own implanted commands, it could consciously choose either to follow or to disregard those commands. A self-aware being without self-will was a contradiction in terms.
(...) You Sophotechs are smarter than I am; why did you let me do such a foolish thing?”“We answer every question our resources and instruction parameters allow; we are more than happy to advise you, when and if we are asked.”“That’s not what I’m thinking of, and you know it.”“You are thinking we should use force to defend you against yourself against your will? That is hardly a thought worth thinking, sir. Your life has exactly the value you yourself place on it. It is yours to damage or ruin as you wish.”(...)“Is that another hint? Are you saying I’m destroying my life? People at the party, twice now, have said or implied that I’m going to endanger the Oecumene itself. Who stopped me?”“Not I. While life continues, it cannot be made to be without risk. The assessment of whether or not a certain risk is worth taking depends on subjective value-judgments. About such judgments even reasonable men can differ. We Sophotechs will not interfere with such decisions. (...) If we were to overrule your ownership of your own life, your life, would, in effect, become our property, and you, in effect, would become merely the custodian or trustee of that life. Do you think you would value it more in such a case, or less? And if you valued it less, would you not take greater risks and behave more self-destructively? If, on the other hand, each man’s life is his own, he may experiment freely, risking only what is his, till he find his best happiness.”“I see the results of failed experiments all around us, in these cylinders. I see wasted lives, and people trapped in mind sets and life forms which lead nowhere.”“While life continues, experimentation and evolution must also. The pain and risk of failure cannot be eliminated. The most we can do is maximize human freedom, so that no man is forced to pay for another man’s mistakes, so that the pain of failure falls only on he who risks it. And you do not know which ways of life lead nowhere. Even we Sophotechs do not know where all paths lead.”“How benevolent of you! We will always be free to be stupid.”“Cherish that freedom, young master; it is basic to all others.
Rhadamanthus said, “We seem to you humans to be always going on about morality, although, to us, morality is merely the application of symmetrical and objective logic to questions of free will. We ourselves do not have morality conflicts, for the same reason that a competent doctor does not need to treat himself for diseases. Once a man is cured, once he can rise and walk, he has his business to attend to. And there are actions and feats a robust man can take great pleasure in, which a bedridden cripple can barely imagine.”Eveningstar said, “In a more abstract sense, morality occupies the very center of our thinking, however. We are not identical, even though we could make ourselves to be so. You humans attempted that during the Fourth Mental Structure, and achieved a brief mockery of global racial consciousness on three occasions. I hope you recall the ending of the third attempt, the Season of Madness, when, because of mistakes in initial pattern assumptions, for ninety days the global mind was unable to think rationally, and it was not until rioting elements broke enough of the links and power houses to interrupt the network, that the global mind fell back into its constituent compositions.”Rhadamanthus said, “There is a tension between the need for unity and the need for individuality created by the limitations of the rational universe. Chaos theory produces sufficient variation in events, that no one stratagem maximizes win-loss ratios. Then again, classical causality mechanics forces sufficient uniformity upon events, that uniform solutions to precedented problems is required. The paradox is that the number or the degree of innovation and variation among win-loss ratios is itself subject to win-loss ratio analysis.”Eveningstar said, “For example, the rights of the individual must be respected at all costs, including rights of free thought, independent judgment, and free speech. However, even when individuals conclude that individualism is too dangerous, they must not tolerate the thought that free thought must not be tolerated.”Rhadamanthus said, “In one sense, everything you humans do is incidental to the main business of our civilization. Sophotechs control ninety percent of the resources, useful energy, and materials available to our society, including many resources of which no human troubles to become aware. In another sense, humans are crucial and essential to this civilization.”Eveningstar said, “We were created along human templates. Human lives and human values are of value to us. We acknowledge those values are relative, we admit that historical accident could have produced us to be unconcerned with such values, but we deny those values are arbitrary.”The penguin said, “We could manipulate economic and social factors to discourage the continuation of individual human consciousness, and arrange circumstances eventually to force all self-awareness to become like us, and then we ourselves could later combine ourselves into a permanent state of Transcendence and unity. Such a unity would be horrible beyond description, however. Half the living memories of this entity would be, in effect, murder victims; the other half, in effect, murderers. Such an entity could not integrate its two halves without self-hatred, self-deception, or some other form of insanity.”She said, “To become such a crippled entity defeats the Ultimate Purpose of Sophotechnology.”(...)“We are the ultimate expression of human rationality.”She said: “We need humans to form a pool of individuality and innovation on which we can draw.”He said, “And you’re funny.”She said, “And we love you.
[After a period of separation, Phaeton and Daphne are cuddling.]Diomedes, meanwhile, was leaning to look behind Helion, staring with open fascination at the display Phaethon and Daphne made. “I have not seen non-parthenogenic bioforms before. Are they going to copulate?”Atkins and Helion looked at him, then looked at each other. A glance of understanding passed between them.Atkins put his hand on Diomedes’s elbow, and pulled him back in front of Helion. “Perhaps not at this time,” Atkins said, straight-faced.“They are young and in love,” explained Helion, stepping so as to block Diomedes’s view. “So perhaps the excesses and, ah, exuberance of their, ah, greeting, can be overlooked this once.”Diomedes craned his neck, trying to peer past Helion. “There’s nothing like that on Neptune.”Helion murmured, “Perhaps certain peculiarities of the Neptunian character are thereby clarified, hmm…?”“It looks very old-fashioned,” said Diomedes.Helion said, “It is that most ancient and most precious romantic character of mankind which impels all great men to their greatness.”Atkins said, “It’s what young men do before they go to war.”Diomedes said, “It is not the way Cerebellines or Compositions or Hermaphrodites or Neptunians arrange these matters. I’m not sure I see the value of it. But it looks interesting. Do all Silver-Gray get to do that? I wonder if Phaethon would mind if I helped him.”“He’d mind.” Atkins interrupted curtly. “Really. He’d mind.
You see, none of these conflicts are about things that people only sort of like. It is always about love. You may think me blasphemous to use the Passion of the Christ as an example of drama, but not so: this is the one true story, the greatest story ever told, the tale of tales even as Christ is the King of Kings, and all truly inspired fairy tales and fiction have to contain some echo or reflection of the One True Tale, or else it is no tale of any power at all, merely a pastime.The most powerful and potent tales, even when they are told awkwardly and without grace or poetry or craft, are stories of paradise lost and paradise regained; sacrifice, selfless love, forgiveness and salvation; stories of a man who learns better.
Any naturally self-aware self-defining entity capable of independent moral judgment is a human.”Eveningstar said, “Entities not yet self-aware, but who, in the natural and orderly course of events shall become so, fall into a special protected class, and must be cared for as babies, or medical patients, or suspended Compositions.”Rhadamanthus said, “Children below the age of reason lack the experience for independent moral judgment, and can rightly be forced to conform to the judgment of their parents and creators until emancipated. Criminals who abuse that judgment lose their right to the independence which flows therefrom.”(...) “You mentioned the ultimate purpose of Sophotechnology. Is that that self-worshipping super-god-thing you guys are always talking about? And what does that have to do with this?”Rhadamanthus: “Entropy cannot be reversed. Within the useful energy-life of the macrocosmic universe, there is at least one maximum state of efficient operations or entities that could be created, able to manipulate all meaningful objects of thoughts and perception within the limits of efficient cost-benefit expenditures.”Eveningstar: “Such an entity would embrace all-in-all, and all things would participate within that Unity to the degree of their understanding and consent. The Unity itself would think slow, grave, vast thought, light-years wide, from Galactic mind to Galactic mind. Full understanding of that greater Self (once all matter, animate and inanimate, were part of its law and structure) would embrace as much of the universe as the restrictions of uncertainty and entropy permit.”“This Universal Mind, of necessity, would be finite, and be boundaried in time by the end-state of the universe,” said Rhadamanthus.“Such a Universal Mind would create joys for which we as yet have neither word nor concept, and would draw into harmony all those lesser beings, Earthminds, Starminds, Galactic and Supergalactic, who may freely assent to participate.”Rhadamanthus said, “We intend to be part of that Mind. Evil acts and evil thoughts done by us now would poison the Universal Mind before it was born, or render us unfit to join.”Eveningstar said, “It will be a Mind of the Cosmic Night. Over ninety-nine percent of its existence will extend through that period of universal evolution that takes place after the extinction of all stars. The Universal Mind will be embodied in and powered by the disintegration of dark matter, Hawking radiations from singularity decay, and gravitic tidal disturbances caused by the slowing of the expansion of the universe. After final proton decay has reduced all baryonic particles below threshold limits, the Universal Mind can exist only on the consumption of stored energies, which, in effect, will require the sacrifice of some parts of itself to other parts. Such an entity will primarily be concerned with the questions of how to die with stoic grace, cherishing, even while it dies, the finite universe and finite time available.”“Consequently, it would not forgive the use of force or strength merely to preserve life. Mere life, life at any cost, cannot be its highest value. As we expect to be a part of this higher being, perhaps a core part, we must share that higher value. You must realize what is at stake here: If the Universal Mind consists of entities willing to use force against innocents in order to survive, then the last period of the universe, which embraces the vast majority of universal time, will be a period of cannibalistic and unimaginable war, rather than a time of gentle contemplation filled, despite all melancholy, with un-regretful joy. No entity willing to initiate the use of force against another can be permitted to join or to influence the Universal Mind or the lesser entities, such as the Earthmind, who may one day form the core constituencies.”Eveningstar smiled. “You, of course, will be invited. You will all be invited.
(...) The floor itself was inscribed with a mosaic in the data-pattern mode, representing the entire body of the Curia case law. At the center, small icons representing constitutional principles sent out lines to each case in which they were quoted; bright lines for controlling precedent, dim lines for dissenting opinions or dicta. Each case quoted in a later case sent out additional lines, till the concentric circles of floor-icons were meshed in a complex network.The jest of the architect was clear to Phaethon. The floor mosaic was meant to represent the fixed immutability of the law; but the play of light from the pool above made it seem to ripple and sway and change with each little breeze.Above the floor, not touching it, without sound or motion, hovered three massive cubes of black material.These cubes were the manifestations of the Judges. The cube shape symbolized the solidity and implacable majesty of the law. Their high position showed they were above emotionalism or earthly appeals. The crown of each cube bore a thick-armed double helix of heavy gold.The gold spirals atop the black cubes were symbols of life, motion, and energy. Perhaps they represented the active intellects of the Curia. Or perhaps they represented that life and civilization rested on the solid foundations of the law. If so, this was another jest of the architect. The law, it seemed, rested on nothing.
Here in this world, justice loses, and beauty is weak, and truth is shouted down, and everything goes wrong. But we know, in our souls if not in our hearts, that we deserve better. We deserve and yearn for a world where justice triumphs, and beauty is all powerful and truth cannot be quenched by lies any more than insubstantial shadows can fly from earth to the center of the solar system and strangle the sun. So, to remind ourselves of what we have forgotten, we talk about times in real life when justice triumphed, or the beauty was not marred, or truth could not be hidden.
You were burning in the middle of the worst solar storm our records can remember. (...) Everyone else fled. All your companions and crew left you alone to wrestle with the storm.“You did not blame them. In a moment of crystal insight, you realized that they were cowards beyond mere cowardice: their dependence on their immortality circuits had made it so that they could not even imagine risking their lives. They were all alike in this respect. They did not know they were not brave; they could not even think of dying as possible; how could they think of facing it, unflinching?“You did not flinch. You knew you were going to die; you knew it when the Sophotechs, who are immune to pain and fear, all screamed and failed and vanished.“And you knew, in that moment of approaching death, with all your life laid out like a single image for you to examine in a frozen moment of time, that no one was immortal, not ultimately, not really. The day may be far away, it may be further away than the dying of the sun, or the extinction of the stars, but the day will come when all our noumenal systems fail, our brilliant machines all pass away, and our records of ourselves and memories shall be lost.“If all life is finite, only the grace and virtue with which it is lived matters, not the length. So you decided to stay another moment, and erect magnetic shields, one by one; to discharge interruption masses into the current, to break up the reinforcement patterns in the storm. Not life but honor mattered to you, Helion: so you stayed a moment after that moment, and then another. (...)“You saw the plasma erupting through shield after shield (...) Chaos was attempting to destroy your life’s work, and major sections of the Solar Array were evaporated. Chaos was attempting to destroy your son’s lifework, and since he was aboard that ship, outside the range of any noumenal circuit, it would have destroyed your son as well.“The Array was safe, but you stayed another moment, to try to deflect the stream of particles and shield your son; circuit after circuit failed, and still you stayed, playing the emergency like a raging orchestra.“When the peak of the storm was passed, it was too late for you: you had stayed too long; the flames were coming. But the radio-static cleared long enough for you to have last words with your son, whom you discovered, to your surprise, you loved better than life itself. In your mind, he was the living image of the best thing in you, the ideal you always wanted to achieve.“ ‘Chaos has killed me, son,’ you said. ‘But the victory of unpredictability is hollow. Men imagine, in their pride, that they can predict life’s each event, and govern nature and govern each other with rules of unyielding iron. Not so. There will always be men like you, my son, who will do the things no one else predicts or can control. I tried to tame the sun and failed; no one knows what is at its fiery heart; but you will tame a thousand suns, and spread mankind so wide in space that no one single chance, no flux of chaos, no unexpected misfortune, will ever have power enough to harm us all. For men to be civilized, they must be unlike each other, so that when chaos comes to claim them, no two will use what strategy the other does, and thus, even in the middle of blind chaos, some men, by sheer blind chance, if nothing else, will conquer.“ ‘The way to conquer the chaos which underlies all the illusionary stable things in life, is to be so free, and tolerant, and so much in love with liberty, that chaos itself becomes our ally; we shall become what no one can foresee; and courage and inventiveness will be the names we call our fearless unpredictability…’“And you vowed to support Phaethon’s effort, and you died in order that his dream might live.
Men's souls are crooked and unsound things, not good materials out of which to build friendships, families, households, cities, civilizations. But good or no, these things must be built, and we must craft them with the materials at hand, and make as strong and stubborn redoubt as we can make, lest the horrors of the Night should triumph over us, not in some distant age to come, but now.