While is common to think of AI as being simply "a computer that is very intelligent", AI researcher Nick Bostrom has identified three different ways in which a superintelligence might exceed human abilities: speed, collective intelligence, and quality. In most cases, people tend to think of superintelligences as a combination of speed and quality types, which is not a bad guess. However, these two types are very different, and it is worth recognizing the differences.
Speed intelligence is both alien and overwhelmingly familiar. When you use a calculator you are using a machine that is much dumber than you are to do one thing much faster than you can. Calculators can complete most calculations tens of thousands of times faster than we can. This speed makes calculators and computers highly valuable to us, but given enough time and motivation, a human could easily take a series of ones and zeros and use the appropriate conversion rules to render by hand anything rendered on a monitor. We would never do this, as this is a ridiculously inefficient way to play Skyrim, but we certainly could. It is only an overwhelming increase in speed that makes this sort of thing worth doing.
Quality intelligence is what we tend to credit ourselves with; we are better than animals not because we are fast, but because we are creative, flexible, have better language skills, are better at problem solving, have deeper conceptual understanding, and etc. A dog that could think 10,000 times as fast as the average dog might keep the house safer, but will not suddenly figure out how to multiply fractions. We can figure out how to multiply fractions because we are different in a significant, qualitative way.
It is noteworthy that a lot of what we might credit as quality intelligence is actually built on a foundation of collective intelligence. Any given human might be able to come up with a method of multiplying fractions, but given free range to do as they wish, most humans wouldn't. Humans are very good at passing information from person to person, often, as in the case of early mathematics education, against a person's will. However, even if our quality intelligence exists within a framework of collective intelligence, it is still quality: it is something more than just breeding enough brains to unlock the 'fractions' skill set.
The quality of intelligence is clearly important, but also a bit mysterious. We developed complex language, art, music, and tool-making abilities long before the agricultural and industrial revolutions. It is evident that these qualities were not sufficient to cause the 'revolutions' that are the basis of modern technology; we futzed around for hundreds of thousands of years before we invented domesticated plants or metalworking.
This illustrates two interesting points about quality intelligence. The first is simply that ability doesn't mean action. We can be fairly certain that zebras are not equipped to compose novels, but if we genetically equipped them to do so, it does not follow that they would write. As it happens, most systems are highly evolved to target a specific task, and we don't have to have an epiphany to figure out what to use our auditory processing pathways for -- although we do require real-word training in their use, or we will grow up sound-blind.
Secondly, many specialized structures are repurposed to give us new intellectual abilities; this is hardly surprising, as this is a large part of how biological evolution works. However, it is useful to purposely reframe human quality intelligence as often being random: we did not decide that we should multiply fractions and therefore develop the neurological structures that enabled us to do so, instead, we decided that multiplying fractions might be useful, and found that we could. Likewise, we did not evolve to be able to read; we evolved a highly advanced pattern recognition system, and found that it worked for reading.
~~~
Quality superintelligence is quality intelligence beyond that held by humans. And as such, this already exists in many splintered forms. For example, computers already have 'neurological' structures that specialize in memory, and record certain types of information more effectively than any human can, because the way computers remember things is fundamentally -- qualitatively -- different.
There are two obvious ways to develop computer quality intelligence; one is through dedicated hardware (e.g., computer memory is better than human memory largely because of the hardware involved) and the other is thorough more specialized programming (e.g., relational databases provide highly structured organization of data). There is also a potential non-obvious way; let the computer try things with the tools it has, and see what the tools allow.
One of the most interesting things about quality intelligence is that some qualities can take on almost mystical traits. For example, the human brain has neurological and cognitive structures that allow it to process and appreciate music. It is reasonable to assume that no amount of increase in general processing capacity would necessarily result in a spontaneous enjoyment of music. It also seems unlikely that, ex nihilo, an intelligent being would suddenly decide that music is something that should exist.
Of course, computer intelligence does not appear ex nihilo, but through the work of many humans trying to get computers to do what humans can already do -- but faster. Often this involves using algorithms that are in fact more targeted and efficient, i.e., higher quality. This includes highly predictable specialized systems, like those used to process radar data, and more unpredictable systems, like deep learning networks. It appears that in many cases, allowing these systems to prime and train each other may be a productive way forward.
The upshot being that quality superintelligence is likely to be hard to design, hard to train, and potentially unpredictable to the point of being incomprehensible. It also suggests that there could be an arbitrary number of skills that are as useful as language, music, and mathematics that are as incomprehensible to us as novels are to zebras.