Since the late 1960s Moore’s law has predicted that the number of transistors on a chip will double about every two years. Over time this law has led to an exponential growth in computing power as we took advantage of more transistors to do more computations.

Yet our ability to rapidly access the data needed for these computations has not kept pace. This mismatch has been compounded by the enormous growth of the datasets we process for everything from biology to social networks and machine learning. Dr. Black-Schaffer sees a need to switch our focus from increasing computational performance to being more clever about how we move data.

“At present about two-thirds of a processor’s computing power is expended on moving data around. Only one-third of the power is used for the computations we mean when we talk about computer performance.”

Memory processing limits performance

While processors can compute values incredibly quickly, the overall performance is limited by how quickly the data can be transferred, read, and copied. Dr. Black-Schaffer points out that we have known about this for some time.

“We’ve developed numerous ways to manage these limitations, but this is done by simplifications and compromises that are far from optimal,” he says.

Today’s approach is to have part of the processor dedicated to data movement by trying to predict the data that will be needed ahead of time. Yet this is difficult since the patterns occurring in different software are so disparate. As a result while most programs benefit from the pattern detection, others can be slowed down.

An extra layer of intelligence

Dr. Black-Schaffer’s solution is to re-think how we program. Instead of one program that specifies what to compute and only implicitly how to move the data, he proposes to separate computational program from the memory processing program. He plans to achieve this by creating a new memory system programming model that works together with the regular computational program to optimize data access and movement.

“We can give precise instructions to the hardware about how it should handle data. This will enable us to adjust the hardware for each program.”

The solution provides an extra programmable layer of flexibility between the software and hardware. This enables precise instructions to be given on how data are to be transferred, the relationship between different types of data, and how transactions should be performed simultaneously. Indeed, this extra layer can be used for the software to precisely specify its own data access pattern, eliminating the need for the hardware to guess it for every program.

“Programs will work better if they can specify how they move data to avoid the compromise of having the hardware guess how data should be moved. I think this may be as large a leap as virtualization, which paved the way for the cloud computing we see today.”

Narrow margins

Today’s systems are highly optimized for the common data movement patterns. Dr. Black-Schaffer’s plan new programmable layer will come at a cost of energy and performance. The crux of the research is to see if this overhead can be beaten by the more precise optimizations from having the software specify its own patterns. The next step will be to develop new tools and software libraries to make it easier for tomorrow’s programmers to co-design their computational and memory programs.

“We need to create an environment in which we can adapt and reuse these new memory system programs. Otherwise, this will incur too much programming cost. If we can show the benefits and the ease of implementation, we will then need to work with processor manufacturers to get the support into the hardware.”

Dr. Black-Schaffer’s research team is currently collaborating with Arm Research, a U.K. processor maker and has worked closely with other manufacturers in the U.S. and Asia in the past. Indeed, most of the researchers trained in the Uppsala University computer architecture group go on to work at major international companies, providing close connections to industry.

“Thanks to the Wallenberg Academy Fellow grant, we can scale up our efforts and involve more people, which is essential to achieve such an ambitious goal.”

“Renewal of the Wallenberg Academy Fellow grant enables me to build on my earlier results and address larger issues than before.”

Education key

Dr. Black-Schaffer arrived at Uppsala University in 2009 having received his PhD at Stanford University in California. In addition to research, he led the development of a popular tool for enabling active digital learning online and in-class. Faced with the choice of commercializing the learning tool or devoting his time to research, he chose the latter.

“Our tool was used by 80,000 students around the world. But it’s very hard to launch a successful education company as you need to convince teachers to put in the extra effort to adapt their teaching.”

He has learned a number of lessons, however, not least of which is that the tool itself is only helpful if teachers have the time and motivation to develop good educational materials. To get there we need better approaches for teaching the new techniques to teachers and better incentives developing teaching materials. He goes on:

“Our most important mission as a university is education. Every person we educate contributes to society for decades after graduation. And this is especially true in research, where the researchers we educate have far more impact over their careers than the articles they publish during their training.”

Text Magnus Trogen Pahlén
Translation Maxwell Arding
Photo Magnus Bergström, David Black-Schaffer,The Swedish Foundation for Strategic Research