Feb 11, 2009Microsoft's Global Foundation Services division, which oversees the platform the company uses to provide its online services, is deploying a wireless sensor network intended to bring visibility into the firm's data centers and reduce energy wasted in keeping a large room of IT equipment cooler than necessary. The system, developed by the company's Microsoft Research division, was first tested in 2006 and is now being installed at Microsoft's data centers worldwide.
The project, known as Data Center Genome (DC Genome), began with testing a few sensors in a Microsoft data center in 2006. Today, it amounts to more than 700 sensors that the company has been gradually installing, tracking temperatures on individual racks and shelves within its data centers worldwide, thus allowing Microsoft to see the temperature or humidity around each rack of servers.
The system was developed following a conversation between Feng Zhao, a principal researcher and manager with Microsoft Research's Networked Embedded Computing group, and Michael Manos, general manager of Microsoft Data Center Services, the group within Microsoft Global Foundation Services that designs, builds and manages the company's many data centers. At that time, Microsoft was growing rapidly as more people worldwide were signing on to the company's Web-based services, such as Hotmail e-mail and Messenger instant messaging. The challenge for Microsoft was to build the necessary infrastructure, such as data centers for servers around the world to support the traffic of consumers using these services. Not only would there need to be thousands of such servers, but there would also need to be adequate energy consumption to keep them operating properly at the correct temperature.
"The problem is, with so many servers, how do we know what is going on with each one?" Zhao says. Generally, it is difficult to determine such things as whether the air-conditioning system is functioning correctly, and whether any specific server is operating at a temperature beyond the safe threshold. Most data centers are large—averaging approximately 470,000 square feet in Microsoft's case—with multiple racks of servers. Air-conditioning in each data center is set to ensure that the temperature never rises above a specified level around the servers at any rack in the large room. For every megawatt directed toward powering a data center, Zhao says, another 0.5 to 1 megawatt is consumed in maintaining the proper temperature. Because it's impossible to know the temperature at any one location in the room, most companies simply turn the air conditioning up high.
The Networked Embedded Computing group's mission is to advance sensor technology and energy-efficient computing research; by 2006, the group had already developed wireless sensors. As such, Zhao was excited to learn, later that year, about Manos' issues related to data center temperature control. "We had been working on sensor networks for quite some time, and I thought this [data center application] was a wonderful space for us to apply our research," he says, citing the benefits gained by reducing the amount of energy consumed by a data center. Using sensors to track temperature, he says, "makes sense from the standpoint of the environment, and it makes sense from a business standpoint."
The system consists of small, battery-powered devices known as Genomotes. There are two types of Genomote—slave and master—that are assembled together to create a wireless sensor node that directs data to a back-end system so Microsoft can monitor its data centers in real time and locate any hot spots.
"If the temperature exceeds certain thresholds, the servers will suffer," Zhao states. "It would impact the performance of Microsoft's Internet services."
Slave Genomotes contain sensors that measure temperature and humidity every 30 seconds, then upload that data to a master Genomote via a cable connection. Master Genomotes, meanwhile, function similarly to certain types of RFID tags, and communicate via a 2.4 GHz signal complying with the IEEE 802.15.4 standard (the same standard on which ZigBee is based), to create a mesh network, transmitting information from one to the next until they reach a base station consisting of a receiver connected to the back-end system via an Ethernet cable. If there is a problem with one base station, the system is designed to enable the transmission to continue on to a working base station.
Zhao's group attached a master Genomote, along with several slave devices, to the front or back of each server rack. "A typical configuration involves three slave sensor nodes," he says, "located at the top, middle and bottom of a rack, and attached to a master node that wirelessly communicates data to other wireless nodes." Microsoft declines to reveal how many of its data centers currently utilize the technology, nor what percentage of racks are fitted with Genomotes in the data centers in which the technology has been deployed.
In that way, Microsoft is alerted if the temperature threshold is exceeded in any specific location. The company uses the data to evaluate the conditions throughout the center, and to adjust the temperature or fan speed. What's more, a server that gets too hot can be powered down, or data can be redirected to a different server.
"Once we had installed the system in a data center," Manos says, "it was allowing us to capture some interesting things." For instance, he says, a room in a data center typically includes cold and hot aisles. In a cold aisle, cool air from the air-conditioning system passes in front of the servers. The hot aisle is where heated exhaust air from the servers gets routed back into the AC system to be chilled. With the sensors, Microsoft could see that in one case, hot server exhaust air was leaking under a rack into the cold aisle, thus compromising its ability to cool the servers. Manos says he simply installed a metal plate to stop the leakage.
Now that the system is installed in dozens of data centers, Manos says, he can access information regarding the conditions of servers worldwide from his own office, and he can also store historical data regarding server temperatures. "When there are significant changes in heat patterns within a rack, we can now capture that," he states. For example, heavy usage of the Messenger instant-messaging application or other Microsoft services by the public at certain times of the day may cause spikes in the temperature of some servers, which Microsoft can now address by adjusting server workloads, or by activating or deactivating some servers during particular times of the day. "This gives us an understanding of the capacity and function of the servers," he says, "and it helps us optimize our data centers."
Microsoft is now piloting the integration of the sensors with the AC system, thereby enabling the company to raise or lower temperatures automatically, based on measurements from those sensors.
