The cloud. If you don’t know what it is, you’re probably on the wrong page. You probably also don’t realize it’s been integrated into just about every service or good in the U.S. market. Those organic wood blocks you bought for your kids—the company website has tutorials on what to build and lets you create your own that are saved under your profile. That Swiss chard you got at the farmer’s market—the farmer uses weather data trends stored on the cloud to find the precise time each year to plant their crops.
Data engineering is everywhere. And it’s not just a skill anymore. Data engineering is now a critical role that ensures companies can meet customer demand—because demand can explode in a short time thanks to social media and other modern social influences. The covid crisis has also driven demand for services created by data engineers. More than 200,000 businesses permanently closed during the COVID-19 crisis; yet Amazon flourished. The company reported a near 200% rise in profits, according to the New York Times. That’s because Amazon’s data engineers created an elastic system that allowed it to stretch and meet customer demand. That paid off when people were afraid to leave their homes.
Amazon’s Web Services sector is also booming—it grew 29% in 2020 and web services accounted for 12% of the company’s total revenue that year. The digital infrastructure that data engineers created is being challenged this holiday season. On December 7th, reports of power outages caused significant problems for companies that use Amazon Web Services, such as Disney+, video game League of Legends, Venmo, and Tinder. You better believe a team of data engineers hustled to get full service restored.
What does that look like? Have you ever seen ants in crisis? Imagine them escaping a nest on one tree that’s on fire, linking their legs to create a bridge four feet across to another tree, transporting the queen and larva, and immediately building a new nest on the new tree. That’s what data engineers do during a crisis.
But there’s not always a crisis. Sometimes they build to benefit users—which benefits the bottom line. Microsoft is a great example of a company shifting one of its main products to the cloud and seeing a resurgence in sales. After Google introduced its all-online G-Suite software, Microsoft’s data engineers got hard at work to create Office 365. They built architecture to store, search, and share files—all for a monthly cost that is similar to the price of a full Microsoft Office download.
At Dropbox, data engineers are changing how we communicate with one another while using their software. They are now working on what the Dropbox CEO calls a Metaverse. It’ll allow every user in the same virtual room to access the same digital materials.
Nintendo, a longtime supporter of keeping gamers offline, finally turned to the cloud to let gamers create Mii avatars and play with each other online. But the most exciting upgrade came in October 2020, when Nintendo announced Cloud Games. Nintendo products are usually built with Nintendo games in mind. You know—Mario, Zelda, Metroid, Super Smash Bros., etc. Nintendo’s consoles often struggled to handle third party games that require more graphical and processing horsepower. Now data engineers have created a cloud system for its Switch console that lets gamers play games that are too powerful for its hardware—such as Resident Evil 7 and Marvel’s Guardians of the Galaxy.
Data engineers aren’t just opening doors that used to be closed. They’re now kicking them down. The cloud lets dreamers dream. Data engineers take those dreams and make them reality.