I see data engineers as being similar to droids in Star Wars. In every movie, they’re in the background repairing ships for starfighters, delivering critical information between Jedi, moving cargo for smugglers, and translating languages for blue milk farmers. It’s easy to overlook droids because most don’t play prominent roles, but they do keep rebel and imperial forces up and running...at least until the blasters start pew pewing.
If you need to be acknowledged and glorified for your work, data engineering is probably not the right role for you. Like droids, you work in the background. You often go unnoticed. You can’t even show your friends the cool app or website that you built.
In fact, many people in your company won’t even recognize the work you do, until something fails. Then is your time to shine—because you’re a jack-of-all-trades. You don’t seek to master the major parts of the company you work for. You master all the little things that go into cloud architecture and maintenance.
There is talk that because you have to master so many little things, it’s hard to get a job as a data engineer. That’s not true. You can learn the critical software you need for your first job in just weeks. Developers have made the software intuitive. You can learn it fast.
Once you do learn the technology, you might notice there aren’t many jobs labeled as “entry level.” Don’t let that deter you from going for a job. The tools data engineers use are not considered by companies to be entry level, even if you can learn them in a few weeks. That’s because not many people know them well enough to use them professionally. Right now there are more jobs open than people who can do them.
Yes, you will have to stand in the background like a droid at a medal ceremony while your executives (Han and Luke) are acknowledged. But you will be well-compensated for the work you do. You won’t always see recognition from those above the line, but you’ll definitely see it on the bottom line.