Skip to main content

11 posts tagged with "data engineering"

View All Tags

· 3 min read
Parham Parvizi

Pass on this opportunity and it will become one of those moments that you look back on with regret—knowing you had the time and dedication to change your life. Data engineering is the hottest job market in tech. It’s a critical role as companies shift to cloud services. The pay is outstanding—averaging $119k. Companies are hiring in every major market—and some are even looking for remote work. In 25 weeks, you could find yourself interviewing for one of those jobs.

· 3 min read
Parham Parvizi

You can search the web and get lost trying to find information about what tools data engineers use every day. Instead of guessing, let’s find out what tools the data engineers at Tura.io use. Their data engineers work with major companies, so they must know what critical tools you need to learn to be a data engineer. Let’s take a look:

· 2 min read
Parham Parvizi

Data engineers need to have a holistic view of a company—because they’re impact is felt across it. They’re in the front end, the back end, and everywhere in-between. They’re in constant interaction with facets in the company as well as people who exist outside the flow chart—like customers. Let’s look at how they interact:

· 3 min read
Parham Parvizi

The most important benefit of a profession is job availability. Right now, data engineering jobs are available in abundance. In fact, there are more jobs than data engineers to do them. The 2020 Dice Tech Jobs Report analyzed six million job postings and found data engineering jobs grew 50% between 2019 and 2020. That’s the highest of any tech industry specialization. A Burtch Works survey confirms the need for data engineers continues. Of 320 data engineering employers polled in mid-2021, 80% said they planned to hire data engineers in the second half of the year.

· 2 min read
Parham Parvizi

I see data engineers as being similar to droids in Star Wars. In every movie, they’re in the background repairing ships for starfighters, delivering critical information between Jedi, moving cargo for smugglers, and translating languages for blue milk farmers. It’s easy to overlook droids because most don’t play prominent roles, but they do keep rebel and imperial forces up and running...at least until the blasters start pew pewing.

· One min read
Parham Parvizi

Every major company is going through a digital transformation. Data engineers are making it happen. They’re the number one priority right now. Stitchdata posted its research on the job market—finding 6,500 data engineers on LinkedIn—and 6,600 open data engineering positions on Indeed. Companies need to switch to the Cloud to meet shifting customer demand but there aren’t enough workers to make it happen.

· 11 min read
Parham Parvizi

A quick deep-dive into Apache Spark, the most popular distributed data engineering tool.

What is Spark? Why is it so popular? When and how to use it?

Learn the difference between the sub components (RDDs, DataFrames, SQL, Streaming, ...), setup PySpark , and learn how to write Spark transformations using Python and Jupyter Notebook.