Job Title: Azure Data Engineer Azure Data Factory (ADF)
Work Location: Redmond,WA-98052
Job Details:
Must Have Skills:
Python, SQL, (bonus: Scala), Apache Spark, PySpark, Spark SQL, Azure Databricks (notebooks, Jobs/Workflows, clusters), Azure Synapse, Delta Lake, Parquet, ADLS Gen2
Nice to have skills:
ETL/ELT, MERGE/upsert, partitioning, Spark performance tuning, Git, Azure DevOps / CI/CD
Detailed Job Description
Techniques (Data Engineering Practices)
ETL / ELT
Extract data from source systems
Transform/clean/reshape into usable form
Load into target tables (ELT = transform after loading, inside the platform)
MERGE / Upsert
Update existing rows when records already exist
Insert new rows when they don't
Handled in a single atomic operation (core Delta Lake pattern)
Partitioning
Split large tables by a column (e.g., date, source type)
Enables partition pruning - scan only relevant data
Improves query speed and reduces cost
Spark Performance Tuning
Handle data skew and manage shuffles
Caching / persistence of reused datasets
Right-size partitions; use broadcast joins
Cluster sizing and resource optimization Orm
Minimum years of experience
8-10 years
...Nurse Practitioner or Physician Assistant Delivering care that is personal!Are you ready to practice medicine with purpose in a well-... ...electronic medical recordAbility to perform occupational and telemedicine patient carePerform non-patient facing duties such as lab...
...Human Resources Intern The Human Resources Intern will support the Human Resources team while serving as a peer leader and primary liaison for Cambridge interns. Through rotations across HR functions, the intern will gain broad exposure to the employee lifecycle and...
...Director of Housekeeping Location: Menlo Park, CA Salary: $100,000-$110,000 + up to 15% bonus Benefits: 100% employer-paid employee health insurance We are hiring on behalf of a luxury hotel in California for an experienced and hands-on Director...
...Join to apply for the Grade 11 Physics Tutors (Instant) role at Varsity Tutors, a Nerdy Company The Varsity Tutors Live Learning Platform has thousands of students looking for immediate online Grade 11 Physics help nationally. As an Instant Tutor on the Varsity Tutors...
...Benefits-at-a-Glance document. Benefits at a glance ( This role is for CWT Defense & Government within CWT, a member of the American Express Global Business Travel family of companies. The #TeamGBT Experience Work and life: Find your happy medium at Amex GBT....