Pyspark Databricks Engineer
Houston, TX
Contracted
Experienced
Job Title: PySpark and Databricks Developer
Location: Houston, TX (Hybrid)
Key Responsibilities:
Required Skills and Qualifications:
Location: Houston, TX (Hybrid)
Key Responsibilities:
- Design, develop, and optimize data pipelines and transformations using PySpark and Databricks.
- Collaborate with data architects and analysts to define and implement scalable data models and frameworks.
- Build and maintain complex data ingestion and processing workflows for large, distributed datasets.
- Develop reusable and efficient code following best practices in coding, testing, and deployment.
- Optimize Spark jobs for performance, scalability, and reliability in production environments.
- Work closely with cross-functional teams to ensure data quality, consistency, and integrity.
- Contribute to continuous improvement of the data engineering ecosystem and CI/CD processes.
Required Skills and Qualifications:
- 5+ years of experience in software development with a focus on Python and PySpark.
- Hands-on expertise in Databricks platform — including cluster management, notebooks, and job orchestration.
- Strong programming fundamentals (data structures, algorithms, debugging, version control).
- Experience with Delta Lake, Spark SQL, and data lake architectures.
- Solid understanding of distributed computing, data partitioning, and Spark performance tuning.
- Familiarity with cloud platforms such as Azure, AWS, or GCP.
- Excellent communication and problem-solving skills — able to explain complex technical concepts clearly.
Apply for this position
Required*