Job Details
Databricks Architect / Data Engineering Architect
Satsyil Corp is currently seeking a highly skilled and motivated Senior Data Architect /
Databricks Architect to join our team and contribute to the success of our Enterprise Data
Services project. As a Databricks Architect, you will play a crucial role in developing and
optimizing Spark applications in AWS Databricks, leveraging your expertise in Python, SQL, and
pySpark. The ideal candidate will have a solid background in data engineering and demonstrate
proficiency in designing and building efficient ETL pipelines using Apache Spark in the
Databricks environment.
Roles and Responsibilities:
• Develop Spark applications in AWS Databricks, utilizing Python, Scala, SQL, and pySpark to
meet project requirements and data processing needs.
• Design and implement robust ETL pipelines using Apache Spark in Databricks, ensuring data
integrity, efficiency, and scalability.
• Collaborate with cross-functional teams to understand business requirements and design
solutions that leverage structured, semi-structured, and unstructured data effectively.
• Write high-quality code in a timely manner, adhering to coding standards, best practices,
and established development processes.
• Utilize version control systems like Git to manage codebase and ensure seamless
collaboration within the team.
• Merge and consolidate various data sets using Pyspark code, enabling streamlined data
processing and analysis.
• Work with APIs to facilitate data ingestion from diverse sources and integrate data into
the ecosystem.
• Apply expertise in Databricks delta lake to optimize data storage, query performance, and
overall data processing efficiency.
• Demonstrate knowledge of application development lifecycles and promote continuous
integration/deployment practices for efficient project delivery.
• Perform query tuning, performance tuning, troubleshooting, and debugging for Spark and
other big data solutions to enhance system efficienc