Role : Principal Data Engineer

 12 - 15 Year
 Bangalore

Job Description

Python Programming, Spark-SQL, Spark-Data frames and Spark Streaming. 

• Experience in Data Ingestion from third party applications using api(s)/ REST api(s). 

• Experience in building solutions for real time ingestion of IOT or Clickstream data from different Applications. 

• Understanding of Azure Blob Storage and Delta Lake concepts. 

• Expertise in Python Programming, Spark-Data frames, Spark Streaming, Spark-SQL. 

• Expertise in building complex business logics using Spark.

 • Experience working on Databricks (preferred) or Azure HDInsight. 

• Experience in Azure App Services, Azure Function App, Azure Logic App.

 • Deep Understanding of Azure Blob Storage and Delta Lake concepts. 

• Experience on Data Orchestration tool preferably Azure Data Factory. 

• Experience on Performance tuning/optimisation on Spark. 

• Parsing & Processing of unstructured and semi-structured data.

 • Collaborate with internal and customers functional and technical teams to ensure Azure based solutions are appropriately Engineered, Tested, & Deployed. 

• Experience in building Data Catalogues, Metadata, Logging & Alerting in the Data Pipelines. 

• Understanding of different Storage technologies on Cloud. 

• Ability to be up to date with new offerings on the Cloud. 

• Ability to translate business requirements / problems into technical Architectures utilizing Azure & Complementary Services and Technologies.

Salary :   Not Disclosed by Recruiter

Industry :   IT Software - Other

Functional Area :   IT Software - Other

Role Category :   Computer/IT

Employment Type :   Full-Time

Education / Qualification :   B.E/ B.Tech