Job Summary 

SMBC Risk Technology Group is seeking a talented and motivated senior Azure Data Engineer. The role will be responsible for developing solutions for risk, regulatory, and compliance projects on Azure platform as well as on-prem to cloud migration initiatives. The Azure Cloud Engineer strategizes, designs, develops, and works with a team of dynamic and passionate data engineers to deliver automated cloud infrastructure and DevOps solutions. The person will play a key role in supporting the growth and scale of Azure cloud-based technology infrastructure. This role will also be responsible for development of complex interfaces, troubleshooting issues, supporting weekend maintenance, and production support.  

Primary Responsibilities

• Responsible for Administration, Configuration, and Optimization of Azure Databricks platform. 
• Provide technical expertise to deploy applications on the platform.
• Work with data engineers, data scientists, and other stakeholders to ensure the platform meets requirements.  
• Manage infrastructure and collaborate with teams to integrate with all components, including network, security, and remote access. 
• Coordinate with SMBC Platform teams, APIM, Azure DevOps, and Support.
• Configure Databricks Cluster, and Unity Catalog repository. 
• Integrate seamlessly with On-prem databases and Power BI reports.  
• Build large-scale batch and real-time data pipelines with data processing frameworks in Azure cloud platform. 
• Partner with architects, engineers, information analysts, business, and technology stakeholders.
• Develop and deploy enterprise grade platforms that enable data-driven solutions. 
• Provide technical guidance on technology implementation and architectural changes. 
• Analyze and design technical solutions on Azure to address business needs. 
• Develop, test, and modify software to improve efficiency of data platforms and applications. 
• Proficient in Python coding and frameworks knowledge. 
• Own projects end to end, work with business stakeholders, IT teams, data stewards, and 3rd parties to ensure accurate and insightful outcomes. 
• Integrate data from a variety of sources, ensure they adhere to data quality and accessibility standards. 
• Enable streaming data use-cases and standard design patterns in a big data environment. 
• Collaborate with data scientists to integrate algorithms and models into automated processes.
• Apply expertise, judgment, and precedents to resolve moderately complex problems. 
• Review and provide feedback on core code changes and support production deployment. 
• Ability to migrate the Data Science models written on native python language to the multi node Databricks compute is desirable.

Key Skills and Competencies

Azure Databricks, Azure Data Factory, Azure Functions, Azure Data Lake storage, Azure Event Grid, Azure Log Analytics, Azure Monitor, Azure Service Bus, R, Python, SQL Server, Oracle / PL-SQL, T-SQL and SQL stored procedures, Azure DevOps.   

 Qualifications

• IT professional experience in Azure Cloud with over 5+ years of Azure Development. 
• Ability to recreate existing legacy application logic and functionality into Azure Databricks/Data Lake, SQL Database, and SQL Datawarehouse.
• Excellent interpersonal and communication skills.
• Experience with software configuration management tools such as Git/Gitlab. 
• Experience with continuous integration and deployment (CI/CD) tools such as Azure DevOps.