Big Data Engineer
As (senior) Big Data Engineer, you will drive the Hadoop platform (PaaS) build up and global scale out – drawing on your deep subject matter expertise gained at significant organizations with a global footprint. Besides your hands-on industry experience i
Pozíció leírása / Job description
As (senior) Big Data Engineer, you will drive the Hadoop platform (PaaS) build up and global scale out – drawing on your deep subject matter expertise gained at significant organizations with a global footprint. Besides your hands-on industry experience in building enterprise-grade big data systems, you shall have strong command of IT architecture on the conceptual side. First expertise in operations and a DevOps stance on building and running distributed systems completes your profile. Your engineering contributions and architectural decisions will be key to the success of the highly dynamic, international and rapidly growing team of big data engineers, developers, architects and project managers.
- As (senior) Big Data Engineer you will be closely working with IT architects and business customers to elicit requirements, to optimize the system performance as well as to advance its technological foundation.
- Manage very large-scale, multi-tenant and secure, highly-available Hadoop infrastructure supporting rapid data growth for a wide spectrum of innovative internal customers
- Provide architectural guidance, planning, estimating cluster capacity, and creating roadmaps for Hadoop cluster deployment
- Design, implement and maintain enterprise-level security (Kerberos, LDAP/AD, Sentry, etc.)
- Install Hadoop distributions, updates, patches, version upgrades
- Identify new components, functions and features and drive from exploration to implementation
- Create run books for troubleshooting, cluster recovery and routine cluster maintenance
- Troubleshoot Hadoop-related applications, components and infrastructure issues at large scale
- Design, configure and manage the strategy and execution for backup and disaster recovery of big data
- 3rd-Level-Support (DevOps) for business-critical applications and use cases
- Evaluate and propose new tools and technologies to meet the needs of the global organization
- Work closely with infrastructure, network, database, application, business intelligence and data science units
Elvárások / Requirements
- University degree ( in computer science, mathematics, business informatics or in another technical field )
- Deep understanding of grid computing design principles and the factors determining and affecting distributed system performance
- Experience with implementing Hadoop in a large scale environment, preferably including multi-tenancy and security with Kerberos
- Excellent hands-on working experience with Hadoop ecosystem for at least 2 years including - HDFS, Map, Reduce, Pig, Hive, Impala, Mahout, Oozie, Zookeeper, Flume, Sentry but also with SQL, R, Python/Shell Scripting, Linux/Unix
- Strong Linux skills; hands-on experience with enterprise-level Linux deployments as well as shell scripting
- Well versed in installing, upgrading & managing distributions of Hadoop (CDH5x), Cloudera Manager, MapR, etc.
- Cluster node configuration, connectivity, capacity, compute architecture, name node/data node/job tracker deployment layout, server requirements, SAN, RAID configurations etc.
- Hands-on experience with automation, virtualization, provisioning, configuration and deployment technologies (Chef, Puppet, Ansible, OpenStack, Docker, etc.)
- Experience working in an agile and international environment – excellent time-management skills
- Excellent communication skills and high level of motivation (self-starter)
- Strong sense of ownership to independently drive a topic to resolution
- Ability and willingness to go the extra mile and support the overall team
- Business fluent English in speech and writing, German is a plus
Amit kínálunk / Offer
- Working as the member of a professional BI/BigData team
- Continuous professional development
- Opportunity for gaining multinational experience
- Comprehensive compensation package
Randstad Hungary Kft.
Lövőház utca 39.
Tel: +36 1 411 2090