United Airlines Principal - Database Engineer in Houston, Texas
We have a wide variety of career opportunities around the world — come find yours.
The United IT team designs, develops and maintains massively scaling technology solutions that are brought to life with innovative architectures, data analytics and digital solutions.
Job overview and responsibilities
This role will be responsible for all activities related to the design, operations, implementation, and 24x7 production support of United's complex, highly-available Kafka, Elastic and Hadoop ecosystem. Responsibilities include defining database standards and best practices for Kafka, Elastic, Hadoop and programming of data access layer. This role will align with architecture, engineering & application portfolios to deliver best-in-class systems in a hybrid data center environment to achieve business goals.
This is a hands-on role with team lead responsibilities and requires strong drive to find the optimal solutions as well as mentoring team members and developers to evangelize Kafka and Elastic technology with United Airlines, Inc.
Kafka Administration and support.
Designing, operationalizing, maintaining and scaling production Kafka clusters
Responsible for Kafka tuning, capacity planning, disaster recovery, replication, and troubleshooting.
Implementing Kafka security, limiting bandwidth usage, enforcing client quotas, backup and restoration.
In-depth understanding of the internals of Kafka cluster management, Zookeeper, partitioning, schema registry, topic replication and mirroring
Elastic Administration and support.
Architect, configure, deploy and maintain Elasticsearch clusters.
Architect and deploy Elastic Cloud Enterprise
Configure Logstash and Beats to collect data necessary to meet client requirements.
Configure X-Pack plugins to include Security, Watcher, Machine Learning, Monitoring, Graph, and Reporting.
Design, implement, and configure Kibana visualizations and dashboards.
Architecture, Design and Standardization.
Modernizing ETL pipelines through the use of Kafka and Elastic.
Maintaining Kafka connectors to move data between systems.
Working with enterprise architects to advise on best practices for Kafka & Elasticsearch both on prem and in the cloud.
Experience training, mentoring and leading an emerging team is beneficial
Knowledge of Cloud Formation/Terraform scripts.
Work closely with Big Data and AWS cloud technology groups.
Experience with open source Kafka distributions as well as enterprise Kafka products preferred.
Familiarity with both cloud native Kafka (on AWS) and on-premise architectures.
Managing Kafka & Elasticsearch clusters & creating tools to automate and improve clusters' reliability and performance.
Design and implement DR solutions for Confluent and Apache Kafka and Elasticsearch.
Lead and train team on Kafka and Elastic implementations.
- Lead and participate in OnCall rotation for production support.
Strong data analysis skills; ability to independently write scripts/code to parse and analyze complex data
Excellent verbal and written communication skills
Communicate and work effectively in a team environment
BS in Computer Science and/or equivalent work experience required
3+ years of experience architecting and implementing Kafka clusters in a large-scale enterprise environment.
3+ years of experience with Elastic (Elastic Search, Logstash, Kibana) in a large-scale enterprise environment.
3+ years of experience supporting Linux environments.
3+ years of experience with Shell scripting
5+ years of related work experience
Must be legally authorized to work in the United States for any employer without sponsorship
Successful completion of interview required to meet job qualification
Reliable, punctual attendance is an essential function of the position
Experience with Elasticsearch Beats packages a plus.
Experience with Big Data and NoSQL technologies a plus
Experience in parsing logs with GROK and regex
Experience in Java, Spring, Spring boot, Apache, Confluent, Kafka, Spark, SQL, Hadoop
Experience in Configuration Management (Puppet, Chef, Ansible), Source code Management (GitHub/Gitlab)
Experience with other features of Elastic Stack (Canvas, Elasticsearch-SQL, and Elasticsearch-Hadoop,)
Field and document modeling for fast and efficient searching and loading.
Elasticsearch Engineer I or II certification.
Previous managerial experience
Experience with configuration management tools like Ansible is a plus.
Equal Opportunity Employer – Minorities/Women/Veterans/Disabled/LGBT
Division: 47 Technology/IT
Function: Information Technology
Equal Opportunity Employer – Minorities/Women/Veterans/Disabled