Senior Infrastructure Engineer: Observability

At Jack Henry, we believe a stronger United States starts with thriving community banks and credit unions-institutions that prioritize people over accounts. Our mission is to empower these financial institutions with best-in-class digital banking and payment solutions, designed to enhance and streamline the user experience. We invite you to be part of this journey.

At its core, Jack Henry is a company built on relationships. Every interaction we have strengthens connections that shape our work, communities, and economy in profound ways. We seek individuals eager to tackle complex challenges in an industry undergoing rapid transformation, bringing innovation and stability to a time of disruption and change.

About the Team

Our team plays a critical role in empowering all product teams across the organization by providing comprehensive and scalable Observability solutions. We are responsible for designing, building, and maintaining robust telemetry pipelines that leverage a strategic blend of open-source tools (e.g., OpenTelemetry, Prometheus) and leading commercial platforms (e.g., Datadog, Honeycomb). This enables us to offer centralized logging, metrics monitoring, and distributed tracing capabilities, ensuring that engineering teams have the insights they need to build, deploy, and operate reliable, high-performing services.

We want our organization to be one that supports, challenges, and motivates you to build strong, dynamic connections with your coworkers. We promote a hardworking, highly communicative, flexible, and transparent culture. How well work integrates with your life is an important quality of a healthy individual and organization. In the world of remote work that is always on and always connected we pay attention to providing margin and space for the things you hold dear.

This role offers the flexibility to work remotely within the U.S. or from any Jack Henry office location. Occasional travel-up to 5%-may be required for meetings, training sessions, or conferences.

The target salary range for this position is $115,000 - 180,000, based on location and experience.

What you'll be responsible for:

• Plan, design and build the observability blueprints used by Jack Henry's development and engineering teams. Craft the overarching strategy and create detailed designs for how observability will be implemented for various products & services across the organization. This includes defining data flows & telemetry pipelines into the appropriate platform (like Datadog, Honeycomb, Prometheus, etc.) and establishing best practices for instrumentation.
• Resolve critical observability incidents that impact the ability to monitor and understand system behavior. This engineer will lead the effort to pinpoint the root cause, implement solutions, and prevent recurrence. This requires deep expertise in observability tools and techniques.
• Design and implement automated pipelines for deploying, configuring, and managing observability tools and instrumentation. This includes automating tasks such as agent installation, configuration updates, and alert provisioning, leveraging IaC principles and tools.
• Responsible for ensuring the observability systems themselves are healthy, reliable, and providing accurate data. Also champion the use of observability to improve the overall health and resilience of production systems, enabling faster detection and mitigation of potential problems.
• Actively engage with product teams to understand their upcoming projects, technology choices, and observability needs. Ensure that observability is embedded early in the development lifecycle and provide the insights necessary to optimize application performance and reliability.
• May perform other job duties as assigned.

What you'll need to have:

• Minimum of 6 years of experience in Software Development, Observability Engineering, or Site Reliability Engineering.
• Minimum of 4 years of in-depth experience with Observability platforms like Datadog, Dynatrace, Honeycomb, New Relic, Splunk, or Prometheus.

• Minimum of 2 years of experience with telemetry pipelines.
• Minimum of 2 years of experience with Kubernetes environments
• Understanding and experience with declarative infrastructure using Terraform.
• Must be able to work an on-call rotation that may include weekends as the business need dictates.

What would be nice for you to have:

• Proven ability to collect logs, metrics & traces and implement Observability solutions for applications and infrastructure.
• Solid understanding of distributed tracing and experience instrumenting applications to analyze performance bottlenecks.
• Hands-on experience configuring and deploying OTEL collectors for telemetry data collection, processing and export.
• Strong understanding of Kubernetes architecture and experience managing observability within K8s environments.
• Proficiency in using Kustomize for Kubernetes configurations and Terraform for infrastructure provisioning.
• Experience integrating observability practices and tools into CI/CD pipelines for automated deployments.
• Exceptional analytical and problem-solving skills to diagnose and resolve complex issues within observability systems, including data pipeline failures, instrumentation errors, and performance bottlenecks.
• Ability to demonstrate a strong Site Reliability Engineering (SRE) mindset with a focus on automation, proactive monitoring, and continuous improvement to ensure system reliability and availability.
• Experience defining and implementing Service Level Objectives (SLOs) and Service Level Indicators (SLIs) to measure system performance and drive data-driven decisions.

If you got this far, we hope you're feeling excited about this opportunity. Even if you don't feel you meet every single requirement on this posting, we still encourage you to apply. We're eager to meet motivated people who align with Jack Henry's mission and can contribute to our company in a variety of ways.

Why Jack Henry?

At Jack Henry, we pride ourselves through our motto of, "Do the right thing, do whatever it takes, and have fun." We recognize the value of our associates and believe much of our company's strength and success depends on their well-being.

We demonstrate our commitment by offering outstanding benefit programs to ensure the physical, mental & financial well-being of our people is always met.

Culture of Commitment

Ask our associates why they love Jack Henry, and many will tell you it is because our culture is exceptional. We do great things together. Rising to meet challenges and seeking opportunities is part of who we are as an organization. Our culture has helped us stay strong through challenging times and we credit our dedicated associates for our success. Visit our Corporate Responsibility site to learn more about our culture and commitment to our people, customers, community, environment, and shareholders.

Equal Employment Opportunity

At Jack Henry, we know we are better together. We value, respect, and protect the uniqueness each of us brings. Innovation flourishes by including all voices and makes our business - and our society - stronger. Jack Henry is an equal opportunity employer and we are committed to providing equal opportunity in all of our employment practices, including selection, hiring, performance management, promotion, transfer, compensation, benefits, education, training, social, and recreational activities to all persons regardless of race, religious creed, color, national origin, ancestry, physical disability, mental disability, genetic information, pregnancy, marital status, sex, gender, gender identity, gender expression, age, sexual orientation, and military and veteran status, or any other protected status protected by local, state or federal law.

No one will be subject to, and Jack Henry prohibits, any form of discipline, reprisal, intimidation, or retaliation for good faith reports or complaints of discrimination of any kind, pursuing any discrimination claim, or cooperating in related investigations.

Requests for full corporate job descriptions may be requested through the interview process at any time.