Senior Big Data Engineer - Hadoop, Spark & Real-Time Data Processing Solutions
Join arenaflex as a Senior Big Data Engineer
Are you ready to transform your career and become part of a dynamic technology powerhouse? At arenaflex, we're not just building software – we're crafting the future of digital financial services and revolutionizing how millions of customers interact with their finances every single day. We're looking for a talented and passionate Senior Big Data Engineer to join our innovative technology team in Phoenix, Arizona, and help us push the boundaries of what's possible in the world of big data and cloud computing.
About arenaflex
At arenaflex, we believe that technology is the backbone of modern financial services. Our diverse tech group is comprised of brilliant engineers, data scientists, and problem-solvers who work collaboratively to create cutting-edge solutions that make us an essential part of our customers' digital lives. Here, you'll work alongside talented architects in an open, supportive, and inclusive environment where your voice is truly valued, and you have the autonomy to make decisions about which technologies to use to solve challenging problems.
arenaflex offers a wide range of opportunities to work with the latest and most innovative technologies in the industry. We encourage our team members to contribute to the broader engineering community through open-source projects and collaborative initiatives. Moreover, because we understand the importance of keeping your skills fresh and relevant in this rapidly evolving field, we provide dedicated time and resources for you to invest in your professional development. Discover your place in technology with #TeamArenaFlex.
What You'll Do in This Role
As a Senior Big Data Engineer at arenaflex, you'll be at the forefront of our data engineering initiatives, designing and building scalable data pipelines that process massive volumes of information in real-time. You'll be an integral part of our product engineering or delivery and integration team, reporting directly to a Engineering Director, and you'll have the opportunity to shape the future of our data infrastructure.
Software Development & Engineering
- Design, develop, and maintain advanced data processing systems using Hadoop, Spark (Core, SQL, and PySpark), and related big data technologies
- Write high-quality, production-ready code and API specifications while adhering to best practices and coding standards
- Participate actively in code reviews and automated testing to ensure code quality and reliability
- Troubleshoot complex data pipeline issues and identify code deficiencies for remediation
- Enable the deployment, support, and monitoring of software across test, integration, and production environments
- Automate deployments in test or production environments using modern CI/CD practices
- Scale applications automatically based on demand projections and usage patterns
- Develop and test software, including ongoing refactoring of code to drive continuous improvement in code structure and quality
- Apply visualization and other techniques to quickly prototype and validate concepts
- Design and develop advanced data pipelines for both batch and real-time data processing
Technical Leadership & Collaboration
- Serve as a core member of an Agile team, driving user story analysis, elaboration, and technical design of data applications
- Lead and mentor a small team of engineers, inspiring them to achieve their full potential and deliver exceptional results
- Influence team members with innovative changes and improvements by challenging the status quo and demonstrating thoughtful risk-taking
- Manage specific platforms/products or serve as part of a dynamic resource pool assigned to projects based on demand and business needs
- Identify opportunities to adopt innovative technologies and methodologies
- Embody an entrepreneurial mindset by looking beyond the obvious for continuous improvement opportunities
What We're Looking For
Essential Qualifications
- Education: Bachelor's degree in Engineering, Computer Science, or a related field, OR Master's in Computer Applications or equivalent practical experience
- Experience: 3+ years of software development experience with leading teams of developers and Scrum teams
- Big Data Expertise: 3+ years of hands-on experience working with MapReduce, Hive, and Spark (Core, SQL, and PySpark)
- Data Warehouse: Strong understanding of data warehousing concepts and architectures
- Programming: Hands-on experience in software development using Core Java or Python/Scala
- Distributed Systems: Mastery of Hadoop and Spark architecture and their working standards
- SQL Proficiency: Hands-on experience writing and understanding complex SQL (Hive/PySpark dataframes), with expertise in optimizing joins while processing large volumes of data
- Scripting: Experience with UNIX shell scripting
- Data Pipeline Design: Ability to design and develop advanced data pipelines for batch and real-time data processing
- Full-Stack Development: Experience in analysis, design, development, testing, and implementation of enterprise applications
- Documentation: Demonstrated ability to develop and document technical and functional specifications, and analyze software and system processing streams
Preferred Qualifications
- Knowledge of cloud platforms like GCP (Google Cloud Platform) or AWS (Amazon Web Services)
- Experience building microservices and scalable solutions
- 3+ years of experience designing and building solutions using Kafka streams or message queues
- Experience with GitHub/Bitbucket and implementing CI/CD pipelines
- Familiarity with NoSQL databases such as HBase, Couchbase, or MongoDB
- Knowledge of financial reporting environments is a plus
- Excellent technical and analytical skills
- Strong communication skills with the ability to explain complex technical concepts to non-technical stakeholders
- Exceptional project management abilities
Why Join arenaflex?
Career Growth & Development
At arenaflex, your career trajectory is limited only by your ambition. We provide extensive opportunities for professional growth through:
- Continuous Learning: Dedicated time for professional development, including access to online courses, certifications, and technical conferences
- Mentorship Programs: Pairing with experienced mentors to accelerate your skill development
- Internal Mobility: Opportunities to explore different roles and teams across the organization
- Technical Challenges: Exposure to complex, high-impact problems that will stretch your capabilities
Work Environment & Culture
We pride ourselves on fostering an inclusive, collaborative, and innovative work environment where every team member can thrive. Our culture is built on:
- Innovation: Encouragement to experiment with new technologies and approaches
- Inclusivity: A diverse workplace where different perspectives are celebrated
- Work-Life Balance: Flexible work arrangements and support for your well-being
- Team Spirit: Strong camaraderie and collaborative spirit across all teams
- Open Communication: Transparent communication channels where your voice matters
Compensation & Benefits
We offer a competitive compensation package that recognizes your skills and contributions:
- Salary: $28 per hour (commensurate with experience and qualifications)
- Employment Type: Full-time position
- Work Hours: 8 hours per day
- Location: Phoenix, Arizona (with potential for hybrid/remote arrangements)
- Comprehensive Benefits: Health insurance, retirement plans, paid time off, and more
- Perks: Professional development budget, wellness programs, and employee discounts
Ready to Make an Impact?
If you're a passionate, high-performing engineer who wants to work on challenging problems with the latest technologies, aren't afraid to think differently, and wants to be part of a team that's shaping the future of financial technology, then arenaflex is the place for you. You'll be part of a fast-moving, pioneering team that enables batch/real-time analytical solutions using cutting-edge technologies including Hadoop/Big Data, Data Science/ML, Python, SQL, UNIX, and Shell Scripting.
Join us and find your place in technology at arenaflex. We can't wait to see what you'll build.
Note: The proposed employment with arenaflex is contingent upon the successful completion of a background verification check, subject to applicable laws and regulations.