Performance Test Lead
Management and Technical Skills
Performance Testing Management and Problem Solving Responsibilities
Communicate performance test objectives, methods and measures of success to management audiences
Ability to manage and drive results across teams within the Billing program
Capable of explaining the value of the performance test plan and metrics to management audiences
Ability to provide direction and performance manage a small team of test automation specialists and test engineers
Ability to personally determine root cause of key solution performance and resiliency issues, understand trends and lead individual contributors through the process
Uses ingenuity in dealing with ambiguous situations, and guides others to cope effectively
Functional/Technical Skills & Market Knowledge
Possesses required functional and technical knowledge and skills to successfully manage, plan and validate solution performance consistent with the needs of a complex business critical solution
The candidate should demonstrate ability to apply the following functional skills:
- Benchmarking - Establishes objective performance test criteria for solution functions across the integrated solution. Identifies and manages critical success factors for applications under load
- Solution Performance Measurement - Assesses the current state of application and its interactions to determine how the functional areas within the solution are performing with regard to what is acceptable from a business perspective. Illustrates which components are performing below, at and above target performance levels
- Solution Resilience Measurement - Assesses the ability of the solution to recover from a failure against established goals for characteristics such as recovery point and recovery time objectives
- Performance Test Automation Acumen– Understands and can explain performance, load, and SQL testing scripts. Describes how performance test automation strategies can standardize and scale the application T+2 years to T+5 years. Demonstrates ROI savings and conduct risk analysis performance test methodologies
Day to Day Role Expectations
Performance Test Planning Deliverables
Provide thought leadership in building robust and scalable enterprise scale software systems
Create Master Performance Test Strategy and Plan identifying response time, throughput, and resource utilization goals and constraints
Prepare the test environment, tools, and resources necessary to execute each strategy as features and components become available for test
Create performance test scenarios and test cases
Facilitate and procure the performance test data required to enable performance testing
Set and monitor test execution rate based on business criteria
Build performance test suites for core application functions and interfaces between systems
Review and validate the architecture, design and technology choices against NFRs
Performance Test Validation Deliverables
Establish a performance test baseline to validate that the performance of the overall system has not been impacted
Institute criteria for measuring and optimizing performance test results
Apply performance counters to the application code to set and record load threshold intervals and timings
Recommend remediation steps for address architecture and design issues in the existing system in order to improve performance and scalability
Analyze the testing issues using log files, heap dump, thread dump, resource usage data, profiling snapshots etc.
Work with vendor and client development teams in implementing the solutions proposed and solving the performance issue
Performance test data conversion functionality on partial and full data loads
Assignment and coordinate testing activities across performance test resources
Collaborate with AB QA Testing leadership as needed
Responsible to System/Integration test REST API's, User interface, Database, reporting, ETL/Batch technology components
Perform complex testing in related to data environments as needed to meet business performance objectives (source to target, data mining, data profiling, creation and execution of stored procedures, etc.)
Execute validated tests for analysis while monitoring results within the performance environment
Prepare and communicate management level performance test metrics, trends and hypothesis on root cause on a scheduled and ad hoc basis
Reprioritize tests as needed to drive performance issue identification and resolution
Qualifications
6+ years of experience, including data mining, profiling of application and its module theoretical minimums and maximums
Experience as a performance test lead for a complex business critical solution with multiple applications and interfaces
SPLUNK certification preferred
Language: Same as Automation Test Lead + experience with PHP
Tools: such as LoadRunner, Neoload, JMeter, LoadNet
Other Requirements
If resources' are working outside of the US ET time zone core hours but the system is unavailable for testing, those resources must be able to test during US ET core hours until the system is operable again during their work shift. Accordingly, resources must be available and accessible during core team work hours, 8AM – 5PM ET.