A New Era of Testing for Firefighters
About the Author
Dr. Hugh Fullagar is a Professor of Sports Science at Reykjavik University in Iceland and an Adjunct Fellow at the University of Technology Sydney. He has also worked as an applied sports scientist and coach and continues to consult with several professional sporting organizations throughout Europe, Australia and North America. Among these are the National Football League (NFL), National Collegiate Athletic Association (NCAA), Fédération Internationale de Football Association (FIFA) Medical Centre of Excellence and elite football clubs.
An award-winning researcher, scientist and educator, his research interests are focused on evidence-based methods to improve performance and reduce injury in sport, government and industry groups, such as Fire and Rescue New South Wales (FRNSW), Defence Science and Technology Group and National Basketball Association (NBA) Global.
In this article, Dr. Fullagar reflects on his team’s work with FRNSW, highlighting how they have used evidence-based methods and advanced technologies like ForceDecks to uncover key insights into tactical performance screening and training, reshaping recruitment and testing standards for firefighting.
No matter where you live, you, a friend or a family member likely knows a firefighter. They might be the volunteer at the local station, a full-time metropolitan worker based in the city or they might even support search and rescue efforts in the aftermath of disasters overseas.
In Australia, when the summer bushfires inevitably hit, firefighters are the ones tasked with the responsibility of protecting people’s lives, homes and one another. Expected to deal with fire temperatures >1,100°C, these brave individuals perform immensely challenging physical tasks while wearing critical yet cumbersome protective clothing.
As their core body temperature rises, cognitive function is diminished, potentially leading to impaired decision-making (Pilcher et al., 2002) and increased casualties (Hand et al., 2015).
Developing Physical Aptitude (Screening) Tests
I have been working with FRNSW for over a decade to help them improve physical employment standards (Fullagar et al., 2015), cooling for performance (Fullagar et al., 2022), recovery (Fullagar et al., 2021) and worker health (Sharp et al., 2023).
The first project we worked on together was creating a screening test tool for new recruits – the Physical Aptitude Test (PAT).
The first project we worked on…was creating a screening test tool for new recruits – the PAT.
Physical Aptitude Tests (PATs) have a long history in sports. Major leagues, especially, run their potential drafted players through extensive assessment tasks to determine player ability, strengths and weaknesses.
In both occupational and athletic scenarios, we rely on a range of data points (often collected by VALD’s ForceDecks technology) to identify the right people for the right role. For firefighters, this involves assessing competencies (ensuring that no individual or group is treated less favorably) to perform the most physically demanding and critical tasks of the job – like dragging a civilian or fellow firefighter out from a burning building.
…we rely on a range of data points to identify the right people for the right role. For firefighters, this involves assessing competencies to perform the most physically demanding and critical tasks of the job…
Ladder raise and lower task performed by a firefighter on ForceDecks with other relevant physical performance attributes from
VALD Hub
displayed.
Our most recent project with FRNSW focused on reviewing a specific test within the original PAT. Reviews like this are increasingly important as complex infrastructures are developed within and around cities, the worsening impacts of climate change take hold (Yu et al., 2020), and the nature of equipment and scenarios evolve. These tests reflect the current demands of firefighting and include:
- High construct validity: How well a test measures the concept it was designed to evaluate (e.g., a test that is reflective of the physical constructs present in firefighting, such as upper body strength).
- High criterion validity: How accurately a test measures the concrete outcome it is designed to measure (e.g., the extent to which a grip strength test might correlate with holding a fire hose).
The test item we looked at within the PAT was the ladder raise and lower task, which is one of the critical and essential tasks performed by firefighters (Taylor et al., 2015). Firefighters use ladders to gain rapid access to elevated areas such as balconies, windows or rooftops to effect firefighting or rescue operations.
Matching the Physical and Perceptual Demands
Led by Honours student Bryany Parker, our research team at the University of Technology Sydney evaluated a series of potential new tests to simulate the ladder raise and lower task. FRNSW recruitment testing happens routinely and rapidly and needs to overcome variable conditions – environmental constraints, equipment inconsistency and logistical challenges. It is often conducted in remote areas where equipment, time, resources and space are scarce.
FRNSW recruitment testing…needs to overcome variable conditions [and] is often conducted in remote areas where equipment, time, resources and space are scarce.
For example, the ladder used in the original task can extend to 10-11 meters and cannot always be feasibly transported across all testing locations. Our job was to ensure that the physical and perceptual demands of the ladder raise and lower were reflected in any newly created test despite these barriers.
We assessed three different tests against the ladder raise and lower task – first in the lab and then in the field. The first phase in the lab involved alternative tests designed in consultation with subject matter experts from FRNSW and then verified via force profiling in career firefighters.
The performance of these tests was broken down into key physical components and then measured both objectively (peak force and average force) and subjectively (exertion, difficulty, fatigue and task representation scores).
Of the tests studied, the overhead modular ladder lift (Modular Under-Running Apparatus (MURA)) best objectively determined whether participants possessed the physical capabilities to perform the ladder raise and lower. Superior subjective scores further illustrated the effectiveness of the MURA in emulating the user experience of the ladder raise and lower.
Of the tests studied, the overhead modular ladder lift (MURA) best objectively determined whether participants possessed the physical capabilities to perform the ladder raise and lower.
From the Lab to the Field
Using ForceDecks, the final phase of testing was assessing the force distributions when using the MURA across individuals of different heights and locations of loads application on the device.
Using ForceDecks… We found that along with the weight of the MURA itself, positioning additional weight between the 8th and 9th rungs of the ladder best reflected the force profiles between the MURA and ladder raise and lower.
We found that along with the weight of the MURA itself, positioning additional weight between the 8th and 9th rungs of the ladder best reflected the force profiles between the MURA and ladder raise and lower. Importantly, the MURA with this load closely matched the vertical force profiles for the three test conditions of start, middle and end of underrunning.
This was a delicate balance as we had to match the original downward force yet not exceed it, as exceeding the value would lead to the elimination of some prospective recruits who were attempting to gain entry to FRNSW based on a target load that exceeded the criterion value, resulting in a false negative (Groeller et al., 2015).
During this phase, we also spoke to firefighters about the strengths and weaknesses of the screening task design and task representation of the MURA, including possible changes to the prototype.
From the Field to Use in Practice – the Final Product
We found the MURA was the best solution to match the force profile and subjective measures of the ladder raise and lower.
Validity testing of the MURA on ForceDecks.
Given environmental constraints and coordination for FRNSW recruit testing, we recommended this equipment and testing procedure be utilized within the PAT. Since our recommendation, FRNSW has welded the additional weight into the existing structure for ease of purpose and accounted for safety considerations and task familiarization for future recruits.
…FRNSW has welded the additional weight into the existing structure for ease of purpose and accounted for safety considerations and task familiarization for future recruits.
Coming full circle from the start of this project, we look forward to this test being rolled out across the state of NSW and ready for the new wave of future brave firefighters to complete.
Summary and Lessons Learned
- Importance of reliable and mobile equipment – we needed equipment that was both lab and field-based to ensure we could match the objective measures of performance with real users (in this case, firefighters). The ForceDecks technology allowed for real-time feedback and in-depth post-lift analysis, which was critical when trialing different loads and methods to match the ladder task.
- Conducting good, quality research takes an army – the success of this research was thanks to not only members of our research team but also the Health and Fitness team at FRNSW, the firefighters who provided their valuable feedback, performance and time, along with help from the Occupational Performance Research Group. Collaboration is key – we had a shared goal and understood our roles, resulting in an adaptable and solution-focused group.
- Understand the process – when designing screening tests in occupational science, there are several either legally bound or recommended steps you should follow (Payne & Harvey, 2010).
- Benefits of evidence-based practice – while the objective force data was critical, collecting real-life experiences of firefighters and their expertise on the equipment and testing was just as important. Including these opinions of key stakeholders is part of the foundation of evidence-based practice (Coutts, 2017) and garnering strong employee feedback was an essential and enjoyable part of our approach.
If you would like to learn how VALD’s human measurement technology, like ForceDecks, can support your organization’s recruitment, performance testing or occupational health efforts, please reach out.
References
- Taylor, N. A., Fullagar, H. H., Mott, B. J., Sampson, J. A., & Groeller, H. (2015). Employment standards for Australian urban firefighters: Part 1: The essential, physically demanding tasks. Journal of Occupational and Environmental Medicine, 57(10), 1063–1071. https://doi.org/10.1097/JOM.0000000000000525
- Pilcher, J. J., Nadler, E., & Busch, C. (2002). Effects of hot and cold temperature exposure on performance: A meta-analytic review. Ergonomics, 45(10), 682–698. https://doi.org/10.1080/00140130210158419
- Hand, M. S., Wibbenmeyer, M. J., Calkin, D. E., & Thompson, M. P. (2015). Risk preferences, probability weighting, and strategy tradeoffs in wildfire management. Risk Analysis, 35(10), 1876–1891. https://doi.org/10.1111/risa.12457
- Fullagar, H. H., Sampson, J. A., Mott, B. J., Burdon, C. A., Taylor, N. A., & Groeller, H. (2015). Employment standards for Australian urban firefighters: Part 4: Physical aptitude tests and standards. Journal of Occupational and Environmental Medicine, 57(10), 1092–1097. https://doi.org/10.1097/JOM.0000000000000528
- Fullagar, H., Notley, S. R., Fransen, J., Richardson, A., Stadnyk, A., Lu, D., Brown, G., & Duffield, R. (2022). Cooling strategies for firefighters: Effects on physiological, physical, and visuo-motor outcomes following fire-fighting tasks in the heat. Journal of Thermal Biology, 106, 103236. https://doi.org/10.1016/j.jtherbio.2022.103236
- Fullagar, H. H. K., Schwarz, E., Richardson, A., Notley, S. R., Lu, D., & Duffield, R. (2021). Australian firefighters’ perceptions of heat stress, fatigue, and recovery practices during fire-fighting tasks in extreme environments. Applied Ergonomics, 95, 103449. https://doi.org/10.1016/j.apergo.2021.103449
- Sharp, P., Caperchione, C. M., Brown, G. A., Stadnyk, A., Marin, E., Hulin, B., Wade, J., Mott, B., Gabriel, M., Impellizzeri, F., & Fullagar, H. H. K. (2023). A pragmatic strength and conditioning intervention for firefighters: Feasibility of the Tactical Athlete Resilience Program (TARP). Health Promotion Journal of Australia, 34(1), 60–69. https://doi.org/10.1002/hpja.656
- Yu, P., Xu, R., Abramson, M. J., Li, S., & Guo, Y. (2020). Bushfires in Australia: A serious health emergency under climate change. The Lancet Planetary Health, 4(1), e7–e8. https://doi.org/10.1016/S2542-5196(19)30267-0
- Groeller, H., Fullagar, H. H., Sampson, J. A., Mott, B. J., & Taylor, N. A. (2015). Employment standards for Australian urban firefighters: Part 3: The transition from criterion task to test. Journal of Occupational and Environmental Medicine, 57(10), 1083–1091. https://doi.org/10.1097/JOM.0000000000000527
- Payne, W., & Harvey, J. (2010). A framework for the design and development of physical employment tests and standards. Ergonomics, 53(7), 858–871. https://doi.org/10.1080/00140139.2010.489964
- Coutts, A. J. (2017). Challenges in developing evidence-based practice in high-performance sport. International Journal of Sports Physiology and Performance, 12(6), 717–718. https://doi.org/10.1123/IJSPP.2017-0455