Loading...
CM-2024-035 - 2/2/2024Fire & Police Selection, Inc. 193 Blue Ravine Rd., Ste. 270 Folsom, CA 95630 Office: 916.294.4242 Fax: 916.294.4240 CONTRACT FOR ROUND ROCK FIRE DEPARTMENT PHYSICAL ABILITY TEST Fire & Police Selection, Inc. (FPSI), a consulting firm located at 193 Blue Ravine Rd., Suite 270, Folsom, California 95630, agrees to perform the tasks below for a fee of $18,000: • Conduct a Physical Ability Test validation study using firefighter subject-matter experts from the participating department (as described in Exhibit 1). It is expressly understood that in selling the right to use its Physical Ability test, FPSI cannot guarantee compliance with the civil rights laws or the requirements of federal or state enforcement agencies. Nor does FPSI agree to stand as surety or otherwise immunize the Round Rock Fire Department from any civil rights liability that may result from the use of the test. The job analysis and validation study associated with this project will greatly improve the defensibility of the PAT. FPSI may provide litigation assistance for the Round Rock Fire Department if it becomes the subject of any litigation or investigation by state or federal enforcement agencies arising from the use of the entry-level Physical Ability Test. This assistance would be provided on a time-and-charges basis at the consultant’s then published hourly rates. • License the Round Rock Fire Department to use the custom-developed Physical Ability Test validated by FPSI for testing candidates applying for employment with the department indefinitely and as an annual maintenance test for incumbent fire suppression personnel. • Provide two (2) cutoff recommendation options for the candidate and incumbent Physical Ability Test standard. • Complete and submit a content validation report to address the requirements of the Uniform Guidelines on Employee Selection Procedures (1978). • Provide one copy of both the Proctor and Candidate videos and a Test Administration Handbook to assist the Round Rock Fire Department with the construction of props. • FPSI shall complete the services described in Exhibit 1 within eight-to-ten weeks following the receipt of the executed contract, barring any unforeseen delays on the part of the Round Rock Fire Department. Initials: FPSI _______ RFFD _______ CM-2023-035 Participation in this study requires that the department provide a random sample of firefighters to complete Job Analysis Surveys and Critical Events Surveys regarding their job as a firefighter and to take the Physical Ability Test for norming and passpoint setting. The Round Rock Fire Department will pay one-half (50%) of the contract upon receipt of the validation surveys and the final one-half (50%) of the contract upon receipt of the cutoff recommendations and Content Validity Report. Round Rock Fire Department Fire & Police Selection, Inc. Name: _________________________ Name: Title: _________________________ Title: President Date: __________________________ Date: December 15, 2023 Signature: __________________________ Signature: __________________________ Stacy L. Bell Stacy L. Bell City Manager 02/02/2024 Laurie Hadley Exhibit 1 December 14, 2023 Re: Proposal for Developing and Validating a Candidate and Incumbent Physical Ability Test (PAT) Tom O’Brien, Captain – Fire Training Coordinator Round Rock Fire Department 2801 N. Mays Street Round Rock, TX 78664 Captain O’Brien: Thank you for your interest in learning about the steps involved with developing and validating an entry-level firefighter Physical Ability Test (PAT) for Round Rock Fire Department. Fire & Police Selection, Inc. (“FPSI”) is pleased to provide you with this information and I have taken the liberty of describing the process below in detail. FPSI has provided firefighter pre-employment selection tests and validity study services to fire departments across the country for over 26 years. Dr. Dan Biddle and Stacy L. Bell would be the lead project managers responsible for providing the products and services described in this proposal to the Round Rock Fire Department throughout the terms of the contract. Dr. Dan Biddle is the CEO of Biddle Consulting Group and of FPSI. Dr. Biddle has his doctoral degree in Industrial/Organizational Psychology. Stacy L. Bell has served as the President/Principal Consultant to FPSI for the last 25 years and has her master’s degree in Industrial/Organizational Psychology. Ms. Bell has served as the project manager, and provided services similar to those requested by the Round Rock Fire Department to other departments in Texas such as Lake Travis Fire Rescue, Kerrville Fire, Eagle Pass, Victoria Fire, Baytown Fire, Marble Falls, Taylor Fire Department, and Rockwall Fire. Additionally, we’ve validated our PAT into large metropolitan fire departments such as Baltimore City Fire Department, Long Beach Fire Department, and Honolulu Fire Department. Collectively, Dr. Biddle and Ms. Bell have well over five decades of combined experience in the development and validation of entry-level tests used for the selection of firefighters and police officers. FPSI’s Experience with Physical Ability Testing FPSI is considered the leading expert in the area of physical ability testing for entry-level firefighters. We are the developers of the world’s largest entry-level firefighter Physical Ability Test consortium, authors of dozens of articles/book chapters, and speakers at national fire conventions on the topic of entry-level firefighter PATs. Our opinion on entry-level firefighter “new hire” testing is that it is always best to use a work- sample physical ability test as opposed to a fitness (i.e., agility) test. The main reasons for this are: (1) work-sample tests provide defensible testing in a way that connects the test events to actual work behaviors performed on-the-job; (2) work-sample tests provided a realistic preview to the candidates on the types of physical work behaviors they may one day perform on the job; and (3) work-sample tests tend to have much lower levels of adverse impact against the protected classes compared to traditional fitness tests. Section 3B of the Federal Uniform Guidelines on Employee Selection Procedures (1978) states: “Where two or more selection procedures are available which serve the user's legitimate interest in efficient and trustworthy workmanship, and which are substantially equally valid for a given purpose, the user should use the procedure which has been demonstrated to have the lesser adverse impact. Accordingly, whenever a validity study is called for by these guidelines, the user should include, as a part of the validity study, an investigation of suitable alternative selection procedures and suitable alternative methods of using the selection procedure which have as little adverse impact as possible, to determine the appropriateness of using or validating them in accord with these guidelines. If a user has made a reasonable effort to become aware of such alternative procedures and validity has been demonstrated in accord with these guidelines, the use of the test or other selection procedure may continue until such time as it should reasonably be reviewed for currency. Whenever the user is shown an alternative selection procedure with evidence of less adverse impact and substantial evidence of validity for the same job in similar circumstances, the user should investigate it to determine the appropriateness of using or validating it in accord with these guidelines...” It is our strong recommendation that Round Rock Fire Department consider transporting the validity of our well-validated PAT. FPSI’s PAT has been successfully used to screen tens of thousands of firefighter candidates across the U.S. without any challenges. Our PAT is an extremely robust measure of the physical abilities required to successfully perform the fire suppression duties of the job, but in a way that doesn’t overemphasize upper body strength, like most fitness/agility tests do. FPSI’s PAT: FPSI and Biddle & Associates (B&A) developed our current PAT back in 1996 in a consortium effort with 41 departments from the Southern California area including the City and County of Los Angeles and Orange County. The end-result was an eleven-event continuously timed work- sample PAT that measures critical work behaviors and endurance. Both content and criterion- related validity data was collected showing that faster performance on our PAT is indicative of better job performance. FPSI was incorporated in 1996 and has validated various entry-level firefighter PATs into over 30 fire departments (in addition to the original 41 departments in our initial validation study). Cities using our PAT range from cities with a population of 1,190 to a population of 9,900,000. The average size city using our PAT is about 100,000. Transporting FPSI’s PAT: The current FPSI PAT test events are used as a “template” in an attempt to transport them to the Fire Department through necessary validation surveys. The data from these surveys will be used to identify which of the 11, continuously-timed, test events will be included in the final PAT, as well as to customize the test events to ensure that proper distances, weights, and pieces of equipment are used to replicate the ways in which the work behaviors are performed by Round Rock Fire Department firefighters. The “template” 11 events on FPSI’s PAT are listed below: Event 1: Dry Hose Deployment Candidate advances pre-connected 1.75-inch dry hose line for 150 feet. This event simulates extending a dry offensive hose line from an apparatus to a fire scene for fire fighting operations. Event 2: Charged Hose Deployment Candidate advances pre-connected 1.75-inch charged hose line for 70 feet. This event simulates extending a charged offensive hose line to the seat of a fire for fire fighting operations. Event 3: Halyard Raise Candidate raises the fly section of a 35-foot aluminum extension ladder. This event simulates extending the fly of a 35-foot aluminum extension ladder in order to access higher floors for rescue, ventilation, and to conduct other fire fighting operations. Event 4: Roof Walk Candidate ascends and descends an 11-foot distance walking/crawling on the rungs of a 12-foot roof ladder while carrying a simulated 20-pound chain saw. This event simulates walking on a roof ladder with the appropriate equipment in order to perform ventilation operations. Event 5: Attic Crawl Candidate crawls a distance of 21 feet across a simulated attic area while carrying a simulated flashlight in his/her hands. This event simulates walking/crawling on joists in an attic to search for the extension of a fire. Event 6: Roof Ventilation Candidate stands on a pitched rooftop and strikes the roof 30 times with an 8-pound sledge hammer. This event simulates making a ventilation opening in a roof with an axe or sledge hammer in order to ventilate a fire. Event 7: Victim Removal Candidate carries or drags a 154-pound dummy for 26 feet. This event simulates removing an injured or unconscious victim from a hazardous area. Event 8: Ladder Removal/Carry Candidate removes a 24-foot aluminum extension ladder from mounted hooks, carries the ladder 54 feet around a diamond shaped course, and replaces the ladder to mounted hooks. This event simulates removing an extension ladder from an apparatus, carrying it to the site of use, and replacing it to the apparatus. Event 9: Stair Climb with Hose Candidate climbs four flights of stairs with a 49-pound hose bundle and places bundle on the landing of the fourth floor. This event simulates climbing stairs with a hose bundle in order to facilitate the extinguishment of a multiple story fire. Event 10: Crawling Search/Stair Descent Candidate crawls on the platform of the fourth floor for a distance of 60 feet. Candidate places the hose bundle on his/her shoulder and takes it back down the four flights of stairs. Candidate places bundle down at the bottom floor. This event simulates crawling while conducting search and rescue, hose deployment, or attic operations in a burning structure. Event 11: Stair Climb with Equipment/Hose Hoist Candidate picks up two air bottles (with shoulder strap) and returns up the stairs to the third floor. On the third floor the candidate places the air bottles on the ground and hoists a 100-foot section of an extended hose line up and over the balcony rail. The candidate then picks up the air bottles and returns to the ground floor to end the test. This event simulates hoisting an extended hose line up a multiple story building to facilitate fire fighting operations. It is important to keep in mind that these events will be customized to fit the specifications resulting from the data provided by Round Rock Fire Department firefighters. Some of the events may be significantly modified and one or two of the events may fall out of the test if there is not sufficient evidence to support its inclusion in the final PAT. There are three important, and simple, steps required for transporting the PAT to your department. These steps involve completing three validation surveys: The Job Analysis Survey; the Critical Events Survey; and the Tool & Equipment Survey. FPSI will randomly select a group of incumbent firefighters to complete these surveys: ▪ Job Analysis Survey–Incumbents will assign various ratings (i.e., content, frequency, importance, performance differentiating) to those critical physical job duties from our validated entry-level firefighter job description. ▪ Critical Events Survey–Incumbents will provide detailed information on the specifics of how they perform the physical aspects of their job (e.g., the length of hose used, how far they drag a charged and uncharged hoseline, the type of roof they most typically work on, etc.). ▪ Tool & Equipment Survey–One Fire Captain or Training Officer will complete a short Tool & Equipment Survey. This survey will be used to identify the weights, lengths, and sizes of the various tools and equipment used by firefighters on the job. Random Sample of Incumbent– Round Rock Fire Department will provide FPSI with a list of all paid incumbents who perform fire suppression duties by station and shift. FPSI will randomly select a group of incumbents to complete the Job Analysis and Critical Events Surveys. The completion of these surveys should take no longer than two to three hours. FPSI will select a diverse group of incumbents from the various stations and shifts to avoid leaving the department with too many vacancies at any particular station or shift. Once all validation surveys have been completed by the randomly selected incumbents and returned to FPSI for analysis, FPSI will determine which of the eleven test events can be transported to Round Rock Fire Department. Upon the completion of this step, FPSI will then provide Round Rock Fire Department with a “Test Administration Handbook” to use for the construction of the props. Any and all props required for this PAT are very basic and require no more than some plywood, hammer, and nails. Our PAT does not require the use of any equipment that the fire department does not already have on hand. We have been told by other clients that the cost associated with the construction of the props is about $2,000-$2,500 in materials. The “Test Administration Handbook” specifically outlines the specifications of the necessary props and how the course should be laid out. Once the props have been constructed and the course has been set-up, the randomly selected incumbents can then participate in the norming process to identify an appropriate cutoff time for the test. Norming of the PAT: Norming the PAT is quite simple. Those randomly selected incumbents will all run through the PAT course and their times will be recorded by the lead proctor. Incumbents will be required to wear their turnout coat, gloves, helmet, and an SCBA (not breathing air). All of the times will be recorded and returned to FPSI for analysis. FPSI will use a variety of statistics to determine an appropriate cutoff time to be used for selection purposes of future candidates. Costs: The costs associated with this project are based upon the size of your department. FPSI would provide Round Rock Fire Department with a ground-up PAT complete with a customized cutoff time and a Content Validity Report to use on new hire candidates and incumbents, and own, in the event that the test was ever challenged for $18,000. Round Rock Fire Department would be provided with different cutoff options for the candidate (i.e., “New Hire”) test and the incumbent test. This is a significant savings off of our standard validation rates and is a one-time fee. This document will be invaluable to your agency should a candidate ever challenge the validation behind the PAT. If you elect to have an onsite consultant certify your site and attend the “norming process,” the one-day training fee is an additional $2,000 and an additional $1,500 for every day thereafter up to three days of attendance. This is an optional service, but many clients take comfort in having the site certified and an onsite consultant to ensure that the proctors are properly trained at running candidates and incumbents through the course. In addition to the onsite daily fees described above, all travel costs would be paid by the department (e.g., hotel, airfare, transportation, per diem, etc.) according to our current billable rates. Please note that at this very moment, we are not offering onsite certification during the pandemic as our staff is not conducting any nonessential travel at this time. This could change in the coming months. In the event that you do not elect to have an onsite consultant, rest assured that your staff will have all of the materials and site certification forms it needs to properly set up the course without having an onsite consultant. The vast majority of our clients decline this optional service. Neither Kerrville, Victoria, Eagle Pass, Baytown, Taylor, nor Marble Falls elected to have an onsite consultant and were quite successful in setting up and running their course using our offsite guidance. Timeframe of Validation The time frame for this project is about 4-6 weeks, assuming that all surveys are returned to FPSI in a timely fashion and the norming process is completed shortly after the props are constructed. Listed below are the requirements for this project: Task Responsibility Amount of Time Send all necessary PAT surveys FPSI One day Complete all necessary surveys RRFD One week Send back all necessary surveys to FPSI RRFD One day Analyze Data from Surveys FPSI One week Send out Test Admin. Handbook FPSI One day Construct Props and Layout Course RRFD Two weeks Run Incumbents Through PAT RRFD Two to Three days Send PAT Test Times to FPSI RRFD One day Analyze Times/Recommend Cutoff FPSI Two days Write Content Validity Report FPSI One week Total Time Necessary Approximately 4 to 6 Weeks List of References Name of City: Lake Travis Fire Rescue Location: 15304 Pheasant Ln. Ste 100, Austin, TX 78734 Contact: Robert Abbott, Fire Chief Phone: (512)266-2533 Date of Service: August 2012/June 2015/April 2018 Name of City: Kerrville Fire Department Location: 87 Coronado Drive Kerrville, TX 78028 Contact: Eric Maloney, Fire Chief Phone: (830) 257-8449 Date of Service: December 2015 and April 2022 Listed below are a number of projects we have worked on in a similar capacity to your request: List of Fire Service Job analyses Completed for other Jurisdictions In 1995-1997, 41 departments participated in a job analysis study with B&A focusing on 70 duties relating to physical abilities. The participating departments were: Los Angeles County Alhambra Hermosa Beach Redondo Beach Arcadia Inglewood San Gabriel Beverly Hills La Verne San Marino Burbank Long Beach Santa Fe Springs Covina Los Angeles City Santa Monica Culver City Los Angeles County South Pasadena Downey Lynwood Vernon El Segundo Manhattan Beach West Covina El Monte Montebello Torrance Gardena Monterey Park Glendale Pasadena Orange County Anaheim Fullerton Orange County Brea Huntington Beach Santa Ana Costa Mesa Newport Beach Fountain Valley Orange City Several of the departments listed above also participated in earlier consortium job analysis and/or test validation workshops with B&A in 1977 and/or 1990. Several agencies above also completed custom job analysis projects with B&A. Other agencies not listed above for which B&A or FPSI has completed job analysis projects include but are not limited to: City of Bakersfield, CA 1998 City of Clovis, CA 1998 City of Fresno, CA 1998 County of Kings, CA 1998 City of Sanger, CA 1998 Anchorage Fire Department, AK 1999 Baton Rouge Fire Department, LA 1999 City of Oceanside, CA 1999 North Lake Tahoe FPD, NV 2001 Soldotna FD, AK 2002 Cripple Creek, CO 2005 Anchorage FD, AK 2006 Soledad FD, CA 2007 Fresno FD, CA 2007 Weber Fire District, UT 2008 Mount Pleasant FD, SC 2008 Wylie Fire Department, TX 2009 Concord Fire Department, IN 2011 Tacoma Fire Department, WA 2011 Tallahassee Fire Department, FL 2012 Aberdeen Fire Department, SD 2012 Lake Travis Fire Rescue, TX 2012 Honolulu Fire Department, HI 2012 Charleston Fire Department, SC 2013 American Fork Fire/Rescue, UT 2014 Central Mat-Su Fire Dept., AK 2014 Lake Travis Fire Rescue, TX 2015 Kerrville Fire Department, TX 2015 Baltimore City Fire Dept., MD 2016 Victoria Fire Department, TX 2017 Albemarle County Fire, VA 2018 Hastings Fire & Rescue, NE 2019 Santa Fe Fire Department, NM 2019 Baytown Fire Department, TX 2020 Washington Fire & Rescue, NC 2020 Marble Falls Fire, TX 2022 Ventura County Fire Dept., CA 2022 Harris County ES #12, TX 2022 Taylor Fire Department, TX 2023 Hobbs Fire Department, NM 2023 Captain O’Brien, if you have any questions about this proposal or any of the steps involved in the process, please feel free to contact me directly at 916.294.4242 x. 245. If you are interested in having FPSI conduct a webinar for your decision-makers to attend a Power Point presentation on this process and to answer any additional questions you might have please let me know and I will be more than happy to schedule a webinar in the coming weeks. There are some significant differences between our PAT and the C-PAT that make validating and administering our PAT much more user-friendly (e.g., course can be set up outside, there are no required practice test sessions, etc.). We have had great success with our PAT across the country. This is the world’s largest PAT consortium and we are pleased with the diversity and quality of candidates produced by this test. Additionally, this test has been used since 1996 to screen tens of thousands of candidates and has never been challenged. I strongly encourage you to contact Chief Bobby Abbott and/or Chief Eric Maloney as a reference to our products and services. Their departments have used this PAT for their entry-level firefighter test for several years now, with great success! I look forward to hearing from you soon and possibly setting up a time to discuss this proposal in more detail. Sincerely, Stacy L. Bell, M.S. President Phone: 916.294.4242 x. 245 Email: stacy@fpsi.com