Scenario Village Proctoring

Shon Christensen (PF&R) guides Verde NET through Scenario Village. May 06, 2023. Credit: Jeremy Van Keuren.

- Contributing writer: Amanda Westervelt, Portland Bureau of Emergency Management, Operations Specialist -

Proctors are a critical part of the Scenario Village exercise experience. Proctors support Survivor Volunteer actors, observe the players as they work through the exercise, evaluate demonstrated skills, identify areas for improvement, deliver injects, provide simulated effects and responses, and monitor the safety of all participants during the course of the event. At the end of the exercise, a Proctor provides the NET players with constructive criticism and positive reinforcement.

The person in charge of the exercise is the Exercise Coordinator. This person is responsible for (or delegating) the exercise design, gathering the needed resources for the exercise (such as props, actors, moulage technicians, proctors, food, etc), communicating with volunteers before the exercise, and designating Proctor responsibilities.

Every Scenario Village exercise should include at least two Proctors, and preferably more. There are several different Proctor roles, and an exercise with few Proctors will mean each individual proctor taking multiple roles (as well as a less safe exercise).

...

...

Preparing to Proctor

Alice Busch and Jeremy Van Keuren proctor at Operation Fort Rock. December12, 2018. Credit: Kathy Reese.

Before the exercise begins:

• What to bring: Clipboard, paper, something to write with, flashlight, and a whistle. You'll do a lot of walking, so wear comfortable footwear. Also, there is no climate control in the Village; dress accordingly.
• Have the Exercise Coordinator's cell phone number. PPB Training is a secure facility with access control between areas, meaning people may get locked out of the building or different parts of it. The Exercise Coordinator can get doors unlocked and people escorted back in.
• Know what the different lanyard colors mean. If you don't, see: Scenario Village Participant Roles and Responsibilities.
• Know the scenario: Maybe an earthquake just happened, we are on a typical city block, several survivors are scattered throughout the village with various injuries, they need assistance, and here comes a NET team…

• Know the objectives of the exercise: Why is the Exercise Coordinator doing this? Generally this is to provide a team of NETs and opportunity to practice ICS, radio, search & rescue, and medical triage skills. Other objectives could include practicing patient management, de-escalation and mental health first aid.

• Know the NET players’ objectives: Is this an easier scenario for first-time players? Or is this a more complex scenario for a team that wants to really practice a very specific set of advanced skills? You’ll want to know what you should be evaluating.

• Familiarize yourself with the Village if you haven’t been there before. You can download a map HERE. You can see pictures of the buildings HERE.

• Ask the Exercise Coordinator if there will be any injects, and if you are providing them; when, where and how. Injects could be gas release sounds or an announcement that an aftershock just occurred, etc.
• Know any simulations you’ll need to provide. For example, telling players if they touch a “live” electrical wire and have to sit out for 5 minutes due to injury, or you may pretend to be the operator if a player pretends to call 911.

Once you have this information, you can choose to create a checklist of critical tasks – specific actions that are required for players to meet their goals, formally known as capability targets. By creating a checklist in advance, you become a more thoughtful, objective, and consistent evaluator. Since teams may run through this exercise two or even three times in a session, consistent evaluation will clearly show where they improve over time. The formal HSEEP Exercise Evaluation Guide for search, rescue, and triage can be found HERE. Feel free to use it, or to develop your own checklist format.

...

Pre-Exercise Roles

The following is a list of the different possible roles the Exercise Coordinator might assign to Proctors before actual play begins:

Front End Proctor

The PPB training facility is divided between the lobby area and Scenario Village itself. For security reasons, access is controlled by locked doors between the two locations. Therefore, some proctors may be assigned to the "front end" lobby area before the exercise while other proctors set up the Village. Front end proctors should:

• Monitor their phone for calls and texts from the Exercise Coordinator, who will be setting up the Village.
• Let in, greet, and sign in volunteer Survivors, Players, and other participants.
• Help moulage techs set up if they need help, and make sure carpets and furniture in their area is sufficiently protected.
• Make sure Survivors are comfortable and know where restrooms are.
• Keep Survivors and Players in separate rooms (Players should avoid seeing Survivors before play).
• Remind Players and Survivors to check themselves for sharp instruments and to leave them in the lobby/moulage room for pick up later.
• Give the Survivor Volunteer Safety Briefing while Survivors get moulaged.
• When given the "OK" by the Coordinator, lead the Survivors into the Village through the security check area.

...

Village Proctor

Village Proctors will join the Exercise Coordinator to set up the Village for play. The Coordinator may or may not have a specific exercise plan; if they do, they will prescribe the locations of props and survivors. Village proctors will work with the Coordinator to:

• Set up props such as "dangerous" electrical wire, smoke machine, "debris", strobe, white noise generator, gas meter, NFPA diamond, etc.
• Hang sheets of paper at building entrances for building markings.
• Do a safety walk-through before the actors come in. Look for any acute hazards.
• Place incoming Survivors in different Village locations.

...

Prep and Safety Briefings

For Players

Alice Busch delivering a Prep and Safety Briefing to Players during Operation Watzek. September 22, 2018.

The safety briefing is given before the Players move from the lobby to the staging area. It is usually given by the Exercise Coordinator, but they may delegate it to a Front End Proctor. It should not take more than a few minutes. Remind Players:

1. To sign in if they haven't already done so.
2. To yell "REAL WORLD" to stop the exercise if they observe anything unsafe taking place (they should trust their gut too).
3. To be aware of their own stress reactions and those of their teammates. Stress is used in SV so players can practice using the parts of their brains that are often impaired in stressful situations. There is no problem with "tapping out" to take a break. Check in with each other on stress levels.
4. (If we brought the beads)...wear Mardi Gras beads if you do not want to be photographed.
5. To do a sharp instrument check...are you sure you don't have any on you?
6. Not to carry Survivor Volunteers down stairs. But instead, to let them stand and walk down and then resume the patient carry.
7. Have a flashlight for darkened areas. If you don't have a flashlight, don't go in.
8. What the different lanyard colors mean.
9. To leave PPB Training in better shape than we found it; don't leave any garbage or anything like that, or we don't get invited back. We are guests here.
10. The basic exercise flow: they'll stage, go into the village, assess, and render aid.
11. Not to take too long for assessment. And acknowledge people you see who need help even if you're not rescuing them yet...don't just ignore them. Talk about "response tempo".
12. If this is a mixed group, indicate the Team Leader.
13. To have fun, and do not stress; it's only a FAIL if someone is hurt in the exercise. Otherwise, this is the place to make mistakes.
14. There will be a hotwash at the end.

If proctors have time, we also recommend reviewing this brief from NW District NET TL Nick McCullar on trauma-informed proctoring for the exercise.

...

For Survivor Volunteers

The safety briefing is given by a proctor before the Survivor Volunteers move from the lobby to the staging area. Be sure to leave adequate time for questions. Let Survivor Volunteers know:

1. To sign in if they haven't already done so.
2. To yell "REAL WORLD" to stop the exercise if they observe anything unsafe taking place (they should trust their gut too).
3. (If we brought the beads)...wear Mardi Gras beads if you do not want to be photographed.
4. To do a sharp instrument check...are you sure you don't have any on you?
5. They should never be carried down stairs. But instead, let them stand and walk down and then resume the patient carry.
6. What the different lanyard colors mean.
7. To leave PPB Training in better shape than we found it; don't leave any garbage or anything like that, or we don't get invited back. We are guests here.
8. The basic exercise flow: staging, going into the village, assessing, and rendering aid.
9. Get ready for the frequent "Hurry up and wait" of being a Volunteer Survivor.
10. There will be a hotwash at the end and they are welcome to participate and offer how things went from their perspective.
11. Situation with lunch and refreshments.

...

Proctoring During the Exercise

Lt. Matthew Silva and Lt. Laurent Picard taking notes during Operation Red Fern. December 12, 2015. Credit: Ernest Jones.

Actors are hidden; play begins. The NET team arrives, and the actors start acting. You take out your clipboard and become a fly on the wall.

Remember the role of the proctor is not to teach or provide guidance. Scenario Village is a place to practice, and the hot wash is the place at Scenario Village where we learn. If you have comments regarding skill performance or methodology, write it on your evaluation to share at the hot wash. The only time you break this rule is if safety of any participant is at risk.

As the NET Team Lead instructs the team and starts sending out search and rescue strike teams, you:

• Observe.
• Inject.
• Simulate, and
• Monitor for safety.

And have fun doing it!

Hotwash

The Scenario Village hotwash is an opportunity for all participants to decide how they think the exercise went. It is important not to frame in terms of "success" or "failure" since this is a practice response. What is more important is what people learned.

A hotwash session at the end of Operation Sasquatch. May 02, 2015. Credit: Ben Pham.

Begin by affirming their hard work. Often, no matter what happened during the exercise or how well they performed, volunteers feel like they failed...be sure to disabuse them of that notion.

Thumb: Something that went well

Index finger: Something you'll remember for next time

Middle finger: Something that went not great

Ring finger: One relationship you're going to keep from this network

Pinky: One thing you promise to do as a result of this exercise

• Organizational roles and responsibilities are clearly identified
• I understood my role in the response
• The incident objectives were met
• Participants communicated clearly
• I brought the right gear with me

JOB AID: Scenario Village Coordinator Checklist

1 - 3 MONTHS PRIOR
Set up NET responder signups
Confirm teams and shifts, send Wiki page
Recruit volunteer survivors, set up signup
Recruit moulage
Recruit proctors	Optimal is one proctor for every 10 responders/survivors on site.
Complete lunch/refreshment purchase paperwork	Recommend $300 to spend in a typical exercise; lunch is for volunteer survivors, proctors, and moulage only.
1 - 2 WEEKS PRIOR
Send confirmation email to PPB Training
Email responders, re-send Wiki page
Email moulage and proctors, include "injury" needs
Print sign-in sheets (responder, survivors, moulage/proctors)
Assess Scenario Village shuttle box	Shuttle box should have lanyards, injury descriptions, clipboards and pens, extension cords, tape.
Grab large post-it pads	These will be used for building markings.
Prep PA system, test with laptop, get microphone
Gather other props as needed	May include electrical cable (for a "down power line"), gas meter and SCBA tank (for a simulated gas leak), building signs, mardi-gras beads.
NIGHT BEFORE/DAY OF
Order lunches	This should be done the night before if pizza...if catered, this should probably be done at least a week prior.
Do any grocery shopping for snacks
Order and pick up coffee	There is a Starbucks at 5826 NE 122nd Ave (503-252-3880); order one "Coffee Traveler" for each shift...takes 40 minutes to brew, so time accordingly.
Make sure this page is on your phone
1 - 3 DAYS AFTER
Prep/email feedback survey to responders and survivors
Input exercise to the CRT Activities sheet