Getting better SME and group feedback
If your team makes decisions on the fly after off-the-cuff discussions with Subject Matter Experts (SMEs) and Stakeholders–and your business is thriving, congratulations. You’ve been incredibly lucky. To ensure your good fortune continues, back off from the cliff’s edge and consider the benefits of Precision Questioning/Precision Answering. (PQ/PA). Tens of thousands of people in universities and companies throughout the world have studied different versions of this system devised at Stanford University by Dennis Matthies and Dr. Monica Worline in the early 1990s. They helped co-found Vervago, the primary company that has been training employees of the nation’s top corporations in how to improve critical thinking skills and the effectiveness of decisions since 1995. I found their certification program to be the most valuable of any I have ever experienced.
PQ/PA integrates with all other decision-making methodologies
PQ /PA is a proven system that significantly improves the efficient, accurate exchange of information required in making effective decisions. It integrates beautifully with other decision process-related communication techniques like rapport building and brainstorming (describe in my last post).
Specificity is all too often lacking both in questions that we ask others and the responses they provide. This gets us in trouble in our personal lives and can lead to fatal business decisions.
PQ/PA clearly exposes gaps in thinking by helping decision makers uncover weaknesses in thinking and to raise the objectivity of fact-finding and decision-making discussions.
Basic PQ/PA principles
PQ/PA requires drilling into a subject to discover critical facts that would otherwise be missed. To do this effectively, certain guidelines need to be respected–
- Practitioners need to demonstrate respect for conversation-partners. There’s no room for shame/blame.
- As the questioner, you should hover and float, while you judge possible points of entry. In the initial phase, you build an overview, size up the situation, and figure out what you want to accomplish. Later, your overview may change and your intention may change.
- The answer is always a simplification. In the ideal PQ/PA scenario, the questioner can focus only on whatever is most important.
- Use questions that relate to the current situation. Avoid tangents.
- In group decision making meetings, carefully explore what is really important and perhaps will require additional research. This helps the team move beyond issues that either have no relevance or immediate resolution.
- The answerer should respond with the core first, then details. In other words, a summary of the answer, then elaboration as needed.
- Let social factors shape how you ask but not what you ask. The culture of your team or group will dictate how you should ask questions, but not what you should ask. Use PQ/PA to explore the range.
Next, I’ll describe PQ/PA’s touchstone values 7 categories of questions, with examples.
Precision questions and precision answers
It was one of my most effective training sessions at Microsoft.
My manager encouraged me to take PQ / PA so that I would be more effective in executive reviews and our group was full of avid PQ / PA practitioners. I had a tendency to tell stories and elaborate, so this was about cutting to the chase crisply, with brutal effectiveness. I took the class a few years back, so my notes may be a little rough.
Improve Your Communication Efficiency and Critical Thinking
You can think of Precision Questions and Precision Answers as a way to improve your communication efficiency and improve your critical thinking. It works by asking and answering questions from 7 categories.
Rather than trying to randomly be smart on the fly, it’s a structured approach that helps you test your thinking.
The rumor I heard was PQ / PA was required training for executives because Bill Gates asked too many questions that people weren’t prepared for. Precision Questions and Precision Answers was the solution. Since then, many Softies use it to improve their own thinking, prepare for executive reviews, analyze project plans, or just about any scenario where the complexity is high, and information exchange is important.
Seven categories of precise questions
There are 7 types of questions for focused drilling down.
- Go / No-go. Do we need to talk about this?
- Clarification. What do you mean?
- Assumptions. What are we assuming?
- Basic Critical Question. How do we know this is true?
- Causes. What’s causing this?
- Effects. What will be the effects?
- Action. What should be done?
Note that these are categories for questions. In other words, within each category, you can ask a variety of questions. The other key point is that you would start with a high-level question in the category, and only drill down as needed, to test credibility, find assumption, or gain information.
Principles of precision questions and precision answers
Here are the key values behind PA / QA:
- Candor. No “show and hide.”
- Intellectual commitment. As the questioner, you stand behind your question and as the answerer, you stand behind your answer
- Depth of preparation, analytical skill, and integrity
Credibility is a key concept in PQ / PA. When you know how to do Precision Questioning and Precision Answering, it’s easy to see 1) depth of preparation, 2) analytical Skill and
3) integrity. This works both ways. In other words, it tests and exposes the questioner and the answers.
- Tone, pace, and conciseness
There are three dimensions to style:
- Tone. Ranges from helping to neutral to impatient to punishing.
- Pace: Ranges from slow and thoughtful to rapid fire.
- Conciseness. Ranges from every word counts to conversational.
- Avoid dangerous simplifications
It’s important to remember that answers are always simplifications. Some are more dangerous than others:
- Give a warning flag + a core answer. When you’re giving an answer that has some caveats, call them out. For example, *based* on what we know today __ *Best* case would be __ … An *aggressive* date would be __. Off the top of my head.
- Do torpedo alerts. As the questioner, you might ask, “Is there anything else I need to know?” As the answerer, you might say, “You haven’t asked about __” or “It’s important that you know __.”
Rather than say the questioner doesn’t know enough to ask the right questions, make an offer. For example, “Would you like me to explain?” or “Would a definition (map, analogy) help?”
Go / No-go Questions
Here is an example set of Go / No-go Questions:
- Basic process questions. What is our agenda? What is our goal? Do we need to adopt any ground rules?
- Motivation questions. Is this urgent to discuss? Is this important? Is this interesting?
- Participation. Do I need to be here? Do you need to be here? Who else should be here? Should we take this off-line?
- Questioning our questioning. Are we focused on the right ideas? Are we asking the right questions? Are we getting to the heart of the issue? Are our questions benefiting the discussion?
Here are some example cause questions:
- What’s the story?
- What’s the mechanism?
- What triggered it?
- What are the forces?
- What are the root causes?
- What are contributing causes?
Here are some example action questions:
- Who should act? By when?
- What are the alternatives?
- What work won’t get done?
- How confident are you?
- Is this a root cause fix?
- Is the problem only going to be contained?
- What will be the strategy?
7 Categories of precision questions
All methods of effective communication require clarity, open dialogue, accurate information and showing respect for others. PQ/PA is a carefully developed system for drilling into a subject to discover critical facts too often missed in the decision-making process.
The story goes that Microsoft required PQ/PA training for all its executives after Bill Gates found his staff was often unprepared to answer many of his questions, both one-on-one and at meetings. It proved so valuable that management subsequently disseminated it throughout the corporation to improve outcomes in other scenarios characterized by complexity and a need for precise, accurate interchange.
Underpinning Values PA/QA can only work if leaders themselves model candor, courage, thorough preparation, analytical skill and a commitment to pursuing all relevant facts. While a qualified–“This is the best answer I have now based on available information” is OK, glossing over information gaps is not.
Categories of precision questions
There are seven types of questions in this system. Within each category, there is a wide array of possible inquiries. Participants begin by asking high-level question and drill down, as needed, to test credibility, uncover an assumption, or gain information.
- Go/No-Go questions begin with asking whether investigating an issue/launching a project should be initiated or dropped. If a ‘go,’ follow up inquiries would define the ‘big picture,’ long-term objectives, an agenda, future participants, and ground rules.
- Clarification questions help avoid ambiguity, as in “Do you mean__or___?”; nailing down when/where/frequency; requesting examples, e.g., range of uses; requesting graphs and other mathematically-based information; and paraphrasing to ensure understanding.
- Assumption questions explore the validity of basic facts/issues. This includes categorizing, making comparisons with an eye to subtle distinctions, defining applicable/target audience(s) and the current and likely future likelihood of change.
- Basic Critical questions address the quality (reliability and validity) of data, triangulation and other confirming/disconfirming methodologies, the credibility of subject matter experts and the cost effectiveness of new, in-depth research.
- Cause questions encompass triggering factors and their context, root cause analysis, defining drivers and inhibitors, feedback loops and optimal intervention points.
- Effects questions help define likely short and long term outcomes; impact in stakeholders; alternative outcome scenarios; statistical probability of excellent to poor outcomes; and possible unintended consequences.
- Action questions identify what needs to be implemented to fix root causes; who will be assigned responsibility for different aspects of implementation; timelines; budget; and how action(s) may impact long-term business interests.
Lastly, I’ll provide an overview of guidelines for providing precision answers to questions.
So, how can PQ/PA have value in this all-too-common situation?
Andrew Mani, our senior project manager, will post an article this coming Tuesday on crisis management. He’ll be sharing guidelines for dealing hands-on with unexpected challenges in implementing new products/services. Among his ‘lessons learned’–managers dealing with emergencies seldom have time to hold staff meetings or do new research.
So, how can PQ/PA have value in this all-too-common situation?
IF your team has applied some version of Precision Questioning/Precision Answering prior to the crisis, you will have immediate access to extensive baseline research information multiple sources, greatly improving the odds of a successful outcome.
Precision answer guidelines
I described seven categories of precision questions that help team members explore the full scope of any inquiry and then drill down to the most relevant information.
Answering questions to improve discovery requires a similar process.
In order to provide more precise, accurate feedback you need to–
- Stay on point; limiting your answer to the question asked.
- Come to the meeting prepared to address core issues efficiently and succinctly.
- Be brief; then inquire whether others have additional questions.
- If the answer includes multiple factors, rank them–perhaps writing them out on a flipchart.
- Say, “I don’t know” if you don’t have an answer, but give team members your plan of action for getting the needed information.
- Ask for clarification if a question is ambiguous. “Are you asking about X or Y?”
- Redirect the questioner with your own question(s) if they get off track.
- Qualify your answers to include critical contingencies, e.g., “There are two main risks to taking that course of action; they are______.”
- To ensure nothing crucial is being overlooked, let the questioner know when you have important information that they are overlooking.