The quantitative method
The quantitative method seeks to collect large amounts of "hard data" - i.e. information that can be measured and quantified. It typically measures fixed parameters via specific questions or themes that are sought to be uncovered.
This is often done through 360-degree evaluations, where each manager's leadership is assessed by the manager themselves - and by their superiors, subordinates and/or peers. Some managers find it useful to include multiple respondent groups on some questions, such as boards of directors and/or specific employee groups that the manager works with but doesn't directly report to. These are 450- or multi-degree evaluations.
The topics that are evaluated can be classic leadership competencies specified by various leadership experts - or perhaps by the organization's own code of good leadership defined in a leadership framework. Leadership in the organization can be measured uniformly in the organization, regardless of what you are the leader of - or leadership can be evaluated based on what role you have - for example, whether you are CEO, manager of employees or perhaps a professional coordinator through the ideas in the Leadership Pipeline (Charan, Drotter & Noel, 2011). It may also be that you want to evaluate leadership on some individual elements, but do not necessarily want to get "all the way around" - for example, by measuring the leader's versatility, which is crucial for leaders to succeed and be effective in their leadership (Kaplan & Kaiser, 2006).
Another way to approach leadership evaluation is by measuring the leadership of leadership groups instead of just the leadership of individuals. Extensive research has been done in this field, with more and more people choosing to measure the parameters within leadership groups that are crucial to whether leadership groups succeed or not (Bang, Midelfart, Molly-Søholm & Elmholdt, 2015).
The decisive factor in the choice of methodology is not necessarily what type of management we choose to evaluate or who we choose to ask. What matters is the purpose of the evaluation. With the quantitative method, we can uncover how a manager is assessed on various fixed parameters through specific questions or themes. If, on the other hand, we want to know more about what the individual manager could do differently or if we want examples of what the manager is already doing that works, qualitative methods are the obvious choice. Quantitative methods are especially useful when we want to compare data across an organization or perform specific analyses such as benchmarking. In addition, continuous quantitative measurements allow us to monitor how the leadership quality of a number of strategically important leadership competencies develop over time in numbers.
The qualitative method
Unlike quantitative methods, qualitative methods seek to obtain data that can be analyzed without the use of statistical models.
This is done, for example, through interviews, follow-up discussions at meetings or observations. These methods make sense when the purpose of the management evaluation is more about supporting learning and development than control and strategic management. Especially if the evaluation seeks to bring out more nuances. Here, we can work with a fixed hypothesis that we want to investigate - or we can take a more anthropological approach, where the method evolves as we learn more as we investigate. After 10 interviews, we've usually realized something new and should start asking questions in new ways. This is allowed in qualitative methods, whereas in the quantitative approach it is important to ask the questions in the same way (Morison, 2011).
Leadership interviews can be conducted one-on-one or in groups. The interview can be conducted by a consultant. It could also be the manager themselves - or the manager's managerial colleague. In reality, good interviews are difficult to do well, but when done competently, they are one of the most important and effective methods for understanding how specific individuals understand and experience relationships, situations and events (Tanggaard & Brinkmann, 2015). We can uncover many more nuances about the individual leader or group's leadership that we wouldn't be able to spot in even a very carefully crafted questionnaire. When analyzing data, it is important to keep in mind that the interview is not a neutral technique, but an active interaction between two or more people (Fontana & Frey, 2005). For example, there are various power imbalances when the manager or the manager's colleague conducts the interview. Before starting an interview, the purpose of the interview must be completely clear to the participants, as it is crucial to the outcome (Kvale, 2006). Is it an interview of a manager's employees to evaluate a manager's leadership - or that of a management team? In other words, a regular management evaluation? And how are these statements used afterwards? Perhaps it's an interview of a management team to clarify why last year's management evaluation didn't lead to the desired results, so that we can make changes this year?
We can also work with interviews in less formal setups, such as weekly whiteboard meetings on specific KPIs or monthly/quarterly staff meetings and debriefs, which can help create reflection and forward-looking learning about leadership (Dinesen et al., 2014). Here, both a forward- and backward-looking focus can be created based on questions such as
- What went well in our management and collaboration - and why?
- What can we do better?
- Where do we want to go with our leadership and collaboration?
- How do we make sure we get there?
Here, instead of the "annual management review", we move towards an ongoing dialogue about good leadership and collaboration in the organization, which also connects to the development of a general culture of evaluation and feedback in the organization.
Observations can be another way to qualitatively investigate, which can be done both inside and outside the organization. However, when studying management, it is most obvious to observe in the usual settings where management is carried out. A consultant or management colleague can observe the manager in specific contexts and provide feedback. This can also be the observation of management group meetings, where video observation is a known method that makes it easier to provide concrete feedback with clear examples through video sequences. Sometimes as an observer you are completely passive - other times you are part of the context. In the past, observation has been referred to as the "fly on the wall", but we have gradually begun to declare the fly
dead. We need to be very clear that we cannot observe without influencing. Therefore, we cannot talk about collecting data, but rather about "generating" data. This applies regardless of the methodology.
Observation makes sense when we want to be open in our research method and want to investigate a practice without necessarily having a hypothesis beforehand. Observations are also useful when we want to investigate conditions that are otherwise inaccessible through other methods. Observation can also be used to subsequently be able to ask the right questions if we choose to "mix" the methods.
READ MORE ABOUT OUR EDUCATION
LEAD
offers certification in the development of agile leadership with the development tool The Leader's Versatility Index (LVI)
With a certification, you will be equipped to use LVI in development processes in your organization at the individual, group and organizational level.
Mixed methods
Mixed Methods is, in the pure sense of the word, when we mix methods. We can do this when we want to create more knowledge, create more revealing or complementary knowledge, or when we want to create more complex knowledge than is possible with one method alone (Greene, 2007). This is often referred to as triangulation, complementarity or complexity, depending on the purpose of the study (Frederiksen, 2014).
When we use triangulation in management evaluation, we use two different methods to investigate the same phenomenon. For example, we may send out a questionnaire and then conduct interviews or observations to investigate the same elements. In this case, the two methods are used to confirm the findings of the analysis with both methods. There could also be different needs to be met. For example, the executive board wants numbers on which leadership competencies are strategically needed to develop - or that managers and employees have a greater need for local learning and development through dialog.
With complementarity, we seek to gain more revealing or complementary knowledge by investigating several areas that are linked to each other. This could, for example, be the connection between wellbeing and leadership - or that after a questionnaire, interviews are conducted to elaborate on the answers in the questionnaire survey. In this case, you don't expect to create a validation between the methods, but the results should be seen as supplements to each other.
In the complexity approach, it's the more conflicted or multifaceted description of leadership that we're after. Here, the intention is to depict "reality" with whatever methods are necessary, which can be both myriad and contradictory - just like reality.
Rounding up
Regardless of which method or methods are chosen to investigate leadership, the most important thing is first and foremost a good analysis of the data that comes in - but most importantly: A good follow-up and "sensemaking" based on the analysis afterwards, which can initiate good dialogues about and development of leadership. The evaluation itself is worth nothing - regardless of the method - if the results are not actively used afterwards. Ideally, through the various evaluations, we should develop a culture where it becomes natural to give and receive feedback on a daily basis.
