Eindrapport_Evaluatie_Opleidingen_In_Bedrijven.pdf by Vlaanderen-be

IMPACTEVALUATIE ESF INTERVENTIE “OPLEIDINGEN IN BEDRIJVEN” Finaal rapport

What drives training transfer effectiveness and how does this transfer work?

Onderzoek uitgevoerd onder leiding van Antwerp Management School in opdracht van het Departement Werk en Sociale Economie Priscilla Álamos-Concha Bart Cambré Josephine Foubert Valérie Pattyn Benoît Rihoux Benjamin Schalembier 2

ACKNOWLEDGMENT The authors would like to thank everyone involved in this evaluation, specifically Benedict Wauters from the Flemish Ministry of Employment and Social Economy for funding this research, professor Derek Beach (Aarhus University) for his guidance and expertise in process-tracing methods and professor Filip Dochy (KULeuven) for his expertise in High Learning Impact. We are grateful for the comments received from experts at the International QCA conferences in Zurich and the 2019 International Public Policy conferences in Montreal. The usual disclaimer applies.

TABLE OF CONTENTS Acknowledgment .......................................................................................................................................... 3 Executive summary ..................................................................................................................................... 10

Introduction .................................................................................................................................... 10

Findings ........................................................................................................................................... 10

Recommendations .......................................................................................................................... 16

Further studies and development .................................................................................................. 18 Introduction ........................................................................................................................................ 21

1.1

Background: ................................................................................................................................ 21

1.2

Purpose, scope and limitations ................................................................................................... 22

1.3

How to read this report .............................................................................................................. 22

Methods and study design .................................................................................................................. 24 2.1

What is QCA? .............................................................................................................................. 24

2.2

Why we chose QCA ..................................................................................................................... 24

2.3

The QCA process in this study ..................................................................................................... 25

2.4

What is Process tracing? ............................................................................................................. 26

2.5

Why we chose Process tracing .................................................................................................... 26

2.6

The application of Process tracing in this study.......................................................................... 26

Data and concept operationalization ................................................................................................. 29 3.1

Case-study data collection .......................................................................................................... 29

3.2

Case-study selection criteria ....................................................................................................... 29

3.3

Conceptualization and operationalization of key conditions ..................................................... 32

3.4

Selected Case-studies ................................................................................................................. 37

Analysis and QCA findings ................................................................................................................... 41 4.1

QCA model .................................................................................................................................. 41

4.2

Single conditions analysis............................................................................................................ 42

4.2.1

Necessity analysis................................................................................................................ 42

4.2.2

Sufficiency analysis.............................................................................................................. 43

4.2.3

The pathways to successful training transfer and the cases in them ................................. 44

4.2.4

Failed training transfer effectiveness ................................................................................. 50

Overarching findings of Process tracing ............................................................................................. 54 5.1

Key causal mechanisms............................................................................................................... 54 4

5.1.1

Signaling and Retention ...................................................................................................... 55

5.1.2

Enhanced learning transfer intervention ............................................................................ 57

5.1.3

Self-management intervention ........................................................................................... 59

5.1.4

Learner agency .................................................................................................................... 63

5.2

5.2.1

Overview of successful cases .............................................................................................. 66

5.2.2

Signaling and Retention triggered by supervisor support: The case of N2......................... 67

5.2.3

Enhanced learning transfer intervention triggered by peer support: The case of D1 ........ 69

5.2.4

Self-management intervention triggered by relapse prevention and goal setting: The case of B3 .................................................................................................................................... 71

5.2.5

Learner agency triggered by sense of urgency: The case of T1 .......................................... 73

5.3 6.

Case studies: Opening the black box to understand training transfer effectiveness ................. 66

Concluding remarks .................................................................................................................... 75

Conclusions ......................................................................................................................................... 78 6.1

Explaining training transfer in Flemish firms .............................................................................. 78

6.2

Understanding how Training transfer takes place in Flemish firms ........................................... 79

Recommendations .............................................................................................................................. 82 7.1

Recommendations for ESF project managers ............................................................................. 82

7.2

Practical recommendations for stakeholders ............................................................................. 82

7.3

Recommendations for further research ..................................................................................... 84

References .......................................................................................................................................... 87

Annex 1: Overview of the evaluandum .................................................................................................... 102 Annex 2: Limitations of QCA ..................................................................................................................... 103 Annex 3: Literature review........................................................................................................................ 104 A.3.1 Approach to the selection of theoretical perspectives ............................................................... 104 A.3.2 Outcome: success versus failure to transfer learned social skills to the workplace ................... 108 A.3.3 Conditions as causes .................................................................................................................... 109 Supervisor support ............................................................................................................................ 110 Peer support ..................................................................................................................................... 110 Sense of urgency ............................................................................................................................... 111 Goal setting and relapse prevention as transfer enhancing interventions before and after training .......................................................................................................................................................... 111 A.3.4 Contextual conditions as organizational facilitators of transfer ................................................. 112 Identical elements............................................................................................................................. 112 Training as aligned with wider strategic organisation goals ............................................................. 113 5

Training policy as a voluntary training attendance policy ................................................................ 114 A.3.5 Contextual conditions related to the training design .................................................................. 115 Training designed as ‘active learning instructional method’ ............................................................ 115 Length of training .............................................................................................................................. 116 Spacing of training sessions .............................................................................................................. 116 Mode of instruction: internal or external trainer, or self-administered training ............................. 116 A.3.6 Contextual conditions related to the employee level ................................................................. 117 Learner readiness.............................................................................................................................. 117 Content acquired in training ............................................................................................................. 117 Workload allowing training and transfer .......................................................................................... 117 Mastery learning goal orientation .................................................................................................... 117 Personal factors ................................................................................................................................ 117 Annex 4: Conceptual framework: combining qca with process-tracing methods. ................................... 118 Annex 5: Process-Tracing methods ........................................................................................................... 122 Annex 6: Causal mechanisms .................................................................................................................... 129 Key causal mechanisms......................................................................................................................... 129 Signaling and retention ..................................................................................................................... 129 Enhanced learning transfer intervention .......................................................................................... 136 Self-management intervention ......................................................................................................... 144 Learner agency .................................................................................................................................. 150 Annex 7: Additional information on data ................................................................................................. 157 Annex 8: Case selection strategy for qca and pt phase ............................................................................ 159 Selection of negative cases ................................................................................................................... 159 Case selection strategy for a post-QCA phase: Process-tracing ........................................................... 160 Annex 9: Qca analysis................................................................................................................................ 161 Necessity analysis for training transfer effectiveness .......................................................................... 161 The eight conditions model .................................................................................................................. 163 Sufficiency analysis for training transfer effectiveness ........................................................................ 165 Necessity analysis for failure training transfer effectiveness ............................................................... 165 Sufficiency analysis for failed training transfer effectiveness............................................................... 168 Robustness test training transfer effectiveness ................................................................................... 169 Dichotomic table ................................................................................................................................... 171 Annex 10: Note on calibration in QCA ...................................................................................................... 173 6

Annex 11: Evaluating the evidence: argument road maps ....................................................................... 175 A11.1 Roadmap Signaling and Retention Causal Mechanism triggered by supervisor support .......... 175 A11.2 Roadmap Enhanced learning transfer intervention triggered by peer support ......................... 188 A11.3 Roadmap Self-management intervention .................................................................................. 206 A11.4 Roadmap Learner Agency ........................................................................................................... 227 Annex 12: Additional evidence from the roadmaps ................................................................................. 243

LIST OF TABLES Table 1: Case-study outcome...................................................................................................................... 31 Table 2: Key conditions, definitions, operationalization and dichotomization .......................................... 32 Table 3: Overview of cases: membership of conditions and outcome ....................................................... 38 Table 4: Sufficient configurations (pathways) ............................................................................................ 43 Table 5: Sufficient configurations for unsuccessful training transfer (pathways) ...................................... 52 Table 6: Overview of grouped cases matching common conditions and contexts .................................... 66 Table 7: Equivalences between Gist et al.’s strategies of self-management and Marx´s self-management steps............................................................................................................................................... 61 Table 8: Contextual conditions necessary for signaling and retention process ......................................... 68 Table 9: Contextual conditions necessary for an enhanced learning intervention .................................... 70 Table 10: Contextual conditions necessary for self-management intervention......................................... 72 Table 11: Contextual conditions necessary for learner agency process ..................................................... 74 Table 12: Process of training transfer in four in-depth case studies .......................................................... 76 Table A1: Overview of potential conditions conducive to ‘value at worker level’ and ‘value at firm level’ ................................................................................................................................................... 105 Table A2: Theory-testing variant of process-tracing................................................................................. 123 Table A3: The temporal dimension of causal mechanisms ...................................................................... 125 Table A4: Probabilities expressed in Bayesian reasoning ......................................................................... 128 Table A5: Causal mechanism Signaling and retention .............................................................................. 133 Table A6: Causal mechanism Enhanced training transfer ........................................................................ 140 Table A7: Causal mechanism self-management intervention .................................................................. 147 Table A8: Causal mechanism Learner agency ........................................................................................... 153 Table A9 Necessity analysis: necessity-consistency-relevance of conditions for successful training transfer ...................................................................................................................................... 162 Table A10 Necessity analysis: relapse prevention and goal setting (in a conjunction) ............................ 162 Table A11 Necessity analysis: necessity-relevance-consistency-of disjunctions ...................................... 162 Table A12 Truth Table for an eight-condition model................................................................................ 164 Table A13 Conservative solution .............................................................................................................. 165 Table A14 Necessity analysis: necessity-consistency-relevance of conditions for not successful training transfer ...................................................................................................................................... 166 Table A15 Necessity analysis: relapse prevention and goal setting in a conjunction for not successful training transfer ......................................................................................................................... 166 Table A16 Necessity analysis: necessity-relevance-consistency-of disjunctions for not successful cases ................................................................................................................................................... 166 Table A17 Truth Table for non-successful training transfer ..................................................................... 167 Table A18 Conservative solution .............................................................................................................. 168

LIST OF FIGURES Figure 1: Pathways to successful training transfer ..................................................................................... 11 Figure 2: QCA procedure............................................................................................................................. 25 Figure 3: The PT process after QCA analysis ............................................................................................... 27 Figure 4: Data collection process ................................................................................................................ 29 Figure 5: QCA model ................................................................................................................................... 42 Figure A1: Defining ontological concepts for the combination of QCA and PT ........................................ 119 Figure A2: Defining causal concepts for the combination of QCA and PT ............................................... 120 Figure A3: The three steps of an in-depth theory-testing process-tracing............................................... 124 Figure A4: Two-stage evidence-evaluation framework for turning empirical material into evidence .... 126

EXECUTIVE SUMMARY 1. INTRODUCTION This study investigates factors and processes driving training transfer effectiveness in Flemish firms. It builds on 50 cases, including 15 successful cases in which in social skills (leadership and stress management) were transferred and 35 failed cases where training transfer effectiveness has not (yet) been achieved. In order to better understand the process of training transfer, we take a more in-depth look at four cases where different mechanisms played a key role in transfer effectiveness. Having a clear view on what causes training transfer and how it causes training transfer will assist in designing more effective learning and development policies. The aim of this research is twofold: (1) to identify patterns across case studies in a systematic and rigorous manner. In particular, it explores the conditions (factors) and combinations of conditions (configurations) associated with successful and unsuccessful training transfer, and (2) to disentangle the mechanisms or processes (the black box) that take place in transfer effectiveness. By ‘successful’ or effective transfer, we mean training content that has been learned in the training context and implemented in the job context by employees and maintained over a period of time as planned, rather than just applied for a short term after the training. This study applies a multimethod design. We use qualitative comparative analysis (QCA) to analyze the cases and determine which (combination of) factors are associated with the outcome. QCA is a suitable and innovative comparative method, which allows for complexity of causality. This means, for instance, that numerous pathways can lead to the same outcome, as has been found in our research. We have used process-tracing methods to study the causal mechanisms that link the conditions or factors and the outcome in a productive manner.

2. FINDINGS In our research, we found that no single condition was required (necessary) for successful training transfer. Rather, several combinations of conditions were sufficient 1 for success, meaning that whenever these pathways were present, the training content was successfully retained and applied to the workplace. We differentiate between causal conditions that could trigger training transfer and contextual conditions that, simply put, allow these transfers to happen. The condition that was most often present in successful training transfer was the contextual condition of training program as an active learning method. How this and other conditions are defined and calibrated is explained in Section 3. However, as the successful training transfer is seldom the result of a single condition, we identified several pathways to success. Figure 1 shows the pathways to successful training transfer in our eight condition model, which includes the following causal conditions: (1) peer support; (2) supervisor support; (3) sense of urgency; (4) relapse prevention and goal setting; and the following contextual conditions: (5) identical elements, (6) training program as active learning method; (7) autonomy, and (8) balanced workload.

See section 4 and Annex 9 for more details on necessity and sufficiency analysis with QCA. 10

Figure 1: Pathways to successful training transfer Case

Peer support

Supervisor support

Sense of urgency

Relapse prevention and goal setting

Identical elements

Training program as active learning method

Autonomy

Balanced workload

J3; V2

B2; K2 M1; D1 N2; B3

W1 T1 S2 T2

Source: authors. Note: White: condition is absent; Grey: condition is present; “-“ not included in the pathway.

Pathway 1 – Absence of support and sense of urgency and presence of relapse prevention-goal setting, within the contexts of absence of identical elements and presence of training program as active learning method and autonomy. Training transfer was effective in cases where employees applied techniques of relapse prevention-goal setting within the contexts of training as active learning method and autonomy. In these cases, peer and supervisor support and sense of urgency did not matter as well as the context of identical elements and balanced workload. This pathway explains two cases out of twelve successful cases in leadership training programs and communication skills. The pathway is significant, especially as it contains two similar cases where the employee was able to cope with relapse prevention-goal setting to transfer under an autonomous work environment. Pathway 2 – Presence of support from peers and supervisor and absence of sense of urgency and relapse prevention-goal setting, within the contexts of identical elements and training program as active learning method and the absence of a balanced workload. An employee training transfer was effective in cases where employees were supported by peers and supervisors, and where the training was aligned with the actual job experience of the employee in equipment, work environment and psychological sphere; where the training programs was carried out as an active learning method even if the workload was not balanced. In these cases, the absence of sense of urgency and relapse prevention-goal setting did not matter. This pathway explains two out of twelve successful cases. Pathway 3 – Presence of support from peers combined with the absence of sense of urgency and the absence of relapse prevention-goal setting, within the contexts of identical elements and training program as active learning method and autonomy and a balanced workload. 11

The third successful combination was having support from peers within which the training was aligned with the actual job experience of the employee in equipment, work environment and psychological sphere; where the training program was carried out as an active learning method and where the trainee enjoyed autonomy. In these cases, it didn’t matter whether the employees felt a sense of urgency to learn and transfer and whether some techniques related to relapse prevention-goal setting were implemented. It is important to note that this was the only pathway where the four contexts were present in combination with a single causal condition. This pathway explained two of the twelve successful cases. Pathway 4 – Presence of supervisor support and relapse prevention-goal setting, and absence of peer support within the contexts of identical elements and training program as active learning method and autonomy and a balanced workload. There were several cases where peer support was not needed as long as the supervisor provided strong support or where trainee had a sense of urgency to learn and transfer or where trainee implemented techniques of relapse prevention-setting goals. This pathway applied to two of the twelve successful cases. In this pathway, peer support does not matter as long the supervisor support was engaged in training transfer, because the trainee had already some form of support. It may be worth noting that sense of urgency was irrelevant since its presence or absence does not make any difference in the impact on transfer in those cases. Pathway 5– Absence of support and relapse prevention-goal setting and presence of sense of urgency, within the contexts of absence of identical elements and presence of balanced workload and presence of training program as active learning method and absence of autonomy. Training transfer was also effective in cases where trainee was motived by a sense of urgency, where employee had a clear need to engage in training due to the identification of a hiatus between his/her knowledge and skills before the training and the required knowledge and skill post-training. Trainee understood that overcoming the hiatus is within reach of his/her capabilities. The contexts here were relevant, for example, the balanced workload enabled the trainee to be focused on learning and training program as active learning methods facilitated the transfer. Other contexts such as identical elements and autonomy did not matter either for this single case. This is the only case where a single causal condition is involved in the outcome achievement. Pathway 6 – Absence of support and presence of sense of urgency and relapse prevention-goal setting within the contexts of identical elements and training program as active learning method, autonomy and unbalanced workload. The sixth successful combination was having the presence of sense of urgency and relapse prevention-goal setting within the contexts of identical elements, training program as active learning method and autonomy. In the single case, support from peers and supervisors did not matter as well as contexts of workload balance, because the efforts to achieve training transfer came from the own employee due to the autonomy, he/she had. The employee feels the need to engage in training due to the identification of a hiatus and feels capable of overcoming this hiatus. This, combined with the coping strategies that keep the training alive, allow the trainee to succeed in transferring the training content to the job.

Pathway 7 – Presence of peer support and absence of supervisor support, combined with the absence of both sense of urgency and relapse prevention-goal setting, within the contexts of absence of identical elements, absence of balanced workload and absence of autonomy and presence of training program as active learning method. Training transfer was effective in cases where employees received support from peers within the context of training program as active learning method. Supervisor support, sense of urgency and relapse prevention-goal setting did not matter, as well as the context of identical elements, autonomy, and balanced workload. This pathway tells us something about the importance of peers in keeping alive the training application within the work environment in this single case. Pathway 8 – Presence of peer support and absence of supervisor support and presence of sense of urgency and absence of relapse prevention-goal setting, within the contexts of identical elements, autonomy and balanced workload and absence of training program as active learning method. The final successful combination was having support from peers and having a sense of urgency to transfer, within the contexts of identical elements and autonomy and balanced workload. It didn’t matter whether the supervisor implemented measures to support the trainees and whether the techniques of relapse prevention-goal setting from trainee were present. Training program as active learning method did not make a difference neither. The main conclusions of our comparative study with QCA highlights the following: •

Training program as active learning method is a high impact context for training transfer effectiveness. ‘Active learning’ refers to a learning method with active trainee engagement through meaningful practice and reflection on what has been learned and encountered (Dochy and Segers, 2018; Prince, 2004; Dewey, 1938). Dochy and Segers (2018) already noticed the importance of active learning in high impact learning, proposing as building block ‘action’ and ‘sharing’. Individuals engaged in active learning methods are capable to have their learning process in their own hands, regulating their own learning experiences. The findings of this QCA study strongly confirm the relevance of the role of training program as active learning method in eleven out of twelve successful cases of transfer. It is not a necessary context, though. However, it is a core context for most of the cases. We can conclude, therefore that many may equate training transfer effectiveness with whether a training program is designed as one of active learning or not. Therefore, this contextual condition would need to be absolutely included in future training design in Flemish firms.

•

Sense of urgency is irrelevant for successful training transfer when training is “mandatory”. The role of sense of urgency (understood in our study to be actors engaged in training because of the identification of a hiatus between current knowledge/skill and the required knowledge/skill in the future, with the understanding that overcoming the hiatus is within reach of the capabilities of the employee (Dochy and Segers, 2018)) in transferring the training came across as limited in our analysis. This was unsurprising as the training program was mandatory for most of the trainees. For example, only three out of twelve cases had a sense of urgency to participate in the training, probably because the motivation as a state of maximum involvement and intrinsic motivation was 13

the result of a balance between task demands and competences (Dochy and Segers, 2018). This is something to think about for further training programs. A balance between tasks demands and competences would need to be evaluated by the organization in order to offer more adjusted training programs to the employees. •

Support plays a moderate role in transfer but cannot easily drive the process to transfer. Surprisingly, our study shows that Flemish firms are not necessarily characterized for having strongly developed ‘support to training transfer’ in their organization training policy. Our analysis treated support as ‘peer support’ or ‘supervisor support’, both independently. As peer support we understood the colleague’s commitment for employees to improve the trainee’s learned content and stimulate the trainee’s use of learned material to the job (Reinhold et.al, 2018; Chauhan et al, 2016; Russ Eft, 2002; Noe, 1986). We understood supervisor support as the superior’s commitment to facilitate the retention and motivate the use of the acquired content in a training on the job by employees, during and after a training program takes place (Govaerts, 2017; Lancaster et al 2013; Nijman, 2006; Cromwell, 2004; Richman-Hirsch, 2001; Salas & Cannon-Bowers, 2001; Holton 1997; Quinones, Ford, Sego, & Smith, 1995). Only two cases out of twelve experienced support by both peers and their supervisor. Six out of twelve cases experienced peer support only and four out of twelve solely experienced supervisor support. In contrast with our expectations, support did not play a key role in the successful cases of transfer. This is contrary to what was raised in theory of training transfer.

•

Relapse prevention and goal setting can be moderately influential, but only under certain conditions and if training program as active learning method and autonomy are present. Relapse prevention and setting goals are important factors to lead to transfer. Our study suggests that such factors acting together can be moderately influential, but within certain limits. First, the context of training program as active learning method needs to be present and combined with autonomy. Moreover, support from peers or supervisor do not need to be present. The analysis indicates that coping with slips and setting goals in parallel can make the difference when some kind of support is absent. In addition, this may occur when the training design is one that enables active learning and the work environment provides opportunities to make decisions (Botke et al, 2018). It is important to note that five out of twelve cases experienced this situation. For five trainees, the work environment and the training design facilitates coping with barriers to transfer even when support was weak or absent.

Regarding to the processes taking place for transfer effectiveness, we identified and unpacked four processes: (1) The process of signaling and retention triggered by supervisor support; (2) The process of enhanced learning intervention triggered by peer support; (3) The process of self-management intervention triggered by relapse prevention and goal setting and (4) the process of learner agency triggered by sense of urgency (Details on these mechanisms can be found in section 5, and Annexes 6, 11 and 12). Our main conclusions in relation to the process taking place in successful training transfer are the following: •

Determination and self-efficacy are key aspects for a well-functioning of the process of selfmanagement. The trainees can organize themselves better when they feel determined to reach their goal. This happens within an environment where trainees can set learning goals and organize 14

their effort accordingly. When feeling determined, employees can feel motivated to maintain the learning and retention goals by applying diverse coping strategies, developing a network, and monitoring the evolution. It is important to highlight that trainees need to recognize the importance of transfer to retain the training content and skills and achieve the goals. The relevance of the training itself acts as a self-reward gained by the trainee. Four general contexts made the process succeed: identical elements, training program as active learning process, autonomy and a balanced workload. When the training program is one of active learning, individuals can grow accustomed to the activities and training skills that they will need to use. However, training situations need to be similar to real life situations to enable transfer (identical elements). Otherwise, there would be a disequilibrium between what is offered and what is feasible to achieve by trainees. A balanced workload enables individuals to be focused on learning experiences and enjoying autonomy allows them to better organize their time. Finally, self-determination and self-efficacy are crucial individual factors that act as facilitators of the whole self-management process. •

Doing the training together within a flat organizational structure and having intervision moments were key aspects of enhanced learning intervention. When everybody is ‘on the same page’, things go better. Individuals following the training together are more likely to go through the process of enhanced training transfer than those who do not follow the training with colleagues. We have seen that this happens within a flat organizational structure that enables individuals to feel more comfortable with peers and to engage with less effort in activities such as communication, sharing different views, reaching common understanding and engaging in coaching activities to discuss and share experiences about the application of the training content. When engaging in communication with peers, employees improve their knowledge on the topic and increase mutual trust. Trust is a context specifically related to this process that enables individuals to share their views with others and valorize the learning moments spent together. Four general contexts made the process succeed: identical elements, training program as active learning process, autonomy and a balanced workload. When a training program mirrors the real life challenges of employees and offers useable tools to enable transfer (identical elements), trainees are more capable to transfer the learned content to the job. In this case, the training program enabled individuals to participate together in the activity, increasing group cohesion. Similarly, when the training program is oriented towards active learning, employees can support learning activities by discussing their experiences related to the training with peers. These coaching moments required time and the balanced workload enabled employees to enjoy subsequent intervision activities. Finally, autonomy facilitated transfer because employees could be focused on learning experiences that were relevant to their own needs.

•

“Keeping the training alive” : How the ‘right message’ can make a difference. The employees that receive support from their supervisor can better transfer when the supervisor ascribes importance to the training, reminds employees to use it and provides feedback. Keeping the training “alive" is key in the retention and application of the training content, mainly when the workload is also balanced. When perceiving the training as relevant for their job, employees can feel motivated to use it and discuss its content with peers, creating also an environment of trust. The latter leads to post-training evaluation feedback for a better transfer. Four general contexts made the process succeed: identical elements, training program as active learning process, autonomy and a balanced 15

workload. The employee needs to gain some hands-on experience during the training, it should therefore be designed as one of active learning. Similarly, the training design would need to be equivalent to the one in which the trainee needs to use the skills that need to be transferred (identical elements). We have seen that the supervisor ascribes importance to the training and communicates this message to the trainees. As already mentioned, the workload was balanced and the autonomy to select the most appropriate tasks related to the training content was also relevant for transfer purposes. To stimulate this, the supervisor should manage the workload so the trainee can also focus on learning. Finally, the ways in which the supervisor increased the perceived relevance of the training for the job were crucial microprocesses that facilitated the mechanisms of signaling and retention. •

The process of learner agency cannot work when training is mandatory. Learning activities are not always a consequence of learner agency. It is sensitive to the context of whether the training was mandatory or voluntary. Learner agency works better when the decision to learn is voluntary (Dochy and Segers, 2018), because in that situation, there is an intrinsic motivation to learn. We have seen that in most of the cases the training was mandatory. In those cases where training was mandatory but still successful, trainees mentioned its importance and their interest in pursuing it. However, learner agency cannot work properly, because what initially triggered participation in the training, was not a “real” gap or challenge identified as relevant for the job tasks. Although the training content improved his/her capabilities and job performance, the trainee mentioned several times that he/she could do his/her job without the training. We see that intrinsic motivation produced by a sense of urgency is not what triggered the process of learner agency in the particular case studied in this research. For this reason, the learner agency process broke down. Three contexts were present in this case: identical elements, autonomy and training program as active learning method. The workload was unbalanced, though. Even when three contexts were present, they did not enable the well-functioning of the process, because of the already mentioned lack of contextual conditions related to the microprocess: the free choice for learning as the original facilitator of learner agency.

3. RECOMMENDATIONS The OiB-call aimed to contribute to an increase in the number of Flemish workers in training. The ultimate goal of such an increase is to support and encourage lifelong learning in companies and organizations. However, in the call, there was little attention for how these trainings would actually be implemented and how their effectiveness would be maximized. This evaluation study focused on whether or not following the training would lead to a successful training outcome, defined here as a durable transfer of the training content to the job context. We investigated what and how conditions can trigger successful training transfer. As discussed in section 3, only about 30% of our cases actually experienced effective training transfer, leaving ample room for improvement. By incorporating guidelines on these conditions and the necessary contexts, future calls could improve effective training transfer, ultimately leading to increased benefits of these trainings. ESF project managers could ask organizations that want to participate in the call to reflect on how they will support their trainees to transfer the training in their application proposal. They could also ask more specific questions such as

“how will you, as an organization, ensure that trainees receive a sufficient amount of peer support during and after the training so that they can transfer the training to the job context?”. Other questions could be: • • • •

During which occasions will employees that followed the training be able to discuss the training and its application? Why is the suggested training relevant for the trainee(s) and how will you convince trainees of this relevance? To what degree will supervisors be involved in transferring the training content to the job context and how will they be convinced of its relevance? What steps will you take to ensure that the use of the training skills is maintained over time?

Furthermore, in the call, they could stress the importance of these conditions and provide some guidelines to organizations (see below). Organizations and trainers that consider these conditions are more likely to cause training transfer. Besides the project proposal stage, it may also be relevant to develop a monitoring system which requires firms to report on these aspects during (if possible, depending on the duration of the training) and after the training. Monitoring these conditions could encourage organizations to attach increased importance to those conditions that could allow trainees to successfully transfer their training. This advice is of course not limited to calls that focus on increasing participation rates in trainings, but can be incorporated in other calls as well (e.g. calls that assist organizations in transforming into learning organizations). That is not to say that calls can or should be the only way through which favorable contexts that maximize the likelihood of training transfer can be achieved. Organizations, as well, have many tools available to create conditions that are favorable to training transfer and will directly benefit from utilizing these tools. Anticipating the training and ensuring that supportive conditions are present before the training starts will stimulate overall training transfer. Below, we provide specific recommendations for stakeholders and for further research. •

Understand that ‘mandatory trainings’ are not effective. Organizations need to focus on making the relevance of training visible to employees. Our study recommends that it is important to start with communicating the added value of a given training. Employees need to understand why the training is relevant and how it can be used in job-related tasks. Similarly, organizations need to facilitate the relevance of the training by balancing the workload (ensuring that it is not too high, nor too low) and providing some sort of autonomy. Employees should be able to work on their own learning process when having the time and understanding the relevance of training content for their further performance. This could help employees to feel motivated to engage in a certain training program and avoid falling back into old habits when transferring training. In future training initiatives, organizations could be focused on how to motivate employees to engage in learning activities, by illustrating its relevance in their careers. Employees have their own views on how to respond to changing demands in the job contexts. Training program design should consider these views.

•

Pay attention to the implementation of meetings between peers during and after training. Plenty of employees who experienced ‘peer support’ noted that they would regularly meet with peers and discuss the (application of the) training. Although it is important to have informal contact as well, stimulating the organization of formal moments to discuss it may be helpful. Some participants noted that they would rarely get to interact with those colleagues with whom they 17

followed the training, which they regretted. Our findings suggest that quick informal ad hoc meetings with peers are not enough to transfer: employees need to share their experiences, frustrations, feelings, get feedback, attention and be encouraged by others on regular occasions. This implies that organizations need to pay attention to formal meetings between peers during and after training. Organizations could support their employees in organizing these intervision moments. In the successful case of peer support triggering an enhanced transfer intervention mechanism, the feedback or coaching provided by colleagues kept the motivation alive. Interaction of this nature has proved to be a powerful building block for successful training transfer and can be a fruitful area for organizations to work further on with their employees to strengthen the transfer process. Likewise, although this may be obvious, meetings between trainees and supervisors could also contribute to training transfer •

Transferability of training depends on what the organization offers to make it possible. As seen, transfer cannot be taken for granted by organizations. Actors, such as supervisors and management, need to implement facilitators and those facilitators need to be communicated to the employees. Employees need to know which resources are available for a better learning environment and transfer. Many employees complain about the workload. They note that sometimes they cannot focus on the learning tasks. This is clearly a barrier to learning and transfer. Actors in organizations not only need to make the training relevant for employees, but also provide a balanced pace for job tasks and training tasks. Some cases studied mentioned that they were fully engaged in training with peers, together, and this worked greatly for transfer. Everybody was on the same page and they gain trust and new knowledge when interacting with colleagues. Similarly, a degree of autonomy in the work management during training program could also be helpful for transfer. We have seen that active learning requires autonomy, where employees are capable to organize their own pace, rhythm and learning experiences according to their preferences and interest. However, it may be challenging for enterprises to modify aspects of a work environment that facilitate training in the short term when they have already acquired government resources to invest in training within their organizations. Reorganizing the job is challenging, and costly. We suggest thinking in terms of solutions that can be feasible in the short-term and in the long-term. For example, when possibilities of subsidized training present themselves, organizations could work with their employees by proposing them to be engaged in the design of training dynamics. They could also assist employees in searching for the best way to deal with potential barriers and support them by creating a climate that facilitates training transfer and secure the quality of these trainings.

4. FURTHER STUDIES AND DEVELOPMENT This is an innovative evaluation study carried out with a complex research design. The methodological approach of combining QCA and process tracing is new and posed us with several challenges that took a lot of effort to overcome. In many ways, this study was refined with the support of experts and other studies can build on and apply this research design in the future. For example, by adding more cases (especially unsuccessful ones) or studying the causal mechanisms that have already been disentangled in other positive cases. We hope that this evaluation research will be used as a practical tool by organizations to identify opportunities to improve their training policy. 18

The most important recommendations for further research are: The combination of QCA and Process-Tracing to study causes and process is not a panacea. There were many decisions that were carefully taken to make this research possible. One of them, was the conceptual design. Pattyn et al. (2020) provide some lessons for concept development when applying these innovative methods: •

•

The most important are related to avoid ‘heterogeneity’ at the level of processes, i.e., keep a level of homogeneity to make the process travel to other cases. This implies that we should be able to explain why training transfer occurred in some cases and not in others and how it worked in the successful cases. With a conceptualization that does not avoid heterogeneity in the process, we cannot generalize or understand the process in more than a single case. In this research, even if we opted for studying single cases per mechanism, there is still the potential to extrapolate mechanisms to the rest of positive cases members of the respective conditions and outcome.

Take the context seriously. We have seen that at the level of the causes of training transfer, some contexts played a key role at the start of training transfer, while other contexts may have played a role at a later stage in the process: •

•

When we zoom in on our results, we can observe that the single case T1 had its particularities: It is a failed case in the mechanism of learner agency. In the theory of learner agency, sense of urgency triggers a motivational force to engage in learning agency. In this case, even if sense of urgency was present at the start, the most important context of learner agency was absent: the free choice for learning engagement. Learning processes are different when the training is mandatory. Expectations related to what to learn, how and when are also different. The trainee’s characteristics are also diverse, as well as the work climate. Therefore, sense of urgency does not guarantee the presence of the learner agency process. We think that in this particular case, T1 searched for alternative processes to transfer, and such processes were more related with individual characteristics. We have seen that relapse prevention and goal setting are present in this case. Therefore, we believe that learner agency elements combined with those from selfmanagement enabled the success in transfer or just a self-management intervention process which was not studied in this case. Future studies could focus on how, and under what circumstances, different mechanisms are connected and work together to produce the outcome.

Some concepts studied in this research are promising and require more attention than they get today. Training transfer and learning are interrelated. Therefore, we expected that sense of urgency (which produces an intrinsic motivation to learn) would lead to training transfer. Because, we were unable to confirm this in our case, sense of urgency as a causal condition needs to be further developed. •

•

We expected sense of urgency to lead to training transfer because it is linked to intrinsic motivation. However, to make it work, there are other characteristics that need to be at play, such as the organization climate and the way in which jobs are designed. It is not sufficient to feel motivated in order to engage in learning. We assumed that this ‘motivational force’ could be stimulated by the work environment so that an employee would be more likely to engage in training. As mentioned earlier, if organizations can

influence their employees’ stimulus to learn, they will gain more in the transfer process and training programs will be more useful for the employees and the organizations. Further case studies. This research was oriented to explain and understand the training transfer effectiveness. • •

•

We think of more in-depth case studies of other typical cases in order to get the whole picture of the process taking place in more than a single case. Similarly, we think of including in future research the study of processes that have broken down in more than a single case, in order to understand what employees and organization need in order to succeed in transfer. In this research, we focused on soft skills training (i.c. leadership training and stress management). Future research could focus on other types of training.

Future evaluations efforts. We suggest including in future research the study of transfer failure, by revising other theories and empirical studies in Flemish firms. This could help to identify what is not working correctly in the rest of the 35 cases of training transfer studied in this research. Methods used. A survey was designed to collect data about training transfer effectiveness. Based on this, we observed that the number of workers that successfully use the training is quite low (15/50). Some improvements to data collection at the level of cross-case could be: •

•

Because we had T0 (before training) and T1 (after training) surveys, we had a lot of attrition (lower survey response rate). When working with large N, it is a challenge to perform interviews or focus groups, for this reason we estimated that a survey could be a better instrument to explore our cases. For further research, we think that adding complementary tools (interviews, focus groups), with more researchers involved in data collection would be an added value to this study. The outcome was measured using several strategies (see section 3.3). One of them was based on attributes. The second one, consisted of the use of scales. For example, for leadership skills, we had two scales that consisted of 7 items (efficacy) and 15 items (empowering leadership) respectively. Because when a lot of items are used, there is often a 3 or lower somewhere. In the case of empowering leadership, this lead to very few cases to be considered to have “good” leadership skills. These scales were better suited for quantitative logic, but less so for QCA and Process tracing, where one need to have a sharp definition of IN/OUT and scores on all dimensions are relevant.

1. INTRODUCTION 1.1 BACKGROUND: This study investigates the factors driving training transfer effectiveness in Flemish firms. Training transfer is a multilevel and context-sensitive process. Theories of training transfer focus on various levels such as the organizational level, characteristics of the individual trainee level and the training design level. The most common understanding of training transfer effectiveness concerns ‘the application of what is learned from the training to the workplace’ (Broad & Newstrom, 1992; Cromwell, 2004; Dochy & Segers, 2018: 163; Newstrom, 1986; DeSimone, Werner, & Harris, 2002: 3; Gumuseli & Ergin, 2002; 81; Hawley & J. Barnard, 2005: 66; Newstrom, 1986). This use of new knowledge on the job is also referred to in the literature as ‘generalization’, meaning that the trainees are capable to ‘activate the resources’, (Hammer et al, 2005) acquired in one context (e.g. training), in another context (e.g. the job) (Baldwin & Ford, 1988; Bransford, Brown, & Cocking, 1999; Campione, Shapiro, & Brown, 1995: 39; Chiaburu, et.al., 2010; Fogarty et al., 1992: x; Gagne et al., 1993: 235; Hawley & J. Barnard, 2005: 66; Kirwan, 2009; Lave, 1988: 122; Ripple and Drinkwater, 1982: 1947), or as ‘productive use of acquired knowledge and skill’ (De Corte, 2003; Gegenfurtner, 2011: 154). Similarly, some scholars introduce the notion of ‘maintenance of the learned material over a period of time on-the-job” (Kirwan, 2009:5; Blume et. al, 2010) when they refer to an effective transfer, arguing that the continued application can lead to a certain standard over time (Broad & Newstrom, 1992:6). Complementing actions regarding positive attitudes to learning, working environments with learning potential, and the climate concerning learning in the organization are factors that influence training transfer (See 2011 HIVA, De Rick & Van Itterbeeck, 2011). As such, training transfer effectiveness in Flemish firms depends on characteristics at different levels. For example, it could be that there is an improvement of the training policy at the level of the organization, and at the trainee level, there is an improvement of competences. However, the effects of these improvements are sometimes limited, because some further work at the level of the framework for training (training policy, before and after care), as well as some efforts to tailor trainings to the needs of specific groups, needs to be carried out. As such, a single training does not have such a strong influence that it can increase adaptability or employability. Previous evaluations of training transfer effectiveness provide useful insights, but several of these merit further investigation. For instance, while a wide range of employee characteristics, attributes of the training projects and characteristics of the organisation have been studied, the impact of each of these characteristics is assessed separately and does not reveal how they interact (for more information see 2011 HIVA (De Rick & Van Itterbeeck, 2011)). In addition, some other evaluations results have concluded positive effects of training on growth in employment, added value, labour productivity, when applying counterfactual design, but they were larger when companies were smaller and younger and when they had been using subsidies for a longer time (see Baert, Decramer and Reynaerts, 2014). These studies were useful in identifying positive effects of training subsidies, but they are not applying an adequate methodology to “unpack” causal processes triggered by different combinations of conditions within diverse contexts. This study builds on beyond what has been previously studied both theoretically and empirically, and it takes a new comparative methodology and case study approach. 21

By studying 50 cases, broadening the range of potential factors and distinguishing between remote (contexts) and proximate factors (causes), we apply a multimethod approach that combines Qualitative Comparative Analysis (QCA) and Process-Tracing (PT). By using QCA, we aim to identify the conditions and pathways 2 associated with effective or not (yet) effective training transfer. In consultation with the commissioner of this evaluation, we decided to focus particularly on leadership trainings and stressmanagement trainings. After identifying such pathways, we aim to study the process linking those paths with transfer effectiveness in the most relevant cases. As relevant cases, we understand those cases where conditions, the contexts and transfer effectiveness are present. As far as we are aware, this is a first time QCA and PT methodologies are combined and used to identify pathways and study causal process for effective training transfer. This study was commissioned by the Flemish authorities. In this study, we analyze the effectiveness of inhouse training program (funded by the European Social Fund – ESF) in Flanders-based firms. More information on the evaluandum can be found in Annex 1.

1.2 PURPOSE, SCOPE AND LIMITATIONS The purpose of this study is to enhance the understanding of how ESF-subsidized training programs produce impact, namely what drives training transfer effectiveness in Flemish firms in Belgium. Specifically, it addresses the research question of what combination of conditions and mechanisms are associated with training transfer effectiveness in Flemish firms. We look at 15 successfully transferred training programs in social skills: leadership and stress management and 35 cases where training transfer effectiveness has not (yet) been achieved. Finally, in order to better understand the process of training transfer, we take a more in-depth look at four cases where different mechanisms played a key role in transfer effectiveness. Having a clear view on what causes training transfer and how it causes training transfer will assist in designing more effective learning and development policies. In this study we do not assess the quality of the training itself, but the impact of thereof, focusing on whether the training program was implemented as intended, in line with its stated aims for creating value to employees and improving the quality of labor, and how the training program was implemented, focusing on what makes a difference, what works and does not works in some cases and not in other instances and why this is the case. We also need to bear in mind that while the population of cases studied represent a range of employees following some training programs, they may not be representative of all training programs in Flemish firms.

1.3 HOW TO READ THIS REPORT This report is divided in 7 sections. Following this introduction, in section 2, we discuss why we use QCA and Process-Tracing and how they are applied to this study. In Section 3, we present the key conditions and their conceptualization and operationalization. We also introduce the 50 cases and the information we have collected on these cases and the relevant conditions. Those more interested in the QCA results can 2

We use the term “pathway” to indicate that different (combinations of) causes can ultimately, possibly through different causal processes, lead to the same outcome. It is thus to be interpreted as a (combination of) condition(s) that leads to success. 22

jump to Section 4, where a QCA analysis is presented with its different pathways and where the key findings of the patterns of training transfer effectiveness in the 50 Flemish cases are discussed. We also discuss the specific cases in each pathway. Section 5 discusses the Process Tracing in which we take a closer look at how different conditions can lead to training transfer. We present four possible mechanisms and illustrate how these manifested themselves in four actual cases. Section 6 summarizes the main implications of these findings for organisations that can influence training programs in Flemish firms. Finally, section 7 presents practical recommendations for stakeholders to support successful training transfer and draw some suggestions for further research. More information on the QCA, Process Tracing and details of this research can be found in various appendices.

2. METHODS AND STUDY DESIGN This section briefly introduces QCA and PT. We briefly discuss each method, why we chose these methods for this study and what the application of these methods looks like in our study.

2.1 WHAT IS QCA? QCA is a systematic cross-case method and approach. Its aim is to explain a given outcome by assuming that a combination of conditions is at play rather than isolated factors. It allows us, and practitioners, to identify which conditions contribute to training transfer and thus what factors to consider when designing policy (e.g. in ESF-calls). The selection of factors to study a given phenomenon is in line with the way in which we assume that such conditions affect the outcome. It is a case-based method, which means that we need to explore the field and to find information related to the cases of interest in an exploratory way, in order to select relevant conditions from others, refine causal relationships and determine where ones have the strongest empirical evidence. QCA enables to explain outcomes from different pathways (equifinality) and is underpinned by configurational or multiple-conjunctural causality. QCA is based on set theory and enables the analysis of necessity and sufficiency, which are explained in greater detail in section 5. The analysis allows us to recognize which conditions are required for an outcome to occur and which ones are sufficient to produce it without being required for its occurrence. Finally, QCA operates with dichotomization of conditions when working with the crisp-set version or calibration when working with the fuzzy variant. In this research we are working with the crisp-set variant of QCA, where numerical values go from 1 to 0, being “1” the presence of the condition/outcome, and “0” the absence (for more information, see Rihoux and Ragin, 2009; Schneider and Wagemann, 2012; Medina et al, 2017). The reason for choosing this variant is its compatibility with the process-tracing approach, i.e. we have more possibilities for homogeneity in the processes leading to training transfer than other variants, which requires the application of other techniques to avoid the heterogeneity. The crisp-set variant also has challenges when combining with process-tracing, but such challenges are more manageable when applying conceptual alignment in the research design (see more about this discussion in Beach and Pedersen, 2016, 2019).

2.2 WHY WE CHOSE QCA Training transfer effectiveness is multilevel and context sensitive. The effective transfer of training programs is not the result of an isolated factor. There are many conditions at play and not only acting as causal but also as contexts (enablers). Certain contexts may be required for an effective training transfer, but they may not be triggering a process to produce the outcome, they just may just facilitate the work functioning of a process to lead to the intended outcome. Causal conditions are required for triggering that process. Causal conditions can be necessary or sufficient or a combination of both. Successful cases of training transfer may require many paths, because some causal conditions may be required for effective training transfer, but they may not be sufficient to guarantee the occurrence of outcome: it depends on the context, the support from colleagues, the role of supervisor and of the trainee´s own interests, we can have ‘different ways of getting to Rome’. 24

QCA as a technique and approach was chosen for this study as it allowed to make sense of the complexity of training transfer. While QCA does not open the black-box of training transfer, it allows cases to be grouped into contexts that provide the basis to study the process in a second part of this study with processtracing and to reconstruct the process starting from the triggers identified in the QCA analysis until the outcome is produced. Annex 2: Limitations of QCA includes a short discussion on the caveats and limitations of this method.

2.3 THE QCA PROCESS IN THIS STUDY There are three main phases in a QCA procedure: (1) Case selection and case description. An overview of cases must contain some ‘thick’, historical and cultural information on the case; (2) selection of conditions (causes and outcome) based on previous empirical and theoretical studies. The information must be summarized in numerical scores based on data collection. An analysis must be performed to get the minimal formulae; (3) Different solutions are obtained and one must choose the most relevant according to the research purposes, goals and research questions (Schneider 2018). The different causal paths are interpreted by returning to the cases and to their narratives (Rihoux and Lobe, 2009:229). It is important to mention that the QCA procedure is not linear, but iterative and one can back to the conditions and refine them as much as convenient (see Figure 2). For doing so, discussions with experts in training transfer (Filip Dochy), experts in methods (Derek Beach) and workshops with the ESF evaluation committee have been carried out to refine the conditions and its conceptualization. Figure 2: QCA procedure

Source: Rihoux and Lobe, 2009 “QCA and the and the funnel of complexity”.

2.4 WHAT IS PROCESS TRACING? According to Beach (2016, 2019) Process-tracing is an innovative approach and method to study processes. Processes are understood as causal mechanisms that play a key role in producing a given outcome. It is context sensitive, and therefore requires making a distinction between factors as causes and factors as contexts needs to be done. It is a within-case method which particularly enables to understand how a causal process operated in real-world cases. For doing so, a case-based knowledge is required, and evidence accordingly. However, process-tracing can be also studied in a comparative way within a causally homogeneous population (and when controlling by contextual conditions). The analytical added value of process-tracing is the type of conclusions what we can draw about how a causal process works, why and under which contexts, and what was missing when the process broke down. Process-tracing consists of three parts: (1) Build up a causal mechanism based on theory linking the identified causes and the intended outcome, (2) Operationalize the theoretical process as to how it looks like in reality; and (3) case selection of those cases that can be relevant for the purposes of the research (understanding a particular case, make generalizations of findings from single-case studies to other causally similar cases).

2.5 WHY WE CHOSE PROCESS TRACING We need to know what is happening in the causal black box. As explained earlier, QCA allows us to explain why training transfer is effective in some cases but not in others, by focusing on the combinations of conditions that are displayed as different pathways. However, only applying QCA does not shed light on the way in which training transfer effectiveness takes place. To do this, we need to unpack the process or reconstruct it based on theory and process-evidence. When working with causal mechanisms with process-tracing, we can better understand the theoretical causal links related with training transfer effectiveness; gain knowledge about how the process operates in real-world cases: identify the contexts that enable a well-functioning of the process and make strong conclusions about the presence of such process in a given case.

2.6 THE APPLICATION OF PROCESS TRACING IN THIS STUDY The process of applying PT after a QCA to this project entailed six key steps, which are captured in Figure 3 and are explained below in more detail.

Figure 3: The PT process after QCA analysis

Step 1: Case selection based on QCA solution

Step 2: Theorization of causal mechanism linking causes and outcome

Step 3: Operationalization of the mechanisms as "fingerprints" and empirical test

Data collection from multiple sources

Evaluation of the data gathered (observations to evidence): content, source, alternative explanations

Conclusion: confirming/disconfirming/nonevaluated presence/absence of the mechanism

Source: authors Step 1: Selecting cases based on QCA analysis. Since we are interested in explaining and understanding training transfer effectiveness, we focus our attention in those cases that are identified as relevant (or typical) for this research. Typical cases are members of the causes and outcome. Similarly, if we are interested in improving our theory we may still be interested in deviant cases, where some causes are present, but the outcome is absent. Step2: Theorization of the causal mechanism linking the causal condition and the outcome. Based on theory and empirical data we disentangle the causal process as a system, consisting of actors engaged in activities in a productive manner. The theorization of the mechanisms depends on the type of process tracing used. The selection of the type of PT is important when conceptualizing the mechanisms, but the three types 3 share common steps. In our research we have chosen a theory-testing process tracing because after a QCA analysis where one can identify the key factors (or combinations of) that lead to the outcome, we are able to trace the process based on the given literature and fieldwork. Step 3: Operationalization of the causal mechanisms based on fingerprints, i.e. how does it look like in reality for the real-life cases. These fingerprints can be traces left by the actors engaged in activities, such as minutes, documents, answers in interviews that can be useful as evidence when testing the presence of the mechanism (and its parts) in particular cases. This also includes the development of empirical test (priors, theoretical and empirical certainty and uniqueness). 4

Theory-testing PT, theory-building PT or explaining-outcome PT. See Beach & Pedersen (2019) for a discussion on these different types of PT. 4 Additional information about how PT works can be found in Annex 4: Conceptual framework: combining qca with process-tracing methods. and Annex 5: Process-Tracing methods. 27

Step 4: Collecting data on the conceptualized mechanism for the group of cases identified as relevant. After having conceptualized the mechanism, we collect “observations” on each part of the mechanism5. These observations can be found in interviews, archival data, visual material, focus groups, etc. We have performed face-to-face semi-structured interviews with 8 participants who have successfully transferred their training, phone interviews with all HR managers and one participant that was selected for process tracing, and also we have received evidence such as emails, minutes or reports from two of the cases that were selected for process tracing. Step 5: Evaluation of the collected observations (not yet evidence) in order to decide what classifies as “mechanistic evidence”. The most important aspects of this step imply to assess: (1) what the observation tells us; (2) Trust in the source; (3) Alternative explanations. When the information is gathered from diverse sources, we may evaluate what the content tells us regarding to the expected information and compare. We may evaluate whether we can trust on the source or not and whether alternative explanations exist. Step 6: Conclusion of the presence/absence of the conceptualized mechanism in the studied case, after the step 5.

5 These individual parts of a mechanism are sometimes also referred to as building blocks of a mechanism. 28

3. DATA AND CONCEPT OPERATIONALIZATION This study draws on 50 cases of training transfer effectiveness in 9 Flemish firms from 2018 to 2020 (see Figure 4). Of these case studies, 15 are considered ‘successfully implemented’ (hereafter referred to as ‘successful case studies’) and 35 have not led to impact in practice (hereafter referred to as ‘failed case studies’). More about the population of cases in this research can be found in the intermediate report.

3.1 CASE-STUDY DATA COLLECTION We collected data on the training participants on several occasions in 2018 and 2019. Two weeks before the training had started, we contacted employees and asked them to fill in a survey (T0). Out of the 203 people we contacted, 106 filled in the survey. Approximately 3 months after the training, we asked the same people to fill in another survey (T1). 65 people filled in the survey. We also asked HR representatives to fill in an additional survey on the training and the training policy at the company. In order to gather more in-depth information about certain cases, we held semi-structured interviews with 8 participants who have successfully transferred their training to the work environment (2019, 2020). During these interviews, we focused on how the training conditions contributed to the outcome. Finally, we contacted some participants and requested some specific documents that provide support to parts of our theorized mechanism on how they have implemented the training. One participant, who did not have time to send us all documents that we requested, was also interviewed again through telephone. Additional information can be found in Annex 7. Figure 4: Data collection process 2018 Before training

2019 After training

•T0 survey •203 contaced •106 answers

Source: authors.

•T1 survey •203 contacted •65 answers + HR representatives

2020 •Semi-structured interviews •8 interviews

Comparative study with QCA

Process-Tracing study

“Why question”

“How question”

3.2 CASE-STUDY SELECTION CRITERIA Because we allowed all employees to complete the T1 survey, and not just those who had also filled in T0, we had multiple respondents who only completed T1. Furthermore, some respondents had indicated that they followed a training to deal with stress, while our record showed that the ESF-funded training was about leadership or vice versa. These observations, and those who did not fill in the survey completely,

were not considered in the dataset. After cleaning and merging both datasets (T0 and T1), we retained 37 employees that filled in both surveys correctly. Case selection strategy for QCA phase Given that all necessary characteristics for the QCA were measured in T1 (and the survey we administered with the HR officer), we decided to solely use the data from T1 for the QCA analysis. This gave us a larger sample of 51 respondents. In applying the possibility principle for selecting negative cases6 (Goertz, 2005), one case was identified as irrelevant because it did not have a positive score on any of the potential conditions that could trigger learning transfer. Out of the 50 remaining cases, 15 had successfully transferred the training, while 35 did not (See Table 1: Case-study outcomeTable 1 for the definition of the outcome). Case selection strategy for Process-tracing Once we have performed our comparative study with QCA, we will select those cases that show membership in the condition, contexts, and outcome. We will situate all relevant cases, which should be scored on conditions that might be relevant for whether and how a mechanism worked. Based on this, we will determine which cases are typical (same contexts/causal conditions/outcome) and which are deviant (cases which show the presence of the context, the conditions but without the outcome). The selection of the typical case will enable us to know how the process worked and to generalize to the other positive cases and confirm theory. In contrast, the selection of the deviant case, will enable us to trace the mechanism till it breaks down, in order to know what did not work and to refine or expand our theory (by exploring omitted conditions /contexts). More information related to the case selection strategy can be found in Annex 8: Case selection strategy for qca and pt phase. Table 1 shows the conceptualization of the outcome. We did define specific attributes and observables manifestations to measure the training transfer effectiveness. Questions about main attributes were based on Govaerts (2017), Hiva (2011) and Lim & Morris (2006). In addition, to gain insight in whether or not the employee established effective training transfer we opted for measuring the skill level after the training (T1). The successful cases were conceived as those that marked as positive in each observable manifestation of generalization and maintenance and also in every substantial insight in leadership skills or stress levels7.

The possibility principle to select negative cases, defines negative cases not only where the outcome is absent, but also where the outcome does not happen yet. We recognize a relevant negative case when this is member of at least one condition. Cases without membership in any condition is not relevant for the research and must be relegated. 7 These are: perceived stress scale, 4 items (Cohen et al., 1983) or Leadership self-efficacy (McCormick et al, 2002), depending on the training that was followed. Originally, empowering leadership (based on Srivastava, Bartol, & Locke, 2006) was also going to be included, but including this would, due to the large number of items that measured this (15), result in an insufficient number of cases. This illustrates that these measuring scales with many items are not necessarily suited for QCA.

Table 1: Case-study outcome OUTCOME Conceptualization

Training Transfer Effectiveness Effective employee training transfer is the application or use of the learned knowledge (content, skills or attitudes) acquired in a training program to the job by trainees, which is maintained over a period of time (Dochy & Segers, 2018; Gegenfurtner, 2011; Chiaburu, et.al., 2010; Blume et. al, 2010; Kirwan, 2009; Hawley & J. Barnard, 2005; Hammer et al, 2005; Hawley & J. Barnard, 2005). Additionally, the trained skills need to be sufficiently high after the training.

Operationalization

Items are based on the criso psychological climate questionnaire. All items were answered on a five-point Likert scale ranging from 1 = "Strongly disagree" to 5 = "Strongly agree".

Generalization

Maintenance

Due to the competences that I learned in the training program, I have changed my job behavior. Right after completing the training, the training was very useful for me. I still use the knowledge and skills from the training nowadays.

Items based on McCormick et al (2002). Likert scale ranging from 1 = "Not confident at all" to 7 = "Very confident". The general question was: “how confident are you when performing the following activities?”

Leadership self-efficacy

Functioning well as a leader in different group settings. Motivating group- or teammembers Building trust with group- or teammembers Developing teamwork Taking charge when it is necessary Communicating effectively Assessing strengths and weaknesses of a group

Items based on Cohen et al. (1983). Likert scale ranging from 1 = "Never" to 5 = "Very often".

Perceived stress

In the last month, how often have you felt that you were unable to control the important things in your life? In the last month, how often have you felt confident about your ability to handle your personal problems? In the last month, how often have you felt that things were going your way? In the last month, how often have you felt difficulties were piling up so high that you could not overcome them?

Dichotomization

Successful (1) – A case is successful if it falls under every observable manifestation (being at least 4 in the Likert-type 5 scale and at least 5 in the Likert type 7 scale). Failed (0) – A case if failed if one attribute is missing. In particular, the case needs to be out of at least one observable manifestation (1,2,3(,4) in any observable manifestation in the Likert-type 5(,7) scale).

Additionally, mainly for the process tracing, we also looked at several open questions to gain more information on whether or not people had transferred their training. These open questions (based on Lim & Morris (2006) and own proposal) were: • • •

Can you give some examples of concrete work situations in which you have applied the training content? If you do not apply the training content, what reasons prevent you from applying the training content? If you apply the training content, what reasons have stimulated the use of the training content?

3.3 CONCEPTUALIZATION AND OPERATIONALIZATION OF KEY CONDITIONS After a vast literature and empirical review within numerous iterations and discussions with experts such as Filip Dochy, Derek Beach, and workshop with ESF stakeholders from the evaluation committee, we conclude that nine conditions can contribute to explaining the effectiveness of training transfer in the set of 50 cases. We differentiate between five causal conditions that could trigger training transfer and four contextual conditions that, simply put, allow these transfers to happen. These conditions are presented in Table 2, along with their descriptions and dichotomizations 8 . The causal conditions are: peer support; supervisor support; sense of urgency; relapse prevention and goal setting; and the contextual conditions are: identical elements, training program as active learning method; autonomy, and balanced workload. A full literature review is found in the Annex 3: Literature review at the end of this report. A conceptual framework is also found in Annex 4: Conceptual framework: combining qca with process-tracing methods. Table 2: Key conditions, definitions, operationalization and dichotomization Condition

Conceptualization

Peer support Peer support is the colleague’s commitment for employees to improve the trainee’s learned content and stimulate the trainee’s use of learned material to the job (Reinhold et.al, 2018: Chauhan et al, 2016; Martin, 2010; Hatala and Fleming, 2007; Gilpin-Jackson & Bushe, 2007; Jellema, Visscher, & Scheerens, 2006; Nijman, 2004; Russ Eft, 2002; Noe, 1986).

Some of these operationalizations differ as to how they were set out to be in the intermediate report. Because they turned out to be too strict, leading to zero (or very few) of our cases to be members of certain conditions. Therefore, we decided to focus on those survey items that were most relevant for the mechanism. The conditions that were operationalized differently are: peer support, supervisor support and sense of urgency.

For the employee T1 survey, the main sources were Cromwell (2004), Holton et al. (1997). Both items were answered on a five-point Likert scale ranging from 1 = "Strongly disagree" to 5 = "Strongly agree". Operationalization

Dichotomization

Condition

Conceptualization

In my work unit, my peers try to minimize breaking-off from work that interfere in the opportunity to practice the newly learned skills. Stimulate generalization ‐ My colleagues try to encourage others to use the skills they learned in trainings. Present (1) – A case is member of the set Peer Support if there are both a commitment to improve the trainee’s learned content and a stimulus to generalize. In particular, the case needs to agree with both statements above (being at least 4 in the Likert-type 5 scale). Commitment to improve the trainee’s learned content

‐

Absent (0) – A case is not member of the set Peer Support if one attribute is missing. In particular, the case needs to have an answer of 3 or lower on the likert scale on at least one of both items. Supervisor support Supervisor support is the superior’s commitment to facilitate the retention and motivate the use of the acquired content in a training to the job by employees, during and after a training program takes place (Govaerts, 2017; Lancaster et al 2013; Nijman, 2006, 2004; Cromwell, 2004; Cromwell & Kolb, 2004; Cromwell & Kolb, 2002; Quinones, Ford, Sego, & Smith, 1995; Richman-Hirsch, 2001; Salas & Cannon-Bowers, 2001; Smith-Jentsch, Salas and Brannick, 2001; Van der Klink et al., 2001; Cohen, Underwood, and Gottlieb, 2000; Elangovan & Karakowsky, 1999; Gregoire, Propp, & Poertner, 1998; Holton 1997; Brinkerhoff & Montesino, 1995; Baldwin & Magjuka, 1991). The employee survey at T1 included questions on supervisor support. During the operationalization we were based on the study of Govaerts (2017), who performed one of the most detailed and informative studies on the role of supervisor support for transfer. We adapt it according to our conceptualization. All items were answered on a five-point Likert scale ranging from 1 = "Strongly disagree" to 5 = "Strongly agree".

Operationalization

Encouragement to retention

‐

Motivation to generalization

‐ ‐

‐

My supervisor encouraged me to share what I’ve learned in training with people in my work environment. My supervisor discusses with me how to apply competences to job situations. My supervisor provided me, when required, advice and coaching immediately following training. My supervisor gave useful feedback after training on my application in my job of what learned.

‐

My supervisor expressed his belief after training that I would successfully apply what I have learned. Present (1) – A case is member of the set Supervisor Support if there are both an encouragement to retention and motivation to generalize the content to the job context. In particular, the case needs to fall under every observable manifestation (being at least 4 in the Likert-type 5 scale). Dichotomization

Condition

Conceptualization

Absent (0) – A case is not member of the set Supervisor Support if one attribute is missing. In particular, the case needs to be out of at least one observable manifestation (1,2,3 in any observable manifestation in the Likert-type 5 scale). Sense of urgency Based on the available literature we understand an employee’s sense of urgency as one’s (1) clear need to engage in training (2) because of the identification of a hiatus between current knowledge and skill and required knowledge and skill in the future (3), with the understanding that overcoming the hiatus is within reach of the capabilities of the employee (Dochy and Segers, 2018; Salas, Tannenbaum, Kraiger and Smith-Jentsch, 2012; Burke and Hutchins, 2007; Ryan and Deci, 2000; Csikszentmihalyi and Beattie, 1979). All items were answered on a five-point Likert scale ranging from 1 = "Strongly disagree" to 5 = "Strongly agree".

Operationalization

Need to engage in training because of the identification of a hiatus between current knowledge and skill and required knowledge and skill in the future

‐ ‐

I felt that overcoming the challenge I identified was within my capabilities I identified a more general hiatus in my knowledge and behaviour that was not bound to a specific task.

Present (1) – A case is member of the set Sense of urgency if the case falls under every observable manifestation (being at least 4 in the Likert-type 5 scale). Dichotomization

Absent (0) – A case is not member of the set Sense of urgency if one attribute is missing. In particular, the case needs to be out of at least one observable manifestation (1,2,3 in any observable manifestation in the Likert-type 5 scale).

Condition

Relapse prevention Relapse prevention is a self-management technique by which individuals can become aware of environmental and intrapersonal threats to skill maintenance to anticipate, prevent, and recover from possible lapses into the old behaviours. The focus is on promoting transfer of training by immunising learners against environmental obstacles to transfer (Botke et al, 2018).

Conceptualization

Operationalization

All items were answered in a YES/NO questions.

When difficulties in applying new knowledge at work there is commitment to overcome the obstacles

‐

When attending the training, did you identify any difficulties that would make it difficult to apply the knowledge and skills at work? During the training, have you discussed certain methods or tools to avoid or overcome any obstacles in applying the training content at work?

Present (1) – If the answer is Yes on both items, the case is fully in the target set. Dichotomization

Absent (0) – If the answer is No on at least one item, the case is fully out the target set.

Condition

Employee Goal setting Goal-setting interventions involve either the actual setting of goals with regard to the implementation of new knowledge, skills and attitudes on the job, or the teaching of how to set such goals (Nijman, 2004).

Conceptualization

All items were answered in a YES/NO questions. Operationalization

Formulation of training goals by employee

‐

Did the employee formulate goals before the training? Which ones?

Present (1) – If the answer is Yes, the case is fully in the target set. Dichotomization Condition

Conceptualization

Absent (0) – If the answer is No, the case is fully out the target set. Identical elements A training design is considered as one with identical elements when the training program mirrors the actual job experience (fidelity) in three domains (fidelity within): equipment, work environment and psychological sphere (Lacerenza et al., 2017; Vanderlocht et al, 2013; Burke and Hutchins, 2007; Saks and Belcourt, 2006; Nijman, 2004; Rehmann, Mitman and Reynolds, 1995; Noe, 1986). All items were answered on a five-point Likert scale ranging from 1 = "Strongly disagree" to 5 = "Strongly agree". ‐

Operationalization

Dichotomization

The content of the examples in the training were relevant for my kind of work. Training program fidelity 3 ‐ The content of the training examples domains: equipment, mimicked my main work tasks. work environment and ‐ The training exercises I performed mimicked psychological sphere. common situations at work (for example discussions with colleagues, or discussions with supervisor, …). Present (1) – A case is member of the set Identical elements if the case falls under every observable manifestation (being at least 4 in the Likert-type 5 scale on every item).

Absent (0) – A case is not member of the set Identical elements if one attribute is missing. In particular, the case needs to be out of at least one observable manifestation (1,2,3 in any observable manifestation in the Likert-type 5 scale). Condition

Conceptualization

Training program as active learning method The main attributes of a training designed as ‘active learning instructional method’ are: (1) a learning method with (2) active student engagement through meaningful practice and (3) reflection on what has been learned and encountered. This is translated in the following operationalization (Dochy and Segers, 2018; Dewey, 1938; Kolb, 1984; Revans, 1982; Salas, Tannenbaum, Kraiger and Smith-Jentsch, 2012; Prince, 2004.). All items were answered in a YES/NO questions.

Active engagement through practice Operationalization

‐ ‐

‐ ‐ Reflection on what has been learned

‐

Did the training consist of practical exercises Were any examples used in the training that departed from relevant professional situations? Was there room for discussion during training? Did employees receive feedback after exercises? Was there room for reflection on what was taught in the training?

Present (1) – If the answer is Yes to all questions, the case is fully in the target set. Dichotomization

Absent (0) – If the answer is No on at least one of the questions, the case is fully out the target set.

Condition

Autonomy Free choice options and opportunity to make decisions (Botke et al, 2018).

Conceptualization

Items based the criso psychological climate questionnaire. All items were answered on a five-point Likert scale ranging from 1 = "Strongly disagree" to 5 = "Strongly agree". Operationalization

Free choice options and opportunity to make decisions

‐ ‐

I have many options for deciding how I do my work. I have the opportunity to make decisions in my work

Present (1) – A case is member of the set autonomy if the case falls under every observable manifestation (being at least 4 in the Likert-type 5 scale). Dichotomization

Absent (0) – A case is not member of the set autonomy if one attribute is missing. In particular, the case needs to be out of at least one observable manifestation (1,2,3 in any observable manifestation in the Likert-type 5 scale).

Condition Conceptualization

(Balanced) Workload A structural factor that might facilitate (or to the contrary impede training participation and transfer) is workload. More specifically, an employee’s workload consists of the personal capacity to transfer and includes factors such as role conflict, overload, and job-generated stress (Russ Eft, 2002). All items were answered on a five-point Likert scale ranging from 1 = "Strongly disagree" to 5 = "Strongly agree".

Operationalization

Dichotomization

Amount of work that allows(not) taskaccomplishment

‐ ‐

I have more work to do than I could ever do. The amount of work I have prevents me from doing a good job.

Present (1) – A case is member of the balanced workload when the score on every observable manifestation is 1,2 or 3 on the Likert scale. This means that there is absence of workload, as the observables manifestations related to the work overloaded job design. Absent (0) – A case is not member of the balanced workload if he or she at least agrees (4 or 5 on the Likert scale) with one of the above statements.

While the best fitting QCA model included nine conditions (5 causal conditions and 4 contextual conditions), information was also collected on additional conditions that were not ultimately a part of the analysis (10 contexts). The main reasons for this was that no necessary contexts were found in the QCA analysis, therefore we abandoned the initial idea of a two-step QCA variant (see Schneider, 2018), where remote and proximate conditions are distinguished and only remote necessary contexts enter in the analysis combined with proximate ones. We went further with a robust QCA that included 9 conditions. More in particular, these 9 conditions consisted of 5 causal conditions, one was a macro variable composed by two conditions (relapse prevention and goal setting), and 4 contextual conditions. The latter are rather to be conceived as enablers of processes that lead to the outcome. More information about other conditions in the analysis could be found in Annex 3: Literature review.

3.4 SELECTED CASE-STUDIES Below, Table 3 presents an overview of all cases and their membership status for each condition. White cells in a row indicate that a condition is absent in a case. Grey cells indicate that the condition is present. As seen, we have drawn on 50 cases, 15 successful case studies and 35 unsuccessful. We have collected data on the training participants in 2018 and 2019 via a survey design. This survey was a good instrument to identify key factors for the training transfer effectiveness. However, this was not the main tool that we used to gather data on training transfer process. For this, we relied, instead, on the semi-structured interview of particular positive cases based on the QCA results, carried out in 2019 and 2020.

Table 3: Overview of cases: membership of conditions and outcome

Type of Training Coaching and situational leadership Coaching and situational leadership Coaching and situational leadership Coaching and situational leadership Connecting communication - leadership and communication Connecting communication - leadership and communication Connecting communication - leadership and communication Connecting communication - leadership and communication Connecting communication - leadership and communication Connecting communication - leadership and communication Connecting communication - leadership and communication Connecting leadership Connecting leadership

Case ID

Effective training transfer

Peer support

Supervisor support

Relapse prevention

Goal setting

Sense of urgency

Identical elements

Training alignment as active learning method

Autonomy

Balanced workload

K1 J5

Connecting leadership Dealing with stress Leadership external Leadership external Leadership external Leadership external Leadership external internal Leadership external internal Leadership external internal Leadership external internal Leadership external internal Leadership external internal Leadership external internal Leadership external internal Leadership external internal Leadership external internal Leadership external internal Leadership external internal Leadership external internal Leadership external internal Leadership external internal

R3 C16 M2 T2 K3 N1 S1

Leadership internal external Leadership internal external Leadership internal external Leadership internal external Professional communication and leadership Professional communication and leadership Professional communication and leadership Professional communication and leadership Professional communication and leadership Self-care and stress management Self-care and stress management Self-care and stress management Stress management Stress management Dealing with stress Stress management Total

V1 M1 W2 H1 J4 15

Note: Grey indicates that the condition is present. White indicates the absence of the condition. Source: authors

4. ANALYSIS AND QCA FINDINGS This section describes the key findings of this QCA part of the evaluation. We start by introducing the QCA model. Then, we conduct a single-condition analysis, in other words, we investigate which individual conditions are necessary (necessity analysis) and which are sufficient (sufficiency analysis) for the outcome to occur9. Details of these analyses can be found in Annex 9, including truth tables. All findings are evaluated in terms of potential generalization. For simplicity, we sometimes shorten the conditions names to make the results easier to read. We also discuss the relevant cases for each pathway that we identify.

4.1 QCA MODEL After an exploratory survey analysis, we have selected those key contexts that can facilitate the training transfer in combination with the causal conditions. Figure 5: QCA model represents how we expect to explain and understand ‘effective employee social skills training transfer’ (i.e. our outcome). The QCA model10 contains (1) the contexts that act as ‘enablers’ of training transfer but without causing it and that facilitate the correct functioning of the causal mechanisms; (2) the casual conditions or factors that cause training transfer effectiveness and (3) the causal mechanisms that play a key role in the process. The causal conditions (in yellow) form a conjuncture of four conditions that jointly act as sufficient to lead to an ‘effective employee social skills training transfer’. The key point in this model is also the relevance of four contexts which will act as facilitators of the different processes taking place in the occurrence of the training transfer.

9 The so-called conservative solution was selected to study ‘training transfer effectiveness’ because we estimate it is the most appropriate when combining QCA with the study of processes or mechanisms via process-tracing. An elaborate discussion of the different solution types that are available in QCA is beyond the scope of this evaluation. We refer the interested reader to Álamos-Concha et al. (forthcoming) for an in-depth discussion on why we have selected the conservative solution in this evaluation study. 10 We have performed these analyses in R (version 4.0.2) with the QCA package (version 3.8.2) and the SetMethods package (version 2.5) .

Figure 5: QCA model

source: authors

4.2 SINGLE CONDITIONS ANALYSIS 4.2.1

Necessity analysis

We found that no single condition was required for training transfer effectiveness11. The only condition closest to being necessary was the presence of the context ‘training program as active learning method’, because of its consistency with the empirical data, however, its values in coverage and relevance of necessity are low12. This condition covers only 31% of cases of transfer effectiveness and its relevance is 16%. So, we have discarded this option. We also tested which disjunctions (or logical unions) of two conditions or more (up to three) were necessary. We found that no disjunction is required for transfer effectiveness. These findings are presented in Annex 9 [Table A9, A10, A11]. Two conditions relapse prevention and goal setting were transformed in a single macrocondition, following the theoretical approaches related to post-training intervention. This condition did not pass the necessity test neither.

The definition of a necessary condition when using crisp set is “whenever the outcome is present, the condition is also present” (Schneider and Wagemann, 2012 :345-347). 12 In simply terms, consistency refers to the percentage of the empirical data that are in line with a postulated subset relation. Coverage expresses how much of the training transfer effectiveness is covered by the explanatory condition. It is well understood in terms of the relevance and trivialness of a necessary condition. It is relevant when a set X is both a superset of Y and when it is not much bigger than Y nor ~X. Otherwise X is not a relevant, but a trivial necessary condition for Y (Schneider and Wagemann, 2012).

4.2.2

Sufficiency analysis

As seen in previous sections, a condition is sufficient if whenever it is present, the training is successfully transferred – but it is not present in all successful cases. We found that no single condition is sufficient by itself for successful training transfer at work. Conditions, instead, act in combination with other conditions to make training transfer successful. Note again that cases in the Table 4 refer to individual employees (configuration). The analysis covered only 12 out 15 positive cases, leaving out three cases with low consistency for the analysis. There is not a single cause that leads to training transfer. Instead, there are multiple combinations of conditions that could lead to the outcome. Each configuration is one part of a big picture of an explanation for transfer effectiveness. This whole picture contains eight pathways where conditions act in combination with others. Each configuration covers just a few cases. The first four configurations are those that explain more cases. This leaves no doubt that many roads lead to Rome and confirm our expectation that many different pathways are conducing to the success of training transfer in Flemish firms. The whole solution covers 80% of the cases and is highly consistent with the theory and empirical evidence found (see Table A12 and A13 in Annex 9). We also applied robustness tests for improving the conclusions (see Annex 9 for details). The result supports the fact that equifinality applies13, and we may focus on slightly different aspects of this single equifinal solution to explain training transfer effectiveness in our twelve successful cases. For example, one way to read the second configuration is: whenever trainees received support from their peer and from their supervisors, even if they did not experience neither sense of urgency nor relapse prevention and goal setting, they were capable to transfer the content of training programs at work. This occurred within the contexts of identical elements with the training, training program as active learning methods and a non-balanced workload. However, this only happened in two of our cases as can be seen in Table 4. Table 4: Sufficient configurations (pathways)

Case

Peer support

Supervisor support

Sense of urgency

Relapse prevention and goal setting

Identical elements

Training program as active learning method

Autonomy

J3; V2 B2; K2 M1; D1 N2; B3

Balanced workload -

W1 T1 S2 T2 13

Equifinality indicates that the same outcome can be reached from different starting points. 43

Source: authors. Note: White: condition is absent; Grey: condition is present; “-“ not included in the pathway. 4.2.3

The pathways to successful training transfer and the cases in them

Even though we started by investigating which single conditions were necessary and sufficient for the effectiveness of training transfer of leadership and stress management programs, our main objective was unravelling the different pathways to successful training transfer. Our analysis evaluated different combinations of the eight most important conditions and contexts that emerged from the myriad of potential conditions and empirical exploration (see section 3). The aim was to find a simple and robust explanation with high consistency, coverage, and relevance that helps understanding training transfer effectiveness(Annex 10 provides details on the important aspects of the calibration and analysis process), and the process behind it with a subsequent research strategy (see section 5). In what follows, we concisely describe the different pathways found. We highlight that the combination that explained most of the successful case studies were fairly different across all of the cases. This can be interpreted as that what serves one does not serve the other. In other words, context matters a lot.

Pathway 1 – Absence of support and sense of urgency and presence of relapse prevention-goal setting, within the contexts of absence of identical elements and presence of training program as active learning method and autonomy. Training transfer was effective in cases where employees applied techniques of relapse prevention-goal setting within the contexts of training as active learning method and autonomy. In these cases, peer and supervisor support and sense of urgency did not matter as well as the context of identical elements and balanced workload. This pathway explains two cases out of 12 successful cases, J3 and V2 in leadership training programs and communication skills and leadership training, respectively. The pathway is significant, especially as it contains two similar cases where the employee was able to cope with relapse preventiongoal setting to transfer under an autonomous work environment. V2 followed a training on coaching and situational leadership. She has been working as a supervisor at the same company as D3 works in for almost twenty years. A company that produces, and sells, bathroom taps, furniture and accessories, amongst other things. J3 works at HR at a hospital and followed a course on leadership skills. The absence of both types of support and a sense of urgency make these interesting cases. We suspect that relapse prevention and goal setting allowed these cases to transfer the training to the work floor. Unfortunately, we were unable to interview these cases to confirm this and therefore have to rely mostly on the surveys that these persons filled in. The survey of V2 revealed that she did set some clear goals before she participated in the training. She mentions that she wanted to “give more concise and clear feedback to the employees that work with her” and that “she needs to have reasonable expectations for those employees”. Although J3 mentions that he set a goal for himself before the training, he did not specify what that goal was. According to our operationalization, J3 did not experience identical elements. However, when asked what stimulated him to apply the training, he said that the training content was clear and applicable to his work situation, suggesting that there were at least some elements that contribute to identical elements. 44

Pathway 2 – Presence of support from peers and supervisor and absence of sense of urgency and relapse prevention-goal setting, within the contexts of identical elements and training program as active learning method and the absence of a balanced workload. An employee training transfer was effective in cases where employees were supported by peers and supervisors, and where the training was aligned with the actual job experience of the employee in equipment, work environment and psychological sphere; where the training programs was carried out as an active learning method even if the workload was not balanced. In these cases, the absence of sense of urgency and relapse prevention-goal setting did not matter. This pathway only explains 2 cases and had high consistency and a low coverage (explaining two out of 12 successful cases). The pathway, as mentioned earlier belongs to the conservative (or complex) solution where most of the conditions conceptually studied, are playing a role in the final explanation. This is relevant when moving to the process tracing study, because we cannot accept losing ‘valuable information’ that can be relevant to the understanding of how the employees behave in the process of training transfer, i.e. the absence of a condition in a given process can completely modify the process itself for each case, therefore we should avoid this risk. One way to read this configuration is: whenever trainees received support from their peers and supervisors, even if they did not experience neither sense of urgency or relapse prevention, they were capable to transfer the content of training programs at work within the contexts of identical elements with the training, training program as active learning methods and a non-balanced workload. There were two cases in our sample in this pathway: B2 and K2. These cases both experienced support from peers as well as supervisors. There was no sense of urgency nor relapse prevention in both cases. K2 works as a head nurse at a hospital. We interviewed her after the training (connecting communication) about how she was able to apply the training. She indicated that the peer support was beneficial for the training transfer. She mentioned that she meets every two weeks with the other head nurses. During these meetings, the training and its application was sometimes discussed. Although she indicated that she receives support from her supervisor, this seemed to be less vital in transferring the training. The support that she received from her supervisor is mostly in the form of trust, rather than specific feedback on how to apply the training. Despite having answered in the survey that the cases that were discussed mimicked those that she experienced in her main work tasks, she also mentioned that what was taught at the course was not always representative of the “real world”. She solved this by picking out the things that were applicable. B2 works as a manager at the same company as N2 and B3 (which are discussed extensively later). The company provides digital signage at events or points-of-sale to companies. Because of her busy schedule, we were unable to interview her. She mentions in the survey that she followed the training “on her own initiative”. Besides that, the answers she provided in the survey did not reveal any surprising findings.

Pathway 3 – Presence of support from peers combined with the absence of sense of urgency and the absence of relapse prevention-goal setting, within the contexts of identical elements and training program as active learning method and autonomy and a balanced workload. The third successful combination was having support from peers within the context of a training that was aligned in equipment, work environment and psychological sphere with the actual job experience of the employee; where the training program was carried out as an active learning method and where the trainee enjoyed autonomy. In these cases, it didn’t matter that the employees did not feel a sense of urgency to learn and transfer and that they did not implement techniques related to relapse prevention-goal setting. For example, in the case of M1 and D1, taking stress management training and communication and leadership social skills trainings, respectively, the most relevant cause for transfer was peer support and the other ones were not present, while all identified contexts , including a balanced workload, were present. It is important to note that this was the only pathway where the four contexts were present in combination with a single causal condition. This pathway explained two of the 12 successful cases. These two cases in pathway 3 were M1 and D1. These cases both experienced support from peers and an absence of sense of urgency or relapse prevention. D1 was selected for a more thorough investigation and we thus refer to the case study where we discuss this case for a more detailed description of this case. In that case study, we focus on the role that peer support played for training transfer. Both in the interview as in the survey, it was clear that she did not experience a sense of urgency. M1 followed a training on stress management. The survey showed that his supervisor was not strongly involved in application of the training. He did not have discussions with his supervisor about the training and also didn’t get feedback from his supervisor in regard to the training. However, he did get support from his peers. This support took the form of providing opportunities to use the training, having the technical knowledge to help others to apply the training, encouraging each other to apply the training and giving positive feedback to each other in order to apply the training.

Pathway 4 – Presence of supervisor support and relapse prevention-goal setting, and absence of peer support within the contexts of identical elements and training program as active learning method and autonomy and a balanced workload. There were several cases where peer support was not needed as long as the supervisor provided strong support or where trainee had a sense of urgency to learn and transfer or where trainee implemented techniques of relapse prevention-setting goals. This pathway, in particular, applied to two of the 12 successful cases, as long the supervisor support was engaged in training transfer, the peer support does not matter, because trainee had already a kind of support. Cases such as the N2 and B3, the training transfer effectiveness was characterized by a lack of peer support within the contexts of autonomy, balanced workload, training program as an active learning methods and identical elements. It may be worth noting that sense of urgency was irrelevant since its presence or absence does not make any difference in the impact on transfer in those cases.

We identified two cases in pathway 4: N2 and B3. Coincidently, both cases work at the same company. They both experienced support by their supervisor and there was relapse prevention and employee goal setting. Although, based on the survey, we concluded that they did not experience clear peer support, during the interviews that we performed with these two employees, they did mention that they received some peer support. However, it could be possible that these answers were the results of a social desirability when answering questions. Because we discuss these cases later in more detail, we refer to those sections in which we discuss these cases, and how they applied the training, in more detail. We saw that, in N2, the effect of supervisor support on training transfer was quite clear, while, with B3, we could see how employee goal setting and relapse prevention contributes to training transfer.

Pathway 5– Absence of support and relapse prevention-goal setting and presence of sense of urgency, within the contexts of absence of identical elements and presence of balanced workload and presence of training program as active learning method and absence of autonomy. Training transfer was also effective in cases where trainee was motived by a sense of urgency, where employee had a clear need to engage in training due to the identification of a hiatus between his/her knowledge and skills before the training and the required knowledge and skill post-training. Trainee understood that overcoming the hiatus is within reach of his/her capabilities. For this single case, W1 the support from peers and supervisor did not matter, as well as the techniques or relapse prevention-setting goal, because the sense of urgency was stronger for keep the training alive. The contexts here were relevant, for example, the balanced workload enabled the trainee to be focused on learning and training program as active learning methods facilitated the transfer. Other contexts such as identical elements and autonomy did not matter either for this single case. This is the only one case where a single individual factor is involved in the outcome achievement. There is one case in pathway 5, W1, who followed a course on self- care and stress management. This case did not experience support or relapse prevention but did identify a sense of urgency. In the survey, he completely agreed with the statement “I was confronted with a challenge or work-related problem that needed to be solved urgently”. Furthermore, in the survey before the training, he said that he often felt like he had no control over important things, rarely felt that things were going the way he wanted and had so much troubles that he would not be able to deal with all of them, illustrating that he was suffering from a serious amount of stress at work. The training helped him deal with this stress. He mentions that the training helped him to make a tough decision: he switched to a different position in the company. This has made him happier in the company and he also mention that he needed this to further develop himself.

Pathway 6 – Absence of support and presence of sense of urgency and relapse prevention-goal setting within the contexts of identical elements and training program as active learning method, autonomy and unbalanced workload. The sixth successful combination was having the presence of sense of urgency and relapse prevention-goal setting within the contexts of identical elements, training program as active learning method and autonomy. In the single case, support from peers and supervisors did not matter as well as contexts of workload 47

balance. The case of T1 is a particular one, because the efforts to achieve training transfer came from the own employee due to the autonomy he had. The need to engage in training due to the identification of a hiatus and the capacity to overcome the hiatus during the training, combined with the coping strategies to keep the training alive, allowed T1 to succeed in transferring the training content to the job. T1 followed a course on leadership skills and works at a hospital. He was able to transfer the training, despite not experiencing significant support from peers or his supervisor. One of the main goals why he followed the training was to increase his confidence. The training also helped him to better help team members in identifying which additional training they need. When we interviewed him, T1 told us that the implementation of specific skills also depends on the training content. For instance, implementing the tips given by the instructors to apply during evaluation meetings with staff members was fairly easy to implement because there was a clear moment when to use these tips. He said that this made it a lot easier to use the training as he was sort of “forced” to use it and he knew when to try it out. Although, in the survey, he indicated that “he did not have all the knowledge and skills required for his job”, it was not as clear during the interview that he experienced a sense of urgency.

Pathway 7 – Presence of peer support and absence of supervisor support, combined with the absence of both sense of urgency and relapse prevention-goal setting, within the contexts of absence of identical elements, absence of balanced workload and absence of autonomy and presence of training program as active learning method. Training transfer was effective in cases where employees received support from peers within the context of training program as active learning method. Supervisor support, sense of urgency and relapse prevention-goal setting did not matter, as well as the context of identical elements, autonomy, and balanced workload. This pathway tells us something about the importance of peers in keeping the training application alive within the work environment. In pathway 7, peer support was one of the few things that could actually trigger training transfer. In our sample, there was only one case in this pathway: S2. He is responsible for production at the same company as D1, who we discuss more extensively in the process tracing part. In D1’s case, there were clear indications that peer support played an important role in transferring the training. S2 is one of the peers that has supported D1 and likely also has experienced a similar level of peer support as she did. We thus expect that peer support also played an important role for S2 during the training transfer. Unfortunately, we were unable to interview S2 to gather additional insights into this process. S2 also noted that his high workload was something that impeded the implementation of the training.

Pathway 8 – Presence of peer support and absence of supervisor support and presence of sense of urgency and absence of relapse prevention-goal setting, within the contexts of identical elements, autonomy and balanced workload and absence of training program as active learning method. The final successful combination was having support from peers and have a sense of urgency to transfer, within the contexts of identical elements and autonomy and balanced workload. In the single case in this 48

pathway, it didn’t matter whether the supervisor implemented measures to support the trainees or if the trainee implemented techniques of relapse prevention-goal setting. Training program as active learning method did not make a difference neither. The final pathway covers one case in our sample: T2 who works as a supervisor at the IT department of a production company. He took the initiative to follow the training himself. After the training, we interviewed T2 and discussed these dimensions and the application of the training in greater detail. Despite, according to what he filled in in the survey, having peer support, he did express the desire to discuss the training with his peers more often. Because the people he followed the training with worked at different departments and there were no formal meetings, he was unable to really discuss the training and its application with peers. His supervisor also followed the training and gave some feedback, but this was limited and unlikely to contribute to training transfer. After the training, he recognized the benefits of it and how it would help him to do his job better. However, during the interview he said that, beforehand, he could not see a clear need for the training, and he was unsure what he would learn during the training. This is in contrast with the answers he gave in the survey.

Cross-case conclusions based on pathways These eight pathways are part of a single solution that explains training transfer effectiveness in the twelve positive cases [J3, V2, B2, K2, M1, D1, N2, B3, W1, T1, S2, T2]. Based on the analysis of each pathway we can infer that: •

•

Many different pathways to training transfer exist. There was not a single case in which all conditions were favorable. There were a couple of similarities. Most of those who successfully transferred their training programs were taught with an active learning method. Given that most of the trainings used such a method, as can also be seen in Table 4, this is not surprising. Perhaps somewhat surprising is the general lack of sense of urgency. Only 1 out of 3 successful cases experienced a sense of urgency. However, this is still substantially higher than the general average in our population (7 out of 50 cases). This clearly shows that a high sense of urgency can contribute to successful adaption of the training. These low rates for sense of urgency may be explained by who took the initiative to follow the training. Most employees were obligated by HR or their supervisor to follow the training. This was also noted by employees in the survey or interview when asked about who’s idea it was to follow the training. They rarely took the initiative themselves. This could explain why they are less prone to see the relevance of the training. Support from peers was relevant for six out twelve positive cases. In addition, the condition relapse prevention-goal setting did not matter, and no common contexts were figured out in those six cases. However, when zooming in closer, the context identical elements turned out to be a facilitator for five out of six cases. Only [S2] did not enter in this set. Similarly, the context training program as active learning method was a facilitator for five out such six cases. Finally, observing those cases by pairs, we can infer that the context balanced workload was present for three of those six cases [M1, D1, T2], while the absence of an unbalanced workload did not matter for the three remaining cases [B2, K2, S2].

•

• •

Support from supervisor was relevant for four out of twelve cases [B2, K2, N2, B3]. The common contexts for these four cases was the presence of identical elements and training program as active learning method. The need to engage in training due to the identification of a hiatus and the capacity to overcoming the hiatus during the training was present in three out of twelve cases [W1, T1 and T2]. In two out of such three cases, identical elements and autonomy acted as facilitators of transfer [T1, T2]. Finally, in two out of three cases, the context balanced workload facilitated transfer [W1, T2]. Keeping the training alive with relapse prevention-goal setting techniques functioned in five out twelve cases [J3, V2, N2, B3, T1]. Autonomy was a facilitator context for those five cases. Three out such five cases shared the context of identical elements as a facilitator of transfer [N2, B3, T1]. All cases share the context of training program as active learning method [N2, B3, J3, T1, V2]. Finally, two out such five cases share the context of balanced workload as a facilitator of transfer [N2, B3]. The support as a whole (from peers and supervisor) did not matter for four out of twelve cases [V2, J3, W1, T1]. These cases, as already analyzed, search other ways to transfer the training content to the job. Sense of urgency did not matter neither for seven out of twelve cases [B2, K2, M1, D1, V2, J3, S2]. Similarly, relapse prevention-goal setting techniques did not matter for seven out twelve cases of training transfer [B2, K2, M1, D1, W1, S2, T2]. When causal conditions of training transfer effectiveness studied in the literature did not matter in real life cases, this means that there are other ways to achieve the goal. Another thing that stood out during the interviews was how trainees often interacted with their peers. Several cases (D1, S1, K1, K2, organized recurring meetings with their peers, which often aided the adaptation of the trainings. T2 noted that he did not have this but would have liked to have this as it would have made it easier to implement the training. Having contact with peers that followed the training at regular intervals is thus also something that could clearly support training transfer.

4.2.4

Failed training transfer effectiveness

In most cases, peer and supervisor support, relapse prevention and sense of urgency are not playing a role in the failed training transfer. Surprisingly, in some cases the contexts are present, but the causal conditions are absent. This is in line with the fact that nothing can be more important than the cause. If the cause is present, the contexts can act as facilitators but not vice versa. In addition, some unsuccessful cases can be explained also by different pathways. Table 5 illustrates the different pathways that can lead to an unsuccessful outcome. The cases of C2, C3, E3, J5, P2, T3 are explained by pathways 1,2 and 3, where the absence of support combined with the absence of relapse prevention are common reasons for the failure. In the solution, there is a single unsuccessful case where all the causal conditions are present, acting within the context of identical elements, autonomy, unbalanced workload, but where training alignment as active learning method is absent. It is interesting to reflect how, in spite of having all the causal conditions present, H1 did not succeed in training transfer. The presence of all causal conditions seems to not guarantee success, if not within certain contexts. A balanced workload and training as active learning could have made a difference. 50

A more detailed picture can be found in the two selected pathways below where most of the unsuccessful cases are concentrated. The first pathway covers 11 cases and the second pathway covers 15 cases.

Table 5: Sufficient configurations for unsuccessful training transfer (pathways) Causal conditions

Peer support

Supervisor support

Case

M2; N1; C5,F1; C2,C3,E3,J5,L1,P2,T3 C2,C3,E3,J5,L1,P2,T3; D3,J2,J4,R1,R2,S4; D2; P1 C2,C3,E3,J5,L1,P2,T3; A1,D4,E2,V1 E1; J1 D3,J2,J4,R1,R2,S4; C1,G1 K3 C4; M3 C6 C16 H1

Contexts

Sense of urgency

Relapse prevention and goal setting

Identical elements

Training program as active learning method

Autonomy

Balanced workload

Note: White: condition is absent; Grey: condition is present; “ –“ not included in the pathway The QCA revealed two major configurations in which trainees failed to transfer the training. We will focus our discussion on these two major pathways. Pathway 1 – Absence of support and sense of urgency and relapse prevention-goal setting, within the contexts of absence of identical elements and a balanced workload. In this pathway, trainees did not receive support. There is a lack of motivation to learn and absence of relapse prevention-goal setting. Furthermore, the trainees experienced an unbalanced workload and training programs was not the one identical elements. This configuration lead them to not transferring the training, because all the causal conditions and contexts are absent. There are 11 cases in this configuration: M2, N1, C5,F1, C2,C3,E3,J5,L1,P2 andT3. Given that we did not contact these cases for further research, the information we have on these cases is limited. In the survey T1, we asked these cases “if you do not apply the training content, what reasons prevent you from applying the training?”. The answers on these questions were diverse: a lack of time because of a busy schedule (M2), the training was not relevant enough for their job (F1), a lack of opportunities to use the training (L1), being rushed so that the trainee forgets to use the training (C5) or an inability to leave old habits (P2) were the given explanations. These answers illustrate the importance of some contexts. For instance, C5, shows that a balanced workload is recommendable if one wants to implement the training.

Pathway 2 – Absence of support and relapse prevention-goal setting, within the contexts of training program as active learning method and autonomy and an absence of identical elements. We have 15 cases to which this pathway applies. The failure of training transfer effectiveness is also explained by the absence of support and relapse prevention-goal setting. In contrast to pathway 1, sense of urgency was not considered. The contexts are also different. These 15 cases shared the presence of training program as an active learning method and autonomy, but they were not relevant to make the causal conditions to work. Identical elements were, once again, absent. The workload was irrelevant in this pathway. There were 15 cases here: C2, C3, E3, J5, L1, P2, T3, D3, J2, J4, R1, R2, S4, D2 and P1. When we asked these cases “if you do not apply the training content, what reasons prevent you from applying the training?” in the T1 survey, the answers on these questions were again diverse: additional training is needed (D2), the training was not applicable enough (J2, J4), a lack of opportunities to use the training (L1), the training felt too artificial (D3), there was no follow-up (R2) or an inability to leave old habits (P2). Several of these reasons (J2, J4, L1, D3) point to the absence of identical elements as an important obstacle to implement the training.

Cross-case conclusions As seen, the failure of training transfer effectiveness can be explained by the absence of peer and supervisor support, relapse prevention and sense of urgency. Even if some contextual conditions are present in some negative cases, the outcome does not occur, because the cause of the process to make transfer succeed was not created. An anomalous case is the one where all the causal conditions for the successful training transfer are present, acting within the context of identical elements, autonomy, unbalanced workload, but where the outcome is absent. This is a deviant case, where the occurrence of the outcome was not present even with the presence of all causal conditions. A balanced workload and training as active learning could have made a difference for this particular case, though. The absence of support seems to be a key reason for the failure of training transfer. Identical elements were also missing in both pathways, and several trainees suggested that this contributed to the failure of the training. However, it is important to highlight that an alternative theory to explain and understand the failure is necessary. Although it is out of our scope to explain the unsuccessful case of transfer, we shed some light on the implications of these factors in the failed cases.

5. OVERARCHING FINDINGS OF PROCESS TRACING In this section, we present the findings of the process-tracing part of our study. The main goal is to provide insight into how various causes can lead to training transfer. We do this by looking at how causal mechanisms played out in several of our cases. We first relate the identified causes to possible mechanisms that could lead to training transfer. We discern four possible mechanism in our analyses: signaling and retention triggered by supervisor support, enhanced training transfer intervention triggered by peer support, self-management intervention triggered by relapse prevention -setting goals and learner agency triggered by sense of urgency. In section 5.1, we discuss these mechanisms and we provide some insight into what could happen in these mechanisms. For a more detailed, step-by-step, description of how we expected these mechanisms to work in theory, we refer to Annex 6. Next, we provide an overview of our successful cases and select those cases where these mechanisms may have been present. In the remainder of section 5.2, we describe how these processes of transfer actually took place in four particular cases. Again, for a more detailed, step-by-step, description of how these mechanisms took place in our cases, we refer to Annexes 11 and 12, that contain road maps and additional evidence that was used to draw conclusions on these mechanisms. Finally, we conclude this section with the implications of these findings in the specific cases studied.

5.1 KEY CAUSAL MECHANISMS In our comparative analysis with QCA, we learnt that no single condition alone can fully explain transfer effectiveness. Rather, different combinations of conditions together can be used to explain the outcome. The explanation is composed of eight pathways, each containing 1 or 2 cases. For this reason, generalization to the population is difficult. We are still in the dark regarding the way in which training transfer effectiveness was achieved. In order to gain insight into this process, we will unpack ‘the black box’ by studying the process that links the combination of conditions and the outcome. As pathways can be generalizable only to the set of cases explained by them, we applied a technique which consists of the isolation of those conditions that we are interested in14. In other words, we isolated our causal conditions in order to open the black box triggered by them. In this section; we discuss how each causal condition could lead to training transfer. We discuss four mechanisms: signaling and retention triggered by supervisor support, enhanced training transfer intervention triggered by peer support, selfmanagement intervention triggered by relapse prevention -setting goals and learner agency triggered by sense of urgency.

Isolation of conditions here, refers to the technique to study key processes triggered by relevant conditions (in this case, causal conditions rather than contextual conditions, because the former has capacity to trigger a process). We may study the whole causal process included in a particular pathway. However, since the scope of this research was focused on studying four mechanisms, we estimated that it was more convenient for the purposes of this research to isolate our four causal conditions and to unravel the process that was taking place. We consider the causes that trigger the self-management mechanism as a single condition. Isolation was a technique suggested in a workshop with professor Derek Beach at Aarhus University (September 2019).

5.1.1

Signaling and Retention

1. What is signaling and retention? In the broader learning organisation literature, signaling and retention is a process through which supervisors influence the behavior of employees Govaerts, Kyndt, Vreye and Dochy (2017). This is in line with the central role that is attributed to the supervisor in developing employees’ learning process and creating cultures for learning (Senge, 1990). Drawing on signaling theory (Sitzmann & Weinhardt, 2015), a supervisor sends strong messages about the value and importance of the training to the trainee. Thus, employees can use the supervisor’s actions to inform their own attitudes and behavior toward training. As such, supervisor support, which may consist of different actions, can influence trainee’s perception concerning the importance of training and the subsequent effort they put in learning and applying it in their job. The reason why signaling, and retention can lead to transfer may lie in the multilevel structure of organisations (Govaerts, 2017). In particular, in hierarchical or hybrid organisations, employees are often held accountable by and have to report to the person higher in the hierarchy. This implies that employees are, in some extent, dependent on their supervisors for accessing resources, equipment and information. Furthermore, supervisors also play an influential role in defining and implementing HR policies, HR practices, and expectations. In short, through the professional and personal relationships with their employees, supervisors send strong messages about the value and importance of training. 2. The role of signaling and retention in the theory of training transfer Govaerts et al (2017) suggest that supervisor signals may increase the training retention of employees. Training retention refers to the degree to which the training content is retained after the training is completed (Govaerts, 2017). Training retention is generally acknowledged to be crucial for a training program to be considered successful: trainees should not only acquire the training content but also retain it in order to apply it in the work context. The role of signaling and retention was already emphasized by Baldwin & Ford (1988) in their core model of training transfer. They posit that learning and retention, as outputs of training, directly influence training transfer. In addition, Baldwin and Ford (1988) contend that supervisor support affects the employee’s learning and retention in the training context and that this link is one of the critical linkages in the transfer process. Working with the different types of supervisor support behavior and attitudes developed by Govaerts et al (2017), the mediating role of training retention in the relation between (perceived) supervisor support and training transfer is empirically evaluated. The conclusion is that only the supervisor’s accountability and involvement in training are positively related to training transfer. They explain that only the employees who perceive that their supervisors know about the content and learning objective of the training, and who perceive that their supervisors expect them to put the training program to use, report greater training use in their job. This is in accordance with Baldwin & Magjuka (1991) who found that trainees feeling to be held accountable by their supervisor for their use of training material on the job, reported stronger intentions to transfer. Moreover, Govaerts et al (2017) also conclude that training retention acts as a mediator, in between supervisors’ involvement and accountability and the training transfer. Those supervisor’s actions seem to make trainees retain the training content and what they have learned after training is completed and makes them more likely to apply the lessons learned in their job. It might be that supervisor’s involvement and accountability most clearly signals the importance of training to trainees, as compared to other types of support. 55

3. Supervisor support as the causal power According to the training literature, we conceptualized supervisor support as ‘sources of encouragement, assistance, reinforcement, opportunities and guidance (feedback) for employees on their use of new knowledge at the workplace’ (Cromwell & Kolb, 2004; Elangovan & Karakowsky, 1999; Gregoire, Propp, & Poertner, 1998; Holton 1997; Lancaster et al 2013; Nijman, 2006; Quinones, Ford, Sego, & Smith, 1995; Richman-Hirsch, 2001; Salas & Cannon-Bowers, 2001;Van der Klink et al., 2001). This support is understood as a behavior (encouraging, reinforcing, providing) or a multidimensional involvement from the supervisor (Lancaster et.al., 2013) either, before, during, and after a training program takes place (Govaerts, 2017; Brinkerhoff & Montesino, 1995; Cohen, Underwood, & Gottlieb, 2000; Cromwell & Kolb, 2002; Lancaster et al 2013). Based on Govaerts, Kyndt and Dochy (2017), we theorize that supervisors who are involved in the training, know about the content and learning objectives, and who expect the employees to put the training to use, send signals about the importance of training. This can affect the trainee’s perceptions of training importance: the supervisors’ involvement and accountability can trigger a causal mechanism related to training retention. The supervisors may influence the effort trainees put in retaining the training content by sending signals about the importance of training. This produces effective training transfer. Of course, this also happens under certain context circumstances. However, the literature does not allow us to clearly describe the context necessary for this process to be enabled. We do assume that this particular trigger and process can only be activated in an organizational context in which there exist hierarchical or hybrid atmosphere and meaningful relations between supervisors and subordinates. The functioning of signaling and retention process is described in the section below. 4. Signaling and retention behavior Signaling and retention is focused on the idea that when supervisor support takes place, this triggers a causal mechanism related to ‘signalling importance and increasing training retention’ that produces effective training transfer, during and after a training program and within a particular organizational context. Shortly, it points attention to the fact that supervisors may send signals about the importance of the training to employees through their supportive attitudes and behaviour. This may increase the training retention of the employee, which will help to apply the acquired content of the training to the job. Signaling and retention acts in learning and performance stages of training. We disentangle the process as a complex mechanism consisting of a single pathway: a cause (supervisor support) that triggers the mechanism consisting of six building block and seven parts. The process of signaling and retention would start in the learning process, with the ascribing importance to training building block. Here the supervisor ascribes importance to the training program and takes initiative to let the employees follow the training. The employees react by putting the training in their agenda [in this context the training is mandatory]. The second building block is employees follow the training. Everybody follows the training in group, in part because it was mandatory to do so by the supervisor. The third building block is facilitating learning climate, where supervisors enable employees to follow the training by taking over the workload during the training period. As such, in parallel, employees can focus on learning the training content. This leads to an intermediate outcome, where an organizational climate is created in which employees perceive the importance of the training for their job, and in which they acknowledge the engagement of the supervisor in encouraging this goal. The next building block is motivation to generalize, where because of the perceived 56

relevance of training, the employees following the training feel motivated to use the learned content and discuss it with peers. Employees also try out/use the training in task-related matters keeping the level of motivation that 'they just have to try it to learn' within an environment of trust and cohesion. The next building block is keeping it alive signaling, where supervisors keep on reminding employees to use the training ("keeping it alive") and provide feedback on the tasks related to the training application. [There is a feedback loop between the motivation to generalize and keeping it alive]. Finally, the last building block is increasing generalization, where, due to the engagement of supervisors (and peers) and trust between trainees and supervisors, post-training evaluation feedback systems are implemented by supervisors until task-oriented new knowledge is retained and improved in its application by employees. This process leads to training transfer effectiveness where employees are capable to apply or use the learned knowledge (content, skills or attitudes) acquired in the training context to the workplace for a long-term period of time. 5.1.2

Enhanced learning transfer intervention

1. What is enhanced learning transfer intervention? Enhanced learning transfer intervention can be understood as a ‘motivation-oriented intervention’. According to Baldwin and Ford (1988), three factors combined lead to training transfer: trainee characteristics (intrinsic motivation), training design (match between needs and what training offers) and work environment (support) (see more in Chauhan, et al, 2016; Baldwin and Ford, 1988). An enhanced learning intervention is a process where these three characteristics enter in a productive dynamic. In this dynamic, the work environment is the most relevant to create motivation. Support at work, opportunities to use the training, a learning culture, a reward system and task constraints (Elangovan & Karakowsky, 1999) are elements that can facilitate or hinder an effective transfer of training. In the literature, the work environment is a core aspect of transfer, because the support received at work can facilitate to use the learned content and skills in a context other than the training (Blume et al., 2010). Among other sources of supports, Baldwin and Ford (1998) have suggested that peer support can play a key role in creating an organizational climate conducive to transfer (Hawley & Barnard, 2005; Kirwan & Birchall, 2006; Lau & McLean, 2013) and that its absence can acts as a barrier to training transfer. For this reason, peer involvement is key in the whole enhanced learning intervention process. Also, because peers are usually closer to trainees than supervisors, they tend to influence them more (Van der Klink et al., 2001). 2. The role of enhanced learning transfer intervention in the theory of training transfer Hutchins and Burke (2007) build on the link between enhanced learning intervention and training transfer. In their views, support and motivation are interwoven when they lead to transfer of training. Bhatti et al. (2013, 2014) also considers transfer motivation as a mediator in the relationship between social support and training transfer. Similarly, the theory of planned behavior (Ajzen, 1985) suggests that employees could have a stronger intent to transfer when peers are also on the same page. This relates to attitudes or perceived control of individuals to perform a behavior in an organizational setting when influenced by perceptions of peers opinions (approval, disapproval) of such behavior. An enhanced learning intervention implies, therefore, the involvement of co-workers who encourage the use the training content at work via improving the ability of peers to effectively apply the learned skills in the workplace (Van den Bossche et al., 2010; Chiaburu, 2010; Bates et al., 2000). The ability to apply the learned skills is also related to the motivation reached by the trainee produced by the different dynamics between peers. Kluger and DeNisi (1996) mention the importance of helpfulness or feedback in supporting learning and the motivation to 57

learn (Annett, 1969; Ashford and Cummings, 1985). When the trainee perceives a gap between the feedback that he received on his own performance and his own goal, the trainee will tend to reduce that gap. This happens by combining the motivation to do the task or the learning of the task itself with ad hoc information so that both simultaneously contribute to a change in behavior that improves performance. In order for this to happen, the trainee needs some sort of network with peers and some diversity in the sources of feedback for behavioral change (Smither et al. 2005). These sources of feedback need to be linked to social support provided by the work environment for stimulating the developmental activities undertaken. 3. Peer support as the causal power In the training transfer literature, ‘peer support’ is understood as the ‘optimization of the trainee’s use of learning on the job by colleagues’ (Noe, 1986; Nijman, 2004), which can also be a ‘perception’ thereof (Reinhold et.al, 2018) or a ‘behaviour’ understood as ‘to optimize the trainee’s use of learned material’, ‘reinforcement for trainee’s use of learning on the job’ (Russ Eft, 2002), or ‘encouragement’ (Martin, 2010). Peer support triggers an enhanced learning transfer process, by considering feedback, encouragement, problem-solving assistance, supplemental information, coaching assistance as key aspects to training transfer (Hatala & Fleming, 2007; Gilpin-Jackson & Bushe, 2007; Jellema, Visscher, & Scheerens, 2006). As seen, enhanced learning process is interwoven with the notion of ‘motivational process to transfer’ which facilitates the skill transfer (Chauhan et al, 2016; Egan et al., 2004; Seyler et al., 1998). This motivational process is the intended effort to use the knowledge and skills acquired in the training, but it is necessary that there is both a commitment of the employee as well as opportunities to use the training content. Peer support can play a key role in motivating colleagues to not only apply the training but also to improve their understanding of the training so that they are better able to transfer the training. The enhanced learning in groups, with coaching sessions, feedback, sharing knowledge and experience also creates a proper work climate of trust and proximity that facilitates the learning and transfer processes (Kirwan and Birchall, 2006). 4. Enhanced learning transfer intervention behavior Peer support triggers the ‘enhanced learning transfer’ causal mechanism, which means that the learned knowledge by the trainee is applied at the workplace. In other words, support by peers enables the improved understanding of the learned content by trainees and the stimulation of generalization to the job context. This happens within the context of a given intervention (training program) and given certain organizational and situational characteristics. The enhanced learning transfer causal mechanism refers to the commitment of peers to provide the trainee with the resources and tasks that allow him or her to improve the skills or knowledge acquired in the training setting and stimulate its usage on the job. This includes networking and information-sharing with peers to enhances skill transfer (Chauhan et. al., 2016). As possible key parts of the causal mechanism, we can mention - as illustration - the provision of resources and ‘follow-up’ from peers to trainees to stimulate them to exert effort in order to improve their skills and to apply the learned content to the job. These resources could take the form of: feedback, encouragement, assistance in problem-solving, supplemental information, and coaching assistance. These actions are what enables training transfer effectiveness. Since the causal mechanism needs to be conceptualized as a system, i.e. entities engaged in activities, the main

parts of this causal mechanism (parsimonious) can be illustrated theoretically (without observable manifestations) as follows: The process of enhanced learning transfer intervention would start with two situations occurring in parallel within a first building block named following the training. The trainees follow the training together within a flat organizational structure. Because peers follow practice-oriented training together, trainees are able to practice what is being learned together and recognize the relevance of its content and the importance of ‘doing it in group’ for their work-performance. In addition, doings this in group leads to employees feeling that they are doing this together. With this knowledge in mind, the second building is building up common understanding, where peers communicate their different views about training implementation, in an open way, with the result that (1) they learn to trust each other even better and 2) that they acknowledge that a different way to work could improve their professional skills. This opens the way to the occurrence of an intermediate outcome where peers gain a ‘common understanding’ of how to implement the training. The third building block is intervision, where most of the process of enhanced learning intervention takes place. Here in parallel with the intermediate outcome, peers propose to organize ‘intervision moments’ as a ‘peer coaching activity’ to better implement the training content to the job. They agree to follow coaching activities – because they trust each other and they recognize the need of a different way of working. As part of the activity, peers meet each other to discuss the implementation of the training, specific cases and share experiences (e.g. issues, challenges or problems). They also ask clarifying questions to other peers to understand the situation and issues at hand when facing issues, challenges or problems. As result of this, peers start to brainstorm and bring up alternatives for action to support other peers. Thus, peer(s) make(s) a synthesis and formulate recommendations to their colleagues with the subsequent debrief : “what did they hear, what do they make of it and what do they take with them”. The following building block is adaptability and application, where, as a result of intervision moments, peers feel more stimulated to apply the content learned and are less resistant (adaptability). Therefore, they apply the content (when it is ad hoc to the problem/challenge identified at work), after a reflection of what they heard during the intervision moment. New processes of intervision moments are proposed (feedback loop) after a period of adaptation, where peers discuss the application and get feedback from other colleagues in subsequent intervision moments (follow-up post-training application). Finally, the last building block is New working thinking, where the employees incorporate the new way of working thinking after an adaptation phase and it becomes routine. Thus, training transfer effectiveness takes place. 5.1.3

Self-management intervention

1. What is self-management? Self-management can be understood as a ‘post-training transfer intervention’ (Rahyuda et al, 2014), as ‘post-training strategies’ (Wexley and Baldwin, 1986), as ‘transfer of training improvement strategies’ (Tziner, Haccoun, and Kadish, 1991) or as ‘post-training supplements´(Tews and Tracey, 2008). It is commonly defined as a series of methods oriented to facilitate positive transfer, as a set of guidance or procedures implemented after training to help transferability (Rahyuda, et al. 2014:421) or as “behavioral techniques relevant to specific trainee characteristics to enhance transferability” (Rahyuda et al, 2014:421). As seen, self-management can be named differently but its focus is on helping to transfer the acquired training content to the job. Self-management has been studied mainly as occurring after training and not during training. However, other approaches that include self-efficacy, studied it also before and during training. When self-management is studied in the early stages of training transfer, its definition integrates 59

a new dimension: self-management in the acquisition of knowledge, going beyond retention of knowledge and also focused on the way in which trainees acquired correctly the knowledge and learn through selfefficacy (cfr. Gist et al,1991: 837). Specifically, here self-management is linked to self-efficacy not only in the maintenance of the acquired complex skills or knowledge, but also in the acquisition of such an interpersonal skill (Gist et al, 1991:838). Self- efficacy plays a key role as being a judgment about tasksspecific capability and because it “subsumes effort expectancies along with considerations of tasks attributes, performance conditions, and ability estimates in a situation-specific judgement” (Gist et al, 1991:839). 2. The role of self-management in the theory of training transfer Luthans and Davis (1979) discuss the link between self-management and training transfer. In their definition, self-management is considered as a “deliberated regulation of stimulus cues, covert processes, and response consequences to achieve personal identified behavioral outcomes” (p.43). Wexley and Baldwin (1986) also highlight the importance of the environmental stimuli in trainees feeling and retention of training skills. Marx (1982), refers to slips situations where self-management can play a key role in identifying strategies for dealing with such situations. Slips situations are defined as situations in which the employee could ‘slip’ back into old habits and do not apply the training content. As seen, the regulations or strategies are deliberate and depend on environmental stimulus. For this reason, trainees need to learn self-management strategies and recognize when to use them. A self-management process is oriented to anticipate potential slips by observing past experiences and present situations that can affect the transferability (Noe, Sears, and Fullenkamp, 1990; Pattni et al, 2007; Rahyuda et al, 2014:422). This process is also in line with social cognitive theory, which states that, if they have a decent level of self-efficacy, individuals can control their behavior and increase the performance when they can handle it accordingly (Rahyuda, 2014 : 422 citing Bandura, 1986, 1999; Wood and Bandura, 1989). Because they understand this environment, individuals can transform and re-structure negative past experiences into more understandable cognitive symbols to overcome potential bad experiences (Zigarmi et al, 2009). Gist et al. (1991) identify some self-management techniques in training transfer: (1) problem identification (disturbances/interferences in the practice related to tasks); (2) coping strategies development (to deal with slips situations); (3) setting ad hoc goals (related to coping with slips situations); (4) progress goalmonitoring (goal achievement evaluation); (5) self-administering rewards/punishment on goal attainment (self-reinforcement methods to motivate accomplishment) (p, 842, 847; cfr. Frayne and Latham, 1987; F.H. Kanfer, 1980; Andrasik and Heimberg, 1981). These strategies are clearly related with the aim to develop and retain the acquired training content and coping with possible barriers to the applicability of such content to the job. Self-management strategies help trainees to anticipate, cope and deal with potential threats to the retention of knowledge and slips back into old habits. The goal of self-management strategies is to become part of a routine to keep motivation alive, maintaining a behavioral change and avoiding slips situations (Hutchins and Burke, 2006; Marx, 1982). Marx (1986), identifies seven steps in self-management as post-training intervention: (1) setting a skill maintenance goal; (2) define a slip and relapse; (3) define advantages/disadvantages of applying new skills; (4) discuss the transfer strategies both cognitive and behavioral; (5) forecast first slip; (6) develop coping skills, (7) monitoring progress. As seen, there are some commonalities with Gist et al. (1991) strategies of selfmanagement (see Table 6): 60

Table 6: Equivalences between Gist et al.’s strategies of self-management and Marx´s self-management steps Gist et al (1999) Problem identification

Marx (1986) Defining slip and relapse

Coping strategies development (to deal with slips Developing coping skills situations) Setting ad hoc goals (related to coping with slips Set skill maintenance goal situations) Progress goal-monitoring (goal achievement Monitoring progress evaluation) Self-administering rewards/punishment on goal Defining advantages/disadvantages of applying attainment (self-reinforcement methods to new skills motivate accomplishment) Source: authors These self-management steps or related strategies to coping with threats to the retention, can be better understood and unpacked when addressing the causes of such a process. A process will not be the same if it is triggered by different conditions and if some contexts are present and absent. In the next section we discuss the causes of a self-management process: relapse prevention combined with employee goal setting. 3. ‘Relapse prevention’ and ‘goal setting’ as the causal powers In the training transfer literature, relapse prevention and goal setting have mainly been studied as posttraining transfer interventions (Rahyuda et al, 2014; Brown, 2005; Burke and Hutchins, 2007; Gaudine and Saks, 2004; Salas and Cannon-Bowers, 2001). These factors seem to be especially relevant when trainees are aiming to successfully apply the skills and knowledge learned on the training at the workplace. For this reason, growing interest has been expressed in examining how to enhance the “employees’ confidence to apply the acquired skills and their inclination to attend future training” (Rahyuda et al, 2004 citing Berk, 2008 and Russ-Eft, 2002). Relapse prevention and goal setting emerged from social cognitive theory and their implementation has had important results for the efficacy of training and organizational performance (Rahyuda et al, 2014: 414: cfr. Brown and Warren, 2009: Gist, Stevens and Bavetta, 1991; Johnson et al, 2012). Structurally, both factors trigger processes that are related, but they can be developed as a single process or as two different processes (Rahyuda et al, 2014: 414). In addition, both factors trigger processes related to post-training methods to facilitate a positive transfer (Wexley and Baldwin, 1986). These methods are behavioral techniques related to the characteristics of the employee to enhance the transferability or (meta)cognitive strategies “that can help trainees to strengthen their awareness about the environment stimuli and use this stimulation to structure, understand, and manipulate their own cognitive process”(Rahyuda et al, 2014: 421, citing Tews and Tracey, 2008 and Wexley and Baldwin, 1986).

In the literature, relapse prevention and goal setting have been studied in isolation and in combination to explain an effective transfer. In regards to how relapse prevention affects training transfer, Hutchins and Burke-Smalley (2006) states that this approach “assumes that trainees who learn certain cognitivebehavioral coping skills will experience a heightened degree of self-efficacy toward transfer of training amidst the obstacles and high-risk situations pervasive in the work environment” (p. 11). Self-efficacy here is acting as an intervening variable. When individuals have certain degree of self-efficacy, they are more likely to avoid relapse, because its capability to make judgments about their competences to perform certain tasks (Bandura, 1982:126-129). Self-efficacy also is connected with enactive mastering (Bandura, 1982), because the success of the performance strengthens self-beliefs of capability (Hutchins and BurkeSmalley, 2006:10). Relapse prevention is most likely to cause training transfer effectiveness when learning and enactive mastering (related skills) combined with learning strategies to maintain or increase selfefficacy are also at play in the self-management process (cfr. Chiaburu and Marinova, 2005; Morin and Latham, 2000). Goal setting “deals with identifying a set of specific, challenging, and difficult goals to help individuals with expressing attention, organizing effort, increasing determination, motivation strategy development” to transfer and improve performance (Hutchins and Burke-Smalley, 2006:424; cfr. Latham and Locke, 2007). Goal setting can enhance expectations about the effects of using the skills acquired in the training and can lead to an increased effort to achieve the previously formulated goals (often in regards to applying the training). This leads to a better use and maintenance of skills when applying these on the job (Hutchins and Burke, 2007; Luthans and Jensen, 2002). Goals can be proximate (attainable in a fairly short time) ,remote (take longer to attain) or a combination of both (Brown and Latham, 2002). When individuals set up combined goals, they are more likely to improve their training transfer capabilities (Brown, 2005; Brown and Warren, 2009). There can be different types of goals. According to scholars, these goals can be oriented to learning, outcomes and can also be assigned (Brown and Latham, 2002; Morin and Latham, 200; Werner et al.,1994). When goals are oriented to learning, rather than outcomes, individuals can focus on understanding how to reach such a goal and feel a sense of urgency to master the training content in a satisfactory way. When oriented to outcomes, individuals can focus on reaching the ultimate goal independent of mastering the way to get there (Seijts et al.,2004). What goal someone sets for him or herself often depends on the characteristics of the training. Gist et al. (1991) interwove relapse prevention and goal setting to highlight that self-efficacy and increased performance work best when both conditions are acting together rather than isolated. In this research, as well, the cause of self-management intervention is theorized to be a conjunctural causal condition of relapse prevention AND goal setting. How a self-management process could lead to training transfer is described in the section below. 4. Self-management behavior In the self-management process triggered by relapse prevention and goal setting, we know that, due to a lack of mechanistic explanation (Rahyuda et al, 2014:421; cfr. Brown and McCracken, 2010; Hutchins and Burke, 2006), an extra effort is needed to fill in the gaps in the process of training transfer. We expect to fill these empty spaces of the process discussed in this section, by combining the process of relapse prevention AND goal setting in a single and coherent process, taking into account the empirical material gathered from the field, and the most relevant aspects discussed in the literature. 62

This process of self-management intervention would start in the learning process, with the direction for attention building block. Individuals identify some goals to help themselves with expressing attention to the implementation of the training. With these goals in mind, they would organize their efforts focused on the learning or performance goals that they wish to reach. Doing this increases feelings of determination. The second building block is mobilizing effort, where individuals feel motivated because they feel determined to reach their goal. Because of this motivation, they develop the best ways to achieve and maintain certain goals (setting the skills maintenance goal, based upon the training, and identifying potential risks of slips). The third building block is Pros & Cons generalization. By identifying potential threats to transfer, the trainee defines the advantages and disadvantages of using the skills at work in order to stay motivated. The fourth building block is coping with slips, where individuals discuss and learn certain strategies to overcome some kind of obstacles in applying the training content at work [anticipation]. The fifth building block is Networking. Individuals understand the difference between training and job context, so, in order to overcome this obstacle, they create a support network for transferability. The following building block is Slip prediction. Individuals predicts some kind of slips in transfer by monitoring past experiences of slip and relapse [anticipation] and the present environmental situations. This is followed by coping strategies building block, where individuals, based on coping methods, applies a threat coping strategy to this 'predicted slip' by selecting only appropriate steps to increase skills retention and generalization [e.g. applying skills in the appropriate setting, reducing interfering and unproductive emotions; retain self-confidence, diagnose support skills needed to maintain training, etc.]. Finally, the process ends with the monitoring and self-rewards building block where individuals monitor the process of skills transfer (self-monitoring if performance; self-evaluation against goal; self-reaction with self-efficacy) and create meaningful self-rewards for skill retention, leading to an effective training transfer. 5.1.4

Learner agency

1. What is learner agency? Learner agency can be understood as part of a process of high impact learning (Dochy and Segers, 2018) where the agency over learning and the decision-making authority is at learner hands (Dochy and Segers, 2018: 34-37). Dochy and Segers define learner agency as the “learner’s awareness of responsibility for his or her own L&D, and accordingly the proactive engagement in learning activities” (p. 34-37). Thus, learners take learning initiatives for their own L&D, managing the learning process by themselves, and choosing freely the different learning alternatives for a better performance. Learner agency also involves “selfregulation” (Gao and Zhang, 2011), which means that the learner is aware of his or her own capacity to learn, engaging in a self-management learning process to achieve the intended goals (Dochy and Segers, 2018: 34-37), because of his or her intrinsic motivation and goal-orientation. 2. The role of learner agency in the theory of training transfer

Dochy and Segers (2018) make the link between learner agency and training transfer in the high impact learning process clear. In their definition, when learners are empowered to make their own decisions, they feel intrinsically motivated to monitor, regulate and control their own learning in the task execution process. The feeling of ownership over their own activities and time investment enable the learners to improve their performances on the basis on their own interest. Learner agency is also interwoven with individual coaching (Dochy and Segers, 2018; Lazonder and Harmsen, 2016), where the support seems to play an important role. In this aspect, support within the process of learning agency is not necessarily a specific one, but it also can be according to the needs of learners. Providing an environment where the learner can act with enough autonomy and supportive coaching within a continuous feedback and reflection dynamic is a key aspect of this process (Sierens et al., 2009). Learning agency and transfer is also connected with self-regulation (Van Dinther et al., 2011). Learners with self-efficacy are more confident about their competences and therefore can engage in a self-regulated process of learning. When there is also an interest in the knowledge to be acquired, learners who perceive themselves as having mastered skills, are also more likely to develop self-regulated behaviors than others. This also means that the more relevant the knowledge or skills are for the learners, the more they will be capable to transfer to new situations. However, for this to happen, a shift towards intrinsic motivation is required (Dochy and Segers, 2018). 3. Sense of urgency as the causal power In the training transfer literature, sense of urgency acts as the starting point of learning process (Dochy and Segers, 2018). Individuals that are feeling sense of urgency will invest energy and efforts to learn (p. 28-30), explaining why they learn and creating a situation where they are committed to learn. Individuals feel the need to undertake activities, because they are in a state of maximum involvement and intrinsic motivation for learning. This may happen when there is an optimal balance between the learner’s task demands and his or her competences (Csikszentmihalyi and Beattie, 1979). Dochy and Segers (2018) states that for feeling a sense of urgency, individuals need to perceive a gap between what they can do and what they need to do later. There is a clear distinction between “can do” and “need to do” that make individuals invest energy on his or her own development. Similarly, this gap can also be perceived between the knowledge and its utility in the short-term. If certain knowledge is required in the short-term to solve tasks issues, individuals will also feel the sense of urgency to learn. For a sense of urgency to trigger a process, contexts are needed. Learning cannot be imposed (Dochy and Segers, 2018: 30-32), it needs to be guided by own motivation. From motivational theories, an empirical support for this is “self-determination” (cfr. Deci and Ryan, 2012). With a sense of self-determination, individuals can keep the flow of an intrinsic motivation for learning. This self-determination explains the individual’s needs for autonomy, relatedness and competence. Autonomy implies that individuals are in charge of own actions, and therefore undertake learning activities about what it is interesting for them, in what way and when is according to their own timing. Relatedness refers to connection with social network. Individuals may strive for secure and satisfying connections with others. Finally, competence involves the need to “understand how to attain the goals set and to be efficacious in undertaking the requisite actions to reach their goals” (Dochy and Segers, 2018). Learning as a free choice can also be complemented with working environmental contexts, that act as facilitators of self-regulation process (Boekaerts and Corno, 2005; Paris and Paris, 2001). Dochy and Segers 64

(2018) have identified four environmental characteristics. The first environmental characteristic is that tasks have a reasonable level of complexity, are beyond routine, the attention is focused on the tasks and curiosity is fueled. This can stimulate the search for solution. The second environmental characteristic is feelings of relevance which play a key role in the outcomes of learning. When reaching certain goals, that are clearly identified and perceived as relevant, individuals are more likely to keep motivation alive to reach such goals. Next, monitoring the process of self-regulated learning involves an “internal feedback” or reflection about how the skills are used during and after application in order to improve them (Schön, 1983). “Reflection-in-action” is described as crucial because it enables adaptability of the learner´s actions in real time with the support of effective feedback from others (Kluger and DeNisi, 1996). Finally, the last environmental context is a safe climate. This implies being oneself in a safe learning environment where mistakes can be committed as well as the strengths and weaknesses. All the characteristics of sense of urgency as a cause of learning process and the contexts taking place in learning agency can be interwoven in a process of learner agency to training transfer. 4. Learner agency behavior In learner agency triggered by sense of urgency, we know that a clear need to engage in training needs to be present, due to the identification of a hiatus between current knowledge and skill and required knowledge and skill in the future. Individual needs to understand that overcoming the hiatus is within reach of his/her own capabilities. With the presence of sense of urgency as a trigger, the process of learner agency would start before the learning process takes place, via perceiving gaps building block. Individuals perceive a gap between what they can do and need to do later at work. This means that individuals notice that extra skills could be necessary for future job performance. In parallel, a second building block takes place feeling of relevance, where individuals reflect (or make sense) about the relevance of the job's tasks requirements they are developing. Based on this reflection, individuals realize that she/he needs to update her/his knowledge for the required tasks and decides to invest energy in own learning process (state of flow building block). As part of this, individuals express expectations about what they will get from the learning process and set goals about their intended outcomes (goal setting building block). As a consequence, individuals enter in a state of maximum involvement driven by intrinsic motivation to engage in learning due to a balance between tasks demands and competences. Individuals make, therefore, a free choice by engaging in a training program of interest and focusing their attention on the tasks to be performed (free choice of learning building block). Guided by such tasks, individuals undertake learning activities related to ‘complex’ tasks (learning at hand building block) and actively reflects about the learning process: "how everything is going" to adapt their learning strategies and ensure the ultimate goal of transfer. Thus, by adapting learning strategies, individuals reach certain goals that are identified and perceived as relevant by themselves. This keep the motivation alive to reach the ultimate goals of transfer. Therefore, individuals undertake activities related to complex tasks to be applied to the job (internal feedback: reflection-in-action). In developing such activities, individuals identify certain strengths and weaknesses to improve themselves in the application of the learned content to the job by focusing on what 65

"to do better" (monitoring building block). However, in order to get an external view, individuals ask other peers for feedback as a way to evaluate the learning and application process objectively. Individuals get feedback from peers and freely adapt their activities to overcome some obstacles to transfer goals (network-feedback building block). Individuals look back on and think about their own actions, evaluating how the process and application went (feedback: reflection-on-action). Because of this self-evaluation, individuals identify and select steps to ensure transfer by focusing on applying the learned content in the appropriate setting and reducing possible interferences to transfer (Coping strategies building block). As a consequence, individuals reflect on the process of learning transfer and create meaningful self-rewards for skill/content retention (Monitoring and self-rewards building block). Finally, the content and skills learned during the training are applied on the job and maintained over time (routine).

5.2 CASE STUDIES: OPENING THE BLACK BOX TO UNDERSTAND TRAINING TRANSFER EFFECTIVENESS 5.2.1

Overview of successful cases

In Table 7 we display the main conditions and the set of cases being included in them. The table also shows whether or not the core contexts studied in this research were present. Table 7: Overview of grouped cases matching common conditions and contexts Cause/context

Training program as active learning method

Peer support and supervisor support

B2; K2

Supervisor support Peer support

Identical elements

Sense of urgency and relapse prevention setting goals Sense of urgency

Balanced workload

B2; K2;N2; B3 B1; K2; M1; D1; S2

B1; K2; M1; D1

Supervisor and relapse prevention setting goals Relapse prevention setting goals

Autonomy

N2; B3 N2;B3; V2; J3;T1

V2; J3; N2; B3 T1 W1

Source: authors. Note: - “does not matter”; ¬ “absent”

In order to study the four potential causal mechanisms, we focused on four cases where these mechanisms would potentially be present. These cases were: 1. N2 to study the causal mechanism signaling and retention triggered by supervisor support. 2. D1 to study the causal mechanism enhanced training transfer intervention triggered by peer support.

3. B3 to study the causal mechanisms self-management intervention triggered by relapse prevention -setting goals. 4. T1 to study the causal mechanism learner agency triggered by sense of urgency. Below, we will summarize how these mechanisms looked like in our cases. The complete mechanism, and how it looked like in our cases, is displayed in four road maps (see Annex 11) that have been created to study the evidence for each causal mechanism in its respective case. We have used a Bayesian-inspired, two-stage evidence evaluation framework (Beach and Pedersen, 2019) to assess whether there is mechanistic evidence confirming or disconfirming how these four causal mechanisms work. In each road map: • • • • •

We distinguish between theoretical and empirical evaluation of evidence We insert the collected observations to be assessed We evaluate the observations in terms of empirical uniqueness and trust We conclude whether the observation confirms/disconfirms(/does not say anything about) the presence of the part in the mechanisms (in terms of strong, moderate or weak). We conclude if the causal mechanisms were present or not in a particular case.

Each road map can be consulted in Annex 11 and additional evidence can be found in Annex 12. 5.2.2

Signaling and Retention triggered by supervisor support: The case of N2

Background information about the case, organization and training In order to discuss the ‘signaling and retention’ mechanism, triggered by supervisor support, we will take a closer look at our case N2. N2 works at a medium-sized company that provides digital signage at events or points-of-sale to companies. It’s a company that aims to have a modern-day approach to modern-day problems. Because, they were also enrolled in another ESF-project (“Organizing Differently”) it was difficult to apply a traditional leadership training. N2 has followed a training that was focused on “Compassionate Communication”. The goal of this training was to get employees to be able to “connect” better with employees, which would allow them to communicate more effectively. Everybody in the firm had to take this training. They were separated into groups based on hierarchy in the firm (so the “supervisors” would not be mixed with the staff members). The training was based on cases that the participants had contributed. They would often focus on difficult conversations with staff members. When we talked to her, she mentioned on several occasions that her supervisor, and the whole management team by extension, were driving forces behind the project. Because N2 is part of the HR team, she needed to use the training often and was also involved in the general implementation of the training in the organization. We have interviewed two participants at the firm. They both agreed that the organizational climate was also something that facilitated the transfer. Because everyone had taken the training, they all seemed to be on the same page. Most people in the organization were open to this “new” way of compassionate communicating. When reconstructing the causal mechanism through which N2 transferred the training, we mainly focused on the interview that we have conducted with her. Before this interview, we told her that the results would be treated anonymously and that her answers would solely be used for scientific purposes and would not affect her or the firm she works at. We have no indications that she was not telling the truth or omitted important things. We have included the theorized necessary contextual conditions in Table 8. 67

Table 8: Contextual conditions necessary for signaling and retention process

Contexts related to supervisor support

Theoretical level

Empirically observable manifestations

Identical elements: This enables the individual team members to transfer the training to the work floor.

Team members recognize the situations discussed during the training or can suggest cases themselves.

Autonomy: The employee needs autonomy in order to select the most appropriate way for her to implement the training Training program as active learning instructional method: A more active, engaging, instructional method will be essential to be able to successfully transfer the training content from the classroom to the workfloor. Non-workload: The employee needs to have time to apply the training. If the workload is too high, the trainee cannot devote enough attention and energy to implement the training.

Team member acknowledges that he or she has autonomy.

The training should involve a lot of interactions such as role-playing games, discussions on specific cases etc.

In the survey, the answers also reflect on workload. The answers on these questions should reflect this.

Source: authors. What worked for N2? N2 experienced support from her supervisor during implementing the training. This became clear during the interview we had with her and when she filled in our survey (see observations C1 road map A11.1 Roadmap Signaling and Retention Causal Mechanism triggered by supervisor support in Annex 11). Her supervisor, being the head of HR, also ascribed significant importance to the training (observations P1 and P2). She noted that he was one of the driving forces behind the project, making sure that all employees followed the training. It was mandatory to follow the training for everybody (observations P3a). Of course, just because it is mandatory does not mean that it is a negative experience. She mentions that, for her, it did not feel mandatory. In part because following this training was mandatory for all employees, they all followed the training in group. One formal way in which they would stimulate participation was to make sure that, while attending, employees did not need to worry about the tasks that would need to be done at the company by making sure someone else was taking over their workload (observations P3b). This facilitated the learning environment. Not only N2 noted this, a colleague of hers confirmed this in another interview. This allowed them to fully focus on the training during the training, instead of worrying about running behind on schedule. We expect that following this training together with other colleagues, and the engagement of the supervisor, creates an organizational climate in which the employee perceives the training to be important for the job (observations IO). During the whole talk, N2 stressed the importance of this organizational climate, acknowledging that she may be a bit biased because of her background in HR. It’s also clear that she recognizes the importance the supervisor attaches to the training. N2 also explicitly mentions that following the training has “been good”, illustrating the relevance of the training. All of these things combined have probably led to our case, N2, to feel motivated to generalize the training. She discusses how the involvement of supervisors causes a lot of enthusiasm and positivity around the training (observation P4). This positive feeling and perceived importance of the training makes her try out the 68

training (observation P5). Although application is not always smooth at the start, this motivation encourages her to keep on trying out the training. Because, after a while, employees may slip back into old habits, we expect that reminders by the supervisor could be beneficial, and sometimes necessary, for successful training transfer (observation P6 iv). N2 discusses several instances where she received feedback from her supervisor on her application of the training (observations P6 i, ii and iii). Because he was often present when she applied the training (e.g. during difficult talks with employees), he was able to assist her in using the training and reminding her of things that she could do differently next time. She also mentions that she, in cooperation with her supervisor, tries to keep the training alive (observation P7). So, not only is she reminded by her supervisor to apply the training, she also reminds other to apply the training, which in turn, could help herself as well. Because of this cycle of trying out the training and receiving feedback, the application of the training becomes natural. N2 mentions on several occasions that they want to “keep this alive”. She is thus often reminded of the training. Over time, this results in her automatically using what she has learned in the training. We then consider the training to be successfully transferred (Observation O1). This successful application can also be seen in her answers in the survey. Conclusions: This case illustrates the importance of supervisor support and how it can contribute to successful training transfer. It highlights the importance of “keeping the training alive” in which the supervisor can play an essential role. By providing feedback and reminding employees to keep on using the training, the chance of successful training transfer can be increased. The importance of the organizational culture is also demonstrated in this case. Having all employees following the training together and enabling them to fully focus on the training leads to a shared understanding of the training, which leads to more interaction and positive feelings around the training. The supervisor can also play a role in this process by also engaging with the employees on the training and reminding them to use the training. 5.2.3

Enhanced learning transfer intervention triggered by peer support: The case of D1

Background information about the case, organization and training D1 is the HR manager of a production company. The “mother company” is a large, German, multinational that is not strongly involved in the day-to-day operations at the Plant where she works. She also said that the organizational structure was quite “flat” or horizontal. There is no “strict” hierarchy and she considers the plant manager more as another colleague, than as a controlling supervisor. She followed a training on “coaching leadership”, in which supervisors shifted towards giving more responsibilities and autonomy to employees. In this course, there was also significant attention to communication. This “new” approach also fitted in the goals the organization has set in striving towards a new organizational approach in another ESF-call (Anders Organiseren). Because they are a production company, everything else but production comes second. So, when something happens in the production hall, this gets priority and certain meetings get cancelled. This sometimes makes it more difficult to implement the training (you are unable to discuss the cases with peers). This was also one of the reasons why they did the training on an external location. Had it been at the company, people would be bothered too often and would sometimes, during breaks, do other things. This would diminish their focus during the training. She mentioned that the high workload was something that sometimes made it difficult to apply the training.

After the training, she clearly experienced support from her peers. She talked about “intervision moments” in which she would discuss certain cases with her colleagues. She also mentioned that the perceived positive results (less resistance from employees, support for it among employees) were something that stimulated the application of the training. When reconstructing the causal mechanism on peer support for D1, we largely draw on the interview that we had with her. Before this interview, we told her that the results would be treated anonymously and that her answers would solely be used for scientific purposes and would not affect her or the firm she works at. We have no indications that she was not telling the truth or omitted important things. We have included the theorized necessary contextual conditions in Table 9.

Table 9: Contextual conditions necessary for an enhanced learning transfer intervention

Contexts related to peer support

Theoretical level

Empirically observable manifestations

Identical elements: This enables the individual team members to transfer the training to the work floor.

Team members recognize the situations discussed during the training or can suggest cases themselves. Team member acknowledges that he or she has autonomy.

The training should involve a lot of interactions such as role-playing games, discussions on specific cases etc. In the survey, the answers also reflect on workload. The answers on these questions should reflect this.

Source: authors. What worked for D1? We believe that support from her peers played an important role in transferring the training for D1. During the interview, she mentions on a couple of occasions that she experiences peer support. This is also reflected in the survey that she completed three months after the training (see observations C1 road map A11.2 Roadmap Enhanced learning transfer intervention triggered by peer supportin Annex 11). D1 and her peers all followed the training together (see observation P1a). It was the company policy that everyone needed to follow the training, so that this was guaranteed. This was also very important for the company. If one person could not attend the training, they would change the date for everybody. D1 mentions that, during this training, everybody actively participated and practiced the content of the training (see observation P1b). There were plenty of discussions and interaction during the training. By all following the training together and actively participating, peers discuss their different views of application of the training (see observation P2). In our case, this was illustrated by the differences between managers who work in production and those who work elsewhere. Given that they work with different profiles and are often confronted with different problems, they generally approached situations differently. These 70

discussions and following the training together lead to a common understanding of the training and how it should be applied in certain cases (see observations IO). Next, having achieved a common understanding of the training, we find evidence of peers planning (see observations part 3) and agreeing to meet each other (see observations P4a and P4b) during intervision moments. It appears that this was not an explicit part of the training, but happened spontaneously and could reflect organizational practice. During these intervision moments they discuss the training and its application. She also told us that it was not only during these moments that they discussed the training, but that they also sometimes discussed the training outside of these meetings. They are able to share experiences during the training and ask each other for clarification on how to apply the training ( see observations P5). Presented with issues, the peers start brainstorming in group on how to handle these issues (see observations P6). Based on this, they offer recommendations to how the employee could handle certain situations (see observations P7). After receiving this advice, we expect that our case considers the suggestions made by her peers (see observations P8). Having a new outlook on how to deal with certain issues, the employee feels stimulated to use the training and try out the suggestions (see observation P9). She argues that it is just “more fun” when you’re on the same page. This could be explained by the absence of conflicts. Because of this stimulation, the employee applies the training content (observations P10). After applying the training, the employee discusses the application of the training, possibly during one of the intervision moments, but also outside of these moments (observation P11). Once again, she receives feedback on the application of the training, improving how she utilizes the training. With these newly acquired insights, the employee uses the training again. Through this iterative process, our case keeps on improving her skills in regard to the training content until they are fully incorporated into her standard way of working (observation 12). The result of this is that the employee has successfully transferred the training (observations O). This successful application can be deduced from the answers she gave on the survey as well as from the interview. Conclusions: The interview clearly demonstrates how peer support can contribute to training transfer. For our case, feedback and conversations were vital to further improve the application of the training. The intervision moments that were organized proved to be essential for this. These meetings allowed the peers to focus on the training and how they could apply it. It would almost “force” peers to have contact with each other and discuss the training. Organizing these types of meeting could thus be very beneficial for application of the training and could stimulate adaptation. By following the training together with other colleagues, there is a shared understanding of what the training is about, which is essential if one wants feedback from peers. It also creates a group dynamic which stimulates contact between peers. Finally, our case also mentioned the importance of a manageable workload. If there is no time left to think about the training and how to apply it, chances are that people will fall back into old habits. 5.2.4

Self-management intervention triggered by relapse prevention and goal setting: The case of B3

Background information about the case, organization and training In this section, we will discuss how the ‘self-management intervention’ mechanism took place in our case B3. B3 works as a warehouse manager for the same company as N2, who we will also discuss later. As such, both cases have a lot in common. He followed a training on “Compassionate Communication”. In order not 71

to repeat ourselves, we refer to her description in section 5.3.1 for a more elaborate description of the company and training. As a warehouse manager, B3 often interacts with his team of blue-collar workers. Because they often have to perform their tasks as a team, communication is crucial. B3 mentioned that, before they followed the training, there regularly were conflicts between employees. According to B3, this has improved after the training. He acknowledged that this, to a large degree, was attributable to him and the other employees following the training. As with N2, we mainly focused on the interview when reconstructing the causal mechanism that allowed B3 to transfer the training. Again, we told the respondent that the results would be treated anonymously and that the provided answers would solely be used for scientific purposes and would not affect him or the firm he works at. There are no indications that he was not telling the truth or omitted important things. We have included the theorized necessary contextual conditions in Table 10. Table 10: Contextual conditions necessary for self-management intervention

Contexts related to selfmanagement

Theoretical level

Empirically observable manifestations

Identical elements: This enables the individual team members to transfer the training to the work floor.

Team members recognize the situations discussed during the training or can suggest cases themselves.

Autonomy: The employee needs autonomy in order to select the most appropriate way for her to implement the training Training program using an active learning instructional method: A more active, engaging, instructional method will be essential to be able to successfully transfer the training content from the classroom to the workfloor. Non-workload: The employee needs to have time to apply the training. If the workload is too high, the trainee cannot devote enough attention and energy to implement the training.

Team member acknowledges that he or she has autonomy.

Source: authors. What worked for B3? In the survey that B3 filled in after the training, he first acknowledges that he set a training goal and then describes this goal as applying the training (see observation C1 in road map A11.3 Roadmap Selfmanagement intervention inAnnex 11). This shows that employee goal setting was present in our case. He also mentioned how implementing this training would help him do his job better. This signals the importance of implementing the training for his job and makes him attach more importance and attention to the implementation (see Part 1 – hereafter P1). This leads the employee to focus his actions in order to be able to reach certain learning or performance goals (see P2). B3 is motivated to apply the training and develops several ways to achieve his goals. He notes that “you need to keep your goal in mind” and that you need to “keep on asking questions” (see P3). Next, the trainee identifies potential threats to apply the training (see P4). This also leads him to consider alternative scenarios and makes him define the advantages and disadvantages of using the training. In our 72

case, B3 mentioned, on several occasions, that “applying the training too artificially” would be something that hinders the implementation. He also considers what would happen if he applied the training, imagining the consequences, suggesting that he defined advantages and disadvantages of using the training. With a clear view on these potential obstacles, the trainee discusses and learns about certain methods or tools to overcome these obstacles (see P5a). One obvious method of overcoming the obstacle of applying the training “too artificially” was just to use the training regularly and learn through “trial and error”. We also observe that B3 understands the differences between the training and job context and, in parallel with discussing these obstacles, creates a support network that will assist him in transferring the training to the work floor (see P5b). One of the main differences that he discusses is the difference in how much time there is available to reflect on certain situations. In training, trainees are presented with a situation and get time to reflect on the most appropriate solution, often in group. In practice, employees need to react almost instantly and cannot always draw on the advice of others. Hence, if implementing the training does not come naturally (instead of feeling artificial), it will be more difficult to adequately implement the training content. Having identified these obstacles, the trainee predicts potential slips based on previous experiences (see P6). This allows him to anticipate such situations. Our case mentioned, both in the survey and during the interview, that reacting too impulsively in a situation could be something that obstructed him from applying the training. However, because our case has identified potential slips, he was also able to apply a coping strategy. The training allowed him to, after a conflict had arisen, go back to the employee later and talk about the issues at hand (see P7). All of this leads to the employee monitoring his own performance and creates self-rewards (p8). For B3 this self-reward is simply the successful application of the training in itself. During the interview (and in the survey), it was clear that he enjoyed learning new skills. He also discusses several things that he did that helped him in implementing the training. Paying this much attention to the implementation of the training resulted in the successful application of the training for B3 (see O1). This successful application can be seen in his answers in the survey. Conclusions: This case illustrates how employee goal setting and relapse prevention can contribute to successful training transfer via a self-management intervention mechanism. Although B3 indicated in the survey that he had also experienced supervisor support, during the interview, it became clear that supervisor support was not the most important dimension that triggered a training transfer. When asked if his supervisor helped him to apply the training content, he replied that that was not really the case, but that he did not think that was necessary. This self-management mechanism shows that it pays off to consider and identify potential obstacles to implement the training. By considering these beforehand, the trainee can adapt and prepare him or herself for these potential issues. By carefully reflecting on these obstacles beforehand, the chance of successful application can thus be increased. 5.2.5

Learner agency triggered by sense of urgency: The case of T1

Background information about the case, organization and training In this section, we will discuss how the ‘learner agency’ mechanism may have taken place in our case T1. T1 works at a medium-sized hospital as a personnel manager. In his job, he manages and supports the head nurses. The course that he followed was on leadership. The training was held during a two-day course (off73

site) in which multiple modules were taught. As such, the training consisted of a broad spectrum of skills that could be used. When we interviewed T1, he mentioned that how easy it was to implement the skills also depended on the specific training content. For example, they received some guidelines to apply during evaluation meetings with staff members. T1 argued that it was fairly easy to use these tips because there were clear opportunities on when to use those recommendations. Because of this, he was sort of “forced” to use it and he knew when to try it out. He also said that the training gave him good, usable tools to use. What they learnt in class was close (enough) to how it was on the workfloor. When we were looking at how sense of urgency could have contributed to training transfer, we mostly focused on the interview that we have performed with T1. Again, we told the respondent that the results would be treated anonymously and that the provided answers would solely be used for scientific purposes and would not affect him or the firm he works at. There are no indications that he was not telling the truth or omitted important things during this interview. We have included the theorized necessary contextual conditions in Table 11. Table 11: Contextual conditions necessary for learner agency process

Contexts related to learner agency

Theoretical level

Empirically observable manifestations

Identical elements: This enables the individual team members to transfer the training to the work floor. Autonomy: The employee needs autonomy in order to select the most appropriate way for her to implement the training Training program as active learning instructional method: A more active, engaging, instructional method will be essential to be able to successfully transfer the training content from the classroom to the workfloor.

Team members recognize the situations discussed during the training or can suggest cases themselves. Team member acknowledges that he or she has autonomy.

The training should involve a lot of interactions such as role-playing games, discussions on specific cases etc.

Source: authors. What worked for T1? There were relatively few cases who experienced sense of urgency. According to our revised operationalization, only seven out of fifty-one cases experienced sense of urgency. Four of these were not able to transfer their training. Out of the three remaining cases, T1 was selected because he followed a training on leadership skills, and he was also retained in the QCA analyses. Although B3 also followed a course on leadership, T1 had less favorable conditions in general so that we expected that it would be more likely that we would be able to uncover a mechanism related to learner agency in this case. However, in contrast to the other mechanisms, there were fewer clear signs that the mechanism that we have theorized was present in our selected case (see road map 11.4 Annex 11. We also looked at the data that we gathered on B3, but this also did not seem to lead to a strong confirmation of the presence of the mechanism. The answers of T1 in the survey revealed that there was some sense of urgency. He agreed with the statement that he “identified a gap between what I could do and what I would be doing in the future”. Furthermore, he was also confident that overcoming this gap lay within his capabilities. This leads us to 74

conclude that T1 experienced a sense of urgency. During the interview, on several occasions, there are moments that make us question whether or not an impact of sense of urgency actually took place. For instance, T1 mentions that he would have been able to do his work without the additional training, albeit not as easily. In the survey, he also did not agree with the statement that he was confronted with a challenge or work-related problem that needed to be solved urgently. As such, the sense of urgency may not have been as clear in this case as we would have hoped. We also expected that the trainee would participate in the training by his own choice. During the interview, the trainee acknowledges that he participated in the training because “it is expected of him”. He also mentions positive experiences with trainings in the past. But he does not clearly indicate that he followed the training because there was an issue that he needed to address. When it comes to feedback from peers or supervisors, the trainee seems to experience a lack of support. He says that not receiving feedback from his supervisor was something that impeded the training transfer. Conclusions: All in all, this evidence does not enable to make inferences about the presence/absence of the mechanism in our case. In an ideal setting, we would have additional evidence on the other parts of the mechanism, even if we already have ample evidence that the mechanism is not present in this case. However, because of the Covid-19 pandemic, we decided not to contact this hospital again with a request for additional evidence.

5.3 CONCLUDING REMARKS With QCA we could explain why training transfer is effective in some cases and not in others. However, we were in the dark with respect to the transfer process that takes place in each case. In this section, we can understand how training was effectively transferred, particularly in these four single cases. We chose one case per mechanisms (process) to understand specificities, but also, we assume that each mechanism is also present in those positive cases members of the condition(s) or combination of conditions, because they are typical cases. There are four mechanisms that enable us to understand the training transfer in four cases: •

•

The first causal mechanism is ‘signaling and retention’. The process was triggered by supervisor support. The particular case studied was N2 and it is the only one where all the evidence is robust. The role played by supervisor support was crucial to transfer, mainly in the stages of ‘keeping the training alive’ where supervisor provided observations and reminded trainees to keep on using the training content to the job context. The key context related to the process was the organizational culture of enable to employee to follow the training together. This led to a better interaction and positive climate to share visions around the training. The second process is ‘enhanced learning transfer intervention’. This process was triggered by peer support. The case studied is D1, where each part of the process was present, and most of the evidence is robust. We can confirm that the role of peers in transfer is a key aspect to take into account. Training transfer becomes better when peers provided feedback to others to improve performance, or when intervision moments were organized to discuss the challenges and achievement in transfer goals. Intervision facilitated to be focused on the training and in the content application, and the fact to have followed the training together with other colleagues enabled more commitment to a shared understanding of what the training was. 75

•

The third process is ‘self-management intervention’. This mechanism was successfully present in B3. We think that the macro condition relapse prevention and goal setting played a key role in triggering a process where the individuals had previous stimulus or motivation and capacity to engage in learning process, and where self-efficacy was a key context related to the process itself - to increase the performance. Each part of the mechanism was present in this case (see Table 12), some of them were more moderate in the quality of the evidence gathered, other were more robust. However, the conclusion is that B3 engaged in a self-management process where he/she was capable to cope with barriers to generalization and achieve the transfer goals. Finally, the last process is ‘learner agency’. This process was triggered by sense of urgency. We focus on T1 as case, because was the only one that had membership in the conditions of sense of urgency and relapse prevention. It seems the T1 has a high self-efficacy and self-determination to coping with barriers to transfer and engage in learning process as free choice. In T1 however, we could not confirm nor evaluate the presence/absence of each part of the process, because the lack of information. Further process of data collection was not possible due to the global context of covid19, however, with the data gathered, it seemed that the process did not work as expected. This does not enable to make inferences about the presence/absence of the mechanism in our case, but at least we can reflect about what worked: Part 9 was present in this case with strong evidence. It seems T1 received feedback from peers and was able to adapt own learning activities to overcome with the barriers to transfer. However, as already said, we cannot make inferences of this case due to the lack of further observations. Table 12: Process of training transfer in four in-depth case studies Signaling and retention N2

Enhanced learning transfer intervention D1

Confirming

No inferences

Confirming

Part 1b

Confirming

Part 2

Confirming

Confirming (moderate) Confirming (moderate)

Confirming (weakly) Disconfirming

Confirming

Part 3/3a

Confirming

Part 3b Intermediate outcome

Confirming

Confirming (moderate) -

Confirming (weakly) -

Confirming

Part 4/4a

Confirming

Part 4b Part 5/5a

Confirming

Confirming (moderate) Confirming (moderate)

Part/CM Case CAUSE Intermediate outcome Part 1/1a

Intermediate outcome

Self-management

Learner agency

T1 Confirming (moderate)

Confirming (moderate) Confirming

No inferences No inferences

Part 5b

Part 6

Confirming

Part 7

Confirming

Part 8

Confirming

Part 9 Part 10 Part 11

Confirming Confirming Confirming (moderate)

Confirming (moderate) Confirming (moderate) Confirming (moderate) Confirming (moderate) -

Part 12

Confirming

Part 13 Part 14 OUTCOME

Confirming

No inferences No inferences No inferences Confirming No inferences No inferences Confirming (weakly) Confirming

Source: authors.

6. CONCLUSIONS In this section we summarize the main results of the comparative study with QCA and the in-depth study with process-tracing to explain and understand training transfer effectiveness in Flemish firms.

6.1 EXPLAINING TRAINING TRANSFER IN FLEMISH FIRMS The results of this comparative evaluation with QCA method are robust, having high coverage (80% ) and consistency (100% - see Annex 9 for details). There are no combinations of conditions that explain most of the successful case studies. Diversity is high, but some conclusions can be drawn for some groups of cases by considering the role of certain contexts and causes across cases. Training program as an active learning method is a high impact context for training transfer effectiveness ‘Active learning’ refers to learning method with active trainee engagement through meaningful practice and reflection on what has been learned and encountered (Dochy and Seger, 2018; Prince, 2004; Dewey, 1938). Dochy and Segers (2018) already noticed the importance of active learning in high impact learning, proposing as building block ‘action’ and ‘sharing’. Individuals engaged in active learning methods are capable to have their learning process at their own hands, regulating their own learning experiences. The findings of this QCA study strongly confirm the relevance of the role of training program as active learning method in eleven out of twelve cases successful cases of transfer. It is not a necessary context, though. However, it is a core context for most of cases. We can conclude, therefore that many may equate training transfer effectiveness with whether a training program is designed as one of active learning or not. Therefore, this contextual condition would need to be absolutely included in future training design in Flemish firms. Sense of urgency is irrelevant for successful training transfer when training is “mandatory”. The role of sense of urgency (understood in our study to be actors engaged in training because of the identification of a hiatus between current knowledge/skill and the required knowledge/skill in the future, with the understanding that overcoming the hiatus is within reach of the capabilities of the employee (Dochy and Segers, 2018)) in transferring the training came across as limited in our analysis. This was unsurprising as the training program was mandatory for most of the trainees. For example, only three out of twelve cases had a sense of urgency to participate in the training, probably because the motivation as a state of maximum involvement and intrinsic motivation was the result of a balance between task demands and competences (Dochy and Segers, 2018). This is something to think about for further training programs. A balance between tasks demands and competences would need to be evaluated by the organization in order to offer more adjusted training programs to the employees. Support plays a moderate role in transfer, but cannot easily drive the process to transfer Surprisingly, our study shows that Flemish firms are not necessarily characterized for having ‘support to training transfer’ strongly developed in their organization training policy. Our analysis treated support as ‘peer support’ or ‘supervisor support’, both independently. As peer support we understood the colleague’s commitment for employees to improve the trainee’s learned content and stimulate the trainee’s use of learned material to the job” (Reinhold et.al, 2018; Chauhan et al, 2016; Nijman, 2004; Russ Eft, 2002; Noe, 78

1986). As supervisor support, we understood the superior’s commitment to facilitate the retention and motivate the use of the acquired content in a training to the job by employees, during and after a training program takes place (Govaerts, 2017; Lancaster et al 2013; Nijman, 2006; Cromwell, 2004; Richman-Hirsch, 2001; Salas & Cannon-Bowers, 2001; Holton 1997; Quinones, Ford, Sego, & Smith, 1995). Only two cases out of twelve experienced support by peers and supervisors. Six cases out twelve experienced only peer support and four out twelve experienced supervisor support. Far from expected, support did not play a key role in the successful cases of transfer. This is contrary to what was raised in theory of training transfer. Relapse prevention and setting goal can be moderately influential, but only under certain conditions and if training program as active learning method and autonomy are present. Relapse prevention and setting goals are important factors to lead to transfer. Our study suggests that such factors acting together can be moderately influential, but within certain limits. First, the context of training program as active learning method needs to be present and combined with autonomy. Moreover, support from peers or supervisor do not need to be present. The analysis indicates that coping with slips and setting goals in parallel can make the difference when some kind of support is absent. In addition, this may occur when the training design is one that enables active learning and the work environment provides opportunities to make decisions (Botke et al, 2018). It is important to note that five out twelve cases experienced this situation. For five trainees, the work environment and the training design facilitates coping with barriers to transfer even when support was weak or absent.

6.2 UNDERSTANDING HOW TRAINING TRANSFER TAKES PLACE IN FLEMISH FIRMS The results of this in-depth evaluation in the four cases studied, provide us some lessons about how the training transfer process took place in such cases (see Annex 11 and 12 for details). As said earlier, four processes were studied, three of them were present and one was not evaluated since it was not possible to make inferences. Contexts played a key role for the well-functioning of the process. Even if single cases were studied, some conclusions can be drawn for such cases by considering what worked and what did not work. “Keeping the training alive” : How the ‘right message’ can make a difference. The employees that receive support from their supervisor can better transfer when the supervisor ascribes importance to the training, reminds employees to use it and provides feedback. Keeping the training “alive" is key in the retention and application of the training content, mainly when the workload is also balanced. When perceiving the training as relevant for their job, employees can feel motivated to use it and discuss its content with peers, creating also an environment of trust. The latter leads to post-training evaluation feedback for a better transfer. For the particular case of N2, four general contexts made the process succeed: identical elements, training program as active learning process, autonomy and a balanced workload. The employee needs to gain some hands-on experience during the training, it should therefore be designed as one of active learning. Similarly, the training design would need to be equivalent to the one in which the trainee needs to use the skills that need to be transferred (identical elements). We have seen that the supervisor ascribes importance to the training and communicates this message to the trainees. As already mentioned, the workload was balanced and the autonomy to select the most appropriate tasks related to the training content was also relevant 79

for transfer purposes. To stimulate this, the supervisor should manage the workload so the trainee can also focus on learning. Finally, the ways in which the supervisor increased the perceived relevance of the training for the job were crucial microprocesses that facilitated the mechanisms of signaling and retention. Doing the training together within a flat organizational structure and having intervision moments were key aspects of training transfer intervention process. When everybody is on the ‘same page’, things go better. Individuals following the training together are more likely to go through the process of enhanced training transfer than those who do not follow the training with colleagues. We have seen that this happens within a flat organizational structure that enables individuals to feel more comfortable with peers and engage with less effort in similar activities, such as communication, sharing different views, getting common understanding and engage in coaching activities to discuss and share experiences about the application of the training content. When engaging in communication with peers, employees improve their knowledge on the topic and increase mutual trust. Trust is a context specifically related to this process that enables individuals to share their views with others and valorize the learning moments spent together. For the particular case of D1, four general contexts made the process succeed: identical elements, training program as active learning process, autonomy and a balanced workload. When a training program mirrors the real life challenges of employees and offers useable tools to enable transfer (identical elements), trainees are more capable to transfer the learned content to the job. In this case, the training program enabled individuals to participate together in the activity, increasing group cohesion. Similarly, when the training program is oriented to active learning, employees can support learning activities by discussing their experiences with the training with peers. These coaching moments required time and the balanced workload enabled employees to enjoy subsequent intervision activities. Finally, autonomy facilitated transfer because employees could be focused on learning experiences, together, according to their needs. Determination and self-efficacy are key aspects for a well-functioning of the process of self-management The trainees can better organize themselves when they feel determined to reach their goal. This happens within an environment where trainees can set learning goals and organize their effort accordingly. When feeling determined, employees can feel motivated to maintain the learning and retention goals by applying diverse coping strategies, developing a network, and monitoring the evolution. It is important to highlight that trainees need to recognize the importance of transfer to retain the training content and skills and achieve the goals. The relevance of the training is the self-reward gained by the trainee itself. For the particular case of B3, four general contexts made the process succeed: identical elements, training program as active learning process, autonomy and a balanced workload. When the training program is one of active learning, individuals can grow accustomed to the activities and trainings skills that they will need to use. However, training situations need to be similar to real life situations to enable transfer (identical elements). Otherwise, there would be a disequilibrium between what is offered and what is feasible to achieve by trainees. A balanced workload enables individuals to be focused on learning experiences and enjoying autonomy allows them to better organize their time. Finally, self-determination and self-efficacy are crucial individual factors that act as facilitators of the whole self-management process. The process of learner agency cannot work when training is mandatory.

Learning activities are not always a consequence of learner agency. It is sensitive to the context of whether the training was mandatory or voluntary. Learner agency works better when the decision to learn is voluntary (Dochy and Segers, 2018), because there is an intrinsic motivation to learn. We have seen that in most of the cases the training was mandatory. In the particular case of T1, even if it was mandatory, the trainee mentioned its importance and interest in pursuing it However, learner agency cannot work properly, because the initial forces were not a “real” gap or challenge identified as relevant for the job tasks. Although the training content improved his/her capabilities and job performance, the trainee mentioned several times that he/she could do his/her job without the training. We see that intrinsic motivation produced by a sense of urgency is not what triggered the process of learner agency in the case of T1. For this reason, the learner agency process broke down. Three contexts were present in this case: identical elements, autonomy and training program as active learning method. The workload was unbalanced, though. Even when three contexts were present, they did not enable the well-functioning of the process, because of the already mentioned lack of contextual conditions related to the microprocess: the free choice for learning as the original facilitator of learner agency.

7. RECOMMENDATIONS The OiB-call aimed to contribute to an increase in the number of Flemish workers in training. The ultimate goal of such an increase is to support and encourage lifelong learning in companies and organizations. However, in the call, there was little attention for how these trainings would actually be implemented and how their effectiveness would be maximized. This evaluation study focused on whether or not following the training would lead to a successful training outcome, defined here as a durable transfer of the training content to the job context. We investigated what and how conditions can trigger successful training transfer. As discussed in section 3, only about 30% of our cases actually experienced effective training transfer, leaving ample room for improvement.

7.1 RECOMMENDATIONS FOR ESF PROJECT MANAGERS By incorporating guidelines on these conditions and the necessary contexts, future calls could improve effective training transfer, ultimately leading to increased benefits of these trainings. ESF project managers could ask organizations that want to participate in the call to reflect on how they will support their trainees to transfer the training in their application proposal. They could also ask more specific questions such as “how will you, as an organization, ensure that trainees receive a sufficient amount of peer support during and after the training so that they can transfer the training to the job context?”. Other questions could be: • • • •

7.2 PRACTICAL RECOMMENDATIONS FOR STAKEHOLDERS Organizations, as well, have many tools available to create conditions that are favorable to training transfer and will directly benefit from utilizing these tools. Anticipating the training and ensuring that supportive 82

conditions are present before the training starts will stimulate overall training transfer. Below, we provide specific recommendations for stakeholders and for further research. Understand that ‘mandatory trainings’ are not effective. Organizations need to focus on making the relevance of training visible to employees. Our study recommends that it is important to start with communicating the added value of a given training. Employees need to understand why the training is relevant and how it can be used in job-related tasks. Similarly, organizations need to facilitate the relevance of the training by balancing the workload (ensuring that it is not too high, nor too low) and providing some sort of autonomy. Employees should be able to work on their own learning process when having the time and understanding the relevance of training content for their further performance. This could help employees to feel motivated to engage in a certain training program and avoid falling back into old habits when transferring training. In future training initiatives, organizations could be focused on how to motivate employees to engage in learning activities, by illustrating its relevance in their careers. Employees have their own views on how to respond to changing demands in the job contexts. Training program design should consider these views. Pay attention to the implementation of meetings between peers during and after training. Plenty of employees who experienced ‘peer support’ noted that they would regularly meet with peers and discuss the (application of the) training. Although it is important to have informal contact as well, stimulating the organization of formal moments to discuss it may be helpful. Some participants noted that they would rarely get to interact with those colleagues with whom they followed the training, which they regretted. Our findings suggest that quick informal ad hoc meetings with peers are not enough to transfer: employees need to share their experiences, frustrations, feelings, get feedback, attention and be encouraged by others on regular occasions. This implies that organizations need to pay attention to formal meetings between peers during and after training. Organizations could support their employees in organizing these intervision moments. In the successful case of peer support triggering an enhanced transfer intervention mechanism, the feedback or coaching provided by colleagues kept the motivation alive. Interaction of this nature has proved to be a powerful building block for successful training transfer and can be a fruitful area for organizations to work further on with their employees to strengthen the transfer process. Likewise, although this may be obvious, meetings between trainees and supervisors could also contribute to training transfer Transferability of training depends on what the organization offers to make it possible. As seen, transfer cannot be taken for granted by organizations. Actors need to implement facilitators and those facilitators need to be communicated to the employees. Employees need to know which resources are available for a better learning environment and transfer. Many employees complain about the workload. They note that sometimes they cannot focus on the learning tasks. This is clearly a barrier to learning and transfer. Actors in organizations not only need to make the training relevant for employees, but also provide a balanced pace for job tasks and training tasks. Some cases studied mentioned that they were fully engaged in training with peers, together, and this worked greatly for transfer. Everybody was on the same page and they gain trust and new knowledge when interacting with colleagues. Similarly, a degree of autonomy in the work management during training program could also be helpful for transfer. We have seen that active learning requires autonomy, where employees are capable to organize their own pace, rhythm and learning experiences according to their preferences and interest. However, it may be challenging for enterprises to 83

modify aspects of a work environment that facilitate training in the short term when they have already acquired government resources to invest in training within their organizations. Reorganizing the job is challenging, and costly. We suggest thinking in terms of solutions that can be feasible in the short-term and in the long-term. For example, when possibilities of subsidized training present themselves, organizations could work with their employees by proposing them to be engaged in the design of training dynamics. They could also assist employees in searching for the best way to deal with potential barriers and support them by creating a climate that facilitates training transfer and secure the quality of these trainings.

7.3 RECOMMENDATIONS FOR FURTHER RESEARCH The combination of QCA and Process-Tracing to study causes and process is not a panacea. There were many decisions that were carefully taken to make this research possible. One of them, was the conceptual design. Pattyn et al. (2020) provide some lessons for concept development when applying these innovative methods: •

•

produces an intrinsic motivation to learn) would lead to training transfer. Because, we were unable to confirm this in our case, sense of urgency as a causal condition needs to be further developed. •

•

Further case studies. This research was oriented to explain and understand the training transfer effectiveness. • •

•

This is an innovative evaluation study carried out with a complex research design. The methodological approach of combining QCA and process tracing is new and posed us with several challenges that took a 85

lot of effort to overcome. In many ways, this study was refined with the support of experts and other studies can build on and apply this research design in the future. For example, by adding more cases (especially unsuccessful ones) or studying the causal mechanisms that have already been disentangled in other positive cases. We hope that this evaluation research will be used as a practical tool by organizations to identify opportunities to improve their training policy.

8. REFERENCES Agoria. (2018). Shaping the future of work. Forem en Actiris en gerealiseerd door Roland Berger in opdracht van Agoria. Ajzen, I. (1985), From Intentions to Actions: A Theory of Planned Behavior (Heidelberg: Springer). Álamos-Concha, P., Pattyn, V., Rihoux, B., Schalembier, B, Beach, D. &. Cambré, B. (forthcoming). On the appropriateness of different solution types when combining Qualitative Comparative Analysis (QCA) and Process Tracing. Andrasik F, Heimberg JS. (1980). Self-management procedures. In Davidson PO, Davidson SM (Eds.), Behavioral medicine: Changing health lqe styles (pp. 219-247). New York: BrunnerMazel. Annett, J. (1969), Feedback and Human Behavior (Harmondsworth: Penguin Books). Arthur, W., Jr., Bennett, W., Jr., Edens, P. S., & Bell, S. T. (2003). Effectiveness of training in organizations: a meta-analysis of design and evaluation features. J Appl Psychol, 88(2), 234-245. Ashford, S. and Cummings, L. (1985), ‘Proactive feedback seeking: the instrumental use of the information environment’, Journal of Occupational Psychology, 58, 1, 67–79. Baert, L., Decramer, S., & Reynaerts, J. (2014). KMO portefeuille - pijler opleiding. Een evaluatie van de opleidingssubsidies in Vlaanderen. Beleidsrapport STORE-B-14-003. Leuven: Steunpunt Ondernemen & Regionale Economie. Baert, Herman, De Witte, Karel, Govaerts, Natalie, and Sterck, Geert. Werk Maken Van Leren. Strategisch VTO-beleid in Organisaties. Garant; Antwerpen, 2011. Web. Baldwin, T. T., & Ford, J. K. (1988). Transfer of Training: A Review and Directions for Future research. Personnel Psychology, 41(1), 63-105. doi:10.1111/j.1744-6570.1988.tb00632.x Baldwin, T. T., & Magjuka, R. J. (1991). Organizational training and signals of importance: Linking pretraining perceptions to intentions to transfer. Human Resource Development Quarterly, 2(1), 25-36. doi:10.1002/hrdq.3920020106. Baldwin, T. T., & Ford, K. J. (1988). Transfer of training: A review and directions for future research. Personnel Psychology, 41, 63-105. Bandura A. (1982). Self-efficacy mechanism in human agency. American Psychologist, 37, 122-147. Bandura, A. (1986). Social foundations of thought and action. Englewood Cliffs, New Jersey: Prentice-Hall. Bandura, A. (1999). Social cognitive theory: An agentic perspective. Asian Journal of Social Psychology, 2(1), 21-41. Bashkar, R. (1978). A Realist Theory of Science. Brighton: Harvester.

Bates, R. A., Holton, E. F. III, Seyler, D. L., Carvalho, M. A. (2000), The role of interpersonal factors in the application of computer-based training in an industrial setting, Human Resource Development International, Vol. 3(1): 19-42. Beach, D., & Pedersen, R. B. (2012). Process Tracing Methods: Foundations and Guidelines: University of Michigan Press. Beach, D., & Pedersen, R. B. (Eds.). (2016). Causal Case Study Methods. Foundations and Guidelines for Comparing, Matching, and Tracing. Ann Harbour: University of Michigan Press. Beach, D., & Rohlfing, I. (2016). Integrating cross-case analyses and process tracing in set-theoretic research: Strategies and parameters of debate. Sociological Methods & Research, online first. Beach, D., & Pedersen, R. B. (Eds.). (2019). Process Tracing Methods: Foundations and Guidelines (2nd ed.): University of Michigan Press. Berg-Schlosser, D., & De Meur, G. (2009). Comparative research design: case and variable selection. In B. Rihoux & C. C. Ragin (Eds.), Configurational comparative methods. Qualitative Comparative Analysis (QCA) and related techniques (pp. 19-32). Thousand Oaks and London: Sage. Berg-Schlosser, D., & De Meur, G. (1997). Reduction of Complexity for a Small-N Analysis: A Stepwise MultiMethodological Approach. Comparative Social Research, 16, 133-162. Berk, J. (2008). The manager’s responsibility for employee learning. Chief Learning Officer, 7(7), 46-48. Bhatti, M. A., Battour, M. M., Sundram, V. P. K. and Othman, A. A. (2013), ‘Transfer of training: does it truly happen? An examination of support, instrumentality, retention and learner readiness on the transfer motivation and transfer of training’, European Journal of Training and Development, 37, 3, 273–97. Bhatti, M. A., Ali, S., Isa, M., Faizal, M. and Battour, M. M. (2014), ‘Training transfer and transfer motivation: the influence of individual, environmental, situational, training design, and affective reaction factors’, Performance Improvement Quarterly, 27, 1, 51–82. Blatter, J., & Haverland, M. (2012). Designing Case Studies: Explanatory Approaches in Small-N Research Basingstoke: Palgrave Macmillan. Blume, B. D., Ford, J. K., Baldwin, T. T., & Huang, J. L. (2010). Transfer of Training: A Meta-Analytic Review. Journal of Management, 36(4), 1065-1105. doi:10.1177/0149206309352880 Boekaerts, M. and Corno, L. (2005) Self-Regulation in the Classroom: A Perspective on Assessment and Intervention. Applied Psychology: An International Review, 54, 199-231. https://doi.org/10.1111/j.14640597.2005.00205.x Botke, J. A., Jansen, P. G. W., Khapova, S. N., & Tims, M (2018). Work factors influencing the transfer stages of soft skills training: A literature review. Educational Research Review, 24, 130-147. doi: 10.1016/j.edurev.2018.04.001 Brady H & David Collier (Eds.) (2004). Rethinking Social Inquiry: Diverse Tools, Shared Standards. Lanham, Md.: Rowman and Littlefield, 362 pages.

Bransford, J. D., Brown, A. L., & Cocking, R. R. (1999). How people learn: Brain, mind, experience, and school. Washington, DC, US: National Academy Press. Brinkerhoff, R. O., & Montesino, M. U. (1995). Partnerships for training transfer: Lessons from a corporate study. Human Resource Development Quarterly, 6(3), 263-274. doi:10.1002/hrdq.3920060305 Broad, M. L. N., John Newstrom. (1992). Transfer of Training: Action-Packed Strategies to Ensure High Payoff from Training Investments.: Perseus Books. Brown, T. C. (2005). Effectiveness of distal and proximal goals as transfer of training intervention: A field experiment. Human Resource Development Quarterly, 16, 369-387. Brown, T. C., & Warren, A. M. (2009). Distal goal and proximal goal transfer of training interventions in an executive education program. Human Resource Development Quarterly, 20, 265-284. Brown, T. C., & Latham, G. P. (2002). The effects of behavioral outcome goals, learning goals, urging people to do their best on teamwork behavior on a group problem-solving task. Canadian Journal of Behavioural Science, 34, 276-285. Brown, T. C., & McCracken, M. (2010). Which goals should participants set to enhance the transfer of learning from management development programmes? Journal of General Management, 35(4), 27-44. Bunge, M. (1997). Mechanism and Explanation. Philosophy of the Social Sciences, 27(4), 410-465. Bunge, M. (2004). How Does It Work? The Search for Explanatory Mechanisms. Philosophy of the Social Sciences, 34(2), 182-210. Burke, L. A. (1997). Improving positive transfer: A test of relapse prevention training on transfer outcomes. Human Resource Development Quarterly, 8(2), 115-128. doi:10.1002/hrdq.3920080204 Burke, L. A., & Hutchins, H. M. (2007). Training Transfer: An Integrative Literature Review. Human Resource Development Review, 6(3), 263-296. doi:10.1177/1534484307303035 Campione, J. C., Shapiro, A. M., & Brown, A. L. (1995). Forms of transfer in a community of learners: Flexible learning and understanding. In Teaching for transfer: Fostering generalization in learning. (pp. 35-68). Hillsdale, NJ, US: Lawrence Erlbaum Associates, Inc. Cannon-Bowers, J. A., Salas, E., Tannenbaum, S. I., & Mathieu, J. E. (1995). Toward Theoretically Based Principles of Training Effectiveness: A Model and Initial Empirical Investigation. Military Psychology, 7(3), 141-164. doi:10.1207/s15327876mp0703_1 Cartwright, Nancy. 1999. The Dappled World: A Study of the Boundaries of Science. Cambridge: Cambridge University Press. Chauhan, R., Ghosh, P., Rai, A., & Shukla, D. (2016). The impact of support at the workplace on transfer of training: a study of an Indian manufacturing unit. International Journal of Training and Development, 20(3), 200-213. doi:10.1111/ijtd.12083 Chiaburu, D.S. and Marinova, S.V. (2005), What predicts skill transfer? An exploratory study of goal orientation, training self ‐ efficacy and organizational supports. International Journal of Training and Development, 9: 110-123. doi:10.1111/j.1468-2419.2005.00225.x 89

Chiaburu, D. S., Van Dam, K., & Hutchins, H. M. (2010). Social Support in the Workplace and Training Transfer: A longitudinal analysis. International Journal of Selection and Assessment, 18(2), 187-200. doi:10.1111/j.1468-2389.2010.00500.x Chiaburu, D. S. (2010), The social context of training: Coworker, supervisor, or organizational support?, Industrial and Commercial Training, Vol. 42, no. 1, pp. 53-56. Cohen, S., Kamarck, T., and Mermelstein, R. (1983). A global measure of perceived stress. Journal of Health and Social Behavior, 24, 386-396. Cohen, S., Gottlieb, B. H., & Underwood, L. G. (2000). Social relationships and health. In Social support measurement and intervention: A guide for health and social scientists. (pp. 3-25). New York, NY, US: Oxford University Press. Coleman, James. 1990. Foundations of Social Theory. Cambridge: Harvard University Press. Collier, H. E. B. D. (2004). Rethinking Social Inquiry: Diverse Tools, Shared Standards: Lanham, Md.: Rowman and Littlefield. Colquitt, J. A., LePine, J.A. and Noe, R.A. (2000). Toward an Integrative Theory of Training Motivation: A Meta-Analytic Path Analysis of 20 Years of Research. Journal of Applied Psychology, 85, 679-707. Craver, Carl F., and Lindley Darden. 2013. In Search of Mechanisms. Chicago: University of Chicago Press. Cromwell, S. E., & Kolb, J. A. (2004). An examination of work-environment support factors affecting transfer of supervisory skills training to the workplace. Human Resource Development Quarterly, 15(4), 449-471. doi:10.1002/hrdq.1115 Csikszentmihalyi, M., & Beattie, O. (1979). Life themes: A theoretical and empirical exploration of their origins and effects. Journal of Humanistic Psychology, 19, 677-693. Deci, E. L., & Ryan, R. M. (2012). Self-determination theory. In P. A. M. Van Lange, A. W. Kruglanski, & E. T. Higgins (Eds.), Handbook of theories of social psychology (p. 416–436). Sage Publications Ltd. https://doi.org/10.4135/9781446249215.n21 De Corte, E. (2003). Transfer as the Productive Use of Acquired Knowledge, Skills, and Motivations. Current Directions in Psychological Science, 12(4), 142-146. doi:10.1111/1467-8721.01250 De Meur, G., Rihoux, B., & Yamasaki, S. (2009). Addressing the critiques of QCA. In B. Rihoux & C. C. Ragin (Eds.), Configurational comparative methods. Qualitative Comparative Analysis (QCA) and related techniques (pp. 147-166). Thousand Oaks and London: Sage. De Meur, G., Bursens, P., & Gottcheiner, A. (2006). MSDO/MDSO revisited for public policy analysis. In B. Rihoux & H. Grimm (Eds.), Innovative comparative methods for policy analysis (pp. 67-94). New York: Springer. De Rick, K., & Van Itterbeeck, K. (2011). Subsidies voor bedrijfsopleidingen als impuls voor kansengroepen. Evaluatie in het kader van de on-going evaluatie ESF Vlaanderen 2007-2013. Retrieved from Leuven. DeSimone, W., & Harris (Ed.) (2002). Human Resource Development: Fort Worth: Harcourt

Dewey, J., (1997 edition) Experience and Education, New York: Touchstone. (1938, (1997 edition)). Experience and Education: New York: Touchstone. Dion, D. (1998). Evidence and Inference in the Comparative Case Study. Comparative Politics, 30(2), 127145. Dochy, F., & Segers, M. (2018). Creating Impact Through Future Learning The High Impact Learning that Lasts (HILL) Model, 1st Edition: Routledge. Dusa, A. (2019). QCA with R: A Comprehensive Resource Springer. Egan, T. M., Yang, B. and Bartlett, K. R. (2004), ‘The effects of organizational learning culture on motivation to transfer learning and turnover intention’, Human Resource Development Quarterly, 5, 3, 279–301. Elangovan, A. R., & Karakowsky, L. (1999). The role of trainee and environmental factors in transfer of training: An exploratory framework. Leadership & Organization Development Journal, 20(5), 268–275. https://doi.org/10.1108/01437739910287180. Elangovan, A. R. a. K., L. (1999). The Effects of Trainee and Environmental Factors on Transfer of Training: A Framework for Research. Leadership and Organizational Development Journal, 20, 268-275. Emmenegger, P., Schraff, D., & Walter, A. (2014). QCA, the Truth Table Analysis and Large - N Survey Data: The Benefits of Calibration and the Importance of Robustness Tests. COMPASSS Working Paper 201479(2014-79). European Skills, C., Qualification and Occupations taxonomy (ESCO). (2018). Skills. Retrieved from https://ec.europa.eu/esco/portal/skill. Fiss, P. C., Sharapov, D., & Cronqvist, L. (2013). Opposites Attract? Opportunities and Challenges for Integrating Large-N QCA and Econometric Analysis. Political Research Quarterly, 66(1), 191-198. Fogarty, R., Perkins, D., & Barell, J. (1992). How to teach for transfer: Palatine: Skylight. Ford, J. K., Quiñones, M. A., Sego, D. J., & Sorra, J. S. (1992). FACTORS AFFECTING THE OPPORTUNITY TO PERFORM TRAINED TASKS ON THE JOB. Personnel Psychology, 45(3), 511-527. doi:10.1111/j.17446570.1992.tb00858.x Forum, W. E. (2015). New Vision for Education: Unlocking the Potential of. Technology. Retrieved from Frayne CA, Latham GP. (1987). Application of social learning theory to employee self-management of attendance. Journal of Applied Psychology, 72,387-392. Gagne, R., Briggs, L., & Wager, W. (1992). Principles of instructional design: Fort Worth, TX: HBJ College. Gao X, Zhang LJ (2011) Joining forces for synergy: agency and metacognition as interrelated theoretical perspectives on learner autonomy. In: Murray G, Gao X, Lamb T (eds) Identity, motivation and autonomy: exploring the links. Multilingual Matters, Bristol, pp 25–41. Gaudine, A. P., & Saks, A. M. (2004). A longitudinal quasi-experiment on the effects of post-training transfer interventions. Human Resource Development Quarterly, 15, 57-76.

Geddes, B. (1990). How the Cases You Choose Affect the Answers You Get: Selection Bias in Comparative Politics. Political Analysis, 2, 131-150. Geddes, B. (2003). Paradigms and Sand Castles: Theory Building and Research Design in Comparative Politics: University of Michigan Press. Gegenfurtner, A. (2011). Motivation and transfer in professional training: A meta-analysis of the moderating effects of knowledge type, instruction, and assessment conditions. Educational Research Review, 6(3), 153-168. doi: https://doi.org/10.1016/j.edurev.2011.04.001 George, Alexander L., and Andrew Bennett. 2005. Case Studies and Theory Development in the Social Sciences. Cambridge: MIT Press. Gilpin-Jackson, Y., Bushe, G. R., & Fraser, S. (2007). Leadership development training transfer: A case study of post-training determinants. Journal of Management Development, 26(10), 980-1004. Gist, M., Stevens, C. K. and Bavetta, A. G. (1991). Effects of self-efficacy and post-training intervention on the acquisition and maintenance of complex interpersonal skills. Personnel Psychology, 44(4), 837-861. Glennan, S. S. (1996). Mechanisms and the Nature of Causation. Erkenntnis, 44(49-71). Glennan, S. S. (2002). Single-Outcome Studies: A Methodological Primer. International Sociology, 69, 342353. Glennan, S. S. (2005). Modeling Mechanisms. Studies in History and Philosophy of Biological and Biomedical Sciences, 36(2), 443-464. Goertz, G. (Ed.) (2005). Social science concepts: a user’s guide: Princeton: Princeton University Press. Goertz, G., & Mahoney, J. (2005). Two-Level Theories and Fuzzy-Set Analysis. Sociological Methods & Research, 33(4), 497-538. doi:10.1177/0049124104266128 Goldstein, I. L. (1986). Training in Organizations: Needs Assessment, Development and Evaluation: Brooks/Cole, Monterey, 111. Govaerts, N. (2017). Transfer of training in corporate settings: Toward an understanding of the multidimensional role of the supervisor. (PhD), KU Leuven, Govaerts, N., Kyndt, E., Vreye, S., & Dochy, F. (2017). A Supervisors' Perspective on Their Role in Transfer of Training. Human Resource Development Quarterly, 28(4), 515-552. doi:10.1002/hrdq.21286 Greckhamer, T., Furnari, S., Fiss, P.C. and Aguilera, R.V. (2018), “Studying configurations with qualitative comparative analysis: best practices in strategy and organisation research”, Strategic Organisation, 16(4), pp. 482–495. Gregoire, T. K., Propp, J., & Poertner, J. (1998). The Supervisor's Role in the Transfer of Training. Administration in Social Work, 22(1), 1-18. doi:10.1300/J147v22n01_01 Grossman, R., & Salas, E. (2011). The transfer of training: what really matters. International journal of training and development, 15(2), 103-120. doi:10.1111/j.1468-2419.2011.00373.x

Gumuseli, A. I., & Ergin, B. (2002). The manager’s role in enhancing the transfer of training: a Turkish case study. International Journal of Training and Development, 6(2), 80-97. doi:10.1111/1468-2419.00151 H. Baert, K. D. W., N. Govaerts & G. Sterck. (2011). Werk maken van leren. Strategisch VTO-beleid in organisaties: Garant Uitgevers; Auflage: 01. Hammer, D., Elby, A, Scherr, R., & Redish, E. F. (2005). Resources, framing, and tranffer. In J.P.Mestre (Ed.), Transfer of Learning from a Modern Multidisciplinary Perspective. (pp. 89-119): Greenwich, CT: IAP. Hatala, J.-P., & Fleming, P. R. (2007). Making Transfer Climate Visible: Utilizing Social Network Analysis to Facilitate the Transfer of Training. Human Resource Development Review, 6(1), 33-63. doi:10.1177/1534484306297116 Hawley, J. D., & Barnard, J. K. (2005). Work Environment Characteristics and Implications for Training Transfer: A Case Study of the Nuclear Power Industry. Human Resource Development International, 8(1), 65-80. doi:10.1080/1367886042000338308 Hedström, Peter, and Richard Swedberg, eds. 1998. Social Mechanisms: An Analytical Approach to Social Theory. Cambridge: Cambridge University Press. Hernes, G. (1998). Real Virtuality. In P. Hedström & R. Swedberg (Eds.), Social Mechanisms an Analytical Approach to Social Theory (pp. 74-101). Cambridge: Cambridge University Press. Hicks, W. D., & Klimoski, R. J. (1987). Entry into Training Programs and Its Effects on Training Outcomes: A Field Experiment. The Academy of Management Journal, 30(3), 542-552. doi:10.2307/256013 Hirst, G., Knippenberg, D. V., & Zhou, J. (2009). A Cross-Level Perspective on Employee Creativity: Goal Orientation, Team Learning Behavior, and Individual Creativity. Academy of Management Journal, 52(2), 280-293. doi:10.5465/amj.2009.37308035 Holton III, E. F., Bates, R. A., & Ruona, W. E. A. (2000). Development of a generalized learning transfer system inventory. Human Resource Development Quarterly, 11(4), 333-360. doi:10.1002/15321096(200024)11:4<333::AID-HRDQ2>3.0.CO;2-P Holton III, E. F., Bates, R. A., Seyler, D. L., & Carvalho, M. B. (1997). Toward construct validation of a transfer climate instrument. Human Resource Development Quarterly, 8(2), 95-113. doi:10.1002/hrdq.3920080203 Hutchins, H. M., & Burke, L. A. (2006). Has relapse prevention received a fair shake? A review and implications for future transfer research. Human Resource Development Review, 5, 8-23. Illari, Phyllis McKay. 2011. Mechanistic Evidence: Disambiguating the Russo-Williamson Thesis. International Studies in the Philosophy of Science 25 (2): 139–57. Itterbeeck, K. D. R. K. V. Subsidies voor bedrijfsopleidingen als impuls voor kansengroepen: Evaluatie in het kader van de on-going evaluatie ESF Vlaanderen 2007-2013. Retrieved from Jellema, F. A., Visscher, A. J., & Scheerens, J. (2006). Measuring change in work behaviour by means of multisource feedback. International journal of training and development. International Journal of Training and Development, 10(2), 121-139.

Jolanda A. Botke, P. G. W. J., Svetlana N. Khapova, Maria Tims,. (2018). Work factors influencing the transfer stages of soft skills training: A literature review. Educational Research Review, 24, 130-147. doi: https://doi.org/10.1016/j.edurev.2018.04.001. Jon, O. (2012). John Dewey and Experiential Learning: Developing the theory of youth work. Johnson, S. K., Garrison, L. L., Hernez-Broome, G., Fleenor, J. W., & Steed, J. L. (2012). Gofor the goal(s): Relationship between goal setting and transfer of training following leadership development. Academy of Management Learning & Education, 11, 555-569. Kalinoski, Z. T., Steele-Johnson, D., Peyton, E. J., Leas, K. A., Steinke, J., & Bowling, N. A. (2013). A metaanalytic evaluation of diversity training outcomes. Journal of Organizational Behavior, 34(8), 1076-1104. doi:10.1002/job.1839 Kanfer FH. (1980). Self-management methods. In Kanfer FH, Goldstein AP (Eds.), Helpingpeople change: A textbook of methods (2nd ed., pp. 334-389). New York Pergamon Press. Kastenmüller, A., Frey, D., Kerschreiter, R., Tattersall, A. J., Traut-Mattausch, E., & Fischer, P. (2012). Perceived openness of climate during training and transfer motivation: testing two short and simple interventions. Journal of Vocational Education & Training, 64(2), 211-225. doi:10.1080/13636820.2011.631711 Kennisnet, S. (2017). Curriculum van de toekomst: 21e eeuwse vaardigheden Retrieved from http://curriculumvandetoekomst.slo.nl/21e-eeuwse-vaardigheden/ King, G., Keohane, R. O., & Verba, S. (Eds.). (1994). Designing social inquiry: scientific inference in qualitative research. Princeton: Princeton University Press. Kirkpatrick, D. L. (1994). Evaluating training programs: the four levels. San Francisco: Berrett-Koehler. Kirwan, C., & Birchall, D. (2006). Transfer of learning from management development programmes: testing the Holton model. International Journal of Training and Development, 10(4), 252-268. doi:10.1111/j.14682419.2006.00259.x Kluger, A. and DeNisi, A. (1996), ‘The effects of feedback intervention on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory’,PsychologicalBulletin,19, 2, 254–84. Kolb, D. A. (1984). Experiential Learning Experience as the Source of Learning and Development. Englewood Cliffs: NJ Prentice Hall. Kozlowski, S., Brown, K., Weissbein, D. A., Cannon-Bowers, J. A., & Salas, E. (2000). A multilevel approach to training effectiveness. In K. Klein & S. Kozlowski (Eds.), Multi level theory, research, and methods in organizations (pp. 157-210). San Francisco, CA: Jossey-Bass. Lacerenza, C. N., Reyes, D. L., Marlow, S. L., Joseph, D. L., & Salas, E. (2017). Leadership training design, delivery, and implementation: A meta-analysis. J Appl Psychol, 102(12), 1686-1718. doi:10.1037/apl0000241 Lancaster, S., Di Milia, L., & Cameron, R. (2013). Supervisor behaviours that facilitate training transfer. Journal of Workplace Learning, 25(1), 6-22. doi:10.1108/13665621311288458

Latham, G. P., & Locke, E. A. (2007). New developments in and directions for goal-setting research. European Psychologist, 12, 290-300. Lau, P. Y. Y. and McLean, G. N. (2013), ‘Factors influencing perceived learning transfer of an outdoor management development programme in Malaysia’, Human Resource Development International, 16(2):186–204. Lave, J. (1988). Cognition in Practice: Mind, mathematics, and culture in everyday life. Cambridge, UK: Cambridge University Press. Lazonder, A. W., & Harmsen, R. (2016). Meta-Analysis of Inquiry-Based Learning: Effects of Guidance. Review of Educational Research, 86(3), 681–718. https://doi.org/10.3102/0034654315627366 Lim, D. H., & Johnson, S. D. (2002). Trainee perceptions of factors that influence learning transfer. International Journal of Training and Development, 6(1), 36-48. doi:10.1111/1468-2419.00148 Lim, D. H., & Morris, M. L. (2006). Influence of trainee characteristics, instructional satisfaction, and organizational climate on perceived learning and training transfer. Human Resource Development Quarterly, 17(1), 85-115. doi:10.1002/hrdq.1162 Locke, E. A., & Latham, G. P. (2002). Building a practically useful theory of goal setting and task motivation: A 35-year odyssey. American Psychologist, 57(9), 705-717. doi:10.1037/0003-066X.57.9.705 Luthans, F. and T. R. V. Davis. “Behavioral Self Manage- ment: The Missing Link in Managerial Effectiveness,” Organizational Dynamics, 8 (1979), 42-60. Luthans, F., & Jensen, S. M. (2002). Hope: A new positive strength for human resource development. Human Resource Development Review, 1, 304-322. Machamer, Peter, Lindley Darden, and Carl F. Craver. (2000). Thinking about Mechanisms. Philosophy of Science 67 (1): 1–25. Machamer, Peter. 2004. Activities and Causation: The Metaphysics and Epistemology of Mechanisms. International Studies in the Philosophy of Science 18 (1): 27–39. Maggetti, M., & Levi-Faur, D. (2013). Dealing with Errors in QCA. Political Research Quarterly, 66(1), 198204. Martin, H. J. (2010). Workplace climate and peer support as determinants of training transfer. Human Resource Development Quarterly, 21(1), 87-104. doi:10.1002/hrdq.20038 Marx, A., Rihoux, B., & Ragin, C. C. (2013). The Origins, Development and Applications of Qualitative Comparative Analysis (QCA): The First 25 Years. European Political Science Review, 6(1), 115-142. Marx, R. D. (1982). Relapse Prevention for Managerial Training: A Model for Maintenance of Behavior Change. Academy of Management Review, 7(3), 433-441. doi:10.5465/amr.1982.4285359 Marx, R. D. (1986). Improving Management Development Through Relapse Prevention Strategies. Journal of Management Development, 5(2), 27-40. doi:10.1108/eb051607

Marx, A., & Dusa, A. (2011). Crisp-Set Qualitative Comparative Analysis (csQCA), Contradictions and Consistency Benchmarks for Model Specification. Methodological Innovations Online, 6(2), 103–148. https://doi.org/10.4256/mio.2010.0037 Mayntz, Renate. 2004. Mechanisms in the Analysis of Social Macro-Phenomena. Philosophy of the Social Sciences 34 (2): 237–59. McCormick, M. J., Tanguma, J., & Lopez-Forment, A. S. (2002). Extending self-efficacy theory to leadership: A review and empirical test. The Journal of Leadership Education, 1(2), 1-15. Medina, I., Castillo Ortiz, P. J., Álamos-Concha, P., & Rihoux, B. (2017). Análisis Cualitativo Comparado (QCA). Madrid: Centro de Investigaciones Sociológicas (CIS). Morin, L., & Latham, G. (2000). The Effect of Mental Practice and Goal Setting as a Transfer of Training Intervention on Supervisors’ Self-efficacy and Communication Skills: An Exploratory Study. Applied Psychology, 49(3), 566-578. doi:10.1111/1464-0597.00032 Morin, L., & Latham, G. P. (2000). The effect of mental practice and goal setting as a transfer of training intervention on supervisors' self-efficacy and communication skills: An exploratory study. Applied Psychology: An International Review, 49(3), 566-578. doi:10.1111/1464-0597.00032 Newstrom, J. W. (1986). Leveraging Management Development through the Management of Transfer. Journal of Management Development, 5(5), 33-45. doi:10.1108/eb051628 Nijman, D. -J. J. M. (2004), ‘Supporting transfer of training: effects of the supervisor’. Doctoral Dissertation, University of Twente, Enschede, the Netherlands. Nijman, D. J. J. M., Nijhof, W. J., Wognum, A. A. M., & Veldkamp, B. P. (2006). Exploring differential effects of supervisor support on transfer of training. Journal of European Industrial Training, 30(7), 529-549. doi:10.1108/03090590610704394 Noe, R. A. (1986). Trainees' Attributes and Attitudes: Neglected Influences on Training Effectiveness. The Academy of Management Review, 11(4), 736-749. doi:10.2307/258393 Noe, R., Sears, J. and Fullenkamp, A. (1990). Relapse training: Does it influence trainees’ post-training behavior and cognitive strategies? Journal of Business Psychology, 4(3), 319-328. Oana, Ioana-Elena and Carsten Q. Schneider. 2018. SetMethods: An Add-on R Package for Advanced QCA. The R Journal 10(1): 507-33. https://journal.r-project.org/archive/2018/RJ-2018-031/index.html Ostroff, C., & Bowen, D. E. (2000). Moving HR to a higher level: HR practices and organizational effectiveness. In Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions. (pp. 211-266). San Francisco, CA, US: Jossey-Bass. Paris, S. G., & Paris, A. H. (2001). Classroom applications of research on self-regulated learning. Educational Psychologist, 36(2), 89–101. https://doi.org/10.1207/S15326985EP3602_4 Pattni, I., Soutar, G. N. and Klobas, J. E. (2007). The impact of a short self-management training intervention in a retail banking environment. Human Resource Development Quarterly, 18(2), 159-178.

Pattyn, V., Álamos-Concha, P., Cambré, B., Rihoux, B., & Schalembier, B. (2020). Policy Effectiveness through Configurational and Mechanistic Lenses: Lessons for Concept Development, Journal of Comparative Policy Analysis: Research and Practice, DOI: 10.1080/13876988.2020.1773263 Peters, B. G. (2013). Strategies for comparative research in political science: theory and methods. Basingstoke: Palgrave. Piezzi, D. (2002). Transferf¨orderung in der betrieblichen Weiterbildung. Die Bedeutung der Arbeitsumgebung und der Integration der Weiterbildung in die Unternehmensf¨uhrung [Supporting the transfer in further education. The influence of work environments and of the integration of further training into the management]. Paderborn, Germany: Eusl-Verlag. Prince, M. (2004). Does Active Learning Work? A Review of the Research. Journal of Engineering Education, 93(3), 223-231. doi:10.1002/j.2168-9830.2004.tb00809.x Przeworski, A., & Teune, H. (1970). The logic of comparative social inquiry. In. New York: Wiley-Interscience. Quiñones, M. A., Ford, J. K., & Teachout, M. S. (1995). The Relationship Between Work Experience and Job Performance: A Conceptual and Meta-Analytic Review. Personnel Psychology, 48(4), 887-910. doi:10.1111/j.1744-6570.1995.tb01785.x Quiñones, M. A., Ford, J. K., Sego, D. J., & Smith, E. M. (1995). The effects of individual and transfer environment characteristics on the opportunity to perform trained tasks. Training Research Journal, 1, 29– 48. Rackham, N. (1979). The Coaching Controversy. Training and Development Journal, 33(11), 12-16. Ragin, C. C. (1987). The Comparative Method: Moving Beyond Qualitative and Quantitative Strategies. Berkeley and Los Angeles, California: University of California Press. Ragin, C. C. (1996). The Distinctiveness of comparative social science. In A. Inkeles & M. Sasaki (Eds.), Comparing Nations and Cultures. Readings in a cross-disciplinary perspective. Englewood Cliffs, New Yersey: Prentice Hall. Rank, B., & Wakenhut, R. (1998). Sicherung des Praxistransfers im Führungskräftetraining: Hampp, R. Rahyuda, A., Syed, J., & Soltani, E. (2014). The Role of Relapse Prevention and Goal Setting in Training Transfer Enhancement. Human Resource Development Review, 13(4), 413–436. https://doi.org/10.1177/1534484314533337 Rehmann AJ, Mitman RD, & MC., R. (1995). A Handbook of Flight Simulation Fidelity Requirements for Human Factors Research: PN. Reinhold, S., Gegenfurtner, A., & Lewalter, D. (2018). Social support and motivation to transfer as predictors of training transfer: testing full and partial mediation using meta-analytic structural equation modelling. International Journal of Training and Development, 22(1), 1-14. doi:10.1111/ijtd.12115 Revans, R. W. (1982). What is Action Learning? Journal of Management Development, 1(3), 64-75. doi:10.1108/eb051529

Richman-Hirsch, W. L. (2001). Post training interventions to enhance transfer: The moderating effects of work environments. Human Resource Development Quarterly, 12(2), 105-120. doi:10.1002/hrdq.2 Rihoux, B. (2013). Qualitative Comparative Analysis (QCA) from ‘The Comparative Method’ (1987) till 2012: from one to several stories? Swiss Political Science Review. Rihoux, B. (Manuscript in progress). Comparative Research Designs. Rihoux, B., Álamos-Concha, P., Bol, D., Marx, A., & Rezsöhazy, I. (2013). From Niche to Mainstream Method? A Comprehensive Mapping of QCA Applications in Journal Articles from 1984 to 2011. Political Research Quarterly, 66(1), 175-184. Rihoux, B., & Lobe, B. (2009). The case for qualitative comparative analysis (QCA): adding leverage for thick cross-case comparison. In D. Byrne & C. C. Ragin (Eds.), The Sage handbook of case-based methods (pp. 222-243). London: Sage. Rihoux, B., & Ragin, C. C. (2009). Configurational comparative methods. Qualitative Comparative Analysis (QCA) and related techniques. Thousand Oaks and London: Sage. Rihoux, B., & Ragin, C. C. (2009). Introduction. In B. Rihoux & C. C. Ragin (Eds.), Configurational comparative methods. Qualitative Comparative Analysis (QCA) and related techniques (pp. xvii-xxv). Thousand Oaks and London: Sage. Rihoux, B. (2017). Configurational Comparative Methods (QCA and Fuzzy Sets): Complex Causation in Crosscase Analysis. In J. Woldendorp & H. Keman (Eds.), Handbook of Research Methods and Applications in Political Science (pp. 383-399). Cheltenham: Edward Elgar. Ripple, R., & Drinkwater, D. (1982). Transfer of learning. In M. C. Alkin (Ed.), Encyclopedia of educational research Vol 4 (Vol. 4, pp. 1947–1953). New York: Macmillan. Russ-Eft, D. (2002). A Typology of Training Design and Work Environment Factors Affecting Workplace Learning and Transfer. Human Resource Development Review, 1(1), 45-65. doi:10.1177/1534484302011003 Russo, Federica, and Jon Williamson. 2011. Generic versus Single-Case Causality: The Case of Autopsy. European Journal of the Philosophy of Science 1 (1): 47–69. Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol, 55(1), 68-78. Sager, F., & Andereggen, C. (2012). Dealing With Complex Causality in Realist Synthesis: The Promise of Qualitative Comparative Analysis. American Journal of Evaluation, 33(1), 60-78. doi:10.1177/1098214011411574 Saks, A. M., & Belcourt, M. (2006). An investigation of training activities and transfer of training in organizations. Human Resource Management, 45(4), 629-648. doi:10.1002/hrm.20135 Salas, E., & Cannon-Bowers, J. A. (2001). The science of training: a decade of progress. Annu Rev Psychol, 52, 471-499. doi:10.1146/annurev.psych.52.1.471

Salas, E., Tannenbaum, S. I., Kraiger, K., & Smith-Jentsch, K. A. (2012). The Science of Training and Development in Organizations: What Matters in Practice. Psychological Science in the Public Interest, 13(2), 74-101. doi:10.1177/1529100612436661 Schneider, C. Q., & Wagemann, C. (2012). Set-Theoretic Methods for the Social Sciences: A Guide to Qualitative Comparative Analysis. Cambridge: Cambridge University Press. Schneider, C.Q. (2018). Two-step QCA revisited: the necessity of context conditions. Qual Quant 53, 1109– 1126 (2019). https://doi.org/10.1007/s11135-018-0805-7 Schneider, Carsten Q, Rohlfing, Ingo. (2016). “Case Studies Nested in Fuzzy-set QCA on Sufficiency: Formalizing Case Selection and Causal Inference.” Sociological Methods & Research 45:526–568. Schön, D.A. 1983. The reflective practitioner: how professionals think in action. Aldershot, England: Ashgate. Seijts, G. H., Latham, G. P., Tasa, K., & Latham, B. (2004). Goal setting and goal orientation: An integration of two different yet related literatures. Academy of Management Journal, 47, 227-239. Sels, L., Vansteenkiste, S., & Knipprath, H. (2017). Toekomstverkenningen arbeidsmarkt 2050. Retrieved from Leuven: Senge, P. M. (1990). The fifth discipline: The art & practice of the learning organization. New York: Doubleday Business. Seyler, D., Holton, E. III, Bates, R., Burnett, M. and Carvalho, M. (1998), ‘Factors affecting motivation to transfer training’, International Journal of Training and Development, 2, 1, 16–17. Sierens, E., Vansteenkiste, M., Goossens, L., Soenens, B. and Dochy, F. (2009), The synergistic relationship of perceived autonomy support and structure in the prediction of self‐regulated learning. British Journal of Educational Psychology, 79: 57-68. doi:10.1348/000709908X304398. Sitzmann, T., & Weinhardt, J. M. (2018). Training Engagement Theory: A Multilevel Perspective on the Effectiveness of Work-Related Training. Journal of Management, 44(2), 732-756. doi:10.1177/0149206315574596 Skaaning, S.-E. (2011). Assessing the Robustness of Crisp-set and Fuzzy-set QCA Results. Sociological Methods and Research, 40(2), 391-408. doi:10.1177/0049124111404818. (SMB), E. S. M. B. Een kmo volgens de Europese kmo-definitie: kleine of middelgrote onderneming. Retrieved from https://www.vlaanderen.be/nl/ondernemen/kapitaal-en-krediet/een-kmo-volgens-deeuropese-kmo-definitie-kleine-middelgrote-onderneming Smith, R. (2006). Epistemological agency: a necessary action-in-context perspective on new employee workplace learning. Studies in Continuing Education, 28(3), 291-304. doi:10.1080/01580370600947553 Smither, J. W., London, M. and Reilly, R. R. (2005), ‘Does performance improve following multisource feedback? A theoretical model, meta-analysis, and review of empirical findings’, Personnel Psychology, 58, 33–66.

Sookhai, F., & Budworth, M.-H. (2010). The trainee in context: Examining the relationship between selfefficacy and transfer climate for transfer of training. Human Resource Development Quarterly, 21(3), 257272. doi:10.1002/hrdq.20044 Srivastava, A., Bartol, K. M., & Locke, E. A. (2006). Empowering Leadership in Management Teams: Effects on Knowledge Sharing, Efficacy, and Performance. The Academy of Management Journal, 49(6), 1239-1251. doi:10.2307/20159830 Taylor, P. J., Russ-Eft, D. F., & Taylor, H. (2009). Transfer of management training from alternative perspectives. Journal of Applied Psychology, 94: 104-121. Tews, M. J. and Tracey, J. B. (2008). An empirical examination of posttraining on-the-job supplements for enhancing the effectiveness of interpersonal skills training. Personnel Psychology, 61(2), 375-401. Tharenou, P., Saks, A. M., & Moore, C. (2007). A review and critique of research on training and organizational-level outcomes. Human Resource Management Review, 17(3), 251-273. Thorndike, E. L., & Woodworth, R. S. (1901). The influence of improvement in one mental function upon the efficiency of other functions. (I). Psychological Review, 8(3), 247–261. Wexley, K. and Baldwin, T. T. (1986). Post-training strategies for facilitating positive transfer: An empirical exploration. Academy of Management Journal, 29(3), 503-520. van der Klink, J. J., Bultmann, U., Burdorf, A., Schaufeli, W. B., Zijlstra, F. R., Abma, F. I., van der Wilt, G. J. (2016). Sustainable employability--definition, conceptualization, and implications: A perspective based on the capability approach. Scand J Work Environ Health, 42(1), 71-79. doi:10.5271/sjweh.3531 Van Den Bossche, P., Segers, M., Jansen, N. (2010), Transfer of training: The role of feedback in supportive social networks, International Journal of Training and Development, Vol. 14(2): 81-94. van der Locht, M., van Dam, K., & Chiaburu, D. S. (2013). Getting the most of management training: the role of identical elements for training transfer. Personnel Review, 42(4), 422-439. doi:10.1108/PR-05-20110072. Vandewalle, D. (1997). Development and validation of a work domain goal orientation instrument. Educational and Psychological Measurement, 57(6), 995-1015. doi:10.1177/0013164497057006009 Van Dinther, M., Dochy, F., & Segers, M. (2011). Factors affecting students’ self-efficacy in higher education. Educational Research Review, 6(2), 95–108. doi:10.1016/j.edurev.2010.10.003. Verba S, K. G., Keohane RO. (1994). Designing Social Inquiry: Scientific Inference in Qualitative Research. Princeton: Princeton University Press. Werner, J. M., O’Leary-Kelly, A. M., Baldwin, T. I., & Wexley, K. N. (1994). Augmenting behavior-modeling training: Testing the effects of pre- and post-training interventions. Human Resource Development Quarterly, 5, 169-183. Wexley, K. and Baldwin, T. T. (1986). Post-training strategies for facilitating positive transfer: An empirical exploration. Academy of Management Journal, 29(3), 503-520. Wexley, K., & Latham, G. (2011). Developing and Training Human Resources in Organizations: Pearson. 100

Wood, R. and Bandura, A. (1989). Social cognitive theory of organizational management. Academy of Management Review, 14(3), 361-384. Zigarmi, D., Nimon, K., Houson, D., Witt, D. and Diehl, J. (2009). Beyond engagement: Toward a framework and operational definition for employee work passion. Human Resource Development Review, 8(3), 300326.

101

ANNEX 1: OVERVIEW OF THE EVALUANDUM This report presents an overview of the results of the evaluation project of the ESF call 395 “Opleidingen in bedrijven”. The evaluation concerns the training projects for the working population in Flanders, funded within the framework of the Flemish European Social Fund Operational Program for the period 2014-2020 (under its priority 2: “preventative career policy” and its investment priority 10iii: life-long learning). The formal objectives of this framework are: (1) to enhance the competences of the working population in Flanders; (2) to increase the visibility (by giving certificates) of these competences, and (3) to stimulate enterprises to create and maintain an optimal policy framework for training and to inject Flanders with knowledge. The applicants of the call are small, medium or large enterprises, as well as sectoral funds. In line with the European Skills Agenda, the training projects within the companies have to focus on the development of either digital or transferable skills, or literacy, numeracy, and general skills specifically for the lowly educated. In addition, trainings that support innovation of a business are also accepted. Call 395 is part of a larger series of calls (307,322,377 and 395) that stimulate participation of the Flemish working population in training. In total, these calls concern a budget of 14 715 457 EUR in terms of ESF co-financing. The projects in ESF-call 395 receive a maximum subsidy of 100 000 EUR, of which 40% is provided by ESF and the remaining 60% by the Flemish co-financing fund. Participating organizations are required to provide at least 30% (micro- or small organizations) or 50% (medium and large organizations) of the necessary resources for the training projects themselves. These training projects took place between October 2017 and September 2018.

102

ANNEX 2: LIMITATIONS OF QCA QCA is a comparative method in constant evolution. For these reasons many of the caveats and limitations are being managed in Good Practices (Schneider and Wagemann, 2010; Greckhamer, et al., 2018). While it is not the aim of this evaluation to present each caveat of QCA, we intent to be transparent with the issues that we have confronted when using this method in relation to other research methods. One limitation of using QCA, v/s standard statistical methods, is that it is not possible to engage in statistical inference from the sample to the whole population (Rihoux, 2017). However, this is not so much of a problem in intermediate-N situations, as it is possible to engage in meaningful, ‘contingent’ generalization beyond the empirically processed cases (Blatter & Haverlandand, 2012; De Meur, Rihoux, & Yamasaki, 2009). Another potential limitation of using QCA is its case-sensitivity, i.e. the fact that small modifications in the protocol (e.g. taking one additional case in, suppressing one case from the analysis, modifying the calibration of a given solution, etc.) can lead to major changes in the QCA solutions (De Meur et al., 2009). This is correct, but this critique should be mitigated in a least three ways: (1) this sensitivity to cases is also a strong asset of the method from a case-oriented perspective (De Meur et al., 2009); (2) there are ways to mitigate this caveat by carefully selecting cases and by applying a rigorous protocol for calibration etc.; (3) some ‘robustness tests’ are being developed for QCA (e.g. Emmenegger, Schraff, & Walter, 2014; Fiss, Sharapov, & Cronqvist, 2013; Maggetti & Levi-Faur, 2013; Skaaning, 2011). The number of conditions in a given QCA model should be kept relatively low. In more technical terms, one faces an issue in terms of the ratio between the number of conditions and the number of cases (BergSchlosser & De Meur, 2009) and one needs to follow a benchmark that has been defined by Marx & Dusa (2011) and that is now quite established. This is of course challenging if one is confronted with multiple potential conditions. However several strategies can be used to mitigate this difficulty, such as implementing ‘two-step’ QCA in order to accommodate more conditions , aggregating some conditions, exploiting the MSDO/MDSO procedure to select some conditions that will be injected in the QCA model (Berg-Schlosser & De Meur, 1997; De Meur, Bursens, & Gottcheiner, 2006), etc.

103

ANNEX 3: LITERATURE REVIEW A.3.1 APPROACH TO THE SELECTION OF THEORETICAL PERSPECTIVES The literature on training effectiveness relates to different academic disciplines, and combines insights from management, human resource development, educational studies, sociology, and psychology. A comprehensive study has been built on all these different disciplines. We have combined two main strategies in the selection of theoretical perspectives in an iterative way. The scientific theories and models have given a relevant complement to the program theories applied by stakeholders. 1. Literature review. In recent years, interesting review articles have been published about the state of evidence on training effectiveness and impacts. These review articles were a useful starting point to select theoretical models and conditions. We refer for instance to the widely referenced work of Burke and Hutchins (2007). They conducted a large-scale literature review that encompasses the variety of disciplines (i.e. management, training, adult learning, performance improvement, psychology, and HRD) in which evidence on the influencing conditions on training effects can be found. The systematic literature review by Burke and Hutchins (2007) yielded a long list of possible conditions, grouped along the three long-standing axes that we mentioned above (see Table A13): learner characteristics; characteristics of the intervention; and work environment influences. In addition, and interestingly, the authors indicate to what extent evidence is available per condition, anno the time of publication (2007), and whether causal evidence is strong, lacking or very ambiguous. Burke and Hutchins’ work provided a useful stepping- stone to select the conditions of most relevance for our evaluation. For instance, conditions which are known to be of strong causal importance are preferably to be included in the QCA model. Since the publication of this literature review other authors, such as Grossman & Salas (2011) and Blume et al. (2010) have worked more in depth on the factors that were shown to consistently impact training effects. These studies were also valuable to select the most relevant theoretical approaches. In our evaluation, we aimed to use these review articles to identify the most interesting theoretical approaches, which we will subsequently present in this annex. 2. The outcome considered in these review articles is ‘transfer’, i.e. the behavior level identified by Kirkpatrick (1994). From a time, perspective, and as argued by Kirkpatrick, the transfer level is assumed to follow prior changes at the learning level. We believe that this transfer level is a very relevant level to consider in an evaluation that focuses on the worker’s level. The definition of transfer is also revealing in this regard: “for transfer to occur learned behavior must be generalized to the job and maintained over a period of time” (Baldwin & Ford, 1988: 63). After all, one of the criteria for eligibility of ESF training programs is that the contents trained should be generic and applicable in other firms and sectors. Of course, in the early stages of the evaluation, we will need to decide which outcome(s) to focus on. 3. As for organizational outcomes , the literature provides a number of models to explain how training might lead to organizational outcomes. The central idea is that when training does result in improvements in relevant knowledge and the acquisition of relevant skills, knowledge and attitudes, employee performance will be improved. In several models, including the one of Kozlowski et al. (2000), training transfer will be the key mechanism in this regard: “because it is the primary 104

leverage point by which training can influence organizational effectiveness” (see also Tharenou et al., 2007). In turn, improvement in job performance is supposed to reflect in changes in organizational outcomes or results criteria, at least if the job is strategically aligned to the organization's needs (Baldwin & Ford, 1988; Tharenou et al., 2007). Despite the existence of a multitude of models that consider the causal chain towards organizational outcomes, evidence on the different steps in this chain remains limited (Tharenou et al., 2007). Therefore, and as mentioned above, we propose to focus the evaluation on effects/impact at the worker level only. 4. Input from the theoretical expert panel: In line with the requirements outlined in the ToR, we have involved theoretical experts in the evaluation. They represent the main disciplinary approaches to training effectiveness. The experts have been asked for input in the identification and operationalization of relevant theories, and have provided guidance in the selection of outcomes, conditions and mechanisms. Later in the evaluation they have also assisted us in the theoretical interpretation of paths yielded by the QCA analysis and will review the Process-Tracing results from a theoretical point of view. Table A13: Overview of potential conditions conducive to ‘value at worker level’ and ‘value at firm level’

Item Firm size Sector of the firm (secondary, tertiary, quaternary) “a strategic (training) plan / orientation as a framework for setting up training activities” Consultation of (future) trainees on their training needs and the (future) training

Level

Source(s)

Work environment Work environment

HIVA evaluation (2011) HIVA evaluation (2011)

Work environment

[call, p. 5]

Work environment

[manual, p. 17] (Huxham, 1996; Salas et al., 2012)

Work environment “existing learning culture [in firm]”

“use of other instruments (e.g. KMOportefeuille)” “[existence of] a career development policy (incl. the extent to which this is oriented to supporting

Work environment Work environment

Remarks

Worker level Outcomes Learning Behavior level level X X

[call, p. 5] Transfer climate & continuous learning culture: (Rouiller & Goldstein, 1993; Tracey, Tannenbaum, & Kavanagh, 1995) [call, p. 5]

[call, p. 5] (De Vos & Cambré, 2017)

linked to ‘strategic training plan’ condition?

Firm level Outcomes Results level X X

105

workers to take charge of their own careers)” “[existence of] a systematic evaluation of training activities” “[implementation of] high quality needs analysis (for example as detected via a personal development plan at worker level)”

Work environment

[call, p. 5] (Salas et al., 2012)

Work environment

[call, p. 5] (Salas et al., 2012)

Work environment

(Bakker, Cambré, Korlaar, & Raab, 2011; Burke & Hutchins, 2007; E. Grossman & Woll, 2007; R. Grossman & Salas, 2011) (Burke & Hutchins, 2007; R. Grossman & Salas, 2011) (Burke & Hutchins, 2007; R. Grossman & Salas, 2011) (Burke & Hutchins, 2007; R. Grossman & Salas, 2011) (Burke & Hutchins, 2007; Noe & Schmitt, 1986) (Burke & Hutchins, 2007) (Burke & Hutchins, 2007; R. Grossman & Salas, 2011) (Burke & Hutchins, 2007; R. Grossman & Salas, 2011); HIVA evaluation (2011) (Burke & Hutchins, 2007; R. Grossman & Salas, 2011) (Burke & Hutchins, 2007) [call, p. 5] Learning orientation (Ford & Weissbein, 1997); career/job attitudes [call, p. 5]

Existence of transfer climate

Support from supervisors and peers Opportunity to perform

Work environment Work environment

Cognitive ability

Learner characteristics

Career planning

Learner characteristics

Openness to experience Pretraining motivation

Perceived utility; match between supply and demand; satisfaction about the training Self-efficacy Organizational commitment “profiles of [training] participants” “gender of [training] participants”

Learner characteristics Learner characteristics Learner characteristics

Learner characteristics

X x

106

Learner characteristics

[call, p. 5] (Kubeck, Delp, Haslett, & McDaniel, 1996); HIVA evaluation (2011)

Learner characteristics Learner characteristics

“age of [training] participants”

“foreign origin of [training] participants” Educational attainment Anxiety, negative affectivity Self-confidence

Locus of control

Practice and feedback

Error based examples

Content relevance

Learning goals

Transversal nature of the training content

Learner characteristics Learner characteristics

Learner characteristics Intervention design and delivery characteristic Intervention design and delivery characteristics Intervention design and delivery characteristics Intervention design and delivery characteristics Intervention design and delivery characteristics

[age categories, e.g. workers older than 54]

[call, p. 5]

HIVA evaluation (2011); cognitive ability (R. Grossman & Salas, 2011) (Burke & Hutchins, 2007) HIVA evaluation (2011); self-efficacy (R. Grossman & Salas, 2011) HIVA evaluation (2011); (Noe, 1986) (Burke & Hutchins, 2007; CannonBowers, Rhodenizer, Salas, & Bowers, 1998) (Burke & Hutchins, 2007; R. Grossman & Salas, 2011)

(Burke & Hutchins, 2007); perceived utility of training (R. Grossman & Salas, 2011) (Burke & Hutchins, 2007); learning goal orientation (Tziner, Fisher, Senior, & Weisberg, 2007) HIVA evaluation (2011)

Source: Adapted and complemented from Kirkpatrick (1994)

107

A.3.2 OUTCOME: SUCCESS VERSUS FAILURE TO TRANSFER LEARNED SOCIAL SKILLS TO THE WORKPLACE As mentioned earlier, we have chosen to narrow our outcome and scoping down to literature which is meaningful in relation with the ESF’s goal of developing sustainable employability. Within the learning and training literature, the concept of ‘effective employee training transfer’ holds a central position. In the following paragraphs we conceptualize the understanding of this outcome. Most of the definitions mapped from the literature focus on ‘training transfer’ as ‘the application of what is learned from the training to the workplace’ (Broad & Newstrom, 1992; Cromwell, 2004; Dochy & Segers, 2018: 163; Newstrom, 1986; DeSimone, Werner, & Harris, 2002: 3; Gumuseli & Ergin, 2002; 81; Hawley & J. Barnard, 2005: 66; Newstrom, 1986). This use of new knowledge into the job is also referred to in the literature as ‘generalization’, meaning that the trainees are capable to ‘activate the resources’, (Hammer et al, 2005) acquired in one context (e.g. training), in another context (e.g. the job) (Baldwin & Ford, 1988; Bransford, Brown, & Cocking, 1999; Campione, Shapiro, & Brown, 1995: 39; Chiaburu, et.al., 2010; Fogarty et al., 1992; Gagne et al., 1993: 235; Hawley & J. Barnard, 2005: 66; Kirwan, 2009; Lave, 1988: 122), or as ‘productive use of acquired knowledge and skill’ (De Corte, 2003; Gegenfurtner, 2011: 154). Similarly, some scholars introduce the notion of ‘maintenance of the learned material over a period of time on-the-job” (Kirwan, 2009:5; Blume et. al, 2010) when they refer to an effective transfer, arguing that the continued application can led to a certain standard over time (Broad & Newstrom, 1992:6). With these concepts at hand, we can build up the concept of ‘effective employee training transfer’ as follows: “Effective employee training transfer is the application or use of the learned knowledge (content, skills or attitudes) acquired in a training program to the job by trainees, which is maintained over a period of time” The main attributes of the concepts are: (1) generalization to other contexts [application of new knowledge from training site into the workplace]; and (2) maintenance of learned content over a period of time onthe-job [effectiveness]. In order to know how an ‘effective employee training transfer’ looks like, we need to observe these two main attributes in all the cases to be considered in the set of the positive pole of the outcome. Thus, the structure of our concept is a conjuncture of two main attributes which need to be jointly present to make the concept exist. In our study, we are not going to study the conditions for the absence of the outcome, but solely the conditions for an effective employee training transfer and hence, the causal mechanisms linking these conditions and the positive pole of the outcome, within certain necessary contexts. For this reason, the negative pole of our outcome let’s say “failure of “, can be understood as that the sole absence of one attribute implies the absence of the concept or the ‘failure of an effective employee training transfer’. Thus, with these concepts at hand, we can build up the conceptual view of the concept ‘failed effective employee training transfer’ as follows: “The failure of an effective employee training transfer is the lack of generalization to the job context or lack of maintenance of the content acquired in the training site to the job” The main attributes of the failure of an effective employee training transfer are: (1) lack of generalization to other contexts of the learned content; or (2) lack of maintenance of learned material over a period of time on-the-job. In order to know how an ineffective training transfer looks like, we need to observe these

108

two main attributes in all the cases to be considered in the set of the negative pole of the outcome. Thus, the structure of our concept is of substitutability of two main attributes which constitute the concept. Negative pole In our definition, we stated that the two attributes of the concept need to be together to make it exist. Thus, the sole absence of one attribute implies the absence of the concept or the ‘failure of an effective employee training transfer’. In our study, we are not going to study the conditions for the absence of the outcome, but solely the conditions for an effective employee training transfer and hence, the causal mechanisms linking these conditions and the positive pole of the outcome, within certain necessary contexts. With these concepts at hand, we can build up the ontological view of the concept ‘failed effective employee training transfer’ as follows: “The failure of an effective employee training transfer is the lack of generalization to the job context or lack of maintenance of the content acquired in the training site to the job” The main attributes of the failure of an effective employee training transfer are: (1) lack of generalization to other contexts of the learned content; or (2) lack of maintenance of learned material over a period of time on-the-job. In order to know how an ineffective training transfer looks like, we need to observe these two main attributes in all the cases to be considered in the set of the negative pole of the outcome. Thus, the structure of our concept is of substitutability of two main attributes which constitute the concept.

A.3.3 CONDITIONS AS CAUSES Within the transfer literature there is an abundance of studies that investigate the role of different factors that might influence the transfer process. However, a drawback of the literature is that currently most of the analyses and conclusions are not formulated in a set theoretical manner. They do not unambiguously state which context elements are necessary for the outcome to be present, or which combinations of causal conditions are necessary or sufficient for the occurrence of the outcome. The literature hints at possible factors that play a role, mostly in terms of correlations. Contrary to most of the literature, we approach the concepts in an ‘essentialist’ and causal view, actively considering their causal attributes when we speak of conditions and more in an ontological view (essentialist view of what constitute a concept) when we refer to the outcome. It is important to note that different attributes of training programs, organization and trainees can be considered as necessary contexts, causal conditions or as crucial parts of a mechanism, depending on the perspective taken. Therefore, some overlap between the sections might be present. Another element we needed to consider was the element of time within an educational cycle related to the development and implementation of training programs within organisations. Transfer is the (preliminary) outcome of a learning process that is affected by many factors that are present of absent even before the training starts. The literature review clearly indicates many factors which affect transfer without necessarily causing it. As such, we reasoned that especially the elements before and during the training might function as enablers rather than causes. We propose to treat them as more remote contextual conditions in a single QCA analysis. Consequently, these conditions will function either as causes or contexts in the follow-up process-tracing. They facilitate the transfer to the work floor after the training without itself being active causes of transfer. In other words: they are necessary requirements training 109

programs within organizations should have attention for. As such, they create the context in which transfer will eventually be performed, given the presence of the causal conditions. Supervisor support Most of the definitions mapped from the literature understand ‘supervisor support’ as ‘sources of encouragement, assistance, reinforcement, opportunities and guidance (feedback) for employees on their use of new knowledge at the workplace’ (Cromwell, 2004; Cromwell & Kolb, 2004; Holton 1997; Lancaster et al 2013; Quinones, Ford, Sego, & Smith, 1995; Richman-Hirsch, 2001; Salas & Cannon-Bowers, 2001; Van der Klink et al., 2001). This support can be understood as a behaviour (encouraging, reinforcing, providing) or a multidimensional involvement from the supervisor (Lancaster et.al., 2013) either, before, during, and after a training program takes place (Govaerts, 2017; Brinkerhoff & Montesino, 1995; Cohen, Underwood, & Gottlieb, 2000; Cromwell & Kolb, 2002; Lancaster et al 2013). In the literature, the supervisor support is also conceptualized in different types such as: instrumental, informational, emotional and appraisal (Nijman, 2006). Similarly, some scholars introduced the notion of ‘perception of supervisor support’ (Baldwin & Magjuka (1991), which is also in line with our approach on human perceptions. With these concepts at hand, we can build up the concept of ‘supervisor support’ as being involved in the whole process, as follows: “Supervisor support is the superior’s commitment to facilitate the retention and motivate the use of the acquired content in a training to the job by employees, during and after a training program takes place” In this evaluation project, the main attributes of supervisor support are: (1) superior’s commitment to motivate the generalization of learned knowledge by trainees to the job; (2) during and after training program. In order to know how the supervisor support looks like, we will observe all these three main causal attributes in the cases to be considered in the set of the positive pole of this condition. Thus, the structure of our concept is a conjuncture of three main attributes which need to be jointly present to make the concept of ‘supervisor support’ exist. In relation to the negative pole of the concept, we estimate that the concept does not exist when any of the necessary attributes is absent. It is important to highlight that the definition of ‘supervisor support’ covers the whole process of training, since retention of training content until the application of learned material and post-training is a prerequisite. Peer support According to the literature and the systematic mapping of definitions, most scholars understand ‘peer support’ as the ‘optimization of the trainee’s use of learning on the job by colleagues’ (Noe, 1986), which can also be a ‘perception’ thereof (Reinhold et.al, 2018) or a ‘behaviour’ understood as ‘to optimize the trainee’s use of learned material’, ‘reinforcement for trainee’s use of learning on the job’ (Russ Eft, 2002), or ‘encouragement (Martin, 2010). In the literature, peer support is also conceptualized in such a way that it triggers an enhanced learning transfer process, by considering feedback, encouragement, problemsolving assistance, supplemental information, coaching assistance as key parts (Hatala & Fleming, 2007; Gilpin-Jackson & Bushe, 2007; Jellema, Visscher, & Scheerens, 2006). Similarly, some scholars introduce the 110

notion of peer support as triggering some ‘motivational process to transfer’ which facilitates the enhanced skill transfer (Chauhan et al, 2016). With these concepts at hand, we can build up the concept of ‘peer support’ as follows: “Peer support is the colleague’s commitment for employees to improve the trainee’s learned content and stimulate the trainee’s use of learned material to the job” The main attributes of training transfer effectiveness are: (1) peer’s commitment for employees to improve the trainee’s learned content to the job, and (2) to stimulate the generalization of learned knowledge by trainees at the workplace. In relation to the negative pole of the concept, we estimate that the concept does not exist when any of the necessary attributes is absent. Sense of urgency Based on the available literature, we understand an employee’s sense of urgency as one’s (1) clear need to engage in training (2) because of the identification of a hiatus between current knowledge and skill and required knowledge and skill in the future (3), with the understanding that overcoming the hiatus is within reach of the capabilities of the employee. Before we referred to the often-mentioned recommendation of performing a training needs analysis in the training literature. It implies the identification of learning needs of the target groups and organization. Indeed, the importance of starting from a learner’s needs to undertake learning activities is highlighted by several scholars (Burke & Hutchins, 2007; Salas, Tannenbaum, Kraiger & Smith-Jentsch, 2012). Dochy & Segers (2018) argue that looking into the training needs as such might be not enough. To bring learners to a state of maximum involvement and intrinsic motivation for learning, one should feel the urgency to know something. One should experience a sense of urgency. This suggestion about the role for sense of urgency is based on motivational theories pointing to the notion of urgency as a powerful driver of learning (Dochy & Segers, 2018). As an example, they refer to the self-determination theory of Ryan and Deci (2000) in which intrinsic motivation is an important factor leading to impactful learning. The direct experience of urgency is presented as a strong intrinsic motivator, that will lead to intrinsically motivated employees. Dochy & Segers (2018) point out this sense of urgency can be originated from different sources: from identifying challenge or a problem that needs to be solved urgently within a project, a task of job, or from a perceived gap between what one can do and what is needed to do later in the future. Moreover, employees may identify a more general hiatus in their knowledge or behaviour that is not bounded to a specific task, usually associated with personal development. Also, employees may deem it important to update their knowledge and skills because of the company’s long-term strategy or goals. Nevertheless, it is important to note that not every challenge will act as a trigger for learning. Based on Csikszentmihalyi & Beattie, 1979), Dochy & Segers (2018) mention that a challenge will only spark learning when there is a balance between what the task demanded from the learner (the challenge) and the competences of the learner. When dealing with the challenge feels out of reach, there is no change to become intrinsically motivated to learn. Goal setting and relapse prevention as transfer enhancing interventions before and after training Within the literature on transfer of soft skills studies discuss the importance of so called transfer enhancing interventions, which focus on identifying both the barriers that hinder the application of learned skills in 111

the work environment, and the trainees' plans to overcome those barriers (Sookhai & Budworth, 2010; Botke et al, 2018). Although the reasons why and how are not very well documented, their execution is expected to facilitate the post-training training transfer. Botke et al (2018) identify three main types of such interventions: goal-setting, relapse preventions and program-framing. First, program-framing includes activities that are expected to lead to a more positive perception of the organisational climate, which in turn may increase transfer (Kastenmüller et al, 2012). Giving optimistic previews is mentioned as an example (Botke et al, 2018). In Hicks & Klimoski (1987) it is considered as a pre-training intervention, consisting of the following elements: outcomes expected from the workshop, content indicating specific topics and expected home-work, expected evaluation procedures, an indication of who should attend, and workshop leaders, dates times and locations (Russ Eft, 2002). As a second type, goal-setting is based on a theory of employee motivation regarding task performance (Morin & Latham, 2000). The theory states that if an employee has the requisite ability, then a difficult, specific goal not only influences the employee's subsequent behaviour through choice, effort and persistence but also affects behaviour cognitively through the search for knowledge of ways to achieve the goal. Goalsetting interventions involve either the actual setting of goals with regard to the implementation of new knowledge, skills and attitudes on the job, or the teaching of how to set such goals (Nijman, 2004). Relapse prevention lastly, is a self-management technique (also called self-management training) by which individuals can become aware of environmental and intrapersonal threats to skill maintenance to anticipate, prevent, and recover from possible lapses into the old behaviours (Botke et al, 2018). The focus is on promoting transfer of training by immunising learners against environmental obstacles to transfer. Marx (1982, 1986) and Burke (1997) described the full relapse prevention method as including the following actions by trainees: choosing a specific skill that they want to main, along with a specific, measurable skillmaintenance goal, defining what constitutes a slip and a relapse, identifying the positive and negative consequences of using the new skill, reviewing cognitive and behavioural transfer strategies, predicting the situation for the first slip, as well as strategies to deal with it, and reviewing a chart to track their progress on their skill-maintenance goal (Russ Eft, 2002).

A.3.4 CONTEXTUAL CONDITIONS AS ORGANIZATIONAL FACILITATORS OF TRANSFER Identical elements We consider a training design as one with identical elements when the training program mirrors the actual job experience (fidelity) in three domains (fidelity within): equipment, work environment and psychological sphere. Training designs may differ in many ways that affect the transfer behaviour of the trainees. One crucial characteristic for interpersonal and open skills, may be ‘identical elements’ (Burke & Hutchins, 2007), often defined as the extent to which stimuli and responses in the training setting are identical to those in the actual performance environment (Saks & Belcourt, 2006). For example, standardized training programs, provided by outsourced training providers, may disproportionately rely on general training design and delivery modes, ignoring the actual work context (Vanderlocht, Van Dam & Chiaburu, 2012). The alignment between training setting and the job has been studied in terms of training content and material, but overall current research has not focused in any detail in what ways stimuli can or cannot be aligned with the work context. In a meta-analysis on the effectiveness of leadership studies, Lacerenza et al (2017) provide more 112

insight in what identical elements in the training setting means based on the flight simulation book of Rehmann, Mitman & Reynolds (1995). They indicate that the alignment of the training stimuli with the actual work environment can be assessed through fidelity or the extent to which a training program accurately mirrors the work floor system. More specifically, training organisers need to have attention for fidelity within three domains: the equipment employed, the broader environment and the psychological sphere. Equipment fidelity is about the alignment between tools, technology, and other system features utilized in training and those used on-the-job), environment fidelity relates to the replication of the actual task environment in regard to sensory information, motion cues, and other features within the training program. Psychological fidelity refers to the degree to which task cues and consequences mirror those experienced on-the-job. Taken this more detailed conceptualization, we consider a training design as a training with identical elements when the training program mirrors the actual job experience (fidelity) in three domains (fidelity within): equipment, work environment and psychological sphere. The concept of identical elements was originally introduced by Thorndike and Woodworth (1901), while noticing that learning is a specific instance of mental adaptation which does not always generalise. They already implied what was picked up by Baldwin and Ford’s (1988) who suggest that trainings designed in such a manner that the training stimuli align with the actual work environment are more likely to encourage transfer. The idea behind training with identical elements is that the presence of similar stimulus and response elements in both settings (the training and work floor) leads to training of transfer (Nijman, 2004:26). It will be easier for employees to apply what has been learned in the training to the job setting, when the stimuli and responses in the two settings matches well (Vanderlocht et al, 2013). Similarity in stimuli increases the relevance of the training situation, and might help to trigger effective responses, emotions and decision-making, developed in training. The importance of identical elements was also confirmed in a meta-analysis on the results of leadership training (Lacerenza et al 2017). Additionally, Vanderlocht et al (2013) also found out that identical elements are influencers of the trainee’s motivation transfer. They argue that identical elements in the training design increase the expectations that the new skills and knowledge can be used back on the job and will lead to improvements in the performance (Burke and Hutchins, 2007; Noe, 1986). As such, the trainees might be more motivated to transfer after a training in which they experience identical elements. Training as aligned with wider strategic organisation goals Sitzmann & Weinhardt (2015) provide a multilevel theory of training engagement that comprises the temporal sequence of events and factors influencing one another and contribute to training effectiveness (understood as enhanced knowledge and skills and ultimately meaningful changes in job performance). It is based on motivational theories of behaviour (Locke & Latham, 2002). They claim that ‘goal establishment’ is the cause that triggers all motivated behaviour and see it as the first necessary phase of training engagement, necessary for training effectiveness. The step of establishing a goal is, however, not only an individual task. It also involves developing strategic training initiatives, which affect a host of factors that in the end will determine whether employees perceive that participating in training is supported by their organisation (Sitzmann & Weinhardt, 2015). Strategic training initiatives developed by the HR system are considered to be a powerful source of competitive advantage that enhances training effectiveness, as they make the link between the business strategy and employees (Sitzmann & Weinhardt, 2015). If the HR system sets clear training goals, and clearly communicates about the importance of training for employees, training effectiveness in enhanced. 113

The organizational environment is the context in which work-related action is perceived and interpreted by employees to formal and information practices, perceptual filtering and collective sense making (Ostroff & Bowen, 2000). It will also act as the context in which the supervisors and colleagues act and react towards the employee following the training and might be more or less supportive towards the training. More concretely, though talking about organizational policies rather than training, Baert, De Witte, Govaerts & Sterck (2011) describe a strategic training policy as a more less explicit and systematic whole of big decisions about learning, training and education of the employees within the organisation, with the aim to stimulate knowledge acquisition and the development of skill, competences and talents, in such a way that it enables the employees and the organisation to keep viable and liveable in the near and far future, and to continue reaching the (changing) goals of the organisation and the career expectations of the employees. As such, a strategic policy is embedded and aligned with the goals of the organisation. The authors distinguish it from an administrative and reactive training policy. The former focuses more on cost, the timing of training and decisions related to training conditions (working time, replacement) and facilities (location, …), the latter makes sure the employees have the competence that are deemed necessary for their current function and tasks. Compared to administrative training policies, reactive one does first explicitly survey the needs and reflect on priorities. However, it reacts more to shortages in the short run and manifest problems. A strategic training policy is both reactive and proactive and used to a lever to support the expected change processes in the organisation. They not only systematically detect which shortages of insights, competences and attitudes are currently lacking among the employees, but also look at the future and determine which talents and competences shall be necessary in the future. Interestingly, Baert et al (2011) indicate that within a strategic training policy the relevance and transfer of what is learned are attention points of the implementations and learning trajectories. Additionally, it is supported by all stakeholders, so there is support and engagement for learning. Based on the foregoing we would understand a training as aligned with the wider strategic goals of the organisation when the training program is 1) explicitly connected with the organisations strategic needs, by which it anticipates on what will come in the (near or far) future of the organisation, and 2) the relevance and transfer of what is learned are attention points within the training implementation. Training policy as a voluntary training attendance policy Companies differ in their training attendance policy: participation in training can be voluntary or mandatory (Blume, Ford, Baldwin & Huang, 2010). Meta-analysis suggests voluntary attendance increases transfer of soft skill training (Lacerenza et al, 2017). The role of a voluntary attendance policy may partially be explained by self-determination theory (Ryan & Deci, 2000), in which autonomy is argued to stimulate motivation (Lacerenza et al, 2017). Providing trainees with a choice to participate in training, foster the learner’s agency (Dochy & Segers, 2018), and might satisfy the need for autonomy. The latter may create a context where a trainee only participates in training when he or she perceives value in the training and is motivated to learn and transfer. Increasing the feeling of autonomy might also be a necessary context for mechanisms at the employee level to be activated (see infra). As such, we will actively consider whether the training attendance policy is voluntary; main attribute is employees can participate in training voluntarily, when they perceive value in the training.

114

A.3.5 CONTEXTUAL CONDITIONS RELATED TO THE TRAINING DESIGN Training designed as ‘active learning instructional method’ Dochy & Segers (2018) present seven building blocks that should stimulate ‘HILL’ or High Impact Learning that Lasts. One of the proposed building block is action and knowledge sharing, often translated into active learning. The idea of active learning, however, dates back to the 1960’s when Dewey’s (1938) ideas about ‘learning by doing’ were argued for. He understood that learners sitting back in their seats and just consuming knowledge, skills and attitudes is an ineffective learning method. Later in the 1980s these ideas were picked up in experiential learning and action learning (Kolb, 1984; Revans, 1982). In the 1990’s when constructivism became the dominant paradigm in the learning sciences, the importance of the active construction of meaning and reflection were also emphasized (Dochy & Segers, 2018). Practice opportunities should require trainees to engage in the same cognitive processes they will need to engage in whey they return to work, which often requires providing meaningful challenges (Salas, Tannenbaum, Kraiger & Smith-Jentsch, 2012). Based on this a training design based on active learning methods is hypothesized to be necessary for a training to lead to learning. As such, we consider it as a necessary context attribute of the training design. Although there is broad support for active learning methods, Prince (2004) correctly notes there exist different interpretations of the concept. In the article, Prince describes more generally accepted definitions and highlights some common distinctions. Active learning is generally defined as any instructional method that requires students to engage in meaningful learning activities and reflect on what they are doing (Prince, 2004:1). These activities refer to what happens in the classroom. It is contrasted to passively receiving information from an external source such an instructor. Different instructional methods are meaningful with regard to the subject being learned. Often mentioned instruction methods under the denominator of active learning are collaborative and cooperative learning, and problem-based learning, each of which represent and stress different elements of active learning (Prince, 2004). Collaborative learning refers to instructional methods in which trainees work together in small groups toward a common goal. Central is the interaction between students, instead of learning as a solitary activity. In the same realm, cooperative learning studies define the latter as a structured form of group work where students pursue common goals while being assessed individually. The core elements held in common among different cooperative learning models are cooperative incentives rather than competition to promote learning. Lastly problem-based learning is an instructional method where relevant problems are introduced in the beginning of the training to provide context and motivation for the learning that follows. Central elements of active learning instructional methods are student activity and engagement within the learning process (Prince, 2004). Dochy and Segers (2018) translate this into learner agency and inductive learning processes. Learner agency refers to students taking responsibility for their learning process and controlling the learning decisions they take related to learning goals, monitoring progress, deciding on new paths). They internally regulate their learning experience. Inductive learning processes involve inductive processes in which learners explore and experiment with tasks in order to understand and grasp general concepts, procedures, rules and strategies that lead to effective performance. In active learning methods learning is seen as a process of linking practice to concepts through different iterations and is created and further deepened through discussions and/or collaboration (Dochy & Segers, 2018:51). Central elements 115

are: exploring, experimenting, testing, formulating hypotheses, evaluation, making errors and learning from them, planning, reflecting and monitoring. To conclude, within this project, the main attributes of a training designed as ‘active learning instructional method’ are: (1) a learning method with (2) active student engagement through meaningful practice and (3) reflection on what has been learned and encountered. Length of training Length of training is a context variable that might affect the effectiveness transfer. Taylor et al (2009) studied the influence of training length of the effect size (measured as the difference between post-test and pre-test scores regarding on-the-job behaviour after training divided by the pre-test standard deviation). These authors found that for short managerial training (one day or less), the effect sizes were small; long programs more than five days were found to have modest effects. Spacing of training sessions Cognitive load theory, as a learning efficiency theory, posits that learners have a finite working memory capacity and once this is met, processing and learning abilities are hindered or lost entirely (Lacerenza et al, 2017). Consequently, there is a need for training programs that are designed to reduce extraneous cognitive load while increasing learner's ability to process salient information and still presenting all the relevant information. One way to do this is to temporally space training sessions, a technique known as spacing. Mode of instruction: internal or external trainer, or self-administered training Kalinoski et al. (2013) posit that the trainer’s background can influence trainee motivation in such a way that programs with an internal trainer will result in increased levels of trainee motivation in comparison to a program with an external trainer, especially if the trainer is a direct manager of the trainee. Within a training program with an internal training, trainees may perceive the organization’s support for the training to be greater because they have a dedicated person on staff who is responsible for the training program. However, trainees participating in a leadership training program organised by an external trainer might also perceive the organization as valuing training because they have paid to bring in an expert (or paid to send the employee to a leadership center) (Lacerenza et al, 2017). On the contrary, a self-administered (leadership) training program could signify to trainees that the organization does not fully support their training because fewer resources are invested in comparison to training programs with an instructor. Because trainees are required to complete the leadership training on their own, they may be less motivated to exert effort as they might believe the training is not valued by the organization (Blume et al., 2010). Based on a meta-analysis of the effectiveness of leadership programs, Lacerenza et al (2017) indeed caution practitioners from using self-administered training programs as the authors conclude those kinds of training appear to be less effective in terms of learning and transfer.

116

A.3.6 CONTEXTUAL CONDITIONS RELATED TO THE EMPLOYEE LEVEL Learner readiness Learner readiness is “the extent to which individuals are prepared to enter and participate in training” (Holton et al, 2000). It includes for example, freedom of choice to attend training and perceptions of the relevance of training. Employees who are free to decide whether to take part in training are expected to develop a greater appreciation of that training than employees who are obliged to take part, possibly resulting in enhanced motivation to learn (Elangovan & Karakowsky, 1999). Similarly, employees who perceive a training program to be relevant to their job are expected to be more motivated to learn. Content acquired in training One of the strongest antecedents of training transfer is what trainees learned during the training (Dochy & Segers, 2018). In order for transfer to occur, there needs to be new knowledge or skills that can be learned. In that sense, Baldwin & Ford (1988) pointed out the amount of learning that occurs during the training program is necessary for transfer to be able to take place. Workload allowing training and transfer A structural factor that might facilitate (or to the contrary impede training participation and transfer) is workload. More specifically, Russ Eft (2002) considers an employee’s workload as the personal capacity to transfer; includes factors such as role conflict, overload, and job-generated stress. Mastery learning goal orientation Goal orientation is the mental framework that one uses to interpret and then shape how to behave in learning-oriented environments (Salas et al., 2012). There are two forms: mastery orientation and performance orientation. Trainees with strong learning (or mastery) orientation seek to acquire new skills and master any novel situations. They exert more effort in learning, engage in more adaptive metacognitive strategies, stay on task after receiving feedback and demonstrate stronger learning outcomes. Conversely, trainees with a strong performance orientation seek to achieve better scores, avoid engagement in situations in which they may fail, and want to be perceived as capable and thus may learn less during training. Personal factors Personality traits (Baldwin and Ford, 1988; Colquitt et al., 2000; Piezzi, 2002), (work-related) attitudes (Cannon-Bowers et al., 1995; Colquitt et al., 2000; Noe, 1986; Piezzi, 2002; Rank and Wakenhut, 1998), expectations (Noe, 1986; Rank and Wakenhut, 1998) are among the additional factors that can be considered.

117

ANNEX 4: CONCEPTUAL FRAMEWORK: COMBINING QCA WITH PROCESSTRACING METHODS. We introduce the guidelines of concept formation when combining QCA and Process-Tracing. We present the different steps that are taken within the project to conceptualize the outcome and conditions. Within multi-methods research, it is prerequisite that conditions and outcome are formulated in a way they are applicable for all the methods applied. For our design specifically, in which we apply a QCA-first design, it is important to ensure that the conclusions drawn from the QCA are informative for the following ProcessTracing analysis. For Process-tracing, the conditions need to be formulated in a causally relevant manner, showing a way how they can contribute to the production of the outcome. Most of the guidelines for PT are compatible with QCA with the exception that in QCA we also focus on the ‘negative pole’. However, for the PT analyses we will focus only in the ‘positive pole’ of the concept for the data collection process in a post-QCA phase.

Four visions to define the concept compatible with Process-Tracing (Goertz, 2005; Beach & Pedersen, 2016; Beach forthcoming, 2019).

The claims we make in process-tracing, as with other case-based methods, are asymmetric claims which means we make claims about the causes of the outcomes, but we are not at the same time making claims about what causes the absence of the outcome. This means we need to define our concepts in a settheoretical fashion (Beach and Rohlfing, 2016; Schneider and Rohlfing, 2016). Set theory as used in social science methodology defines causes and outcomes in terms of the attributes that determine whether a given case is a member of the set of the concept, and theoretical relationships between causes and outcomes as subset relationships (e.g. a necessary condition is one where the cases that are members of the outcome are a subset of cases that are members of the necessary condition). Cases that are not members of the concept (i.e. the attributes that define the positive pole) are just ‘everything else.’ This translates into four elements are central to the conceptual approach applied within the project. Related to the outcome, in order to be compatible with PT we need to discuss the ontological view and the causal view. Furthermore, it is insightful to stress the realist and empirical perspective in which we work. Lastly, the definition of contexts is also made explicit. 1. Ontological view of the outcome (the constituents part of a phenomenon ‘essentialist position’; how the phenomenon looks like). In order to conceptualize it, one needs to think in terms of “what does the ‘phenomenon’ mean? E.g. what is training transfer effectiveness? and What is not transfer effectiveness? and not what causes a successful training transfer. How can we recognize the existence of training transfer in a case? We need to unpack the outcome in terms of attributes/dimension that constitute the outcome itself. Outcomes need to be defined as something that can be produced or influenced by the preceding mechanism and causal conditions. In QCA we need also to conceptualize the ‘negative pole’, e.g. What transfer effectiveness is not? In Figure 6, we can observe the process to conceptualize qualitative concepts within a multimethod research design: 118

Figure 6: Defining ontological concepts for the combination of QCA and PT

Abstract ontological concept

Concept defined (theoretical definition)

Measure of concept (observable manifestation)

Measure of the concept (case specific interpretation)

Phase 3 – Assessing case membership Interpreting whether observable manifestations are actually present/absent in particular cases using ‘contextual’ knowledge

(Source: Based on Beach 2016)

2. Causal conditions (see Figure 7). Causal conditions need to be defined in terms of attributes that can trigger a mechanism (or set of mechanisms) to produce the outcome. We only need to include attributes than are ‘relevant’ as those most used by Scholars. For PT, we only need to focus on the ‘positive pole’ of concepts, and conceptualize it, as including the attributes for a case to be a member of the concept. For QCA, we need also to conceptualize the negative pole of the concept and measurement. Concepts should be defined thickly due to the need to create as causally homogeneous a population of cases as possible by creating contextually specific defined concepts. Finally, the definition should be compatible with mechanistic claims.

119

Figure 7: Defining causal concepts for the combination of QCA and PT

Abstract causal concept

Concept defined (theoretical definition)

Measure of concept (observable manifesttaion)

Measure of the concept (case specific interpretation)

Phase 1 – Defining concepts QCA & PT -Defining (+) pole and threshold of concept membership. - Define (-) pole and threshold of concept membership. -Attributes of positive and negative pole Phase 2 – Measure Choosing what observable manifestations define set membership of concepts Phase 3 – Assessing case membership Interpreting whether observable manifestations are actually present/absent in particular cases using ‘contextual’ knowledge

(Source: Based on Beach 2016)

Both for outcome and conditions and once we have identified the attributes of each concept, we need to establish the relationship between the dimensions in terms of: conjuncture (logical AND); Union (logical OR), or combination of both (substitutability). With the AND structure we can have a conjunction of attributes to build up a concept, but we can have less cases as member of the set of that concept. With the OR structure our concept will have less attributes, and we will have more cases in the set of the concept. With a AND/OR combination we will have cases in both structures.

3. Contextual conditions. Contextual condition can be understood as factors that determine whether a causal relationships functions as theorized (Beach & Pedersen, 2016). Contexts are not causes but ‘enablers’, that contribute to the outcome but does not cause it. While in QCA, the distinction between contextual and causal conditions is only there implicitly, a mechanism-based understanding of causation demands the researchers to actively distinguish between the two. In a mechanism-based understanding, a cause is defined as something that triggers a mechanism that in in a productive relationship with the outcome. A cause ‘does something’, whereas the contextual condition is the enabler. In Process-Tracing, contexts need to be necessary to the mechanisms operate correctly, therefore one needs to choose only those contexts that are necessary to perform analysis with process-tracing techniques. Two-step QCA proofs itself as a good medium for this selection. 4. Realist and empirical view. The phenomenon is observed in the reality (i.e. empirical phenomenon (Goertz, 2005). ii.

Defining concept – A three step approach 120

To define concepts, a three step approach a proposed by Beach & Pedersen (2016) is followed: 1. Mapping existing theories (How other scholars have defined the concept that one is defining). This strategy is only inspiring and help to guide the conceptualization, but we do not need to be trapped by them. The only attributes of the concept to be choose are those we expect to be relevant for the existence/non-existence of the outcome (if the conceptualization is dichotomic or a ‘continuum’) and ‘causally’ relevant for producing it. For instance, in terms of causally relevance, we can ask ourselves “What attributes are causally relevant to include if we are theorizing that the condition A is necessary to produce the outcome Z?”. 2. Brainstorm about which attributes are causally relevant for the research question. Which attributes might define one’s concept using existing definitions as sources of inspiration. What aspect of the causal condition can be a cause of the outcome? What is it about the concept that can be a cause or outcome? What are the causally relevant attributes of an organization structure that might produce the successful training transfer, in a situation where they otherwise could have gone to the failure? Additionally, when defining attributes, the terms utilized should be as unambiguous as possible. 3. Choose the attributes that are causally relevant and develop how they relate to each other (concept structure). After the brainstorming strategy, when selecting the attributes, we expect to use in our concept, there should be a link between the terms used and the meanings of the attributes that are causally relevant, e.g. the attributes of a concept should coherently fit together. Do different attributes have contrasting causal effect? If so, we need to disaggregate the defined concept in subtypes. It is also important to be sure that the attributes of the outcome are not also included in the definition of the cause. Once we have all the attributes identified we need to develop how each other are related in terms of MINUS or PLUS concepts, by theorizing the structure with the logical AND, OR or both.

121

ANNEX 5: PROCESS-TRACING METHODS We use a systems-understanding of mechanism (Glennan, 1996, 2002; Bunge, 1997, 2004; Cartwright, 1999; Machamer et al., 2000; Russo & Williamson, 2007, 2011; Craver & Darden, 2013; Illari & Russo, 2014), where “the core elements of a causal mechanism are unpacked theoretically and studied empirically in the form of the traces left by the activities associated with each part of the process” (Beach & Pedersen, 2019:3). Thus, each part of the mechanism is described in terms of entities that engage in activities (Machamer et al., 2000; Machamer, 2004; Beach & Pedersen, 2013; Beach & Pedersen, 2019), where entities are “the factors (actors, organizations or structures) engaging in activities, whereas the activities are the producers of change or what transmits causal forces or powers through a mechanism” (Beach & Pedersen, 2019: 34). In order to apply process-tracing, we need to focus on its three core components (crf. Beach & Pedersen, 2019), that will be developed in this section: (1) Case selection for process-tracing, in order to facilitate the generalization of findings from single case studies to other causally similar cases; (2) The theorization about causal mechanisms linking causes and outcomes; and (3) the analysis of the observable empirical manifestations of theorized mechanisms. Case selection and future generalization We use process-tracing to understand how a given mechanism works within a bounded population of causally similar cases. Note that all our positive cases of training transfer effectiveness could be ‘typical’ cases, because they are positive on the combination of condition (causes), on the outcome, and on the contextual condition that might affect how a process works. These typical cases are, therefore, positive with causal homogeneity at the level of combination of conditions and we expect to see causal homogeneity also at the level of mechanism. As process-tracing “involves the detailed empirical tracing of the operation of mechanisms within an individual case. Mechanistic evidence then either confirms or disconfirms our theories about the operation of a causal mechanism in the studied case” (originally referred to Illari 2011, see Beach & Pedersen, 2019). Theorization about causal mechanisms linking causes and outcomes We use theory-testing process-tracing with the aim of testing theories of causal mechanisms that can in principle be present in our whole population of positive cases (see Table A14). We start by conceptualizing plausible hypothetical causal mechanisms based on existing theorization and empirical research, in a more unpacked form of a system (system-understanding of mechanism). We theorize each of the constituent parts of the mechanism in terms of entities engaging in activities that provide the causal link for the next part of the mechanism, enabling to discover ‘how it works’ (Craver & Darden, 2013: 83-95).

122

Table A14: Theory-testing variant of process-tracing Theory-testing process-tracing Research purpose

Is hypothesized causal mechanism present and does it function as theorized?

Analytical focus

Theory-focused

Source: Adapted from Beach & Pedersen (2019: 9) The level of abstraction of our mechanisms is pitched as mid-range mechanisms (Beach & Pedersen, 2019), and formulated in a way that they can in theory be present in many different cases within our bounded population (e.g. focusing mainly on the most critical elements that are shared across a range of similar [team] cases) but keeping the essence of describing key interlocking parts between a cause and the outcome, and exhibiting some form of productive continuity (Machamer et al., 2000; Machamer, 2004; Mayntz, 2004; Beach & Pedersen, 2016a). The theorized causal mechanism is then “operationalized in terms of developing propositions about potential empirical fingerprints that might have been left in a given case by the activities associated with a mechanism and its parts” (Beach & Pedersen, 2019: 9). The mechanistic evidence is formulated as predictions as clearly as possible, “making it easier to determine whether or not they are then actually found in the subsequent case study” (Beach & Pedersen, 2019:10). We collect then and assess the available empirical record to determine whether there is mechanistic evidence suggesting that the mechanism was present and operated as theorized, or whether the theory needs to be modified. Thus, according to Beach and Pedersen “If the predicted evidence is found, we can then infer that the hypothesized causal mechanism is present in the case and worked as we theorized” (2019:10). In contrast, a theory-building variant of process-tracing would need to be selected when “evidence is not found for a given part (or for the overall mechanism)” (Ibid.) consisting of an abductive process, where the researcher can utilize the “insights gained from the empirical analysis of what went wrong as inspiration for building theories of new parts of the mechanism” (Ibid.). The inferences produced with theory-testing process-tracing will be related to whether a causal mechanism – through its constituent parts – was operative and functioned as expected. Middle-range ‘in-depth theory-testing process-tracing’ In order to conduct an in-depth theory-testing process-tracing analysis, we theorized a plausible mechanism in such a way that enable to get productive continuity between the combination of conditions and the outcome, but with a mid-range theoretic approach with the purpose to be generalizable to other similar cases (add the level of abstraction of the CM to be generalizable). Our goal here is to dig deeper into how things works in particular typical cases by unpacking causal mechanisms into parts composed of entities engaged in activities, operationalizing empirical fingerprints for each part, and then tracing empirically whether there is actual evidence that the mechanisms worked as hypothesized in the typical 123

case (Figure 8). Further, by tracing each part of the mechanisms empirically using mechanistic evidence, we expect to make stronger causal inferences about how causal processes actually worked in the typical cases of training transfer effectiveness (Russo & Williamson, 2007; Illari, 2011). The process in an in-depth theory-testing process-tracing consists of: (1) conceptualizing the cause and outcome in a set-theoretical manner, as well as the causal mechanism hypothesized to link the two together; (2) Operationalizing the causal mechanism by translating theoretical expectations into casespecific propositions (expected evidence) about what empirical fingerprints the activities linked with each of the parts of the mechanism left if they are actually operating as theorized in the case. The prior confidence in the causal mechanism being present in a given case also may have to be developed as this determines the strength and type of empirical tests that should be utilized; and (3) collecting and evaluating the evidence, where the empirical material is gathered to establish whether the predicted evidence (proposition) was present or not, and then to evaluate it in context to determine whether the predicted evidence for each part of the mechanism was actually found and whether it can be trusted. Figure 8: The three steps of an in-depth theory-testing process-tracing

Source: Beach & Pedersen (2019)

124

Conceptualization of causal mechanisms The conceptualization of causal mechanisms involves detailing each of the parts (entities engaging in activities) between the conditions and the outcome that transmits causal forces through the mechanism to produce the outcome. Conceptualizing in these terms enables us to theoretically capture the actual theorized process whereby causal forces are transmitted through a causal mechanism to produce the outcome; these forces are black-boxed in both frequentist and set-theoretical causal theorization. Further, since causal mechanisms vary on the temporal dimension according to the time horizon of the causal forces that produce the success of team problem solving and the time horizon of the manifestation of that outcome, we will theorize causal relationships in terms of short, tornado-like time horizons (Pierson, 2003, 2004), both regarding the cause, mechanisms, and the outcome (see Table 15).

Table 15: The temporal dimension of causal mechanisms Time horizon of outcome Time horizon of

Short

mechanism producing an outcome

Long

Short

Long

Normal

Cumulative effects

‘Tornado-like’

‘Meteorite/extinction’

Incremental

Cumulative causes

‘Earthquake-

‘Global warming’

like’ Source: Beach & Pedersen, 2019 (Adapted from Pierson, 2003: 179, 192)

The level of theorization of the causal mechanisms will be at the individual/mezzo levels. Although many studies privileges microlevel theoretical explanations at the level of individual actors, the theory of training transfer as organizational phenomena is also multilevel being not reduced solely to the singular actor level. In this research, instead the causal mechanisms are better theorized at the individual level as being the actor who receive the direct impact of a training program. We shared the Hedström and Swedburg (1998) vision that there are no purely macrolevel mechanisms, because as Coleman (1990) and George and Bennett (2005) argue, the human interaction has microlevel causes, because they are “processes through which agents with causal capacities operate” (George & Bennett, 2005:137-142). In addition, as suggested by Beach and Pedersen (2019) we adopt a more “pragmatic middle-ground position, where the choice of level that is theorized is related to the level at which the implications of the existence of a theorized causal mechanism are best studied” (2019: 50). Our two causal mechanisms can, of course, operate at different levels of analysis as mentioned above, and the level at which they have been theorized is determined by the theoretical tradition within which one is working.

125

Two-stage evidence evaluation framework We have selected to use a Bayesian-inspired, two-stage evidence-evaluation framework that allows us to assess the presence of mechanistic evidence either confirming or disconfirming the operation of the theorized causal mechanism and its parts. Beach and Pedersen distinguish between theoretical and empirical evaluations of evidence. The first one, it is a reflection about the empirical fingerprints that the evidence-generating process of the parts of mechanisms should leave within the general context of the relevance of the magnitude of the dissonance for the successful dissonance reduction process. We reflect about what fingerprints we might have to find (theoretical certainty), and whether there are alternative explanations for finding a fingerprint (theoretical uniqueness). The second one, it is the observation of “whether the posited fingerprints are actually present in the accessible empirical record, determining what the found/not found observation means and whether we can trust it” (Ibid.). Figure 9 illustrates a two-stage evidence-evaluation framework for turning empirical material into evidence of mechanisms. Based on Figure 9’s logic, we need to ask ourselves three core questions when “trying to figure out what evidence in theory can tell us about the operation of a mechanism” (Beach & Pedersen, 2019: 158): (1) What is our prior confidence in the mechanism?; (2) Do we have to find the posited observables (theoretical certainty)? And (3) If we find them, are there alternative explanations for finding the observables other than the mechanism or part was operating as theorized (theoretical uniqueness)?

Figure 9: Two-stage evidence-evaluation framework for turning empirical material into evidence

Source: Beach & Pedersen (2019: 156)

Basic concepts Before engaging in a two-stage evidence-evaluation framework, it is essential to define the basic concepts associated with it, for a better understanding of the next phases. These concepts are: mechanistic evidence, proposition, types of mechanistic evidence, theoretical uniqueness and certainty and priors.

126

The mechanistic evidence One of the core concepts in Process-Tracing is as of that ‘mechanistic evidence’, which means the relevant evidence where a given form(s) of “empirical material can act as mechanistic evidence in case-based designs” (Beach & Pedersen, 2019: 171. Similarly, the ‘evidence’ itself can be considered as any “type of material that might be left by the workings of our theorized causal mechanisms that enables us to say something about whether or not the relationship was present in a case” (Ibid.). Propositions In Process-Tracing, a proposition is an “hypothetical empirical fingerprints of mechanisms” (Beach & Pedersen, 2019: 172) and the term ‘evidence’ is more appropriate for the empirical material once it has been theoretically and empirically evaluated for what inferences it can enable (Ibid.). Type of mechanistic evidence There are four types of mechanistic evidence: pattern, sequence, trace, and account. According to Beach and Pedersen, pattern evidence “relates to predictions of statistical patterns in the empirical record”; sequence evidence refers to “the temporal and spatial chronology of events that are predicted by a hypothesized causal mechanism”; trace evidence refers to the “mere existence provides proof” and finally, account evidence “deals with the content of empirical material” (2019: 172). This latter can be meeting minutes, oral account, discourse in speeches, interviews, etc. Theoretical certainty When engaging in Bayesian reasoning for evaluating mechanistic evidence at the theoretical level, a core concept arises, which is the ‘theoretical certainty’ (see Table A16). This concept relates to the “disconfirming power of evidence” (Beach & Pedersen, 2019: 180). We deal with ‘disconfirming power of evidence’ in two different situations: (1) high theoretic certain that the evidence-generating process associated with an activity should have left an empirical fingerprint and absence of it in the case. Our confidence in the hypothesis is downgraded; (2) when engaging in an empirics-first study where the empirical material is present (found out), the certainty is not evaluated because it is already there, rather the “theoretical certainty is thought of as the rate of false negatives” (Ibid.), which means that “if it is highly unlikely that we do not find the proposed evidence, not finding it would suggest that the mechanism (or part) is not present” (Ibid.). Theoretical uniqueness Similarly than above, in Bayesian reasoning engagement, the theoretical uniqueness refers to “the expected probability of finding the observable empirical fingerprints if the mechanism (or part thereof) does not exist, telling us about the confirmatory power of evidence” (Ibid). It can also be defined “as the rate of false positives” (Ibid.) which means that “If it is highly unlikely that we would find the evidence without the mechanism (or part) being operative, then finding it is confirmatory to some degree” (Ibid.).

127

Table A16: Probabilities expressed in Bayesian reasoning Prior/posterior confidence

Likelihood ratio

Higher levels, more confident that Level of certainty (higher levels, mechanism present based on more certain to find) existing knowledge Lower levels, less confidence that Level of uniqueness (lower levels, mechanism present based on more unique if found) existing knowledge Source: Adapted from Beach & Pedersen (2019: 179) Priors confidence Priors are a key aspect of Bayesian reasoning when applying case study research. Prior confidence in a causal mechanism means that we combine “our assessment of our confidence in its validity based both on existing research about the overall causal relationship (at the population level and, if it exists, within the chosen case), and about the particular mechanism (both within the case itself, but also more broadly)” (Beach & Pedersen, 2019: 182). According to Beach and Pedersen, the level of prior confidence affects the focus on developing confirming (i.e., theoretically unique predictions) or disconfirming (i.e., theoretically certain predictions) evidence. Thus, we have two different situations: (1) when our prior confidence is high, only very strong confirming evidence would further increase our confidence. Therefore, we should focus on disconfirming evidence; (2) when our prior confidence is low, weak confirming evidence will update our confidence (Ibid.). We should engage in an “assessment of the plausibility of the presence of a given theoretical mechanism and of its working as theorized in a case” (Ibid.) in order to determine the prior confidence.

128

ANNEX 6: CAUSAL MECHANISMS As we have seen in section two, the research design is one based on a multimethod, which combines QCA and Process-Tracing techniques. We have explained the different conditions included in this research as well the causal process in the form of a causal mechanism. We have argued that the explanation QCA provides for training transfer effectiveness is limited in the sense that we cannot see what happens in the black box. For this reason, we have tried to understand the process behind this transfer by unpacking such processes as causal mechanisms in a productive way to understand how training transfer becomes effective.

KEY CAUSAL MECHANISMS Based on (1) a literature review combined (2) with the collection of empirics from the field as well as (3) feedback from experts in process-tracing (prof. Derek Beach) and experts in training and learning (Prof. Filip Dochy), we identified four causal mechanisms that could be triggered by the causal conditions described in section 3. These causal mechanisms may be more likely to take place when the four contexts previously defined are also present in the case. The causal mechanisms related to the four causal conditions (supervisor support, peer support, relapse prevention and goal setting, and sense of urgency) are signaling and retention, enhanced training transfer, self-management intervention and learner agency. These four mechanisms act in the learning and performance stages of training. Their conceptualization is conceived as a complex mechanism (as a system) that concerns the dynamic transmission of causal forces which can produce the outcome (training transfer effectiveness). The causal mechanisms are treated as middle range theories and are expected to be present in the population of cases of Flemish firms when certain contexts related to the training design, job organization, work climate and trainee characteristics are present. Once we have theorized the causal mechanisms which links a cause (or set of causes) with an outcome in a particular context, how can we study them empirically? Given that we have theorized the steps of the mechanism, we should then empirically assess whether each step operated as we theorized (see section 6 and Annex 11 and 12). In what follows, we systematically discuss the four causal mechanisms. Signaling and retention Signaling and retention acts in learning and performance stages of training. We disentangle the process as a complex mechanism consisting of a single pathway: a cause (supervisor support) that trigger the mechanism consisting of six building block and seven parts). Cause: Supervisor support We have theorized our causal condition as superior’s commitment to facilitate the retention and motivate the use of the acquired content in a training to the job by employees, during and after a training program takes place.

129

As observable manifestation of this condition, we expect to find evidence in the empirical record of supervisor support in the form of supervisors encouraging trainees to share what they've learned in training with people in their work environment. Similarly, we expect to see observables manifestations of discussions between the supervisor and trainees about how to apply competences to job situations; supervisors giving coaching advice and useful feedback after training on the application in the job of what learned when required. Finally, we also assume that the supervisor trusts that the trainees are capable to successfully apply what he or she has learned). Block: Ascribing importance to training Part 1 – Theorization: Supervisor ascribes importance to the training program and takes initiative to let the employees follow the training. Fingerprints: If part 1 of the theorized causal mechanism exists in a case, we expect to find evidence of supervisor’s engagement with the training and trainees. Possible evidence can be the supervisor who takes initiative to let the employees follow the training and signs that reveals the ‘importance’ of the training for him/her. We expect that this can take the form of account evidence as well as trace evidence of actions which the supervisor has taken. Part 2 - Theorization: Employees react by putting the training in their agenda. [because they do not have choice]. Fingerprints: If part 2 of the theorized causal mechanism exists in a case, we expect to find evidence on employees reacting by accepting some sort of invitation to attend the training. We assume there could be both account evidence of this as well as trace evidence. Block: Employees follow the training Part 3a – Theorization: Everybody follows the training in group, in part because it was mandatory to do so by the supervisor. Fingerprint: If part 3a exists in a case, we expect to find that, the supervisor makes everyone (all employees) follow the training together. Block: Facilitating learning climate Part 3b – Theorization: In parallel supervisors enable employees to follow the training by taking over the workload during the training period. As such, employees can focus on learning the training content. Fingerprint: If part 3b exists in a case, we expect to find evidence of supervisors arranging the workload of the employee to be taken over during the training period so that the employee can focus on the training. This can take the form of e-mails where these arrangements are discussed or verbatims provided by trainees. We expect to find account evidence and also trace evidence to measure this proposition.

130

Intermediate outcome Theorization: This create a sort of organizational climate where employees perceive the importance of the training for their job, and where they acknowledge the engagement of the supervisor encouraging this goal. Observable manifestation: We expect to find evidence of employees who perceive the training to be important for their job or acknowledge the engagement of the supervisor. We expect that this can take the form of account evidence. Block: Motivation to generalize Part 4 – Theorization: Because the relevance of training is perceived, employees following the training feel motivated to use the learned content and discuss with peers. Fingerprints: We expect to find evidence on the motivation/inspiration of employees. We expect to find this mainly in account evidence. Part 5 – Theorization: Employees try out/use the training in tasks-related matters keeping the level of motivation that 'they just have to try it to learn' within an environment of trust and cohesion. Fingerprints: Expect to find evidence of employees who try out how to use the training in their tasks. This will probably be account evidence, but can also include trace evidence, such as documents that attest that employees prepared for related conversations. Block: "Keeping it alive" signaling. Part 6 – Theorization: Supervisors keep on reminding to use the training ("keeping it alive") and provides feedback on the tasks related to the training application. [There is feedback loop between part 5 and 6]. Fingerprint: If part 6 exists in a case, we expect to find reminders of the supervisor to use the training and feedback on the use of the training. This can take the form of both account evidence as well as trace evidence. Block: Increasing generalization Part 7 – Theorization: Due to the peers-supervisor engagement and trust between supervisor and trainee, post-training evaluations feedback systems are implemented by supervisors until task-oriented new knowledge is retained and improved in its application by employees. Fingerprint: If part 7 exists in a case, we expect to see supervisors implementing post-training evaluations feedback. This can take the form of trace evidence or account evidence. Outcome: Training Transfer Effectiveness Theorized outcome: Effective employee training transfer is the application or use of the learned knowledge (content, skills or attitudes) acquired in a training program to the job by trainees, which is maintained over a period of time. 131

Observable manifestations: The employee applies the learned content and skills and they are maintained on the job. Evidence for this can take the form of account evidence.

132

Table 17: Causal mechanism Signaling and retention

Cause- Supervisor support

Ascribing importance to training Part 1

Part 2

Employees follow the training Part 3

Facilitating training climate Part 3b

Theorization

Superior’s commitment to facilitate the retention and motivate the use of the acquired content in a training to the job by employees, during and after a training program takes place.

Supervisor ascribes importance to the training program and takes initiative to let the employees follow the training.

Employees react by putting the training in their agenda. [because they do not have choice].

Everybody follows the training in group, in part because it was mandatory to do so by the supervisor.

In parallel supervisors enable employees to follow the training by taking over the workload during the training period, with the aim that employees can be focused on learning the training content.

Operationalization

Supervisor support in the form of supervisors encouraging trainees to share what they've learned in training with people in their work environment. Similarly, we expect to see observables manifestations of discussions between the supervisor and trainees about how to apply competences to job situations; supervisors giving coaching advice and useful feedback after training on the application in the job of what learned when required. Finally, we also assume that the supervisor trusts that the trainees are capable to successfully apply what he or she has learned).

Supervisor’s engagement with the training and trainees, such as taking initiative to let the employees follow the training and signs that reveals the ‘importance’ of the training for him/her. We expect that this can take the form of account evidence as well as trace evidence of actions that the supervisor has undertaken.

Employees reacting by accepting some sort of invitation to attend the training. We assume there could be both account evidence of this as well as trace evidence.

Supervisor makes everyone (all employees) follow the training together in group.

Supervisors arranging for the workload of the employee to be taken over during the training period so that the employee can focus on the training. This can take the form of emails where these arrangements are discussed or verbatims provided by trainees. We expect to find account evidence and also trace evidence to measure this proposition.

133

Intermediate outcome

Motivation to generalize Part 4

Theorization

Operationalization

Part 5

This create a sort of organizational climate where employees perceive the importance of the training for their job, and where they acknowledge the engagement of the supervisor encouraging this goal.

Because the relevance of training is perceived, employees following the training feel motivated to use it and discuss the training content with other peers.

Employees tries out/use the training in tasks-related matters keeping the level of motivation that 'they just have to try it to learn' within an environment of trust and cohesion.

Employees who perceive the training to be important for their job or acknowledge the engagement of the supervisor. We expect that this can take the form of account evidence.

We expect to find evidence on the motivation/inspiration of employees. We expect to find this mainly in account evidence.

Employees try out to evaluate themselves about how to use the training in their tasks. This will probably be account evidence, but there could also be other trace evidence, such as documents that show preparation for specific conversations.

134

“Keeping it alive” signaling Part 6

Increasing generalization Part 7

OUTCOME

Theorization

Supervisors keep on reminding to use the training ("keeping it alive") and provides feedback on the tasks related to the training application. [There is feedback loop between part 5 and 6].

Due to the peers-supervisor engagement and trust between trainee and supervisor, post-training evaluation feedback systems are implemented by supervisors until task-oriented new knowledge is retained and improved in its application by employees.

Learned content and skills are applied on the job context and maintained over time (routine)

Operationalization

Reminders of the supervisor to use the training and feedback on the use of the training. This can take the form of both account evidence as well as trace evidence.

Supervisors implementing post-training evaluations feedback. This can take the form of trace evidence or account evidence.

The employee applies the learned content and skills and they are maintained on the job. Evidence for this can take the form of account evidence.

Source: Authors.

135

Enhanced learning transfer intervention Enhanced learning transfer intervention also acts in learning and performance stages of training. We disentangle the process as a complex mechanism consisting of a single pathway: a cause (peer support) that trigger the mechanism consisting of six building block and twelve parts). Cause: Peer support We have theorized our causal condition as the commitment of peers (colleague’s) to improve the trainee’s retention of the learned content and stimulate the trainee’s use of learned material to the job. It is expected that this cause can be generalized to other cases of training transfer effectiveness when the context identical elements, a training program as active learning methods, autonomy and balanced workload are present. As observable manifestation of this condition, we expect to find evidence of colleagues trying to minimize interruptions at work that interfere with the opportunity to practice the newly learned skills in the peer’s work unit; peers helping others with technical knowledge to apply the techniques learned during the training; peers encouraging others to use the skills they learned in trainings; peers providing positive feedback to others about their performances; peers reinforcing the use of new knowledge acquired by others in the training. We expect to find evidence of this from interviews with involved actors. Block: Following the training Part 1a – Theorization: Peers follow practice-oriented training together within a ‘flat’ atmosphere. Fingerprints: If part 1 of the theorized causal mechanism exists in a case, we expect to find evidence in the empirical record of the employees who have followed the training together, within a non-hierarchical organization or hybrid organization15. Since, having followed the training was an obligation to participate in this research, it may therefore be a prior in each case. We may easily find account evidence and trace evidence of participation in the training. Part 1b – Theorization: Because peers follow practice-oriented training together within a ‘flat’ atmosphere, they practice the content of the training together and recognize the ‘relevance’ of the training and the importance of ‘doing it in group’ for their work-performance. [they need to be on the same page!]. Fingerprints: If part 1b of the theorized causal mechanism exists in a case, we expect to find evidence on peers practicing the content of the training in group and recognizing the relevance during the training and the importance of doing it in group. Given that we do not expect there to be any trace evidence (e.g. video recordings of the training or used exercise sheets), we will need to rely mainly on account evidence.

Hybrid organizations are a mix of hierarchical and nonhierarchical structures. According to Filip Dochy in an interview held on February 20th, 2020, the Belgian context is characterized as a more hybrid structure.

136

Block: Building up common understanding Part 2 – Theorization: During the training, peers communicate their different views about training implementation, in an open way, with the result that (1) they learn to trust each other even better and (2) that they acknowledge that a different way to work could improve their professional skills. Fingerprints: If part 2 of the theorized causal mechanism exists in a case, we expect to find that, during the training, peers communicate and discuss their different views of applying the training. We expect that this takes the form of discussing specific, sometimes sensitive, cases and different approaches on how to handle them. Fingerprints on this part can be more difficult to find given that what is discussed is sometimes sensitive and a record on these discussions is unlikely to exist. We should, however, expect participants to acknowledge that these discussions happened. Intermediate outcome Theorization: Through this interaction, peers gain a ‘common understanding’ of how to implement the training. Observable manifestation: If the intermediate outcome is present in a case, we expect to find evidence concerning the existence of a common understanding of the training. This could take the form of both account evidence as well as trace evidence if there is some official stance on the topic in the company. The absence of certain evidence (for example of disagreements) can also improve our belief in this part. Block: Intervision Part 3 – Theorization: In parallel, peers propose to handle ‘intervision moments’ as a ‘peer coaching activity’ to better implement the training content to the job. Fingerprint: If part 3 exists in a case, we expect to find evidence of ‘intervision moments’ which were planned, and in which peers discuss the training and its application. This evidence of planning can take the form of trace evidence (e.g. an invitation to meetings via emails) or account evidence. Part 4a – Theorization: Peers agree to follow coaching activities – because they feel trust and they recognizes the need of a different way of working. Fingerprint: If part 4a exists in a case, we expect to see evidence of agreements to discuss the implementation of the training in certain kind of coaching activities. This could be found in the form of trace evidence or sequence evidence. Part 4b – Theorization: As part of the activity, peers meet each other to discuss the implementation of the training, specific cases and share experiences (e.g. issue, challenge or problem). Fingerprint: If part 4b exists in a case, we expect to find evidence on scheduled meetings by colleagues with the aim to discuss the implementation of the training. This can be found in the form of trace evidence or account evidence.

137

Part 5 – Theorization: Peers ask clarifying questions to understand the situation and issue at hand of other peers when facing issues, challenges or problems. Fingerprint: If part 5 exists in a case, we expect to see evidence on peers asking for clarification during the intervision moments. Given that these questions are probably mostly oral, we mainly expect account evidence here. Part 6 – Theorization: As result of it, peers start to brainstorm and bring up alternatives for action to support other peers. Fingerprint: We expect to see peers brainstorming about issues on application of the training and offer suggestions to the employee. Given that these suggestions are rarely written down, we only expect account evidence here. Part 7 – Theorization: Peer(s) make(s) a synthesis and formulates recommendations to their peers. Fingerprint: We expect to see peers making a synthesis of the issues and giving each other recommendations on how to apply the training correctly/ handle certain situations. This could take the form of account evidence in the form of quotes from interviews or trace evidence in the form of email/documents with actual recommendations. Part 8 – Theorization: Peers debrief : “what did they hear, what do they make of it, what do they take with them”. Fingerprint: We expect to find evidence of peers debriefing on what they hear, and lessons learned. Here we mostly expect account evidence. Block: Adaptability and application Part 9 – Theorization: As a result of that, peers feel more stimulated to apply the content learned and are less resistant (adaptability). Fingerprint: If part 9 exists in a case, we expect to find evidence of employees feeling more stimulated. We expect account evidence in which employee express feelings of motivation to apply the training. Part 10 – Theorization: Peers apply the content (when it is ad hoc to the problem/challenge identified at work), after a reflection of what they heard during the intervision moment. Fingerprints: We expect to see documents or minutes that reveals some sort of meeting preparation related to the training application. Because a part of the course was on leadership and communication, we expect there to be some preparation of difficult conversations. This evidence could take the form of trace evidence but also of account evidence. Block: Intervision (after adaptability) [feedback loop parts 3-10]. Part 11 – Theorization: Peers discuss the application and get feedback from other peers in subsequent intervision moments (follow-up post-training application).

138

Fingerprint: If part 7 exists in a case, we expect to find evidence of the employees asking and discussing feedback in the context of training application. Evidence can take the form of account evidence. Block: New working thinking Part 12 – Theorization: Because of that, peers incorporate the new way of working thinking after an adaptation phase and it becomes routine. Fingerprint: If part 12 exists in a case, we expect to see evidence of new routines applied to the job by trainees, after that the training content has been absorbed. Evidence for this could be account evidence. Outcome: Training Transfer Effectiveness Theorized outcome: Effective employee training transfer is the application or use of the learned knowledge (content, skills or attitudes) acquired in a training program to the job by trainees, which is maintained over a period of time. Observable manifestations: The employee applies the learned content and skills and they are maintained on the job. Evidence for this can take the form of account evidence.

139

Table 18: Causal mechanism Enhanced learning transfer intervention Cause- Peer support

Following the training

Building up common understanding

Part 1a

Part 1b

Part 2

Intermediate outcome

Theorization

the commitment of peers (colleague’s) to improve the trainee’s retention of the learned content and stimulate the trainee’s use of learned material to the job

Peers follow practiceoriented training together within a ‘flat’ atmosphere.

Because peers follow practice-oriented training together within a ‘flat’ atmosphere, they practice the content of the training together and recognize the ‘relevance’ of the training and the importance of ‘doing it in group’ for their workperformance. [they need to be on the same page!].

During the training, peers communicate their different views about training implementation, in an open way, with the result that (1) they learn to trust each other even better and 2) that they acknowledge that a different way to work could improve their professional skills.

Through this interaction, peers gain a ‘common understanding’ on how to implement the training.

Operationalization

Colleagues trying to minimize interruptions from work that interferes in the opportunity to practice the newly learned skills in the peer’s work unit; peer helping others with technical knowledge to apply the techniques learned in the training; peers encouraging others to use the skills they learned in trainings; peers providing positive feedback to others about their performances; peers reinforcing the use of new knowledge acquired by others in the training. We expect to find evidence of this from interviews with involved actors.

Employees who have followed the training together, within a nonhierarchical organization. Since, having followed the training was an obligation to participate in this research, it may therefore be a prior in each case. We may easily find account evidence and trace evidence of participation in the training.

peers practicing the content of the training in group and recognizing the relevance during the training and the importance of doing it in group. Given that we do not expect there to be any trace evidence (e.g. video recordings of the training or used exercise sheets), we will need to rely mainly on account evidence.

Peers communicate and discuss their different views of applying the training. We expect that this takes the form of discussing specific, sometimes sensitive, cases and different approaches on how to handle them. Fingerprints on this part can be more difficult to find given that what is discussed is sometimes sensitive and a record on these discussions is unlikely to exist. We should, however, expect participants to acknowledge that these discussions happened.

Existence of a common understanding of the training. This could take the form of both account evidence as well as trace evidence if there is some official stance on the topic in the company. The absence of certain evidence (for example of disagreements) can also improve our belief in this part.

140

Intervision Part 3

Part 4a

Part 4b

Part 5

Theorization

In parallel, peers propose to handle ‘intervision moments’ as a ‘peer coaching activity’ to better implement the training content to the job.

Peers agree to follow coaching activities – because they feel trust and they recognizes the need of a different way of working – and meet each other to discuss the implementation of the training, specific cases and share experiences (e.g. issue, challenge or problem).

As part of the activity, peers meet each other to discuss the implementation of the training, specific cases and share experiences (e.g. issue, challenge or problem).

Peers ask clarifying questions to understand the situation and issue at hand of other peers when facing issues, challenges or problems.

Operationalization

Planning of ‘intervision moments’ in which peers discuss the training and its application. This evidence of planning can take the form of trace evidence in the form of invitation through emails or account evidence.

Agreements to discuss the implementation of the training in certain kind of coaching activities. This could be found in the form of trace evidence or sequence evidence.

Scheduled meetings by colleagues with the aim to discuss the implementation of the training. This can be found in the form of trace evidence or account evidence.

Peers asking for clarification during the intervision moments. Given that these questions are probably mostly oral, we mainly expect account evidence here.

141

Part 6

Intervision Part 7

Part 8

Adaptability and application Part 9 Part 10

Theorization

As result of it, peers start to brainstorm and bring up alternatives for action to support other peers.

Peer(s) make(s) a synthesis and formulates recommendations to their peers.

Peers debrief : “what did they hear, what do they make of it, what do they take with them”.

As a result of that, peers feel more stimulated to apply the content learned and are less resistant (adaptability).

Peers apply the content (when it is ad hoc to the problem/challenge identified at work), after a reflection of what they heard during the intervision moment.

Operationalization

Peers brainstorming about issues on application of the training and offer suggestions to the employee. Given that these suggestions are rarely written down, we only expect account evidence here.

Peers making a synthesis of the issues and give each other recommendations on how to apply the training correctly/ handle certain situations. This could take the form of account evidence in the form of quotes from interviews or trace evidence in the form of email/documents with actual recommendations.

We expect to find evidence for peer debrief on: what they hear and lessons learned. Here we mostly expect account evidence.

Employees feeling more stimulated. We expect account evidence in which employee express feelings of motivation to apply the training.

Documents or minutes that reveals some sort of meeting preparation related to the training application. Because a part of the course was on communication, we expect there to be some preparation for difficult conversations. This evidence could take the form of trace evidence but also of account evidence.

142

Intervision (after adaptability) [feedback loop parts 3-10] Part 11 Theorization

Operationalization

New working thinking

OUTCOME

Part 12

Peers discuss the application and get feedback from other peers in subsequent intervision moments (follow-up posttraining application).

Because of that, peers incorporate the new way of working thinking on their own after an adaptation phase and it becomes routine.

Learned content and skills are applied on the job context and maintained over time (routine)

Employee asking and discussing for feedback in a context of training application in subsequent moment. Evidence can take the form of account evidence.

New routines applied to the job by trainees, after that training content has been absorbed. Evidence for this could be account evidence.

The employee applies the learned content and skills and they are maintained on the job. Evidence for this can take the form of account evidence.

Source: Authors.

143

Self-management intervention Self-management intervention is active in the learning and performance stages of training. We reconceptualize the theory as a complex mechanism consisting of a single pathway: a cause (relapse prevention and goal setting) that triggers the mechanism consisting of eight building block and eight parts). Cause: Relapse prevention and goal setting We have theorized our causal condition as “Formulation of training goals by employee AND Commitment to overcome the obstacles when addressing difficulties in applying new knowledge at work”. It is expected that this cause can be generalized to other cases of training transfer effectiveness when the contexts of ‘identical elements’, ‘training program as active learning method’, ‘autonomy’ and ‘balanced workload’ are present. The mechanism is also theorized as functioning in the absence of a ‘balanced workload’. As observable manifestations of this condition, we expect to find evidence of trainees setting specific training goals and activities related to how they deal with the challenges in the achievement of these goals, when addressing difficulties in transfer. We expect to find evidence of this in the survey (so called ‘account evidence’) which we asked trainees to complete after having attended the training. Block: Provide direction for attention Part 1 – Theorization: Based on the new training skills to be acquired, the trainee identifies some kind of goals [distal and proximal] (either specific, challenging or difficult) to help her/himself with expressing attention. Fingerprints: If part 1 of the theorized causal mechanism exists in a case, we expect to find evidence in the empirical record of goal-oriented reasons that stimulated training application. There might be evidence in the form of trace evidence or account evidence. Part 2 – Theorization: With this knowledge in mind, the trainee organizes her/his effort either focused on learning or performance goals that he/she wishes to apply and maintain from this training - increasing (perception of his own) determination Fingerprints: If part 2 of the theorized causal mechanism exists in a case, we expect to find evidence on aspects of the training that the trainee absorbs during the training itself, that enable him/her to get focused in his/her efforts to apply and maintain it from the training. This evidence could take the form of account evidence.

144

Block: Mobilizing effort Part 3 - Theorization: The trainee feels motivated due to this perceived determination and develops the best ways to achieve and maintain such goals: setting the skills maintenance goal, based upon the training, and identifying potential risks of slips. Fingerprints: If part 3 of the theorized causal mechanism exists in a case, we should expect to find evidence of the trainee feeling motivated to reach a goal and actions carried out to implement the training. We expect that this evidence will mainly be account evidence. Block: Pros & Cons generalization Part 4 – Theorization: By identifying potential threats to transfer, the trainee defines the advantages and disadvantages of using the skills at work in order to stay motivated. Fingerprint: If part 4 is present in a case, we expect to find evidence of the trainee contrasting approaches about using/not using the skills at work, identifying potential threats to the training application and some ways to resolve it, and formulating advantages of its applicability. We expect to mostly find account evidence here. Block: Coping with slips Part 5a – Theorization: The trainee discusses and learns certain methods or tools to avoid or overcome some kind of obstacles in applying the training content at work [anticipation]. Fingerprint: If part 5a exists in a case, we expect to find evidence on the trainee dealing with challenging situations in the application of the training to the job, as well as communication exchange about ways to overcoming it. We mainly expect to find account evidence for this. Block: Networking Part 5b – Theorization: The trainee understands the difference between training and job context, so she/he creates a support network for transferability. Fingerprint: If part 5b exists in a case, we expect to find evidence on active supportive networks created by trainees in order to get transferability. We expect to find account evidence that can confirm this proposition. Block: Slip prediction Part 6 – Theorization: The trainee predicts some kind of slips in transfer by monitoring past experiences of slip and relapse [anticipation] and the present environmental situations. Fingerprint: If part 6 exists in a case, we expect to find evidence based on the trainee’s reflections from past slips experiences and relapse as a way to anticipate such situations. This evidence can take the form of account evidence of the employee identifying some obstacles that arise when attempting to apply the training. 145

Block: Coping strategies Part 7 – Theorization: Based on coping methods, the trainee applies a threat coping strategy to this 'predicted slip' by selecting only appropriate steps to increase skills retention and generalization [e.g. applying skills in the appropriate setting, reducing interfering and unproductive emotions; retain selfconfidence, diagnose support skills needed to maintain training, etc.] Fingerprint: If part 7 exists in a case, we expect to see the trainee dealing with slipping up and overcoming threats to generalization, by reducing interfering and unproductive emotions and applying skills in the appropriate setting. We mostly assume that the evidence here will be account evidence. If the trainee has formalized this strategy, there is a small chance that we will find some trace evidence. Block: Monitoring and self-rewards Part 8 – Theorization: The trainee monitors the process of skills transfer (self-monitoring if performance; self-evaluation against goal; self-reaction with self-efficacy) and creates meaningful self-rewards for skill retention. Fingerprint: If part 8 exists in a case, we expect to see evidence of activities performed by the trainee related to self-monitoring of his/her own performance and some sort of self-rewards for skill retention. We expect that this will mostly be account evidence in the form of the trainee discussing what helps him/her in implementing the training. We can also find account evidence in the survey on how the trainee experiences the training. Outcome: Training Transfer Effectiveness Theorized outcome: Effective employee training transfer is the application or use of the learned knowledge (content, skills or attitudes) acquired in a training program to the job by trainees, which is maintained over a period of time. Observable manifestations: The employee applies the learned content and skills and they are maintained on the job. Evidence for this can take the form of account evidence.

146

Table 19: Causal mechanism self-management intervention Cause- Goal setting AND Relapse prevention

Provide direction for attention

Mobilizing effort

Pros & Cons generalization

Part 1

Part 2

Part 3

Part 4

Theorization

Formulation of training goals by employee AND commitment to overcome the obstacles when addressing difficulties in applying new knowledge at work.

Based on the new training skills to be acquired, the trainee identifies some kind of goals [distal and proximal] (either specific, challenging or difficult) to help her/himself with expressing attention.

With this knowledge in mind, the trainee organizes her/his effort either focused on learning or performance goals that he/she wishes to apply and maintain from this training increasing perception of determination.

The trainee feels motivated due to this perceived determination and develops the best ways to achieve and maintain such goals: setting the skills maintenance goal, based upon the training, and identifying potential risks of slips.

By identifying potential threats to transfer, the trainee defines the advantages and disadvantages of using the skills at work in order to keep motivated.

Operationalization

Trainees setting specific training goals and activities related to how they deal with the challenges in achievement such goals, when addressing difficulties in transfer. We expect to find evidence of this in the survey (account evidence) which we asked trainees to complete after having attended the training.

Goal-oriented reasons that stimulated training application. There might be evidence in the form of trace evidence or account evidence.

Aspects that the trainee absorbs during the training itself, that enable him/her to get focused in his/her efforts to apply and maintain from the training. This evidence could take the form of account evidence.

Trainee feeling motivated to reach a goal and actions carried out to implement the training. We expect that this evidence will mainly be account evidence.

Trainee contrasting approaches about using/not using the skills at work, identifying potential threats to the training application and some ways to resolve it, and formulating advantages of its applicability. We expect to mostly find account evidence here.

147

Coping with slips Part 5a

Networking Part 5b

Slip prediction Part 6

Theorization

Trainee discusses and learns certain methods or tools to avoid or overcome some kind of obstacles in applying the training content at work [anticipation].

Trainee understands the difference between training and job context, so she/he creates a support network for transferability.

Trainee predicts some kind of slips in transfer by monitoring past experiences of slip and relapse [anticipation] and the present environmental situations.

Operationalization

Trainee dealing with challenging situations in the application of training to the job, as well as communication exchange about ways to overcoming it. We mainly expect to find account evidence for this.

Active supportive networks created by trainees in order to get transferability. We expect to find account evidence that could confirm this proposition.

Trainee’s reflections from past slips experiences and relapse as a way to anticipate such situations. This evidence can take the form of account evidence of the employee identifying some obstacles that arise when attempting to apply the training.

148

Coping strategies Part 7

Monitoring and self-rewards Part 8

OUTCOME

Theorization

Based on coping methods, the trainee applies a threat coping strategy to this 'predicted slip' by selecting only appropriate steps to increase skills retention and generalization [e.g. applying skills in the appropriate setting, reducing interfering and unproductive emotions; retain selfconfidence, diagnose support skills needed to maintain training, etc.]

The trainee monitors the process of skills transfer (self-monitoring if performance; self-evaluation against goal; self-reaction with self-efficacy) and creates meaningful self-rewards for skill retention.

Effective employee training transfer is the application or use of the learned knowledge (content, skills or attitudes) acquired in a training program to the job by trainees, which is maintained over a period of time. Learned content and skills are applied on the job context and maintained over time (routine)

Operationalization

Trainee dealing with slipping up and overcoming threats to generalization, by reducing interfering and unproductive emotions and applying skills in the appropriate setting. We mostly assume that the evidence here will be account evidence. If the trainee has formalized this strategy, there could be a small chance we will find some trace evidence.

Activities performed by the trainee related to self-monitoring of his/her own performance and some sort of selfrewards for skill retention. We expect that this will mostly be account evidence in the form of the trainee discussing what helps him/her in implementing the training. We can also find account evidence in the survey on how the trainee experiences the training.

The employee applies the learned content and skills and they are maintained on the job. Evidence for this can take the form of account evidence.

Source: authors.

149

Learner agency Learner agency acts before, during and after training, mostly in learning process and less in performance. We disentangle the process as a complex mechanism consisting of a single pathway: a cause (sense of urgency) that trigger the mechanism consisting of twelve building block and fourteen parts). As said, it is particularly a complex process because plays a role during the whole process of pushing individuals to learn, learning, and application. Cause: Sense of urgency We have theorized our causal condition at the individual level, as employees feeling a (1) clear need to engage in training (2) because of the identification of a hiatus between current knowledge and skill and required knowledge and skill in the future (3), with the understanding that overcoming the hiatus is within reach of the capabilities of the employee. As observable manifestation of this condition, we expect to find evidence in the empirical record of employee´s needs to engage in training because of the identification of a hiatus between current knowledge and skill and required knowledge and skill in the future: (1) I felt that overcoming the challenge I identified was within my capabilities; (2) I identified a more general hiatus in my knowledge and behaviour that was not bound to a specific task. Intermediate Outcome Theorization: A state of maximum involvement and intrinsic motivation is achieved due to a balance between tasks demands and competences. Observable manifestations: If the intermediate outcome is present, we expect to find evidence of the employee’s motivation to learn new things. This can take the form of messages shared with other colleagues or own reflection that we expect to get via interviews or trace evidence. Block: Goal setting Part 1 – Theorization: Because of this intrinsic motivation, employee develops a feeling of relevance of the job's tasks requirements and expresses expectations about the learning process by setting goals about what to get from this learning experience. Fingerprint: If part 1 exists in a case, we expect to see evidence of an employee with clear and firm commitment with learning. This can take the form of informal discussions with colleagues or e-mails or messages where this is mentioned. This will be measured using account evidence with participants and trace evidence (e-mails, messages). This can also be in the survey that was administered with the employee. Block: Free choice of learning Part 2 – Theorization: Following his/her goals, the employee makes a free choice by engaging in a training program and focus his/her attention on the tasks to be performed. Fingerprints: We expect to see evidence of employees participating in training by own choice. This can be measured using account evidence or trace evidence. Block: Learning at hands 150

Part 3 – Theorization: Based on such attention, the employee undertakes learning activities related to complex tasks. Fingerprints: We expect to see evidence of employee developing own learning activities which matches with his/her goals. We will measure this with account evidence from interviews with participants Block: Internal feedback: reflection-in-action Part 4 – Theorization: Employee actively reflects about the learning process: "how everything is going on" and adapts learning strategies to ensure the ultimate goal of transfer. Fingerprint: If part 6 exists in a case, we expect to see employees attempting to modify his/her learning strategies to a better transfer. This can take the form of spending more hours in completing complex tasks related to the new content acquired in the training or any activity where employees spend more effort to achieve own goal. We will measure this with account evidence from interviews with participants. Part 5 – Theorization: By adapting learning strategies, employee reaches certain goals that are identified and perceived as relevant by her/himself. This keep the motivation alive to reach the ultimate goals: transfer. Fingerprint: If part 5 exists in a case, we expect to see employee going ahead with training and activities related to learning improvement and performance improvement. We will measure this using account evidence from interviews with participants. Part 6 – Theorization: Based on such ultimate goal, employee undertakes activities related to complex tasks to be applied to the job. Fingerprint: If part 6 exists in a case, we expect to see employee performing complex tasks that she/he knows will be required in the job context. We will measure this using account evidence from interviews with participants. Block: Monitoring Part 7 – Theorization: Employee identifies the strength and weakness to improve themselves in the application of the learned content to the job by focusing on what "to do better". Fingerprint: If part 7 exists in a case, we expect to see brainstorming carried out by the employee in order to evaluate the good, the bad and the ugly of the learning process to improve him/herself. We will measure this using account evidence from interviews with participants. This can also take the form of trace evidence, if some document related to this brainstorm was created and recorded. Block: Networking-feedback Part 8 – Theorization: Employee asks for feedback to others as a way to evaluate the learning and application process objectively. Fingerprint: If part 8 exists in a case, we expect to find evidence in the form of reports, minutes, e-mails from trainee asking to colleagues providing some sort of feedback about learning and performance. We will measure this with trace evidence.

151

Part 9 – Theorization: Employee receives feedback from peers and adapt their activities to overcome some obstacles to transfer goals. Fingerprint: If part 9 exists in a case, we expect to see reports, minutes, e-mails from colleagues providing some sort of feedback to employee taking the training. We will measure this with trace evidence. Block: Internal feedback: reflection-on-action Part 10 – Theorization: Employee looks back on and think on own action about how learning process and applicability went. Fingerprint: If part 10 exists in a case, we expect to see employee’s reflections about his/her experiences in learning process in a retrospective way. We will measure this with account evidence from interviews with participants. Block: Coping strategies [Modularity, part 7 of the causal mechanism self-management which acts as module to travel across classes of homologous mechanisms (Beach and Pedersen, 2019). Part 11 – Theorization: Employee identifies and selects steps to ensure transfer by focusing on applying the learned content in the appropriate setting and reducing possible interferences to transfer. Fingerprint: If part 11 exists in a case, we expect to see employee selecting only appropriate steps to increase skills retention and generalization [e.g. applying skills in the appropriate setting, reducing interfering and unproductive emotions; retain self-confidence, diagnose support skills needed to maintain training, etc.]. We mostly assume that the evidence here will be account evidence. Block: Monitoring and self-rewards [Modularity, part 8 of the causal mechanism self-management which acts as module to travel across classes of homologous mechanisms (Beach and Pedersen, 2019). Part 12 – Theorization: Employee monitors the process of skills transfer (self-monitoring if performance; self-evaluation against goal; self-reaction with self-efficacy) and create meaningful self-rewards for skill retention. Fingerprint: If part 12 exists in a case, we expect to see evidence of activities performed by the trainee related to self-monitoring of his/her own performance and some sort of self-rewards for skill retention. We expect that this will mostly be account evidence in the form of the trainee discussing what helps him/her in implementing the training. We can also find account evidence in the survey on how the trainee experiences the training. Outcome: Training Transfer Effectiveness Theorized outcome: Effective employee training transfer is the application or use of the learned knowledge (content, skills or attitudes) acquired in a training program to the job by trainees, which is maintained over a period of time. Observable manifestations: The employee applies the learned content and skills and they are maintained on the job. Evidence for this can take the form of account evidence.

152

Table 20: Causal mechanism Learner agency Goal setting

Free choice of learning

Learning at hands

Part 1

Part 2

Part 3

A state of maximum involvement and intrinsic motivation is achieved due to a balance between tasks demands and competences

Because of this intrinsic motivation, employee develops a feeling of relevance of the job's tasks requirements and expresses expectations about the learning process by setting goals about what to get from this learning experience.

Following his/her goals, the employee makes a free choice by engaging in a training program and focus his/her attention on the tasks to be performed

Based on such attention, the employee undertakes learning activities related to complex tasks

We expect to find evidence of the employee’s motivation to learn new things. This can take the form of messages shared with other colleagues or own reflection that we expect to get via interviews or trace evidence.

Employee with clear and firm commitment with learning. This can take the form of informal discussions with colleagues or e-mails or messages where this is mentioned. This will be measured using account evidence with participants and trace evidence (e-mails, messages). This can also been in the survey that was administered with the employee.

We expect to see evidence of employees participating in training by own choice. This can be measured using account evidence or trace evidence

We expect to see evidence of employee developing own learning activities which matches with his/her goals. We will measure this with account evidence from interviews with the participant

Cause- Sense of urgency

Intermediate Outcome

Theorization

Clear need to engage in training, because of the identification of a hiatus between current knowledge and skill and required knowledge and skill in the future (3), with the understanding that overcoming the hiatus is within reach of the capabilities of the employee.

Operationalization

Need to engage in training because of the identification of a hiatus between current knowledge and skill and required knowledge and skill in the future: ‐I felt that overcoming the challenge I identified was within my capabilities ‐I identified a more general hiatus in my knowledge and behaviour that was not bound to a specific task.

Before training

During training

153

Part 4 Theorization

Employee actively reflects about the learning process: "how everything is going on" and adapts learning strategies to ensure the ultimate goal of transfer

Operationalization

Employees attempting to modify his/her learning strategies to a better transfer. This can take the form of spending more hours in completing complex tasks related to the new content acquired in the training or any activity where employees spend more effort to achieve own goal. We will measure this with account evidence from interviews with participants.

During training

Internal feedback: reflection-in-action Part 5 By adapting learning strategies, employee reaches certain goals that are identified and perceived as relevant by her/himself. This keeps the motivation alive to reach the ultimate goals: transfer Employee going ahead with training and activities related to learning improvement and performance improvement. We will measure this using account evidence from interviews with participants

Part 6

Based on such ultimate goal, employee undertakes activities related to complex tasks to be applied to the job

Employee performing complex tasks that she/he knows will be required in the job context. We will measure this using account evidence from interviews with participants

After training

154

Monitoring

Network-feedback Part 9

Internal feedback – reflection-onaction Part 10

Part 7

Part 8

Theorization

Employee identifies the strength and weakness to improve themselves in the application of the learned content to the job by focusing on what "to do better".

Employee asks for feedback to others as a way to evaluate the learning and application process objectively.

Employee receives feedback from peers and adapt their activities to overcome some obstacles to transfer goals.

Employee looks back on and think on own action about how learning process and applicability went.

Operationalization

We expect to see brainstorming carried out by the employee in order to evaluate the good, the bad and the ugly of the learning process to improve him/herself. We will measure this using account evidence from interviews with participants. This can also take the form of trace evidence, if some document related to this brainstorm was created and recorded

Report, minutes, e-mails from trainee asking to colleagues providing some sort of feedback about learning and performance. We could measure this with trace evidence.

We expect to find report, minutes, e-mails from colleagues providing some sort of feedback to employee taking the training. We will measure this with trace evidence or account evidence.

We expect to see employee reflections about his/her experiences in learning process in a retrospective way. We will measure this with account evidence from interviews with participants.

After training

155

Theorization

Operationalization

Coping strategies Part 11 Employee identifies and selects steps to ensure transfer by focusing on applying the learned content in the appropriate setting and reducing possible interferences to transfer

Monitoring and self-rewards Part 12 Employee monitors the process of skills transfer (self-monitoring if performance; self-evaluation against goal; self-reaction with self-efficacy) and create meaningful self-rewards for skill retention.

Employee selecting only appropriate steps to increase skills retention and generalization [e.g. applying skills in the appropriate setting, reducing interfering and unproductive emotions; retain self-confidence, diagnose support skills needed to maintain training, etc.]. We mostly assume that the evidence here will be account evidence.

We expect to see evidence of activities performed by the trainee related to self-monitoring of his/her own performance and some sort of selfrewards for skill retention. We expect that this will mostly be account evidence in the form of the trainee discussing what helps him/her in implementing the training. We can also find account evidence in the survey on how the trainee experiences the training.

OUTCOME Learned content and skills are applied on the job context and maintained over time (routine)

The employee applies the learned content and skills and they are maintained on the job. Evidence for this can take the form of account evidence.

After training

156

ANNEX 7: ADDITIONAL INFORMATION ON DATA The evaluation project was set up with a time scope that considers both the situation before and after the training. To this end, we set up three structured surveys. Employees For the employees within the selected training projects, we developed two standardized survey instruments, for the T0 and T1 data collection respectively. To contact the employees in the training programs, we obtained the professional email addresses of their employers. We explained that addresses would only be used for scientific purposes (the survey access and follow-up). Two organisations recommended postal surveys as the employees did not have a professional computer at their disposal. Before the start of the survey, the contacted employees were informed about the goals of the survey, were also asked to agree with the terms and conditions of the research. For the T0 data collection, we focused on training needs and needs analysis, motivation, and present skills before the training. For the T1 data collection, we zoomed in on the use and application of the training content, goals and framing of the training, social support of supervisor and peers and constraints. In both surveys, we combined closed and open questions to get the best possible insight. Where possible, we based ourselves on existing validated measurement instruments to operationalize our concepts. However, as most of the research on the effectiveness of training and training transfer stems for correlational research, the existing operationalization instruments might not pay sufficient attention to the main attributes of the concepts as how we defined them in a set-theoretical manner. We outline the strategy and the details of the operationalization of the concepts in Appendix B. For the T0 data, employees were contacted two weeks before the start of their training. Given the different schedules of the companies, the T0 data collection started in April 2018 and was finalized in August 2018. In total, we contacted 183 employees via an online survey and 20 employees via a postal survey in the T0 data collection. We received 94 online surveys and 12 postal survey, which comes down to a response rate of 51% and 60% respectively. The total response rate for the T0 data is 52% (=106/183). For T1, the employees were contacted no less than two months after the completion of their training, but preferably after at least three months. The T1 data collection started in November 2018 and finished in February 2019. The timing of the T1 contact was based on a specific literature review, as there are indications that the moment of data collection might affect the type of transfer that is examined. More specifically, when we learn, there is often a results dip or incorporation lag, a period that is characterized by challenges and stress, during which we do not necessarily perform better (Dochy, 2017; Rachkam, 1979). When we during and after this period, persist and keep on applying the new skill, applying the skill might start to feel more natural and result in better performance. For the research questions within this evaluation project it is important to prevent collecting data before or during the results dip, but to know under which combinations of contexts and conditions a more sustainable transfer occurs. Based on the literature search, we decided not to set the data collection right after the completion of the training, but to opt for a 3 till 6 months’ time lag.

157

Additionally, we invited the employees who did not reply to the T0 call to fill in the T1 survey. While this was not foreseen in the initial design of the evaluation project, we nonetheless engaged in this additional data collection, as it might increase the generalization potential of research conclusions, and the potential pool of positive cases. In total, we contacted 203 employees for the T1 data collection. HR and Training Officers Additional data collection was deemed necessary to be able to operationalize characteristics of training projects and the companies that might act as necessary contexts. This way we collected data that was unavailable to employees and could also gather alternative observable manifestations of our conditions and contexts. We contacted the HR or training officers responsible for the organization of the ESF training projects via a standardized online survey in November 2018. After this first online data collection, a telephonic follow-up contact was established to clarify their survey responses early in December 2018. The main topics of the HR survey were the training policy and basis for organization of training projects, transfer enhancing intervention and characteristics of the specific training the employees followed.

158

ANNEX 8: CASE SELECTION STRATEGY FOR QCA AND PT PHASE Where and when does “an effective employee social skills training transfer occur? The practice of selecting cases on their values on the outcome has been subject to much debate (Peters, 2013; Brady & Collier, 2004: Chap.6; Ragin, 1996; King et al., 1994: Chap.4; Geddes 1990, 2003). The risk of this strategy is that “the research design does allow variation on the dependent variable, but that variation is truncated (…) we limit our observations to less than the full range of variation on the dependent variable that exists in the real world. In these cases, something can be said about the causes of the dependent variable; but the inferences are likely to be biased …” (King, Keohane & Verba, 1994: 130). However, a qualified added value of analyzing a small number of cases selected on the outcome, is that this strategy is valid if the researcher is evaluating/testing necessary conditions (Dion, 1998: 127; Goertz, 2005). In our research we are actually focused on three interrelated goals: to figure out necessary contexts, causal conditions and mechanisms that lead to our outcome of interest. Therefore, our strategy of case selection is justified, based on the following arguments: (1) Case selection based on the outcome is not a universal rule in social research design (in quantitative research, it is better to avoid it); (2) It is suitable for qualitative research (especially if it is a set-theoretic one); (3) It is applicable for testing/evaluating necessary conditions in a relatively low number of cases; (4) It has proven to be a useful method for gathering information when there are relatively few data and (5) It is useful to control the problem of selection bias thru the selection of negative cases as well, by applying the ‘principle of possibility’ (Goertz & Mahoney, 2005), because it helps the researcher to avoid errors and to maximize leverage for making valid causal inferences.

SELECTION OF NEGATIVE CASES Where and when do “failure of the effective employee social skills training transfer occur? Many cases can qualify as ‘cases of failure of an effective employee training transfer’, but also many of them will never be considered in this research. This is because the selection of negative cases is not simply about choosing cases in which the ‘an effective employee training transfer’ did not happen; rather, those cases need to be relevant and appropriate for exploring the causal mechanisms linking conditions and outcome. It is therefore necessary to apply systematic rules and to justify why certain cases will be located within the set of cases experiencing the occurrence of an effective employee training transfer. In this evaluation, we propose to apply the Possibility Principle “which provides explicit, rigorous, and theoretically informed guidelines for choosing a set of negative cases” (Goetz & Mahoney, 2006:178). Following the Possibility Principle, only cases in which an effective employee training transfer is possible should be included in the set of negative cases. Conversely, cases in which this outcome is impossible should be considered as irrelevant negative cases. This Principle is useful to the extent that it can help to “avoid errors and maximize leverage for making valid causal inferences” (Goertz & Mahoney, 2005: 179). The Basic Rules of the ‘Possibility Principle’ for case selection In order to interpret ‘possibility’, we follow two basic rules to implement this Principle in qualitative analysis: A Rule of Inclusion and a Rule of Exclusion. The rule of inclusion states that an outcome is seen as possible if at least one condition of the theories under study predicts its occurrence – even if other conditions predict its absence (Goertz & Mahoney, 2005:186). In other words: “Cases are relevant if their value on at least one [causal condition] is positively related to the outcome of interest” (Ibid.). The rule of 159

exclusion states that a case is considered as irrelevant if it possesses a value on a condition (that is not a core causal condition under test) that is known to make the outcome of interest impossible (Op. cit., :187). In other words: “Cases are irrelevant if their value on any eliminatory causal condition predicts the nonoccurrence of the outcome of interest” (Ibid.). This basic rule is closely related to the use of scope conditions (which describe the necessary contexts) in social research. In this research the ‘eliminatory causal condition’ is “intervention focused on the improvement of working conditions”.

CASE SELECTION STRATEGY FOR A POST-QCA PHASE: PROCESS-TRACING Once we have performed our comparative study with QCA, we will select those cases that present membership in the condition, contexts, and outcome. We will situate all relevant cases, which should be scored on conditions that might be relevant for whether and how a mechanism worked. Based on this, we will determine which cases are typical (same contexts/causal conditions/outcome) and which are deviant (cases which the presence of the context, the conditions but without the outcome). The selection of the typical case will enable us to know how the process worked and to generalize to the other positive cases and confirm theory. In contrast, the selection of the deviant case, will enable us to trace the mechanism till it break down, in order to know what did not work and to refine or expand our theory (by exploring omitted conditions /contexts).

160

ANNEX 9: QCA ANALYSIS In this Annex, the outcome and the ten conditions are shortened as follows: Training transfer effectiveness (outcome) Peer support Supervisor support Relapse prevention Goal setting Relapse prevention AND employee goal setting combined Sense of urgency Identical elements with training Training program as active learning method Autonomy Non workload (balanced workload)

TRANSFER PEERSUP SUPERV RELAPSE GOALSETTING RELAPSEGOAL SURG IDENT TRAPO AUTO NONWL

When the condition is written in uppercase in the following analyses, it denotes the presence of the condition, and when in lowercase, the absence of it.

NECESSITY ANALYSIS FOR TRAINING TRANSFER EFFECTIVENESS No single conditions are identified as necessary for transfer. Therefore, several disjunctions or logical unions (functional equivalents) of two conditions are required. Table 21 shows the necessity-consistencyrelevance-of-necessity of nine single conditions and its negations, and Table A11 shows the functional equivalents.

161

Table 21 Necessity analysis: necessity-consistency-relevance of conditions for successful training transfer Condition

Cons.Nec Cov.Nec RoN

PEERSUP SUPERV RELAPSE GOALSETTING SURG IDENT TRAPO AUTO NONWL peersup superv relapse goalsetting surg ident trapo auto nonwl

0.4000 0.3333 0.4000 0.6667 0.2667 0.7333 0.9333 0.6667 0.5333 0.6000 0.6667 0.6000 0.3333 0.7333 0.2667 0.0667 0.3333 0.4667

0.5455 0.5556 0.2500 0.5263 0.5714 0.5238 0.3182 0.2703 0.3333 0.2308 0.2439 0.3462 0.1613 0.2558 0.1379 0.1667 0.3846 0.2692

0.8864 0.9111 0.5909 0.7750 0.9348 0.7436 0.1667 0.3250 0.6190 0.2683 0.2250 0.5854 0.4222 0.1795 0.4565 0.8980 0.8222 0.5581

A macrocondition was created by combining relapse prevention and goal setting: RELAPSEGOAL. The test of necessity of this new condition displays the following result (see Table 22), confirming the absence of necessary condition in this analysis. Table 22 Necessity analysis: relapse prevention and goal setting (in a conjunction) Condition

Cons.Nec Cov.Nec RoN

RELAPSEGOAL Relapsegoal

0.3333 0.6667

0.5000 0.8889 0.2500 0.2500

Table 2311 shows three pairs of conditions united in disjunctions. Each disjunction has high but not perfect consistency. We could observe that they are also high in relevance and coverage, with 68% and 50% respectively. So, no necessary disjunctions have been figured out. Table 23 Necessity analysis: necessity-relevance-consistency-of disjunctions Condition

Cons.Nec Cov.Nec

RoN

PEERSUP+IDENT

0.800

0.500

0.684

SURG+IDENT

0.800

0.500

0.684

RELAPSEGOAL+IDENT 0.867

0.500

0.649

162

THE EIGHT CONDITIONS MODEL After several iterations with QCA, case-base knowledge and theory, we decided to select 4 causal conditions and 4 contexts from the original theoretical model consisting of 4 conditions and 10 contexts. This model clearly best-fit with our cases.

163

Table 24 Truth Table for an eight-condition model Row PEERSUP SUPERV SURG RELAPSEGOAL IDENT TRAPO AUTO NONWL

OUT

incl

PRI

cases

1.000 1.000 J3

1.000 1.000 V2

1.000 1.000 W1

1.000 1.000 T1

1.000 1.000 N2

128

1.000 1.000 B3

133

1.000 1.000 S2

144

1.000 1.000 M1

172

1.000 1.000 T2

205

1.000 1.000 B2

207

1.000 1.000 K2

208

1.000 1.000 D1

0.667 0.667 K1,S1,R3

0.500 0.500 W2,S3

0.000 0.000 C2,C3,E3, J5,L1,P2,T3 0.000 0.000 D3,J2,J4, R1,R2,S4 0.000 0.000 A1,D4, E2,V1 0.000 0.000 C5,F1

0.000 0.000 C1,G1

152

0.000 0.000 C4,M3

0.000 0.000 M2

0.000 0.000 N1

0.000 0.000 E1

0.000 0.000 K3

0.000 0.000 J1

0.000 0.000 D2

0.000 0.000 P1

158

0.000 0.000 C6

203

0.000 0.000 C16

251

0.000 0.000 H1

Note: OUT = Outcome TRANSFER; n= number of cases covered; incl= consistency; PRI16= Proportional Reduction in Inconsistency

PRI is an “Acronym for Proportional Reduction in Inconsistency. It expresses how much it helps to know that a given condition X is a subset of outcome Y rather than a subset of either Y, its complement ~Y, or the intersection between Y and ~Y” (Schneider and Wagemann, 2012: 345-346). 164

SUFFICIENCY ANALYSIS FOR TRAINING TRANSFER EFFECTIVENESS Table A25 Conservative solution Pathways to Training Transfer Effectiveness 1 peersup*superv*surg*RELAPSEGOAL*ident*TRAPO*AUTO

inclS 1.000

PRI 1.000

covS 0.133

covU 0.133

cases J3; V2

2 PEERSUP*SUPERV*surg*relapsegoal*IDENT*TRAPO*nonwl

1.000

0.133

B2; K2

3 PEERSUP*surg*relapsegoal*IDENT*TRAPO*AUTO*NONWL

1.000

0.133

M1; D1

4 peersup*SUPERV*RELAPSEGOAL*IDENT*TRAPO*AUTO*NONWL

1.000

0.133

N2; B3

5 peersup*superv*SURG*relapsegoal*ident*TRAPO*auto*NONWL

1.000

0.067

6 peersup*superv*SURG*RELAPSEGOAL*IDENT*TRAPO*AUTO*nonwl

1.000

0.067

7 PEERSUP*superv*surg*relapsegoal*ident*TRAPO*auto*nonwl

1.000

0.067

8 PEERSUP*superv*SURG*relapsegoal*IDENT*trapo*AUTO*NONWL

1.000

0.067

1.000

0.800

Note: Pathways are the eight configurations that conform a single solution to explain the outcome TRANSFER; inclS= consistency of the solution; PRI17= Proportional Reduction in Inconsistency; covS=solution coverage18; covU=unique coverage19; cases= cases studied in the research, in this cases, trainees.

We have eight pathways that are part of a single solution. In QCA we have also the parsimonious and intermediate solution, but for the purposes of this evaluation, we have selected the most conservative solution, which is more compatible with the study of causal mechanisms with process-tracing.

NECESSITY ANALYSIS FOR FAILURE TRAINING TRANSFER EFFECTIVENESS No single conditions are identified as necessary for the failure of transfer effectiveness. Therefore, we proceed with several disjunctions or logical unions of two or more conditions. Table 26 shows the necessityconsistency-relevance-of-necessity of nine single conditions and its negations, and Table 28 the disjunctions.

PRI is an “Acronym for Proportional Reduction in Inconsistency. Expresses how much it helps to know that a given condition X is a subset of outcome Y rather than a subset of either Y, its complement ~Y, or the intersection between Y and ~Y” (Schneider and Wagemann, 2012: 345-346). 18 It refers to the percentage of all cases’ set membership in the outcome TRANSFER covered by the solution term. 19 It refers to the percentage of all cases’ set membership in the outcome TRANSFER uniquely covered by a single path of an equifinal solution term (see equifinality ). 165

Table 26 Necessity analysis: necessity-consistency-relevance of conditions for not successful training transfer

Condition PEERSUP SUPERV RELAPSE GOALSETTING SURG IDENT TRAPO AUTO NONWL peersup superv relapse goalsetting surg ident trapo auto nonwl

Cons.Nec 0.1429 0.1143 0.5143 0.2571 0.0857 0.2857 0.8571 0.7714 0.4571 0.8571 0.8857 0.4857 0.7429 0.9143 0.7143 0.1429 0.2286 0.5429

Cov.Nec RoN 0.4545 0.8667 0.4444 0.8913 0.7500 0.8125 0.4737 0.7561 0.4286 0.9149 0.4762 0.7250 0.6818 0.3000 0.7297 0.5652 0.6667 0.7647 0.7692 0.5500 0.7561 0.4737 0.6538 0.7273 0.8387 0.7917 0.7442 0.3889 0.8621 0.8400 0.8333 0.9778 0.6154 0.8810 0.7308 0.7742

Table 27 Necessity analysis: relapse prevention and goal setting in a conjunction for not successful training transfer Condition

Cons.Nec Cov.Nec RoN

RELAPSEGOAL relapsegoal

0.1429 0.8571

0.5000 0.8889 0.7500 0.5000

Table 28 shows a single union of two conditions that are absent. It has high value in consistency, but it is not a perfect one, i.e. it is not 1 (for this crisp-set variant). In addition, the coverage is 78% and the relevance 50%. In conclusion, no necessary unions have been figured out for the outcome negated. Table 28 Necessity analysis: necessity-relevance-consistency-of disjunctions for not successful cases

Condition

Cons.Nec Cov.Nec

nonwl+ident

0.914

0.780

RoN 0.500

166

Table 29 Truth Table for non-successful training transfer Row 7

PEERSUP SUPERV SURG RELAPSEGOAL IDENT TRAPO AUTO NONWL OUT n 0 0 0 0 0 1 1 0 1 7

5 72 152 1 3 6 10 22 39 40 158 203 251 16 13

0 0 1 0 0 0 0 0 0 0 1 1 1 0 0

0 1 0 0 0 0 0 0 0 0 0 1 1 0 0

0 0 0 0 0 0 0 0 1 1 0 0 1 0 0

0 0 1 0 0 0 0 1 0 0 1 0 1 0 0

0 0 0 0 0 0 1 0 0 0 1 1 1 1 1

1 1 1 0 0 1 0 1 1 1 1 0 0 1 1

0 1 1 0 1 0 0 0 1 1 0 1 1 1 0

0 1 1 0 0 1 1 1 0 1 1 0 0 1 0

1 1 1 1 1 1 1 1 1 1 1 1 1 0 0

2 2 2 1 1 1 1 1 1 1 1 1 1 2 3

23 24 38 63 96 128 133 144 172 205 207 208

0 0 0 0 0 0 1 1 1 1 1 1

0 0 0 0 1 1 0 0 0 1 1 1

0 0 1 1 0 1 0 0 1 0 0 0

1 1 0 1 1 1 0 0 0 0 0 0

0 0 0 1 1 1 0 1 1 1 1 1

1 1 1 1 1 1 1 1 0 1 1 1

1 1 0 1 1 1 0 1 1 0 1 1

0 1 1 0 1 1 0 1 1 0 0 1

0 0 0 0 0 0 0 0 0 0 0 0

1 1 1 1 1 1 1 1 1 1 1 1

incl PRI cases 1.000 1.000 C2,C3, E3,J5, L1,P2, T3 1.000 1.000 D3,J2, J4,R1, R2, S4 1.000 1.000 A1,D4, E2,V1 1.000 1.000 C5,F1 1.000 1.000 C1,G1 1.000 1.000 C4,M3 1.000 1.000 M2 1.000 1.000 N1 1.000 1.000 E1 1.000 1.000 K3 1.000 1.000 J1 1.000 1.000 D2 1.000 1.000 P1 1.000 1.000 C6 1.000 1.000 C16 1.000 1.000 H1 0.500 0.500 W2,S3 0.333 0.333 K1,S1, R3 0.000 0.000 J3 0.000 0.000 V2 0.000 0.000 W1 0.000 0.000 T1 0.000 0.000 N2 0.000 0.000 B3 0.000 0.000 S2 0.000 0.000 M1 0.000 0.000 T2 0.000 0.000 B2 0.000 0.000 K2 0.000 0.000 D1

167

SUFFICIENCY ANALYSIS FOR FAILED TRAINING TRANSFER EFFECTIVENESS Table 30 Conservative solution The failed training Transfer Effectiveness 1 peersup*superv*surg*relapsegoal*ident*nonwl

inclS 1.000

PRI 1.000

covS 0.314

covU 0.114

2 peersup*superv*relapsegoal*ident*TRAPO*AUTO

1.000

0.429

0.057

1.000

0.314

0.114

4 peersup*superv*surg*ident*TRAPO*auto*NONWL 5 peersup*surg*relapsegoal*ident*TRAPO*AUTO*NONWL

1.000 1.000

0.057 0.229

0.057 0.057

6 peersup*superv*surg*relapsegoal*IDENT*trapo*auto*NONWL

1.000

0.029

cases M2;N1; C5,F1; C2,C3, E3,J5, L1,P2, T3 C2,C3, E3,J5, L1,P2, T3; D3,J2, J4,R1, R2,S4; D2; P1 C2,C3, E3,J5, L1,P2, T3; A1,D4, E2,V1 E1; J1 D3,J2, J4,R1, R2,S4; C1,G1 K3

7 PEERSUP*superv*surg*RELAPSEGOAL*ident*TRAPO*AUTO*NONWL

1.000

0.057

C4,M3

8 PEERSUP*superv*surg*RELAPSEGOAL*IDENT*TRAPO*auto*NONWL

1.000

0.029

9 PEERSUP*SUPERV*surg*relapsegoal*IDENT*trapo*AUTO*nonwl

1.000

0.029

C16

10 PEERSUP*SUPERV*SURG*RELAPSEGOAL*IDENT*trapo*AUTO*nonwl

1.000

0.029

1.000

0.943

peersup*superv*surg*relapsegoal*TRAPO*AUTO*nonwl

168

ROBUSTNESS TEST TRAINING TRANSFER EFFECTIVENESS Robustness against consistency threshold changes The original consistency of the conservative solution is 0.91. We performed the analysis of sufficiency and choose two different consistency thresholds: 0.7 and 0.87. Solution with threshold 0.7 1 2 3 4 5 6 7

Conservative solution [0.7 threshold] peersup*superv*RELAPSEGOAL*surg*ident*TRAPO*AUTO PEERSUP*SUPERV*relapsegoal*surg*IDENT*TRAPO*nonwl PEERSUP*relapsegoal*surg*IDENT*TRAPO*AUTO*NONWL peersup*SUPERV*RELAPSEGOAL*IDENT*TRAPO*AUTO*NONWL peersup*superv*relapsegoal*SURG*ident*TRAPO*auto*NONWL peersup*superv*RELAPSEGOAL*SURG*IDENT*TRAPO*AUTO*nonwl PEERSUP*superv*relapsegoal*surg*ident*TRAPO*auto*nonwl

inclS 1.000 1.000 1.000 1.000 1.000 1.000 1.000

PRI 1.000 1.000 1.000 1.000 1.000 1.000 1.000

covS 0.133 0.133 0.133 0.133 0.067 0.067 0.067

covU 0.133 0.133 0.133 0.133 0.067 0.067 0.067

8 PEERSUP*superv*relapsegoal*SURG*IDENT*trapo*AUTO*NONWL 1.000 1.000 0.067 0.067 M1 1.000 1.000 0.800

cases J3; V2 B2; K2 M1; D1 N2; B3 W1 T1 S2 T2

Solution with threshold 0.87 1 2 3 4 5 6 7 8

Conservative solution [0.87 threshold] peersup*superv*RELAPSEGOAL*surg*ident*TRAPO*AUTO PEERSUP*SUPERV*relapsegoal*surg*IDENT*TRAPO*nonwl PEERSUP*relapsegoal*surg*IDENT*TRAPO*AUTO*NONWL peersup*SUPERV*RELAPSEGOAL*IDENT*TRAPO*AUTO*NONWL peersup*superv*relapsegoal*SURG*ident*TRAPO*auto*NONWL peersup*superv*RELAPSEGOAL*SURG*IDENT*TRAPO*AUTO*nonwl PEERSUP*superv*relapsegoal*surg*ident*TRAPO*auto*nonwl PEERSUP*superv*relapsegoal*SURG*IDENT*trapo*AUTO*NONWL

inclS PRI covS covU 1.000 1.000 0.133 0.133 1.000 1.000 0.133 0.133 1.000 1.000 0.133 0.133 1.000 1.000 0.133 0.133 1.000 1.000 0.067 0.067 1.000 1.000 0.067 0.067 1.000 1.000 0.067 0.067 1.000 1.000 0.067 0.067 M1 1.000 1.000 0.800

cases J3; V2 B2; K2 M1; D1 N2; B3 W1 T1 S2 T2

We can see that solutions are the same. In addition, we calculated the robustness parameters, i.e. the ratio of the parameters of fit for the core vis-a-vis for the initial conservative solution. The result of robustness fit for this test is: RF_cov RF_cons Robustness_Fit

RF_SC SSR 1

Robustness against dropping cases We drop the first case (B2). B2 was previously member of the path PEERSUP*SUPERV*surg*relapsegoal*IDENT*TRAPO*nonwl with K2.

169

K2 is member of PEERSUP*SUPERV*relapsegoal*surg*IDENT*TRAPO*AUTO with D1. The unique difference is that the context “unbalanced workload” was not included in the minimization process, which mean that it does not care for the analysis. Pathways to training transfer effectiveness

inclS

PRI

covS

covU

cases

1 peersup*superv*RELAPSEGOAL*surg*ident*TRAPO*AUTO

1.000

0.143

J3; V2

2 PEERSUP*SUPERV*relapsegoal*surg*IDENT*TRAPO*AUTO

1.000

0.143

0.071

K2; D1

3 PEERSUP*relapsegoal*surg*IDENT*TRAPO*AUTO*NONWL

1.000

0.143

0.071

M1; D1

4 peersup*SUPERV*RELAPSEGOAL*IDENT*TRAPO*AUTO*NONWL

1.000

0.143

N2; B3

5 peersup*superv*RELAPSEGOAL*SURG*IDENT*TRAPO*AUTO*nonwl 1.000

1.000

0.071

6 peersup*superv*relapsegoal*SURG*ident*TRAPO*auto*NONWL 1.000

1.000

0.071

7 PEERSUP*superv*relapsegoal*surg*ident*TRAPO*auto*nonwl 1.000

1.000

0.071

8 PEERSUP*superv*relapsegoal*SURG*IDENT*trapo*AUTO*NONWL 1.000

1.000

0.071

1.000

0.786

Robustness against calibration. As being a crisp set dichotomization and having used a survey to get the data, it is not possible to modify the way in which the Likert scales were used. For 5 values Likert-scales (being 5 “strongly agree” and 1 “strongly disagree”) the cut point was 4 to be considered as [1] (positive/IN the set) all the values. Values 1,2, 3 were considered [0]. For 7 Likert scales values, the cut point was 4 (Dusa, 2019). Robustness against number of remainders. As being a conservative solution, we do not treat (modify) the minimum number of cases under which a truth table row is declared as a remainder.

170

DICHOTOMIC TABLE CASE ID B2 B3 D1 J3 K1 K2 M1 N2 S1 S2 T1 T2 V2 W1 W2 A1 C16 C6 C1 C2 C3 C4 C5 D2 D3 D4 E1

TRANSFER PEERSUP 1 1 1 0 1 1 1 0 1 0 1 1 1 1 1 0 1 0 1 1 1 0 1 1 1 0 1 0 1 0 0 0 0 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0

SUPERV 1 1 1 0 0 1 0 1 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0

RELAPSE GOALSETTING 0 1 1 1 0 1 1 1 0 0 0 1 0 0 1 1 0 1 0 1 1 1 1 0 1 1 0 0 0 0 0 0 1 0 1 1 0 0 1 0 0 0 1 1 0 1 0 1 0 0 0 0 0 0

SURG 0 1 0 0 0 0 0 0 0 0 1 1 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0

IDENT 1 1 1 0 1 1 1 1 1 0 1 1 0 0 1 1 1 1 0 0 0 0 0 0 0 1 0

TRAPO 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1

AUTO 0 1 1 1 0 1 1 1 0 0 1 1 1 0 1 1 1 0 1 1 1 1 0 1 1 1 0

NONWL 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 0 0 1 1 0 0 1 0 0 1 0 1 171

E2 E3 F1 G1 H1 J1 J2 J4 J5 K3 L1 M2 M3 N1 P1 P2 R1 R2 R3 S3 S4 T3 V1

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0

0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

1 1 0 0 1 1 1 1 0 1 0 1 1 0 1 0 0 1 1 0 1 0 1

0 0 0 0 1 1 0 0 0 0 1 0 1 0 0 0 0 0 0 1 0 0 0

0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0

1 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 1 1 0 0 1

1 1 1 1 0 1 1 1 1 0 1 0 1 0 1 1 1 1 1 1 1 1 1

1 1 0 1 1 0 1 1 1 0 1 0 1 1 1 1 1 1 0 1 1 1 1

0 0 0 1 0 1 1 1 0 1 0 0 1 0 1 0 1 1 0 1 1 0 0

172

ANNEX 10: NOTE ON CALIBRATION IN QCA Since the comparative part of this multimethod research is a cross-case comparison, we have chosen for a crisp set QCA, because of two reasons: (1) the dichotomization of conditions has a lower risk of being a source of mechanistic heterogeneity as fuzzy set; and (2) within a PT approach it makes sense to think of a condition being present or not. From a mechanism perspective, it does not make sense to have a condition that is only present ‘to a certain extent’. In strictu sensu when performing crisp-set we do not use the term ‘calibration’ since “they involve a simple recoding of the raw data instead of a seemingly complicated calibration process” (Dusa, 2019:68). However, in this research we do use the term ‘calibration’, because of the difficulties of dealing with Likert scale values in the presence of conjunctions. In the following three sections we perform the calibration process for the outcome, remote or contextual conditions and proximate or causal conditions. In the following three sections we perform the calibration process for the outcome, remote or contextual conditions and proximate or causal conditions. We have chosen to dichotomize the Likert 5 scales values as considering the values [1,2,3] fully out of the set target and values [4,5] as fully in the target. However, we think of the utility to open a debate about the best strategy to calibrate conditions when using Likert scales with the following theoretical notes to be taken into account: ▪

Conjunction of attributes

The presence or absence of a concept (whether it is the outcome, a remote or a proximate condition) which was theorized as having two or more main attributes will be based on a conjunction as studied in the theory. The crisp set score will be calculated using the logical AND operator. ▪

Multiple items to operationalize one attribute:

When we propose to operationalize an attribute of a concept with multiple Likert-scale items, we envisioned and discussed three main strategies to calibrate the score on the attribute: o

Conjunction principle

This calibration guideline states that one case is member of the condition, when it is also member of each of the attributes of the concept (operationalized as observable manifestation (OM)). The score for the attribute will be calculated with the logical AND operator. The advantage of this approach is that is true to a set theoretical way of conceptualizing, giving the research a good overview on which cases are in the set. This can minimize the sources of mechanistic heterogeneity and therefore be more compatible with a PT approach. The disadvantage is that this rule will probably limit the number of positive cases as members of each condition and outcome under study. However, as Beach says “…in process-tracing, we need to define what attributes must be present for a case to be a member of the given causal concept.” (Beach & Pedersen, 2019: 62). We need also to keep in mind that this research is not only a QCA analysis, but a multimethod research with PT. o

Mean principle

173

This calibration guideline states that an attribute can be present when the mean score on the items is rated as present. This means that not all items need to reach the level of the set target, only the mean score needs to. This approach offers more flexibility than the conjunction principle, especially when the different items tap into different ways of operationalizing the same concept. However, the mean is not the most appropriate when combining QCA and PT for three reasons: -

In a Likert scale of 5 values, the mean will include also cases where the responses are negatives for those observables manifestations (OM) that are necessary to make the attribute to exist. Therefore, one case will be considered as positive if it has value [2] in some items, violating the theory and the conjunctural structure between the OM of each attribute we theorized. A better procedure will be to consider only as IN - positive cases-, the Likert values [4, 5] in the Likert scale, and [1,2,3] as negative or OUT (Dusa, 2019); The mean is not well considered in the QCA community when working with Likert scales (See Dusa, 2019: 96). So, if chosen we would need to provide good reasons to manage this option; The mean is a source of mechanistic heterogeneity for Process-Tracing, for the same reasons mentioned above. We will have in the set of positive cases, those cases with membership in different OM, but member the concept though. This could be risked when studying mechanismoriented causation, because some cases will be explained by different process triggered by the condition, where the cases are members of different OM per attribute. o

Disjunction principle

The disjunction principle proposes that the attribute score will be based on the maximum score among the items used to operationalize it. This principle is draws on the idea that there might be many different ways a certain theoretical attribute or behavior might manifest itself; different ways that might be powerful enough independently to decide the attribute is present. This strategy would lead to more positive cases, but also leads us to expect more mechanistic heterogeneity for the same reasons explained above. We would need to make sure that “presence or absence of the causal concept (…) are compatible with the mechanism-based claim” (Beach & Pedersen, 2019: 62).

174

ANNEX 11: EVALUATING THE EVIDENCE: ARGUMENT ROAD MAPS A11.1 ROADMAP SIGNALING AND RETENTION CAUSAL MECHANISM TRIGGERED BY SUPERVISOR SUPPORT Case 1: N2 Causal relationship Causal mechanism linking supervisor support to improve the trainee’s learned content and stimulate the trainee’s use of learned material to the job with the effectiveness of training transfer. Prior relatively low ●There is no existing research that documents the mechanism that connects supervisor support to training transfer in Flemish firms. Theorized cause: Superior’s commitment to facilitate the retention and motivate the use of the acquired content in a training to the job by employees, during and after a training program takes place. C Observable manifestation: We expect to find evidence in the empirical record of supervisor support in the form of supervisors a encouraging trainees to share what they've learned in training with people in their work environment. Similarly, we expect to see u observables manifestations of discussions between the supervisor and trainees about how to apply competences to job situations; s supervisors giving coaching advice and useful feedback after training on the application in the job of what learned when required. e Finally, we also assume that the supervisor trusts that the trainees is capable to successfully apply what he or she has learned). Supervisor support can manifest itself in many ways. The empirical fingerprints it leaves will be mainly account evidence. There may also be trace evidence, although we expect that most support is given in an informal way, leaving few traces. — Htc, Htu20 ●Relatively high theoretical certainty. If supervisor support is present, we expect to find evidence of this. ●Relatively high theoretical uniqueness: It is unlikely that we find these observable manifestations if supervisor support is not present. ●observatio Account evidence: The survey that was filled in by the Trainee agreed with all items in a list of statements n C1(i) respondent. that measured the dimensions of supervisor support Trainee had a positive response on questions on having mentioned above. supervisor support • Hu: We do not expect that high scores on these items indicate something else than supervisor 20

Note: Htc= High theoretical certainty; Htu = High theoretical uniqueness; Ltu= Low theoretical uniqueness; Hu= High uniqueness; Lu: Low uniqueness, Mu: Moderate uniqueness.

175

●observatio n C1(ii)

support. It seems unlikely that the employee would think that her supervisor would see the results, because anonymity was guaranteed when filling in the survey. Therefore, we can trust on the source. • Strong confirmation of condition 1 Account evidence: Interview with trainee. During the interview, trainee claimed to have received support from supervisor.

22:54 “Q Did you experience support from your supervisor? Yes absolutely”

•

Mu: The information provided is modest, because it was oriented to get an affirmation about the involvement of supervisor in training transfer. We expect trainee to be honest, because there is no clear incentive to hide about how she perceives the support by her supervisor. Anonymity was also stressed before the interview. • Moderate confirmation of condition 1 ●Aggregatio Our overall judgement is that it’s very likely that there was indeed supervisor support in this case, because both pieces n of are found. evidence for We can, therefore, confirm the presence of the condition ‘supervisor support’. proposition If only C(i) is found, we can confirm the evidence. C1 Sources are relatively independent. Overall confirmation: strongly warranted, given C1(i) is accurate evidence of the presence of supervisor support and C1(ii) even if it is modest information, it confirms an environment of supervisor involvement. Building block: Ascribing importance to training. Theorized part: Supervisor ascribes importance to the training program and takes initiative to let the employees follow the training. Fingerprints: We expect to find evidence of supervisor’s engagement with the training and trainees, such as taking initiative to let the employees follow the training and signs that reveals the ‘importance’ of the training for him/her. We expect that this can take the form of account evidence as well as trace evidence of actions that the supervisor has undertaken. — Htu ●Theoretical certainty not formulated (no priors) ●Relatively high theoretical uniqueness. Highly unlikely to find this fingerprint, as it looks like, without the mechanism of ‘signaling and retention’ being operative. 176

●observatio n P1(i)

Account evidence: Interview with trainee. During the interview, trainee discussed the signs of engagement of the supervisor (‘they’re convinced; it was pleasant’). • Hu: This piece of information is empirically unique. Trainees mentions three relevant aspects here: the training organization comes from above; they show positive signs of the training relevance (convincement) and they think it was ‘pleasant’. These elements clearly reflect the engagement from supervisors. We can trust in the source because there are no reasons for why the trainee would affirm this statement or why direction had other motives. • Strong confirmation proposition 1

31:00 “Direction organized this [the training], and they were really convinced by this and that was pleasant. They also joined some sessions and asked for input. So yes, from the organization, I think there was enough engagement”

●observatio n P1(ii)

Trace evidence: the company also participates in a different project “Anders Organiseren”, as advertised on their website. This project is focused on changing the management style at the company, which also has implications for how leadership and communication happens. This illustrates that the company is open for innovation and willing to change how they organize (including providing training to employees). • Lu: The proof of participation, if found, means that they are open to innovation, but does not tell us much on whether or not the supervisor made the employees follow the training. Therefore, we cannot trust on this source as such. • Weak confirmation of proposition 1 Account evidence: Interview with trainee. In this quote, trainee notes that the supervisor was one of the driving forces.

They advertise participating in this project on their website. This proof of participation is found in annex 12.

●observatio n P1(iii)

23:00” he was also one of the driving forces behind the project. He is a “believer”.

177

•

• ●observatio n P1(iv)

Hu: This piece of information tells us something about how supervisor is being perceived by trainees, e.g. ‘believer’, which is a clear proof of ‘signs’ that the training is important for the trainees. We can trust in this source, because there are no alternative reasons for why the trainee would say this statement. Moderate confirmation of proposition 1

Account evidence: Interview with trainee. This excerpt shows that they are “on board”. • Mu: The piece of evidence tells us something about the signs showed by the supervisor. The fact to be ‘on board’ can be perceived by trainees as a ‘sign’ of engagement and relevance of training for the employees. We can interpret that to be “on board”, is a ‘sign’ of relevance, but we would need probably more observations to confirm that this quote implies this interpretation. • Moderate confirmation proposition 1

“Yes, it was good. And of course, if “directive” [supervisor] is not on board, it would be totally different. They are at the steering wheel of the company.”

●Aggregatio Due to these four pieces of evidence are found, we can confirm the presence of the part 1 of the mechanism. n of If only p1(i), or p1(iii) or p1(iv) is found, we might infer that P1 is present. evidence for Sources are relatively independent. proposition Overall confirmation: strongly warranted, there is a significant amount of evidence that the supervisor(s) attached 1 quite a bit of importance to the training. Building block: Ascribing importance to training. Theorized part: Employees react by putting the training in their agenda. [because they do not have choice]. Fingerprints: We expect to find evidence on employees reacting by accepting some sort of invitation to attend the training. We assume there could be both account evidence of this as well as trace evidence. — Htu ●Theoretical certainty not formulated (no priors). ●Relatively low theoretical uniqueness. Just because the trainee planned to attend the training, this does not really say anything on why the supervisor was essential for the training transfer to occur. 178

●observatio n P2(i)

3 a

Trace evidence: The acceptance of the invitation to participate in training. This may have taken the form of a (positive) reply on an email. • Hu: These pieces of evidence as a whole show that there was agreement to meet each other. If the list of attendance for the meeting can be seen, this would also count as sequence evidence as the invitation than must have been accepted beforehand. There are few alternative explanations as to why the trainee would put the training in their agenda, if they did not plan to attend it. • Strong confirmation proposition 2

The acceptance of the invitation. She said she can’t find these. We do have a mail in which she says “I will send everyone an outlook request” which is indirect proof of an invitation. This, however, applies more to the colleagues than to her and think the invitation is not for the training itself, but for the “follow-up meeting”. List of attendance for the meeting. We have the attendance list (or something that looks like it) VC indeling groepen verdiepingssessies 2019.xlsx. I think this is our best evidence, maybe combined with some reasoning how this step cannot not have happened it the previous and next step are confirmed. (See Annex 12)

●Aggregatio Overall, it looks quite clear that this step took place in our case, because the pieces were found, and they are accurate n of evidence of the presence of proposition 2. evidence for Sources are relatively independent. proposition Overall confirmation: strongly warranted, p2(i) is accurate evidence of the trainee reacting to the invitation to take the 2 training by putting it in agenda. Building block: Employees follow the training. Theorized part: Everybody follows the training in group, in part because it was mandatory to do so by the supervisor. Fingerprints: We expect to find that, the supervisor makes everyone (all employees) follow the training together in group. — Ltu ●Theoretical certainty not formulated (no priors) ●Relatively low theoretical uniqueness. Just because the trainee attended the training does not mean that the mechanism of ‘signaling and retention’ is being operative in this case. ●observatio Account evidence: Interview with case. 34:15 “Q was the following of the training voluntary? n P3a(i) This interview excerpt illustrates that it was obligatory to No. It didn’t feel mandatory, perhaps for others it did, follow the training. but it was mandatory.” • Hu: This piece of information shows that trainee claims that training was obligatory. We think there is no reason to affirm the ‘obligatoriness’ when it was not, therefore we could trust on this source. • Strong confirmation proposition 3a 179

●observatio n P3a(ii)

3 b

Account evidence: The survey the trainee took after the To the statement “My supervisor informed me that the training. training was mandatory”, she replied “agreed”. When asked if her supervisor told her that the training was obligated. She replied “agreed”. • Hu: In an anonymous survey there is no reason to say ‘agree’ when is ‘disagree’ and vice-versa. We, therefore, interpret this observation as evidence that training was mandatory for trainees. We can therefore trust on this source. • Strong confirmation of proposition 3a ●observatio Account evidence: Interview with other employee at the 2:32 “Q Why did you participate in the training? n P3a(iii) organisation. On the one hand because we were obligated to. But This interview excerpt illustrates that it was obligatory to even though it was obligated. I still would have done follow the training. it.” • Hu: This piece of information shows that other employees also claim that training was obligatory. This further strengthens our confidence in this proposition. • Strong confirmation proposition 3a ●Aggregatio In general, we think that these observations are mostly strong pieces of evidence to confirm the proposition 3a. n of Sources are relatively independent. evidence for If only p3a(i) is found, or p3a(iii) is found, we might infer that P3a is present, because we think that just finding one proposition piece of these pieces of evidence is already sufficient to claim that there is significant support for this proposition. 3a Overall confirmation: strongly warranted, given our two supportive observations are accurate evidence of that supervisor make everyone follow the training together in group. Building block: Facilitating learning climate. Theorized part: In parallel supervisors enable employees to follow the training by taking over the workload during the training period, with the aim that employees can be focused on learning the training content. Fingerprints: We expect to find evidence of supervisors arranging for the workload of the employee to be taken over during the training period so that the employee can focus on the training. This can take the form of e-mails where these arrangements are discussed or verbatims provided by trainees. We expect to find account evidence and also trace evidence to measure this proposition. — Htu ●Theoretical certainty not formulated (no priors) 180

●Relatively high theoretical uniqueness. Highly unlikely to find this fingerprint of supervisor taking over the workload, as it looks like, without the mechanism of ‘signaling and retention’ being operative.

●observatio n P3b(i)

●observatio n P3b(ii)

Account evidence: Interview with case. During the interview, trainee mentioned that the workload is taken over. • Hu: The two observations contain information about the balanced workload to apply the training. The organization of space and time has played an important role. We can trust on this source because there are no reasons why trainee would claim such a thing in two different questions. • Strong confirmation proposition 3b Account evidence: Interview of colleague. Trainee also stated that his/her workload was taken over when he/she needed to follow the training. • Mu: Trainee mentions that everybody has a back-up during the training so that they do not have to worry about their job. This allows them to focus on the training better. Since he/she was also in the training, this includes him/her as well. However, because he/she was also involved in this training as an employee of the HR department, he/she may have been focusing on others when answering this interview question. • Moderate confirmation proposition 3b

30:09 “Q Did the organization support you in applying the training They're the same. Not structurally, but in some specific cases: space, time, the fact that you ’re workload is taken care of” And 23:08” Q how did they concretely give you support? Well the fact that it is organized, that it is followed, that your workload is taken over.” 43:17 “Q What according to you was important to absorb the training? For me personally, I need to be able to be separated from the work environment. So that’s very important I think. Thus everyone has a back-up that does their job at that time. And that with the ten, twelve people that are there, you can focus on the training. That you don’t get disturbed or that you are stressing and thinking that you should have been at your work for a certain reason. The fact that you can just get away from your work for two days, if I think back to the last part of the training.” And 2:20 “Your job was also done by someone else so that you could go there without distractions.” 181

●Aggregatio n of evidence for proposition 3b In te r m e di at e o ut c o m e

Everything considered, there seems to be enough evidence to conclude that proposition 3 is present. Sources are relatively independent. If only p3a(i) and p3a(ii) is found, we might infer that P3a is present, because we think that just finding both pieces of these observations, we can claim that there is significant support for this proposition. Overall confirmation: strongly warranted, given our two supportive observations are accurate evidence of that supervisor made arrangements for the workload of the employee to be taken over during the training period. Theorized part: This create a sort of organizational climate where employees perceive the importance of the training for their job, and where they acknowledge the engagement of the supervisor encouraging this goal. Observable manifestations: We expect to find evidence of employees who perceive the training to be important for their job or acknowledge the engagement of the supervisor. We expect that this can take the form of account evidence. — theoretical Ltu ●Theoretical certainty not formulated (no priors). ●Relatively high theoretical uniqueness. It is unlike to find trainees acknowledging the relevance of the training and the engagement of the supervisor in this intervention, if an ad hoc organizational work climate is not being taking place. The work climate is dynamic, and the signals provided from supervisor since the beginning helped to create this work environment, as an intermediate outcome within the causal mechanism. ●observatio n IO(i)

Account evidence. Interview with case. During the interview, trainee acknowledges that, in hindsight, it was good that she followed the training. • Hu: This observation tells us something about the way the training was valorized. Trainees mentions that in the beginning she did not know what to expect from the training but that afterwards she recognizes that it was ‘definitively good’. We trust on this source because we do not see alternative empirical explanations for the existence of this piece of evidence. • Moderate confirmation intermediate outcome.

24:12 “Q beforehand, did you feel like you needed the training? It’s not a training that I indicated that people had to follow, does not mean that I was hesitant to take the training, but I didn’t know what to expect. Beforehand, I did not ask specific questions… or felt like I needed to be trained in this. But, in hindsight, it was definitely good.”

●Aggregatio n of evidence for Intermediate Outcome

The piece of evidence was found. We can, therefore, confirm the presence of the intermediate outcome in this case. Source is relatively independent. Overall confirmation: warranted, given C01(i) is accurate evidence of the presence of some sort of acknowledge of the training afterwards and we can trust on the source, confirming thus the presence of this intermediate outcome.

182

Building block: Motivation to generalize. Theorized part: Because the relevance of training is perceived, employees following the training feel motivated to use it and discuss the training content with other peers. Fingerprints: We expect to find evidence on the motivation/inspiration of employees. We expect to find this mainly in account evidence. — theoretical Hc, Lu ●High theoretical certainty. We expect to find some account evidence on this. ●Low uniqueness; This does not directly relate to how supervisors would assist in transferring training content. ●observatio Account evidence. Interview with trainee. 30:09 “Q Did the organization support you in applying n P4(i) In the interview, the trainee noted that there is a lot of the training enthusiasm around the training. […] the fact that direction also asks to HR: “brainstorm • Hu: The observation suggests that indeed there was with us how we are going to keep this alive in the period to come”. This makes that there is a lot of some sort of enthusiasm on this training as well as enthusiasm and positivity around this.” discussions with peers such as brainstorms about the use of the training (‘keep this alive’). We cannot see a clear alternative explanation for this quote, so we can trust on this source and what it means. • Moderate confirmation of proposition 4 ●Aggregatio The piece of evidence is found. We can, therefore, confirm the presence of proposition 4. n of Source is relatively independent. evidence for Overall confirmation: warranted, given P1(i) is accurate evidence of the presence of motivation/enthusiasm around proposition the training application and work environment (brainstorm with peers). 6 Building block: Motivation to generalize. Theorized part: Employees tries out/use the training in tasks-related matters keeping the level of motivation that 'they just have to try it to learn' within an environment of trust and cohesion. Fingerprints: Expect to find evidence on employees who try out to evaluate themselves about how to use the training in their tasks. This will probably be account evidence, but there could also be other trace evidence, such as documents that show preparation for specific conversations. — Htu ●Theoretical certainty not formulated (no priors). ●Relatively moderate theoretical uniqueness. This does not directly relate to how supervisors would assist in transferring training content.

183

●observatio n P5(i)

●observatio n P5(ii)

●Aggregatio n of evidence for proposition 5

Account evidence. Interview with case. 29:40 “Q How did you handle this obstacle? This shows that, even though they experienced difficulties, That is keep on trying and evaluating yourself. Or they just kept on trying to apply the training. thinking with someone else about how you could try certain things again. This sounds very “heavy”. • Hu: This observation tells us something about the (laughs). There are also many moments in which it just way in which trainees handle with difficulties in the goes well.” use of training. Trainees mentions that they just kept on trying and evaluating themselves, also within an autonomous context or a non-hierarchical organization. We therefore, trust on this source, because there is no reasons to make such a reflections if to handle obstacles is not present. • Strong confirmation proposition 5 Account evidence. Interview with case. 43:00 “ For me it’s not about being able to perfectly to This shows that trainee tried to apply (parts of) the training. replicate it perfectly. It’s about “picking up the glove” and during the training making a couple of practical • Hu: This observation tells us something about the things that you think of of which you know “that is my effort made by the trainees to apply the training pitfall” or “this is something I never do, but it would be content. We can observe that trainee refers to better if I would do it. “ For me it is about 5 things that ‘taking the glove off’ rather than ‘replicate the training perfectly’. This observation also reveals the are anchored in my head and that I will effectively fact that trainee recognized his/her own pitfalls and apply. becomes better in identifying where to effectively apply the training. We can therefore trust on this source since is quite unique and non-alternative empirical explanations have been found beyond the presence of this part. • Strong confirmation proposition 5 The two pieces of evidences were found. We can confirm the existence of proposition 5. Sources are relatively independent. If only p5(i) or p5(ii) is found, we might infer that P5 is present, because we think that just finding one piece of this proposition is already sufficient to claim that there is significant evidence for the use of training in tasks related matters. Overall confirmation: strongly warranted, given our three supportive observations are accurate evidence of employees who try out to evaluate themselves about how to use the training in their tasks.

184

Building block: "Keeping it alive" signaling. Theorized part: Supervisors keep on reminding to use the training ("keeping it alive") and provides feedback on the tasks related to the training application. [There is feedback loop between part 5 and 6]. Fingerprints: We expect to find reminders of the supervisor to use the training and feedback on the use of the training. This can take the form of both account evidence as well as trace evidence. — Htu ●Theoretical certainty not formulated (no priors). ●Relatively high theoretical uniqueness. Highly unlikely to find this fingerprint, as it looks like, without the mechanism of ‘signaling and retention’ being operative, because supervisor engagement and feedback as ‘keeping it alive’ signaling are core aspects of the theorized causal mechanism. ●observatio n P6(i)

Account evidence. Interview with case. During the interview, trainee discussed an interaction she had with her supervisor. • Mu: This fragment reveals that trainee talked about the application of the training with her supervisor. She mentions ‘a big correction’ that can be related to some sort of feedback. The piece of information confirm the existence of supervisor involvement in possible remainders but we are still in the dark about what exactly trainee mean. • Moderate confirmation proposition 6

●observatio n P6(ii)

Account evidence. Interview with case. During the interview, trainee discussed the feedback she got from her supervisor. • Hu: In this piece of evidence, trainee responds positively when asked if she received feedback from her supervisor in her training applications, highlighting that she was also actively involved in recognizing what did work or not in her application,

17:30 “earlier you said, you also applied the training content when talking to a colleague. And afterwards, you also think for yourself: I have forgotten “the request” or something like that. Is that something that you talk about with other colleagues? Perhaps the colleague you talked with, or other colleagues? And where they themselves also indicate : “I also followed the training and you forgot the request” or “this was unclear to me”. Sometimes yes and sometimes not. In a couple of cases, especially from HR, there are some conversations that I afterwards discuss with my supervisor… In some conversations, there are also references to the training. Recently we had a big “correction” talk with an employee and a supervisor.” 22:16 “did you ever get feedback from your supervisor? About how you applied the techniques in conversations? Yes, in my case yes, because I often report conversations to my supervisor. You make some remarks yourself. “this worked” or “this didn’t work, I should handle this differently”. Yes.”

185

• ●observatio n P6(iii)

●observatio n P6(iv)

●Aggregatio n of evidence for proposition 6

and how she reported these reflections to her supervisor. We can trust on this source, since we did not see alternative empirical explanations for the existence of this piece other than the involvement of supervisor with feedback in training application. Moderate confirmation proposition 6

Account evidence. Interview with case. During the interview, trainee mentions again, spontaneously, that she gets feedback from her supervisor. • Hu: This piece of evidence tells us something about the presence of supervisor involvement in keeping alive the training content and the feedback received in team meetings and beyond. Trainee clearly mentions the feedback she gets from her supervisor. We can trust on this source for these reasons. • Strong confirmation proposition 6 Account evidence. Interview with case. During the interview, trainee mentions that her supervisor paid attention to the application of the training and tried to “keep it alive”. • Hu: This piece of information reveals that supervisor tried to keep the training alive, despite the obstacles and difficulties to do so. We can trust on this source since it reveals the (perceived) involvement of supervisor in keeping the training alive. • Strong confirmation proposition 6

23:15 “Well of course I’m looking together with my supervisor into how we can keep this alive. We are quite in the middle of it. So specifically, that is something we talk about in some team meetings. “oh that is something that we have to discuss again”, “what is your feeling, what is my feeling?” yes.. some time that he wants to spend on this. The conversations we have about it, the feedback he gives. Those are the concrete examples”

46:27 “but if someone else uses something, it’s not really an obstacle, but rather a memory? Yes, that’s right… an obstacle… the difficulty for every training is to keep it moving in the very long term. That’s always a potential danger. But again, I think that walk your talk has done some effort and delivered some tools, to keep these things alive. And the fact that there has been some attention for this from the supervisor in HR.”

We have found many observations for proposition 6. We can confirm the presence of this proposition for the causal mechanism. Sources are relatively independent. Every single observation, if found, can confirm the evidence, except for P6(i) which has a moderate confirmation. Overall confirmation: strongly warranted, given P6(ii-iv) are accurate evidence of the presence of supervisor support in keeping alive the training and the feedback provided to the trainees. 186

Building block: Increasing generalization. Theorized part: Due to the peers-supervisor engagement and trust, post-training evaluations feedback systems are implemented by supervisors until task-oriented new knowledge is retained and improved in its application by employees. Fingerprints: We expect to see supervisors implementing post-training evaluations feedback. This can take the form of trace evidence or account evidence. — theoretical Hc, Hu ●Theoretical certainty not formulated (no priors). ●Relatively high theoretical uniqueness. Highly unlikely to find this fingerprint, as it looks like, without the mechanism of ‘signaling and retention’ being operative. ●observatio Account evidence. Interview with case. “And this is something we are looking at from HR, how n P7(i) During the interview, trainee mentions that they want to do we keep this alive in the future? And those who “keep it alive” the training content in a post-training context trained us, walk you talk, they left some tools to keep this alive. And that is good. We are searching for how • Hu: This piece of information tells us something about the interest of pursuing some activity that can we will apply this.” enable to retain and improve the application of content training. Trainee mentions “how to keep this alive in the future?” showing her/his interest. • Moderate confirmation proposition 7 ●observatio n P7(ii)

Trace evidence. Slides that were used after the training to evaluate the training. These slides show that afterwards, the supervisor also gave additional attention to the training and it’s (successful) implementation. • Hu: This piece of information tells us something about the interest of pursuing some activity that can enable to retain and improve the application of content training. When looking in the document properties at the person who last edited the slides, the name of the supervisor was listed, indicating his active involvement. • Strong confirmation proposition 7

After the training, the supervisor organized additional meetings on the training and its implementation. Slides for a meeting on the training (and what is going well/wrong) can be found in annex 12.

187

●Aggregatio n of evidence for proposition 7 O u t c o m e

Two observations have been found, but it does not tell us a whole story about the engagement of supervisor in posttraining activities. We cannot confirm the presence of proposition 7 in a strong way, but rather in a moderate way. Sources are relatively independent. Overall confirmation: strongly warranted, given p7(i) tells us something about the interest in keeping the training content alive in the future from the side of peers and p7(ii) shows us the additional attention provided by supervisor after training. Theorized outcome: Effective employee training transfer is the application or use of the learned knowledge (content, skills or attitudes) acquired in a training program to the job by trainees, which is maintained over a period of time. Observable manifestations: The employee applies the learned content and skills and they are maintained on the job. Evidence for this can take the form of trace evidence or account evidence. — theoretical Hc, Hu ●Relatively high theoretical certainty. We have to find evidence of the presence of training transfer. ●Relatively high theoretical uniqueness. It is unlike to find these fingerprints if training transfer is not present. ●observatio Account evidence. The survey that was filled in by the In the survey, the trainee answered several questions n O1(i) respondent. on the training and the specific skills that were trained. Trainee had a positive response on questions on the For generalization and maintenance based on Govaerts application of the training She answered 4 on all questions . • Hu: This observation reveals the presence of training transfer in this particular case for questions related to training and specific skills and for generalization and maintenance aspects of the outcome. • Strong confirmation of outcome ●Aggregatio n of evidence for proposition O1

The evidence for the presence of the outcome was found previously to the study of causal mechanism. We have already confirmed its presence in the comparative study. Overall confirmation: strongly warranted since the outcome was already studied in the comparative part of this study.

A11.2 ROADMAP ENHANCED LEARNING TRANSFER INTERVENTION TRIGGERED BY PEER SUPPORT Case 1: D1 188

Causal relationship Causal mechanism linking peer support as the colleague’s commitment for employees to improve the trainee’s learned content and stimulate the trainee’s use of learned material to the job with the effectiveness of training transfer. Prior relatively low ●There is no existing research that documents the mechanism that connects peer support to training transfer in our empirical cases. Theorized cause: Colleague’s commitment for employees to improve the trainee’s learned content and stimulate the trainee’s use of learned material to the job. C Observable manifestations: We expect to find evidence in the empirical record of colleagues trying to minimize breaking-off from a work that interfere in the opportunity to practice the newly learned skills in the peer’s work unit; peer helping others with technical u knowledge to apply the techniques learned in the training; peers encouraging others to use the skills they learned in trainings; peers s providing positive feedback to others about their performances; peers reinforcing the use of new knowledge acquired by others in e the training. This could be measured using account evidence (from interviews with involved actors). — Htu21 ●Theoretical certainty not formulated (no priors) ●Relatively high theoretical uniqueness, as observing these fingerprints necessarily means the presence of peer support, based on theory. ●observatio Account evidence: The survey that was filled in by the On the peer support question in our survey (based on n C1(i) respondent. Cromwell (2004) and Holton et al. (1997)), trainee Observables manifestations as a whole: Trainee had a agreed with all the statements (score of 4). These positive response on questions that measured if she received specific items can be found in Table 2. peer support • Hu: This observation tells us something about the presence of peer support in this particular case. The trainee agreed with all the statements related to the support provided by colleagues in the transfer of training. It is, therefore, unlikely that trainee would answer these questions this way if she/he did not experience peer support. Anonymity in the survey was guaranteed. We can trust on this source. • Moderate confirmation of condition 1.

Note: Htu= high theoretical uniqueness; Hu = High uniqueness; Mu = Moderate uniqueness; Lu = Low uniqueness.

189

1 a

●observatio n C1(ii)

Account evidence: Interview with case. Observable manifestations as a whole: During the interview, trainee discussed the support from peers that she/he received from colleagues. • Hu: These pieces of evidence are complementary and tells us something about the presence of peer support as a causal condition for transfer. We can observe that peer support is considered as ‘complementary’, ‘needed’, within a context where the ‘work culture’ is oriented to a ‘collaborative mentality’. In addition, the atmosphere and daily working also have been mentioned as facilitators of per support. We can trust on this source because it is unlikely to be explained by alternative explanations, since there are no reasons to make affirmation that does not corresponds with reality when talking about training transfer and support from colleagues. Why someone could say that experienced ‘peer support’ if this was not the case? These pieces of information are also in line with the information we get from trainee in the survey. • Strong confirmation of condition 1

39:20 “They’re also complementary because it’s a group that has certain strengths and points for improvement. They help each other in that regard. Q So the atmosphere is good? Yes. It’s also necessary because they have to work together every day. And they have to set the right example. Q I think it’s interesting you say “they help each other” .. Yes, but we do have that mentality. We’re not a company that has an “island” culture [meaning that everyone stays on his own job and ignores the rest] Q and what make you notice that? Hoh, everything. First, the atmosphere. Second, the daily way of working. If supervisor [name protected] has an issue “I really don’t know how to handle this”, then she will just discuss this with someone like [name protected] or [name protected] or [name protected]. It’s not that afterwards. how do I say this? There is some sort of abuse. No everyone helps each other if its needed.

●Aggregatio n of evidence for proposition C1

In general, the presence of the cause seems supported by evidence, because both pieces are found. We can, therefore, confirm the presence of the condition ‘peer support’. If only C(i) is found, we can confirm the evidence. Sources are relatively independent. Overall confirmation: strongly warranted, given C1(i) and C1(ii) are accurate evidence of the presence of peer support.

Building block: Following the training. Theorized part: Peers follow practice-oriented training together within a 'flat' atmosphere.

190

Fingerprints: We expect to find evidence in the empirical record of the employees who have followed the training together, within a non-hierarchical organization. Since, having followed the training was an obligation to participate in this research, it may therefore be a prior in each case. We may easily find account evidence and trace evidence of participation in the training. — Mtc — Htu ●Medium theoretical certainty, since we need to find this fingerprint. Based on general theory (not Flemish firms) and our empirical priors (survey), data reveal that this fingerprint was present at the first phase of the project, since we have to find it. ●Relatively high theoretical uniqueness. Highly unlikely to find this fingerprint, as it looks like, without the mechanism of ‘enhanced training transfer’ being operative. ●observatio Trace evidence: attendance list of training. The attendance list can be found in Annex 12 . n P1a(i) The attendance list displays who participated in the training. The name of our case is included in this list. All attendants have signed this document with their autograph. • Hu: The attendance list is quite empirically unique, because we can trust the source and it matches with the information provided by the trainee. Finding this piece of evidence is difficult to account for with alternative empirical explanations. • Strong confirmation of proposition 1a ●observatio n P1a(ii)

Account evidence: Interview with case. During the interview, trainee mentioned that it was important that everybody was there (training together). • Hu: The evidence is empirically unique. There are no reasons to say that one changed the date of the training because someone was missing, if this really did not happen. Trainee recognize the importance to follow the training together from the beginning to the end. The reasons are clearly related to pave the way for a common vision on leadership. Therefore, finding this piece of evidence are difficult to account for empirical alternative explanations, other than the trainees following the training together within a nonhierarchical firm. • Strong confirmation proposition 1a

57:30 “We just changed the date. Because we really think it was necessary that everyone was there and that everyone could follow the trajectory from start to finish. To have the shared vision on leadership… at the end of the trajectory.”

191

1 b

●Aggregatio Because these two pieces of evidence are found, we can confirm the presence of the part 1 of the mechanism. n of If only p1a(i) is found, or p1a(ii) is found, we might infer that P1 is present. evidence for Sources are relatively independent. proposition Overall confirmation: strongly warranted, given our three pieces are accurate evidence of the peers participating in 1a the training, together. Building block: Following the training. Theorized part: Because peers follow practice-oriented training together within a 'flat' atmosphere, peers practice quite a lot the content of the training, and recognize the 'relevance' of its content (during the training) and the 'fact to do it together' for their workperformance [to be on the same page!]. Fingerprints: We expect to find evidence on peers practicing the content of the training and recognizing the relevance during the training. Given that we do not expect there to be any trace evidence (e.g. video recordings of the training or used exercise sheets), we will need to rely mainly on account evidence. — Htu ●Theoretical certainty not formulated (no priors) ●Relatively high theoretical uniqueness. Highly unlikely to find this fingerprint, as it looks like, without the mechanism of ‘enhanced training transfer’ being operative. ●observatio Account evidence: Interview with case. 55:29 “Q Was there, during the training, a lot of n P1b(i) During the interview, trainee mentioned that she “tried” the interaction? training content during the training. Very much! But this was also our demand. We’re an audience that requests this. Don’t put us in a training • Hu: The piece of evidence is quite unique, because there are no alternative empirical explanations as to where they have multiple days of talking about theory. There will be resistance against that. why the trainee would be dishonest about Q and what did this interaction look like? ‘practicing the training’ and ‘recognizing it Well, yes by exercising in different ways relevance’. Trainee recognize the interaction and Q by exercising… are that role-playing games the fact that they demand the training with a Roleplaying games, coaching conversations that are practical approach. We can therefore trust on this heard, yes… source. Q discussions… • Strong confirmation proposition 1b Yes” ●Aggregatio The single piece of evidence is found, we can confirm the presence of the proposition 1b. n of Sources are relatively independent. evidence for Overall confirmation: strongly warranted, given p1b(i) is strong evidence of trainees practicing the training and proposition recognizing its relevance during the training as well. 1b 192

In te

Building block: Building up common understanding. Theorized part: During the training, peers communicate their different views about training implementation, in an open way, with the result that (1) they learn to trust each other even better and 2) that they acknowledge that a different way to work could improve their professional skills. Fingerprints: We expect to find that, during the training, peers communicate and discuss their different views of applying the training. We expect that this takes the form of discussing specific, sometimes sensitive, cases and different approaches on how to handle them. Fingerprints on this part can be more difficult to find given that what is discussed is sometimes sensitive and a record on these discussions is unlikely to exist. We should, however, expect participants to acknowledge that these discussions happened. — Htu ●Theoretical certainty not formulated (no priors) ●Relatively high theoretical uniqueness. Highly unlikely to find this fingerprint, as it looks like, without the mechanism of ‘enhanced training transfer’ being operative. ●observatio Account evidence: Interview with case. 49:18: “Q Following the training together. Is that n P2(i) This interview excerpt illustrates that there have been something that is important? discussions during the training on how the content should be Yes. applied. Q and why? Because… to be on the same page. Yes really. During • Hu: This observation tells us something about how the training you deal with others opinion, that differ the training content may be applied. The trainee from your own opinion, in an open way. But, by mentions how they dealt with different views in an working on this in group, 1) you get to know each open way, collectively (in group) and how to know other even better and 2) supervisors that may be each other better was a good thing. There are no working on production less, learn that you have to alternative empirical explanations as to why the work in a different way if you are a supervisor in trainee would say this affirmation. We expect that her recollection of the training is good and truthful. production. “ We can trust on this source. • Strong confirmation proposition 2 ●Aggregatio The single piece of evidence is found. We can confirm the presence of the proposition 2 because we trust on the n of source. evidence for Sources are relatively independent. proposition Overall confirmation: strongly warranted, given p2 (i) is strong evidence of trainees having discussions during the 2 training on how to implement it and the results of that on their collective understanding. Building block: Building up common understanding. Theorized intermediate outcome: Because of that, peers gain a 'common understanding' on how to implement the training.

193

r m e di at e o ut c o m e

Observable manifestations: We expect to find evidence concerning the existence of a common understanding of the training. This could take the form of both account evidence as well as trace evidence if there is some official stance on the topic in the company. The absence of certain evidence (for example of disagreements) can also improve our belief in this part. ●observatio Trace evidence of the organisation of a day during which a Email invitation to a meeting to discuss a common n IO(i) common vision is communicated to all employees vision on leadership can be found in annex 12. Document that shows that the organisation aims to have a specific vision on leadership. We found an e-mail invitation for several leaders so that they could discuss how they view leadership in their organization. • Hu: The document clearly reflects the organization aims to have a specific vision on leadership. There are few credible alternative explanations for finding this piece of evidence. We can also trust on this source, because it is an official invitation from the organization. • Strong confirmation of intermediate Outcome ●observatio n IO(ii)

Trace evidence of the organisation of a day during which a common vision is communicated to all employees Document that shows that the organisation aims to have a specific vision on leadership. We have the slides that were used when this vision was communicated to all employees. • Hu: The document clearly reflects the organization aims to have a specific vision on leadership. There are few credible alternative explanations for finding this piece of evidence. We can also trust on this source, because it is an official invitation from the organization. • Strong confirmation of intermediate Outcome

The slides that were used during this day can be found in annex 12.

●observatio n IO(iii)

Account evidence. Interview with case. During the interview, trainee implied that there was a shared understanding of the training, which if it would have been absent, they had not followed the training together.

50:15 “I can only see benefits to do it in group together. If you would send everybody to the training individually, I don’t think that the result would be the same. 194

•

• ●Aggregatio n of evidence for intermediate outcome 3

Hu: This observation tells us something about the benefits of pursuing the training in group, with good results. Trainee is convinced about this add value. This piece of evidence is unlikely to be explained by alternative explanations. We can therefore trust on this source. Strong confirmation of intermediate outcome

Q What would then be the result? I think that everyone would have his own implementation of the training. And the shared would not have been there to the same degree, I think.”

Due to these three pieces of evidence are found, we can confirm the presence of the part our intermediate outcome. If only IO(i), is found we cannot confirm, whereas if IO (ii) or IO (iii) is found, we might infer that IO is present, because they are more at the level of trainee. IO(ii) is important in the extent to which can be combined with the other pieces of evidence. Overall confirmation: strongly warranted, there is a significant amount of evidence on the presence of peers’ common understanding to implement the training in this case.

Building block: Intervision Theorized part: In parallel, peers propose to handle 'intervision moments' as a 'peer coaching activity' to better implement the training content to the job. Fingerprints: We expect to find evidence of ‘intervision moments’ which were planned, and in which peers discuss the training and its application. This evidence of planning can take the form of trace evidence in the form of invitation through emails or account evidence. ●observatio Account evidence. Interview with case. 15:25 “We made sure that, internally, that we regularly n P3(i) During the interview, trainee mentioned that ‘they made had “intervision moments” with the supervisors and sure that’ there would be meetings between supervisors to that we came together and that we… talked about for discuss the training. This implies that these meetings were example “what is going smooth for me? What is going planned. less smooth? What are we running into that makes it more difficult to have that coaching” • Hu: This piece of evidence clearly mentions the ‘intervision moments’ followed by trainees, in a regular manner, together as peers and also with the supervisor. We can trust on this source, as it is literally mentioned the type of meeting followed. There is no alternative explanations for this evidence. • Strong confirmation proposition 3

195

●observatio n P3(ii)

Trace evidence. Invitation for meeting If the meeting was registered in an agenda after an invitation through e-mail. This can be seen as sequence evidence. The actual appointment in the agenda shows that an invitation must have been send beforehand. • Hu: This piece of evidence is clear proof that a meeting was put in agenda after receiving an invitation. This meeting reflects the dynamic of intervision to discuss the training and its implementation. We can trust on this source, since there is no reason to propose this event if there is no a clear interest in carry it out. • Strong confirmation of proposition 3

A proof of this invitation can be found in annex 12.

●Aggregatio n of evidence for proposition 3 4 a

Overall, it looks quite clear that P3 took place in our case, because the pieces were found, and they are accurate evidence of the presence of intervision initiatives. Sources are relatively independent. If only P3(i) or P3(ii) are found, we might infer that P3 is present. Overall confirmation: strongly warranted, p3(i) and p3(ii) are accurate evidence of how peers propose to handle 'intervision moments' as a 'peer coaching activity'. Building block: Intervision Theorized part: Peers agree to follow coaching activities - because they feel trust and they recognizes the need of a different way of working. Fingerprints: We expect to see evidence of agreements to discuss the implementation of the training in certain kind of coaching activities. This could be found in the form of trace evidence or sequence evidence. ●observatio Telephone conversation with the case. This is account 1:14 “Q so you had these intervision moments with n P4a(i) evidence. She acknowledges that she accepted the invitation your colleagues.. You accepted the invitation for those which shows that trainees agreed to meet each other. intervision moments …? What do you mean? That I accepted that invitation? • Hu: This piece of evidence reflects the fact that trainee agreed to participate in intervision moment. Q yes. Yes, yes, of course!.” • Strong confirmation proposition 4a ●Aggregatio Only one piece of evidence has been found. It is accurate evidence of agreement to participate in intervision coaching. n of Therefore, we can confirm the presence of proposition 4a. evidence for 196

4 b

proposition Overall confirmation: strongly warranted, there is a significant amount of evidence on the presence of peers’ 4a agreement to follow the intervision coaching. Building block: Intervision Theorized part: As part of the activity, peers meet each other to discuss the implementation of the training, specific cases and share experiences (e.g. issue, challenge or problem). Fingerprints: Expect to find evidence on scheduled meetings by colleagues with the aim to discuss the implementation of the training. This can be found in the form of trace evidence or account evidence. ●observatio n P4b(i)

Interview with case. This is account evidence. This shows that, outside of these intervision moments, there are also other moments in which they meet and discuss the training. • Mu: This piece of evidence is ‘good to have’ because show us the interest of discussing the implement of the training beyond intervision moments. In our piece of information, trainee mentions that ‘meetings’ are also an ‘everyday activity’ with supervisor to discuss personal issues or productionrelated topics that generates an open opportunity to discuss them in team. However, these meetings are not necessarily linked with the framework of intervision moments, since there must be other reasons to get involved in this meeting such as discussing production or an issue with employees as mentioned by the trainee. • Moderate confirmation of proposition 4b

41:33 “Q You’ve talked about intervision moments, are there other examples of where you help each other? Well, just in the everyday working. Every day we have a meeting in the morning with the supervisors. Then the problems of the day, if I can put it like that, are discussed. Often it is just about production, but it happens that there are issues with employees and that […] all openness is created, this is discussed in team.”

●Aggregatio A single piece of evidence is found, we can confirm the presence of proposition 4b. n of Sources are relatively independent. evidence for Overall confirmation: moderately warranted, since in general there is evidence that employees have met each other proposition later to discuss the implementation of the training in the framework of intervision moment. 4b Building block: Intervision Theorized part: Peers ask clarifying questions to understand the situation and issue at hand of other peers when facing issues, challenges or problems. 197

Fingerprints: We expect to see evidence on peers asking for clarification during the intervision moments. Given that these questions are probably mostly oral, we mainly expect account evidence here. ●observatio Account evidence. Interview with case. 17:40 “We also saw it more as exchanging experiences. n P5(i) During the interview, trainee discussed the contents of these Had this shown that extra coaching or support was intervision moments needed, than that would have happened. But, people indicated that “okay, in the training, we got enough • Mu: This piece of observation tells us something tools to work with that”. […] about the way in which peers perceive the intervision moments. It is mentioned that ‘exchanging experience´ is positive as well as the extra coaching or support. However, the piece of information tells us little about how the dynamic is when asking questions to understand issues, challenging or problem from other peers. One explanation for this is the trainee’s remark that it has been “a long time”´. So, there is probably some missing information in the memory, indicating that her/his recollection of the event is inaccurate. We can just partially trust on this source. • Strong confirmation of proposition 5 ●observatio n P5(ii)

Account evidence, Interview with case. During the interview, trainee discussed the contents of these intervision moments • Hu: This piece of information shows us the way in which peers deal with problems. When the problem is challenging, peers relies on colleagues and ask questions to get the view from others, in order to search alternatives ways to deal with. Trainee also mentions that the culture in the team is also ‘open’, so this is also an important context to make this happen. We can trust on this source and also in what this piece of information means. • Strong confirmation of proposition 5

39:00 “If a supervisor has a certain problem, and he himself doesn’t really know how to deal with it. Then he comes to me or talks to other supervising colleagues. “look, I would deal with it like that, do you think that’s a good way? Or would you do it differently?”. We have an open culture here. In general, but definitely among the supervisors.”

198

●Aggregatio n of evidence for proposition 5

In general, two pieces of evidence are found, we can confirm the presence of proposition 5. If only P5(i), is found we cannot confirm, whereas if P5(ii) is found, we can infer that P5 is present, because this piece as a whole confirms the presence of the expected dynamic. Sources are relatively independent. Overall confirmation: moderately warranted, since in general there is evidence that employees have carried out activities that reflects the dynamic of ´asking clarifying questions´ to understand a given situation when peers are facing issues, challenges or problems with the implementation of the training.

Building block: Intervision Theorized part: As result of it, peers start to brainstorm and bring up alternatives for action to support other peers. Fingerprints: We expect to see peers brainstorming about issues on application of the training and offer suggestions to the employee. Given that these suggestions are rarely written down, we only expect account evidence here. ●observatio Account evidence. Interview with case. 37:14 “Q Has there been talked a lot about the n P6(i) Trainee discusses the content of the conversations during training? these intervision moments. Well the fact that we had these intervision moments. Well then we are meant to talk about “well, okay, • Hu: The piece of evidence shows us some of the content discussed in this peer-coaching activity: the we’ve had this training” “what do you use out of it?” , “how come there are there certain things that you use topic seen in the training, the use of such content, less often?” or you think it’s more difficult to use?” the content that is more/less used, the difficulties. Interviewee mentions what they are meant to talk in intervision moment, which reflects that there are also some guidelines to make it more effective. We can trust on this source, since what it means is related to the expected proposition. Trainee also talks directly from his/her experience and there are no alternative explanations for the existence of this piece other than having ‘intervision’. • Strong confirmation of proposition 6 ●Aggregatio n of evidence for proposition 6

A single piece of evidence is found therefore, we can confirm the presence of proposition 6 because it as a whole confirms the presence of the expected content and dynamic discussed. Sources are relatively independent. Overall confirmation: strongly warranted, since in general there is evidence that employees have carried out activities that reflects the dynamic of ´brainstorming’ and ‘alternatives for action’ to support peers’ implementation of the training. 199

Building block: Intervision Theorized part: Peer(s) make(s) a synthesis and formulates recommendations to their peers. Fingerprints: We expect to see peers make a synthesis of the issues and give each other recommendations on how to apply the training correctly/ handle certain situations. This could take the form of account evidence in the form of quotes from interviews or trace evidence in the form of email/documents with actual recommendations. ●observatio Account evidence. Interview with case. 18:00 “It’s mostly finding a way to implement it in your n P7(i) Trainee discusses how people try to help each other during daily job. And why does it work for some people more these intervision moments. and for others less? And in that way you try to help • Hu: The piece of evidence tells us something about each other. That was sort of the point of those the dynamic of intervision: it is a way to implement intervision conversations.” the training in daily work, finding the reasons per case about what work and does not work for everybody (synthesis) and how they can help each other (recommendations). We can also trust on this source since, trainee explicitly mention that these activities are part of intervision moment. • Strong confirmation of proposition 7 ●Aggregatio A single piece of evidence is found therefore, we can confirm the presence of proposition 7 because it as a whole n of confirms the presence of the expected synthesis and recommendation activity linked to intervision moment. evidence for Sources are relatively independent. proposition Overall confirmation: warranted, since in general there is evidence that employees have carried out activities that 7 reflects the dynamic of ´synthesis’ and ‘recommendation to peers’ to support peers’ implementation of the training. Building block: Intervision Theorized part: Peers debrief : “what did they hear, what do they make of it, what do they take with them”. Fingerprints: We expect to find evidence for peer debriefing on: what they hear and lessons learned. Here we mostly expect account evidence. ●observatio Telephone conversation with the case. This is account 1:38 ”Q I have a question on those intervision n P8 (i) evidence. She mentions that, during these intervision moments. Did you hear or learn things about the moments, they talked about the training and its application. training and how you could apply it? Things that you She says that colleagues suggested certain ways of phrasing took with you? questions, so that they would reach the desired result. We yes, […] the way in which certain questions can be can trust this source. asked. So that you can get to the core of a problem.

200

•

That helps to have better results when you are having Hu: This piece of evidence reflects that the trainee a conversation with employees.” learned things from her colleagues during the intervision moments. • Strong confirmation proposition 8 A single piece of evidence is found therefore, we can confirm the presence of proposition 8 because it as a whole confirms the presence of the peer debrief activity linked to intervision moment. Overall confirmation: strongly warranted, since there is a specific example of behavior which can serve as evidence of peer debrief.

●Aggregatio n of evidence for proposition 8 Building block: Adaptability and application Theorized part: As a result of that, peers feel more stimulated to apply the content learned and are less resistant (adaptability). Fingerprints: We expect to find evidence of employees feeling more stimulated. We expect account evidence in which employee express feelings of motivation to apply the training. ●observatio Account evidence. Interview with case. 46:50 “Q that encouragement and interest of your n P9(i) During the interview, trainee mentions that it’s more “fun” if colleagues. Is that something that influences your colleagues are interested in the training and encouraging the feeling with the training? application of the training. This signals a positive feeling Yes, it’s more fun when you’re on the same page. If towards the training that is instigated by others’ interests. there is no resistance, then it’s easier to convey it to your employees. If you’re not on the same page, or • Hu: This piece of information is empirically unique. Trainee mentions a key relevant aspect here: “to be you have a different vision. Then it’s more difficult. We don’t have that very often. And if we have different on the same page”, and “If there is no resistance”, opinions. Then we discuss that beforehand and we which involves some kind of motivation or stimulus make sure we are on the same line before we talk to to apply the training content to the job. We can the employee. trust on this source, because there is no alternative empirical evidence that can explain this observation other than ‘gain stimulus to apply the content to the job and get less resistant’. • Moderate confirmation of intermediate outcome ●Aggregatio n of evidence for

A single piece of evidence is found therefore, we can confirm the presence of proposition 9 because it as a whole confirms the presence of stimulus to apply the training content to the job and get less resistant to do it. Sources are relatively independent.

201

1 0

1 1

proposition Overall confirmation: warranted, since in general there is evidence that employees have carried out activities that 9 reflects the dynamic of ´feeling motivated’ and ‘being less resistant’ to use training to the job. Building block: Adaptability and application Theorized part: Peers apply the content (when it is ad hoc to the problem/challenge identified at work), after a reflection of what they heard during the intervision moment. Fingerprints: We expect to see documents or minutes that reveals some sort of meeting preparation related to the training application. Because a part of the course was on leadership and communication, we expect there to be some preparation for difficult conversations. This evidence could take the form of trace evidence but also of account evidence. ●observatio Telephone conversation with the case. This is account 3:10 Q “And if you have learned things there. Did you n P10(i) evidence. Continuing on what she said earlier (that her use those things? Can you give an example of a colleagues suggested certain ways of phrasing questions), situation in which you use the feedback that you got? she notes that she actually uses this advice and phrases Well, for example when you are having a conversation questions in line with how they have been suggested by with an employee. A “corrective” conversation, in colleagues. We can trust this source. regards to how this person does his job. […] by asking the right questions, you try to make the employee • Hu: This piece of evidence reflects that the trainee reflect themselves on the issue.” used things that she learned during the intervision moments when applying the training. • Strong confirmation proposition 10 ●Aggregatio A single account evidence is found. We can therefore confirm the presence of proposition 10 because it as a whole n of confirms the presence of the application of training content to the job by trainees. evidence for Sources are relatively independent. proposition Overall confirmation: strongly warranted, since in general there is evidence that employees have carried out activities 10 that reflects the dynamic of ´application of the training content to the job’ after peer coaching activity experience. Building block: Intervision (after adaptability) [feedback loop parts 3-10]. Theorized part: Peers discuss the application and get feedback from other peers in subsequent intervision moments (follow-up posttraining application). Fingerprints: We expect to find evidence the employee asking and discussing feedback in the context of training application. Evidence can take the form of account evidence. ●observatio Account evidence. Interview with case. 38:35 Q “on those intervision moments, experiences n P11(i) During the interview, trainee discussed how colleagues ask were shared [humms agreeingly], where there for feedback and that they discuss how they apply the sometimes explicit questions like “how would you deal training. with that?”

202

•

Mu: The piece of evidence tells us something about how trainee shared experiences in intervision moments and also outside of such peer coaching dynamic. However, it tells us little about the kind of dynamic carried out here. There is not enough information related to ‘asking for feedback’ and the intention of discussing the use of training. So, we cannot evaluate this evidence as highly unique. We can, however, trust on this source, since it comes from the trainee who was experiencing this process of enhancing training transfer triggered by peer support. Moderate confirmation of proposition 11

Yes, yes, it also happens outside of these intervision moments as well.”

●Aggregatio n of evidence for proposition 11 1 2

A single account evidence is found. We can therefore confirm the presence of proposition 11 because it as a whole tells us something about the presence of intervision after adaptability. Sources are relatively independent. Overall confirmation: moderately warranted, since in general there is evidence of follow-up post-training application, but the observation is not telling us more about the dynamic conducted with feedback and discussions. The piece therefore is moderately accurate. Building block: New working thinking Theorized part: Because of that, peers incorporate the new way of working thinking on their own after an adaptation phase and it becomes routine. Fingerprints: We expect to see evidence of new routines applied to the job by trainees, after that training content has been absorbed. Evidence for this could be account evidence. ●observatio Account evidence. Interview with case. 42:20 “Q Do you encourage each other to apply the n P12(i) Trainee mentions that at the start, they would have to training? actively think about the training, but now, it has become a Yes, definitely at the start, now it’s not as necessary routine. because people have already absorbed this in their • Hu: This piece of evidence is quite unique. It tells us way of working. But at the start.. again, sometimes you are in your daily routine, the drag, and then you really how trainees absorbed the training, after have to think consciously about it.” adaptation, in their own way to work and how different it is when compared with the start of training. The new way of thinking is part of a work 203

• ●observatio n P12(ii)

routine after training. We can trust on this source and in what it means. Strong confirmation of proposition 12

Account evidence. Interview with case. Trainee mentions that at the start, they would have to actively think about the training, but now, it has become a routine. • Hu: This piece of evidence clearly reflects that the incorporation of a new way of thinking was taking place, with adaptability and effort in the implementation of training content. It is recognized that the process was challenging and that the frequency of the use of training makes a difference in how it becomes a routine afterwards. We can trust on this source and in what it means, since trainees tells us this information from his/her own experience with training. • Strong confirmation of proposition 12

25:50 “Q So you recognize situation in which you can apply the training. That was easy? To recognize these situations.? At first, it’s not really easy. Once you have done it more often. You start to incorporate that way of thinking. But at the start, it’s not really straightforward. At the start, you need some time to adapt. To make that way of working, thinking your own. But after a while, the more you apply it. The more used to it you get. After a while it becomes routine. Or a way of working, let me put it like that.

●Aggregatio n of evidence for proposition 12

O u t c o

Both pieces of evidence are found. We can therefore confirm the presence of proposition 12 because they as a whole tell us something about the presence of a new routine after training content application and new way of thinking absorbed. If only P12(i) or P12(ii) is found, we can infer proposition 12 is taking place. These pieces of information are substitutable. Sources are relatively independent. Overall confirmation: strongly warranted, since in general there is evidence of a new working thinking in the trainee routines. Theorized outcome: Effective employee training transfer is the application or use of the learned knowledge (content, skills or attitudes) acquired in a training program to the job by trainees, which is maintained over a period of time. Observable manifestations: The employee applies the learned content and skills and they are maintained on the job. Evidence for this can take the form of account evidence. — theoretical Hc, Hu ●Relatively high theoretical certainty. We have to find evidence of the presence of training transfer. 204

m e

●Relatively high theoretical uniqueness. It is unlike to find these fingerprints if training transfer is not present. ●observatio Account evidence. Interview with case. Q and how do you see that it [training] improves? That n O1(i) During the interview, trainee discusses the application of the you apply it more? training on several instances Well, you experience it. Q you experience it? • Hu: This observation reflects how the training The quality of your conversations improves and you transfer look like for trainees: It is an experience notice that the preparation of your conversations takes (people were involved, they are not the same); a bit less time than at the start? You are more used to conversation quality becomes better (they incorporate the learnt knowledge to communicate), the techniques” preparation of conversations takes less time (efficiency in preparation). We can trust on this source because of it means and because trainee share this observation from her own experience with training. • Strong confirmation of outcome ●observatio n O1(ii)

Account evidence. Interview with case. During the interview, trainee mentions the application of the training on several instances • Mu: This piece of evidence tells us something about the positive consequences of applying the training as a whole (more oriented to the attitude) and the role of coaching and feedback to keep on doing it over time. However, it tells us little about what has been applied in terms of content or skills. We can trust on this source since the trainee share her/his experience from his/her own experiences. • Moderate confirmation of outcome

29:50 Q So that you see the results and that those results.. Yes, encourage you to keep on doing it like that. Because you notice that you get results. Like I said, more support and less resistance. By communicating in a coaching way, as a supervisor, you get feedback that you didn’t consider beforehand.

observation O1(iii)

The survey that was filled in by the respondent She had a positive response on questions on the application of the training • Hu: We expect that she would be honest when filling in the survey. There were some questions on

In the survey, the trainee answered several questions on the training and the specific skills that were trained. For the generalization and maintenance based on Govaerts (2017) and Hiva (2011) she “agreed” (a score of 4) on all statements. For the two leadership scales, 205

●Aggregatio n of evidence for proposition O1

the application of the training and questions about she scored quite high on all statements (on average, leadership skills in particular. she “agreed” with most statements). • Strong confirmation of outcome In general, three pieces of evidence has been found. We can therefore confirm the presence of outcome ‘training transfer effectiveness’. If O1(i) or O1(ii) are found, we cannot confirm, whereas if O1(iii) is found, we might infer that outcome is present in the case. Observations O1(i) and O1(ii) reinforces the observation O1(iv). Sources are relatively independent. Overall confirmation: strongly warranted, since in general there is evidence that training transfer effectiveness occurred in this case, because of the presence of the application of the learned knowledge acquired in the training and its maintenance over a period of time.

A11.3 ROADMAP SELF-MANAGEMENT INTERVENTION Case 1: B3 Causal relationship Causal mechanism linking employee goal setting and relapse prevention to the trainee’s use of learned material to the job with the effectiveness of training transfer. Prior relatively low ●There is no existing research that documents the mechanism that connects employee goal setting and relapse prevention to training transfer. Theorized cause: Formulation of training goals by employee AND Commitment to overcome the obstacles when addressing difficulties in applying new knowledge at work. C Observable manifestations: we expect to find evidence of trainees setting specific training goals and activities related to how they a deal with the challenges in achievement such goals, when addressing difficulties in transfer. We expect to find evidence of this in the u survey (account evidence) which we asked trainees to complete after having attended the training. s — Htu22 e ●Theoretical certainty not formulated (no priors). ●High theoretical uniqueness as observing these observable manifestations necessarily means the presence of goal setting and relapse prevention based on theory (Rahyuda, 2014; Ghosh, 2015; Nijman, 2004, 2010; Gist, 2006; Hutchins, 2006). 22

Note: Htu= high theoretical uniqueness; Hu = High uniqueness; Mu = Moderate uniqueness; Lu = Low uniqueness.

206

●observatio n C1(i)

Account evidence: The survey that was filled in by the respondent after the training took place (T1). Trainee confirms goal setting before training participation. • Hu: This observation reflects that trainee had a goal before he participated in the training. There is also a clear commitment to overcome potential obstacle to apply the training content at work. There are no empirical alternative reasons for why he would claim he had a goal, when in fact he didn’t. One of these could be that the trainee gave an answer because he thought this specific answer was socially desirable over not having a goal. However, we cannot confirm this possible explanation. Therefore, we can trust on this trust and in what it means. • Strong confirmation of condition

When asked “Did you, personally, set a couple of goals before you participated in the training?”, the trainee replied “yes” and clarified that “my absolute goal was to use the tips and theory that have been taught in the training and to apply them in practice.”

●observatio n C1(ii)

Account evidence: The survey that was filled in by the respondent after the training took place (T1). Trainee confirms that during the training, there was attention for relapse prevention • Hu: This piece of evidence shows us how trainee directly claims that, during the training, they discussed methods to deal with potential obstacles. The evidence is “yes” to questions that are clearly related to challenges. We can trust on this source and in what it means, but we cannot consider it selfexplanatory. It needs to be combined with other observation. • Strong confirmation of condition

The trainee answered yes on two questions

●Aggregatio n of evidence for

“Did you, when following the training, identify potential difficulties that would make it difficult to apply the knowledge and skills at the workplace?” and “Did you, during the training, discuss certain methods or tools to deal with or prevent potential obstacles when applying the training?”

In general, the presence of the cause seems supported by evidence, because the evidence is found. We can, therefore, confirm the presence of the condition ‘goal setting and relapse prevention’. Sources are relatively independent.

207

proposition C1 1

Overall confirmation: strongly warranted, given C1(i) and C1(ii) are accurate evidence of the presence of the mentioned conjunctural condition.

Building block: Provide direction for attention Theorized part: Based on the new training skills to be acquired, trainee identifies some kind of goals [distal and proximal] (either specific, challenging or difficult) to help her/himself with expressing attention. Fingerprints: We expect to find evidence in the empirical record of goal-oriented reasons that stimulated training application. There might be evidence in the form of trace evidence or account evidence. — Htu ●Theoretical certainty not formulated (no priors). ●High theoretical uniqueness. Highly unlikely to find this fingerprint, as it looks like, without this part of mechanism being operative. ●observatio n P1(i)

●observatio n P1(ii)

Account evidence: The survey that was filled in by the respondent after the training (T1). Goal oriented reasons that stimulated application of the training. • Hu: This piece of evidence shows us the reason that has stimulated the training content application at work. Trainee wants to get better at communicating and this is the goal in the beginning of the process. We do not know anything about whether or not this goal has enabled him to focus on the implementation of the training more, but here the most important is whether a goal was set up at the start of self-management process. We can therefore trust on this source and in what it tells us. • Strong confirmation of proposition 1 Account evidence: Interview with trainee. In this quote, trainee mentions that he really “needed” the training. • Mu: This piece of evidence reflects a motivation related to the training application: ‘I needed it’ which helped the functioning of the process of selfmanagement. However, this one tells us little about the goal behind such a need. Therefore, what this

When he was asked in an open question what has stimulated him to apply the training contents, he replied “my conviction that my communication can be better so that my message gets across clearer and more respectfully to my staff members/colleagues.”

7:25 Me as a person, I think I needed it. Like I said. [My communication] it was too direct.

208

• ●Aggregatio n of evidence for proposition 1 2

piece means is not confirmatory of the presence of goal-setting. Moderate confirmation of proposition 1

In general, the presence of the proposition 1 seems supported by evidence, however both pieces found are different in accuracy. We can, therefore, confirm the presence of the proposition 1, if only P1(i) is found. However, P1(ii) acted as supportive evidence for P1(i) which is not relevant for this purpose. Sources are relatively independent. Overall confirmation: moderately warranted, given P1(i) is accurate evidence of the presence of goal setting.

Building block: Provide direction for action. Theorized part: With this knowledge in mind, the trainee organizes her/his effort either focused on learning or performance goals that he/she wishes to apply and maintain from this training - increasing perception of determination. Fingerprints: We expect to find evidence on aspects that the trainee absorbs during the training itself, that enable him/her to get focused in his/her efforts to apply and maintain it from the training. This evidence could take the form of account evidence. — Htu ●Theoretical certainty not formulated (no priors). ●High theoretical uniqueness as observing these fingerprints necessarily means the presence of this part of the causal mechanism. ●observatio Account evidence: Interview with trainee. 39:35 Q with what mindset did you start the training? n P2(i) In this quote, trainee mentions what he does to learn from Open. Because I knew, here you give direction to a the training. He tries to pick up as much as possible during team of warehouseworkers [inaudible] I come from a the training (“stealing with your eyes and ears”) and then group where I had to lead 50 to 70 people. With all apply what you believe in. different nationalities. In different languages. There you don’t have time to talk 5 minutes to everybody • Hu: This piece of evidence reflects what trainee does to learn from the training. Trainee tries to pick “how are you really doing?” If you’re not educated like up as much as possible during the training (“stealing that, as a company, then you don’t have it. Maybe on the inside, but then it has to be developed. So with an with your eyes and ears”) and then apply what it is believed in. This part does not explicitly say anything open view. Stealing with your eyes and ears and afterwards make it your own and apply what you about how the trainee organizes his effort, but the believe in. intention to focus the effort on learning is there when trainee says that a company with different education have to develop what is it missing. We can trust on this source because it is the trainee who speaks from his experience. • Moderate confirmation of proposition 2 209

●observatio n P2(ii)

Account evidence: Interview with trainee. In this quote, trainee mentions some things that are important to absorb the training during the training itself, that enable him to get focused in his efforts. • Mu: This piece of evidence tells us something about trainee effort on learning. Firstly, he mentions the need to be separate from the work environment, as a way to be focused on the training with a certain group of peers. This aspect enable to the training don’t get disturbed or stressed. Secondly, trainee mentions the importance to be connected with people, being yourself, and the training was taught in this way. This enable to trainee to focus his efforts in a better leaning, under a trust environment. This evidence tells us something about how the trainee organizes his effort in learning, but it is not as much explicit as to evaluate this piece as high uniqueness. We, can however, fully trust on this source and in what it means moderately. • Moderate confirmation of proposition 2

43:17 Q What according to you was important to absorb the training? For me personally, I need to be able to be separate from the work environment. So that’s very important I think. That everyone has a back-up that does their job at that time. And that with the ten, twelve people that are there, you can focus on the training. That you don’t get disturbed or that you are stressing and thinking that you should have been at your work for a certain reason. The fact that you can just get away from your work for two days, if I think back to the last part of the training. That’s the first thing. The second thing. In a certain way, I think it’s important that you have a connection with the people at the training, that you can be yourself. And that certainly was the case for everybody. And then I think, how it is taught. Not too artificial. Not with a powerpoint and note after note after note. I’m not a fan of that because I’m convinced that people don’t remember a lot from all of those. I think conversations and exercises and just communicating with the teacher, I think that from that, you learn the most. And then trust.

●Aggregatio n of evidence for proposition 2

In general, the presence of the proposition 2 is supported by evidence. Both pieces have been found, although they are different in what they tell us. We can, therefore, confirm the presence of the proposition 2. If only P2(i) is found, we can confirm, whereas if only P2(ii) is found we cannot. P2(ii) acts as supportive evidence for P2(i). Sources are relatively independent. Overall confirmation: moderately warranted, given P2(i) is accurate evidence of the presence of trainee effort on learning or performance goals, and P2(ii) tells us something about the presence of P2.

Building block: Mobilizing efforts. Theorized part: The trainee feels motivated due to this perceived determination and develops the best ways to achieve and maintain such goals: setting the skills maintenance goal, based upon the training, and identifying potential risks of slips.

210

Fingerprints: We expect to find evidence of trainee feeling motivated to reach a goal and actions carried out to implement the training. We expect that this evidence will mainly be account evidence. — Htu. ●Theoretical certainty not formulated (no priors). ●High uniqueness. Finding evidence on this would be a serious indication that the causal mechanism is present. ●observatio n P3(i)

Account evidence: Interview with case Motivation to reach goal and develops ways to achieve that goal. • Hu: This piece of evidence illustrates that the interviewee is clearly motivated to reach his goal, which is communicating better and also develops ways to achieve that goal (‘keep on asking questions’; ‘keep on communicating connectively’). It is also clear from this excerpt that he has set a goal to reach. • Moderate confirmation of proposition 3

“You have to be spontaneous when you talk. And maybe you don’t get the answer you want. Or not your preferred goal. That you keep your goal in mind and that by keeping on asking questions, by keeping on communicating connectively, that you do reach your goal.”

●observatio n P3(ii)

Account evidence: The survey that was filled in by the respondent before the training (T0). • Lu: The observation tells us something about the motivation to reach goal and develops ways to achieve that goal. We can observe that all answers of these statements combined, indicate that the trainee is motivated to learn and apply what has been acquired in the training. Further, the trainee is confident that he can develop the best ways to achieve implementation of the training. These survey questions do not specifically address specific ways to achieve goals. • Weak confirmation of proposition 3

The trainee agrees or completely agrees with the following statements “When I participate in trainings, I want to improve the practical/technical knowledge that I can use in my job”; “I think it is important to learn new things from the training activitities”; “usually, I can apply the things that I learn in a training in my job” and “If I’m involved in training activities, I’m convinced to learn the new knowledge that has been taught during the training activities”.

●observatio n P3(iii)

Account evidence: Interview with trainee Actions to implement the training. • Lu: This piece of evidence tells us something about the actions carried out by the trainee to implement

21:05 Q and if you talk about it. What is being said about it? Of course, it’s too long ago to say exactly. But then, from both groups, supervisors and employees, certain 211

• ●Aggregatio n of evidence for proposition 3

it. Trainee mentions that they have carried out certain actions to make sure that they could deal with certain aspects better (and would implement the training). However, what is mentioned is not specific for the trainee, but he talks about the organization in general. Also, the specific actions are not mentioned and are only seen as an indirect consequence of the training. Weak confirmation of proposition 3

ideas emerged on how to deal with certain things better. And those were incorporated into action plans later. And so there are certain initiatives, some bigger than others, that are in effect that are indirect consequences of the training back then. Because they are about communication.

In general, the presence of the proposition 3 is supported by moderate evidence. Three pieces have been found, although they are different in what they tell us. We can, therefore, confirm the presence of the proposition 3. If only P3(i) is found, we can confirm, whereas if only P3(ii) or P3(iii) is found we cannot. P3 (ii) and P3(iii) acted as supportive evidence for P3(i). Sources are relatively independent. Overall confirmation: moderately warranted, given P3(i) is accurate evidence of the presence of motivation and development of actions to achieve and maintain certain goals.

Building block: Pros & cons generalization. Theorized part: By identifying potential threats to transfer, the trainee defines the advantages and disadvantages of using the skills at work in order to stay motivated. Fingerprints: We expect to find evidence of the trainee contrasting approaches about using/not using the skills at work, identifying potential threats to the training application and some ways to resolve it, and formulating advantages of its applicability. We expect to mostly find account evidence here. — Htu. ● Theoretical certainty not formulated (no priors). ● Relatively high theoretical uniqueness: Finding this evidence would necessary mean this part of the mechanisms is present. ●observatio Account evidence: Interview with trainee. 5:50 Q beforehand, did you think the training would be n P4(i) Contrasting approaches useful? And why? Well, I know that earlier, well not that much earlier, I • Lu: This piece of evidence tells us how the trainee had quite a “direct” way of leadership. So from that sometimes wonder how things would have been if perspective, to be able to think more about it. And he would have approached situations differently. This implies that he contrasts these two approaches. two, this was through walk your talk [the company that organize the training], you look things up or you see However, the piece of evidence tells us little or indirectly on the (dis) advantages of using the skills. things on the television and [you think] “they approach 212

•

Weak confirmation of proposition 4

it in a completely different way”. And then the “Whatif”-question becomes important. What if I would also do it like that? Or try. So beforehand, I knew it would interest me. I also know before these types of trainings that I always try to take things away from the training [ to learn things] there are always things that you remember.

●observatio n P4(ii)

Account evidence: Interview with trainee. Identifying potential threats as being “too artificial” • Mu: This observation tells us something about how trainee identifies a potential threat to transferring the training to the work floor. He argues that the application cannot be too “artificial”. This illustrates that he identified potential threats as well the challenge to apply the training in a more spontaneous and natural way, taking into account that it must be different from before. The observation, however, does not tells us much about the advantages or disadvantages of using skills at work. • Moderate confirmation of proposition 4

10:15 Just, we got some “steps”. If you follow those steps literally, it’s too artificial. So that is something that I needed to… And that is something that is actually still, because you can’t change immediately. That is where the biggest challenge lays, to do it in a way where it looks very spontaneous and natural, but where it is different from before.

●observatio n P4(iii)

Account evidence: Interview with trainee. In this fragment, trainee identifies the need “to get some spontaneity” as opposed to artificial to transferring the training to the work floor. He also mentions that it takes a while before you really learn how to apply the training contents. • Hu: This observation tells us something about how trainee makes contrasts between an artificial application of the training versus the spontaneity that they need to get. Trainee also valued the training as ‘they bring something… a theoretical explanation’, that can be considered as a potential

6:30 In the sense that they bring something.. a theoretical explanation. You apply it and work around it. But it’s not like (snaps fingers) you know tomorrow how it works [It takes a while to learn to apply things] … you need to get some spontaneity. But, from the start, I was convinced that it would interest me.

213

• ●observatio n P4(iv)

●observatio n P4(v)

advantages. We can trust on this source and in what it means, since the interviewed was asked many times about the presence of this fingerprints, with more examples at hand and same version of the facts. Moderate confirmation of proposition 4

Account evidence: Interview with trainee. In this fragment, training identifies some potential threats and some advantages. • Mu: This piece of evidence tells us something about how trainee identifies “being too artificial” as potential threat to transferring the training to the work floor. Trainee also mentions that if other employees are more aware of the training content, that itself will already lead to an advantage. We can trust on this source and in what it means, because the trainee speaks from his own experience. • Moderate confirmation of proposition 4 Account evidence: Interview with trainee. In this fragment, training identifies advantages of following and applying the training together. • Mu: This piece of evidence tells us something about how trainee identifies “advantages of following training in group” to become better in it use, keep motivated and for reinforcing the group identity and cohesiveness. However, this piece of evidence tells us little about potential threats to transfer. We can trust on this source, because trainee speaks from his own experience in the training, and there is no reason to mention this if it did not occurred. • Moderate confirmation of proposition 4

15:00 It can’t be too artificial and people are who they are. Someone who has been doing things a certain way for 40 years, three days of training won’t easily change that. Some of them. Others not. But if they are more aware of it. If we can address it later, that they are able to show their feelings and can honestly say “sorry, I overreacted”. Then you’ve had a gain.

“Then you can really talk about things in depth.. also emotionally. If certain situations are talked about or certain techniques are used that are going much deeper into people.. that’s sometimes emotional. Then you become closer as a group. And that, for me, is the second important part, asides from the contents, that as group, you become stronger. I think you see some colleagues day in and day out, and you say hi to them sincerely , but apart from that, you don’t really have a connection with them. The training results in that you become closer to those that you don’t work with too often. The colleagues definitely.”

214

5 a

●observatio n P4(vi)

Account evidence: Interview with trainee. In this fragment, training identifies advantages of using training content at work. • Hu: This piece of evidence shows us how trainee identifies “advantages” of the training content acquired. There is a situation before/after training between colleagues with unbalanced knowledge. It is explicitly mentioned how training improved the communication between peers, in a better understanding. We can trust on this source and in what it tells us. • Strong confirmation of proposition 4

“If you, for example, are forced to discuss a situation that you have to discuss with your direct colleague that in the past you had difficulties with. At that moment, you gain from training the technique. But you also talk about a subject that in the past you had some difficulties with. But it has also really brought us closer together.”

●Aggregatio n of evidence for proposition 4

In general, six observations have been found out. The presence of the proposition 4 is supported by moderate evidence. Two of them are strong evidence, whereas one is weakly confirmed. We can, therefore, confirm the presence of the proposition 4. If only one observation is found, we cannot confirm, whereas if P4(iii) and P4(vi) are found in combination with either P4(ii), or with P4(iv), or P4(v) we can confirm. Sources are relatively independent. Overall confirmation: moderately warranted, given P4(iii) combined with P4(vi) are accurate evidence of the identification of potential threats to transfer, as well as dis(advantages) of using the skills at work, but the supportive complementary evidence is modest.

Building block: Coping with slips. Theorized part: The trainee discusses and learn certain methods or tools to avoid or overcome some kind of obstacles in applying the training content at work [anticipation]. Fingerprints: We expect to find evidence on the trainee dealing with challenging situations in the application of the training to the job, as well as communication exchange about ways to overcoming it. We mainly expect to find account evidence for this. — Htu. ●Theoretical certainty not formulated (no priors). ●Relatively high theoretical uniqueness: if we find evidence for this part, this does necessarily indicate that this part of the mechanism is present.

215

●observatio n P5a(i)

Account evidence: Interview with trainee. Trainee identifying obstacles to application of training • Lu: This piece of evidence, shows us the trainee’s reflections about how obstacles were discussed in the training. Two of them are discussed. The first is that there is less time to apply the training contents than during the training. This gives the trainee less time to reflect on the appropriate course of action. The second factor that is mentioned is the behavior of others, which also differs from how it has been portrayed in the training. This piece of evidence shows that these obstacles were considered, but does not tell us explicitly how the obstacles are overcome or dealt with. We can trust on this source, although the evidence is low in uniqueness because of it means. • Weak confirmation of proposition 5a

37:18 Q Is this something that, in the training itself, took into account? Yes of course, it’s said sometimes. Here it goes easy and in 5 minutes of talking, you are finished. But when you’re talking to a warehouse worker, or someone else, than it can turn out differently. Of course you are aware of that. 37:42 Q In what way was the behavior of others, like you said, other employees, also discussed in the training? Examples that were discussed. Cases, and this will be the same in other groups. There were groups with only warehouse workers. There you will also have discussed situations in which they think a supervisor not reacted/communicated 100% correctly. That’s sensible. It goes in both directions.

●observatio n P5a(ii)

Account evidence: Interview with trainee. Discussion of potential obstacles to implement the training. • Hu: This piece of evidence shows us that certain obstacles were considered in the training and how trainees were asked to reflect about their eventual reactions to a given challenging situation. Trainee mentioned how they, during the training, considered other possible cases. In that way, potential obstacles to implement the training were identified and discussed beforehand. • Strong confirmation of proposition 5a

13:02 Q You give concrete examples, was it easy to recognize situations in which you could apply the training? Most of the time, we suggested the cases. She asked us. You have ten minutes with a group of 2 or 3. “Try to think about situations, what happened? How did you deal with it?” afterwards, some theory or , during the training, “how would you deal with this?”. There were situations that had happened or could have happened. It was very accessible. 14:05 Q so in the training there were also specific examples from on the workfloor? Yes. Of course anonymously.

●observatio n P5a(iii)

Account evidence: Interview with trainee. Dealing with issues when implementing the training. .

34:12 Q How do you deal with that? Trial and error. Sometimes having a conversation and thinking afterwards “that was not how it should be”. 216

•

Hu: This piece of evidence tells us something about how the trainee deals with issues when implementing the training. He mentions that a large part is just trial and error. In addition, in two occasions to apply the training, the trainee was able to reflect on how he applied the training and in what way he could improve his application. This piece of evidence shows obstacles that were considered and dealt with in a certain way. We can trust on this source and in what it means, because trainee tells us his own experience with training challenges. Moderate confirmation of proposition 5a

And for some conversation, my manager was there. That was agreed upon. And then on the one hand you had the evaluation of the person that was just there. “okay, how did you think about that conversation” and then two seconds later “good, how did you contentwise feel about how the conversation went?” Did you apply the techniques? And so on. But the difficulty was mostly in the spontaneity. I think many people will say this. Thinking about it is one thing. Saying it is another thing.

●Aggregatio n of evidence for proposition 5a

5 b

In general, three observations has been found out. The presence of the proposition 5a is supported by strong evidence. Two of them are strong evidence [P5a(ii) and P5a(iii], whereas one is weakly confirmed [P5a(i)]. We can, therefore, confirm the presence of the proposition 5a. If only P5a(ii) or P5a(iii) is found, we might infer the presence of P5a, whereas if P5a(i) is found, we cannot confirm. Sources are relatively independent. Overall confirmation: strongly warranted, given P5a(ii) or P5a(iii) are accurate evidence of the presence of discussions or ways to deal with the challenges/obstacles in applying the training content to the job. Building block: Networking. Theorized part: The trainee understands the difference between training and job context, so she/he creates a support network for transferability. Fingerprints: Expect to find evidence on active supportive networks created by trainees in order to get transferability. We expect to find account evidence that could confirm this proposition. — Htu. ●Theoretical certainty not formulated (no priors). ●Relatively moderated uniqueness. Finding evidence for the existence of a support network in itself could have many alternative explanations, other than the existence of this (part of) mechanism. ●observatio Account evidence: Interview with trainee. 35:58 Q what are the differences between what you n P5b(i) Trainee talks about the differences between the training and have done on the training and how it is on the reality. workfloor? You already indicated that there are a lot of similarities because there are cases you suggested • Hu: This piece of evidence tells us something about the difference between the training and job context. yourselves. But what are the differences? 217

Trainee is contrasting both scenarios, where in real life a quick reaction to handle problems is desirable, versus the time to reflect they have when they are playing a game. Trainee knows in the new scenario what is important, where to focus, pay attention and react in a shorter time. We can trust on this source and in what it means because it is the trainee’s own experience about dealing with reality to get transferability. Strong confirmation of proposition 5b

Difference is very simple. If there is now a situation that is unacceptable, than you don’t have 10 minutes to think about it. You have to handle now. While in a training if there is a situation, you have, as manner of speaking, 30 minutes with three people to discuss it. You can discuss the situation and context.. and you can think about it in the meantime. And you can think “how will I deal with it later during the role—playing game? If it happens now, you have to act now and you have to pay attention now to how you communicate it. So your, reaction time is way shorter. The situation is more real, you can’t prepare. “what will he say?” , “How will he react?”. That’s the difficulty.

Account evidence: Interview with trainee The existence of a support network (partially) created by the trainee. • Hu: This piece of evidence shows us how trainee relies on his direct colleagues to reflect on the needs of the warehouse workers. This illustrates the existence of a support network that was (partially) created by the trainee. We can trust on this source and in what it means, since trainee talks from his own experience. • Strong confirmation of proposition 5b Account evidence: Interview with trainee The existence of a support network (partially) created by the trainee. • Mu: This piece of evidence tells us something about how trainee coping with the difficulties in the training transferability. It is mentioned the importance of sharing experiences, provide feedback, arranging meetings to be on the same page and act collectively. However, it seems that the support network is implicit and it is not clear

So on the one hand, I try to, with my direct colleagues, try to think about the need of our warehouse workers. (…)Why does it go wrong everytime. So purely the thinking about it was something. And two, earlier, we would have said it in a certain way. Now, we also think about the way in which we would say it.

•

●observatio n P5b(ii)

●observatio n P5b(iii)

32:28 Q and afterwards, have there been meetings or talks? Official or unofficial? [well, for official] Absolutely yes. 32: 44 Q What was mainly discussed during these conversations? In fact, mostly about the experience that you had with very difficult situations, because that’s where they mostly surface. Just casual about how they dealt with that and sharing those experiences. There was then given some feedback from “look, very good”. I would 218

•

●observatio n P5b(iv)

who has created this support group. Since the evidence is based on the own experience of the trainee, we can trust on this source and in what it means. Moderate confirmation of proposition 5b

Account evidence: Interview with trainee Following the training together creates a natural network with peers. • Mu: This piece of evidence tells us something about how following the training together can implicitly create a support network among peers. This scenario makes easier to apply the training, because there is a shared understanding of what application should look like and more chances to realize about the mistakes and solutions. However, since it is an implicit information, that does not directly say anything on the creation of a support network, we can consider the uniqueness of this observation as moderate. We can trust, however, on this source, because the interviewee talks about his own experience. • Moderate confirmation of proposition 5b

have done it like that or I did it like that. And even now sometimes. We are with 4 team coordinators, spread over some divisions. We used to see each other when passing each other, but now we also arrange meetings with those 4 to stick our heads together [to talk about things] and to make sure that we are on the same line and that the four of us act in the same way [hetzelfde uitdragen]. So that it’s clear. That it’s the same for people. That we all know the same. That we can report the same things to our people. Q 14:12 Did it take a lot of effort to use the training on the work floor? (thinks) in a certain way, not really, and I think the biggest reason for that is that everyone got the training. Imagine if you would only take the training with the supervisors, then there is a basis, but if you try to implement something for the warehouse workers, for them it’s not as relevant to them so because they followed the training, they also know the purpose. Which makes it accessible/easier to use for everyone. Did it take a lot of effort? No. And it also takes a lot more [probably means less] effort to now say, when somebody does something wrong, “that was not connecting”. And that person immediately knows “ah, it was about then, I need to wait for a bit, I need to stop for a bit” and it still happens that it goes out. But I’ve noticed that those who have an impulsive character, not aggressive, that they come a half an hour later “sorry about earlier, but you have to understand…”. And then they start to talk about it. And I think that that is a gain.

219

●Aggregatio n of evidence for proposition 5b

In general, four observations has been found out. The presence of the proposition 5b is supported by moderate and strong evidence. Two of them are strong evidence [P5b(i) and P5b(ii)], whereas the rest of observation are moderately confirmed. We can, therefore, confirm the presence of the proposition 5b. If only P5b(i) or P5b(ii) is found, we cannot confirm, whereas if both are found we might infer that P5b is present. P5b (iii) and P5b(iv) are supportive evidence for the observations (i) and (ii). Sources are relatively independent. Overall confirmation: moderately warranted, given P5b(i) and P5b(ii) are accurate evidence of the presence of supportive networks for transferability and the reasoning about the difference between training and job context. Building block: Slip prediction. Theorized part: The trainee predicts some kind of slips in transfer by monitoring past experiences of slip and relapse [anticipation] and the present environmental situations. Fingerprints: We expect to find evidence based on the trainee’s reflections from past slips experiences and relapse as a way to anticipate such situations. This evidence can take the form of account evidence of the employee identifying some obstacles that arise when attempting to apply the training. — Htu. ●Theoretical certainty not formulated (no priors). ●Relatively high theoretical uniqueness. We expect that if we find support for this proposition, this is quite unique for the mechanism. ●observatio Account evidence: interview with trainee. 33:48 Q What were the difficulties in applying the n P6(i) If the implementation is not “smooth”, it will make it more training on the workfloor? difficult. What I said earlier. Make sure it’s not too artificial. At • Mu: This piece of evidence tells us something about the start, it’s just the way how you say things. Don’t look to artificial, like you are straight from the book. how the trainee considers “applying the training to artificially” something that made it more difficult to You have to be spontaneous. use the training on the job. According to him, if the implementation is not “smooth”, it will make it more difficult. For this reason, the implementation need to be smooth. Trainee also recognizes a potential slip in transfer, he does not mention how or why this would affect the implementation of the training, though. We can trust on this source because trainee speaks from his own experience. • Moderate confirmation of proposition 6

220

●observatio n P6(ii)

Account evidence: The survey that was filled in by the When asked the open question: “If you are not respondent after the training took place (T1). applying the training content, what reasons obstruct Trainee explicitly acknowledges that his temper may be an you to apply the training content?” in the survey, he issue in applying the training content. replies “If I react too impulsively in a situation.” • Hu: This evidence is quite directly related to the proposition. It is clear why a temper could interfere with applying what has been learned in a course on communication. We can trust on this source because it is a complement of the first observation which is in line with the argument given. • Strong confirmation of proposition 6 ●Aggregatio In general, there seems to be sufficient evidence for this proposition. The trainee clearly identifies some potential slips n of in transfer and denotes his capacity to anticipate or predict it. evidence for If only P6(i) or P6(ii) is found, we cannot confirm, whereas if both are found we might infer that P6 is present. proposition Sources are relatively independent. 6 Overall confirmation: moderately warranted, given P6(i) and P6(ii) are not fully unique but accurate evidence of the presence of anticipation via some kind of slip monitoring of past experiences. Building block: Coping strategies. Theorized part: Based on coping methods, the trainee applies a threat coping strategy to this 'predicted slip' by selecting only appropriate steps to increase skills retention and generalization [e.g. applying skills in the appropriate setting, reducing interfering and unproductive emotions; retain self-confidence, diagnose support skills needed to maintain training, etc.] Fingerprints: We expect to see the trainee dealing with slipping up and overcoming threats to generalization, by reducing interfering and unproductive emotions and applying skills in the appropriate setting. We mostly assume that the evidence here will be account evidence. If the trainee has formalized this strategy, there could be a small chance we will find some trace evidence. — Htu. ●Theoretical certainty not formulated (no priors). ●Relatively high uniqueness. Find this proposition means that this part of the mechanisms (or the mechanism itself) is present. ●observatio n P7(i)

Account evidence: interview with trainee. Slipping up and overcoming. • Hu: This piece of evidence tells us something about how trainee is coping with slipping situations due to emotions (failing back into old habits) and how he overcomes this by realizing it is a mistake

11:30 Of course, If you’re having some private problems, you’re a bit more on edge. You’re only a human being and then it can happen that, by accident, you say something of which you think later “what have I said now?”, but that training also makes me feel supported to go to that person an hour later and say 221

•

[reflection]. Trainee explicitly addresses a problem “look, it came out like this, that was not my intention” that arises when applying the training and dealt with and then talk it out in a minute, where earlier, you may that obstacle [reducing interfering emotions]. There just wind up in another discussion. are no alternative empirical explanations for this finding. Emotional reactions have been recognized as a possible obstacle to implement the training and, therefore the appropriate strategy have been reducing interfering and unproductive emotions. Strong confirmation of proposition 7

●observatio n P7(ii)

Account evidence: interview with trainee. Applying skills in the appropriate setting. • Mu: This piece of evidence shows us the application of learned skills in the appropriate setting. As a supervisor, trainee tries to set a good example for everyone and create an atmosphere in which the training is often used It is, however, not clear if this behavior is part of a threat-coping strategy or something else. Therefore the uniqueness is moderate. We can trust on this source since it is the own trainee experience what is being shared in the interview. • Moderate confirmation of proposition 7

25:00 I can only say, that we as supervisors, always try to give a start to it, and always try to keep on giving. To do it in a connecting way. To do the communication in a different way than earlier. Instead of discussing things. But you can’t force anyone.

●Aggregatio n of evidence for proposition 7

In general, there are some serious indications that the trainee employs a threat-coping strategy. We therefore think we have some confirmation for this proposition. If only P7(i) is found, we can confirm, whereas if P7(ii) is found we might need P7(i) to infer the presence of proposition 7. The observation (ii) is only supportive for the observation (i). Sources are relatively independent. Overall confirmation: moderately warranted, given P7(i) is accurate evidence of the presence threat-coping strategies.

Building block: Monitoring and self-rewards. Theorized part: The trainee monitors the process of skills transfer (self-monitoring if performance; self-evaluation against goal; selfreaction with self-efficacy) and creates meaningful self-rewards for skill retention.

222

Fingerprints: We expect to see evidence of activities performed by the trainee related to self-monitoring of his/her own performance and some sort of self-rewards for skill retention. We expect that this will mostly be account evidence in the form of the trainee discussing what helps him/her in implementing the training. We can also find account evidence in the survey on how the trainee experiences the training. — Htu. ●Theoretical certainty not formulated (no priors). ●Relatively high theoretical uniqueness. If we find this evidence, it seems likely that it would relate to the causal mechanism that we have discussed. ●observatio Account evidence: The survey that was filled in by the The trainee “agreed” with the statements “I enjoy n P8(i) respondent after the training took place (T1). challenging and difficult job tasks in which I learn new skills” and “I prefer to work in situations that require a • Lu: Trainee had a positive response on whether or high level of talent and capabilities”. not he enjoys challenging and difficult job tasks in which he learns new skills. He also indicates that he prefers to work in situations that require a high level of skills. This indicates that learning new skills and applying them could be rewarding for him. We expect that high scores on this item indicates that learning these skills in itself is rewarding for the trainee. We see no clear reason why the trainee would be dishonest about this. This does not say anything on the monitoring of the process of skills transfer. This piece of evidence, however, tells us little about some kind of monitoring and selfrewards activities. • Weak confirmation of proposition 8 ●observatio Account evidence: The survey that was filled in by the He completely agreed with the statement “Learning n P8(ii) respondent before the training took place (T0). new things is fun” • Lu: Trainee had a positive response on whether or not he thinks learning new things is fun. This indicates that learning new skills and applying them could be rewarding for him. We expect that high scores on this item indicates that learning these skills in itself is rewarding for the trainee. We see no clear reason why the trainee would be dishonest 223

●observatio n P8(iii)

●observatio n P8(iv)

about this. This does not say anything on the monitoring of the process of skills transfer. • Weak confirmation of proposition 8 Account evidence: interview with trainee. The trainee acknowledges that taking the training is fun. • Mu: This piece of evidence tells us something about how trainee acknowledges that taking the training is something he enjoys. This could indicate that following the training can serve as a self-reward. Enjoying following the training, however, does not mean that trainee is applying a self-monitoring strategy, but only a kind of self-reward. • Moderate confirmation of proposition 8 Account evidence: interview with trainee. Monitoring the process of skill transfer. • Hu: This piece of evidence tells us something about monitoring the process of skill transfer. Trainee talks about three things that he does so that he would be able to implement the training: (1) He looks back at the summary they received after the training; (2) He talks to other people about the training (this also relates to the supportive group) and (3) just the need to ‘do it’. Trainee claims he learns by trial and error. He discusses several things that help him monitoring the process of implementation. We can trust on this source and in what it means. • Strong confirmation of proposition 8

40:28 Q Did you think it was fun? Taking the training? Yes, I always like those things. Because I’m also convinced that, apart from the training in itself, it’s also beneficial for the group. And that was also the case. 40:44 Q in general, so not specifically this training, but you like trainings? Yes

46:28 Q So you followed the training, were you able to apply it directly, or were there in-between steps? There were certainly in-between steps. But for myself, we also got some sort of summary. Where every step was listed. It was often about needs and feelings. There was also a list of those. And, by accident, I had some individual conversations with some of my employees. That was, apart from the preparation we de beforehand on paper, looking to the sheet to see what “need” fits this the best. Then you are doing some study work. But that summary is not big, but it has the core of the training. And it’s something that comes back during the conversation. We peek at it. To keep it alive. 47:51 Q So that little summary helped you to prepare the conversations? Yes. What else helped? Well my own conversations with the manager when certain things are talked about. “look, last time I was sitting there during the conversation. This was good and that was not good” 224

that is also discussed. I think that that is a second thing, in addition to the little book we got [the summary] and the third one is the most important one. Just do it. Just do it. I don’t think that there is anyone from our group has the same level of the teacher. Because it’s their way of life. But I do think that a lot of people here still have the some view on how to communicate. And when they need to have a conversation, they think for a bit. “How should I address this?” and if possible from the connective communication. Maybe “peep” [into the summary]. I’m quite convinced by that. ●Aggregatio n of evidence for proposition 8

O u t c o m e

In general, four pieces of evidence have been found. We can confirm the presence of proposition 8. Every observation by itself is not enough to confirm the presence of P8, whereas if P8(iv) is found in combination with P8(iii) we might infer that proposition 8 is present. The observation (i) and (ii) are only supportive for the observation (iii) and (iv). Sources are relatively independent. Overall confirmation: moderately warranted, given P8(iii) and P8(iv) are accurate evidence of the presence of monitoring and self-rewards activities. Theorized outcome: Effective employee training transfer is the application or use of the learned knowledge (content, skills or attitudes) acquired in a training program to the job by trainees, which is maintained over a period of time. Observable manifestations: The employee applies the learned content and skills and they are maintained on the job. Evidence for this can take the form of account evidence. — theoretical Hc, Hu ●Relatively high theoretical certainty. We have to find evidence of the presence of training transfer. ●Relatively high theoretical uniqueness. It is unlike to find these fingerprints if training transfer is not present.— theoretical Hc, Hu ●observatio Account evidence. The survey that was filled in by the In the survey, the trainee answered several questions n O1(i) respondent. on the training and the specific skills that were trained. Trainee had a positive response on questions on the For the generalization and maintenance based on application of the training Govaerts (2017) and Hiva (2011) he “agreed” (a score of 4) on all statements. • Hu: This piece of evidence is directly related to the question on generalization and generalization of the training content to the job. Trainee had a positive 225

•

response on these questions. We expect that trainee would be honest when filling in the survey. We can trust on this source and in what it meant. Strong confirmation of outcome

●observatio n O1(ii)

The survey that was filled in by the respondent. This is account evidence. He agreed with the majority of the statements that tested for leadership skills. • Hu: This piece of evidence is directly related to the questions on the application of the training and leadership skills in particular. Trainee had a positive response in the majority of the statements. We expect that trainee would be honest when filling in the survey. We can trust on this source and in what it meant. • Strong confirmation of outcome

On the two leadership scales, he scored himself quite high on all statements. There was not a single statement with which he disagreed.

●Aggregatio n of evidence for proposition O1

There seems to be a sufficient amount of evidence that supports this proposition. In general, two pieces of evidence has been found. We can therefore confirm the presence of outcome ‘training transfer effectiveness’. If O1(i) or O1(ii) are found, we cannot confirm, whereas if both pieces of evidence are found, we might infer that outcome is present in the case. Both observations reinforce each other since one is more related directly to the outcome and the other to the context of leadership skills. Sources are relatively independent. Overall confirmation: strongly warranted, since in general there is evidence that training transfer effectiveness occurred in this case, because of the presence of the application of the learned knowledge acquired in the training and its maintenance over a period of time and the particular aspect of leadership skills.

226

A11.4 ROADMAP LEARNER AGENCY Case 1: T1

Causal relationship Causal mechanism linking sense of urgency to the trainee’s use of learned material to the job with the effectiveness of training transfer. Prior relatively low ●There is no existing research that documents the mechanism that connects employee learner agency to training transfer. Theorized cause: Based on the available literature we understand an employee’s sense of urgency as one’s (1) clear need to engage in training (2) because of the identification of a hiatus between current knowledge and skill and required knowledge and skill in the future (3), with the understanding that overcoming the hiatus is within reach of the capabilities of the employee. C Observable manifestations: Need to engage in training because of the identification of a hiatus between current knowledge and skill a and required knowledge and skill in the future: (1) “I felt that overcoming the challenge I identified was within my capabilities”; (2) “I u identified a more general hiatus in my knowledge and behaviour that was not bound to a specific task.” We expect to find account s evidence for this and perhaps trace evidence. e — Htc, Htu ●Relatively high theoretical certainty. If sense of urgency is present, we expect to find evidence of this in the case. ●Relatively high theoretical uniqueness: It is unlikely that we find these observables manifestations if sense of urgency is not present. ●observatio Account evidence: The survey that was completed by the The respondent answered positively to the statements: n C1(i) respondent after the training took place (T1). “Overcoming this challenge lay within my capabilities “, Trainee confirms sense of urgency in the survey. “I identified a gap between what I could do and what I would be doing in the future.” • Hu: This observation shows that the trainee experienced some sort of sense of urgency. He agreed with the statements related to the observable manifestations of sense of urgency (overcoming the challenge and identification of hiatus). • Strong confirmation of condition 1

227

●observatio n C1(ii)

Account evidence: Interview with trainee Trainee notes that even if he could have done his work without the training, the training make his/her work better and she/he acknowledges its value. • Mu: This piece of evidence tells us that trainee recognizes the importance of the training in his/her job performance. There is no acknowledgement of a need, but it is noticed that there is an improvement. Even if trainee also recognize that he/she can always do his/her job without training, the fact to have it pursued was a surplus, because the new tools and insights have make the job better. • Moderate confirmation of condition 1

5:25 “Q Did beforehand, for you personally, you feel like there was a need for that training? Would you be able to do your work without the training? Yes, I think you always can Q Yes, there is a need? Yes, in the sense that I could do my work without it. Because those are things you have done in the past and are things that come from you education. But it is a surplus if you can finetune in that. To get some new tools, some new insights. So I think yes I could have done it without. But I just think it goes better, easier with. Also it’s a bit more uniform in the organization. Thanks to the training.”

●observatio n C1(iii)

Account evidence: Interview with trainee In this excerpt, the trainee notes that he was aware that he could improve his skills. • Mu: The piece of evidence shows us that trainees recognize the utility of the training, even if the needed was not absolute. There is a reflection about the improvement of skills that make the job better. This piece, however, tells us not much about overcoming challenges. • Moderate confirmation of condition 1

23:54 “ Q We’ve also briefly talked about the need that you experienced beforehand. We’re you aware that it could be better. You said afterwards that it was working. So the need was not absolute. But, .. the usefulness.. Yes. That yes. Those are topics that you are already working with. But where you do experience “ okay, this can go better” or “I’m struggling with this”. Or I didn’t do that well. 24:35 Q So that was something, beforehand, you did realize that “okay,…” (completes) “this can be done better, and here training is useful”. 24:44 Q Who has… you said training is useful. Who says that? Me (laughs). It’s not that I ever got feedback on “that’s not good”. But if you’re having yearly evaluation conversations, to stick with the example, you experience that that conversation was not useful, or it went difficult. Or I can’t make it work with this 228

I n t e r m e d i a t e O u t c o m e 1

employee. A bit of self-reflection teaches you that some improvement there is possible.” ●Aggregatio Three pieces of evidence have been found. We can confirm the presence of condition 1 “sense of urgency” in this n of case. evidence for If only C(i) is found, we can confirm the evidence. Observation (i) and (ii) are supportive evidence for (i). proposition Sources are relatively independent. C1 Overall confirmation: moderately warranted, given C1(i) is accurate evidence of the presence of sense of urgency and C1(ii) and C(iii) even if they are moderate in confirmation, overcoming challenges and identification of a need are not really confirmed in this case. Theorized outcome:. A state of maximum involvement and intrinsic motivation is achieved due to a balance between tasks demands and competences Observable manifestations: We expect to find evidence of the employee’s motivation to learn new things. This can take the form of messages shared with other colleagues or own reflection that we expect to get via interviews or trace evidence. — Htu ● Theoretical certainty not formulated (no priors). ● Relatively low theoretical uniqueness. Find this observable manifestation does not necessarily mean that the mechanism ‘learner agency’ is being operative. It is an intermediate outcome. ●observatio n O1(i)

•

There is no observations. Difficulties in the access to the informant were addressed due to the global context of covid19. Therefore, the absence of this observation does not mean the observable manifestation is absent. • No inferences are possible to make. No observations about the intermediate outcome were found in this case. This does not mean the observable manifestations are absent. No further access to the sources were possible. Overall confirmation: No inferences can be made.

●Aggregatio n of evidence for proposition O1 Building block: Goal setting Theorized part: Because of this intrinsic motivation, employee develops a feeling of relevance of the job's tasks requirements and expresses expectations about the learning process by setting goals about what to get from this learning experience.

229

Fingerprints: Employee with clear and firm commitment with learning. This can take the form of informal discussions with colleagues or e-mails or messages where this is mentioned. This will be measured using account evidence with participants and trace evidence (e-mails, messages). This can also been in the survey that was administered with the employee. — Htu ●Theoretical certainty not formulated (no priors) ●Relatively high theoretical uniqueness. Highly unlikely to find this fingerprint, as it looks like, without the mechanism of ‘learner agency’ being operative. ●observatio Account evidence: The survey that was filled in by the Answered “yes” on the following question “Did you n P1(i) respondent after the training took place (T1). personally set some goals before you participated in These answers indicate that he did set some goal, but it’s the training?”. He did not specify which goals in a unclear what that goal was. follow-up question. • Lu: This piece of evidence tells us little about the firm commitment with learning. There is no further explanation about the reasons to set goals before participating in the training. We can, however, trust on this source. • Weak confirmation of part 1

●Aggregatio A single piece of evidence has been found. n of With this single piece we cannot infer that P1 is present. evidence for Overall confirmation: weakly warranted. There is a single piece of evidence which cannot be interpreted as firm proposition commitment with learning. 1 Building block: Free choice of learning Theorized part: Following his/her goals, the employee makes a free choice by engaging in a training program and focus his/her attention on the tasks to be performed Fingerprints: We expect to see evidence of employees participating in training by own choice. This can be measured using account evidence or trace evidence. — Htu ●Theoretical certainty not formulated (no priors). ● Relatively high theoretical uniqueness. Finding evidence that employees participated in the training by own choice would mean that the causal mechanisms “learner agency” is taking place. ●observatio Account evidence: The survey that was filled in by the The trainee replied yes to the question “did you signal n P2(i) respondent before the training took place (T0). your need for the ESF-training yourself?”. In the 230

The trainee indicates that he himself signaled the need for this training to his supervisors during evaluation conversations. • Lu: This piece of evidence tells us little about the reasons to follow the training. Even if the trainee indicates “yes” to the respective question, we do not know if this is related to free choice or not. We can trust on the source, but we cannot interpret this as strong evidence. • Weak confirmation of part 2

following question he specified that he did this during evaluation conversations with his supervisor.

●observatio n P2(ii)

Account evidence: The survey that was filled in by the respondent after the training took place (T1). These answers indicate that there was no formal obligation to participate in the training, but the trainee did think that this was “expected of him” by his supervisor. • Mu: This piece of observation tells us something about the commitment of the trainee with the training by their own. Trainee’s responses reflects that the training was not declared as mandatory for his/her supervisor. Since this is account evidence from a survey, we did not find more observations related to the reasons of selecting the training. There is not clarity whether the trainee freely chose or not the training. • Moderate confirmation of part 2

The trainee agreed with the statement “my supervisor expected me to participate in the training”, but disagreed with the statement “my supervisor told me that following the training was obligated”.

●observatio n P2(iii)

Account evidence: Interview with trainee When asked about why trainee participated, he/she does not talk about motivation or wanting to improve his/her skills at the start. The main reason seems to be “because it’s expected of you” and he mentions that it is mandatory. • Lu: This observation tells us that even if the training was mandatory, employee expects it each year. In addition, he/she valorizes the training because its

3:13 Q why did you participate in the training? First, because it’s expected of you, from your role. So its obligated. But on the other hand, we have been doing it for a couple of years now like that, preparing us for the year that is coming. And it has always been positive. The trainings are always relevant to work with. They are relevant in practice. So you really come home thinking “okay, I can do something with this, I 231

•

importance in getting new tools. It is perceived as positive. This observation, however tells us that there is no free choice to engage in training. Weak confirmation of part 2

got some tools”. And with the experience of the past years, I think it’s useful to participate every time.

●observatio n P2(iv)

Account evidence: Interview with trainee This fragment shows that the trainee did not really have a big say in which training he would follow. • Lu: This observation tells us something about the goals to follow the training. It seems that the goals were oriented to support the organization. • Weak confirmation of part 2

2:35 “Q could you choose which trainings you followed? Could you select them? No, everybody did everything, but with rotation. But in fact, the program was composed in such a way that it covered the goals for the next year. Everybody needed them. To go through all trainings modules. The goal was to support the organization goals.”

●observatio n P2(v)

Account evidence: Interview with trainee In this fragment, the trainee mentions that the training was obligatory. • Lu: This piece of evidence tells us two things. For one side, trainee tells us that training was mandatory and that, in a situation where it would have been voluntary, he/she would have participated. There is not information about goals. • Weak confirmation of part 2

31:32” Q the training was partially obligated. But you also indicated that you would still participate if it wasn’t. It was obligated, but if it wouldn’t be obligated, I would also have participated. Because it really gives you tools to work with in the coming years. 32:00 Q Do you think that it being obligated, in itself, is something that bothers you? No, because it’s useful. That’s why it doesn’t bother me.”

●Aggregatio n of evidence for proposition 2

We have found evidence that can be interpreted as supporting the absence of free choice and goals to follow the training. Training was mandatory. Some observations showed us that even if it is mandatory, trainee is not struggling with that. Due to these four pieces of evidence found tells us the contrary to what we expected to find, we can disconfirm the presence of the part 2 of the mechanism. Sources are relatively independent. Overall confirmation: disconfirming. There is a significant amount of evidence that confirms that free choice and goal setting are not present in this case as theorized. Building block: Learning at hands Theorized part:. Based on such attention, the employee undertakes learning activities related to complex tasks

232

Fingerprints: We expect to see evidence of employee developing own learning activities which matches with his/her goals. We will measure this with account evidence from interviews with the participant — Htu. ●Theoretical certainty not formulated (no priors). ● Relatively high theoretical uniqueness. It is unlike to find this fingerprint if the mechanism “learner agency” is not being operative. ●observatio n P3(i)

Account evidence: Interview with trainee To be “forced” to use the training is “good”. • Lu: This piece of evidence tells us that being ‘forced’ to use the training is something good’. When talking about the role of the supervisor, he mentions that it is good to be “forced” to use the training. Although this does not directly say anything about his own learning activities, it does imply that external pressure to use the training improves adaptation of the training. This would not be necessary if the trainee experienced a strong sense of urgency and was intrinsically motivated. • Weak confirmation of condition 3

●observatio n P3(ii)

Account evidence: Interview with trainee The trainee mentions that they got specific tools to use the training. So although he did use tools, he did not develop them himself. • Lu: This observation tell us little about the learning activities carried out by the trainee. The trainee

19: 25 Q if you would need to think about the ideal situation about those actors [the supervisor], how would you expect that they react to the training? Wait, that’s a difficult question… at the training itself? Q afterwards, so let’s say we are talking about the supervisor. What kind of behavior do you think is best to implement the training? My supervisor towards me to make sure that I will apply what I have learnt? Q yes A bit what I said at the start, setting concrete expectations. Force to use certain things. Like those evaluation conversations or working out a policy about the patient-centered care or employee satisfaction where it’s concretely expected to suggest points that can be improved so report that and show that you are making progress. In that way, I think you are forced to apply it. 20:45 Q so this is also done? Yes 20:52 Q and that’s the ideal scenario? Or do you see improvements? (thinks) not really, I think it’s okay like that. 14:28 “Q did it take a lot of effort to apply what you have learned on the workfloor? Euhm. No, not really, because the trainings are quite hands-on. You leave the classroom with specific tools. That makes it quite easy to translate this to the worklfoor and apply it.” 233

•

●Aggregatio n of evidence for proposition 3 4

makes references to specific tools, but he/she does not go further wit this. We can trust on the source but we cannot interpret this observation as Part 3 being present. Weak confirmation of proposition 3

Two pieces of evidence were found. With P3(i) and P3(ii) we cannot infer that P3 is present. Sources are relatively independent. Overall confirmation: weakly warranted, since what the observations tells us little regarding to the fingerprints of part 3.

Building block: Internal feedback: reflection-in-action Theorized part: Employee actively reflects about the learning process: "how everything is going on" and adapts learning strategies to ensure the ultimate goal of transfer Fingerprints: Expect to see employees attempting to modify his/her learning strategies to a better transfer. This can take the form of spending more hours in completing complex tasks related to the new content acquired in the training or any activity where employees spend more effort to achieve own goal. We will measure this with account evidence from interviews with participants. — Htu. ● Theoretical certainty not formulated (no priors). ●Relatively high theoretical uniqueness. It is unlike to find this fingerprint is the mechanism “learner agency” is not taking place. ●observatio • There are no observations. n P4(i) • No inferences can be made ●Aggregatio Since no observations have been found, no inferences can be made regarding the presence/absence of P4. n of Overall confirmation: No inferences. evidence for proposition 4 Building block: Internal feedback: reflection-in-action Theorized part: By adapting learning strategies, employee reaches certain goals that are identified and perceived as relevant by her/himself. This keeps the motivation alive to reach the ultimate goals: transfer

234

Fingerprints: Employee going ahead with training and activities related to learning improvement and performance improvement. We will measure this using account evidence from interviews with participants — Htu. ●Theoretical certainty not formulated (no priors) ● Relatively high theoretical uniqueness. To find this fingerprint actually would mean that the mechanism “learner agency” is taking place in this case. ●observatio • There are no observations. n P5(i) • No inferences can be made

●Aggregatio Since no observations have been found, no inferences can be made regarding the presence/absence of P5. n of Overall confirmation: No inferences. evidence for proposition 5 Building block: Internal feedback: reflection-in-action Theorized part: Based on such ultimate goal, employee undertakes activities related to complex tasks to be applied to the job Fingerprints: Employee performing complex tasks that she/he knows will be required in the job context. We will measure this using account evidence from interviews with participants — Htu. ●Theoretical certainty not formulated (no priors). ● Relatively low theoretical uniqueness. Just because the trainee undertakes activities related to complex tasks required in the job context, does not mean that the mechanisms “learner agency” is taking place in this case. ●observatio • There are no observations. n P6(i) • No inferences can be made ●Aggregatio Since no observations have been found, no inferences can be made regarding the presence/absence of P6. n of Overall confirmation: No inferences. evidence for proposition 6 Building block: Monitoring Theorized part: Employee identifies the strength and weakness to improve themselves in the application of the learned content to the job by focusing on what "to do better".

235

Fingerprints: We expect to see brainstorming carried out by the employee in order to evaluate the good, the bad and the ugly of the learning process to improve him/herself. We will measure this using account evidence from interviews with participants. This can also take the form of trace evidence, if some document related to this brainstorm was created and recorded — Htu. ●Theoretical certainty nor formulated (no priors). ● Relatively high theoretical uniqueness. It is unlike to find this fingerprint if the mechanisms “learner agency” is not taking place in this case. ●observatio • There are no observations. n P7(i) • No inferences can be made

●Aggregatio Since no observations have been found, no inferences can be made regarding the presence/absence of P7. n of Overall confirmation: No inferences. evidence for proposition 7 Building block: Networking-feedback Theorized part: Employee asks for feedback from others as a way to evaluate the learning and application process objectively Fingerprints: Report, minutes, e-mails from trainee asking to colleagues providing some sort of feedback about learning and performance. We could measure this with trace evidence. — Htu. ●Theoretical certainty not formulated (no priors). ● Relatively low theoretical uniqueness. Just because the trainee asks for feedback from others to evaluate own learning and performance, does not mean that the mechanisms “learner agency” is taking place. This can be part of a routine of peer support or self-management. ●observatio • There are no observations. n P8(i) • No inferences can be made. ●Aggregatio Since no observations have been found, no inferences can be made regarding the presence/absence of P8. n of Overall confirmation: No inferences. evidence for proposition 8 Building block: Networking-feedback Theorized part: Employee receives feedback from peers and adapt their activities to overcome some obstacles to transfer goals 236

Fingerprints: We expect to find report, minutes, e-mails from colleagues providing some sort of feedback to employee taking the training. We will measure this with trace evidence or account evidence. — Htu. ●Theoretical certainty not formulated (no priors). ● Relatively low theoretical uniqueness. Just because the trainee received feedback from peers and adapt activities to overcome with obstacles does not mean that the mechanism “learner agency” is taking place. This can also be part of peer support related activities or self-management intervention. ●observatio Account evidence: Interview with trainee. 6:30 […] For example these evaluation conversation n P9(i) The trainee discusses how he interacts with peers to apply was a roleplaying game with another colleague that the training. observed. And then feedback was given and experiences were shared. Those are valuable tools, so • Hu: This observation tells us something about the that when you have to it, where you think back about feedback between peers and the experiences it, or certain tips or tricks.. shared. It is called a ‘ropleplaying’ to transfer. The 7:26 Q you think back to the content of the training? trainee mentions the importance of brainstorming Yes. So there you notice you can use it. Apart from to share vision and best practices to handle better that, there was also the brainstorm about our vision the use of training content to the job. and best practices. Where when you draft your policy • Strong confirmation of proposition 9 plan and think with your colleagues about how you will handle it, where you also get things that you can go back to, from the brainstorm or discussion with colleagues. So in that sense, you get things out of that, that you can use for your job. ●observatio Account evidence: Interview with trainee. 10:15 “Q Would you say that the sharing of ideas and n P9(ii) The trainee mentions that he does receive some feedback, the feedback is essential for implementation? but that it would be better it this person could also be there To implement it in the long term… I think so. You get when the training is being applied. your training, you train your skills in a simulation environment, but the moment you will apply it, I think • Hu: The observation tells us something about the if you want to finetune those skills and embed them feedback received from peers to overcome the and make them your own. At that moment, you need barriers to transfer. The trainee highlights the some coaching or someone to talk to. importance of sharing visions for the long-term 10:51 Q Do you, in your environment, have people implementation of the training. The ‘coaching that could help you with questions? activity’ is perceived as essential for a better Yes, you can talk to your colleagues or your supervisor. transfer. The trainee also highlights the need to But, that is outside of the setting, afterwards. That you have someone, an ‘observer’ to evaluate the 237

training application immediately and provide a more useful feedback for better mastering. Strong confirmation of proposition 9

1 0

1 1

say “I have done it like that, could I do it differently, is this okay”. It would also be interesting that at the moment you are applying something, you have • someone, an observer, standing next to you that can give you feedback afterwards. I think. Insofar that it is feasible of course. I think that is an effective way to gain new skills and to master them and apply them.” ●Aggregatio Two pieces of evidence were found. We can confirm the presence of proposition 9. n of If only P9(i) is found, we cannot confirm; whereas if P9(ii) is found, we can infer that P9 is present. P9(i) is supportive evidence for evidence for P9(ii). proposition Sources are relatively independent. 9 Overall confirmation: strongly warranted, P9(i) and P9(ii) are accurate evidence of the trainee receiving feeback from peers for a better transfer. Building block: Internal feedback: reflection-on-action Theorized part: Employee looks back on and think on own action about how learning process and applicability went. Fingerprints: We expect to see employee reflections about his/her experiences in learning process in a retrospective way. We will measure this with account evidence from interviews with participants. — Htu. ●Theoretical certainty not formulated (no priors). ● Relatively high theoretical uniqueness. It is unlike to find this fingerprint if the mechanism ‘learner agency’ is not taking place in this case. ●observatio • There are no observations. n P10(i) • No inferences can be made ●Aggregatio Since no observations have been found, no inferences can be made regarding the presence/absence of P8. n of Overall confirmation: No inferences. evidence for proposition 10 Building block: Coping strategies Theorized part: Employee identifies and selects steps to ensure transfer by focusing on applying the learned content in the appropriate setting and reducing possible interferences to transfer

238

Fingerprints: Employee selecting only appropriate steps to increase skills retention and generalization [e.g. applying skills in the appropriate setting, reducing interfering and unproductive emotions; retain self-confidence, diagnose support skills needed to maintain training, etc]. We mostly assume that the evidence here will be account evidence. — Htu. ●Theoretical certainty not formulated (no priors). ● Relatively low theoretical uniqueness. Just because the trainee selects appropriated steps to transfer, does not mean that the mechanisms “learner agency” is taking place in this case. This can also be part of self-management intervention. ●observatio • There are no observations. n P11(i) • No inferences can be made

1 2

●Aggregatio Since no observations have been found, no inferences can be made regarding the presence/absence of P8. n of Overall confirmation: No inferences. evidence for proposition 11 Building block: Monitoring and self-rewards. Theorized part: Trainee monitors the process of skills transfer (self-monitoring if performance; self-evaluation against goal; selfreaction with self-efficacy) and create meaningful self-rewards for skill retention. Fingerprints: We expect to see evidence of activities performed by the trainee related to self-monitoring of his/her own performance and some sort of self-rewards for skill retention. We expect that this will mostly be account evidence in the form of the trainee discussing what helps him/her in implementing the training. We can also find account evidence in the survey on how the trainee experiences the training. — Htu. ●Theoretical certainty not formulated (no priors). ●Relatively high theoretical uniqueness. If we find this evidence, it seems likely that it would relate to the causal mechanism that we have discussed. ●observatio Account evidence: The survey that was filled in by the The trainee “agreed” with the statements “I enjoy n P12(i) respondent after the training took place (T1). challenging and difficult job tasks in which I learn new skills” and “I prefer to work in situations that require a • Lu: Trainee had a positive response on whether or high level of talent and capabilities”. not he enjoys challenging and difficult job tasks in which he learns new skills. He also indicates that he prefers to work in situations that require a high level of skills. This indicates that learning new skills and applying them could be rewarding for him. We 239

• ●observatio n P12(ii)

O u t c o

expect that high scores on this item indicates that learning these skills in itself is rewarding for the trainee. We see no clear reason why the trainee would be dishonest about this. This does not say anything on the monitoring of the process of skills transfer. This piece of evidence, however, tells us little about some kind of monitoring and selfrewards activities. Weak confirmation of proposition 12

Account evidence: The survey that was filled in by the respondent before the training took place (T0). • Lu: Trainee had a positive response on whether or not he thinks learning new things is fun. This indicates that learning new skills and applying them could be rewarding for him. We expect that high scores on this item indicates that learning these skills in itself is rewarding for the trainee. We see no clear reason why the trainee would be dishonest about this. This does not say anything on the monitoring of the process of skills transfer. • Weak confirmation of proposition 12

He agreed with the statement “Learning new things is fun”

●Aggregatio Two pieces of evidence have been found. However, due to these pieces are not strong evidence, we can only weakly n of infer the presence of proposition 12. evidence for Sources are relatively independent. proposition Overall confirmation: weakly warranted, given our two observations are not accurate evidence of some sort of 12 monitoring and self-rewards int his case. Theorized outcome: Effective employee training transfer is the application or use of the learned knowledge (content, skills or attitudes) acquired in a training program to the job by trainees, which is maintained over a period of time. Observable manifestations: The employee applies the learned content and skills and they are maintained on the job. Evidence for this can take the form of account evidence. — theoretical Hc, Hu ●Relatively high theoretical certainty. We have to find evidence of the presence of training transfer. 240

m e

●Relatively high theoretical uniqueness. It is unlike to find these fingerprints if training transfer is not present. ●observatio Account evidence. The survey that was filled in by the In the survey, the trainee answered several questions n O1(i) respondent. (T1) on the training and the specific skills that were trained. Trainee had a positive response on questions on the For the generalization and maintenance based on application of the training Govaerts (2017) and Hiva (2011) he “agreed” (a score of 4) on all statements. • Hu: This piece of evidence is directly related to the question on generalization and generalization of the training content to the job. Trainee had a positive response on these questions. We expect that trainee would be honest when filling in the survey. We can trust on this source and in what it meant. • Strong confirmation of outcome ●observatio n O1(ii)

The survey that was filled in by the respondent. This is account evidence. (T1) He agreed with the majority of the statements that tested for leadership skills. • Hu: This piece of evidence is directly related to the questions on the application of the training and leadership skills in particular. Trainee had a positive response in the majority of the statements. We expect that trainee would be honest when filling in the survey. We can trust on this source and in what it meant. • Strong confirmation of outcome

On the two leadership scales, he scored himself quite high on all statements on leadership self-efficacy and empowering leadership.

●Aggregatio n of evidence for proposition O1

Overall confirmation: strongly warranted, since in general there is evidence that training transfer effectiveness occurred in this case, because of the presence of the application of the learned knowledge acquired in the training and its maintenance over a period of time and the particular aspect of leadership skills.

242

ANNEX 12: ADDITIONAL EVIDENCE FROM THE ROADMAPS The additional evidence from the roadmaps is, under certain conditions (e.g. signing a data usage agreement), available on request. Please contact Benjamin.schalembier@vlaanderen.be for more information.

Additional evidence roadmap Signaling and Retention Causal Mechanism triggered by supervisor support (15 pages) Additional evidence roadmap Enhanced learning intervention triggered by peer support (31 pages)

243