The evolution of approaches to the measurement of organizational and strategic constructs with the use of computational text analysis

With the increased availability of textual documents published by organizations, computational text analysis (CTA) opens new avenues for management and strategy research. However, while existing text processing techniques are well documented, approaches to measuring organizational and strategic constructs based on these techniques — that represent only a portion of measurement methodologies within CTA-based research — remain considerably less developed. Although the use of CTA-based methodologies offers new opportunities for researchers, the literature employing them remains scattered, with the limitations of such methodologies being infrequently discussed. In this regard, the authors aim to examine the evolution of this line of research, focusing explicitly on measure development practices within CTA-based methodologies. This study offers an overview of methodological progress and challenges in this area based on a critical review of over 20 specialized empirical research articles, and presents the typology of measure development approaches in CTAbased assessments of organizational and strategic constructs. The authors illustrate the use of the proposed typology by examining the literature that employs CTA to assess dynamic capabilities. The present work seeks to provide a useful methodological foundation for further study and enhancement of CTA-based measurement of organizational and strategic constructs, offering an accessible introduction to this interdisciplinary area for both management and strategy scholars as well as practitioners in the field.

1. Katkalo, V. S. (2008). The evolution of strategic management theory. (3rd ed.). SPbU.

2. Arora, S. K., Li, Y., Youtie, J., & Shapira, P. (2020). Measuring dynamic capabilities in new ventures: exploring strategic change in US green goods manufacturing using website data. The Journal of Technology Transfer, 45(5), 1451–1480. https://doi.org/10.1007/s10961019-09751-y.

3. Belderbos, R., Grabowska, M., Leten, B., Kelchtermans, S., & Ugur, N. (2017). On the use of computer-aided text analysis in international business research. Global Strategy Journal, 7(3), 312–331. https://doi.org/10.1002/gsj.1162.

4. Boyd, B. K., Bergh, D. D., Ireland, R. D., & Ketchen Jr, D. J. (2013). Constructs in strategic management. Organizational Research Methods, 16(1), 3–14.

5. Boyd, B. K., Gove, S., & Hitt, M. A. (2005). Construct measurement in strategic management research: illusion or reality? Strategic Management Journal, 26(3), 239–257.

6. Bhandari, K. R., Rana, S., Paul, J., & Salo, J. (2020). Relative exploration and fi rm performance: Why resource-theory alone is not suffi cient? Journal of Business Research, 118, 363–377. https://doi.org/10.1016/j.jbusres.2020.07.001.

7. den Hertog, P., van der Aa, W., & de Jong, M. W. (2010). Capabilities for managing service innovation: towards a conceptual framework. Journal of Service Management, 21(4), 490–514. https://doi.org/10.1108/09564231011066123.

8. Eggers, J. P., & Kaplan, S. (2009). Cognition and renewal: Comparing CEO and organizational eff ects on incumbent adaptation to technical change. Organization Science, 20(2), 461–477. https://doi.org/10.1287/orsc.1080.0401.

9. Eisenhardt, K. M., & Martin, J. A. (2000). Dynamic capabilities: what are they?Strategic Management Journal, 21(10–11), 1105–1121. https://doi.org/10.1002/10970266(200010/11)21:10/11<1105::AID-SMJ133>3.0.CO;2-E.

10. Fyff e, S., Lee, P., & Kaplan, S. (2023). “Transforming” Personality Scale Development: Illustrating the Potential of State-of-the-Art Natural Language Processing. Organizational Research Methods, 27(2), 265–300. https://doi.org/10.1177/10944281231155771.

11. Garcia, R., & Calantone, R. (2002). A critical look at technological innovation typology and innovativeness terminology: a literature review. Journal of Product Innovation Management, 19(2), 110–132. Portico. https://doi.org/10.1111/1540-5885.1920110.

12. Gokhberg, L., Meissner, D., & Kuzminov, I. (2022). What semantic analysis can tell us about long term trends in the global STI policy agenda. The Journal of Technology Transfer, 48(6), 2249–2277. https://doi.org/10.1007/s10961-022-09959-5.

13. Hickman, L., Thapa, S., Tay, L., Cao, M., & Srinivasan, P. (2020). Text Preprocessing for Text Mining in Organizational Research: Review and Recommendations. Organizational Research Methods, 25(1), 114–146. https://doi.org/10.1177/1094428120971683.

14. Kabanoff , B., Waldersee, R., & Cohen, M. (1995). Espoused values and organizational change themes. Academy of Management Journal, 38(4), 1075–1104. https://doi.org/10.5465/256621.

15. Kaplan, S. (2008). Cognition, capabilities, and incentives: Assessing fi rm response to the fi ber-optic revolution. Academy of Management Journal, 51(4), 672–695. https://doi.org/10.5465/amr.2008.33665141.

16. Keil, T., Maula, M., & Syrigos, E. (2017). CEO entrepreneurial orientation, entrenchment, and fi rm value creation. Entrepreneurship Theory and Practice, 41(4), 475–504. https://doi.org/10.1111/etp.12213.

17. Kohler, M., Feldmann, N., Kimbrough, S. O., & Fromm, H. (2014). Service innovation analytics: Leveraging existing unstructured data to assess service innovation capability. International Journal of Information System Modeling and Design (IJISMD), 5(2), 1–21. https://doi.org/10.4018/ijismd.2014040101.

18. Laaksonen, O., & Peltoniemi, M. (2016). The Essence of Dynamic Capabilities and their Measurement. International Journal of Management Reviews, 20(2), 184–205. Portico. https://doi.org/10.1111/ijmr.12122.

19. Levinthal, D. A., & March, J. G. (1993). The myopia of learning. Strategic Management Journal, 14(S2), 95–112. https://doi.org/10.1002/smj.4250141009.

20. Lu, Q., Meng, X., Su, J., Ming, A. A. K., Wu, Y., & Wang, C. (2023). TMT functional background heterogeneity and SMEs’ performance: The role of dynamic capabilities and business environment. Journal of Business Research, 160, 113807. https://doi.org/10.1016/j.jbusres.2023.113807.

21. Lumpkin, G. T., & Dess, G. G. (1996). Clarifying the entrepreneurial orientation construct and linking it to performance. Academy of Management Review, 21(1), 135–172. https://doi.org/10.5465/amr.1996.9602161568.

22. March, J. G. (1991). Exploration and Exploitation in Organizational Learning. Organization Science, 2(1), 71–87. https://doi.org/10.1287/orsc.2.1.71.

23. McKenny, A. F., Aguinis, H., Short, J. C., & Anglin, A. H. (2018). What doesn’t get measured does exist: Improving the accuracy of computer-aided text analysis. Journal of Management, 44(7), 2909–2933. https://doi.org/10.1177/0149206316657594.

24. Menon, A., Choi, J., & Tabakovic, H. (2018). What You Say Your Strategy Is and Why It Matters: Natural Language Processing of Unstructured Text. Academy of Management Proceedings, (1), 18319. https://doi.org/10.5465/ambpp.2018.18319abstract.

25. Morris, R. (1994). Computerized content analysis in management research: A demonstration of advantages & limitations. Journal of Management, 20(4), 903–931. https://doi.org/10.1016/0149-2063(94)90035-3.

26. Moss, T. W., Payne, G. T., & Moore, C. B. (2014). Strategic consistency of exploration and exploitation in family businesses. Family Business Review, 27(1), 51–71. https://doi.org/10.1177/0894486513504434.

27. Mousa, F.-T., Wales, W. J., & Harper, S. R. (2015). When less is more: EO’s infl uence upon funds raised by young technology fi rms at IPO. Journal of Business Research, 68(2), 306–313. https://doi.org/10.1016/j.jbusres.2014.07.003.

28. Neuendorf, K. A. (2002). The Content Analysis Guidebook. Thousand Oaks, CA: Sage Publications.

29. Pandey, S., & Pandey, S. K. (2019). Applying natural language processing capabilities in computerized textual analysis to measure organizational culture. Organizational Research Methods, 22(3), 765–797. https://doi.org/10.1177/1094428117745648.

30. Pandey, S., Pandey, S. K., & Miller, L. (2017). Measuring innovativeness of public organizations: Using natural language processing techniques in computer-aided textual analysis. International Public Management Journal, 20(1), 78–107. https://doi.org/10.1080/10967494.2016.1143424.

31. Porac, J. F., Wade, J. B., & Pollock, T. G. (1999). Industry Categories and the Politics of the Comparable Firm in CEO Compensation. Administrative Science Quarterly, 44(1), 112– 144. https://doi.org/10.2307/2667033.

32. Short, J. C., Broberg, J. C., Cogliser, C. C., & Brigham, K. H. (2010). Construct validation using computer-aided text analysis (CATA) an illustration using entrepreneurial orientation. Organizational Research Methods, 13(2), 320–347. https://doi.org/10.1177/1094428109335949.

33. Short, J. C., & Palmer, T. B. (2008). The application of DICTION to content analysis research in strategic management. Organizational Research Methods, 11(4), 727–752. https://doi.org/10.1177/1094428107304534.

34. Short, J. C., Payne, G. T., Brigham, K. H., Lumpkin, G. T., & Broberg, J. C. (2009). Family Firms and Entrepreneurial Orientation in Publicly Traded Firms. Family Business Review, 22(1), 9–24. https://doi.org/10.1177/0894486508327823.

35. Snyder, H. (2019). Literature review as a research methodology: An overview and guidelines. Journal of Business Research, 104, 333–339. https://doi.org/10.1016/j.jbusres.2019.07.039.

36. Stone, P. J., Dunphy, D. C., Smith, M. S., & , Ogilvie, D. M. (1966). The General Inquirer: A Computer Approach to Content Analysis. MIT Press, Cambridge, MA.

37. Teece, D. J. (2007). Explicating dynamic capabilities: the nature and microfoundations of (sustainable) enterprise performance. Strategic Management Journal, 28(13), 1319–1350. Portico. https://doi.org/10.1002/smj.640.

38. Teece, D. J. (2023). The Evolution of the Dynamic Capabilities Framework. In: Adams, R., Grichnik, D., Pundziene, A., Volkmann, C. (Eds.), Artificiality and Sustainability in Entrepreneurship (p. 113–129).

39. FGF Studies in Small Business and Entrepreneurship. Springer, Cham. https://doi.org/10.1007/978-3-031-11371-0_6.

40. Teece, D., & Pisano, G. (1994). The Dynamic Capabilities of Firms: an Introduction. Industrial and Corporate Change, 3(3), 537–556. https://doi.org/10.1093/icc/3.3.537-a.

41. Teece, D. J., Pisano, G., & Shuen, A. (1997). Dynamic capabilities and strategic management. Strategic Management Journal, 18(7), 509–533. https://doi.org/10.1002/(sici)1097-0266(199708)18:7<509::aid-smj882>3.0.co;2-z.

42. Ugur, N., Belderbos, R., Kelchtermans, S., & Leten, B. (2024). The Long March: The quest for valid text-based indicators of exploration and exploitation.Strategic Organization, 22(2), 269–296. https://doi.org/10.1177/14761270241231724.

43. Uotila, J., Maula, M., Keil, T., & Zahra, S. A. (2009). Exploration, exploitation, and fi nancial performance: Analysis of S&P 500 corporations. Strategic Management Journal, 30(2), 221–231. https://doi.org/10.1002/smj.738.

44. Vaara, E., & Fritsch, L. (2022). Strategy as language and communication: Theoretical and methodological advances and avenues for the future in strategy process and practice research. Strategic Management Journal, 43(6), 1170–1181. Portico. https://doi.org/10.1002/smj.3360.

45. Winter, S. G. (2003). Understanding dynamic capabilities. Strategic Management Journal, 24(10), 991–995. Portico. https://doi.org/10.1002/smj.318.

46. Wolfe, R. A., Gephart, R. P., & Johnson, T. E. (1993). Computer-facilitated qualitative data analysis: Potential contributions to management research. Journal of Management, 19(3), 637–660. https://doi.org/10.1177/014920639301900307.

47. Wolfe, M. T., & Shepherd, D. A. (2015). “Bouncing back” from a loss: Entrepreneurial orientation, emotions, and failure narratives. Entrepreneurship Theory and Practice, 39(3), 675–700. https://doi.org/10.1111/etap.12057.

48. Yang, J., Liu, J., Yao, Z., & Ma, C. (2024). Measuring digitalization capabilities using machine learning. Research in International Business and Finance, 70, 102380. https://doi.org/10.1016/j.ribaf.2024.102380.

49. Zhang, Y., DeCarlo, T. E., Manikas, A. S., & Bhattacharya, A. (2023). To exploit or explore? The impact of crowdfunding project descriptions and backers’ power states on funding decisions. Journal of the Academy of Marketing Science, 51(2), 444–462. https://doi.org/10.1007/s11747-022-00871-w.

50. Zollo, M., & Winter, S. G. (2002). Deliberate Learning and the Evolution of Dynamic Capabilities. Organization Science, 13(3), 339–351. https://doi.org/10.1287/orsc.13.3.339.2780.