Skip to main content Accessibility help
Hostname: page-component-747cfc64b6-xl4lj Total loading time: 0.26 Render date: 2021-06-13T18:57:32.144Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "metricsAbstractViews": false, "figures": true, "newCiteModal": false, "newCitedByModal": true, "newEcommerce": true }

Extracting and Analysing Design Process Data from Log Files of ICT Supported Co-Creative Sessions

Published online by Cambridge University Press:  26 July 2019

Niccolo' Becattini
Politecnico di Milano;
Gaetano Cascini
Politecnico di Milano;
Jamie Alexander O'Hare
University of Bath;
Federico Morosi
Politecnico di Milano;
Jean-Francois Boujut
Grenoble Institute of Technology


The observation of designers' behaviour in collaborative design activities and the analysis of protocols improved the understanding of how novel ideas emerge, what occurs among designers and, indirectly, what methods have a good impact on the outcomes. Yet, protocol analysis requires recording the design sessions, often in a simulated environment, thus introducing a bias in the observation. Moreover, the analysis takes up to 1000 times the duration of the observed design session. These limitations definitely hinder the scalability of this practice to large experiments in real operational environments.

This paper investigates the possibility to use the data collected in log files, automatically recorded during collaborative design sessions assisted by an ICT design support tool, as a means to extract relevant information about the design process and ultimately to infer insights about co-designers' cognition during the session. In this perspective, the paper proposes a set of metrics tailored to an Augmented Reality-based collaborative design tool. The study has been carried about by processing the data collected in 5 real case studies conducted in three different design companies.

Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (, which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
© The Author(s) 2019


Bannert, M., Sonnenberg, C., Mengelkamp, C. and Pieger, E. (2015), “Short-and long-term effects of students’ self-directed metacognitive prompts on navigation behavior and learning performance”. Computers in Human Behavior, Vol. 52, pp. 293306.CrossRefGoogle Scholar
Becattini, N., Cascini, G., O’Hare, J.A. and Masclet, C. (2018), “Coding schemes for the analysis of ICT supported co-creative design sessions”. In DS92: Proceedings of the DESIGN 2018 15th International Design Conference (pp. 533544).CrossRefGoogle Scholar
Becattini, N., Cascini, G. and Rotini, F. (2015), “An OTSM-TRIZ based framework towards the computer-aided identification of cognitive processes in design protocols”. In Design Computing and Cognition’14 (pp. 99117). Springer, Cham.CrossRefGoogle Scholar
Becattini, N., Borgianni, Y., Cascini, G. and Rotini, F. (2012), “Experiencing protocol analysis of computer-guided design tasks”. In DS 70: Proceedings of DESIGN 2012, the 12th International Design Conference, Dubrovnik, Croatia (pp. 18211830).Google Scholar
Bouabid, A., Lepreux, S. and Kolski, C. (2018), “Study on generic tangible objects used to collaborate remotely on RFID tabletops”. Journal on Multimodal User Interfaces, pp. 120.Google Scholar
Blessing, L. T. and Chakrabarti, A. (2009). DRM, a design research methodology. Springer Science & Business Media.CrossRefGoogle Scholar
Carmel, E., Crawford, S. and Chen, H. (1992), “Browsing in hypertext: A cognitive study.” IEEE transactions on Systems, Man, and Cybernetics, Vol. 22, No. 5, pp. 865884.CrossRefGoogle Scholar
Catledge, L.D. and Pitkow, J.E., 1995. Characterizing browsing behaviors on the world-wide web. Georgia Institute of Technology.CrossRefGoogle Scholar
Ferre, X., Villalba, E., Julio, H. and Zhu, H. (2017), September. “Extending Mobile App Analytics for Usability Test Logging”, In IFIP Conference on Human-Computer Interaction (pp. 114131). Springer, Cham.Google Scholar
Greiff, S., Wüstenberg, S. and Avvisati, F. (2015), “Computer-generated log-file analyses as a window into students’ minds? A showcase study based on the PISA 2012 assessment of problem solving.” Computers & Education, Vol. 91, pp. 92105.CrossRefGoogle Scholar
Jiang, H. and Yen, C.-C. (2009), “Protocol analysis in design research: a review”. In Rigor and Relevance in Design: IASDR 2009, Seoul, Korea October 18–22, Seoul, pp. 147156. International Association of Societies of Design Research.Google Scholar
Kan, J.W. and Gero, J.S. (2009), “Using the FBS ontology to capture semantic design information in design protocol studies”. In About: Designing. Analysing Design Meetings (pp. 213229). CRC Press.Google Scholar
Kafai, Y. and Fields, D. (2012), “Connecting play: Understanding multimodal participation in virtual worlds”. In Proceedings of the 14th ACM international conference on Multimodal interaction (pp. 265–272). ACM.CrossRefGoogle Scholar
Kornmann, J., Kammerer, Y., Anjewierden, A., Zettler, I., Trautwein, U. and Gerjets, P. (2016), “How children navigate a multiperspective hypermedia environment: The role of spatial working memory capacity”. Computers in human behavior, Vol. 55, pp. 145158.CrossRefGoogle Scholar
Krieter, P. and Breiter, A. (2018), “Analyzing mobile application usage: generating log files from mobile screen recordings”. In Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services (p. 9). ACM.CrossRefGoogle Scholar
Landauer, M., Wurzenberger, M., Skopik, F., Settanni, G. and Filzmoser, P. (2018), “Dynamic log file analysis: An unsupervised cluster evolution approach for anomaly detection, computers & security, 79, pp. 94116.Google Scholar
Lee, G., Eastman, C.M. and Zimring, C. (2003), “Avoiding design errors: a case study of redesigning an architectural studio”. Design Studies, Vol. 24 No. 5, pp. 411435.CrossRefGoogle Scholar
Lee, Y., 2018. “Using Self-Organizing Map and Clustering to Investigate Problem-Solving Patterns in the Massive Open Online Course: An Exploratory Study”. Journal of Educational Computing Research, p. 0735633117753364.Google Scholar
Patil, D.N. and Meshram, B.B. (2019), “Web Browser Analysis for Detecting User Activities”. In Recent Findings in Intelligent Computing Techniques (pp. 279291). Springer, Singapore.CrossRefGoogle Scholar
Peruzzini, M., Grandi, F. and Pellicciari, M. (2018), “How to analyse the workers’ experience in integrated product-process design”. Journal of Industrial Information Integration, Vol. 12, pp. 3146.CrossRefGoogle Scholar
Rahimian, F.P. and Ibrahim, R. (2011), “Impacts of VR 3D sketching on novice designers’ spatial cognition in collaborative conceptual architectural design”. Design Studies, Vol. 32, No. 3, pp. 255291.CrossRefGoogle Scholar
Saliminamin, S., Becattini, N. and Cascini, G. (2018), “Sources of creativity stimulation for designing the next generation of technical systems: correlations with R&D designers’ performance”. Research in Engineering Design, pp. 121.Google Scholar
Someren, M. V., Barnard, Y. F. and Sandberg, J. A. ((1994), The think aloud method: a practical approach to modelling cognitive processes. Academic Press.Google Scholar
Schulz, A.H. and Breiter, A., 2012, August. “Monitoring user patterns in school information systems using logfile analysis”. In IFIP Conference on Information Technology in Educational Management (pp. 94103). Springer, Berlin, Heidelberg.Google Scholar
Snider, C., Škec, S., Gopsill, J.A. and Hicks, B.J. (2017), The characterisation of engineering activity through email communication and content dynamics, for support of engineering project management. Design Science, 3.CrossRefGoogle Scholar
Theobald, M., Bellhäuser, H. and Imhof, M. (2018), “Identifying individual differences using log-file analysis: Distributed learning as mediator between conscientiousness and exam grades”. Learning and Individual Differences, Vol. 65, pp. 112122.CrossRefGoogle Scholar
You have Access
Open access
Cited by

Send article to Kindle

To send this article to your Kindle, first ensure is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

Note you can select to send to either the or variations. ‘’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Extracting and Analysing Design Process Data from Log Files of ICT Supported Co-Creative Sessions
Available formats

Send article to Dropbox

To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

Extracting and Analysing Design Process Data from Log Files of ICT Supported Co-Creative Sessions
Available formats

Send article to Google Drive

To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

Extracting and Analysing Design Process Data from Log Files of ICT Supported Co-Creative Sessions
Available formats

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *