Skip to main content Accessibility help
×
Home

Feature Engineering for Design Thinking Assessment

  • Ryan Arlitt (a1), Sumbul Khan (a2) and Lucienne Blessing (a2)

Abstract

As design and design thinking become increasingly important competencies for a modern workforce, the burden of assessing these fuzzy skills creates a scalability bottleneck. Toward addressing this need, this paper presents an exploratory study into a scalable computational approach for design thinking assessment. In this study, student responses to a variety of contextualized design questions – gathered both before and after participation in a design thinking training course – are analyzed. Specifically, a variety of text features are engineered, tested, and interpreted within a design thinking framework in order to identify specific markers of design thinking skill acquisition. Key findings of this work include identification of text features that may enable scalable measurement of (1) user-centric language and (2) design thinking concept acquisition. These results contribute toward the creation of computational tools to ease the burden of providing feedback about design thinking skills to a wide audience.

    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Feature Engineering for Design Thinking Assessment
      Available formats
      ×

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Feature Engineering for Design Thinking Assessment
      Available formats
      ×

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Feature Engineering for Design Thinking Assessment
      Available formats
      ×

Copyright

This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.

Corresponding author

Contact: Arlitt, Ryan Michael, Technical University of Denmark, Mechanical Engineering, Denmark, rmarl@mek.dtu.dk

References

Hide All
Abdi, H. and Williams, L.J. (2010), “Principal Component Analysis”, WIREs Comput. Stat., Vol. 2 No. 4, pp. 433459, John Wiley & Sons, Inc., New York, NY, USA.
Aflatoony, L., Wakkary, R. and Neustaedter, C. (2018), “Becoming a Design Thinker: Assessing the Learning Process of Students in a Secondary Level Design Thinking Course”, International Journal of Art & Design Education, Vol. 37 No. 3, pp. 438453, Wiley/Blackwell (10.1111).
Balfour, S.P. (2013), “Assessing Writing in MOOCs: Automated Essay Scoring and Calibrated Peer ReviewTM.”, Research & Practice in Assessment, ERIC, Vol. 8, pp. 4048.
Bartholomew, S.R. (2017), “Assessing open-ended design problems”, Technology and Engineering Teacher, Vol. 76 No. 6, pp. 1317.
Beyerlein, S., Davis, D., Trevisan, M., Harrison, K. and Thompson, L. (2006), “Assessment Framework for Capstone Design Courses”, Proceedings of American Society for Engineering Education Annual Conference.
Blei, D.M., Ng, A.Y. and Jordan, M.I. (2003), “Latent dirichlet allocation”, Journal of Machine Learning Research : JMLR, Vol. 3, pp. 9931022.
Blizzard, J., Klotz, L., Potvin, G., Hazari, Z., Cribbs, J. and Godwin, A. (2015), “Using survey questions to identify and learn more about those who exhibit design thinking traits”, Design Studies, Vol. 38, pp. 92110. Elsevier Ltd.
Camburn, B.A., Auernhammer, J.M., Sng, K.H.E., Mignone, P.J., Arlitt, R.M., Perez, K.B., Huang, Z., et al. (2017), “Design Innovation: A Study of Integrated Practice”, Volume 7: 29th International Conference on Design Theory and Methodology, pp. 110.
Carroll, M., Goldman, S., Britos, L., Koh, J., Royalty, A. and Hornstein, M. (2010), “Destination, imagination and the fires within: Design thinking in a middle school classroom”, International Journal of Art and Design Education, Available at: https://doi.org/10.1111/j.1476-8070.2010.01632.x
Christensen, K.S., Hjorth, M., Iversen, O.S. and Blikstein, P. (2016), “Towards a formal assessment of design literacy: Analyzing K-12 students’ stance towards inquiry”, Design Studies, Vol. 46, pp. 125151.
Cross, N. (2011), Design Thinking : Understanding How Designers Think and Work, Berg.
Davis, D.C., Gentili, K.L., Trevisan, M.S. and Calkins, D.E. (2002), “Engineering design assessment processes and scoring scales for program improvement and accountability”, Journal of Engineering Education, Vol. 91 No. 2, pp. 211221, Blackwell Publishing Ltd.
Deerwester, S., Dumais, S.T., Furnas, G.W., Landauer, T.K. and Harshman, R. (1990), “Indexing by latent semantic analysis”, Journal of the American Society for Information Science, Vol. 41 No. 6, pp. 391407, Wiley Online Library.
Demirkan, H. and Afacan, Y. (2012), “Assessing creativity in design education: Analysis of creativity factors in the first-year design studio”, Design Studies, Vol. 33 No. 3, pp. 262278, Elsevier Ltd.
DesignSingapore Council (2016), Design 2025.
Diefes-Dux, H.A., Zawojewski, J.S. and Hjalmarson, M.A. (2010), “Using Educational Research in the Design of Evaluation Tools for Open-Ended Problems”, International Journal of Engineering Education, Vol. 26 No. 4, pp. 807819, SI.
Domingos, P. (2012), “A few useful things to know about machine learning”, Communications of the ACM, ACM, Vol. 55 No. 10, pp. 7887.
Doppelt, Y. (2009), “Assessing creative thinking in design-based learning”, International Journal of Technology and Design Education, Vol. 19 No. 1, pp. 5565.
Jonassen, D., Strobel, J. and Lee, C.B. (2006), “Everyday problem solving in engineering: Lessons for engineering educators”, Journal of Engineering Education, Available at: https://doi.org/10.1002/j.2168-9830.2006.tb00885.x
Kelley, T. and Kelley, D. (2013), “Creative confidence: Unleashing the creative potential within us all”, The Business Source, Available at: https://doi.org/10.4103/0255-0857.93014
Kimbell, R. (2012), “Evolving project e-scape for national assessment”, International Journal of Technology and Design Education, Vol. 22 No. 2, pp. 135155, Springer Netherlands.
Kluyver, T., Ragan-Kelley, B., Pérez, F., Granger, B.E., Bussonnier, M., Frederic, J., Kelley, K., et al. (2016), “Jupyter Notebooks-a publishing format for reproducible computational workflows”, ELPUB, pp. 8790.
Le, Q.V. and Mikolov, T. (2014), “Distributed Representations of Sentences and Documents”, CoRR, abs/1405.4, Available at: http://arxiv.org/abs/1405.4053.
Mc Laughlin, G.H. (1969), “SMOG grading-a new readability formula”, Journal of Reading, Vol. 12 No. 8, pp. 639646, JSTOR.
McLaren, S.V. and Stables, K. (2008), “Exploring key discriminators of progression: relationships between attitude, meta-cognition and performance of novice designers at a time of transition”, Design Studies, Vol. 29 No. 2, pp. 181201.
Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S. and Dean, J. (2013), “Distributed representations of words and phrases and their compositionality”, Advances in Neural Information Processing Systems, pp. 31113119.
Miller, T. (2003), “Essay assessment with latent semantic analysis”, Journal of Educational Computing Research, Vol. 29 No. 4, pp. 495512, SAGE Publications Sage CA, Los Angeles, CA.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., et al. (2011), “Scikit-learn: Machine Learning in {P}ython”, Journal of Machine Learning Research, Vol. 12, pp. 28252830.
Řehůřek, R. and Sojka, P. (2010), “Software Framework for Topic Modelling with Large Corpora”, Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, ELRA, Valletta, Malta, pp. 4550.
Shermis, M.D. and Burstein, J. (2013), Handbook of Automated Essay Evaluation: Current Applications and New Directions, Routledge.
Sparck Jones, K. (1972), “A statistical interpretation of term specificity and its application in retrieval”, Journal of Documentation, MCB UP Ltd, Vol. 28 No. 1, pp. 1121.
SUTD-MIT International Design Centre (2018), Design Odyssey, Available at: https://idc.sutd.edu.sg/programmes/design-odyssey/.
“Textacy” (2018), Chartbeat Labs, Available at: https://github.com/chartbeat-labs/textacy.
Wrigley, C., Mosely, G. and Tomitsch, M. (2018), “Design Thinking Education: A Comparison of Massive Open Online Courses”, She Ji: The Journal of Design, Economics, and Innovation, Elsevier, Vol. 4 No. 3, pp. 275292.
Wrigley, C. and Straker, K. (2017), “Design Thinking pedagogy: the Educational Design Ladder”, Innovations in Education and Teaching International, Available at: https://doi.org/10.1080/14703297.2015.1108214

Keywords

Feature Engineering for Design Thinking Assessment

  • Ryan Arlitt (a1), Sumbul Khan (a2) and Lucienne Blessing (a2)

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed.