Skip to main content Accessibility help
×
Home
Hostname: page-component-684899dbb8-xt4p2 Total loading time: 0.492 Render date: 2022-05-28T23:32:30.386Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "useRatesEcommerce": false, "useNewApi": true }

On the subjectivity of human-authored summaries*

Published online by Cambridge University Press:  01 April 2009

BALAKRISHNA KOLLURU
Affiliation:
Department of Computer Science, University of Sheffield, Sheffield S1 4DP, United Kingdom e-mail: b.kolluru@dcs.shef.ac.uk, y.gotoh@dcs.shef.ac.uk
YOSHIHIKO GOTOH
Affiliation:
Department of Computer Science, University of Sheffield, Sheffield S1 4DP, United Kingdom e-mail: b.kolluru@dcs.shef.ac.uk, y.gotoh@dcs.shef.ac.uk

Abstract

Human-generated summaries are a blend of content and style, bound by the task restrictions, but are ‘subject to subjectiveness’ of the individuals summarising the documents. We study the impact of various facets that cause subjectivity such as brevity, information content and information coverage on human-authored summaries. The scale of subjectivity is quantitatively measured among various summaries using a question–answer-based cross-comprehension test. The test evaluates summaries for meaning rather than exact words based on questions, framed by the summary authors, derived from the summary. The number of questions that cannot be answered after reading the candidate summary reflects its subjectivity. The qualitative analysis of the outcome of the cross-comprehension test shows the relationship between the length of a summary, information content and nature of questions framed by the summary author.

Type
Papers
Copyright
Copyright © Cambridge University Press 2008

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Boret, M. and Peyrot, J. 1969. Le Résumé de Texte: Épreuve d'une Pédagogie Nouvelle. Paris: Chotard et Associés.Google Scholar
Cieri, C., Graff, D., Liberman, M., Martey, N. and Strassel, S. 1999. The TDT-2 text and speech corpus. In Proceedings of DARPA Broadcast News Workshop, Herndon, VA, pp. 57–60.Google Scholar
Cohen, J. 1968. A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20: 3746.CrossRefGoogle Scholar
Cremmins, E. T. 1982. The Art of Abstracting. Philadelphia, PA: ISI Press.Google Scholar
Dang, H. and Lin, J. 2007. Different structures for evaluating answers to complex questions: pyramids won't topple, and neither will human assessors. In Proceedings of the ACL 2007, Prague, pp. 768–775.Google Scholar
Endres-Niggemeyer, B. and Neugebauer, E. 1998. Professional summarizing: no cognitive simulation without observation. Journal of the American Society for Information Science 49 (6): 486506.3.0.CO;2-Y>CrossRefGoogle Scholar
Hirschmann, L., Burger, J., Palmer, D. and Robinson, P. 1999. Evaluating content extraction from audio source. In Proceedings of the ESCA Workshop: Accessing Information in Spoken Audio, Cambridge, UK, pp. 54–59.Google Scholar
Hovy, E., Lin, C.-Y., Zhou, L. and Fukumoto, J. 2006. Automatic summarization evaluation with basic elements. In Proceedings of the 5th International Conference on Language Resources and Evaluation (LREC 2006), Genoa.Google Scholar
Jing, H. 2002. Using hidden Markov modeling to decompose human-written summaries. Computational Linguistics 28 (4): 527543.CrossRefGoogle Scholar
Kintsch, W. and van Dijk, T. A. 1978. Toward a model of text comprehension and production. Psychological Review 85 (5): 363394.CrossRefGoogle Scholar
Kolluru, B., Christensen, H. and Gotoh, Y. 2005. Multi-stage compaction approach to broadcast news summarisation. In Proceedings of Interspeech 2005, Lisbon, pp. 69–72.Google Scholar
Kolluru, B. and Gotoh, Y. 2007. Relative evaluation of informativeness in machine generated summaries. In Proceedings of Interspeech 2007, Antwerp, pp. 1338–1341.Google Scholar
Lin, C.-Y. and Hovy, E. 2003. The potential and limitations of automatic sentence extraction for summarization. In Proceedings of the HLT-NAACL 2003 Workshop on Automatic Summarization, Edmonton, pp. 73–80.Google Scholar
Lin, J. and Demner-Fushman, D. 2006. Will Pyramids built of nuggets topple over? In Proceedings of HLT/NAACL 2006, New York, pp. 383–90.Google Scholar
Mani, I. 2001. Automatic Summarization. Jon Benjamins Publishing Company, Amsterdam.CrossRefGoogle Scholar
Nenkova, A. and Passonneau, R. 2004. Evaluating content selection in summarization: the pyramid method. In Proceedings of the HLT-NAACL 2004, Boston, pp. 145–152.Google Scholar
Nenkova, A., Passonneau, R. and McKeown, K. 2007. The Pyramid method: incorporating human content selection variation in summarization evaluation. ACM Transactions on Speech and Language Processing 4 (2), article 4.CrossRefGoogle Scholar
PintoMolina, M. Molina, M. 1995. Documentary abstracting: toward a methodological model. Journal of the American Society for Information Science 46 (3): 225234.Google Scholar
Radev, D. R., Jing, H., Styś, M. and Tam, D. 2004. Centroid-based summarization of multiple documents. Information Processing and Management 40, 919938.CrossRefGoogle Scholar
Rowley, J. E. 1988. Abstracting and Indexing, 2nd ed.London: Bingley.Google Scholar
Teufel, S. and van Halteren, H. 2004. Evaluating information content by factoid analysis: human annotation and stability. In Proceedings of the EMNLP 2004, Barcelona, pp. 419–426.Google Scholar
Van Halteren, H. and Teufel, S. 2003. Examining the consensus between human summaries: initial experiments with factoid analysis. In Proceedings of the HLT-NAACL 2003 Workshop on Automatic Summarization, Edmonton, pp. 57–64.Google Scholar
Vanderwende, L., Banko, M. and Menezes, A. 2004. Event-centric summary generation. In Proceedings of DUC 2004, Boston, pp. 76–81.Google Scholar

Save article to Kindle

To save this article to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

On the subjectivity of human-authored summaries*
Available formats
×

Save article to Dropbox

To save this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Dropbox account. Find out more about saving content to Dropbox.

On the subjectivity of human-authored summaries*
Available formats
×

Save article to Google Drive

To save this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Google Drive account. Find out more about saving content to Google Drive.

On the subjectivity of human-authored summaries*
Available formats
×
×

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *