Comparative analysis of term distributions in a sentence and in a document for sentence retrieval

Kyoung Soo Han; Hae Chang Rim

doi:10.1007/978-3-540-24630-5_59

Comparative analysis of term distributions in a sentence and in a document for sentence retrieval

Kyoung Soo Han, Hae Chang Rim

* Honorary professors

Research output: Chapter in Book/Report/Conference proceeding › Chapter

Abstract

Most of previous works of finding relevant sentences applied document retrieval models to sentence retrieval. However, the performance was very poor. This paper analyzes the reason of this poor performance by comparing term statistics in a document with those in a sentence. The analysis shows that the distribution of within-document and within-sentence term frequency is not similar, and the distribution of document frequency is similar to that of sentence frequency. Considering the discrepancy between the term statistics, it is not appropriate that document retrieval models, as they stand, are applied to sentence retrieval.

Original language	English
Title of host publication	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Editors	Alexander Gelbukh
Publisher	Springer Verlag
Pages	484-487
Number of pages	4
ISBN (Print)	3540210067, 9783540210061
DOIs	https://doi.org/10.1007/978-3-540-24630-5_59
Publication status	Published - 2004

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	2945
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

ASJC Scopus subject areas

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/978-3-540-24630-5_59

Cite this

Han, K. S., & Rim, H. C. (2004). Comparative analysis of term distributions in a sentence and in a document for sentence retrieval. In A. Gelbukh (Ed.), Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (pp. 484-487). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 2945). Springer Verlag. https://doi.org/10.1007/978-3-540-24630-5_59

Comparative analysis of term distributions in a sentence and in a document for sentence retrieval. / Han, Kyoung Soo; Rim, Hae Chang.
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). ed. / Alexander Gelbukh. Springer Verlag, 2004. p. 484-487 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 2945).

Research output: Chapter in Book/Report/Conference proceeding › Chapter

Han, KS & Rim, HC 2004, Comparative analysis of term distributions in a sentence and in a document for sentence retrieval. in A Gelbukh (ed.), Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 2945, Springer Verlag, pp. 484-487. https://doi.org/10.1007/978-3-540-24630-5_59

Han KS, Rim HC. Comparative analysis of term distributions in a sentence and in a document for sentence retrieval. In Gelbukh A, editor, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer Verlag. 2004. p. 484-487. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-540-24630-5_59

Han, Kyoung Soo ; Rim, Hae Chang. / Comparative analysis of term distributions in a sentence and in a document for sentence retrieval. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). editor / Alexander Gelbukh. Springer Verlag, 2004. pp. 484-487 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inbook{0ef4766e1e6f44b7973f579cbc328b0a,

title = "Comparative analysis of term distributions in a sentence and in a document for sentence retrieval",

abstract = "Most of previous works of finding relevant sentences applied document retrieval models to sentence retrieval. However, the performance was very poor. This paper analyzes the reason of this poor performance by comparing term statistics in a document with those in a sentence. The analysis shows that the distribution of within-document and within-sentence term frequency is not similar, and the distribution of document frequency is similar to that of sentence frequency. Considering the discrepancy between the term statistics, it is not appropriate that document retrieval models, as they stand, are applied to sentence retrieval.",

author = "Han, {Kyoung Soo} and Rim, {Hae Chang}",

year = "2004",

doi = "10.1007/978-3-540-24630-5_59",

language = "English",

isbn = "3540210067",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "484--487",

editor = "Alexander Gelbukh",

booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

}

TY - CHAP

T1 - Comparative analysis of term distributions in a sentence and in a document for sentence retrieval

AU - Han, Kyoung Soo

AU - Rim, Hae Chang

PY - 2004

Y1 - 2004

N2 - Most of previous works of finding relevant sentences applied document retrieval models to sentence retrieval. However, the performance was very poor. This paper analyzes the reason of this poor performance by comparing term statistics in a document with those in a sentence. The analysis shows that the distribution of within-document and within-sentence term frequency is not similar, and the distribution of document frequency is similar to that of sentence frequency. Considering the discrepancy between the term statistics, it is not appropriate that document retrieval models, as they stand, are applied to sentence retrieval.

AB - Most of previous works of finding relevant sentences applied document retrieval models to sentence retrieval. However, the performance was very poor. This paper analyzes the reason of this poor performance by comparing term statistics in a document with those in a sentence. The analysis shows that the distribution of within-document and within-sentence term frequency is not similar, and the distribution of document frequency is similar to that of sentence frequency. Considering the discrepancy between the term statistics, it is not appropriate that document retrieval models, as they stand, are applied to sentence retrieval.

UR - http://www.scopus.com/inward/record.url?scp=35048828543&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=35048828543&partnerID=8YFLogxK

U2 - 10.1007/978-3-540-24630-5_59

DO - 10.1007/978-3-540-24630-5_59

M3 - Chapter

AN - SCOPUS:35048828543

SN - 3540210067

SN - 9783540210061

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 484

EP - 487

BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

A2 - Gelbukh, Alexander

PB - Springer Verlag

ER -

Comparative analysis of term distributions in a sentence and in a document for sentence retrieval

Abstract

Publication series

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this