Automatic new topic identification using multiple linear regression

Ozmutlu S.

INFORMATION PROCESSING & MANAGEMENT, vol.42, no.4, pp.934-950, 2006 (SCI-Expanded) identifier identifier

  • Publication Type: Article / Article
  • Volume: 42 Issue: 4
  • Publication Date: 2006
  • Doi Number: 10.1016/j.ipm.2005.10.002
  • Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Social Sciences Citation Index (SSCI), Scopus
  • Page Numbers: pp.934-950
  • Bursa Uludag University Affiliated: Yes


The purpose of this study is to provide automatic new topic identification of search engine query logs, and estimate the effect of statistical characteristics of search engine queries on new topic identification. By applying multiple linear regression and multi-factor ANOVA on a sample data log from the Excite search engine, we demonstrated that the statistical characteristics of Web search queries, such as time interval, search pattern and position of a query in a user session, are effective on shifting to a new topic. Multiple linear regression is also a successful tool for estimating topic shifts and continuations. The findings of this study provide statistical proof for the relationship between the non-semantic characteristics of Web search queries and the occurrence of topic shifts and continuations. (c) 2005 Elsevier Ltd. All rights reserved.