Content Oriented Automatic Text Categorization

  IJETT-book-cover  International Journal of Engineering Trends and Technology (IJETT)          
© 2011 by IJETT Journal
Volume-1 Issue-1                          
Year of Publication : 2011
Authors :N.Balasundaraganapathy, P.Yuvarasu, Mr.Prabhakar


N.Balasundaraganapathy, P.Yuvarasu, Mr.Prabhakar. "Content Oriented Automatic Text Categorization". International Journal of Engineering Trends and Technology (IJETT),V1(1):128-130 May to June 2011. ISSN:2231-5381. Published by Seventh Sense Research Group.


The project is to implement a web spam classifier, which given a web page, will analyze its features and try to determine whether the page is spam or not. The efficiency of the classifier will be compared t o the results spam detection in the text datasets using Naïve Baye’s classifier text representation is the task of transforming the content of a textual document into a vector in the term space so that the document could be recognized and classified by a c omputer or a classifier. Different terms (i.e. words, phrases, or any other indexing units used to identify the contents of a text) have different importance in a text. The term weighting methods assign appropriate weights to the terms to improve the perfo rmance of text categorization. In this study, the investigate several widely - used unsupervised (traditional) and supervised term weighting methods on benchmark data collections in combination with NLP and Clustering algorithms. In consideration of the dist ribution of relevant documents in the collection, the propose a new simple supervised term weighting method, i.e. tf.rf, to improve the terms` discriminating power for text categorization task. a consistently better performance while other supervised term weighting methods based on information theory or statistical metric perform the worst in all experiments. On the other hand, the popularly used tf.idf method has not shown a uniformly good performance in terms of different data sets.


1] T. Joachims, “Text Categorization with Support Vector Machines: Learning with Many Relevant Features,” Proc. 10th European Conf.Machine Learning (ECML), pp. 137 - 142, 1998.
[2] A. A. Benczúr, K. Csalogány, T. Sarlós, and M. Uher, “Spamrank — Fully automatic link spam detection,” in Proc. First Int. Workshop on Adversarial Information Retrieval on the Web (AIRWeb , C hiba, Japan, 2005, pp. 25 – 38.
[3] C. Castillo, D. Donato, L. Becchetti, P. Boldi, S. Leonardi, M. Santini, and S. Vigna, “A reference collection for web spam,” SIGIR Forum ,vol. 40, no. 2, pp. 11 – 24, 2006.
[4] T. Joachims, Learning to Classify Text Using Su pport Vector Machines — Methods, Theory, and Algorithms. Kluwer/Springer, 2002.
[5] T. M. Cover and J. A. Thomas , Elements of Information Theory . New York: Wiley - Interscience, 1991
[6] K. Bharat and M. R. Henzinger, “Improved algorithms for topic distillatio n in a hyperlinked environment,” in Proc. 21st Annu. Int. ACM SIGIR Conf. Research and Development in Information Retrieval , New York, 1998, pp. 104 – 111, ACM