A fast template-based approach to automatically identify primary text content of a web page

Dat Quoc Nguyen; Dai Quoc Nguyen; Son Bao Pham; The Duy Bui

doi:10.1109/KSE.2009.39

A fast template-based approach to automatically identify primary text content of a web page

Dat Quoc Nguyen, Dai Quoc Nguyen, Son Bao Pham, The Duy Bui

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

7 Citations (Scopus)

Abstract

Search engines have become an indispensable tool for browsing information on the Internet. The user, however, is often annoyed by redundant results from irrelevant web pages. One reason is because search engines also look at non-informative blocks of web pages such as advertisement, navigation links, etc. In this paper, we propose a fast algorithm called FastContentExtractor to automatically detect main content blocks in a web page by improving the ContentExtractor algorithm. By automatically identifying and storing templates representing the structure of content blocks in a website, content blocks of a new web page from the website can be extracted quickly. The hierarchical order of the output blocks is also maintained which guarantees that the extracted content blocks are in the same order as the original ones.

Original language	English
Title of host publication	KSE 2009 - The 1st International Conference on Knowledge and Systems Engineering
Place of Publication	Washington, DC
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
Pages	232-236
Number of pages	5
ISBN (Print)	9780769538464
DOIs	https://doi.org/10.1109/KSE.2009.39
Publication status	Published - 2009
Externally published	Yes
Event	1st International Conference on Knowledge and Systems Engineering, KSE 2009 - Hanoi, Viet Nam Duration: 13 Oct 2009 → 17 Oct 2009

Other

Other	1st International Conference on Knowledge and Systems Engineering, KSE 2009
Country/Territory	Viet Nam
City	Hanoi
Period	13/10/09 → 17/10/09

Keywords

Data mining
Template detection
Web mining

Access to Document

10.1109/KSE.2009.39

Cite this

@inproceedings{27cfa1bc749a400b8a63a7ecb5872705,

title = "A fast template-based approach to automatically identify primary text content of a web page",

abstract = "Search engines have become an indispensable tool for browsing information on the Internet. The user, however, is often annoyed by redundant results from irrelevant web pages. One reason is because search engines also look at non-informative blocks of web pages such as advertisement, navigation links, etc. In this paper, we propose a fast algorithm called FastContentExtractor to automatically detect main content blocks in a web page by improving the ContentExtractor algorithm. By automatically identifying and storing templates representing the structure of content blocks in a website, content blocks of a new web page from the website can be extracted quickly. The hierarchical order of the output blocks is also maintained which guarantees that the extracted content blocks are in the same order as the original ones.",

keywords = "Data mining, Template detection, Web mining",

author = "Nguyen, {Dat Quoc} and Nguyen, {Dai Quoc} and Pham, {Son Bao} and Bui, {The Duy}",

year = "2009",

doi = "10.1109/KSE.2009.39",

language = "English",

isbn = "9780769538464",

pages = "232--236",

booktitle = "KSE 2009 - The 1st International Conference on Knowledge and Systems Engineering",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

address = "United States",

note = "1st International Conference on Knowledge and Systems Engineering, KSE 2009 ; Conference date: 13-10-2009 Through 17-10-2009",

}

Nguyen, DQ, Nguyen, DQ, Pham, SB & Bui, TD 2009, A fast template-based approach to automatically identify primary text content of a web page. in KSE 2009 - The 1st International Conference on Knowledge and Systems Engineering., 5361702, Institute of Electrical and Electronics Engineers (IEEE), Washington, DC, pp. 232-236, 1st International Conference on Knowledge and Systems Engineering, KSE 2009, Hanoi, Viet Nam, 13/10/09. https://doi.org/10.1109/KSE.2009.39

A fast template-based approach to automatically identify primary text content of a web page. / Nguyen, Dat Quoc; Nguyen, Dai Quoc; Pham, Son Bao et al.
KSE 2009 - The 1st International Conference on Knowledge and Systems Engineering. Washington, DC: Institute of Electrical and Electronics Engineers (IEEE), 2009. p. 232-236 5361702.

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

TY - GEN

T1 - A fast template-based approach to automatically identify primary text content of a web page

AU - Nguyen, Dat Quoc

AU - Nguyen, Dai Quoc

AU - Pham, Son Bao

AU - Bui, The Duy

PY - 2009

Y1 - 2009

N2 - Search engines have become an indispensable tool for browsing information on the Internet. The user, however, is often annoyed by redundant results from irrelevant web pages. One reason is because search engines also look at non-informative blocks of web pages such as advertisement, navigation links, etc. In this paper, we propose a fast algorithm called FastContentExtractor to automatically detect main content blocks in a web page by improving the ContentExtractor algorithm. By automatically identifying and storing templates representing the structure of content blocks in a website, content blocks of a new web page from the website can be extracted quickly. The hierarchical order of the output blocks is also maintained which guarantees that the extracted content blocks are in the same order as the original ones.

AB - Search engines have become an indispensable tool for browsing information on the Internet. The user, however, is often annoyed by redundant results from irrelevant web pages. One reason is because search engines also look at non-informative blocks of web pages such as advertisement, navigation links, etc. In this paper, we propose a fast algorithm called FastContentExtractor to automatically detect main content blocks in a web page by improving the ContentExtractor algorithm. By automatically identifying and storing templates representing the structure of content blocks in a website, content blocks of a new web page from the website can be extracted quickly. The hierarchical order of the output blocks is also maintained which guarantees that the extracted content blocks are in the same order as the original ones.

KW - Data mining

KW - Template detection

KW - Web mining

UR - http://www.scopus.com/inward/record.url?scp=77951456974&partnerID=8YFLogxK

U2 - 10.1109/KSE.2009.39

DO - 10.1109/KSE.2009.39

M3 - Conference proceeding contribution

AN - SCOPUS:77951456974

SN - 9780769538464

SP - 232

EP - 236

BT - KSE 2009 - The 1st International Conference on Knowledge and Systems Engineering

PB - Institute of Electrical and Electronics Engineers (IEEE)

CY - Washington, DC

T2 - 1st International Conference on Knowledge and Systems Engineering, KSE 2009

Y2 - 13 October 2009 through 17 October 2009

ER -

A fast template-based approach to automatically identify primary text content of a web page

Abstract

Other

Keywords

Access to Document

Other files and links

Fingerprint

Cite this