Automatic Topic Extraction from Research Articles Using N-gram Analysis

Abstract

Identifying the topic of an article can involve a lot of manual work. The manual processes can be exhaustive when it comes to a large volume of articles. In order to tackle this problem, we propose an automated topic extraction approach, which is able to extract topics for a large number of articles with a consideration to efficiency. To support the automatic topic extraction, our research focuses on existing N-gram analysis, which only calculates the words appearing frequency in a document. But in our research, we apply our customized filtering standards to improve the efficiency. And also to eliminate the irrelevant or noncritical phrases as many as possible. By doing that, we can make sure that our final selected keyphrases to each article are unique labels, which can represent the core idea of each specific article. In our case, we choose to focus on the research papers within the autonomous vehicle domain because the research papers are highly demanded in our daily life. Since most of the research papers are available only in PDF format, we need to process the PDF format files into the editable file types such as TXT. In order to realize the automation, we have selected a large number of autonomous vehicle-related articles to test our proposed idea. Then we observe the result and compare it with the manual topic extraction result to evaluate our approach.