An RFID and computer vision fusion system for book inventory using mobile robot

Jiuwu Zhang, Xiulong Liu*, Tao Gu, Bojun Zhang, Dongdong Liu, Zijuan Liu, Keqiu Li

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionpeer-review

4 Citations (Scopus)


Mobile robot-assisted book inventory such as book identification and book order detection has become increasingly popular in smart library, replacing the manual book inventory which is time-consuming and error-prone. The existing systems are either computer vision (CV)-based or RFID-based, however several limitations are inevitable. CV-based systems may not be able to identify books effectively due to low accuracy of detecting texts on book spine. RFID tags attached to books can be used to identify a book uniquely. However, in high tag density scenarios such as library, tag coupling effects of adjacent tags may seriously affect the accuracy of tag reading. To overcome these limitations, this paper presents a novel RFID and CV fusion system for Book Inventory using mobile robot (RC-BI). RFID and CV are first used individually to obtain book order, then the information will be fused by the sequence based matching algorithm to remove ambiguity and improve overall accuracy. Specifically, we address three technical challenges. We design a deep neural network (DNN) model with multiple inputs and mixed data to filter out interference of RFID tags on other tiers, and propose a video information extracting schema to extract book spine information accurately, and use strong link to align and match RFID- and CV-based timestamp vs. book-name sequences to avoid errors during fusion. Extensive experiments indicate that our system achieves an average accuracy of 98.4% for tier filtering and an average accuracy of 98.9% for book order, significantly outperforming the state-of-the-arts.

Original languageEnglish
Title of host publicationIEEE INFOCOM 2022 - IEEE Conference on Computer Communications
Place of PublicationPiscataway, NJ
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Number of pages10
ISBN (Electronic)9781665458221
ISBN (Print)9781665458238
Publication statusPublished - 2022
Event41st IEEE Conference on Computer Communications, INFOCOM 2022 - Virtual, London, United Kingdom
Duration: 2 May 20225 May 2022

Publication series

ISSN (Print)0743-166X
ISSN (Electronic)2641-9874


Conference41st IEEE Conference on Computer Communications, INFOCOM 2022
Country/TerritoryUnited Kingdom
CityVirtual, London


Dive into the research topics of 'An RFID and computer vision fusion system for book inventory using mobile robot'. Together they form a unique fingerprint.

Cite this