Footwear has become fashion merchandise with a need of personalized design. Previous studies realized the idea of virtual footwear try-on in augmented reality by implementing its functional prototypes. The use of depth camera imposes limitations on the practical value of the virtual try-on. New functions are developed to generate the intelligences for 3D recognition and tracking of human foot from one single color image using deep learning techniques. A convolution neural network model is constructed from training data of two different sources: synthesized and real data. The model learns to estimate the 3D pose of human foot existing in a color image with random background. The developed functions eliminate the lengthy calculation required by 3D registration of depth data. They show a reduced sensitivity to the lighting condition in the use environment. The technology works with regular RGB camera of a smart phone or tablet. It enables both smart e-commerce and mobile commerce.

This new technology overcomes the major limitations of previous virtual try-on functions. A convolution neural network model (CNN) was trained to recognize and track human foot in a color image without the use of depth camera. We also constructed the world’s first training database that provides annotated data for 3D pose estimation of human foot. There was no previous literature that realized the same pose estimation from one single color image. A previous AI team in Google has launched an APP for commerce use of virtual shoe try-on, but the technical details are not revealed.

The technology of virtual try-on realizes the business concept of online to offline (O2O) retail by creating novel shopping experience in augmented reality. It allows customers to evaluate how a personalized product matches with each individual, without any location restrictions, and to quickly access a large amount of designs. Our new technology can be deployed in a regular mobile device or smart phone. It supports smart e-commerce, mobile commerce, and interaction device in real environment by working as an effective tool for product display and marketing.