In order to prevent medical accidents related to drugs, not only developing safe drugs but also their proper use is important. In this paper, we proposed the method to measure look-alikeness between not only backside designs but also front side designs. Since the major difference between backside designs and front sides is the existence of tables or capsules, we applied image inpainting technique to convert the original PTP front side images to “tabletless” PTP sheet images. In order to employ inpainting technique, a neural network was used to obtain a mask image which masks tablet/capsule parts. To the resultant images, the similarity index calculation was applied as proposed in our previous study.