Measuring Repeatability across multiple images with code

This was posted on 2023-05-12


Jump to Output Files: ImageRepeatabilityCalculator/dist at master · iamJohnnySam/ImageRepeatabilityCalculator (github.com)

One of the recent challenges that I faced is the measurement of repeatability data from a series of images. The requirement was to measure and validate that a set of images taken of an object from a fixed camera facing down has positional repeatability of +/- 0.1mm in the X and Y directions.

To take images, the camera (Basler) used had a resolution of 12.3 MP (4096 px x 3000 px) with a field of view of 2.355 x 1.725mm. Since the requirement of 0.1mm measurement will fall within ~174 pixels of each photo, the accuracy of the detecting algorithm can be a little relaxed.

I also used a second camera (Cognex) for low repeatability figures which had a FoV of 31.127mm x 23.346mm with an output image resolution of 1024px x 768px.

Using OpenCV Image Processing

One of the methods for this that was used widely on the internet was template matching using OpenCV and python. Using this algorithm, I was able to develop a simple application which locates a template inside another image and calculated the X & Y distance to the center of the localized area. However, there are a number of challenges when selecting the template as the template must have a unique design which is both scale and rotation invariant. The template must only match one location in any image and cannot match anywhere else at a different scale, position or rotation.

template = cv.imread(os.path.join(location,'template.bmp'))

# Check with First image and check best match algorithm
img_orig = cv.imread(os.path.join(location,'Images/1.bmp'))
w = template.shape[0]
h = template.shape[1]

img2 = img_orig.copy()
# All the 6 methods for comparison in a list
methods = ['cv.TM_CCOEFF', 'cv.TM_CCOEFF_NORMED', 'cv.TM_CCORR',
            'cv.TM_CCORR_NORMED', 'cv.TM_SQDIFF', 'cv.TM_SQDIFF_NORMED']
for meth in methods:
    img = img2.copy()
    method = eval(meth)
    # Apply template Matching
    res = cv.matchTemplate(img,template,method)
    min_val, max_val, min_loc, max_loc = cv.minMaxLoc(res)
    # If the method is TM_SQDIFF or TM_SQDIFF_NORMED, take minimum
    if method in [cv.TM_SQDIFF, cv.TM_SQDIFF_NORMED]:
        top_left = min_loc
    else:
        top_left = max_loc
    bottom_right = (top_left[0] + w, top_left[1] + h)
    cv.rectangle(img,top_left, bottom_right, 255, 2)
    plt.subplot(121),plt.imshow(res,cmap = 'gray')
    plt.title('Matching Result'), plt.xticks([]), plt.yticks([])
    plt.subplot(122),plt.imshow(img,cmap = 'gray')
    plt.title('Detected Point'), plt.xticks([]), plt.yticks([])
    plt.suptitle(meth)
    plt.show()

# Run the best selected algorithm for the first image to confirm.
img = img_orig.copy()
res = cv.matchTemplate(img, template, cv.TM_CCOEFF_NORMED)
threshold = 0.8
loc = np.where(res >= threshold)

x_orig = 0
y_orig = 0
i = 0

print(loc)
for pt in zip(*loc[::-1]):
    cv.rectangle(img, pt, (pt[0] + w, pt[1] + h), (0, 255, 255), 2)
    i += 1
    x_orig = x_orig + pt[0]
    y_orig = y_orig + pt[1]
  
plt.imshow(img, cmap='gray')


# Run for remaining images in the file directory

directory = os.path.join(location,'Images/')
x_arr = np.array([0])
y_arr = np.array([0])

for filename in os.listdir(directory):
    f = os.path.join(directory, filename)
    # checking if it is a file
    if os.path.isfile(f):
        img2 = cv.imread(f)
        res = cv.matchTemplate(img2, template, cv.TM_CCOEFF_NORMED)
        threshold = 0.8
        loc = np.where(res >= threshold)
        x = 0
        y = 0
        i = 0
        
        for pt in zip(*loc[::-1]):
            cv.rectangle(img2, pt, (pt[0] + w, pt[1] + h), (0, 255, 255), 2)
            i += 1
            x = x+pt[0]
            y = y+pt[1]

        if(i != 0):
            x = ((x/i) - x_orig)*2.355/4096   
            y = ((y/i) - y_orig)*1.725/3000

            if (x<0.1 and y<0.1) and (x> -0.1 and y> -0.1):
              x_arr = np.append(x_arr, x)
              y_arr = np.append(y_arr, y)
            else:
                print(f)

        print(filename+'  -  '+str(x)+', '+str(y))

fig1 = plt.figure()
fig1.set_figwidth(30)
fig1.set_figheight(15)
fig1 = plt.title('X and Y Offset')
fig1 = plt.plot(x_arr - np.mean(x_arr))
fig1 = plt.plot(y_arr - np.mean(y_arr))

Althogh this method worked without many problems, I ran in to a number of issues when selecting suitable templates. If the OpenCV algorithm is unable to find the exact location the variation in the data is unpredictable.

Manual Entry

Next going back to basics, I developed a quick solution which is able to display 1 image after the other to provide the user the option to lock on to one feature on the image and select the same feature across a number of images. Since the click accuracy might not be the best, it was better to select a smaller feautre where the mouse can be directed to, with the lease amount of devation.

import cv2 as cv
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.pyplot import figure
import os

# Set Directory here for Images
directory = 'location/'

x_arr = np.array([0])
y_arr = np.array([0])

  
def mouse_callback(event, x, y, flags, param):
    global x_arr
    global y_arr
    if event == cv.EVENT_LBUTTONUP:
        print('Mouse clicked at X:', x, 'Y:', y)
        x_arr = np.append(x_arr, x)
        y_arr = np.append(y_arr, y)

h = 1900
# Open Each Photo
for filename in os.listdir(directory):
    f = os.path.join(directory, filename)
    if os.path.isfile(f):
        img = cv.imread(f)
        dim = (int(h*img.shape[1]/img.shape[0]), h)
        img = cv.resize(img, dim, interpolation = cv.INTER_AREA)
        cv.namedWindow(f)
        cv.setWindowProperty(f, cv.WND_PROP_FULLSCREEN, cv.WINDOW_NORMAL)
        cv.setMouseCallback(f, mouse_callback)
        cv.imshow(f, img)
        cv.waitKey(0)
        cv.destroyAllWindows()
        
x_arr = np.delete(x_arr, 0) 
y_arr = np.delete(y_arr, 0)

imgWpx = 4096
imgHpx = 3000

imgWSpx = 4096 * h / imgHpx
imgHSpx = 3000

imgWmm = 2.355
imgHmm = 1.725

x_img = x_arr * imgWmm / imgWSpx
y_img = y_arr * imgHmm / imgHSpx       

x_set = x_img - np.mean(x_img)
y_set = y_img - np.mean(y_img)


print('Min X: ', np.min(x_img), ' | Max X: ', np.max(x_img),  ' | Pk-Pk X: ', np.max(x_img)-np.min(x_img))
print('Min Y: ', np.min(y_img), ' | Max Y: ', np.max(y_img),  ' | Pk-Pk Y: ', np.max(y_img)-np.min(y_img))
 

#%% Cpk Calculation
USL = 0.15
LSL = -0.15

CpiX = (np.mean(x_set) - LSL)/(3*np.std(x_set))
CpuX = (USL - np.mean(x_set))/(3*np.std(x_set))
print('Cpk in X:', min(CpiX, CpuX))

CpiY = (np.mean(y_set) - LSL)/(3*np.std(y_set))
CpuY = (USL - np.mean(y_set))/(3*np.std(y_set))
print('Cpk in Y:', min(CpiY, CpuY))

#%% Figure
fig1 = plt.figure() 
fig1.set_figwidth(30)
fig1.set_figheight(15)
fig1 = plt.title('X and Y Offset')
fig1 = plt.plot(x_set, label='X Data')
fig1 = plt.plot(y_set, label='Y Data')
fig1 = plt.axhline(y=USL, color='r', linestyle='-', label='Control Limits')
fig1 = plt.axhline(y=LSL, color='r', linestyle='-')

fig1 = plt.axhline(y=3*np.std(x_set), color='m', linestyle='--', label='3 sigma for X')
fig1 = plt.axhline(y=-3*np.std(x_set), color='m', linestyle='--', label='3 sigma for X')
fig1 = plt.axhline(y=3*np.std(y_set), color='c', linestyle='--', label='3 sigma for Y')
fig1 = plt.axhline(y=-3*np.std(y_set), color='c', linestyle='--', label='3 sigma for Y')

plt.title('Repeatability data for '+directory.replace('/', ''))
fig1 = plt.xlabel('Iterations')
fig1 = plt.ylabel('Devation in mm')
plt.legend(loc='upper right')
fig1 = plt.text(1, USL, 'Cpk in X: ' + str(min(CpiX, CpuX)), fontsize = 12)
fig1 = plt.text(1, USL-0.01, 'Cpk in Y: ' + str(min(CpiY, CpuY)), fontsize = 12)

plt.savefig(directory.replace('/', '')+'.png', dpi=100)
fig1 = plt.show()

Similar Posts

See other projects and posts in the same category as this post

blog item Robot Layout Simulator
A simple tool to determine the output of your layout
python 2024-02-10 | Read More....
blog item Measuring Repeatability across multiple images with code
repeatable, measurement 2023-05-12 | Read More....
Comment Box is loading comments...