Hybrid Image

2 minute read


1. Project #1 - Text Classification

    1. Goal: Train a model to classifiy positive and negative movie reviews.
    1. Methods: In three different ways:
        1. perceptron algorithm

How to create hybrid image using image filtering

1. Main idea

    1. Hybrid image: an image with chaning perception based on distance.
    1. Mechanism: Higher frequency survives in longer distance.
    1. covariance matrix 协方差矩阵: if we examine N-dimensional samples (N rows), the Cij element in matrix will be the covariance of the row i and row j.
    1. covariance 协方差: a metric showing how “correlated” the two variables are, or the tendency in the linear relationship between the two. If the covx,y=0, the plot of points of the two vairable will has a shape of circle. If covx,y>0, if will go “direct proportional”.

3. Gaussian Filter 高斯滤波

    1. 1D Gaussian kernel 一维高斯核函数:
      • a. What: a (k, 1) vector
      • b. How:
    1. 2D Gaussian kernel 二维高斯核函数:
      • a. What: a (?, ?) matrix.
      • b. How: Multi-dimensional Gaussian functions are separable, that is they can be calculated as the product of the Gaussian function along all axes. For the 2-d case it means: terms[x, y] = terms[x]*terms[y]. Thus, we can create 2D kernel by axis = np.array(create_Gaussian_kernel_1D(k, sigma)) and kernel = axis * axis.T
    1. 2D convolution 二维卷积 ref
      • a. What:
      • b: How:
      • (1). Calculate padding to preserve dimension: In many cases, we will want to set ph=kh−1 and pw=kw−1 to give the input and output the same height and width

4. Frequency Filter - High pass vs low pass 高通滤波器 和 低通滤波器

    1. Low pass:
      • a. What: image with only the low frequencies
      • b. How: apply 2D convolution using the Gaussian kernel. Then do clipping of the image.
    1. High pass:
      • a. What: image with only the high frequencies
      • b. How: image - low_pass. Then do clipping of the image.
    1. Hybrid image:
      • a. What: image with both low pass and high pass of two different sources.
      • b. How: hybrid = low_pass + high_pass. Then do the clipping.
    1. Clipping:
      • a. What: normalize each pixel value to range 0 to 1.
      • b. How:
      • (1) np.clip(image, 0, 1)
      • (2) hybrid_img_min = hybrid_image_unclip.min() hybrid_img_max = hybrid_image_unclip.max() hybrid_image = (hybrid_image_unclip - hybrid_img_min) / (hybrid_img_max - hybrid_img_min).
    1. Cutoff frequency:
      • a. What is the range?
      • b. How is it related to standard deviation: (1) narrower Gaussian = higher cutoff (2) wider Gaussian = lower cutoff

4. Pytorch

    1. Convert 2D kernel into Pytorch kernels with channels:
      • a. What: from (k, k) with float to (c, 1, k, k) with tensor.
      • b. How:
      • (1). k1 = np.reshape(k1, (1, 1, kernel.shape[0], kernel.shape[1]))
      • (2). k2 = np.tile(k1, (self.n_channels, 1, 1, 1)) # repeat n_channels of times on the axis dim of 0
      • (3). new_kernel = torch.Tensor(k2)