Yumi's Blog

What would happen if image is analyzed with 1D-DCT vs 2D-DCT?

In the previous blog, I reviewed 1D-DCT and 2D-DCT. In this blog, I will check what would happen if you denoise the image via 1D-DCT on flattened image.

Denoising

I will denoise the image by first transforming an image to frequency domain and pick the top $k$ largest DCT coefficients in absolute values. The remaining DCT coefficients are sparcified to zero. The new DCT coefficients are then inverse DCTed to spacial domain. The recovered image is visualized.

For the number of DCT Coefficients, I consider the following ks:

In [1]:
import numpy as np
height, width = 28, 28 ## height and width of the image
ks = [int(height*width*p) for p in np.linspace(0.1, 1, 3)[::-1]] + [50,40,30,20,15,10,5,1]
ks
Out[1]:
[784, 431, 78, 50, 40, 30, 20, 15, 10, 5, 1]

Import image

In [2]:
import matplotlib.pyplot as plt
import matplotlib.image as mpimg 
import numpy as np
import cv2
 
# use MNIST data
from keras.datasets import mnist as data_keras
(x_train, y_train), (x_test, y_test) = data_keras.load_data()
img = x_train[0]
plt.figure(figsize=(10,10))
plt.imshow(img,cmap="gray") 
plt.show()

Using TensorFlow backend.

1-D DCT

In [3]:
from scipy.fftpack import dct, idct
F = np.transpose(dct(np.eye(height*width), type=2, norm='ortho'))
y = img.flatten()
signal_1d = F @ y 
plt.plot(signal_1d)
plt.title("1D DCT signals prop zero = {}%".format(np.mean(signal_1d == 0)*100))
plt.show()
In [4]:
def sparcity_plot(signal,thresholds):
    abs_signal = np.abs(signal)
    prop_importance = np.sort(abs_signal)[::-1]/np.sum(abs_signal)*100
    plt.plot(prop_importance,"p")
    plt.title("% of | DCT coefficients | weight")
    for threshold in thresholds:
        print("N of DCT coefficients with weight > {} % = {}".format(
            threshold,np.sum(prop_importance > threshold)))
    plt.show()
thresholds = np.linspace(0.001,1,10)    
sparcity_plot(signal_1d,thresholds)

N of DCT coefficients with weight > 0.001 % = 774
N of DCT coefficients with weight > 0.112 % = 214
N of DCT coefficients with weight > 0.223 % = 96
N of DCT coefficients with weight > 0.334 % = 63
N of DCT coefficients with weight > 0.445 % = 48
N of DCT coefficients with weight > 0.556 % = 34
N of DCT coefficients with weight > 0.667 % = 24
N of DCT coefficients with weight > 0.778 % = 13
N of DCT coefficients with weight > 0.889 % = 12
N of DCT coefficients with weight > 1.0 % = 11

2-D DCT

In [5]:
F_ = np.transpose(dct(np.eye(height), type=2, norm='ortho'))
FF = np.kron(F_,F_)
vec_img = img.flatten()

signal_2d = FF @ vec_img
plt.plot(signal_2d)
plt.title("2D DCT signals prop zero = {}%".format(np.mean(signal_2d == 0)*100))
plt.show()
In [6]:
sparcity_plot(signal_2d,thresholds)

N of DCT coefficients with weight > 0.001 % = 781
N of DCT coefficients with weight > 0.112 % = 210
N of DCT coefficients with weight > 0.223 % = 109
N of DCT coefficients with weight > 0.334 % = 73
N of DCT coefficients with weight > 0.445 % = 42
N of DCT coefficients with weight > 0.556 % = 31
N of DCT coefficients with weight > 0.667 % = 20
N of DCT coefficients with weight > 0.778 % = 16
N of DCT coefficients with weight > 0.889 % = 13
N of DCT coefficients with weight > 1.0 % = 11

Compare the results of 1D and 2D DCT in spacial domain

In [7]:
def plot(ax,signal,ks,F):
    from copy import copy
    signal_=copy(signal)
    argsort_signal = np.argsort(np.abs(signal_))
    for i, k in enumerate(ks):
        signal_[argsort_signal[:-k]] = 0
        mytitle = "k = {}".format(np.sum(signal_ !=0))
        ax[i].set_title(mytitle)
        img_ = (np.transpose(F) @ signal_).reshape(height,width)
        ax[i].imshow(img_,cmap="gray")
fig, ax = plt.subplots(len(ks),2,figsize=(10,50))        
plot(ax[:,0],signal_1d,ks,F)
ax[0,0].set_title("1D DCT, k = {}".format(ks[0]))
plot(ax[:,1],signal_2d,ks,FF)
ax[0,1].set_title("2D DCT, k = {}".format(ks[0]))
plt.show()

Conclusion:

  • 2D-DCT coefficients are better when k is small (K <= 40).

Comments