Continuity-attenuation captured network for frame deletion detection

Abstract

Frame deletion detection has played an essential role in digital forensics. The existing literature suggests that detection work is accomplished by appropriately revealing continuity-attenuation traces of video contents caused by frame deletion in the temporal direction. In this work, we propose a new network architecture, one module of which is exploited as a detector to capture the spatiotemporal features with continuity-attenuation in the forgery videos. First, through a study on the statistical characteristics of the motion trajectory of moving objects, we reveal a new continuity-attenuation trace, based on which the inter-frame residual feature is selected as the basis for continuity-attenuation tracking. Second, to capture the continuity-attenuation phenomenon, we design a network framework consisting of three components: a detector module, a reference module, and a decision module. Three modules work cooperatively under the contrast learning strategy to make the detector more sensitive to capture the forensic trace. The experiment results show that the detection rate can reach 93.85%, indicating the effectiveness of our proposed deep learning-based detection strategy.

Appendix

1.1 Proof of Eq. (4)

On the basis of the first-order Markovian hypothesis, the variance of motion intensity is constant. ${\text{D}} \left( {u_{t}^{x} } \right)$ is expanded as follows:

$$ \begin{gathered} {\text{D}} \left( {u_{t}^{x} } \right) \,= {\text{E}} \left[ {\left( {{\text{D}} \left( {u_{t}^{x} } \right)^{2} } \right)} \right] \hfill\\ \quad = {\text{E}} \left[ {\left( {\sum\limits_{i = 1}^{t} {a^{t - i} \varepsilon_{i}^{x} } } \right) \cdot \left( {\sum\limits_{i = 1}^{t} {a^{t - i} \varepsilon_{i}^{x} } } \right)} \right] \hfill\\ \quad = {\text{E}} \left[ \begin{gathered} a^{{2\left( {t - 1} \right)}} \left( {\varepsilon_{1}^{x} } \right)^{2} + a^{2t - 3} \varepsilon_{1}^{x} \varepsilon_{2}^{x} + \cdots + a^{t - 1} \varepsilon_{1}^{x} \varepsilon_{t}^{x} \hfill \\ \qquad + a^{2t - 3} \varepsilon_{2}^{x} \varepsilon_{1}^{x} + a^{{2\left( {t - 2} \right)}} \left( {\varepsilon_{2}^{x} } \right)^{2} + \cdots + a^{t - 2} \varepsilon_{2}^{x} \varepsilon_{t}^{x} + \hfill \\ \qquad \cdots \hfill \\ \qquad + a^{t - 1} \varepsilon_{t}^{x} \varepsilon_{1}^{x} + a^{t - 2} \varepsilon_{t}^{x} \varepsilon_{2}^{x} + \cdots + \left( {\varepsilon_{t}^{x} } \right)^{2} \hfill \\ \end{gathered} \right] \hfill\\ \quad = a^{{2\left( {t - 1} \right)}} {\text{E}} \left[ {\left( {\varepsilon_{1}^{x} } \right)^{2} } \right] + a^{2t - 3} {\text{E}} \left[ {\varepsilon_{1}^{x} \varepsilon_{2}^{x} } \right] + \cdots + a^{t - 1} {\text{E}} \left[ {\varepsilon_{1}^{x} \varepsilon_{t}^{x} } \right] \hfill\\ \qquad + a^{2t - 3} {\text{E}} \left[ {\varepsilon_{2}^{x} \varepsilon_{2}^{x} } \right] + a^{{2\left( {t - 2} \right)}} {\text{E}} \left[ {\varepsilon_{2}^{x} } \right] + \cdots + a^{t - 2} {\text{E}} \left[ {\varepsilon_{2}^{x} \varepsilon_{t}^{x} } \right] \hfill\\ \qquad + \cdots \hfill\\ \qquad + a^{t - 1} {\text{E}} \left[ {\varepsilon_{t}^{x} \varepsilon_{1}^{x} } \right] + a^{t - 2} {\text{E}} \left[ {\varepsilon_{t}^{x} e_{2} } \right] + \cdots + {\text{E}} \left[ {\left( {\varepsilon_{t}^{x} } \right)^{2} } \right] \hfill\\ \end{gathered}$$

(23)

Since the random interference is assumed to be white noise, the following applies:

$$ \begin{gathered} E\left[ {\varepsilon_{i}^{x} \varepsilon_{j}^{x} } \right] = 0,i \ne j \hfill \\ E\left[ {\left( {\varepsilon_{i}^{x} } \right)^{2} } \right] = \delta^{2} ,i = 1,2,3, \ldots ,t \hfill \\ \end{gathered} $$

(24)

According to Eq. (30), ${\text{D}} \left[ {f_{x} \left( t \right)} \right]$ is obtained as follows:

$$ \begin{gathered} {\text{D}} \left( {u_{t}^{x} } \right) = \delta^{2} \cdot \left( {a_{{}}^{{2\left( {t - 1} \right)}} + a_{{}}^{{2\left( {t - 2} \right)}} + \cdots + 1} \right) \\ = \frac{{1 - a_{{}}^{2t} }}{{1 - a_{{}}^{2} }} \cdot \delta^{2} \\ \end{gathered} $$

(25)

Similarly, ${\text{COV}} \left[ {u_{t}^{x} ,u_{t + 1}^{x} } \right]$ can be calculated as follows:

$$ \begin{gathered} {\text{COV}} \left[ {u_{t}^{x} ,u_{t + 1}^{x} } \right] = {\text{E}} \left[ {\left( {\sum\limits_{i = 1}^{t} {a^{t - i} \varepsilon_{i}^{x} } } \right)\left( {\sum\limits_{i = 1}^{t + 1} {a^{t + 1 - i} \varepsilon_{i}^{x} } } \right)} \right] \\ = {\text{E}} \left[ \begin{gathered} a^{2t - 1} \left( {\varepsilon_{1}^{x} } \right)^{2} + a^{2t - 2} \varepsilon_{1}^{x} \varepsilon_{2}^{x} + \cdots + a^{t - 1} \varepsilon_{1}^{x} \varepsilon_{t + 1}^{x} + \hfill \\ a^{2t - 2} \varepsilon_{2}^{x} \varepsilon_{1}^{x} + a^{2t - 3} \left( {\varepsilon_{2}^{x} } \right)^{2} + \cdots + a^{t - 2} \varepsilon_{2}^{x} \varepsilon_{t + 1}^{x} + \hfill \\ \vdots + \hfill \\ a^{t} \varepsilon_{t}^{x} \varepsilon_{1}^{x} + a^{t - 1} \varepsilon_{t}^{x} \varepsilon_{2}^{x} + \cdots + a^{1} \left( {\varepsilon_{t}^{x} } \right)^{2} + \cdots \varepsilon_{t}^{x} \varepsilon_{t + 1}^{x} \hfill \\ \end{gathered} \right] \\ = a^{2t - 1} {\text{E}} \left[ {\left( {\varepsilon_{1}^{x} } \right)^{2} } \right] + a^{2t - 2} {\text{E}} \left[ {\varepsilon_{1}^{x} \varepsilon_{2}^{x} } \right] + \cdots + a^{t - 1} {\text{E}} \left[ {\varepsilon_{1}^{x} \varepsilon_{t}^{x} } \right] + \\ a^{2t - 2} {\text{E}} \left[ {\varepsilon_{2}^{x} \varepsilon_{1}^{x} } \right] + a^{2t - 3} {\text{E}} \left[ {\left( {\varepsilon_{2}^{x} } \right)^{2} } \right] + \cdots + a^{t - 2} {\text{E}} \left[ {\varepsilon_{2}^{x} \varepsilon_{t}^{x} } \right] + \\ \cdots + \\ a^{t} {\text{E}} \left[ {\varepsilon_{t}^{x} \varepsilon_{1}^{x} } \right] + a^{t - 1} {\text{E}} \left[ {\varepsilon_{t}^{x} \varepsilon_{2}^{x} } \right] + \cdots + a^{1} {\text{E}} \left[ {\left( {\varepsilon_{t}^{x} } \right)^{2} } \right] + {\text{E}} \left[ {\varepsilon_{t}^{x} \varepsilon_{t + 1}^{x} } \right] \\ \end{gathered} $$

(26)

and resulted in:

$$ {\text{COV}} \left[ {u_{t}^{x} ,u_{t + 1}^{x} } \right]{ = }\frac{{1 - a_{{}}^{2t} }}{{1 - a_{{}}^{2} }} \cdot a \cdot \delta^{2} $$

(27)

Further, the correlation coefficient $\rho_{x}^{t}$ can be obtained as follows:

$$ \rho_{x}^{t} { = }a $$

(28)

1.2 Proof of Eq. (10)

According to Eq. (7), $f_{x} \left( {t + k} \right)$ is expressed as follows:

$$ u_{t + k}^{x} = a^{k} \varepsilon_{t}^{x} + a^{k - 1} \varepsilon_{t + 1}^{x} + a^{k - 2} \varepsilon_{t + 2}^{x} + \cdots + \varepsilon_{t + k}^{x} $$

(29)

Therefore, ${\text{COV}} \left[ {u_{t}^{x} ,u_{t + k}^{x} } \right]$ is calculated as follows:

$$ \begin{gathered} {\text{COV}} \left[ {u_{t}^{x} ,u_{t + k}^{x} } \right] = {\text{E}} \left[ {\left( {\sum\limits_{i = 1}^{t} {a^{t - i} \varepsilon_{i}^{x} } } \right)\left( {\sum\limits_{i = 1}^{t + k} {a^{t + k - i} \varepsilon_{i}^{x} } } \right)} \right] \\ = {\text{E}} \left[ \begin{gathered} a^{2t + k - 2} \left( {\varepsilon_{1}^{x} } \right)^{2} + a^{2t + k - 3} \varepsilon_{1}^{x} \varepsilon_{2}^{x} + \cdots + a^{t - 1} \varepsilon_{1}^{x} \varepsilon_{t}^{x} + \hfill \\ a^{2t + k - 3} \varepsilon_{2}^{x} \varepsilon_{1}^{x} + a^{2t + k - 4} \left( {\varepsilon_{2}^{x} } \right)^{2} + \cdots + a^{t - 2} \varepsilon_{2}^{x} \varepsilon_{t}^{x} + \hfill \\ \vdots + \hfill \\ a^{t + k - 1} \varepsilon_{t}^{x} \varepsilon_{1}^{x} + a^{t + k - 2} \varepsilon_{t}^{x} \varepsilon_{2}^{x} + \cdots + a^{k} \left( {\varepsilon_{t}^{x} } \right)^{2} + \cdots \varepsilon_{t}^{x} \varepsilon_{t + k}^{x} \hfill \\ \end{gathered} \right] \\ = a^{2t + k - 2} {\text{E}} \left[ {\left( {\varepsilon_{1}^{x} } \right)^{2} } \right] + a^{2t + k - 3} {\text{E}} \left[ {\varepsilon_{1}^{x} \varepsilon_{2}^{x} } \right] + \cdots + a^{t - 1} {\text{E}} \left[ {\varepsilon_{1}^{x} \varepsilon_{t}^{x} } \right] + \\ a^{2t + k - 3} {\text{E}} \left[ {\varepsilon_{2}^{x} \varepsilon_{1}^{x} } \right] + a^{2t + k - 4} {\text{E}} \left[ {\left( {\varepsilon_{2}^{x} } \right)^{2} } \right] + \cdots + a^{t - 2} {\text{E}} \left[ {\varepsilon_{2}^{x} \varepsilon_{t}^{x} } \right] + \\ \cdots + \\ a^{t + k - 1} {\text{E}} \left[ {\varepsilon_{t}^{x} \varepsilon_{1}^{x} } \right] + a^{t + k - 2} {\text{E}} \left[ {\varepsilon_{t}^{x} \varepsilon_{2}^{x} } \right] + \cdots + a^{k} {\text{E}} \left[ {\left( {\varepsilon_{t}^{x} } \right)^{2} } \right] + {\text{E}} \left[ {\varepsilon_{t}^{x} \varepsilon_{t + k}^{x} } \right] \\ \end{gathered} $$

(30)

according to Eq. (30), ${\text{COV}} \left[ {u_{t}^{x} ,u_{t + k}^{x} } \right]$ results in

$$ {\text{COV}} \left[ {u_{t}^{x} ,u_{t + k}^{x} } \right] = \frac{{\left( {1 - a^{2t} } \right) \cdot a^{k} }}{{1 - a^{2} }} \cdot \delta^{2} $$

(31)

Then, the k-step correlation coefficient is

$$ \rho_{k} = \frac{{{\text{COV}} \left( {u_{t} ,u_{t + k} } \right)}}{{\sqrt {{\text{D}} \left( {u_{t}^{x} } \right)} \sqrt {{\text{D}} \left( {u_{t + k}^{x} } \right)} }} = a^{k} $$

(32)