https://github.com/bao231/liveness-detection-introduction
活体检测方法调研，introduction to liveness detection
Science Score: 10.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
○codemeta.json file
-
○.zenodo.json file
-
○DOI references
-
✓Academic publication links
Links to: arxiv.org, ieee.org -
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (4.5%) to scientific vocabulary
Last synced: 10 months ago
·
JSON representation
Repository
活体检测方法调研，introduction to liveness detection
Statistics
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
- Releases: 0
Fork of Jing-Xiaolong/liveness-detection-introduction
Created about 5 years ago
· Last pushed almost 7 years ago
https://github.com/bao231/liveness-detection-introduction/blob/master/
- [](#) - [](#) - [PA](#PA) - [](#)
## **motivation** PA - presentation attacksprint, replay, 3D-mask, facial cosmetic makeup, etc PA2017CNN **methods** | | traditional methods | CNN-based methods | | :------------: | ------------------------------------------------------------ | ------------------------------------------------------------ | | |
LBPSIFTSURF | CNNanti-spoofing | | | 1. PAreplay/ 3D mask attack
2. /PAs | 1. CNN
2. PA
- | | | 1. **** | 1. ****
-
-
2. ** **
-
3. ****
- rPPG()
- (PA) |
## **** 1. 2. SIFTSURFLBPHOG 3. 4. real/spoofSVMLR
> [2015icip - Face anti-spoofing based on color texture analysis](https://ieeexplore.ieee.org/abstract/document/7351280) > [2016 - Face spoofing detection using color texture analysis](https://ieeexplore.ieee.org/abstract/document/7454730) - PALBP (local binary patterns )LBP***LBP***
-
> [2018 - Ccolbp: Chromatic cooccurrence > of local binary pattern for face presentation attack > detection](https://ieeexplore.ieee.org/document/8487325) - -
> [2012 - On the effectiveness > of local binary patterns in face anti-spoofing](https://ieeexplore.ieee.org/abstract/document/6313548) - -
> [2012 - LBP-TOP based countermeasure against face spoofing attacks](http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=271BCC281BDD5D9B869D3DB92A278BB0?doi=10.1.1.493.6222&rep=rep1&type=pdf) > > [2013 - Can face anti-spoofing countermeasures work in a real world scenario?](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.650.8406&rep=rep1&type=pdf) - -
> [2017 - Face anti-spoofing based on color texture analysis](https://ieeexplore.ieee.org/abstract/document/7351280) - -
> [2013 - Face Liveness Detection with Component Dependent Descriptor](http://www.cbsr.ia.ac.cn/users/zlei/papers/ICB2013/YANG-ICB13.pdf) - - 1. 6 2. LBPHOG 3. SVMreal/spoofing
## PA [**1. Replay-attack 2012**](https://www.idiap.ch/dataset/replayattack) 1300[EULA](https://www.idiap.ch/dataset/replayattack/eula.pdf)(End User License Agreement) [**2. MSU-USSA**](http://biometrics.cse.msu.edu/Publications/Databases/MSU_USSA/) MSU Unconstraint Smartphone Spoof Attack Database9000(1000live+8000spoof)[MSU USSA Agreement](http://biometrics.cse.msu.edu/Publications/Databases/MSU_USSA/MSU_USSA_Database_Agreement.pdf) [**3. oulu-npu 2017**](https://sites.google.com/site/oulunpudatabase/) 4950CNN-based[EULA](https://drive.google.com/open?id=1m5qWHpdvM4SNOewKbw43u95makMTfD68) [**4. SiW 2018**](http://cvlab.cse.msu.edu/spoof-in-the-wild-siw-face-anti-spoofing-database.html) 165subjects4478[DRA](https://www.cse.msu.edu/computervision/SiW_Dataset_Release_Agreement_Form.pdf)(dataset release agreement) [**5. CASIA-SURF 2019.6**](https://sites.google.com/qq.com/chalearnfacespoofingattackdete/welcome) 1000subjects21000[CASIA-SURF release agreement](http://www.cbsr.ia.ac.cn/users/jwan/database/CASIA-SURF_agreement.pdf)
## #### An original face anti-spoofing approach using partial convolutional neural network [](https://ieeexplore.ieee.org/document/7821013) **motivation/** - ****PA **models/** VGG
#### Face anti-spoofing using patch and depth-based cnns [](http://cvlab.cse.msu.edu/pdfs/FaceAntiSpoofingUsingPatchandDepthBasedCNNs.pdf) **** **motivation/** - ****** 1. PA 2. PAPA**models/** - **Supervision Signals** - ***patch spoof scores***patchespatchpatch spoof scorespatch-based CNN **patch spoof scores**1 - live0 - spoof - ***Depth Map***depth-based CNN ******[3DMM(2003)](https://ieeexplore.ieee.org/abstract/document/1227983)**3D$A\in\mathbb{R}^{3\times Q}$2D$M\in\mathbb{R}^{52\times 52}$
- **** - ***patch-based CNN*** patches1. 2. resize 3. patchesRGB, HSV, YCbCr, pixel-wised LBP pacth spoof scores - ***Depth-based CNN*** anti-spoofingresizeFCNsize HSV + YCbCr $\widehat{M}\in\mathbb{R}^{52\times52}$ - ******
#### Deep Convolutional Dynamic Texture Learning with Adaptive Channel-discriminability for 3D Mask Face Anti-spoofing [](https://ieeexplore.ieee.org/document/8272765) **motivation/** ***3D mask***PA - ****maskmask 3D mask **models/**- ****1. [**CLNF(2014)**](https://www.cl.cam.ac.uk/research/rainbow/projects/ccnf/files/iccv2014.pdf) 2. VGG 3. Subtle Facial Motion Estimation 4. Deep Conv Dynamic texture 5. spoofing/live - **** - ***Subtle Facial Motion Estimation*** $X_t\in\mathbb{R}^{W\times H\times K}$$W,H,K$$t=1,2,...,T$ $X_t$$K$$\{C_k\in\mathbb{R}^{W\times H\times T}\}_{k=1}^K$$\frac{\part{I}}{\part{x}}u+\frac{\part{I}}{\part{y}}v+\frac{\part{I}}{\part{t}}=0$ $I(x,y,t)$$(x,y)$$u,v$$(x,y)$optical flow $V\in\mathbb{R}^{K\times N},N= W \times H$ - ***Adaptive Channel-discriminability constraint learning*** $V$3D mask$D\in\mathbb{R}^{1\times K}$$D$40 - ***Deep Convolutional Dynamic Texture Learning*** $min_{D, U^l}\frac{1}{2}\sum_{l=1}^N||DV^l-U^l||^2, s.t.||D||_2=1$$U\in\mathbb{R}^{1\times N}$ $U^l=DV^l, l=1,2,3,...,N$ **implementation/** - **** - [**CLNF(2014)**](https://www.cl.cam.ac.uk/research/rainbow/projects/ccnf/files/iccv2014.pdf) 68landmarksalign - VGG 5VGGconv3-3$F\in\mathbb{R}^{56\times56\times256}$ - classifier SVM - **** Intra-dataset
cross-dataset
#### Learning Deep Models for Face Anti-Spoofing: Binary or Auxiliary Supervision **[](https://ieeexplore.ieee.org/document/8578146)** **src codes** **** **motivation/** - ****PAPA - ****rPPG(remote Photoplephysmography)********models/** - **Supervision Signals** - ***Depth Map***CNN ******[DeFA(2017)](https://arxiv.org/abs/1709.01442)**3D$S\in\mathbb{R}^{3\times Q}$2D$D\in\mathbb{R}^{32\times32}$ - ***rPPG***rPPGrPPGreplay attackRNN **rPPG**rPPGrPPG DeFA$x_f=3r_f-2g_f$$y_f=1.5r_f+g_f-1.5b_f$$\gamma=\frac{\sigma(x_f)}{\sigma(y_f)}$$p=3(1-\frac{\gamma}{2})r_f-2(1+\frac{\gamma}{2})g_f+\frac{2\gamma}{2}b_f$$p$FFTrPPG
- **** - ***CNN*** blockblockconv+exponential linear+bnpooling blockresize layerresize$64\times64$ branchesdepth mapfeature map $\Theta_D=argmin_{\Theta_{D}}\sum_{i=1}^{N_d}||CNN_D(I_i; \Theta_D)-D_i||_1^2$ - ***RNN*** $\Theta_R=argmin_{\Theta_{R}}\sum_{i=1}^{N_s}||RNN_R([{F_j}_{j=1}^{N_F}]_i; \Theta_R)-f_i||_1^2$ $F_j$frontilizationfeature map - ***Non-rigid Registration*** 3D$S$feature mapalignRNNRNN
**implementation/** - **** - two-stream training 1. $I$$D$CNN 2. ${I_j}_{j=1}^{N_f}$${D_j}_{j=1}^{N_f}$3D${S_j}_{j=1}^{N_f}$rPPGCNNRNN - testing $score=||\widehat{f}||_2^2+\lambda||\widehat{D}||_2^2$ - **** intro-testcross-test results
#### Face De-Spoofing: Anti-Spoofing via Noise Modeling **[](http://openaccess.thecvf.com/content_ECCV_2018/html/Yaojie_Liu_Face_De-spoofing_ECCV_2018_paper.html)** **src codes**[](https://github.com/yaojieliu/ECCV2018-FaceDeSpoofing)README ****spoofing noisePA **motivation/** ***print, replay, make-up***PA - ****spoof facelive facespoof noisespoof noisePAs $x\in\mathbb{R}^m$$x=A\widehat{x}+n=\widehat{x}+(A-\mathbb{I})\widehat{x}+n=\widehat{x}+N(\widehat{x})$$\widehat{x}$$A\in\mathbb{R}^{m\times m}$$n\in\mathbb{R}^m$$N(\widehat{x})=(A-\mathbb{I})\widehat{x}$image-dependent$\widehat{x},N(\widehat{x})$spoof noise$\widehat{x}$$x=\widehat{x}$spoof noise = 0 spoof1. spoof 2.- **Spoof noise pattern study** - 1. color distortionspoof 2. Display artifacts 3. Presenting artifacts:additive 4. Imaging artifactsCMOSCCDreplay attackprint attackalisingmoire pattern -
**models/**
- **** - $I\in\mathbb{R}^{256\times 256 \times 6}$RGB+HSV - ***DS Net*** De-Spoof Netencoder-decoder$N(\widehat{I})$$\widehat{I}=I-N(\widehat{I})$ encoder$F\in\mathbb{R}^{32\times 32\times 32}$spoof decoder$F$$N(\widehat{I})$ 0\1 Map Net0 - live face1 - spoof - ***DQ Net*** discriminative quality net[CNN](#**[2018cvpr - Learning Deep Models for Face Anti-Spoofing: Binary or Auxiliary Supervision](https://ieeexplore.ieee.org/document/8578146)**)DQ Net$\widehat{I}$ DQ Net$\widehat{I}$$CNN_{DQ}(\widehat{I})$2D$D\in\mathbb{R}^{32\times32}$$\widehat{I}$$D$ $J_{DQ}=||CNN_{DQ}(\widehat{I})-D||_1$ - ***VQ Net*** visual quality netGAN$\widehat{I}$visual qualityVQ$\widehat{I}I$$I$PAsVQ Net DS NetVQ Net$J_{VQ_{train}}=-\mathbb{E}_{I\in R}log(CNN_{VQ}(I))-\mathbb{E}_{I\in S}log(1-CNN_{VQ}(CNN_{DS}(I)))$ VQ NetDS Net$J_{VQ_{test}}=-\mathbb{E}_{I\in S}log(CNN_{VQ}(CNN_{DS}(I)))$ $R,S$ - **** - magnitude lossspoof noise0 $J_m=||N||_1$ - Zero\One map lossspoof noise $J_z=||CNN_{01map}(F;\Theta)-M||_1$$M\in0^{32\times 32} or M\in1^{32\times 32}$ - repetitive lossspoof noisespoof$N$$F$spoof noise $J_r=\begin{cases}-max(H(F(N),k)),I\in Spoof \\ ||max(H(F(N),k))||_1, I\in Live\end{cases}$$H$$k\times k$ - $J_{Total}=J_z+\lambda_1J_m+\lambda_2J_r+\lambda_3J_{DQ}+\lambda_4J_{VQ_{test}}$ **implementation/** - ****
#### Exploiting Temporal and Depth Information for Multi-frame face Anti-Spoofing **[](https://arxiv.org/abs/1811.05118)** **src models** **motivation/** - ****PAPAliveness$(a):\alpha>\beta_{2}, \beta_{1}<\gamma, (b):\alpha'<\beta_{2}', \beta_{1}'>\gamma'$
PA$d1, d2$ real scene: $d1'/d2'=d1/d2$$d1'$$d1$ print attack: $d1'=0,d2'=0$ replay attack: $d1'/d2'=d1/d2$ in perfect spoofing scene (keep stationary, nearly impossible) $d1'/d2'\ne d1/d2$ in usual spoofing scene PA() **Models**: - **** - ********[PRNet(2017)](http://openaccess.thecvf.com/content_ECCV_2018/html/Yao_Feng_Joint_3D_Face_ECCV_2018_paper.html)**$V_{n\times3}$n3D2DUV$D\in\mathbb{R}^{32\times32}$$D^t$
- ******
- ***depth loss*** = + (contrast) $L_{single}^{absolute}=||D_{single}-D||_2^2$ $L_{single}^{contrast}=\sum_{i}||K_i^{contrast}\bigodot D_{single}-K_i^{contrast}\bigodot D||_2^2$ $L_{single}=L_{single}^{absolute}+L_{single}^{contrast}$ $K_{i}^{contrast}$
- **** / -> OFFB -> ConvGRU - ******
- ***OFFB*** optical flow guided feature block******** $t$$(x,y)$I(x,y,t)
vx, vyx,y(vx, vy)(vx, vy, 1)F=(Fx, Fy, Ft)Foptical flow guided features
- ***ConvGRU*** Convolution gated recurrent unit****()ConvGRU
$X_{t},H_{t},U_{t},R_{t}$update gatereset gate$K_{r},K_{u},K_{h}$$H_{t}$ConvGRU - ****** = + $D_{fusion}^{t}=\alpha*D_{single}^t+(1-\alpha)*D_{multi}^t, \alpha\in[0,1]$ 1. = + $L_{multi}^{absolute}(t)=||D_{fusion}^t-D^t||_2^2$ $L_{multi}^{contrast}(t)=\sum_{i}||K_i^{contrast}\bigodot D_{fusion}^t-K_i^{contrast}\bigodot D^t||_2^2$ $L_{multi}^{depth}=\sum_{t}^{N_F-1}(L_{multi}^{absolute}(t)+L_{multi}^{contrast}(t))$ $D^t$$t$ 2. $L_{multi}^{binary}=-B^t*log(fcs([{D_{fusion}^t}^{N_f-1}]))$ $B^t$$t$${D_{fusion}^t}^{N_f-1}$$N_f-1$()$t$ $fcs$ + softmax()score + 0/1 3. = + $L_{multi}=\betaL_{multi}^{binary}+(1-\beta)L_{multi}^{depth}$ **implementation/** - **** - two-staged training 1. 2. $N_f$ - testing: $\widehat{b}$living logtis$\widehat{b}=fcs(D_{fusion}^t)$living score $score=\beta\widehat{b}+(1-\beta)\frac{\sum_{t}^{N_f-1}||D_{fusion}^t*M_{fusion}^t||_1}{N_f-1}$$M_{fusion}^t$PRNett - **** OULU-NPUintra-testing
cross-testing
hard examples
Owner
- Login: bao231
- Kind: user
- Repositories: 84
- Profile: https://github.com/bao231