├── README.md ├── figures ├── fig1.png ├── fig5.png ├── fig6.png ├── fig7.png ├── fig8.png ├── fig9.png ├── network structrue.png ├── table1.png ├── table10.png ├── table11.png ├── table6.png ├── table7.png ├── table8.png ├── table9.png └── visual resuts_on_distribution_of_salars_SEFCNN_channels.png ├── hm_source ├── TDecGop.cpp ├── TEST.py ├── TEncGOP.cpp ├── UTILS.py └── callPython.cpp ├── rhcnn ├── RHCNN.py ├── RHCNN_B_qp37 │ ├── RHCNN_B_qp37_550.ckpt.data-00000-of-00001 │ ├── RHCNN_B_qp37_550.ckpt.index │ └── RHCNN_B_qp37_550.ckpt.meta ├── RHCNN_I_qp37 │ ├── RHCNN_I_qp37_589.ckpt.data-00000-of-00001 │ ├── RHCNN_I_qp37_589.ckpt.index │ └── RHCNN_I_qp37_589.ckpt.meta └── RHCNN_P_qp37 │ ├── RHCNN_ldp_qp37_566.ckpt.data-00000-of-00001 │ ├── RHCNN_ldp_qp37_566.ckpt.index │ └── RHCNN_ldp_qp37_566.ckpt.meta ├── sefcnn ├── SEFCNN.py ├── VDSR12.py ├── VDSR16.py └── model │ ├── VDSR12_qp22 │ ├── VDSR12_qp22_420.ckpt.data-00000-of-00001 │ ├── VDSR12_qp22_420.ckpt.index │ └── VDSR12_qp22_420.ckpt.meta │ ├── VDSR16_qp27 │ ├── VDSR16_qp27_389.ckpt.data-00000-of-00001 │ ├── VDSR16_qp27_389.ckpt.index │ └── VDSR16_qp27_389.ckpt.meta │ ├── VDSRx21_SE_qp40_B │ ├── VDSRx21_SE_qp40_576.ckpt.data-00000-of-00001 │ ├── VDSRx21_SE_qp40_576.ckpt.index │ └── VDSRx21_SE_qp40_576.ckpt.meta │ ├── VDSRx21_SE_qp40_I │ ├── VDSRx21_SE_qp40_I_set2K+1238_394.ckpt.data-00000-of-00001 │ ├── VDSRx21_SE_qp40_I_set2K+1238_394.ckpt.index │ └── VDSRx21_SE_qp40_I_set2K+1238_394.ckpt.meta │ ├── VDSRx21_SE_qp40_P │ ├── VDSRx21_SE_qp40_P_2618_129.ckpt.data-00000-of-00001 │ ├── VDSRx21_SE_qp40_P_2618_129.ckpt.index │ └── VDSRx21_SE_qp40_P_2618_129.ckpt.meta │ ├── gradient_Model │ ├── VDSRx15_SE_qp37_I_2K_High_Grad_147.ckpt.data-00000-of-00001 │ ├── VDSRx15_SE_qp37_I_2K_High_Grad_147.ckpt.index │ └── VDSRx15_SE_qp37_I_2K_High_Grad_147.ckpt.meta │ ├── sefcnn_B_qp32 │ ├── sefcnn_B_qp32_500.ckpt.data-00000-of-00001 │ ├── sefcnn_B_qp32_500.ckpt.index │ └── sefcnn_B_qp32_500.ckpt.meta │ ├── sefcnn_B_qp37 │ ├── sefcnn_B_qp37_368.ckpt.data-00000-of-00001 │ ├── sefcnn_B_qp37_368.ckpt.index │ └── sefcnn_B_qp37_368.ckpt.meta │ ├── sefcnn_I_qp32 │ ├── sefcnn_I_qp32_599.ckpt.data-00000-of-00001 │ ├── sefcnn_I_qp32_599.ckpt.index │ └── sefcnn_I_qp32_599.ckpt.meta │ ├── sefcnn_I_qp37 │ ├── sefcnn_I_qp37_model_570.ckpt.data-00000-of-00001 │ ├── sefcnn_I_qp37_model_570.ckpt.index │ └── sefcnn_I_qp37_model_570.ckpt.meta │ └── sefcnn_P_qp37 │ ├── sefcnn_P_qp37_588.ckpt.data-00000-of-00001 │ ├── sefcnn_P_qp37_588.ckpt.index │ └── sefcnn_P_qp37_588.ckpt.meta ├── trainSet └── trainSet&testSet.md └── vrcnn ├── model ├── QP22.caffemodel ├── QP27.caffemodel ├── QP32.caffemodel ├── QP37_B.caffemodel ├── QP37_I.caffemodel └── QP37_P.caffemodel └── vrcnn_test.prototxt /README.md: -------------------------------------------------------------------------------- 1 | # A Hierarchical Deep Learning Approach for In-loop Filtering in Video Coding 2 | ------ 3 | The paper proposes a deep learning based systematic approach that includes an effective Convolutional Neural Network (CNN) structure, a hierarchical training strategy, and a video codec oriented switchable mechanism. In brief, the contributions of this work are as follows: 4 | 5 | - A novel network,named as ***Squeezeand-Excitation Filtering CNN (SEFCNN)***, is designed, which is comprised of two subnets: *Feature EXtracting (FEX) net and Feature ENhancing (FEN) net*. The FEX is a stack of convolutional layers characterizing the spatial and channel-wise correlation, while the FEN is a squeeze-andexcitation net that fully explores the relationship between channels. 6 | - ***A hierarchical model training strategy*** is developed. During the encoding, (a) different Quantization Parameters (QPs) cause different levels of artifacts; (b) different frame types employ different coding tools and thus exhibit different artifact proprieties. In contrast to prior researches that design a single powerful network for all kinds of artifacts, we propose to **hierarchically deploy two subnets for different coding scenarios**. 7 | - When incorporating CNN model into video encoder, ***we conduct an adaptive mechanism that switches between the CNN-based and the traditional methods to selectively enhance some frames or some regions of a frame.*** Compared to previous work that applies one model to every single frame, our approach takes advantage of coding reference structure and obtains the superiority in both encoder computational complexity and overall coding efficiency. 8 | ## State-of-the-art solutions 9 | ![State-of-the-art solutions ](https://github.com/IVC-Projects/cnn_In-loop_filter/blob/master/figures/fig1.png) 10 | ## Network Structure 11 | ![Network Structure](https://github.com/IVC-Projects/cnn_In-loop_filter/blob/master/figures/network%20structrue.png) 12 | ## Hierarchical CNN Models for Different Frame Types 13 | ![Hierarchical CNN Models for Different Frame Types](https://github.com/IVC-Projects/cnn_In-loop_filter/blob/master/figures/table6.png) 14 | ![fig 6](https://github.com/IVC-Projects/cnn_In-loop_filter/blob/master/figures/fig6.png) 15 | ## visual comparisons 16 | ![visual comparisons](https://github.com/IVC-Projects/cnn_In-loop_filter/blob/master/figures/fig7.png) 17 | ## EXPERIMENTAL RESULTS 18 | ![CODING PERFORMANCE COMPARED WITH VRCNN (BD-RATE) 19 | ](https://github.com/IVC-Projects/cnn_In-loop_filter/blob/master/figures/table7.png) 20 | ![CODING PERFORMANCE OF THE PROPOSED SEFCNN COMPARED TO VRCNN AND RHCNN (QP = 37) 21 | ](https://github.com/IVC-Projects/cnn_In-loop_filter/blob/master/figures/table8.png) 22 | ![CODING PERFORMANCE COMPARED WITH WORK [39] (BD-RATE COMPARED TO HM12.0) 23 | ](https://github.com/IVC-Projects/cnn_In-loop_filter/blob/master/figures/table9.png) 24 | ![ PSNR histogram statistics of different network structures on common test sequences. 25 | ](https://github.com/IVC-Projects/cnn_In-loop_filter/blob/master/figures/fig8.png) 26 | ![CODING PERFORMANCE COMPARED WITH WORK [40] (BD-RATE COMPARED TO HM7.0) 27 | ](https://github.com/IVC-Projects/cnn_In-loop_filter/blob/master/figures/table10.png) 28 | ![Visualization of the parameter quantity versus PSNR gain in various network deployments. 29 | ](https://github.com/IVC-Projects/cnn_In-loop_filter/blob/master/figures/fig9.png) 30 | ![AVERAGE ENCODING TIME COMPARED TO HM16.9 (SECOND/FRAME) 31 | ](https://github.com/IVC-Projects/cnn_In-loop_filter/blob/master/figures/table11.png) 32 | -------------------------------------------------------------------------------- /figures/fig1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/figures/fig1.png -------------------------------------------------------------------------------- /figures/fig5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/figures/fig5.png -------------------------------------------------------------------------------- /figures/fig6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/figures/fig6.png -------------------------------------------------------------------------------- /figures/fig7.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/figures/fig7.png -------------------------------------------------------------------------------- /figures/fig8.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/figures/fig8.png -------------------------------------------------------------------------------- /figures/fig9.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/figures/fig9.png -------------------------------------------------------------------------------- /figures/network structrue.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/figures/network structrue.png -------------------------------------------------------------------------------- /figures/table1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/figures/table1.png -------------------------------------------------------------------------------- /figures/table10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/figures/table10.png -------------------------------------------------------------------------------- /figures/table11.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/figures/table11.png -------------------------------------------------------------------------------- /figures/table6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/figures/table6.png -------------------------------------------------------------------------------- /figures/table7.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/figures/table7.png -------------------------------------------------------------------------------- /figures/table8.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/figures/table8.png -------------------------------------------------------------------------------- /figures/table9.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/figures/table9.png -------------------------------------------------------------------------------- /figures/visual resuts_on_distribution_of_salars_SEFCNN_channels.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/figures/visual resuts_on_distribution_of_salars_SEFCNN_channels.png -------------------------------------------------------------------------------- /hm_source/TDecGop.cpp: -------------------------------------------------------------------------------- 1 | /* The copyright in this software is being made available under the BSD 2 | * License, included below. This software may be subject to other third party 3 | * and contributor rights, including patent rights, and no such rights are 4 | * granted under this license. 5 | * 6 | * Copyright (c) 2010-2016, ITU/ISO/IEC 7 | * All rights reserved. 8 | * 9 | * Redistribution and use in source and binary forms, with or without 10 | * modification, are permitted provided that the following conditions are met: 11 | * 12 | * * Redistributions of source code must retain the above copyright notice, 13 | * this list of conditions and the following disclaimer. 14 | * * Redistributions in binary form must reproduce the above copyright notice, 15 | * this list of conditions and the following disclaimer in the documentation 16 | * and/or other materials provided with the distribution. 17 | * * Neither the name of the ITU/ISO/IEC nor the names of its contributors may 18 | * be used to endorse or promote products derived from this software without 19 | * specific prior written permission. 20 | * 21 | * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 22 | * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 23 | * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 24 | * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS 25 | * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR 26 | * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF 27 | * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS 28 | * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN 29 | * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) 30 | * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF 31 | * THE POSSIBILITY OF SUCH DAMAGE. 32 | */ 33 | 34 | /** \file TDecGop.cpp 35 | \brief GOP decoder class 36 | */ 37 | 38 | #include "TDecGop.h" 39 | #include "TDecCAVLC.h" 40 | #include "TDecSbac.h" 41 | #include "TDecBinCoder.h" 42 | #include "TDecBinCoderCABAC.h" 43 | #include "libmd5/MD5.h" 44 | #include "TLibCommon/SEI.h" 45 | 46 | 47 | 48 | #include 49 | 50 | //! \ingroup TLibDecoder 51 | //! \{ 52 | static Void calcAndPrintHashStatus(TComPicYuv& pic, const SEIDecodedPictureHash* pictureHashSEI, const BitDepths &bitDepths, UInt &numChecksumErrors); 53 | // ==================================================================================================================== 54 | // Constructor / destructor / initialization / destroy 55 | // ==================================================================================================================== 56 | 57 | TDecGop::TDecGop() 58 | : m_numberOfChecksumErrorsDetected(0) 59 | { 60 | m_dDecTime = 0; 61 | } 62 | 63 | TDecGop::~TDecGop() 64 | { 65 | 66 | } 67 | 68 | Void TDecGop::create() 69 | { 70 | 71 | } 72 | 73 | 74 | Void TDecGop::destroy() 75 | { 76 | } 77 | 78 | Void TDecGop::init( TDecEntropy* pcEntropyDecoder, 79 | TDecSbac* pcSbacDecoder, 80 | TDecBinCABAC* pcBinCABAC, 81 | TDecCavlc* pcCavlcDecoder, 82 | TDecSlice* pcSliceDecoder, 83 | TComLoopFilter* pcLoopFilter, 84 | TComSampleAdaptiveOffset* pcSAO 85 | ) 86 | { 87 | m_pcEntropyDecoder = pcEntropyDecoder; 88 | m_pcSbacDecoder = pcSbacDecoder; 89 | m_pcBinCABAC = pcBinCABAC; 90 | m_pcCavlcDecoder = pcCavlcDecoder; 91 | m_pcSliceDecoder = pcSliceDecoder; 92 | m_pcLoopFilter = pcLoopFilter; 93 | m_pcSAO = pcSAO; 94 | m_numberOfChecksumErrorsDetected = 0; 95 | } 96 | 97 | 98 | // ==================================================================================================================== 99 | // Private member functions 100 | // ==================================================================================================================== 101 | // ==================================================================================================================== 102 | // Public member functions 103 | // ==================================================================================================================== 104 | 105 | Void TDecGop::decompressSlice(TComInputBitstream* pcBitstream, TComPic* pcPic) 106 | { 107 | TComSlice* pcSlice = pcPic->getSlice(pcPic->getCurrSliceIdx()); 108 | // Table of extracted substreams. 109 | // These must be deallocated AND their internal fifos, too. 110 | TComInputBitstream **ppcSubstreams = NULL; 111 | 112 | //-- For time output for each slice 113 | clock_t iBeforeTime = clock(); 114 | m_pcSbacDecoder->init( (TDecBinIf*)m_pcBinCABAC ); 115 | m_pcEntropyDecoder->setEntropyDecoder (m_pcSbacDecoder); 116 | 117 | const UInt uiNumSubstreams = pcSlice->getNumberOfSubstreamSizes()+1; 118 | 119 | // init each couple {EntropyDecoder, Substream} 120 | ppcSubstreams = new TComInputBitstream*[uiNumSubstreams]; 121 | for ( UInt ui = 0 ; ui < uiNumSubstreams ; ui++ ) 122 | { 123 | ppcSubstreams[ui] = pcBitstream->extractSubstream(ui+1 < uiNumSubstreams ? (pcSlice->getSubstreamSize(ui)<<3) : pcBitstream->getNumBitsLeft()); 124 | } 125 | 126 | m_pcSliceDecoder->decompressSlice( ppcSubstreams, pcPic, m_pcSbacDecoder); 127 | // deallocate all created substreams, including internal buffers. 128 | for (UInt ui = 0; ui < uiNumSubstreams; ui++) 129 | { 130 | delete ppcSubstreams[ui]; 131 | } 132 | delete[] ppcSubstreams; 133 | 134 | m_dDecTime += (Double)(clock()-iBeforeTime) / CLOCKS_PER_SEC; 135 | } 136 | 137 | Void TDecGop::filterPicture(TComPic* pcPic) 138 | { 139 | TComSlice* pcSlice = pcPic->getSlice(pcPic->getCurrSliceIdx()); 140 | 141 | //-- For time output for each slice 142 | clock_t iBeforeTime = clock(); 143 | 144 | // deblocking filter 145 | Bool bLFCrossTileBoundary = pcSlice->getPPS()->getLoopFilterAcrossTilesEnabledFlag(); 146 | m_pcLoopFilter->setCfg(bLFCrossTileBoundary); 147 | #if CNN_LF 148 | TComPicYuv* pcPicYuvRec = pcPic->getPicYuvRec(); 149 | Pel* pPic = pcPicYuvRec->getAddr(COMPONENT_Y, 0); 150 | Pel* writePic = pPic; 151 | Pel* pPicFwrite = pPic; 152 | Pel* firstLineInCtu = pPic; 153 | Pel* firstLineInCu = pPic; 154 | int stride = pcPicYuvRec->getStride(COMPONENT_Y); 155 | int Height = pcPicYuvRec->getHeight(COMPONENT_Y); 156 | int Width = pcPicYuvRec->getWidth(COMPONENT_Y); 157 | if (pcPic->getPOC()==0) 158 | cfgQP = pcSlice->getSliceQp(); 159 | 160 | 161 | //cout << "QP=" << pcSlice->getSliceQp() << "qpD=" << pcSlice->getSliceQpDelta() << "qpOS" << pcSlice->getPPS()->getPicInitQPMinus26() << endl; 162 | //getchar(); 163 | cv::Mat cnnImg; 164 | if (pcSlice->getSliceType()==I_SLICE||(cfgQP<=27&&pcPic->getPOC()%4==1)) 165 | cnnImg = SplitAndStitch(pPic, Height, Width, stride, 1, cfgQP); 166 | if (pcSlice->getSliceType() != I_SLICE && cfgQP > 27&&pcPic->getPOC()%4==1) 167 | cnnImg = SplitAndStitch(pPic, Height, Width, stride, 2, cfgQP); 168 | if (!(pcSlice->getSliceType() == I_SLICE || pcPic->getPOC() % 4 == 1)) 169 | m_pcLoopFilter->loopFilterPic(pcPic); 170 | #endif 171 | 172 | 173 | if( pcSlice->getSPS()->getUseSAO() ) 174 | { 175 | m_pcSAO->reconstructBlkSAOParams(pcPic, pcPic->getPicSym()->getSAOBlkParam()); 176 | m_pcSAO->SAOProcess(pcPic); 177 | m_pcSAO->PCMLFDisableProcess(pcPic); 178 | } 179 | #if CNN_LF 180 | if (pcSlice->getSliceType() == I_SLICE || pcPic->getPOC() % 4 == 1) 181 | { 182 | uchar *temp; 183 | for (int i = 0; i < Height; i++){ 184 | temp = cnnImg.ptr(i); 185 | for (int j = 0; j < Width; j++){ 186 | (*(writePic + j)) = (Pel)temp[j]; 187 | } 188 | writePic += pcPicYuvRec->getStride(COMPONENT_Y); 189 | } 190 | 191 | } 192 | #endif 193 | pcPic->compressMotion(); 194 | TChar c = (pcSlice->isIntra() ? 'I' : pcSlice->isInterP() ? 'P' : 'B'); 195 | if (!pcSlice->isReferenced()) 196 | { 197 | c += 32; 198 | } 199 | 200 | //-- For time output for each slice 201 | printf("POC %4d TId: %1d ( %c-SLICE, QP%3d ) ", pcSlice->getPOC(), 202 | pcSlice->getTLayer(), 203 | c, 204 | pcSlice->getSliceQp() ); 205 | 206 | m_dDecTime += (Double)(clock()-iBeforeTime) / CLOCKS_PER_SEC; 207 | printf ("[DT %6.3f] ", m_dDecTime ); 208 | m_dDecTime = 0; 209 | 210 | for (Int iRefList = 0; iRefList < 2; iRefList++) 211 | { 212 | printf ("[L%d ", iRefList); 213 | for (Int iRefIndex = 0; iRefIndex < pcSlice->getNumRefIdx(RefPicList(iRefList)); iRefIndex++) 214 | { 215 | printf ("%d ", pcSlice->getRefPOC(RefPicList(iRefList), iRefIndex)); 216 | } 217 | printf ("] "); 218 | } 219 | if (m_decodedPictureHashSEIEnabled) 220 | { 221 | SEIMessages pictureHashes = getSeisByType(pcPic->getSEIs(), SEI::DECODED_PICTURE_HASH ); 222 | const SEIDecodedPictureHash *hash = ( pictureHashes.size() > 0 ) ? (SEIDecodedPictureHash*) *(pictureHashes.begin()) : NULL; 223 | if (pictureHashes.size() > 1) 224 | { 225 | printf ("Warning: Got multiple decoded picture hash SEI messages. Using first."); 226 | } 227 | calcAndPrintHashStatus(*(pcPic->getPicYuvRec()), hash, pcSlice->getSPS()->getBitDepths(), m_numberOfChecksumErrorsDetected); 228 | } 229 | 230 | printf("\n"); 231 | 232 | pcPic->setOutputMark(pcPic->getSlice(0)->getPicOutputFlag() ? true : false); 233 | pcPic->setReconMark(true); 234 | } 235 | 236 | /** 237 | * Calculate and print hash for pic, compare to picture_digest SEI if 238 | * present in seis. seis may be NULL. Hash is printed to stdout, in 239 | * a manner suitable for the status line. Theformat is: 240 | * [Hash_type:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx,(yyy)] 241 | * Where, x..x is the hash 242 | * yyy has the following meanings: 243 | * OK - calculated hash matches the SEI message 244 | * ***ERROR*** - calculated hash does not match the SEI message 245 | * unk - no SEI message was available for comparison 246 | */ 247 | static Void calcAndPrintHashStatus(TComPicYuv& pic, const SEIDecodedPictureHash* pictureHashSEI, const BitDepths &bitDepths, UInt &numChecksumErrors) 248 | { 249 | /* calculate MD5sum for entire reconstructed picture */ 250 | TComPictureHash recon_digest; 251 | Int numChar=0; 252 | const TChar* hashType = "\0"; 253 | 254 | if (pictureHashSEI) 255 | { 256 | switch (pictureHashSEI->method) 257 | { 258 | case HASHTYPE_MD5: 259 | { 260 | hashType = "MD5"; 261 | numChar = calcMD5(pic, recon_digest, bitDepths); 262 | break; 263 | } 264 | case HASHTYPE_CRC: 265 | { 266 | hashType = "CRC"; 267 | numChar = calcCRC(pic, recon_digest, bitDepths); 268 | break; 269 | } 270 | case HASHTYPE_CHECKSUM: 271 | { 272 | hashType = "Checksum"; 273 | numChar = calcChecksum(pic, recon_digest, bitDepths); 274 | break; 275 | } 276 | default: 277 | { 278 | assert (!"unknown hash type"); 279 | break; 280 | } 281 | } 282 | } 283 | 284 | /* compare digest against received version */ 285 | const TChar* ok = "(unk)"; 286 | Bool mismatch = false; 287 | 288 | if (pictureHashSEI) 289 | { 290 | ok = "(OK)"; 291 | if (recon_digest != pictureHashSEI->m_pictureHash) 292 | { 293 | ok = "(***ERROR***)"; 294 | mismatch = true; 295 | } 296 | } 297 | 298 | printf("[%s:%s,%s] ", hashType, hashToString(recon_digest, numChar).c_str(), ok); 299 | 300 | if (mismatch) 301 | { 302 | numChecksumErrors++; 303 | printf("[rx%s:%s] ", hashType, hashToString(pictureHashSEI->m_pictureHash, numChar).c_str()); 304 | } 305 | } 306 | //! \} 307 | -------------------------------------------------------------------------------- /hm_source/TEST.py: -------------------------------------------------------------------------------- 1 | import sys 2 | if not hasattr(sys, 'argv'): 3 | sys.argv = [''] 4 | import numpy as np 5 | import tensorflow as tf 6 | import os, time 7 | from SEFCNN import model #网络 8 | 9 | from UTILS import * 10 | 11 | I_MODEL_PATH = r"..........." #模型路径 12 | P_MODEL_PATH = r"..........." 13 | B_MODEL_PATH = r"..........." 14 | 15 | def prepare_test_data(fileOrDir): 16 | original_ycbcr = [] 17 | gt_y = [] 18 | fileName_list = [] 19 | imgCbCr = 0 20 | fileName_list.append(fileOrDir) 21 | imgY = np.reshape(fileOrDir,(1, len(fileOrDir), len(fileOrDir[0]), 1)) 22 | imgY = normalize(imgY) 23 | original_ycbcr.append([imgY, imgCbCr]) 24 | return original_ycbcr, gt_y, fileName_list 25 | 26 | def test_all_ckpt(modelPath, fileOrDir,flags): 27 | tf.reset_default_graph() 28 | 29 | tf.logging.warning(modelPath) 30 | tem = [f for f in os.listdir(modelPath) if 'data' in f] 31 | ckptFiles = sorted([r.split('.data')[0] for r in tem]) 32 | config = tf.ConfigProto() 33 | config.gpu_options.allow_growth = True 34 | with tf.Session(config=config) as sess: 35 | input_tensor = tf.placeholder(tf.float32, shape=(1, None, None, 1)) 36 | shared_model = tf.make_template('shared_model', model) 37 | output_tensor, weights = shared_model(input_tensor) 38 | output_tensor = tf.clip_by_value(output_tensor, 0., 1.) 39 | output_tensor = output_tensor * 255 40 | 41 | saver = tf.train.Saver() 42 | sess.run(tf.global_variables_initializer()) 43 | 44 | original_ycbcr, gt_y, fileName_list = prepare_test_data(fileOrDir) 45 | 46 | for ckpt in ckptFiles: 47 | epoch = int(ckpt.split('_')[-1].split('.')[0]) 48 | if flags==0: 49 | if epoch != 555: 50 | continue 51 | elif flags==1: 52 | if epoch!= 555: 53 | continue 54 | else: 55 | if epoch != 555: 56 | continue 57 | 58 | tf.logging.warning("epoch:%d\t"%epoch) 59 | saver.restore(sess,os.path.join(modelPath,ckpt)) 60 | total_imgs = len(fileName_list) 61 | for i in range(total_imgs): 62 | imgY = original_ycbcr[i][0] 63 | out = sess.run(output_tensor, feed_dict={input_tensor: imgY}) 64 | out = np.reshape(out, (out.shape[1], out.shape[2])) 65 | out = np.around(out) 66 | out = out.astype('int') 67 | out = out.tolist() 68 | return out 69 | 70 | def modelI(inp): 71 | tf.logging.warning("python, in I") 72 | i = test_all_ckpt(I_MODEL_PATH, inp,0) 73 | return i 74 | def modelP(inp): 75 | tf.logging.warning("python, in p") 76 | p = test_all_ckpt(P_MODEL_PATH, inp,1) 77 | return p 78 | def modelB(inp): 79 | tf.logging.warning("python, in B") 80 | b = test_all_ckpt(B_MODEL_PATH, inp,2) 81 | return b 82 | -------------------------------------------------------------------------------- /hm_source/TEncGOP.cpp: -------------------------------------------------------------------------------- 1 | /* The copyright in this software is being made available under the BSD 2 | * License, included below. This software may be subject to other third party 3 | * and contributor rights, including patent rights, and no such rights are 4 | * granted under this license. 5 | * 6 | * Copyright (c) 2010-2016, ITU/ISO/IEC 7 | * All rights reserved. 8 | * 9 | * Redistribution and use in source and binary forms, with or without 10 | * modification, are permitted provided that the following conditions are met: 11 | * 12 | * * Redistributions of source code must retain the above copyright notice, 13 | * this list of conditions and the following disclaimer. 14 | * * Redistributions in binary form must reproduce the above copyright notice, 15 | * this list of conditions and the following disclaimer in the documentation 16 | * and/or other materials provided with the distribution. 17 | * * Neither the name of the ITU/ISO/IEC nor the names of its contributors may 18 | * be used to endorse or promote products derived from this software without 19 | * specific prior written permission. 20 | * 21 | * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 22 | * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 23 | * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 24 | * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS 25 | * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR 26 | * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF 27 | * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS 28 | * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN 29 | * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) 30 | * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF 31 | * THE POSSIBILITY OF SUCH DAMAGE. 32 | */ 33 | 34 | /** \file TEncGOP.cpp 35 | \brief GOP encoder class 36 | */ 37 | 38 | #include 39 | #include 40 | #include 41 | 42 | #include "TEncTop.h" 43 | #include "TEncGOP.h" 44 | #include "TEncAnalyze.h" 45 | #include "libmd5/MD5.h" 46 | #include "TLibCommon/SEI.h" 47 | #include "TLibCommon/NAL.h" 48 | #include "NALwrite.h" 49 | #include 50 | #include 51 | #include 52 | #include 53 | #include 54 | #include 55 | #include 56 | 57 | 58 | using namespace std; 59 | 60 | 61 | 62 | //extern FILE* fpt; 63 | extern void conv_y_to_mat(cv::Mat &YYY, unsigned char* pY, int nWidth, int nHeight, int bit_depth); 64 | extern cv::Mat ItoCaffe(cv::Mat img); 65 | extern cv::Mat PtoCaffe(cv::Mat img); 66 | extern cv::Mat BtoCaffe(cv::Mat img); 67 | 68 | //! \ingroup TLibEncoder 69 | //! \{ 70 | 71 | // ==================================================================================================================== 72 | // Constructor / destructor / initialization / destroy 73 | // ==================================================================================================================== 74 | Int getLSB(Int poc, Int maxLSB) 75 | { 76 | if (poc >= 0) 77 | { 78 | return poc % maxLSB; 79 | } 80 | else 81 | { 82 | return (maxLSB - ((-poc) % maxLSB)) % maxLSB; 83 | } 84 | } 85 | 86 | TEncGOP::TEncGOP() 87 | { 88 | m_iLastIDR = 0; 89 | m_iGopSize = 0; 90 | m_iNumPicCoded = 0; //Niko 91 | m_bFirst = true; 92 | m_iLastRecoveryPicPOC = 0; 93 | 94 | m_pcCfg = NULL; 95 | m_pcSliceEncoder = NULL; 96 | m_pcListPic = NULL; 97 | 98 | m_pcEntropyCoder = NULL; 99 | m_pcCavlcCoder = NULL; 100 | m_pcSbacCoder = NULL; 101 | m_pcBinCABAC = NULL; 102 | 103 | m_bSeqFirst = true; 104 | 105 | m_bRefreshPending = 0; 106 | m_pocCRA = 0; 107 | m_numLongTermRefPicSPS = 0; 108 | ::memset(m_ltRefPicPocLsbSps, 0, sizeof(m_ltRefPicPocLsbSps)); 109 | ::memset(m_ltRefPicUsedByCurrPicFlag, 0, sizeof(m_ltRefPicUsedByCurrPicFlag)); 110 | m_lastBPSEI = 0; 111 | m_bufferingPeriodSEIPresentInAU = false; 112 | m_associatedIRAPType = NAL_UNIT_CODED_SLICE_IDR_N_LP; 113 | m_associatedIRAPPOC = 0; 114 | #if W0038_DB_OPT 115 | m_pcDeblockingTempPicYuv = NULL; 116 | #endif 117 | return; 118 | } 119 | 120 | TEncGOP::~TEncGOP() 121 | { 122 | } 123 | 124 | /** Create list to contain pointers to CTU start addresses of slice. 125 | */ 126 | Void TEncGOP::create() 127 | { 128 | m_bLongtermTestPictureHasBeenCoded = 0; 129 | m_bLongtermTestPictureHasBeenCoded2 = 0; 130 | } 131 | 132 | Void TEncGOP::destroy() 133 | { 134 | #if W0038_DB_OPT 135 | if (m_pcDeblockingTempPicYuv) 136 | { 137 | m_pcDeblockingTempPicYuv->destroy(); 138 | delete m_pcDeblockingTempPicYuv; 139 | m_pcDeblockingTempPicYuv = NULL; 140 | } 141 | #endif 142 | } 143 | 144 | Void TEncGOP::init ( TEncTop* pcTEncTop ) 145 | { 146 | m_pcEncTop = pcTEncTop; 147 | m_pcCfg = pcTEncTop; 148 | m_seiEncoder.init(m_pcCfg, pcTEncTop, this); 149 | m_pcSliceEncoder = pcTEncTop->getSliceEncoder(); 150 | m_pcListPic = pcTEncTop->getListPic(); 151 | 152 | m_pcEntropyCoder = pcTEncTop->getEntropyCoder(); 153 | m_pcCavlcCoder = pcTEncTop->getCavlcCoder(); 154 | m_pcSbacCoder = pcTEncTop->getSbacCoder(); 155 | m_pcBinCABAC = pcTEncTop->getBinCABAC(); 156 | m_pcLoopFilter = pcTEncTop->getLoopFilter(); 157 | 158 | m_pcSAO = pcTEncTop->getSAO(); 159 | m_pcRateCtrl = pcTEncTop->getRateCtrl(); 160 | m_lastBPSEI = 0; 161 | m_totalCoded = 0; 162 | 163 | } 164 | 165 | Int TEncGOP::xWriteVPS (AccessUnit &accessUnit, const TComVPS *vps) 166 | { 167 | OutputNALUnit nalu(NAL_UNIT_VPS); 168 | m_pcEntropyCoder->setBitstream(&nalu.m_Bitstream); 169 | m_pcEntropyCoder->encodeVPS(vps); 170 | accessUnit.push_back(new NALUnitEBSP(nalu)); 171 | return (Int)(accessUnit.back()->m_nalUnitData.str().size()) * 8; 172 | } 173 | 174 | Int TEncGOP::xWriteSPS (AccessUnit &accessUnit, const TComSPS *sps) 175 | { 176 | OutputNALUnit nalu(NAL_UNIT_SPS); 177 | m_pcEntropyCoder->setBitstream(&nalu.m_Bitstream); 178 | m_pcEntropyCoder->encodeSPS(sps); 179 | accessUnit.push_back(new NALUnitEBSP(nalu)); 180 | return (Int)(accessUnit.back()->m_nalUnitData.str().size()) * 8; 181 | 182 | } 183 | 184 | Int TEncGOP::xWritePPS (AccessUnit &accessUnit, const TComPPS *pps) 185 | { 186 | OutputNALUnit nalu(NAL_UNIT_PPS); 187 | m_pcEntropyCoder->setBitstream(&nalu.m_Bitstream); 188 | m_pcEntropyCoder->encodePPS(pps); 189 | accessUnit.push_back(new NALUnitEBSP(nalu)); 190 | return (Int)(accessUnit.back()->m_nalUnitData.str().size()) * 8; 191 | } 192 | 193 | 194 | Int TEncGOP::xWriteParameterSets (AccessUnit &accessUnit, TComSlice *slice) 195 | { 196 | Int actualTotalBits = 0; 197 | 198 | actualTotalBits += xWriteVPS(accessUnit, m_pcEncTop->getVPS()); 199 | actualTotalBits += xWriteSPS(accessUnit, slice->getSPS()); 200 | actualTotalBits += xWritePPS(accessUnit, slice->getPPS()); 201 | 202 | return actualTotalBits; 203 | } 204 | 205 | Void TEncGOP::xWriteAccessUnitDelimiter (AccessUnit &accessUnit, TComSlice *slice) 206 | { 207 | AUDWriter audWriter; 208 | OutputNALUnit nalu(NAL_UNIT_ACCESS_UNIT_DELIMITER); 209 | 210 | Int picType = slice->isIntra() ? 0 : (slice->isInterP() ? 1 : 2); 211 | 212 | audWriter.codeAUD(nalu.m_Bitstream, picType); 213 | accessUnit.push_front(new NALUnitEBSP(nalu)); 214 | } 215 | 216 | // write SEI list into one NAL unit and add it to the Access unit at auPos 217 | Void TEncGOP::xWriteSEI (NalUnitType naluType, SEIMessages& seiMessages, AccessUnit &accessUnit, AccessUnit::iterator &auPos, Int temporalId, const TComSPS *sps) 218 | { 219 | // don't do anything, if we get an empty list 220 | if (seiMessages.empty()) 221 | { 222 | return; 223 | } 224 | OutputNALUnit nalu(naluType, temporalId); 225 | m_seiWriter.writeSEImessages(nalu.m_Bitstream, seiMessages, sps, false); 226 | auPos = accessUnit.insert(auPos, new NALUnitEBSP(nalu)); 227 | auPos++; 228 | } 229 | 230 | Void TEncGOP::xWriteSEISeparately (NalUnitType naluType, SEIMessages& seiMessages, AccessUnit &accessUnit, AccessUnit::iterator &auPos, Int temporalId, const TComSPS *sps) 231 | { 232 | // don't do anything, if we get an empty list 233 | if (seiMessages.empty()) 234 | { 235 | return; 236 | } 237 | for (SEIMessages::const_iterator sei = seiMessages.begin(); sei!=seiMessages.end(); sei++ ) 238 | { 239 | SEIMessages tmpMessages; 240 | tmpMessages.push_back(*sei); 241 | OutputNALUnit nalu(naluType, temporalId); 242 | m_seiWriter.writeSEImessages(nalu.m_Bitstream, tmpMessages, sps, false); 243 | auPos = accessUnit.insert(auPos, new NALUnitEBSP(nalu)); 244 | auPos++; 245 | } 246 | } 247 | 248 | Void TEncGOP::xClearSEIs(SEIMessages& seiMessages, Bool deleteMessages) 249 | { 250 | if (deleteMessages) 251 | { 252 | deleteSEIs(seiMessages); 253 | } 254 | else 255 | { 256 | seiMessages.clear(); 257 | } 258 | } 259 | 260 | // write SEI messages as separate NAL units ordered 261 | Void TEncGOP::xWriteLeadingSEIOrdered (SEIMessages& seiMessages, SEIMessages& duInfoSeiMessages, AccessUnit &accessUnit, Int temporalId, const TComSPS *sps, Bool testWrite) 262 | { 263 | AccessUnit::iterator itNalu = accessUnit.begin(); 264 | 265 | while ( (itNalu!=accessUnit.end())&& 266 | ( (*itNalu)->m_nalUnitType==NAL_UNIT_ACCESS_UNIT_DELIMITER 267 | || (*itNalu)->m_nalUnitType==NAL_UNIT_VPS 268 | || (*itNalu)->m_nalUnitType==NAL_UNIT_SPS 269 | || (*itNalu)->m_nalUnitType==NAL_UNIT_PPS 270 | )) 271 | { 272 | itNalu++; 273 | } 274 | 275 | SEIMessages localMessages = seiMessages; 276 | SEIMessages currentMessages; 277 | 278 | #if ENC_DEC_TRACE 279 | g_HLSTraceEnable = !testWrite; 280 | #endif 281 | // The case that a specific SEI is not present is handled in xWriteSEI (empty list) 282 | 283 | // Active parameter sets SEI must always be the first SEI 284 | currentMessages = extractSeisByType(localMessages, SEI::ACTIVE_PARAMETER_SETS); 285 | assert (currentMessages.size() <= 1); 286 | xWriteSEI(NAL_UNIT_PREFIX_SEI, currentMessages, accessUnit, itNalu, temporalId, sps); 287 | xClearSEIs(currentMessages, !testWrite); 288 | 289 | // Buffering period SEI must always be following active parameter sets 290 | currentMessages = extractSeisByType(localMessages, SEI::BUFFERING_PERIOD); 291 | assert (currentMessages.size() <= 1); 292 | xWriteSEI(NAL_UNIT_PREFIX_SEI, currentMessages, accessUnit, itNalu, temporalId, sps); 293 | xClearSEIs(currentMessages, !testWrite); 294 | 295 | // Picture timing SEI must always be following buffering period 296 | currentMessages = extractSeisByType(localMessages, SEI::PICTURE_TIMING); 297 | assert (currentMessages.size() <= 1); 298 | xWriteSEI(NAL_UNIT_PREFIX_SEI, currentMessages, accessUnit, itNalu, temporalId, sps); 299 | xClearSEIs(currentMessages, !testWrite); 300 | 301 | // Decoding unit info SEI must always be following picture timing 302 | if (!duInfoSeiMessages.empty()) 303 | { 304 | currentMessages.push_back(duInfoSeiMessages.front()); 305 | if (!testWrite) 306 | { 307 | duInfoSeiMessages.pop_front(); 308 | } 309 | xWriteSEI(NAL_UNIT_PREFIX_SEI, currentMessages, accessUnit, itNalu, temporalId, sps); 310 | xClearSEIs(currentMessages, !testWrite); 311 | } 312 | 313 | // Scalable nesting SEI must always be the following DU info 314 | currentMessages = extractSeisByType(localMessages, SEI::SCALABLE_NESTING); 315 | xWriteSEISeparately(NAL_UNIT_PREFIX_SEI, currentMessages, accessUnit, itNalu, temporalId, sps); 316 | xClearSEIs(currentMessages, !testWrite); 317 | 318 | // And finally everything else one by one 319 | xWriteSEISeparately(NAL_UNIT_PREFIX_SEI, localMessages, accessUnit, itNalu, temporalId, sps); 320 | xClearSEIs(localMessages, !testWrite); 321 | 322 | if (!testWrite) 323 | { 324 | seiMessages.clear(); 325 | } 326 | } 327 | 328 | 329 | Void TEncGOP::xWriteLeadingSEIMessages (SEIMessages& seiMessages, SEIMessages& duInfoSeiMessages, AccessUnit &accessUnit, Int temporalId, const TComSPS *sps, std::deque &duData) 330 | { 331 | AccessUnit testAU; 332 | SEIMessages picTimingSEIs = getSeisByType(seiMessages, SEI::PICTURE_TIMING); 333 | assert (picTimingSEIs.size() < 2); 334 | SEIPictureTiming * picTiming = picTimingSEIs.empty() ? NULL : (SEIPictureTiming*) picTimingSEIs.front(); 335 | 336 | // test writing 337 | xWriteLeadingSEIOrdered(seiMessages, duInfoSeiMessages, testAU, temporalId, sps, true); 338 | // update Timing and DU info SEI 339 | xUpdateDuData(testAU, duData); 340 | xUpdateTimingSEI(picTiming, duData, sps); 341 | xUpdateDuInfoSEI(duInfoSeiMessages, picTiming); 342 | // actual writing 343 | xWriteLeadingSEIOrdered(seiMessages, duInfoSeiMessages, accessUnit, temporalId, sps, false); 344 | 345 | // testAU will automatically be cleaned up when losing scope 346 | } 347 | 348 | Void TEncGOP::xWriteTrailingSEIMessages (SEIMessages& seiMessages, AccessUnit &accessUnit, Int temporalId, const TComSPS *sps) 349 | { 350 | // Note: using accessUnit.end() works only as long as this function is called after slice coding and before EOS/EOB NAL units 351 | AccessUnit::iterator pos = accessUnit.end(); 352 | xWriteSEISeparately(NAL_UNIT_SUFFIX_SEI, seiMessages, accessUnit, pos, temporalId, sps); 353 | deleteSEIs(seiMessages); 354 | } 355 | 356 | Void TEncGOP::xWriteDuSEIMessages (SEIMessages& duInfoSeiMessages, AccessUnit &accessUnit, Int temporalId, const TComSPS *sps, std::deque &duData) 357 | { 358 | const TComHRD *hrd = sps->getVuiParameters()->getHrdParameters(); 359 | 360 | if( m_pcCfg->getDecodingUnitInfoSEIEnabled() && hrd->getSubPicCpbParamsPresentFlag() ) 361 | { 362 | Int naluIdx = 0; 363 | AccessUnit::iterator nalu = accessUnit.begin(); 364 | 365 | // skip over first DU, we have a DU info SEI there already 366 | while (naluIdx < duData[0].accumNalsDU && nalu!=accessUnit.end()) 367 | { 368 | naluIdx++; 369 | nalu++; 370 | } 371 | 372 | SEIMessages::iterator duSEI = duInfoSeiMessages.begin(); 373 | // loop over remaining DUs 374 | for (Int duIdx = 1; duIdx < duData.size(); duIdx++) 375 | { 376 | if (duSEI == duInfoSeiMessages.end()) 377 | { 378 | // if the number of generated SEIs matches the number of DUs, this should not happen 379 | assert (false); 380 | return; 381 | } 382 | // write the next SEI 383 | SEIMessages tmpSEI; 384 | tmpSEI.push_back(*duSEI); 385 | xWriteSEI(NAL_UNIT_PREFIX_SEI, tmpSEI, accessUnit, nalu, temporalId, sps); 386 | // nalu points to the position after the SEI, so we have to increase the index as well 387 | naluIdx++; 388 | while ((naluIdx < duData[duIdx].accumNalsDU) && nalu!=accessUnit.end()) 389 | { 390 | naluIdx++; 391 | nalu++; 392 | } 393 | duSEI++; 394 | } 395 | } 396 | deleteSEIs(duInfoSeiMessages); 397 | } 398 | 399 | 400 | Void TEncGOP::xCreateIRAPLeadingSEIMessages (SEIMessages& seiMessages, const TComSPS *sps, const TComPPS *pps) 401 | { 402 | OutputNALUnit nalu(NAL_UNIT_PREFIX_SEI); 403 | 404 | if(m_pcCfg->getActiveParameterSetsSEIEnabled()) 405 | { 406 | SEIActiveParameterSets *sei = new SEIActiveParameterSets; 407 | m_seiEncoder.initSEIActiveParameterSets (sei, m_pcCfg->getVPS(), sps); 408 | seiMessages.push_back(sei); 409 | } 410 | 411 | if(m_pcCfg->getFramePackingArrangementSEIEnabled()) 412 | { 413 | SEIFramePacking *sei = new SEIFramePacking; 414 | m_seiEncoder.initSEIFramePacking (sei, m_iNumPicCoded); 415 | seiMessages.push_back(sei); 416 | } 417 | 418 | if(m_pcCfg->getSegmentedRectFramePackingArrangementSEIEnabled()) 419 | { 420 | SEISegmentedRectFramePacking *sei = new SEISegmentedRectFramePacking; 421 | m_seiEncoder.initSEISegmentedRectFramePacking(sei); 422 | seiMessages.push_back(sei); 423 | } 424 | 425 | if (m_pcCfg->getDisplayOrientationSEIAngle()) 426 | { 427 | SEIDisplayOrientation *sei = new SEIDisplayOrientation; 428 | m_seiEncoder.initSEIDisplayOrientation(sei); 429 | seiMessages.push_back(sei); 430 | } 431 | 432 | if(m_pcCfg->getToneMappingInfoSEIEnabled()) 433 | { 434 | SEIToneMappingInfo *sei = new SEIToneMappingInfo; 435 | m_seiEncoder.initSEIToneMappingInfo (sei); 436 | seiMessages.push_back(sei); 437 | } 438 | 439 | if(m_pcCfg->getTMCTSSEIEnabled()) 440 | { 441 | SEITempMotionConstrainedTileSets *sei = new SEITempMotionConstrainedTileSets; 442 | m_seiEncoder.initSEITempMotionConstrainedTileSets(sei, pps); 443 | seiMessages.push_back(sei); 444 | } 445 | 446 | if(m_pcCfg->getTimeCodeSEIEnabled()) 447 | { 448 | SEITimeCode *seiTimeCode = new SEITimeCode; 449 | m_seiEncoder.initSEITimeCode(seiTimeCode); 450 | seiMessages.push_back(seiTimeCode); 451 | } 452 | 453 | if(m_pcCfg->getKneeSEIEnabled()) 454 | { 455 | SEIKneeFunctionInfo *sei = new SEIKneeFunctionInfo; 456 | m_seiEncoder.initSEIKneeFunctionInfo(sei); 457 | seiMessages.push_back(sei); 458 | } 459 | 460 | if(m_pcCfg->getMasteringDisplaySEI().colourVolumeSEIEnabled) 461 | { 462 | const TComSEIMasteringDisplay &seiCfg=m_pcCfg->getMasteringDisplaySEI(); 463 | SEIMasteringDisplayColourVolume *sei = new SEIMasteringDisplayColourVolume; 464 | sei->values = seiCfg; 465 | seiMessages.push_back(sei); 466 | } 467 | if(m_pcCfg->getChromaResamplingFilterHintEnabled()) 468 | { 469 | SEIChromaResamplingFilterHint *seiChromaResamplingFilterHint = new SEIChromaResamplingFilterHint; 470 | m_seiEncoder.initSEIChromaResamplingFilterHint(seiChromaResamplingFilterHint, m_pcCfg->getChromaResamplingHorFilterIdc(), m_pcCfg->getChromaResamplingVerFilterIdc()); 471 | seiMessages.push_back(seiChromaResamplingFilterHint); 472 | } 473 | #if U0033_ALTERNATIVE_TRANSFER_CHARACTERISTICS_SEI 474 | if(m_pcCfg->getSEIAlternativeTransferCharacteristicsSEIEnable()) 475 | { 476 | SEIAlternativeTransferCharacteristics *seiAlternativeTransferCharacteristics = new SEIAlternativeTransferCharacteristics; 477 | m_seiEncoder.initSEIAlternativeTransferCharacteristics(seiAlternativeTransferCharacteristics); 478 | seiMessages.push_back(seiAlternativeTransferCharacteristics); 479 | } 480 | #endif 481 | } 482 | 483 | Void TEncGOP::xCreatePerPictureSEIMessages (Int picInGOP, SEIMessages& seiMessages, SEIMessages& nestedSeiMessages, TComSlice *slice) 484 | { 485 | if( ( m_pcCfg->getBufferingPeriodSEIEnabled() ) && ( slice->getSliceType() == I_SLICE ) && 486 | ( slice->getSPS()->getVuiParametersPresentFlag() ) && 487 | ( ( slice->getSPS()->getVuiParameters()->getHrdParameters()->getNalHrdParametersPresentFlag() ) 488 | || ( slice->getSPS()->getVuiParameters()->getHrdParameters()->getVclHrdParametersPresentFlag() ) ) ) 489 | { 490 | SEIBufferingPeriod *bufferingPeriodSEI = new SEIBufferingPeriod(); 491 | m_seiEncoder.initSEIBufferingPeriod(bufferingPeriodSEI, slice); 492 | seiMessages.push_back(bufferingPeriodSEI); 493 | m_bufferingPeriodSEIPresentInAU = true; 494 | 495 | if (m_pcCfg->getScalableNestingSEIEnabled()) 496 | { 497 | SEIBufferingPeriod *bufferingPeriodSEIcopy = new SEIBufferingPeriod(); 498 | bufferingPeriodSEI->copyTo(*bufferingPeriodSEIcopy); 499 | nestedSeiMessages.push_back(bufferingPeriodSEIcopy); 500 | } 501 | } 502 | 503 | if (picInGOP ==0 && m_pcCfg->getSOPDescriptionSEIEnabled() ) // write SOP description SEI (if enabled) at the beginning of GOP 504 | { 505 | SEISOPDescription* sopDescriptionSEI = new SEISOPDescription(); 506 | m_seiEncoder.initSEISOPDescription(sopDescriptionSEI, slice, picInGOP, m_iLastIDR, m_iGopSize); 507 | seiMessages.push_back(sopDescriptionSEI); 508 | } 509 | 510 | if( ( m_pcEncTop->getRecoveryPointSEIEnabled() ) && ( slice->getSliceType() == I_SLICE ) ) 511 | { 512 | if( m_pcEncTop->getGradualDecodingRefreshInfoEnabled() && !slice->getRapPicFlag() ) 513 | { 514 | // Gradual decoding refresh SEI 515 | SEIGradualDecodingRefreshInfo *gradualDecodingRefreshInfoSEI = new SEIGradualDecodingRefreshInfo(); 516 | gradualDecodingRefreshInfoSEI->m_gdrForegroundFlag = true; // Indicating all "foreground" 517 | seiMessages.push_back(gradualDecodingRefreshInfoSEI); 518 | } 519 | // Recovery point SEI 520 | SEIRecoveryPoint *recoveryPointSEI = new SEIRecoveryPoint(); 521 | m_seiEncoder.initSEIRecoveryPoint(recoveryPointSEI, slice); 522 | seiMessages.push_back(recoveryPointSEI); 523 | } 524 | if (m_pcCfg->getTemporalLevel0IndexSEIEnabled()) 525 | { 526 | SEITemporalLevel0Index *temporalLevel0IndexSEI = new SEITemporalLevel0Index(); 527 | m_seiEncoder.initTemporalLevel0IndexSEI(temporalLevel0IndexSEI, slice); 528 | seiMessages.push_back(temporalLevel0IndexSEI); 529 | } 530 | 531 | if( m_pcEncTop->getNoDisplaySEITLayer() && ( slice->getTLayer() >= m_pcEncTop->getNoDisplaySEITLayer() ) ) 532 | { 533 | SEINoDisplay *seiNoDisplay = new SEINoDisplay; 534 | seiNoDisplay->m_noDisplay = true; 535 | seiMessages.push_back(seiNoDisplay); 536 | } 537 | 538 | // insert one Colour Remapping Info SEI for the picture (if the file exists) 539 | if (!m_pcCfg->getColourRemapInfoSEIFileRoot().empty()) 540 | { 541 | SEIColourRemappingInfo *seiColourRemappingInfo = new SEIColourRemappingInfo(); 542 | const Bool success = m_seiEncoder.initSEIColourRemappingInfo(seiColourRemappingInfo, slice->getPOC() ); 543 | 544 | if(success) 545 | { 546 | seiMessages.push_back(seiColourRemappingInfo); 547 | } 548 | else 549 | { 550 | delete seiColourRemappingInfo; 551 | } 552 | } 553 | } 554 | 555 | Void TEncGOP::xCreateScalableNestingSEI (SEIMessages& seiMessages, SEIMessages& nestedSeiMessages) 556 | { 557 | SEIMessages tmpMessages; 558 | while (!nestedSeiMessages.empty()) 559 | { 560 | SEI* sei=nestedSeiMessages.front(); 561 | nestedSeiMessages.pop_front(); 562 | tmpMessages.push_back(sei); 563 | SEIScalableNesting *nestingSEI = new SEIScalableNesting(); 564 | m_seiEncoder.initSEIScalableNesting(nestingSEI, tmpMessages); 565 | seiMessages.push_back(nestingSEI); 566 | tmpMessages.clear(); 567 | } 568 | } 569 | 570 | Void TEncGOP::xCreatePictureTimingSEI (Int IRAPGOPid, SEIMessages& seiMessages, SEIMessages& nestedSeiMessages, SEIMessages& duInfoSeiMessages, TComSlice *slice, Bool isField, std::deque &duData) 571 | { 572 | const TComVUI *vui = slice->getSPS()->getVuiParameters(); 573 | const TComHRD *hrd = vui->getHrdParameters(); 574 | 575 | // update decoding unit parameters 576 | if( ( m_pcCfg->getPictureTimingSEIEnabled() || m_pcCfg->getDecodingUnitInfoSEIEnabled() ) && 577 | ( slice->getSPS()->getVuiParametersPresentFlag() ) && 578 | ( hrd->getNalHrdParametersPresentFlag() || hrd->getVclHrdParametersPresentFlag() ) ) 579 | { 580 | Int picSptDpbOutputDuDelay = 0; 581 | SEIPictureTiming *pictureTimingSEI = new SEIPictureTiming(); 582 | 583 | // DU parameters 584 | if( hrd->getSubPicCpbParamsPresentFlag() ) 585 | { 586 | UInt numDU = (UInt) duData.size(); 587 | pictureTimingSEI->m_numDecodingUnitsMinus1 = ( numDU - 1 ); 588 | pictureTimingSEI->m_duCommonCpbRemovalDelayFlag = false; 589 | pictureTimingSEI->m_numNalusInDuMinus1.resize( numDU ); 590 | pictureTimingSEI->m_duCpbRemovalDelayMinus1.resize( numDU ); 591 | } 592 | pictureTimingSEI->m_auCpbRemovalDelay = std::min(std::max(1, m_totalCoded - m_lastBPSEI), static_cast(pow(2, static_cast(hrd->getCpbRemovalDelayLengthMinus1()+1)))); // Syntax element signalled as minus, hence the . 593 | pictureTimingSEI->m_picDpbOutputDelay = slice->getSPS()->getNumReorderPics(slice->getSPS()->getMaxTLayers()-1) + slice->getPOC() - m_totalCoded; 594 | if(m_pcCfg->getEfficientFieldIRAPEnabled() && IRAPGOPid > 0 && IRAPGOPid < m_iGopSize) 595 | { 596 | // if pictures have been swapped there is likely one more picture delay on their tid. Very rough approximation 597 | pictureTimingSEI->m_picDpbOutputDelay ++; 598 | } 599 | Int factor = hrd->getTickDivisorMinus2() + 2; 600 | pictureTimingSEI->m_picDpbOutputDuDelay = factor * pictureTimingSEI->m_picDpbOutputDelay; 601 | if( m_pcCfg->getDecodingUnitInfoSEIEnabled() ) 602 | { 603 | picSptDpbOutputDuDelay = factor * pictureTimingSEI->m_picDpbOutputDelay; 604 | } 605 | if (m_bufferingPeriodSEIPresentInAU) 606 | { 607 | m_lastBPSEI = m_totalCoded; 608 | } 609 | 610 | if( hrd->getSubPicCpbParamsPresentFlag() ) 611 | { 612 | Int i; 613 | UInt64 ui64Tmp; 614 | UInt uiPrev = 0; 615 | UInt numDU = ( pictureTimingSEI->m_numDecodingUnitsMinus1 + 1 ); 616 | std::vector &rDuCpbRemovalDelayMinus1 = pictureTimingSEI->m_duCpbRemovalDelayMinus1; 617 | UInt maxDiff = ( hrd->getTickDivisorMinus2() + 2 ) - 1; 618 | 619 | for( i = 0; i < numDU; i ++ ) 620 | { 621 | pictureTimingSEI->m_numNalusInDuMinus1[ i ] = ( i == 0 ) ? ( duData[i].accumNalsDU - 1 ) : ( duData[i].accumNalsDU- duData[i-1].accumNalsDU - 1 ); 622 | } 623 | 624 | if( numDU == 1 ) 625 | { 626 | rDuCpbRemovalDelayMinus1[ 0 ] = 0; /* don't care */ 627 | } 628 | else 629 | { 630 | rDuCpbRemovalDelayMinus1[ numDU - 1 ] = 0;/* by definition */ 631 | UInt tmp = 0; 632 | UInt accum = 0; 633 | 634 | for( i = ( numDU - 2 ); i >= 0; i -- ) 635 | { 636 | ui64Tmp = ( ( ( duData[numDU - 1].accumBitsDU - duData[i].accumBitsDU ) * ( vui->getTimingInfo()->getTimeScale() / vui->getTimingInfo()->getNumUnitsInTick() ) * ( hrd->getTickDivisorMinus2() + 2 ) ) / ( m_pcCfg->getTargetBitrate() ) ); 637 | if( (UInt)ui64Tmp > maxDiff ) 638 | { 639 | tmp ++; 640 | } 641 | } 642 | uiPrev = 0; 643 | 644 | UInt flag = 0; 645 | for( i = ( numDU - 2 ); i >= 0; i -- ) 646 | { 647 | flag = 0; 648 | ui64Tmp = ( ( ( duData[numDU - 1].accumBitsDU - duData[i].accumBitsDU ) * ( vui->getTimingInfo()->getTimeScale() / vui->getTimingInfo()->getNumUnitsInTick() ) * ( hrd->getTickDivisorMinus2() + 2 ) ) / ( m_pcCfg->getTargetBitrate() ) ); 649 | 650 | if( (UInt)ui64Tmp > maxDiff ) 651 | { 652 | if(uiPrev >= maxDiff - tmp) 653 | { 654 | ui64Tmp = uiPrev + 1; 655 | flag = 1; 656 | } 657 | else ui64Tmp = maxDiff - tmp + 1; 658 | } 659 | rDuCpbRemovalDelayMinus1[ i ] = (UInt)ui64Tmp - uiPrev - 1; 660 | if( (Int)rDuCpbRemovalDelayMinus1[ i ] < 0 ) 661 | { 662 | rDuCpbRemovalDelayMinus1[ i ] = 0; 663 | } 664 | else if (tmp > 0 && flag == 1) 665 | { 666 | tmp --; 667 | } 668 | accum += rDuCpbRemovalDelayMinus1[ i ] + 1; 669 | uiPrev = accum; 670 | } 671 | } 672 | } 673 | 674 | if( m_pcCfg->getPictureTimingSEIEnabled() ) 675 | { 676 | pictureTimingSEI->m_picStruct = (isField && slice->getPic()->isTopField())? 1 : isField? 2 : 0; 677 | seiMessages.push_back(pictureTimingSEI); 678 | 679 | if ( m_pcCfg->getScalableNestingSEIEnabled() ) // put picture timing SEI into scalable nesting SEI 680 | { 681 | SEIPictureTiming *pictureTimingSEIcopy = new SEIPictureTiming(); 682 | pictureTimingSEI->copyTo(*pictureTimingSEIcopy); 683 | nestedSeiMessages.push_back(pictureTimingSEIcopy); 684 | } 685 | } 686 | 687 | if( m_pcCfg->getDecodingUnitInfoSEIEnabled() && hrd->getSubPicCpbParamsPresentFlag() ) 688 | { 689 | for( Int i = 0; i < ( pictureTimingSEI->m_numDecodingUnitsMinus1 + 1 ); i ++ ) 690 | { 691 | SEIDecodingUnitInfo *duInfoSEI = new SEIDecodingUnitInfo(); 692 | duInfoSEI->m_decodingUnitIdx = i; 693 | duInfoSEI->m_duSptCpbRemovalDelay = pictureTimingSEI->m_duCpbRemovalDelayMinus1[i] + 1; 694 | duInfoSEI->m_dpbOutputDuDelayPresentFlag = false; 695 | duInfoSEI->m_picSptDpbOutputDuDelay = picSptDpbOutputDuDelay; 696 | 697 | duInfoSeiMessages.push_back(duInfoSEI); 698 | } 699 | } 700 | 701 | if( !m_pcCfg->getPictureTimingSEIEnabled() && pictureTimingSEI ) 702 | { 703 | delete pictureTimingSEI; 704 | } 705 | } 706 | } 707 | 708 | Void TEncGOP::xUpdateDuData(AccessUnit &testAU, std::deque &duData) 709 | { 710 | if (duData.empty()) 711 | { 712 | return; 713 | } 714 | // fix first 715 | UInt numNalUnits = (UInt)testAU.size(); 716 | UInt numRBSPBytes = 0; 717 | for (AccessUnit::const_iterator it = testAU.begin(); it != testAU.end(); it++) 718 | { 719 | numRBSPBytes += UInt((*it)->m_nalUnitData.str().size()); 720 | } 721 | duData[0].accumBitsDU += ( numRBSPBytes << 3 ); 722 | duData[0].accumNalsDU += numNalUnits; 723 | 724 | // adapt cumulative sums for all following DUs 725 | // and add one DU info SEI, if enabled 726 | for (Int i=1; igetDecodingUnitInfoSEIEnabled()) 729 | { 730 | numNalUnits += 1; 731 | numRBSPBytes += ( 5 << 3 ); 732 | } 733 | duData[i].accumBitsDU += numRBSPBytes; // probably around 5 bytes 734 | duData[i].accumNalsDU += numNalUnits; 735 | } 736 | 737 | // The last DU may have a trailing SEI 738 | if (m_pcCfg->getDecodedPictureHashSEIType()!=HASHTYPE_NONE) 739 | { 740 | duData.back().accumBitsDU += ( 20 << 3 ); // probably around 20 bytes - should be further adjusted, e.g. by type 741 | duData.back().accumNalsDU += 1; 742 | } 743 | 744 | } 745 | Void TEncGOP::xUpdateTimingSEI(SEIPictureTiming *pictureTimingSEI, std::deque &duData, const TComSPS *sps) 746 | { 747 | if (!pictureTimingSEI) 748 | { 749 | return; 750 | } 751 | const TComVUI *vui = sps->getVuiParameters(); 752 | const TComHRD *hrd = vui->getHrdParameters(); 753 | if( hrd->getSubPicCpbParamsPresentFlag() ) 754 | { 755 | Int i; 756 | UInt64 ui64Tmp; 757 | UInt uiPrev = 0; 758 | UInt numDU = ( pictureTimingSEI->m_numDecodingUnitsMinus1 + 1 ); 759 | std::vector &rDuCpbRemovalDelayMinus1 = pictureTimingSEI->m_duCpbRemovalDelayMinus1; 760 | UInt maxDiff = ( hrd->getTickDivisorMinus2() + 2 ) - 1; 761 | 762 | for( i = 0; i < numDU; i ++ ) 763 | { 764 | pictureTimingSEI->m_numNalusInDuMinus1[ i ] = ( i == 0 ) ? ( duData[i].accumNalsDU - 1 ) : ( duData[i].accumNalsDU- duData[i-1].accumNalsDU - 1 ); 765 | } 766 | 767 | if( numDU == 1 ) 768 | { 769 | rDuCpbRemovalDelayMinus1[ 0 ] = 0; /* don't care */ 770 | } 771 | else 772 | { 773 | rDuCpbRemovalDelayMinus1[ numDU - 1 ] = 0;/* by definition */ 774 | UInt tmp = 0; 775 | UInt accum = 0; 776 | 777 | for( i = ( numDU - 2 ); i >= 0; i -- ) 778 | { 779 | ui64Tmp = ( ( ( duData[numDU - 1].accumBitsDU - duData[i].accumBitsDU ) * ( vui->getTimingInfo()->getTimeScale() / vui->getTimingInfo()->getNumUnitsInTick() ) * ( hrd->getTickDivisorMinus2() + 2 ) ) / ( m_pcCfg->getTargetBitrate() ) ); 780 | if( (UInt)ui64Tmp > maxDiff ) 781 | { 782 | tmp ++; 783 | } 784 | } 785 | uiPrev = 0; 786 | 787 | UInt flag = 0; 788 | for( i = ( numDU - 2 ); i >= 0; i -- ) 789 | { 790 | flag = 0; 791 | ui64Tmp = ( ( ( duData[numDU - 1].accumBitsDU - duData[i].accumBitsDU ) * ( vui->getTimingInfo()->getTimeScale() / vui->getTimingInfo()->getNumUnitsInTick() ) * ( hrd->getTickDivisorMinus2() + 2 ) ) / ( m_pcCfg->getTargetBitrate() ) ); 792 | 793 | if( (UInt)ui64Tmp > maxDiff ) 794 | { 795 | if(uiPrev >= maxDiff - tmp) 796 | { 797 | ui64Tmp = uiPrev + 1; 798 | flag = 1; 799 | } 800 | else ui64Tmp = maxDiff - tmp + 1; 801 | } 802 | rDuCpbRemovalDelayMinus1[ i ] = (UInt)ui64Tmp - uiPrev - 1; 803 | if( (Int)rDuCpbRemovalDelayMinus1[ i ] < 0 ) 804 | { 805 | rDuCpbRemovalDelayMinus1[ i ] = 0; 806 | } 807 | else if (tmp > 0 && flag == 1) 808 | { 809 | tmp --; 810 | } 811 | accum += rDuCpbRemovalDelayMinus1[ i ] + 1; 812 | uiPrev = accum; 813 | } 814 | } 815 | } 816 | } 817 | Void TEncGOP::xUpdateDuInfoSEI(SEIMessages &duInfoSeiMessages, SEIPictureTiming *pictureTimingSEI) 818 | { 819 | if (duInfoSeiMessages.empty() || (pictureTimingSEI == NULL)) 820 | { 821 | return; 822 | } 823 | 824 | Int i=0; 825 | 826 | for (SEIMessages::iterator du = duInfoSeiMessages.begin(); du!= duInfoSeiMessages.end(); du++) 827 | { 828 | SEIDecodingUnitInfo *duInfoSEI = (SEIDecodingUnitInfo*) (*du); 829 | duInfoSEI->m_decodingUnitIdx = i; 830 | duInfoSEI->m_duSptCpbRemovalDelay = pictureTimingSEI->m_duCpbRemovalDelayMinus1[i] + 1; 831 | duInfoSEI->m_dpbOutputDuDelayPresentFlag = false; 832 | i++; 833 | } 834 | } 835 | 836 | static Void 837 | cabac_zero_word_padding(TComSlice *const pcSlice, TComPic *const pcPic, const std::size_t binCountsInNalUnits, const std::size_t numBytesInVclNalUnits, std::ostringstream &nalUnitData, const Bool cabacZeroWordPaddingEnabled) 838 | { 839 | const TComSPS &sps=*(pcSlice->getSPS()); 840 | const Int log2subWidthCxsubHeightC = (pcPic->getComponentScaleX(COMPONENT_Cb)+pcPic->getComponentScaleY(COMPONENT_Cb)); 841 | const Int minCuWidth = pcPic->getMinCUWidth(); 842 | const Int minCuHeight = pcPic->getMinCUHeight(); 843 | const Int paddedWidth = ((sps.getPicWidthInLumaSamples() + minCuWidth - 1) / minCuWidth) * minCuWidth; 844 | const Int paddedHeight= ((sps.getPicHeightInLumaSamples() + minCuHeight - 1) / minCuHeight) * minCuHeight; 845 | const Int rawBits = paddedWidth * paddedHeight * 846 | (sps.getBitDepth(CHANNEL_TYPE_LUMA) + 2*(sps.getBitDepth(CHANNEL_TYPE_CHROMA)>>log2subWidthCxsubHeightC)); 847 | const std::size_t threshold = (32/3)*numBytesInVclNalUnits + (rawBits/32); 848 | if (binCountsInNalUnits >= threshold) 849 | { 850 | // need to add additional cabac zero words (each one accounts for 3 bytes (=00 00 03)) to increase numBytesInVclNalUnits 851 | const std::size_t targetNumBytesInVclNalUnits = ((binCountsInNalUnits - (rawBits/32))*3+31)/32; 852 | 853 | if (targetNumBytesInVclNalUnits>numBytesInVclNalUnits) // It should be! 854 | { 855 | const std::size_t numberOfAdditionalBytesNeeded=targetNumBytesInVclNalUnits - numBytesInVclNalUnits; 856 | const std::size_t numberOfAdditionalCabacZeroWords=(numberOfAdditionalBytesNeeded+2)/3; 857 | const std::size_t numberOfAdditionalCabacZeroBytes=numberOfAdditionalCabacZeroWords*3; 858 | if (cabacZeroWordPaddingEnabled) 859 | { 860 | std::vector zeroBytesPadding(numberOfAdditionalCabacZeroBytes, UChar(0)); 861 | for(std::size_t i=0; i(&(zeroBytesPadding[0])), numberOfAdditionalCabacZeroBytes); 866 | printf("Adding %d bytes of padding\n", UInt(numberOfAdditionalCabacZeroWords*3)); 867 | } 868 | else 869 | { 870 | printf("Standard would normally require adding %d bytes of padding\n", UInt(numberOfAdditionalCabacZeroWords*3)); 871 | } 872 | } 873 | } 874 | } 875 | 876 | class EfficientFieldIRAPMapping 877 | { 878 | private: 879 | Int IRAPGOPid; 880 | Bool IRAPtoReorder; 881 | Bool swapIRAPForward; 882 | 883 | public: 884 | EfficientFieldIRAPMapping() : 885 | IRAPGOPid(-1), 886 | IRAPtoReorder(false), 887 | swapIRAPForward(false) 888 | { } 889 | 890 | Void initialize(const Bool isField, const Int gopSize, const Int POCLast, const Int numPicRcvd, const Int lastIDR, TEncGOP *pEncGop, TEncCfg *pCfg); 891 | 892 | Int adjustGOPid(const Int gopID); 893 | Int restoreGOPid(const Int gopID); 894 | Int GetIRAPGOPid() const { return IRAPGOPid; } 895 | }; 896 | 897 | Void EfficientFieldIRAPMapping::initialize(const Bool isField, const Int gopSize, const Int POCLast, const Int numPicRcvd, const Int lastIDR, TEncGOP *pEncGop, TEncCfg *pCfg ) 898 | { 899 | if(isField) 900 | { 901 | Int pocCurr; 902 | for ( Int iGOPid=0; iGOPid < gopSize; iGOPid++ ) 903 | { 904 | // determine actual POC 905 | if(POCLast == 0) //case first frame or first top field 906 | { 907 | pocCurr=0; 908 | } 909 | else if(POCLast == 1 && isField) //case first bottom field, just like the first frame, the poc computation is not right anymore, we set the right value 910 | { 911 | pocCurr = 1; 912 | } 913 | else 914 | { 915 | pocCurr = POCLast - numPicRcvd + pCfg->getGOPEntry(iGOPid).m_POC - isField; 916 | } 917 | 918 | // check if POC corresponds to IRAP 919 | NalUnitType tmpUnitType = pEncGop->getNalUnitType(pocCurr, lastIDR, isField); 920 | if(tmpUnitType >= NAL_UNIT_CODED_SLICE_BLA_W_LP && tmpUnitType <= NAL_UNIT_CODED_SLICE_CRA) // if picture is an IRAP 921 | { 922 | if(pocCurr%2 == 0 && iGOPid < gopSize-1 && pCfg->getGOPEntry(iGOPid).m_POC == pCfg->getGOPEntry(iGOPid+1).m_POC-1) 923 | { // if top field and following picture in enc order is associated bottom field 924 | IRAPGOPid = iGOPid; 925 | IRAPtoReorder = true; 926 | swapIRAPForward = true; 927 | break; 928 | } 929 | if(pocCurr%2 != 0 && iGOPid > 0 && pCfg->getGOPEntry(iGOPid).m_POC == pCfg->getGOPEntry(iGOPid-1).m_POC+1) 930 | { 931 | // if picture is an IRAP remember to process it first 932 | IRAPGOPid = iGOPid; 933 | IRAPtoReorder = true; 934 | swapIRAPForward = false; 935 | break; 936 | } 937 | } 938 | } 939 | } 940 | } 941 | 942 | Int EfficientFieldIRAPMapping::adjustGOPid(const Int GOPid) 943 | { 944 | if(IRAPtoReorder) 945 | { 946 | if(swapIRAPForward) 947 | { 948 | if(GOPid == IRAPGOPid) 949 | { 950 | return IRAPGOPid +1; 951 | } 952 | else if(GOPid == IRAPGOPid +1) 953 | { 954 | return IRAPGOPid; 955 | } 956 | } 957 | else 958 | { 959 | if(GOPid == IRAPGOPid -1) 960 | { 961 | return IRAPGOPid; 962 | } 963 | else if(GOPid == IRAPGOPid) 964 | { 965 | return IRAPGOPid -1; 966 | } 967 | } 968 | } 969 | return GOPid; 970 | } 971 | 972 | Int EfficientFieldIRAPMapping::restoreGOPid(const Int GOPid) 973 | { 974 | if(IRAPtoReorder) 975 | { 976 | if(swapIRAPForward) 977 | { 978 | if(GOPid == IRAPGOPid) 979 | { 980 | IRAPtoReorder = false; 981 | return IRAPGOPid +1; 982 | } 983 | else if(GOPid == IRAPGOPid +1) 984 | { 985 | return GOPid -1; 986 | } 987 | } 988 | else 989 | { 990 | if(GOPid == IRAPGOPid) 991 | { 992 | return IRAPGOPid -1; 993 | } 994 | else if(GOPid == IRAPGOPid -1) 995 | { 996 | IRAPtoReorder = false; 997 | return IRAPGOPid; 998 | } 999 | } 1000 | } 1001 | return GOPid; 1002 | } 1003 | 1004 | 1005 | static UInt calculateCollocatedFromL1Flag(TEncCfg *pCfg, const Int GOPid, const Int gopSize) 1006 | { 1007 | Int iCloseLeft=1, iCloseRight=-1; 1008 | for(Int i = 0; igetGOPEntry(GOPid).m_numRefPics; i++) 1009 | { 1010 | Int iRef = pCfg->getGOPEntry(GOPid).m_referencePics[i]; 1011 | if(iRef>0&&(iRefiCloseLeft||iCloseLeft==1)) 1016 | { 1017 | iCloseLeft=iRef; 1018 | } 1019 | } 1020 | if(iCloseRight>-1) 1021 | { 1022 | iCloseRight=iCloseRight+pCfg->getGOPEntry(GOPid).m_POC-1; 1023 | } 1024 | if(iCloseLeft<1) 1025 | { 1026 | iCloseLeft=iCloseLeft+pCfg->getGOPEntry(GOPid).m_POC-1; 1027 | while(iCloseLeft<0) 1028 | { 1029 | iCloseLeft+=gopSize; 1030 | } 1031 | } 1032 | Int iLeftQP=0, iRightQP=0; 1033 | for(Int i=0; igetGOPEntry(i).m_POC==(iCloseLeft%gopSize)+1) 1036 | { 1037 | iLeftQP= pCfg->getGOPEntry(i).m_QPOffset; 1038 | } 1039 | if (pCfg->getGOPEntry(i).m_POC==(iCloseRight%gopSize)+1) 1040 | { 1041 | iRightQP=pCfg->getGOPEntry(i).m_QPOffset; 1042 | } 1043 | } 1044 | if(iCloseRight>-1&&iRightQP& rcListPic, 1091 | TComList& rcListPicYuvRecOut, std::list& accessUnitsInGOP, 1092 | Bool isField, Bool isTff, const InputColourSpaceConversion snr_conversion, const Bool printFrameMSE ) 1093 | { 1094 | // TODO: Split this function up. 1095 | 1096 | TComPic* pcPic = NULL; 1097 | TComPicYuv* pcPicYuvRecOut; 1098 | TComSlice* pcSlice; 1099 | TComOutputBitstream *pcBitstreamRedirect; 1100 | pcBitstreamRedirect = new TComOutputBitstream; 1101 | AccessUnit::iterator itLocationToPushSliceHeaderNALU; // used to store location where NALU containing slice header is to be inserted 1102 | 1103 | xInitGOP( iPOCLast, iNumPicRcvd, isField ); 1104 | 1105 | m_iNumPicCoded = 0; 1106 | SEIMessages leadingSeiMessages; 1107 | SEIMessages nestedSeiMessages; 1108 | SEIMessages duInfoSeiMessages; 1109 | SEIMessages trailingSeiMessages; 1110 | std::deque duData; 1111 | SEIDecodingUnitInfo decodingUnitInfoSEI; 1112 | 1113 | EfficientFieldIRAPMapping effFieldIRAPMap; 1114 | if (m_pcCfg->getEfficientFieldIRAPEnabled()) 1115 | { 1116 | effFieldIRAPMap.initialize(isField, m_iGopSize, iPOCLast, iNumPicRcvd, m_iLastIDR, this, m_pcCfg); 1117 | } 1118 | 1119 | // reset flag indicating whether pictures have been encoded 1120 | for ( Int iGOPid=0; iGOPid < m_iGopSize; iGOPid++ ) 1121 | { 1122 | m_pcCfg->setEncodedFlag(iGOPid, false); 1123 | } 1124 | 1125 | for ( Int iGOPid=0; iGOPid < m_iGopSize; iGOPid++ ) 1126 | { 1127 | if (m_pcCfg->getEfficientFieldIRAPEnabled()) 1128 | { 1129 | iGOPid=effFieldIRAPMap.adjustGOPid(iGOPid); 1130 | } 1131 | 1132 | //-- For time output for each slice 1133 | clock_t iBeforeTime = clock(); 1134 | 1135 | UInt uiColDir = calculateCollocatedFromL1Flag(m_pcCfg, iGOPid, m_iGopSize); 1136 | 1137 | /////////////////////////////////////////////////////////////////////////////////////////////////// Initial to start encoding 1138 | Int iTimeOffset; 1139 | Int pocCurr; 1140 | 1141 | if(iPOCLast == 0) //case first frame or first top field 1142 | { 1143 | pocCurr=0; 1144 | iTimeOffset = 1; 1145 | } 1146 | else if(iPOCLast == 1 && isField) //case first bottom field, just like the first frame, the poc computation is not right anymore, we set the right value 1147 | { 1148 | pocCurr = 1; 1149 | iTimeOffset = 1; 1150 | } 1151 | else 1152 | { 1153 | pocCurr = iPOCLast - iNumPicRcvd + m_pcCfg->getGOPEntry(iGOPid).m_POC - ((isField && m_iGopSize>1) ? 1:0); 1154 | iTimeOffset = m_pcCfg->getGOPEntry(iGOPid).m_POC; 1155 | } 1156 | 1157 | if(pocCurr>=m_pcCfg->getFramesToBeEncoded()) 1158 | { 1159 | if (m_pcCfg->getEfficientFieldIRAPEnabled()) 1160 | { 1161 | iGOPid=effFieldIRAPMap.restoreGOPid(iGOPid); 1162 | } 1163 | continue; 1164 | } 1165 | 1166 | if( getNalUnitType(pocCurr, m_iLastIDR, isField) == NAL_UNIT_CODED_SLICE_IDR_W_RADL || getNalUnitType(pocCurr, m_iLastIDR, isField) == NAL_UNIT_CODED_SLICE_IDR_N_LP ) 1167 | { 1168 | m_iLastIDR = pocCurr; 1169 | } 1170 | // start a new access unit: create an entry in the list of output access units 1171 | accessUnitsInGOP.push_back(AccessUnit()); 1172 | AccessUnit& accessUnit = accessUnitsInGOP.back(); 1173 | xGetBuffer( rcListPic, rcListPicYuvRecOut, iNumPicRcvd, iTimeOffset, pcPic, pcPicYuvRecOut, pocCurr, isField ); 1174 | 1175 | // Slice data initialization 1176 | pcPic->clearSliceBuffer(); 1177 | pcPic->allocateNewSlice(); 1178 | m_pcSliceEncoder->setSliceIdx(0); 1179 | pcPic->setCurrSliceIdx(0); 1180 | 1181 | m_pcSliceEncoder->initEncSlice ( pcPic, iPOCLast, pocCurr, iGOPid, pcSlice, isField ); 1182 | 1183 | //Set Frame/Field coding 1184 | pcSlice->getPic()->setField(isField); 1185 | 1186 | pcSlice->setLastIDR(m_iLastIDR); 1187 | pcSlice->setSliceIdx(0); 1188 | //set default slice level flag to the same as SPS level flag 1189 | pcSlice->setLFCrossSliceBoundaryFlag( pcSlice->getPPS()->getLoopFilterAcrossSlicesEnabledFlag() ); 1190 | 1191 | if(pcSlice->getSliceType()==B_SLICE&&m_pcCfg->getGOPEntry(iGOPid).m_sliceType=='P') 1192 | { 1193 | pcSlice->setSliceType(P_SLICE); 1194 | } 1195 | if(pcSlice->getSliceType()==B_SLICE&&m_pcCfg->getGOPEntry(iGOPid).m_sliceType=='I') 1196 | { 1197 | pcSlice->setSliceType(I_SLICE); 1198 | } 1199 | 1200 | // Set the nal unit type 1201 | pcSlice->setNalUnitType(getNalUnitType(pocCurr, m_iLastIDR, isField)); 1202 | if(pcSlice->getTemporalLayerNonReferenceFlag()) 1203 | { 1204 | if (pcSlice->getNalUnitType() == NAL_UNIT_CODED_SLICE_TRAIL_R && 1205 | !(m_iGopSize == 1 && pcSlice->getSliceType() == I_SLICE)) 1206 | // Add this condition to avoid POC issues with encoder_intra_main.cfg configuration (see #1127 in bug tracker) 1207 | { 1208 | pcSlice->setNalUnitType(NAL_UNIT_CODED_SLICE_TRAIL_N); 1209 | } 1210 | if(pcSlice->getNalUnitType()==NAL_UNIT_CODED_SLICE_RADL_R) 1211 | { 1212 | pcSlice->setNalUnitType(NAL_UNIT_CODED_SLICE_RADL_N); 1213 | } 1214 | if(pcSlice->getNalUnitType()==NAL_UNIT_CODED_SLICE_RASL_R) 1215 | { 1216 | pcSlice->setNalUnitType(NAL_UNIT_CODED_SLICE_RASL_N); 1217 | } 1218 | } 1219 | 1220 | if (m_pcCfg->getEfficientFieldIRAPEnabled()) 1221 | { 1222 | if ( pcSlice->getNalUnitType() == NAL_UNIT_CODED_SLICE_BLA_W_LP 1223 | || pcSlice->getNalUnitType() == NAL_UNIT_CODED_SLICE_BLA_W_RADL 1224 | || pcSlice->getNalUnitType() == NAL_UNIT_CODED_SLICE_BLA_N_LP 1225 | || pcSlice->getNalUnitType() == NAL_UNIT_CODED_SLICE_IDR_W_RADL 1226 | || pcSlice->getNalUnitType() == NAL_UNIT_CODED_SLICE_IDR_N_LP 1227 | || pcSlice->getNalUnitType() == NAL_UNIT_CODED_SLICE_CRA ) // IRAP picture 1228 | { 1229 | m_associatedIRAPType = pcSlice->getNalUnitType(); 1230 | m_associatedIRAPPOC = pocCurr; 1231 | } 1232 | pcSlice->setAssociatedIRAPType(m_associatedIRAPType); 1233 | pcSlice->setAssociatedIRAPPOC(m_associatedIRAPPOC); 1234 | } 1235 | // Do decoding refresh marking if any 1236 | pcSlice->decodingRefreshMarking(m_pocCRA, m_bRefreshPending, rcListPic, m_pcCfg->getEfficientFieldIRAPEnabled()); 1237 | m_pcEncTop->selectReferencePictureSet(pcSlice, pocCurr, iGOPid); 1238 | if (!m_pcCfg->getEfficientFieldIRAPEnabled()) 1239 | { 1240 | if ( pcSlice->getNalUnitType() == NAL_UNIT_CODED_SLICE_BLA_W_LP 1241 | || pcSlice->getNalUnitType() == NAL_UNIT_CODED_SLICE_BLA_W_RADL 1242 | || pcSlice->getNalUnitType() == NAL_UNIT_CODED_SLICE_BLA_N_LP 1243 | || pcSlice->getNalUnitType() == NAL_UNIT_CODED_SLICE_IDR_W_RADL 1244 | || pcSlice->getNalUnitType() == NAL_UNIT_CODED_SLICE_IDR_N_LP 1245 | || pcSlice->getNalUnitType() == NAL_UNIT_CODED_SLICE_CRA ) // IRAP picture 1246 | { 1247 | m_associatedIRAPType = pcSlice->getNalUnitType(); 1248 | m_associatedIRAPPOC = pocCurr; 1249 | } 1250 | pcSlice->setAssociatedIRAPType(m_associatedIRAPType); 1251 | pcSlice->setAssociatedIRAPPOC(m_associatedIRAPPOC); 1252 | } 1253 | 1254 | if ((pcSlice->checkThatAllRefPicsAreAvailable(rcListPic, pcSlice->getRPS(), false, m_iLastRecoveryPicPOC, m_pcCfg->getDecodingRefreshType() == 3) != 0) || (pcSlice->isIRAP()) 1255 | || (m_pcCfg->getEfficientFieldIRAPEnabled() && isField && pcSlice->getAssociatedIRAPType() >= NAL_UNIT_CODED_SLICE_BLA_W_LP && pcSlice->getAssociatedIRAPType() <= NAL_UNIT_CODED_SLICE_CRA && pcSlice->getAssociatedIRAPPOC() == pcSlice->getPOC()+1) 1256 | ) 1257 | { 1258 | pcSlice->createExplicitReferencePictureSetFromReference(rcListPic, pcSlice->getRPS(), pcSlice->isIRAP(), m_iLastRecoveryPicPOC, m_pcCfg->getDecodingRefreshType() == 3, m_pcCfg->getEfficientFieldIRAPEnabled()); 1259 | } 1260 | 1261 | pcSlice->applyReferencePictureSet(rcListPic, pcSlice->getRPS()); 1262 | 1263 | if(pcSlice->getTLayer() > 0 1264 | && !( pcSlice->getNalUnitType() == NAL_UNIT_CODED_SLICE_RADL_N // Check if not a leading picture 1265 | || pcSlice->getNalUnitType() == NAL_UNIT_CODED_SLICE_RADL_R 1266 | || pcSlice->getNalUnitType() == NAL_UNIT_CODED_SLICE_RASL_N 1267 | || pcSlice->getNalUnitType() == NAL_UNIT_CODED_SLICE_RASL_R ) 1268 | ) 1269 | { 1270 | if(pcSlice->isTemporalLayerSwitchingPoint(rcListPic) || pcSlice->getSPS()->getTemporalIdNestingFlag()) 1271 | { 1272 | if(pcSlice->getTemporalLayerNonReferenceFlag()) 1273 | { 1274 | pcSlice->setNalUnitType(NAL_UNIT_CODED_SLICE_TSA_N); 1275 | } 1276 | else 1277 | { 1278 | pcSlice->setNalUnitType(NAL_UNIT_CODED_SLICE_TSA_R); 1279 | } 1280 | } 1281 | else if(pcSlice->isStepwiseTemporalLayerSwitchingPointCandidate(rcListPic)) 1282 | { 1283 | Bool isSTSA=true; 1284 | for(Int ii=iGOPid+1;(iigetGOPSize() && isSTSA==true);ii++) 1285 | { 1286 | Int lTid= m_pcCfg->getGOPEntry(ii).m_temporalId; 1287 | if(lTid==pcSlice->getTLayer()) 1288 | { 1289 | const TComReferencePictureSet* nRPS = pcSlice->getSPS()->getRPSList()->getReferencePictureSet(ii); 1290 | for(Int jj=0;jjgetNumberOfPictures();jj++) 1291 | { 1292 | if(nRPS->getUsed(jj)) 1293 | { 1294 | Int tPoc=m_pcCfg->getGOPEntry(ii).m_POC+nRPS->getDeltaPOC(jj); 1295 | Int kk=0; 1296 | for(kk=0;kkgetGOPSize();kk++) 1297 | { 1298 | if(m_pcCfg->getGOPEntry(kk).m_POC==tPoc) 1299 | { 1300 | break; 1301 | } 1302 | } 1303 | Int tTid=m_pcCfg->getGOPEntry(kk).m_temporalId; 1304 | if(tTid >= pcSlice->getTLayer()) 1305 | { 1306 | isSTSA=false; 1307 | break; 1308 | } 1309 | } 1310 | } 1311 | } 1312 | } 1313 | if(isSTSA==true) 1314 | { 1315 | if(pcSlice->getTemporalLayerNonReferenceFlag()) 1316 | { 1317 | pcSlice->setNalUnitType(NAL_UNIT_CODED_SLICE_STSA_N); 1318 | } 1319 | else 1320 | { 1321 | pcSlice->setNalUnitType(NAL_UNIT_CODED_SLICE_STSA_R); 1322 | } 1323 | } 1324 | } 1325 | } 1326 | arrangeLongtermPicturesInRPS(pcSlice, rcListPic); 1327 | TComRefPicListModification* refPicListModification = pcSlice->getRefPicListModification(); 1328 | refPicListModification->setRefPicListModificationFlagL0(0); 1329 | refPicListModification->setRefPicListModificationFlagL1(0); 1330 | pcSlice->setNumRefIdx(REF_PIC_LIST_0,min(m_pcCfg->getGOPEntry(iGOPid).m_numRefPicsActive,pcSlice->getRPS()->getNumberOfPictures())); 1331 | pcSlice->setNumRefIdx(REF_PIC_LIST_1,min(m_pcCfg->getGOPEntry(iGOPid).m_numRefPicsActive,pcSlice->getRPS()->getNumberOfPictures())); 1332 | 1333 | // Set reference list 1334 | pcSlice->setRefPicList ( rcListPic ); 1335 | 1336 | // Slice info. refinement 1337 | if ( (pcSlice->getSliceType() == B_SLICE) && (pcSlice->getNumRefIdx(REF_PIC_LIST_1) == 0) ) 1338 | { 1339 | pcSlice->setSliceType ( P_SLICE ); 1340 | } 1341 | pcSlice->setEncCABACTableIdx(m_pcSliceEncoder->getEncCABACTableIdx()); 1342 | 1343 | if (pcSlice->getSliceType() == B_SLICE) 1344 | { 1345 | pcSlice->setColFromL0Flag(1-uiColDir); 1346 | Bool bLowDelay = true; 1347 | Int iCurrPOC = pcSlice->getPOC(); 1348 | Int iRefIdx = 0; 1349 | 1350 | for (iRefIdx = 0; iRefIdx < pcSlice->getNumRefIdx(REF_PIC_LIST_0) && bLowDelay; iRefIdx++) 1351 | { 1352 | if ( pcSlice->getRefPic(REF_PIC_LIST_0, iRefIdx)->getPOC() > iCurrPOC ) 1353 | { 1354 | bLowDelay = false; 1355 | } 1356 | } 1357 | for (iRefIdx = 0; iRefIdx < pcSlice->getNumRefIdx(REF_PIC_LIST_1) && bLowDelay; iRefIdx++) 1358 | { 1359 | if ( pcSlice->getRefPic(REF_PIC_LIST_1, iRefIdx)->getPOC() > iCurrPOC ) 1360 | { 1361 | bLowDelay = false; 1362 | } 1363 | } 1364 | 1365 | pcSlice->setCheckLDC(bLowDelay); 1366 | } 1367 | else 1368 | { 1369 | pcSlice->setCheckLDC(true); 1370 | } 1371 | 1372 | uiColDir = 1-uiColDir; 1373 | 1374 | //------------------------------------------------------------- 1375 | pcSlice->setRefPOCList(); 1376 | 1377 | pcSlice->setList1IdxToList0Idx(); 1378 | 1379 | if (m_pcEncTop->getTMVPModeId() == 2) 1380 | { 1381 | if (iGOPid == 0) // first picture in SOP (i.e. forward B) 1382 | { 1383 | pcSlice->setEnableTMVPFlag(0); 1384 | } 1385 | else 1386 | { 1387 | // Note: pcSlice->getColFromL0Flag() is assumed to be always 0 and getcolRefIdx() is always 0. 1388 | pcSlice->setEnableTMVPFlag(1); 1389 | } 1390 | } 1391 | else if (m_pcEncTop->getTMVPModeId() == 1) 1392 | { 1393 | pcSlice->setEnableTMVPFlag(1); 1394 | } 1395 | else 1396 | { 1397 | pcSlice->setEnableTMVPFlag(0); 1398 | } 1399 | 1400 | // set adaptive search range for non-intra-slices 1401 | if (m_pcCfg->getUseASR() && pcSlice->getSliceType()!=I_SLICE) 1402 | { 1403 | m_pcSliceEncoder->setSearchRange(pcSlice); 1404 | } 1405 | 1406 | Bool bGPBcheck=false; 1407 | if ( pcSlice->getSliceType() == B_SLICE) 1408 | { 1409 | if ( pcSlice->getNumRefIdx(RefPicList( 0 ) ) == pcSlice->getNumRefIdx(RefPicList( 1 ) ) ) 1410 | { 1411 | bGPBcheck=true; 1412 | Int i; 1413 | for ( i=0; i < pcSlice->getNumRefIdx(RefPicList( 1 ) ); i++ ) 1414 | { 1415 | if ( pcSlice->getRefPOC(RefPicList(1), i) != pcSlice->getRefPOC(RefPicList(0), i) ) 1416 | { 1417 | bGPBcheck=false; 1418 | break; 1419 | } 1420 | } 1421 | } 1422 | } 1423 | if(bGPBcheck) 1424 | { 1425 | pcSlice->setMvdL1ZeroFlag(true); 1426 | } 1427 | else 1428 | { 1429 | pcSlice->setMvdL1ZeroFlag(false); 1430 | } 1431 | pcPic->getSlice(pcSlice->getSliceIdx())->setMvdL1ZeroFlag(pcSlice->getMvdL1ZeroFlag()); 1432 | 1433 | 1434 | Double lambda = 0.0; 1435 | Int actualHeadBits = 0; 1436 | Int actualTotalBits = 0; 1437 | Int estimatedBits = 0; 1438 | Int tmpBitsBeforeWriting = 0; 1439 | if ( m_pcCfg->getUseRateCtrl() ) // TODO: does this work with multiple slices and slice-segments? 1440 | { 1441 | Int frameLevel = m_pcRateCtrl->getRCSeq()->getGOPID2Level( iGOPid ); 1442 | if ( pcPic->getSlice(0)->getSliceType() == I_SLICE ) 1443 | { 1444 | frameLevel = 0; 1445 | } 1446 | m_pcRateCtrl->initRCPic( frameLevel ); 1447 | estimatedBits = m_pcRateCtrl->getRCPic()->getTargetBits(); 1448 | 1449 | #if U0132_TARGET_BITS_SATURATION 1450 | if (m_pcRateCtrl->getCpbSaturationEnabled() && frameLevel != 0) 1451 | { 1452 | Int estimatedCpbFullness = m_pcRateCtrl->getCpbState() + m_pcRateCtrl->getBufferingRate(); 1453 | 1454 | // prevent overflow 1455 | if (estimatedCpbFullness - estimatedBits > (Int)(m_pcRateCtrl->getCpbSize()*0.9f)) 1456 | { 1457 | estimatedBits = estimatedCpbFullness - (Int)(m_pcRateCtrl->getCpbSize()*0.9f); 1458 | } 1459 | 1460 | estimatedCpbFullness -= m_pcRateCtrl->getBufferingRate(); 1461 | // prevent underflow 1462 | #if V0078_ADAPTIVE_LOWER_BOUND 1463 | if (estimatedCpbFullness - estimatedBits < m_pcRateCtrl->getRCPic()->getLowerBound()) 1464 | { 1465 | estimatedBits = max(200, estimatedCpbFullness - m_pcRateCtrl->getRCPic()->getLowerBound()); 1466 | } 1467 | #else 1468 | if (estimatedCpbFullness - estimatedBits < (Int)(m_pcRateCtrl->getCpbSize()*0.1f)) 1469 | { 1470 | estimatedBits = max(200, estimatedCpbFullness - (Int)(m_pcRateCtrl->getCpbSize()*0.1f)); 1471 | } 1472 | #endif 1473 | 1474 | m_pcRateCtrl->getRCPic()->setTargetBits(estimatedBits); 1475 | } 1476 | #endif 1477 | 1478 | Int sliceQP = m_pcCfg->getInitialQP(); 1479 | if ( ( pcSlice->getPOC() == 0 && m_pcCfg->getInitialQP() > 0 ) || ( frameLevel == 0 && m_pcCfg->getForceIntraQP() ) ) // QP is specified 1480 | { 1481 | Int NumberBFrames = ( m_pcCfg->getGOPSize() - 1 ); 1482 | Double dLambda_scale = 1.0 - Clip3( 0.0, 0.5, 0.05*(Double)NumberBFrames ); 1483 | Double dQPFactor = 0.57*dLambda_scale; 1484 | Int SHIFT_QP = 12; 1485 | Int bitdepth_luma_qp_scale = 0; 1486 | Double qp_temp = (Double) sliceQP + bitdepth_luma_qp_scale - SHIFT_QP; 1487 | lambda = dQPFactor*pow( 2.0, qp_temp/3.0 ); 1488 | } 1489 | else if ( frameLevel == 0 ) // intra case, but use the model 1490 | { 1491 | m_pcSliceEncoder->calCostSliceI(pcPic); // TODO: This only analyses the first slice segment - what about the others? 1492 | 1493 | if ( m_pcCfg->getIntraPeriod() != 1 ) // do not refine allocated bits for all intra case 1494 | { 1495 | Int bits = m_pcRateCtrl->getRCSeq()->getLeftAverageBits(); 1496 | bits = m_pcRateCtrl->getRCPic()->getRefineBitsForIntra( bits ); 1497 | 1498 | #if U0132_TARGET_BITS_SATURATION 1499 | if (m_pcRateCtrl->getCpbSaturationEnabled() ) 1500 | { 1501 | Int estimatedCpbFullness = m_pcRateCtrl->getCpbState() + m_pcRateCtrl->getBufferingRate(); 1502 | 1503 | // prevent overflow 1504 | if (estimatedCpbFullness - bits > (Int)(m_pcRateCtrl->getCpbSize()*0.9f)) 1505 | { 1506 | bits = estimatedCpbFullness - (Int)(m_pcRateCtrl->getCpbSize()*0.9f); 1507 | } 1508 | 1509 | estimatedCpbFullness -= m_pcRateCtrl->getBufferingRate(); 1510 | // prevent underflow 1511 | #if V0078_ADAPTIVE_LOWER_BOUND 1512 | if (estimatedCpbFullness - bits < m_pcRateCtrl->getRCPic()->getLowerBound()) 1513 | { 1514 | bits = estimatedCpbFullness - m_pcRateCtrl->getRCPic()->getLowerBound(); 1515 | } 1516 | #else 1517 | if (estimatedCpbFullness - bits < (Int)(m_pcRateCtrl->getCpbSize()*0.1f)) 1518 | { 1519 | bits = estimatedCpbFullness - (Int)(m_pcRateCtrl->getCpbSize()*0.1f); 1520 | } 1521 | #endif 1522 | } 1523 | #endif 1524 | 1525 | if ( bits < 200 ) 1526 | { 1527 | bits = 200; 1528 | } 1529 | m_pcRateCtrl->getRCPic()->setTargetBits( bits ); 1530 | } 1531 | 1532 | list listPreviousPicture = m_pcRateCtrl->getPicList(); 1533 | m_pcRateCtrl->getRCPic()->getLCUInitTargetBits(); 1534 | lambda = m_pcRateCtrl->getRCPic()->estimatePicLambda( listPreviousPicture, pcSlice->getSliceType()); 1535 | sliceQP = m_pcRateCtrl->getRCPic()->estimatePicQP( lambda, listPreviousPicture ); 1536 | } 1537 | else // normal case 1538 | { 1539 | list listPreviousPicture = m_pcRateCtrl->getPicList(); 1540 | lambda = m_pcRateCtrl->getRCPic()->estimatePicLambda( listPreviousPicture, pcSlice->getSliceType()); 1541 | sliceQP = m_pcRateCtrl->getRCPic()->estimatePicQP( lambda, listPreviousPicture ); 1542 | } 1543 | 1544 | sliceQP = Clip3( -pcSlice->getSPS()->getQpBDOffset(CHANNEL_TYPE_LUMA), MAX_QP, sliceQP ); 1545 | m_pcRateCtrl->getRCPic()->setPicEstQP( sliceQP ); 1546 | 1547 | m_pcSliceEncoder->resetQP( pcPic, sliceQP, lambda ); 1548 | } 1549 | 1550 | UInt uiNumSliceSegments = 1; 1551 | 1552 | // Allocate some coders, now the number of tiles are known. 1553 | const Int numSubstreamsColumns = (pcSlice->getPPS()->getNumTileColumnsMinus1() + 1); 1554 | const Int numSubstreamRows = pcSlice->getPPS()->getEntropyCodingSyncEnabledFlag() ? pcPic->getFrameHeightInCtus() : (pcSlice->getPPS()->getNumTileRowsMinus1() + 1); 1555 | const Int numSubstreams = numSubstreamRows * numSubstreamsColumns; 1556 | std::vector substreamsOut(numSubstreams); 1557 | 1558 | // now compress (trial encode) the various slice segments (slices, and dependent slices) 1559 | { 1560 | const UInt numberOfCtusInFrame=pcPic->getPicSym()->getNumberOfCtusInFrame(); 1561 | pcSlice->setSliceCurStartCtuTsAddr( 0 ); 1562 | pcSlice->setSliceSegmentCurStartCtuTsAddr( 0 ); 1563 | 1564 | for(UInt nextCtuTsAddr = 0; nextCtuTsAddr < numberOfCtusInFrame; ) 1565 | { 1566 | m_pcSliceEncoder->precompressSlice( pcPic ); 1567 | m_pcSliceEncoder->compressSlice ( pcPic, false, false ); 1568 | 1569 | const UInt curSliceSegmentEnd = pcSlice->getSliceSegmentCurEndCtuTsAddr(); 1570 | if (curSliceSegmentEnd < numberOfCtusInFrame) 1571 | { 1572 | const Bool bNextSegmentIsDependentSlice=curSliceSegmentEndgetSliceCurEndCtuTsAddr(); 1573 | const UInt sliceBits=pcSlice->getSliceBits(); 1574 | pcPic->allocateNewSlice(); 1575 | // prepare for next slice 1576 | pcPic->setCurrSliceIdx ( uiNumSliceSegments ); 1577 | m_pcSliceEncoder->setSliceIdx ( uiNumSliceSegments ); 1578 | pcSlice = pcPic->getSlice ( uiNumSliceSegments ); 1579 | assert(pcSlice->getPPS()!=0); 1580 | pcSlice->copySliceInfo ( pcPic->getSlice(uiNumSliceSegments-1) ); 1581 | pcSlice->setSliceIdx ( uiNumSliceSegments ); 1582 | if (bNextSegmentIsDependentSlice) 1583 | { 1584 | pcSlice->setSliceBits(sliceBits); 1585 | } 1586 | else 1587 | { 1588 | pcSlice->setSliceCurStartCtuTsAddr ( curSliceSegmentEnd ); 1589 | pcSlice->setSliceBits(0); 1590 | } 1591 | pcSlice->setDependentSliceSegmentFlag(bNextSegmentIsDependentSlice); 1592 | pcSlice->setSliceSegmentCurStartCtuTsAddr ( curSliceSegmentEnd ); 1593 | // TODO: optimise cabac_init during compress slice to improve multi-slice operation 1594 | // pcSlice->setEncCABACTableIdx(m_pcSliceEncoder->getEncCABACTableIdx()); 1595 | uiNumSliceSegments ++; 1596 | } 1597 | nextCtuTsAddr = curSliceSegmentEnd; 1598 | } 1599 | } 1600 | 1601 | duData.clear(); 1602 | pcSlice = pcPic->getSlice(0); 1603 | 1604 | // SAO parameter estimation using non-deblocked pixels for CTU bottom and right boundary areas 1605 | if( pcSlice->getSPS()->getUseSAO() && m_pcCfg->getSaoCtuBoundary() ) 1606 | { 1607 | m_pcSAO->getPreDBFStatistics(pcPic); 1608 | } 1609 | 1610 | //-- Loop filter 1611 | Bool bLFCrossTileBoundary = pcSlice->getPPS()->getLoopFilterAcrossTilesEnabledFlag(); 1612 | m_pcLoopFilter->setCfg(bLFCrossTileBoundary); 1613 | if ( m_pcCfg->getDeblockingFilterMetric() ) 1614 | { 1615 | #if W0038_DB_OPT 1616 | if ( m_pcCfg->getDeblockingFilterMetric()==2 ) 1617 | { 1618 | applyDeblockingFilterParameterSelection(pcPic, uiNumSliceSegments, iGOPid); 1619 | } 1620 | else 1621 | { 1622 | #endif 1623 | applyDeblockingFilterMetric(pcPic, uiNumSliceSegments); 1624 | #if W0038_DB_OPT 1625 | } 1626 | #endif 1627 | } 1628 | /////////////////////////////////////////////////////// 1629 | // Take the reconstructed image and send it to the neural network 1630 | // edit by kong 1631 | ////////////////////////////////////////////////////// 1632 | 1633 | 1634 | TComPicYuv* pcPicYuvRec = pcPic->getPicYuvRec(); 1635 | TComSlice* pcPicSlice = pcPic->getSlice(0); 1636 | Pel* pPic = pcPicYuvRec->getAddr(COMPONENT_Y, 0); 1637 | Pel* bkuPic = pPic; 1638 | Pel* pPicFwrite = pPic; 1639 | int buflen = pcPicYuvRec->m_picHeight*pcPicYuvRec->m_picWidth; 1640 | 1641 | 1642 | uint8_t *buf = new uint8_t[buflen]; 1643 | uint8_t *p = buf; 1644 | for (int y = 0; y < pcPicYuvRec->m_picHeight; ++y) { 1645 | for (int x = 0; x < pcPicYuvRec->m_picWidth; ++x) { 1646 | uint8_t uctemp = (unsigned char)(*(pPic + x)); 1647 | *p = uctemp; 1648 | p++; 1649 | } 1650 | pPic += pcPicYuvRec->getStride(COMPONENT_Y); 1651 | } 1652 | cv::Mat yImg, recImg; 1653 | conv_y_to_mat(yImg, buf, pcPicYuvRec->m_picWidth, pcPicYuvRec->m_picHeight, 8); 1654 | 1655 | 1656 | m_pcLoopFilter->loopFilterPic(pcPic); 1657 | 1658 | if (pcPicSlice->getSliceType() == I_SLICE) 1659 | { 1660 | recImg = ItoCaffe(yImg); 1661 | 1662 | 1663 | uchar *temp; 1664 | for (int y = 0; y < pcPicYuvRec->m_picHeight; ++y) { 1665 | temp = recImg.ptr(y); 1666 | for (int x = 0; x < pcPicYuvRec->m_picWidth; ++x) { 1667 | *(bkuPic + x) = (Pel)temp[x]; 1668 | } 1669 | bkuPic += pcPicYuvRec->getStride(COMPONENT_Y); 1670 | } 1671 | } 1672 | 1673 | 1674 | 1675 | 1676 | /////////////////////////////////////////////////////////////////////////////////////////////////// File writing 1677 | // Set entropy coder 1678 | m_pcEntropyCoder->setEntropyCoder ( m_pcCavlcCoder ); 1679 | 1680 | if ( m_bSeqFirst ) 1681 | { 1682 | // write various parameter sets 1683 | actualTotalBits += xWriteParameterSets(accessUnit, pcSlice); 1684 | 1685 | // create prefix SEI messages at the beginning of the sequence 1686 | assert(leadingSeiMessages.empty()); 1687 | xCreateIRAPLeadingSEIMessages(leadingSeiMessages, pcSlice->getSPS(), pcSlice->getPPS()); 1688 | 1689 | m_bSeqFirst = false; 1690 | } 1691 | if (m_pcCfg->getAccessUnitDelimiter()) 1692 | { 1693 | xWriteAccessUnitDelimiter(accessUnit, pcSlice); 1694 | } 1695 | 1696 | // reset presence of BP SEI indication 1697 | m_bufferingPeriodSEIPresentInAU = false; 1698 | // create prefix SEI associated with a picture 1699 | xCreatePerPictureSEIMessages(iGOPid, leadingSeiMessages, nestedSeiMessages, pcSlice); 1700 | 1701 | /* use the main bitstream buffer for storing the marshalled picture */ 1702 | m_pcEntropyCoder->setBitstream(NULL); 1703 | 1704 | pcSlice = pcPic->getSlice(0); 1705 | if (!((pcPic->getPOC() % 2 == 1 || pcPicSlice->getSliceType() == I_SLICE) && pcPic->getPOC() != 1)) 1706 | { 1707 | if (pcSlice->getSPS()->getUseSAO()) 1708 | { 1709 | Bool sliceEnabled[MAX_NUM_COMPONENT]; 1710 | TComBitCounter tempBitCounter; 1711 | tempBitCounter.resetBits(); 1712 | m_pcEncTop->getRDGoOnSbacCoder()->setBitstream(&tempBitCounter); 1713 | m_pcSAO->initRDOCabacCoder(m_pcEncTop->getRDGoOnSbacCoder(), pcSlice); 1714 | #if OPTIONAL_RESET_SAO_ENCODING_AFTER_IRAP 1715 | m_pcSAO->SAOProcess(pcPic, sliceEnabled, pcPic->getSlice(0)->getLambdas(), 1716 | m_pcCfg->getTestSAODisableAtPictureLevel(), 1717 | m_pcCfg->getSaoEncodingRate(), 1718 | m_pcCfg->getSaoEncodingRateChroma(), 1719 | m_pcCfg->getSaoCtuBoundary(), 1720 | m_pcCfg->getSaoResetEncoderStateAfterIRAP()); 1721 | #else 1722 | m_pcSAO->SAOProcess(pcPic, sliceEnabled, pcPic->getSlice(0)->getLambdas(), m_pcCfg->getTestSAODisableAtPictureLevel(), m_pcCfg->getSaoEncodingRate(), m_pcCfg->getSaoEncodingRateChroma(), m_pcCfg->getSaoCtuBoundary()); 1723 | #endif 1724 | m_pcSAO->PCMLFDisableProcess(pcPic); 1725 | m_pcEncTop->getRDGoOnSbacCoder()->setBitstream(NULL); 1726 | 1727 | //assign SAO slice header 1728 | for (Int s = 0; s < uiNumSliceSegments; s++) 1729 | { 1730 | pcPic->getSlice(s)->setSaoEnabledFlag(CHANNEL_TYPE_LUMA, sliceEnabled[COMPONENT_Y]); 1731 | assert(sliceEnabled[COMPONENT_Cb] == sliceEnabled[COMPONENT_Cr]); 1732 | pcPic->getSlice(s)->setSaoEnabledFlag(CHANNEL_TYPE_CHROMA, sliceEnabled[COMPONENT_Cb]); 1733 | } 1734 | } 1735 | } 1736 | // pcSlice is currently slice 0. 1737 | std::size_t binCountsInNalUnits = 0; // For implementation of cabac_zero_word stuffing (section 7.4.3.10) 1738 | std::size_t numBytesInVclNalUnits = 0; // For implementation of cabac_zero_word stuffing (section 7.4.3.10) 1739 | 1740 | for( UInt sliceSegmentStartCtuTsAddr = 0, sliceIdxCount=0; sliceSegmentStartCtuTsAddr < pcPic->getPicSym()->getNumberOfCtusInFrame(); sliceIdxCount++, sliceSegmentStartCtuTsAddr=pcSlice->getSliceSegmentCurEndCtuTsAddr() ) 1741 | { 1742 | pcSlice = pcPic->getSlice(sliceIdxCount); 1743 | if(sliceIdxCount > 0 && pcSlice->getSliceType()!= I_SLICE) 1744 | { 1745 | pcSlice->checkColRefIdx(sliceIdxCount, pcPic); 1746 | } 1747 | pcPic->setCurrSliceIdx(sliceIdxCount); 1748 | m_pcSliceEncoder->setSliceIdx(sliceIdxCount); 1749 | 1750 | pcSlice->setRPS(pcPic->getSlice(0)->getRPS()); 1751 | pcSlice->setRPSidx(pcPic->getSlice(0)->getRPSidx()); 1752 | 1753 | for ( UInt ui = 0 ; ui < numSubstreams; ui++ ) 1754 | { 1755 | substreamsOut[ui].clear(); 1756 | } 1757 | 1758 | m_pcEntropyCoder->setEntropyCoder ( m_pcCavlcCoder ); 1759 | m_pcEntropyCoder->resetEntropy ( pcSlice ); 1760 | /* start slice NALunit */ 1761 | OutputNALUnit nalu( pcSlice->getNalUnitType(), pcSlice->getTLayer() ); 1762 | m_pcEntropyCoder->setBitstream(&nalu.m_Bitstream); 1763 | 1764 | pcSlice->setNoRaslOutputFlag(false); 1765 | if (pcSlice->isIRAP()) 1766 | { 1767 | if (pcSlice->getNalUnitType() >= NAL_UNIT_CODED_SLICE_BLA_W_LP && pcSlice->getNalUnitType() <= NAL_UNIT_CODED_SLICE_IDR_N_LP) 1768 | { 1769 | pcSlice->setNoRaslOutputFlag(true); 1770 | } 1771 | //the inference for NoOutputPriorPicsFlag 1772 | // KJS: This cannot happen at the encoder 1773 | if (!m_bFirst && pcSlice->isIRAP() && pcSlice->getNoRaslOutputFlag()) 1774 | { 1775 | if (pcSlice->getNalUnitType() == NAL_UNIT_CODED_SLICE_CRA) 1776 | { 1777 | pcSlice->setNoOutputPriorPicsFlag(true); 1778 | } 1779 | } 1780 | } 1781 | 1782 | pcSlice->setEncCABACTableIdx(m_pcSliceEncoder->getEncCABACTableIdx()); 1783 | 1784 | tmpBitsBeforeWriting = m_pcEntropyCoder->getNumberOfWrittenBits(); 1785 | m_pcEntropyCoder->encodeSliceHeader(pcSlice); 1786 | actualHeadBits += ( m_pcEntropyCoder->getNumberOfWrittenBits() - tmpBitsBeforeWriting ); 1787 | 1788 | pcSlice->setFinalized(true); 1789 | 1790 | pcSlice->clearSubstreamSizes( ); 1791 | { 1792 | UInt numBinsCoded = 0; 1793 | m_pcSliceEncoder->encodeSlice(pcPic, &(substreamsOut[0]), numBinsCoded); 1794 | binCountsInNalUnits+=numBinsCoded; 1795 | } 1796 | 1797 | { 1798 | // Construct the final bitstream by concatenating substreams. 1799 | // The final bitstream is either nalu.m_Bitstream or pcBitstreamRedirect; 1800 | // Complete the slice header info. 1801 | m_pcEntropyCoder->setEntropyCoder ( m_pcCavlcCoder ); 1802 | m_pcEntropyCoder->setBitstream(&nalu.m_Bitstream); 1803 | m_pcEntropyCoder->encodeTilesWPPEntryPoint( pcSlice ); 1804 | 1805 | // Append substreams... 1806 | TComOutputBitstream *pcOut = pcBitstreamRedirect; 1807 | const Int numZeroSubstreamsAtStartOfSlice = pcPic->getSubstreamForCtuAddr(pcSlice->getSliceSegmentCurStartCtuTsAddr(), false, pcSlice); 1808 | const Int numSubstreamsToCode = pcSlice->getNumberOfSubstreamSizes()+1; 1809 | for ( UInt ui = 0 ; ui < numSubstreamsToCode; ui++ ) 1810 | { 1811 | pcOut->addSubstream(&(substreamsOut[ui+numZeroSubstreamsAtStartOfSlice])); 1812 | } 1813 | } 1814 | 1815 | // If current NALU is the first NALU of slice (containing slice header) and more NALUs exist (due to multiple dependent slices) then buffer it. 1816 | // If current NALU is the last NALU of slice and a NALU was buffered, then (a) Write current NALU (b) Update an write buffered NALU at approproate location in NALU list. 1817 | Bool bNALUAlignedWrittenToList = false; // used to ensure current NALU is not written more than once to the NALU list. 1818 | xAttachSliceDataToNalUnit(nalu, pcBitstreamRedirect); 1819 | accessUnit.push_back(new NALUnitEBSP(nalu)); 1820 | actualTotalBits += UInt(accessUnit.back()->m_nalUnitData.str().size()) * 8; 1821 | numBytesInVclNalUnits += (std::size_t)(accessUnit.back()->m_nalUnitData.str().size()); 1822 | bNALUAlignedWrittenToList = true; 1823 | 1824 | if (!bNALUAlignedWrittenToList) 1825 | { 1826 | nalu.m_Bitstream.writeAlignZero(); 1827 | accessUnit.push_back(new NALUnitEBSP(nalu)); 1828 | } 1829 | 1830 | if( ( m_pcCfg->getPictureTimingSEIEnabled() || m_pcCfg->getDecodingUnitInfoSEIEnabled() ) && 1831 | ( pcSlice->getSPS()->getVuiParametersPresentFlag() ) && 1832 | ( ( pcSlice->getSPS()->getVuiParameters()->getHrdParameters()->getNalHrdParametersPresentFlag() ) 1833 | || ( pcSlice->getSPS()->getVuiParameters()->getHrdParameters()->getVclHrdParametersPresentFlag() ) ) && 1834 | ( pcSlice->getSPS()->getVuiParameters()->getHrdParameters()->getSubPicCpbParamsPresentFlag() ) ) 1835 | { 1836 | UInt numNalus = 0; 1837 | UInt numRBSPBytes = 0; 1838 | for (AccessUnit::const_iterator it = accessUnit.begin(); it != accessUnit.end(); it++) 1839 | { 1840 | numRBSPBytes += UInt((*it)->m_nalUnitData.str().size()); 1841 | numNalus ++; 1842 | } 1843 | duData.push_back(DUData()); 1844 | duData.back().accumBitsDU = ( numRBSPBytes << 3 ); 1845 | duData.back().accumNalsDU = numNalus; 1846 | } 1847 | } // end iteration over slices 1848 | 1849 | ///////////////////////////////////////////////////////////////////////////////////////// 1850 | ///////////////////////////////////////////////////////////////////////////////////////////////////kly 1851 | 1852 | /////////////////////////////////////////////////////////////////////////////////////////////////// 1853 | 1854 | 1855 | 1856 | 1857 | ////////////////////////////////////////////////////////////////////////////////////////// 1858 | 1859 | // cabac_zero_words processing 1860 | cabac_zero_word_padding(pcSlice, pcPic, binCountsInNalUnits, numBytesInVclNalUnits, accessUnit.back()->m_nalUnitData, m_pcCfg->getCabacZeroWordPaddingEnabled()); 1861 | 1862 | pcPic->compressMotion(); 1863 | 1864 | 1865 | //-- For time output for each slice 1866 | Double dEncTime = (Double)(clock()-iBeforeTime) / CLOCKS_PER_SEC; 1867 | 1868 | std::string digestStr; 1869 | if (m_pcCfg->getDecodedPictureHashSEIType()!=HASHTYPE_NONE) 1870 | { 1871 | SEIDecodedPictureHash *decodedPictureHashSei = new SEIDecodedPictureHash(); 1872 | m_seiEncoder.initDecodedPictureHashSEI(decodedPictureHashSei, pcPic, digestStr, pcSlice->getSPS()->getBitDepths()); 1873 | trailingSeiMessages.push_back(decodedPictureHashSei); 1874 | } 1875 | xWriteTrailingSEIMessages(trailingSeiMessages, accessUnit, pcSlice->getTLayer(), pcSlice->getSPS()); 1876 | 1877 | m_pcCfg->setEncodedFlag(iGOPid, true); 1878 | 1879 | xCalculateAddPSNRs( isField, isTff, iGOPid, pcPic, accessUnit, rcListPic, dEncTime, snr_conversion, printFrameMSE ); 1880 | 1881 | printHash(m_pcCfg->getDecodedPictureHashSEIType(), digestStr); 1882 | 1883 | if ( m_pcCfg->getUseRateCtrl() ) 1884 | { 1885 | Double avgQP = m_pcRateCtrl->getRCPic()->calAverageQP(); 1886 | Double avgLambda = m_pcRateCtrl->getRCPic()->calAverageLambda(); 1887 | if ( avgLambda < 0.0 ) 1888 | { 1889 | avgLambda = lambda; 1890 | } 1891 | 1892 | m_pcRateCtrl->getRCPic()->updateAfterPicture( actualHeadBits, actualTotalBits, avgQP, avgLambda, pcSlice->getSliceType()); 1893 | m_pcRateCtrl->getRCPic()->addToPictureLsit( m_pcRateCtrl->getPicList() ); 1894 | 1895 | m_pcRateCtrl->getRCSeq()->updateAfterPic( actualTotalBits ); 1896 | if ( pcSlice->getSliceType() != I_SLICE ) 1897 | { 1898 | m_pcRateCtrl->getRCGOP()->updateAfterPicture( actualTotalBits ); 1899 | } 1900 | else // for intra picture, the estimated bits are used to update the current status in the GOP 1901 | { 1902 | m_pcRateCtrl->getRCGOP()->updateAfterPicture( estimatedBits ); 1903 | } 1904 | #if U0132_TARGET_BITS_SATURATION 1905 | if (m_pcRateCtrl->getCpbSaturationEnabled()) 1906 | { 1907 | m_pcRateCtrl->updateCpbState(actualTotalBits); 1908 | printf(" [CPB %6d bits]", m_pcRateCtrl->getCpbState()); 1909 | } 1910 | #endif 1911 | } 1912 | 1913 | xCreatePictureTimingSEI(m_pcCfg->getEfficientFieldIRAPEnabled()?effFieldIRAPMap.GetIRAPGOPid():0, leadingSeiMessages, nestedSeiMessages, duInfoSeiMessages, pcSlice, isField, duData); 1914 | if (m_pcCfg->getScalableNestingSEIEnabled()) 1915 | { 1916 | xCreateScalableNestingSEI (leadingSeiMessages, nestedSeiMessages); 1917 | } 1918 | xWriteLeadingSEIMessages(leadingSeiMessages, duInfoSeiMessages, accessUnit, pcSlice->getTLayer(), pcSlice->getSPS(), duData); 1919 | xWriteDuSEIMessages(duInfoSeiMessages, accessUnit, pcSlice->getTLayer(), pcSlice->getSPS(), duData); 1920 | 1921 | pcPic->getPicYuvRec()->copyToPic(pcPicYuvRecOut); 1922 | 1923 | 1924 | pcPic->setReconMark ( true ); 1925 | m_bFirst = false; 1926 | m_iNumPicCoded++; 1927 | m_totalCoded ++; 1928 | /* logging: insert a newline at end of picture period */ 1929 | printf("\n"); 1930 | fflush(stdout); 1931 | 1932 | if (m_pcCfg->getEfficientFieldIRAPEnabled()) 1933 | { 1934 | iGOPid=effFieldIRAPMap.restoreGOPid(iGOPid); 1935 | } 1936 | } // iGOPid-loop 1937 | 1938 | delete pcBitstreamRedirect; 1939 | 1940 | assert ( (m_iNumPicCoded == iNumPicRcvd) ); 1941 | //system("pause"); 1942 | } 1943 | 1944 | Void TEncGOP::printOutSummary(UInt uiNumAllPicCoded, Bool isField, const Bool printMSEBasedSNR, const Bool printSequenceMSE, const BitDepths &bitDepths) 1945 | { 1946 | assert (uiNumAllPicCoded == m_gcAnalyzeAll.getNumPic()); 1947 | 1948 | 1949 | //--CFG_KDY 1950 | const Int rateMultiplier=(isField?2:1); 1951 | m_gcAnalyzeAll.setFrmRate( m_pcCfg->getFrameRate()*rateMultiplier / (Double)m_pcCfg->getTemporalSubsampleRatio()); 1952 | m_gcAnalyzeI.setFrmRate( m_pcCfg->getFrameRate()*rateMultiplier / (Double)m_pcCfg->getTemporalSubsampleRatio()); 1953 | m_gcAnalyzeP.setFrmRate( m_pcCfg->getFrameRate()*rateMultiplier / (Double)m_pcCfg->getTemporalSubsampleRatio()); 1954 | m_gcAnalyzeB.setFrmRate( m_pcCfg->getFrameRate()*rateMultiplier / (Double)m_pcCfg->getTemporalSubsampleRatio()); 1955 | const ChromaFormat chFmt = m_pcCfg->getChromaFormatIdc(); 1956 | 1957 | //-- all 1958 | printf( "\n\nSUMMARY --------------------------------------------------------\n" ); 1959 | m_gcAnalyzeAll.printOut('a', chFmt, printMSEBasedSNR, printSequenceMSE, bitDepths); 1960 | 1961 | printf( "\n\nI Slices--------------------------------------------------------\n" ); 1962 | m_gcAnalyzeI.printOut('i', chFmt, printMSEBasedSNR, printSequenceMSE, bitDepths); 1963 | 1964 | printf( "\n\nP Slices--------------------------------------------------------\n" ); 1965 | m_gcAnalyzeP.printOut('p', chFmt, printMSEBasedSNR, printSequenceMSE, bitDepths); 1966 | 1967 | printf( "\n\nB Slices--------------------------------------------------------\n" ); 1968 | m_gcAnalyzeB.printOut('b', chFmt, printMSEBasedSNR, printSequenceMSE, bitDepths); 1969 | 1970 | if (!m_pcCfg->getSummaryOutFilename().empty()) 1971 | { 1972 | m_gcAnalyzeAll.printSummary(chFmt, printSequenceMSE, bitDepths, m_pcCfg->getSummaryOutFilename()); 1973 | } 1974 | 1975 | if (!m_pcCfg->getSummaryPicFilenameBase().empty()) 1976 | { 1977 | m_gcAnalyzeI.printSummary(chFmt, printSequenceMSE, bitDepths, m_pcCfg->getSummaryPicFilenameBase()+"I.txt"); 1978 | m_gcAnalyzeP.printSummary(chFmt, printSequenceMSE, bitDepths, m_pcCfg->getSummaryPicFilenameBase()+"P.txt"); 1979 | m_gcAnalyzeB.printSummary(chFmt, printSequenceMSE, bitDepths, m_pcCfg->getSummaryPicFilenameBase()+"B.txt"); 1980 | } 1981 | 1982 | if(isField) 1983 | { 1984 | //-- interlaced summary 1985 | #if TEMPORAL_SUBSAMPLE 1986 | m_gcAnalyzeAll_in.setFrmRate( m_pcCfg->getFrameRate() / (Double)m_pcCfg->getTemporalSubsampleRatio()); 1987 | #else 1988 | m_gcAnalyzeAll_in.setFrmRate( m_pcCfg->getFrameRate()); 1989 | #endif 1990 | m_gcAnalyzeAll_in.setBits(m_gcAnalyzeAll.getBits()); 1991 | // prior to the above statement, the interlace analyser does not contain the correct total number of bits. 1992 | 1993 | printf( "\n\nSUMMARY INTERLACED ---------------------------------------------\n" ); 1994 | m_gcAnalyzeAll_in.printOut('a', chFmt, printMSEBasedSNR, printSequenceMSE, bitDepths); 1995 | 1996 | if (!m_pcCfg->getSummaryOutFilename().empty()) 1997 | { 1998 | m_gcAnalyzeAll_in.printSummary(chFmt, printSequenceMSE, bitDepths, m_pcCfg->getSummaryOutFilename()); 1999 | } 2000 | } 2001 | 2002 | printf("\nRVM: %.3lf\n" , xCalculateRVM()); 2003 | } 2004 | 2005 | Void TEncGOP::preLoopFilterPicAll( TComPic* pcPic, UInt64& ruiDist ) 2006 | { 2007 | Bool bCalcDist = false; 2008 | m_pcLoopFilter->setCfg(m_pcCfg->getLFCrossTileBoundaryFlag()); 2009 | m_pcLoopFilter->loopFilterPic( pcPic ); 2010 | 2011 | if (!bCalcDist) 2012 | { 2013 | ruiDist = xFindDistortionFrame(pcPic->getPicYuvOrg(), pcPic->getPicYuvRec(), pcPic->getPicSym()->getSPS().getBitDepths()); 2014 | } 2015 | } 2016 | 2017 | // ==================================================================================================================== 2018 | // Protected member functions 2019 | // ==================================================================================================================== 2020 | 2021 | 2022 | Void TEncGOP::xInitGOP( Int iPOCLast, Int iNumPicRcvd, Bool isField ) 2023 | { 2024 | assert( iNumPicRcvd > 0 ); 2025 | // Exception for the first frames 2026 | if ( ( isField && (iPOCLast == 0 || iPOCLast == 1) ) || (!isField && (iPOCLast == 0)) ) 2027 | { 2028 | m_iGopSize = 1; 2029 | } 2030 | else 2031 | { 2032 | m_iGopSize = m_pcCfg->getGOPSize(); 2033 | } 2034 | assert (m_iGopSize > 0); 2035 | 2036 | return; 2037 | } 2038 | 2039 | 2040 | Void TEncGOP::xGetBuffer( TComList& rcListPic, 2041 | TComList& rcListPicYuvRecOut, 2042 | Int iNumPicRcvd, 2043 | Int iTimeOffset, 2044 | TComPic*& rpcPic, 2045 | TComPicYuv*& rpcPicYuvRecOut, 2046 | Int pocCurr, 2047 | Bool isField) 2048 | { 2049 | Int i; 2050 | // Rec. output 2051 | TComList::iterator iterPicYuvRec = rcListPicYuvRecOut.end(); 2052 | 2053 | if (isField && pocCurr > 1 && m_iGopSize!=1) 2054 | { 2055 | iTimeOffset--; 2056 | } 2057 | 2058 | for ( i = 0; i < (iNumPicRcvd - iTimeOffset + 1); i++ ) 2059 | { 2060 | iterPicYuvRec--; 2061 | } 2062 | 2063 | rpcPicYuvRecOut = *(iterPicYuvRec); 2064 | 2065 | // Current pic. 2066 | TComList::iterator iterPic = rcListPic.begin(); 2067 | while (iterPic != rcListPic.end()) 2068 | { 2069 | rpcPic = *(iterPic); 2070 | rpcPic->setCurrSliceIdx(0); 2071 | if (rpcPic->getPOC() == pocCurr) 2072 | { 2073 | break; 2074 | } 2075 | iterPic++; 2076 | } 2077 | 2078 | assert (rpcPic != NULL); 2079 | assert (rpcPic->getPOC() == pocCurr); 2080 | 2081 | return; 2082 | } 2083 | 2084 | UInt64 TEncGOP::xFindDistortionFrame (TComPicYuv* pcPic0, TComPicYuv* pcPic1, const BitDepths &bitDepths) 2085 | { 2086 | UInt64 uiTotalDiff = 0; 2087 | 2088 | for(Int chan=0; changetNumberValidComponents(); chan++) 2089 | { 2090 | const ComponentID ch=ComponentID(chan); 2091 | Pel* pSrc0 = pcPic0 ->getAddr(ch); 2092 | Pel* pSrc1 = pcPic1 ->getAddr(ch); 2093 | UInt uiShift = 2 * DISTORTION_PRECISION_ADJUSTMENT(bitDepths.recon[toChannelType(ch)]-8); 2094 | 2095 | const Int iStride = pcPic0->getStride(ch); 2096 | const Int iWidth = pcPic0->getWidth(ch); 2097 | const Int iHeight = pcPic0->getHeight(ch); 2098 | 2099 | for(Int y = 0; y < iHeight; y++ ) 2100 | { 2101 | for(Int x = 0; x < iWidth; x++ ) 2102 | { 2103 | Intermediate_Int iTemp = pSrc0[x] - pSrc1[x]; 2104 | uiTotalDiff += UInt64((iTemp*iTemp) >> uiShift); 2105 | } 2106 | pSrc0 += iStride; 2107 | pSrc1 += iStride; 2108 | } 2109 | } 2110 | 2111 | return uiTotalDiff; 2112 | } 2113 | 2114 | Void TEncGOP::xCalculateAddPSNRs( const Bool isField, const Bool isFieldTopFieldFirst, const Int iGOPid, TComPic* pcPic, const AccessUnit&accessUnit, TComList &rcListPic, const Double dEncTime, const InputColourSpaceConversion snr_conversion, const Bool printFrameMSE ) 2115 | { 2116 | xCalculateAddPSNR( pcPic, pcPic->getPicYuvRec(), accessUnit, dEncTime, snr_conversion, printFrameMSE ); 2117 | 2118 | //In case of field coding, compute the interlaced PSNR for both fields 2119 | if(isField) 2120 | { 2121 | Bool bothFieldsAreEncoded = false; 2122 | Int correspondingFieldPOC = pcPic->getPOC(); 2123 | Int currentPicGOPPoc = m_pcCfg->getGOPEntry(iGOPid).m_POC; 2124 | if(pcPic->getPOC() == 0) 2125 | { 2126 | // particular case for POC 0 and 1. 2127 | // If they are not encoded first and separately from other pictures, we need to change this 2128 | // POC 0 is always encoded first then POC 1 is encoded 2129 | bothFieldsAreEncoded = false; 2130 | } 2131 | else if(pcPic->getPOC() == 1) 2132 | { 2133 | // if we are at POC 1, POC 0 has been encoded for sure 2134 | correspondingFieldPOC = 0; 2135 | bothFieldsAreEncoded = true; 2136 | } 2137 | else 2138 | { 2139 | if(pcPic->getPOC()%2 == 1) 2140 | { 2141 | correspondingFieldPOC -= 1; // all odd POC are associated with the preceding even POC (e.g poc 1 is associated to poc 0) 2142 | currentPicGOPPoc -= 1; 2143 | } 2144 | else 2145 | { 2146 | correspondingFieldPOC += 1; // all even POC are associated with the following odd POC (e.g poc 0 is associated to poc 1) 2147 | currentPicGOPPoc += 1; 2148 | } 2149 | for(Int i = 0; i < m_iGopSize; i ++) 2150 | { 2151 | if(m_pcCfg->getGOPEntry(i).m_POC == currentPicGOPPoc) 2152 | { 2153 | bothFieldsAreEncoded = m_pcCfg->getGOPEntry(i).m_isEncoded; 2154 | break; 2155 | } 2156 | } 2157 | } 2158 | 2159 | if(bothFieldsAreEncoded) 2160 | { 2161 | //get complementary top field 2162 | TComList::iterator iterPic = rcListPic.begin(); 2163 | while ((*iterPic)->getPOC() != correspondingFieldPOC) 2164 | { 2165 | iterPic ++; 2166 | } 2167 | TComPic* correspondingFieldPic = *(iterPic); 2168 | 2169 | if( (pcPic->isTopField() && isFieldTopFieldFirst) || (!pcPic->isTopField() && !isFieldTopFieldFirst)) 2170 | { 2171 | xCalculateInterlacedAddPSNR(pcPic, correspondingFieldPic, pcPic->getPicYuvRec(), correspondingFieldPic->getPicYuvRec(), snr_conversion, printFrameMSE ); 2172 | } 2173 | else 2174 | { 2175 | xCalculateInterlacedAddPSNR(correspondingFieldPic, pcPic, correspondingFieldPic->getPicYuvRec(), pcPic->getPicYuvRec(), snr_conversion, printFrameMSE ); 2176 | } 2177 | } 2178 | } 2179 | } 2180 | 2181 | Void TEncGOP::xCalculateAddPSNR( TComPic* pcPic, TComPicYuv* pcPicD, const AccessUnit& accessUnit, Double dEncTime, const InputColourSpaceConversion conversion, const Bool printFrameMSE ) 2182 | { 2183 | Double dPSNR[MAX_NUM_COMPONENT]; 2184 | 2185 | for(Int i=0; igetWidth(COMPONENT_Y), pcPicD->getHeight(COMPONENT_Y), pcPicD->getChromaFormat() ); 2194 | TVideoIOYuv::ColourSpaceConvert(*pcPicD, cscd, conversion, false); 2195 | } 2196 | TComPicYuv &picd=(conversion==IPCOLOURSPACE_UNCHANGED)?*pcPicD : cscd; 2197 | 2198 | //===== calculate PSNR ===== 2199 | Double MSEyuvframe[MAX_NUM_COMPONENT] = {0, 0, 0}; 2200 | 2201 | for(Int chan=0; changetNumberValidComponents(); chan++) 2202 | { 2203 | const ComponentID ch=ComponentID(chan); 2204 | const TComPicYuv *pOrgPicYuv =(conversion!=IPCOLOURSPACE_UNCHANGED) ? pcPic ->getPicYuvTrueOrg() : pcPic ->getPicYuvOrg(); 2205 | const Pel* pOrg = pOrgPicYuv->getAddr(ch); 2206 | const Int iOrgStride = pOrgPicYuv->getStride(ch); 2207 | Pel* pRec = picd.getAddr(ch); 2208 | const Int iRecStride = picd.getStride(ch); 2209 | const Int iWidth = pcPicD->getWidth (ch) - (m_pcEncTop->getPad(0) >> pcPic->getComponentScaleX(ch)); 2210 | const Int iHeight = pcPicD->getHeight(ch) - ((m_pcEncTop->getPad(1) >> (pcPic->isField()?1:0)) >> pcPic->getComponentScaleY(ch)); 2211 | 2212 | Int iSize = iWidth*iHeight; 2213 | 2214 | UInt64 uiSSDtemp=0; 2215 | for(Int y = 0; y < iHeight-0; y++ ) 2216 | { 2217 | for(Int x = 0; x < iWidth-0; x++ ) 2218 | { 2219 | Intermediate_Int iDiff = (Intermediate_Int)( pOrg[x] - pRec[x] ); 2220 | uiSSDtemp += iDiff * iDiff; 2221 | } 2222 | pOrg += iOrgStride; 2223 | pRec += iRecStride; 2224 | } 2225 | const Int maxval = 255 << (pcPic->getPicSym()->getSPS().getBitDepth(toChannelType(ch)) - 8); 2226 | const Double fRefValue = (Double) maxval * maxval * iSize; 2227 | dPSNR[ch] = ( uiSSDtemp ? 10.0 * log10( fRefValue / (Double)uiSSDtemp ) : 999.99 ); 2228 | MSEyuvframe[ch] = (Double)uiSSDtemp/(iSize); 2229 | } 2230 | 2231 | 2232 | /* calculate the size of the access unit, excluding: 2233 | * - any AnnexB contributions (start_code_prefix, zero_byte, etc.,) 2234 | * - SEI NAL units 2235 | */ 2236 | UInt numRBSPBytes = 0; 2237 | for (AccessUnit::const_iterator it = accessUnit.begin(); it != accessUnit.end(); it++) 2238 | { 2239 | UInt numRBSPBytes_nal = UInt((*it)->m_nalUnitData.str().size()); 2240 | if (m_pcCfg->getSummaryVerboseness() > 0) 2241 | { 2242 | printf("*** %6s numBytesInNALunit: %u\n", nalUnitTypeToString((*it)->m_nalUnitType), numRBSPBytes_nal); 2243 | } 2244 | if ((*it)->m_nalUnitType != NAL_UNIT_PREFIX_SEI && (*it)->m_nalUnitType != NAL_UNIT_SUFFIX_SEI) 2245 | { 2246 | numRBSPBytes += numRBSPBytes_nal; 2247 | } 2248 | } 2249 | 2250 | UInt uibits = numRBSPBytes * 8; 2251 | m_vRVM_RP.push_back( uibits ); 2252 | 2253 | //===== add PSNR ===== 2254 | m_gcAnalyzeAll.addResult (dPSNR, (Double)uibits, MSEyuvframe); 2255 | TComSlice* pcSlice = pcPic->getSlice(0); 2256 | if (pcSlice->isIntra()) 2257 | { 2258 | m_gcAnalyzeI.addResult (dPSNR, (Double)uibits, MSEyuvframe); 2259 | } 2260 | if (pcSlice->isInterP()) 2261 | { 2262 | m_gcAnalyzeP.addResult (dPSNR, (Double)uibits, MSEyuvframe); 2263 | } 2264 | if (pcSlice->isInterB()) 2265 | { 2266 | m_gcAnalyzeB.addResult (dPSNR, (Double)uibits, MSEyuvframe); 2267 | } 2268 | 2269 | TChar c = (pcSlice->isIntra() ? 'I' : pcSlice->isInterP() ? 'P' : 'B'); 2270 | if (!pcSlice->isReferenced()) 2271 | { 2272 | c += 32; 2273 | } 2274 | 2275 | #if ADAPTIVE_QP_SELECTION 2276 | printf("POC %4d TId: %1d ( %c-SLICE, nQP %d QP %d ) %10d bits", 2277 | pcSlice->getPOC(), 2278 | pcSlice->getTLayer(), 2279 | c, 2280 | pcSlice->getSliceQpBase(), 2281 | pcSlice->getSliceQp(), 2282 | uibits ); 2283 | #else 2284 | printf("POC %4d TId: %1d ( %c-SLICE, QP %d ) %10d bits", 2285 | pcSlice->getPOC()-pcSlice->getLastIDR(), 2286 | pcSlice->getTLayer(), 2287 | c, 2288 | pcSlice->getSliceQp(), 2289 | uibits ); 2290 | #endif 2291 | 2292 | printf(" [Y %6.4lf dB U %6.4lf dB V %6.4lf dB]", dPSNR[COMPONENT_Y], dPSNR[COMPONENT_Cb], dPSNR[COMPONENT_Cr] ); 2293 | if (printFrameMSE) 2294 | { 2295 | printf(" [Y MSE %6.4lf U MSE %6.4lf V MSE %6.4lf]", MSEyuvframe[COMPONENT_Y], MSEyuvframe[COMPONENT_Cb], MSEyuvframe[COMPONENT_Cr] ); 2296 | } 2297 | printf(" [ET %5.0f ]", dEncTime ); 2298 | 2299 | // printf(" [WP %d]", pcSlice->getUseWeightedPrediction()); 2300 | 2301 | for (Int iRefList = 0; iRefList < 2; iRefList++) 2302 | { 2303 | printf(" [L%d ", iRefList); 2304 | for (Int iRefIndex = 0; iRefIndex < pcSlice->getNumRefIdx(RefPicList(iRefList)); iRefIndex++) 2305 | { 2306 | printf ("%d ", pcSlice->getRefPOC(RefPicList(iRefList), iRefIndex)-pcSlice->getLastIDR()); 2307 | } 2308 | printf("]"); 2309 | } 2310 | 2311 | cscd.destroy(); 2312 | } 2313 | 2314 | Void TEncGOP::xCalculateInterlacedAddPSNR( TComPic* pcPicOrgFirstField, TComPic* pcPicOrgSecondField, 2315 | TComPicYuv* pcPicRecFirstField, TComPicYuv* pcPicRecSecondField, 2316 | const InputColourSpaceConversion conversion, const Bool printFrameMSE ) 2317 | { 2318 | const TComSPS &sps=pcPicOrgFirstField->getPicSym()->getSPS(); 2319 | Double dPSNR[MAX_NUM_COMPONENT]; 2320 | TComPic *apcPicOrgFields[2]={pcPicOrgFirstField, pcPicOrgSecondField}; 2321 | TComPicYuv *apcPicRecFields[2]={pcPicRecFirstField, pcPicRecSecondField}; 2322 | 2323 | for(Int i=0; igetChromaFormat()==apcPicRecFields[1]->getChromaFormat()); 2344 | const UInt numValidComponents=apcPicRecFields[0]->getNumberValidComponents(); 2345 | 2346 | for(Int chan=0; changetWidth(ch)==apcPicRecFields[1]->getWidth(ch)); 2350 | assert(apcPicRecFields[0]->getHeight(ch)==apcPicRecFields[1]->getHeight(ch)); 2351 | 2352 | UInt64 uiSSDtemp=0; 2353 | const Int iWidth = apcPicRecFields[0]->getWidth (ch) - (m_pcEncTop->getPad(0) >> apcPicRecFields[0]->getComponentScaleX(ch)); 2354 | const Int iHeight = apcPicRecFields[0]->getHeight(ch) - ((m_pcEncTop->getPad(1) >> 1) >> apcPicRecFields[0]->getComponentScaleY(ch)); 2355 | 2356 | Int iSize = iWidth*iHeight; 2357 | 2358 | for(UInt fieldNum=0; fieldNum<2; fieldNum++) 2359 | { 2360 | TComPic *pcPic=apcPicOrgFields[fieldNum]; 2361 | TComPicYuv *pcPicD=apcPicRecFields[fieldNum]; 2362 | 2363 | const Pel* pOrg = (conversion!=IPCOLOURSPACE_UNCHANGED) ? pcPic ->getPicYuvTrueOrg()->getAddr(ch) : pcPic ->getPicYuvOrg()->getAddr(ch); 2364 | Pel* pRec = pcPicD->getAddr(ch); 2365 | const Int iStride = pcPicD->getStride(ch); 2366 | 2367 | 2368 | for(Int y = 0; y < iHeight; y++ ) 2369 | { 2370 | for(Int x = 0; x < iWidth; x++ ) 2371 | { 2372 | Intermediate_Int iDiff = (Intermediate_Int)( pOrg[x] - pRec[x] ); 2373 | uiSSDtemp += iDiff * iDiff; 2374 | } 2375 | pOrg += iStride; 2376 | pRec += iStride; 2377 | } 2378 | } 2379 | const Int maxval = 255 << (sps.getBitDepth(toChannelType(ch)) - 8); 2380 | const Double fRefValue = (Double) maxval * maxval * iSize*2; 2381 | dPSNR[ch] = ( uiSSDtemp ? 10.0 * log10( fRefValue / (Double)uiSSDtemp ) : 999.99 ); 2382 | MSEyuvframe[ch] = (Double)uiSSDtemp/(iSize*2); 2383 | } 2384 | 2385 | UInt uibits = 0; // the number of bits for the pair is not calculated here - instead the overall total is used elsewhere. 2386 | 2387 | //===== add PSNR ===== 2388 | m_gcAnalyzeAll_in.addResult (dPSNR, (Double)uibits, MSEyuvframe); 2389 | 2390 | printf("\n Interlaced frame %d: [Y %6.4lf dB U %6.4lf dB V %6.4lf dB]", pcPicOrgSecondField->getPOC()/2 , dPSNR[COMPONENT_Y], dPSNR[COMPONENT_Cb], dPSNR[COMPONENT_Cr] ); 2391 | if (printFrameMSE) 2392 | { 2393 | printf(" [Y MSE %6.4lf U MSE %6.4lf V MSE %6.4lf]", MSEyuvframe[COMPONENT_Y], MSEyuvframe[COMPONENT_Cb], MSEyuvframe[COMPONENT_Cr] ); 2394 | } 2395 | 2396 | for(UInt fieldNum=0; fieldNum<2; fieldNum++) 2397 | { 2398 | cscd[fieldNum].destroy(); 2399 | } 2400 | } 2401 | 2402 | /** Function for deciding the nal_unit_type. 2403 | * \param pocCurr POC of the current picture 2404 | * \param lastIDR POC of the last IDR picture 2405 | * \param isField true to indicate field coding 2406 | * \returns the NAL unit type of the picture 2407 | * This function checks the configuration and returns the appropriate nal_unit_type for the picture. 2408 | */ 2409 | NalUnitType TEncGOP::getNalUnitType(Int pocCurr, Int lastIDR, Bool isField) 2410 | { 2411 | if (pocCurr == 0) 2412 | { 2413 | return NAL_UNIT_CODED_SLICE_IDR_W_RADL; 2414 | } 2415 | 2416 | if(m_pcCfg->getEfficientFieldIRAPEnabled() && isField && pocCurr == 1) 2417 | { 2418 | // to avoid the picture becoming an IRAP 2419 | return NAL_UNIT_CODED_SLICE_TRAIL_R; 2420 | } 2421 | 2422 | if(m_pcCfg->getDecodingRefreshType() != 3 && (pocCurr - isField) % m_pcCfg->getIntraPeriod() == 0) 2423 | { 2424 | if (m_pcCfg->getDecodingRefreshType() == 1) 2425 | { 2426 | return NAL_UNIT_CODED_SLICE_CRA; 2427 | } 2428 | else if (m_pcCfg->getDecodingRefreshType() == 2) 2429 | { 2430 | return NAL_UNIT_CODED_SLICE_IDR_W_RADL; 2431 | } 2432 | } 2433 | if(m_pocCRA>0) 2434 | { 2435 | if(pocCurr0) 2446 | { 2447 | if (pocCurr < lastIDR) 2448 | { 2449 | return NAL_UNIT_CODED_SLICE_RADL_R; 2450 | } 2451 | } 2452 | return NAL_UNIT_CODED_SLICE_TRAIL_R; 2453 | } 2454 | 2455 | Double TEncGOP::xCalculateRVM() 2456 | { 2457 | Double dRVM = 0; 2458 | 2459 | if( m_pcCfg->getGOPSize() == 1 && m_pcCfg->getIntraPeriod() != 1 && m_pcCfg->getFramesToBeEncoded() > RVM_VCEGAM10_M * 2 ) 2460 | { 2461 | // calculate RVM only for lowdelay configurations 2462 | std::vector vRL , vB; 2463 | size_t N = m_vRVM_RP.size(); 2464 | vRL.resize( N ); 2465 | vB.resize( N ); 2466 | 2467 | Int i; 2468 | Double dRavg = 0 , dBavg = 0; 2469 | vB[RVM_VCEGAM10_M] = 0; 2470 | for( i = RVM_VCEGAM10_M + 1 ; i < N - RVM_VCEGAM10_M + 1 ; i++ ) 2471 | { 2472 | vRL[i] = 0; 2473 | for( Int j = i - RVM_VCEGAM10_M ; j <= i + RVM_VCEGAM10_M - 1 ; j++ ) 2474 | { 2475 | vRL[i] += m_vRVM_RP[j]; 2476 | } 2477 | vRL[i] /= ( 2 * RVM_VCEGAM10_M ); 2478 | vB[i] = vB[i-1] + m_vRVM_RP[i] - vRL[i]; 2479 | dRavg += m_vRVM_RP[i]; 2480 | dBavg += vB[i]; 2481 | } 2482 | 2483 | dRavg /= ( N - 2 * RVM_VCEGAM10_M ); 2484 | dBavg /= ( N - 2 * RVM_VCEGAM10_M ); 2485 | 2486 | Double dSigamB = 0; 2487 | for( i = RVM_VCEGAM10_M + 1 ; i < N - RVM_VCEGAM10_M + 1 ; i++ ) 2488 | { 2489 | Double tmp = vB[i] - dBavg; 2490 | dSigamB += tmp * tmp; 2491 | } 2492 | dSigamB = sqrt( dSigamB / ( N - 2 * RVM_VCEGAM10_M ) ); 2493 | 2494 | Double f = sqrt( 12.0 * ( RVM_VCEGAM10_M - 1 ) / ( RVM_VCEGAM10_M + 1 ) ); 2495 | 2496 | dRVM = dSigamB / dRavg * f; 2497 | } 2498 | 2499 | return( dRVM ); 2500 | } 2501 | 2502 | /** Attaches the input bitstream to the stream in the output NAL unit 2503 | Updates rNalu to contain concatenated bitstream. rpcBitstreamRedirect is cleared at the end of this function call. 2504 | * \param codedSliceData contains the coded slice data (bitstream) to be concatenated to rNalu 2505 | * \param rNalu target NAL unit 2506 | */ 2507 | Void TEncGOP::xAttachSliceDataToNalUnit (OutputNALUnit& rNalu, TComOutputBitstream* codedSliceData) 2508 | { 2509 | // Byte-align 2510 | rNalu.m_Bitstream.writeByteAlignment(); // Slice header byte-alignment 2511 | 2512 | // Perform bitstream concatenation 2513 | if (codedSliceData->getNumberOfWrittenBits() > 0) 2514 | { 2515 | rNalu.m_Bitstream.addSubstream(codedSliceData); 2516 | } 2517 | 2518 | m_pcEntropyCoder->setBitstream(&rNalu.m_Bitstream); 2519 | 2520 | codedSliceData->clear(); 2521 | } 2522 | 2523 | // Function will arrange the long-term pictures in the decreasing order of poc_lsb_lt, 2524 | // and among the pictures with the same lsb, it arranges them in increasing delta_poc_msb_cycle_lt value 2525 | Void TEncGOP::arrangeLongtermPicturesInRPS(TComSlice *pcSlice, TComList& rcListPic) 2526 | { 2527 | if(pcSlice->getRPS()->getNumberOfLongtermPictures() == 0) 2528 | { 2529 | return; 2530 | } 2531 | // we can only modify the local RPS! 2532 | assert (pcSlice->getRPSidx()==-1); 2533 | TComReferencePictureSet *rps = pcSlice->getLocalRPS(); 2534 | 2535 | // Arrange long-term reference pictures in the correct order of LSB and MSB, 2536 | // and assign values for pocLSBLT and MSB present flag 2537 | Int longtermPicsPoc[MAX_NUM_REF_PICS], longtermPicsLSB[MAX_NUM_REF_PICS], indices[MAX_NUM_REF_PICS]; 2538 | Int longtermPicsMSB[MAX_NUM_REF_PICS]; 2539 | Bool mSBPresentFlag[MAX_NUM_REF_PICS]; 2540 | ::memset(longtermPicsPoc, 0, sizeof(longtermPicsPoc)); // Store POC values of LTRP 2541 | ::memset(longtermPicsLSB, 0, sizeof(longtermPicsLSB)); // Store POC LSB values of LTRP 2542 | ::memset(longtermPicsMSB, 0, sizeof(longtermPicsMSB)); // Store POC LSB values of LTRP 2543 | ::memset(indices , 0, sizeof(indices)); // Indices to aid in tracking sorted LTRPs 2544 | ::memset(mSBPresentFlag , 0, sizeof(mSBPresentFlag)); // Indicate if MSB needs to be present 2545 | 2546 | // Get the long-term reference pictures 2547 | Int offset = rps->getNumberOfNegativePictures() + rps->getNumberOfPositivePictures(); 2548 | Int i, ctr = 0; 2549 | Int maxPicOrderCntLSB = 1 << pcSlice->getSPS()->getBitsForPOC(); 2550 | for(i = rps->getNumberOfPictures() - 1; i >= offset; i--, ctr++) 2551 | { 2552 | longtermPicsPoc[ctr] = rps->getPOC(i); // LTRP POC 2553 | longtermPicsLSB[ctr] = getLSB(longtermPicsPoc[ctr], maxPicOrderCntLSB); // LTRP POC LSB 2554 | indices[ctr] = i; 2555 | longtermPicsMSB[ctr] = longtermPicsPoc[ctr] - longtermPicsLSB[ctr]; 2556 | } 2557 | Int numLongPics = rps->getNumberOfLongtermPictures(); 2558 | assert(ctr == numLongPics); 2559 | 2560 | // Arrange pictures in decreasing order of MSB; 2561 | for(i = 0; i < numLongPics; i++) 2562 | { 2563 | for(Int j = 0; j < numLongPics - 1; j++) 2564 | { 2565 | if(longtermPicsMSB[j] < longtermPicsMSB[j+1]) 2566 | { 2567 | std::swap(longtermPicsPoc[j], longtermPicsPoc[j+1]); 2568 | std::swap(longtermPicsLSB[j], longtermPicsLSB[j+1]); 2569 | std::swap(longtermPicsMSB[j], longtermPicsMSB[j+1]); 2570 | std::swap(indices[j] , indices[j+1] ); 2571 | } 2572 | } 2573 | } 2574 | 2575 | for(i = 0; i < numLongPics; i++) 2576 | { 2577 | // Check if MSB present flag should be enabled. 2578 | // Check if the buffer contains any pictures that have the same LSB. 2579 | TComList::iterator iterPic = rcListPic.begin(); 2580 | TComPic* pcPic; 2581 | while ( iterPic != rcListPic.end() ) 2582 | { 2583 | pcPic = *iterPic; 2584 | if( (getLSB(pcPic->getPOC(), maxPicOrderCntLSB) == longtermPicsLSB[i]) && // Same LSB 2585 | (pcPic->getSlice(0)->isReferenced()) && // Reference picture 2586 | (pcPic->getPOC() != longtermPicsPoc[i]) ) // Not the LTRP itself 2587 | { 2588 | mSBPresentFlag[i] = true; 2589 | break; 2590 | } 2591 | iterPic++; 2592 | } 2593 | } 2594 | 2595 | // tempArray for usedByCurr flag 2596 | Bool tempArray[MAX_NUM_REF_PICS]; ::memset(tempArray, 0, sizeof(tempArray)); 2597 | for(i = 0; i < numLongPics; i++) 2598 | { 2599 | tempArray[i] = rps->getUsed(indices[i]); 2600 | } 2601 | // Now write the final values; 2602 | ctr = 0; 2603 | Int currMSB = 0, currLSB = 0; 2604 | // currPicPoc = currMSB + currLSB 2605 | currLSB = getLSB(pcSlice->getPOC(), maxPicOrderCntLSB); 2606 | currMSB = pcSlice->getPOC() - currLSB; 2607 | 2608 | for(i = rps->getNumberOfPictures() - 1; i >= offset; i--, ctr++) 2609 | { 2610 | rps->setPOC (i, longtermPicsPoc[ctr]); 2611 | rps->setDeltaPOC (i, - pcSlice->getPOC() + longtermPicsPoc[ctr]); 2612 | rps->setUsed (i, tempArray[ctr]); 2613 | rps->setPocLSBLT (i, longtermPicsLSB[ctr]); 2614 | rps->setDeltaPocMSBCycleLT (i, (currMSB - (longtermPicsPoc[ctr] - longtermPicsLSB[ctr])) / maxPicOrderCntLSB); 2615 | rps->setDeltaPocMSBPresentFlag(i, mSBPresentFlag[ctr]); 2616 | 2617 | assert(rps->getDeltaPocMSBCycleLT(i) >= 0); // Non-negative value 2618 | } 2619 | for(i = rps->getNumberOfPictures() - 1, ctr = 1; i >= offset; i--, ctr++) 2620 | { 2621 | for(Int j = rps->getNumberOfPictures() - 1 - ctr; j >= offset; j--) 2622 | { 2623 | // Here at the encoder we know that we have set the full POC value for the LTRPs, hence we 2624 | // don't have to check the MSB present flag values for this constraint. 2625 | assert( rps->getPOC(i) != rps->getPOC(j) ); // If assert fails, LTRP entry repeated in RPS!!! 2626 | } 2627 | } 2628 | } 2629 | 2630 | Void TEncGOP::applyDeblockingFilterMetric( TComPic* pcPic, UInt uiNumSlices ) 2631 | { 2632 | TComPicYuv* pcPicYuvRec = pcPic->getPicYuvRec(); 2633 | Pel* Rec = pcPicYuvRec->getAddr(COMPONENT_Y); 2634 | Pel* tempRec = Rec; 2635 | Int stride = pcPicYuvRec->getStride(COMPONENT_Y); 2636 | UInt log2maxTB = pcPic->getSlice(0)->getSPS()->getQuadtreeTULog2MaxSize(); 2637 | UInt maxTBsize = (1<getWidth(COMPONENT_Y); 2640 | const UInt picHeight = pcPicYuvRec->getHeight(COMPONENT_Y); 2641 | const UInt noCol = (picWidth>>log2maxTB); 2642 | const UInt noRows = (picHeight>>log2maxTB); 2643 | assert(noCol > 1); 2644 | assert(noRows > 1); 2645 | std::vector colSAD(noCol, UInt64(0)); 2646 | std::vector rowSAD(noRows, UInt64(0)); 2647 | UInt colIdx = 0; 2648 | UInt rowIdx = 0; 2649 | Pel p0, p1, p2, q0, q1, q2; 2650 | 2651 | Int qp = pcPic->getSlice(0)->getSliceQp(); 2652 | const Int bitDepthLuma=pcPic->getSlice(0)->getSPS()->getBitDepth(CHANNEL_TYPE_LUMA); 2653 | Int bitdepthScale = 1 << (bitDepthLuma-8); 2654 | Int beta = TComLoopFilter::getBeta( qp ) * bitdepthScale; 2655 | const Int thr2 = (beta>>2); 2656 | const Int thr1 = 2*bitdepthScale; 2657 | UInt a = 0; 2658 | 2659 | if (maxTBsize > minBlockArtSize) 2660 | { 2661 | // Analyze vertical artifact edges 2662 | for(Int c = maxTBsize; c < picWidth; c += maxTBsize) 2663 | { 2664 | for(Int r = 0; r < picHeight; r++) 2665 | { 2666 | p2 = Rec[c-3]; 2667 | p1 = Rec[c-2]; 2668 | p0 = Rec[c-1]; 2669 | q0 = Rec[c]; 2670 | q1 = Rec[c+1]; 2671 | q2 = Rec[c+2]; 2672 | a = ((abs(p2-(p1<<1)+p0)+abs(q0-(q1<<1)+q2))<<1); 2673 | if ( thr1 < a && a < thr2) 2674 | { 2675 | colSAD[colIdx] += abs(p0 - q0); 2676 | } 2677 | Rec += stride; 2678 | } 2679 | colIdx++; 2680 | Rec = tempRec; 2681 | } 2682 | 2683 | // Analyze horizontal artifact edges 2684 | for(Int r = maxTBsize; r < picHeight; r += maxTBsize) 2685 | { 2686 | for(Int c = 0; c < picWidth; c++) 2687 | { 2688 | p2 = Rec[c + (r-3)*stride]; 2689 | p1 = Rec[c + (r-2)*stride]; 2690 | p0 = Rec[c + (r-1)*stride]; 2691 | q0 = Rec[c + r*stride]; 2692 | q1 = Rec[c + (r+1)*stride]; 2693 | q2 = Rec[c + (r+2)*stride]; 2694 | a = ((abs(p2-(p1<<1)+p0)+abs(q0-(q1<<1)+q2))<<1); 2695 | if (thr1 < a && a < thr2) 2696 | { 2697 | rowSAD[rowIdx] += abs(p0 - q0); 2698 | } 2699 | } 2700 | rowIdx++; 2701 | } 2702 | } 2703 | 2704 | UInt64 colSADsum = 0; 2705 | UInt64 rowSADsum = 0; 2706 | for(Int c = 0; c < noCol-1; c++) 2707 | { 2708 | colSADsum += colSAD[c]; 2709 | } 2710 | for(Int r = 0; r < noRows-1; r++) 2711 | { 2712 | rowSADsum += rowSAD[r]; 2713 | } 2714 | 2715 | colSADsum <<= 10; 2716 | rowSADsum <<= 10; 2717 | colSADsum /= (noCol-1); 2718 | colSADsum /= picHeight; 2719 | rowSADsum /= (noRows-1); 2720 | rowSADsum /= picWidth; 2721 | 2722 | UInt64 avgSAD = ((colSADsum + rowSADsum)>>1); 2723 | avgSAD >>= (bitDepthLuma-8); 2724 | 2725 | if ( avgSAD > 2048 ) 2726 | { 2727 | avgSAD >>= 9; 2728 | Int offset = Clip3(2,6,(Int)avgSAD); 2729 | for (Int i=0; igetSlice(i)->setDeblockingFilterOverrideFlag(true); 2732 | pcPic->getSlice(i)->setDeblockingFilterDisable(false); 2733 | pcPic->getSlice(i)->setDeblockingFilterBetaOffsetDiv2( offset ); 2734 | pcPic->getSlice(i)->setDeblockingFilterTcOffsetDiv2( offset ); 2735 | } 2736 | } 2737 | else 2738 | { 2739 | for (Int i=0; igetSlice(i)->setDeblockingFilterOverrideFlag(false); 2742 | pcPic->getSlice(i)->setDeblockingFilterDisable( pcPic->getSlice(i)->getPPS()->getPicDisableDeblockingFilterFlag() ); 2743 | pcPic->getSlice(i)->setDeblockingFilterBetaOffsetDiv2( pcPic->getSlice(i)->getPPS()->getDeblockingFilterBetaOffsetDiv2() ); 2744 | pcPic->getSlice(i)->setDeblockingFilterTcOffsetDiv2( pcPic->getSlice(i)->getPPS()->getDeblockingFilterTcOffsetDiv2() ); 2745 | } 2746 | } 2747 | } 2748 | 2749 | #if W0038_DB_OPT 2750 | Void TEncGOP::applyDeblockingFilterParameterSelection( TComPic* pcPic, const UInt numSlices, const Int gopID ) 2751 | { 2752 | enum DBFltParam 2753 | { 2754 | DBFLT_PARAM_AVAILABLE = 0, 2755 | DBFLT_DISABLE_FLAG, 2756 | DBFLT_BETA_OFFSETD2, 2757 | DBFLT_TC_OFFSETD2, 2758 | //NUM_DBFLT_PARAMS 2759 | }; 2760 | const Int MAX_BETA_OFFSET = 3; 2761 | const Int MIN_BETA_OFFSET = -3; 2762 | const Int MAX_TC_OFFSET = 3; 2763 | const Int MIN_TC_OFFSET = -3; 2764 | 2765 | TComPicYuv* pcPicYuvRec = pcPic->getPicYuvRec(); 2766 | TComPicYuv* pcPicYuvOrg = pcPic ->getPicYuvOrg(); 2767 | 2768 | const Int currQualityLayer = (pcPic->getSlice(0)->getSliceType() != I_SLICE) ? m_pcCfg->getGOPEntry(gopID).m_temporalId+1 : 0; 2769 | assert(currQualityLayer create( m_pcEncTop->getSourceWidth(), m_pcEncTop->getSourceHeight(), m_pcEncTop->getChromaFormatIdc(), pcPic->getSlice(0)->getSPS()->getMaxCUWidth(), pcPic->getSlice(0)->getSPS()->getMaxCUHeight(), pcPic->getSlice(0)->getSPS()->getMaxTotalCUDepth(),true ); 2775 | memset(m_DBParam, 0, sizeof(m_DBParam)); 2776 | } 2777 | 2778 | //preserve current reconstruction 2779 | pcPicYuvRec->copyToPic(m_pcDeblockingTempPicYuv); 2780 | 2781 | const Bool bNoFiltering = m_DBParam[currQualityLayer][DBFLT_PARAM_AVAILABLE] && m_DBParam[currQualityLayer][DBFLT_DISABLE_FLAG]==false /*&& pcPic->getTLayer()==0*/; 2782 | const Int maxBetaOffsetDiv2 = bNoFiltering? Clip3(MIN_BETA_OFFSET, MAX_BETA_OFFSET, m_DBParam[currQualityLayer][DBFLT_BETA_OFFSETD2]+1) : MAX_BETA_OFFSET; 2783 | const Int minBetaOffsetDiv2 = bNoFiltering? Clip3(MIN_BETA_OFFSET, MAX_BETA_OFFSET, m_DBParam[currQualityLayer][DBFLT_BETA_OFFSETD2]-1) : MIN_BETA_OFFSET; 2784 | const Int maxTcOffsetDiv2 = bNoFiltering? Clip3(MIN_TC_OFFSET, MAX_TC_OFFSET, m_DBParam[currQualityLayer][DBFLT_TC_OFFSETD2]+2) : MAX_TC_OFFSET; 2785 | const Int minTcOffsetDiv2 = bNoFiltering? Clip3(MIN_TC_OFFSET, MAX_TC_OFFSET, m_DBParam[currQualityLayer][DBFLT_TC_OFFSETD2]-2) : MIN_TC_OFFSET; 2786 | 2787 | UInt64 distBetaPrevious = std::numeric_limits::max(); 2788 | UInt64 distMin = std::numeric_limits::max(); 2789 | Bool bDBFilterDisabledBest = true; 2790 | Int betaOffsetDiv2Best = 0; 2791 | Int tcOffsetDiv2Best = 0; 2792 | 2793 | for(Int betaOffsetDiv2=maxBetaOffsetDiv2; betaOffsetDiv2>=minBetaOffsetDiv2; betaOffsetDiv2--) 2794 | { 2795 | UInt64 distTcMin = std::numeric_limits::max(); 2796 | for(Int tcOffsetDiv2=maxTcOffsetDiv2; tcOffsetDiv2 >= minTcOffsetDiv2; tcOffsetDiv2--) 2797 | { 2798 | for (Int i=0; igetSlice(i)->setDeblockingFilterOverrideFlag(true); 2801 | pcPic->getSlice(i)->setDeblockingFilterDisable(false); 2802 | pcPic->getSlice(i)->setDeblockingFilterBetaOffsetDiv2( betaOffsetDiv2 ); 2803 | pcPic->getSlice(i)->setDeblockingFilterTcOffsetDiv2( tcOffsetDiv2 ); 2804 | } 2805 | m_pcDeblockingTempPicYuv->copyToPic(pcPicYuvRec); // restore reconstruction 2806 | m_pcLoopFilter->loopFilterPic( pcPic ); 2807 | const UInt64 dist = xFindDistortionFrame(pcPicYuvOrg, pcPicYuvRec, pcPic->getPicSym()->getSPS().getBitDepths()); 2808 | if(dist < distMin) 2809 | { 2810 | distMin = dist; 2811 | bDBFilterDisabledBest = false; 2812 | betaOffsetDiv2Best = betaOffsetDiv2; 2813 | tcOffsetDiv2Best = tcOffsetDiv2; 2814 | } 2815 | if(dist < distTcMin) 2816 | { 2817 | distTcMin = dist; 2818 | } 2819 | else if(tcOffsetDiv2 <-2) 2820 | { 2821 | break; 2822 | } 2823 | } 2824 | if(betaOffsetDiv2<-1 && distTcMin >= distBetaPrevious) 2825 | { 2826 | break; 2827 | } 2828 | distBetaPrevious = distTcMin; 2829 | } 2830 | 2831 | //update: 2832 | m_DBParam[currQualityLayer][DBFLT_PARAM_AVAILABLE] = 1; 2833 | m_DBParam[currQualityLayer][DBFLT_DISABLE_FLAG] = bDBFilterDisabledBest; 2834 | m_DBParam[currQualityLayer][DBFLT_BETA_OFFSETD2] = betaOffsetDiv2Best; 2835 | m_DBParam[currQualityLayer][DBFLT_TC_OFFSETD2] = tcOffsetDiv2Best; 2836 | 2837 | m_pcDeblockingTempPicYuv->copyToPic(pcPicYuvRec); //restore reconstruction 2838 | 2839 | if(bDBFilterDisabledBest) 2840 | { 2841 | for (Int i=0; igetSlice(i)->setDeblockingFilterOverrideFlag(true); 2844 | pcPic->getSlice(i)->setDeblockingFilterDisable(true); 2845 | } 2846 | } 2847 | else if(betaOffsetDiv2Best ==pcPic->getSlice(0)->getPPS()->getDeblockingFilterBetaOffsetDiv2() && tcOffsetDiv2Best==pcPic->getSlice(0)->getPPS()->getDeblockingFilterTcOffsetDiv2()) 2848 | { 2849 | for (Int i=0; igetSlice(i)->setDeblockingFilterOverrideFlag(false); 2852 | pcPic->getSlice(i)->setDeblockingFilterDisable( pcPic->getSlice(i)->getPPS()->getPicDisableDeblockingFilterFlag() ); 2853 | pcPic->getSlice(i)->setDeblockingFilterBetaOffsetDiv2( pcPic->getSlice(i)->getPPS()->getDeblockingFilterBetaOffsetDiv2() ); 2854 | pcPic->getSlice(i)->setDeblockingFilterTcOffsetDiv2( pcPic->getSlice(i)->getPPS()->getDeblockingFilterTcOffsetDiv2() ); 2855 | } 2856 | } 2857 | else 2858 | { 2859 | for (Int i=0; igetSlice(i)->setDeblockingFilterOverrideFlag(true); 2862 | pcPic->getSlice(i)->setDeblockingFilterDisable( false ); 2863 | pcPic->getSlice(i)->setDeblockingFilterBetaOffsetDiv2(betaOffsetDiv2Best); 2864 | pcPic->getSlice(i)->setDeblockingFilterTcOffsetDiv2(tcOffsetDiv2Best); 2865 | } 2866 | } 2867 | } 2868 | 2869 | #endif 2870 | //! \} 2871 | -------------------------------------------------------------------------------- /hm_source/UTILS.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import tensorflow as tf 3 | import math, os, random 4 | from PIL import Image 5 | from VDSRUNK import BATCH_SIZE 6 | from VDSRUNK import PATCH_SIZE 7 | 8 | def normalize(x): 9 | x = x / 255. 10 | return truncate(x, 0., 1.) 11 | 12 | def denormalize(x): 13 | x = x * 255. 14 | return truncate(x, 0., 255.) 15 | 16 | def truncate(input, min, max): 17 | input = np.where(input > min, input, min) 18 | input = np.where(input < max, input, max) 19 | return input 20 | 21 | def remap(input): 22 | input = 16+219/255*input 23 | #return tf.clip_by_value(input, 16, 235).eval() 24 | return truncate(input, 16.0, 235.0) 25 | 26 | def deremap(input): 27 | input = (input-16)*255/219 28 | #return tf.clip_by_value(input, 0, 255).eval() 29 | return truncate(input, 0.0, 255.0) 30 | 31 | def load_file_list(directory): 32 | list = [] 33 | for filename in [y for y in os.listdir(directory) if os.path.isfile(os.path.join(directory,y))]: 34 | list.append(os.path.join(directory,filename)) 35 | return sorted(list) 36 | 37 | def get_train_list(lowList, highList): 38 | assert len(lowList) == len(highList), "low:%d, high:%d"%(len(lowList), len(highList)) 39 | train_list = [] 40 | for i in range(len(lowList)): 41 | train_list.append([lowList[i], highList[i]]) 42 | return train_list 43 | 44 | def prepare_nn_data(train_list, idx_img=None): 45 | i = np.random.randint(len(train_list)) if (idx_img is None) else idx_img 46 | 47 | input_image = c_getYdata(train_list[i][0]) 48 | gt_image = c_getYdata(train_list[i][1]) 49 | 50 | input_list = [] 51 | gt_list = [] 52 | inputcbcr_list = [] 53 | 54 | for idx in range(BATCH_SIZE): 55 | 56 | #crop images to the disired size. 57 | input_imgY, gt_imgY = crop(input_image, gt_image, PATCH_SIZE[0], PATCH_SIZE[1], "ndarray") 58 | 59 | #normalize 60 | input_imgY = normalize(input_imgY) 61 | gt_imgY = normalize(gt_imgY) 62 | 63 | input_list.append(input_imgY) 64 | gt_list.append(gt_imgY) 65 | #inputcbcr_list.append(input_imgCbCr) 66 | 67 | input_list = np.resize(input_list, (BATCH_SIZE, PATCH_SIZE[0], PATCH_SIZE[1], 1)) 68 | gt_list = np.resize(gt_list, (BATCH_SIZE, PATCH_SIZE[0], PATCH_SIZE[1], 1)) 69 | #inputcbcr_list = np.resize(inputcbcr_list, (BATCH_SIZE, PATCH_SIZE[0], PATCH_SIZE[1], 2)) 70 | 71 | return input_list, gt_list, inputcbcr_list 72 | 73 | def getWH(yuvfileName): 74 | #w_included , h_included = os.path.basename(yuvfileName).split('x') 75 | #w = w_included.split('_')[-1] 76 | #h = h_included.split('_')[0] 77 | wxh = os.path.basename(yuvfileName).split('_')[-2] 78 | w, h = wxh.split('x') 79 | return int(w), int(h) 80 | 81 | def getYdata(path, size): 82 | w= size[0] 83 | h=size[1] 84 | #print(w,h) 85 | Yt = np.zeros([h, w], dtype="uint8", order='C') 86 | with open(path, 'rb') as fp: 87 | fp.seek(0, 0) 88 | Yt = fp.read() 89 | tem = Image.frombytes('L', [w, h], Yt) 90 | 91 | Yt = np.asarray(tem, dtype='float32') 92 | 93 | # for n in range(h): 94 | # for m in range(w): 95 | # Yt[n, m] = ord(fp.read(1)) 96 | 97 | return Yt 98 | 99 | def c_getYdata(path): 100 | return getYdata(path, getWH(path)) 101 | 102 | def img2y(input_img): 103 | if np.asarray(input_img).shape[2] == 3: 104 | input_imgY = input_img.convert('YCbCr').split()[0] 105 | input_imgCb, input_imgCr = input_img.convert('YCbCr').split()[1:3] 106 | 107 | input_imgY = np.asarray(input_imgY, dtype='float32') 108 | input_imgCb = np.asarray(input_imgCb, dtype='float32') 109 | input_imgCr = np.asarray(input_imgCr, dtype='float32') 110 | 111 | 112 | #Concatenate Cb, Cr components for easy, they are used in pair anyway. 113 | input_imgCb = np.expand_dims(input_imgCb,2) 114 | input_imgCr = np.expand_dims(input_imgCr,2) 115 | input_imgCbCr = np.concatenate((input_imgCb, input_imgCr), axis=2) 116 | 117 | elif np.asarray(input_img).shape[2] == 1: 118 | print("This image has one channal only.") 119 | #If the num of channal is 1, remain. 120 | input_imgY = input_img 121 | input_imgCbCr = None 122 | else: 123 | print("The num of channal is neither 3 nor 1.") 124 | exit() 125 | return input_imgY, input_imgCbCr 126 | 127 | def crop(input_image, gt_image, patch_width, patch_height, img_type): 128 | assert type(input_image) == type(gt_image), "types are different." 129 | #return a ndarray object 130 | if img_type == "ndarray": 131 | in_row_ind = random.randint(0,input_image.shape[0]-patch_width) 132 | in_col_ind = random.randint(0,input_image.shape[1]-patch_height) 133 | 134 | input_cropped = input_image[in_row_ind:in_row_ind+patch_width, in_col_ind:in_col_ind+patch_height] 135 | gt_cropped = gt_image[in_row_ind:in_row_ind+patch_width, in_col_ind:in_col_ind+patch_height] 136 | 137 | #return an "Image" object 138 | elif img_type == "Image": 139 | in_row_ind = random.randint(0,input_image.size[0]-patch_width) 140 | in_col_ind = random.randint(0,input_image.size[1]-patch_height) 141 | 142 | input_cropped = input_image.crop(box=(in_row_ind, in_col_ind, in_row_ind+patch_width, in_col_ind+patch_height)) 143 | gt_cropped = gt_image.crop(box=(in_row_ind, in_col_ind, in_row_ind+patch_width, in_col_ind+patch_height)) 144 | 145 | return input_cropped, gt_cropped 146 | 147 | def save_images(inputY, inputCbCr, size, image_path): 148 | """Save mutiple images into one single image. 149 | 150 | Parameters 151 | ----------- 152 | images : numpy array [batch, w, h, c] 153 | size : list of two int, row and column number. 154 | number of images should be equal or less than size[0] * size[1] 155 | image_path : string. 156 | 157 | Examples 158 | --------- 159 | >>> images = np.random.rand(64, 100, 100, 3) 160 | >>> tl.visualize.save_images(images, [8, 8], 'temp.png') 161 | """ 162 | def merge(images, size): 163 | h, w = images.shape[1], images.shape[2] 164 | img = np.zeros((h * size[0], w * size[1], 3)) 165 | for idx, image in enumerate(images): 166 | i = idx % size[1] 167 | j = idx // size[1] 168 | img[j*h:j*h+h, i*w:i*w+w, :] = image 169 | return img 170 | 171 | inputY = inputY.astype('uint8') 172 | inputCbCr = inputCbCr.astype('uint8') 173 | output_concat = np.concatenate((inputY, inputCbCr), axis=3) 174 | 175 | assert len(output_concat) <= size[0] * size[1], "number of images should be equal or less than size[0] * size[1] {}".format(len(output_concat)) 176 | 177 | new_output = merge(output_concat, size) 178 | 179 | new_output = new_output.astype('uint8') 180 | 181 | img = Image.fromarray(new_output, mode='YCbCr') 182 | img = img.convert('RGB') 183 | img.save(image_path) 184 | 185 | def get_image_batch(train_list,offset,batch_size): 186 | target_list = train_list[offset:offset+batch_size] 187 | input_list = [] 188 | gt_list = [] 189 | inputcbcr_list = [] 190 | for pair in target_list: 191 | input_img = Image.open(pair[0]) 192 | gt_img = Image.open(pair[1]) 193 | 194 | #crop images to the disired size. 195 | input_img, gt_img = crop(input_img, gt_img, PATCH_SIZE[0], PATCH_SIZE[1], "Image") 196 | 197 | #focus on Y channal only 198 | input_imgY, input_imgCbCr = img2y(input_img) 199 | gt_imgY, gt_imgCbCr = img2y(gt_img) 200 | 201 | #input_imgY = normalize(input_imgY) 202 | #gt_imgY = normalize(gt_imgY) 203 | 204 | input_list.append(input_imgY) 205 | gt_list.append(gt_imgY) 206 | inputcbcr_list.append(input_imgCbCr) 207 | 208 | input_list = np.resize(input_list, (batch_size, PATCH_SIZE[0], PATCH_SIZE[1], 1)) 209 | gt_list = np.resize(gt_list, (batch_size, PATCH_SIZE[0], PATCH_SIZE[1], 1)) 210 | 211 | return input_list, gt_list, inputcbcr_list 212 | 213 | def save_test_img(inputY, inputCbCr, path): 214 | assert len(inputY.shape) == 4, "the tensor Y's shape is %s"%inputY.shape 215 | assert inputY.shape[0] == 1, "the fitst component must be 1, has not been completed otherwise.{}".format(inputY.shape) 216 | 217 | inputY = np.squeeze(inputY, axis=0) 218 | inputY = inputY.astype('uint8') 219 | 220 | inputCbCr = inputCbCr.astype('uint8') 221 | 222 | output_concat = np.concatenate((inputY, inputCbCr), axis=2) 223 | img = Image.fromarray(output_concat, mode='YCbCr') 224 | img = img.convert('RGB') 225 | img.save(path) 226 | 227 | def psnr(hr_image, sr_image, max_value=255.0): 228 | eps = 1e-10 229 | if((type(hr_image)==type(np.array([]))) or (type(hr_image)==type([]))): 230 | hr_image_data = np.asarray(hr_image, 'float32') 231 | sr_image_data = np.asarray(sr_image, 'float32') 232 | 233 | diff = sr_image_data - hr_image_data 234 | mse = np.mean(diff*diff) 235 | mse = np.maximum(eps, mse) 236 | return float(10*math.log10(max_value*max_value/mse)) 237 | else: 238 | assert len(hr_image.shape)==4 and len(sr_image.shape)==4 239 | diff = hr_image - sr_image 240 | mse = tf.reduce_mean(tf.square(diff)) 241 | mse = tf.maximum(mse, eps) 242 | return 10*tf.log(max_value*max_value/mse)/math.log(10) 243 | -------------------------------------------------------------------------------- /hm_source/callPython.cpp: -------------------------------------------------------------------------------- 1 | #include "stdio.h" 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | #include 9 | using namespace std; 10 | 11 | extern void conv_y_to_mat(cv::Mat &YYY, unsigned char* pY, int nWidth, int nHeight, int bit_depth); 12 | 13 | cv::Mat callPython(cv::Mat img ,int ccc_type){ 14 | 15 | 16 | int height = img.rows; 17 | int width = img.cols; 18 | cv::Mat recMat; 19 | 20 | PyObject * pModule = NULL; 21 | PyObject * pFuncI = NULL; 22 | PyObject * pFuncP = NULL; 23 | PyObject * pFuncB = NULL; 24 | PyObject * pArgs = NULL; 25 | 26 | Py_Initialize(); 27 | 28 | 29 | pModule = PyImport_ImportModule("TEST_qp37"); 30 | 31 | if (!pModule) { 32 | printf("don't get Pmodule\n"); 33 | PyErr_Print(); 34 | Py_Finalize(); 35 | return recMat; 36 | } 37 | pFuncI = PyObject_GetAttrString(pModule, "modelI"); 38 | if (!pFuncI) { 39 | printf("don't get I function!"); 40 | Py_Finalize(); 41 | return recMat; 42 | } 43 | pFuncP = PyObject_GetAttrString(pModule, "modelP"); 44 | if (!pFuncP) { 45 | printf("don't get P function!"); 46 | Py_Finalize(); 47 | return recMat; 48 | } 49 | 50 | pFuncB = PyObject_GetAttrString(pModule, "modelB"); 51 | 52 | if (!pFuncB) { 53 | printf("don't get B function!"); 54 | Py_Finalize(); 55 | return recMat; 56 | } 57 | 58 | PyObject* list = PyList_New(height); 59 | pArgs = PyTuple_New(1); 60 | PyObject** lists = new PyObject*[height]; 61 | uchar *temp; 62 | for (int i = 0; i < height; i++) 63 | { 64 | temp = img.ptr(i); 65 | lists[i] = PyList_New(0); 66 | for (int j = 0; j < width; j++) 67 | { 68 | PyList_Append(lists[i], Py_BuildValue("i", temp[j])); 69 | } 70 | PyList_SetItem(list, i, lists[i]); 71 | } 72 | PyTuple_SetItem(pArgs, 0, list); 73 | 74 | PyObject *presult = NULL; 75 | if (ccc_type == 1){ 76 | printf("c++, in I\n"); 77 | presult = PyEval_CallObject(pFuncI, pArgs); 78 | } 79 | else if (ccc_type == 2){ 80 | printf("c++, in P\n"); 81 | presult = PyEval_CallObject(pFuncP, pArgs); 82 | } 83 | else{ 84 | printf("c++, in B\n"); 85 | presult = PyEval_CallObject(pFuncB, pArgs); 86 | } 87 | Py_ssize_t high = PyList_Size(presult), wid = PyList_Size(PyList_GetItem(presult, 0)); 88 | int buflen = high * wid; 89 | unsigned char *buf = new unsigned char[buflen]; 90 | unsigned char *p = buf; 91 | int s; 92 | 93 | for (int i = 0; i < high; i++) 94 | { 95 | for (int j = 0; j < wid; j++) 96 | { 97 | PyArg_Parse(PyList_GetItem(PyList_GetItem(presult, i), j), "i", &s); 98 | *p = s; 99 | p++; 100 | } 101 | } 102 | 103 | 104 | conv_y_to_mat(recMat, buf, wid, high, 8); 105 | return recMat; 106 | } 107 | -------------------------------------------------------------------------------- /rhcnn/RHCNN.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | import numpy as np 3 | 4 | 5 | def model(input_tensor): 6 | # with tf.device("/gpu:0"): 7 | arrayNum = [64, 128, 256, 128, 64, 1] 8 | # arrayNum = [64, 32, 1] 9 | weights = [] 10 | tensor = input_tensor 11 | convId = 0 12 | tem = None 13 | 14 | conv_w_min = tf.get_variable('conv_%02d_Middle_w' % (convId), [3, 3, 1, 64], 15 | initializer=tf.contrib.layers.xavier_initializer()) 16 | conv_b_min = tf.get_variable('conv_%02d_Middle_b' % (convId), [64], initializer=tf.constant_initializer(0)) 17 | tf.add_to_collection(tf.GraphKeys.WEIGHTS, conv_w_min) 18 | tf.add_to_collection(tf.GraphKeys.BIASES, conv_b_min) 19 | weights.append(conv_w_min) 20 | weights.append(conv_b_min) 21 | tensorMin3 = tf.nn.relu( 22 | tf.nn.bias_add(tf.nn.conv2d(tensor, conv_w_min, strides=[1, 1, 1, 1], padding='SAME'), conv_b_min)) 23 | 24 | conv_wL = tf.get_variable('conv_%02d_Left_w' % (convId), [3, 3, 64, 64], 25 | initializer=tf.contrib.layers.xavier_initializer()) 26 | conv_bL = tf.get_variable('conv_%02d_Left_b' % (convId), [64], initializer=tf.constant_initializer(0)) 27 | weights.append(conv_wL) 28 | weights.append(conv_bL) 29 | 30 | tensorL = tf.nn.relu( 31 | tf.nn.bias_add(tf.nn.conv2d(tensorMin3, conv_wL, strides=[1, 1, 1, 1], padding='SAME'), conv_bL)) 32 | conv_wR = tf.get_variable('conv_%02d_Right_w' % (convId), [3, 3, 64, 64], 33 | initializer=tf.contrib.layers.xavier_initializer()) 34 | conv_bR = tf.get_variable('conv_%02d_Right_b' % (convId), [64], initializer=tf.constant_initializer(0)) 35 | weights.append(conv_wR) 36 | weights.append(conv_bR) 37 | tensorR = tf.nn.relu( 38 | tf.nn.bias_add(tf.nn.conv2d(tensorMin3, conv_wR, strides=[1, 1, 1, 1], padding='SAME'), conv_bR)) 39 | tensorConcat = tf.concat([tensorL, tensorR], axis=3) # axis = 0??? 40 | conv_1x1 = tf.get_variable('conv_%02d_1x1_w' % (convId), [1, 1, 128, 64], 41 | initializer=tf.contrib.layers.xavier_initializer()) 42 | convb_1x1 = tf.get_variable('conv_%02d_1x1_b' % (convId), [64], 43 | initializer=tf.constant_initializer(0)) 44 | weights.append(conv_1x1) 45 | weights.append(convb_1x1) 46 | tensor = tf.nn.relu(tf.nn.bias_add(tf.nn.conv2d(tensorConcat, conv_1x1, strides=[1, 1, 1, 1], padding='SAME'), convb_1x1)) 47 | convId += 1 48 | 49 | for i in range(1, 6): 50 | conv_w_min = tf.get_variable('conv_%02d_Middle_w' % (convId), [3, 3, arrayNum[i - 1], arrayNum[i]], 51 | initializer=tf.contrib.layers.xavier_initializer()) 52 | conv_b_min = tf.get_variable('conv_%02d_Middle_b' % (convId), [arrayNum[i]], 53 | initializer=tf.constant_initializer(0)) 54 | 55 | weights.append(conv_w_min) 56 | weights.append(conv_b_min) 57 | tensorMin = tf.nn.relu( 58 | tf.nn.bias_add(tf.nn.conv2d(tensor, conv_w_min, strides=[1, 1, 1, 1], padding='SAME'), conv_b_min)) 59 | 60 | conv_wL = tf.get_variable('conv_%02d_Left_w' % (convId), [3, 3, arrayNum[i], arrayNum[i]], 61 | initializer=tf.contrib.layers.xavier_initializer()) 62 | conv_bL = tf.get_variable('conv_%02d_Left_b' % (convId), [arrayNum[i]], initializer=tf.constant_initializer(0)) 63 | weights.append(conv_wL) 64 | weights.append(conv_bL) 65 | tensorL = tf.nn.relu( 66 | tf.nn.bias_add(tf.nn.conv2d(tensorMin, conv_wL, strides=[1, 1, 1, 1], padding='SAME'), conv_bL)) 67 | conv_wR = tf.get_variable('conv_%02d_Right_w' % (convId), [3, 3, arrayNum[i], arrayNum[i]], 68 | initializer=tf.contrib.layers.xavier_initializer()) 69 | conv_bR = tf.get_variable('conv_%02d_Right_b' % (convId), [arrayNum[i]], initializer=tf.constant_initializer(0)) 70 | weights.append(conv_wR) 71 | weights.append(conv_bR) 72 | tensorR = tf.nn.relu( 73 | tf.nn.bias_add(tf.nn.conv2d(tensorMin, conv_wR, strides=[1, 1, 1, 1], padding='SAME'), conv_bR)) 74 | tensorConcat = tf.concat([tensorL, tensorR], axis=3) # axis = 0??? 75 | conv_1x1 = tf.get_variable('conv_%02d_1x1_w' % (convId), [1, 1, 2 * arrayNum[i], arrayNum[i]], 76 | initializer=tf.contrib.layers.xavier_initializer()) 77 | convb_1x1 = tf.get_variable('conv_%02d_1x1_b' % (convId), [arrayNum[i]], 78 | initializer=tf.constant_initializer(0)) 79 | weights.append(conv_1x1) 80 | weights.append(convb_1x1) 81 | if i == 3: 82 | tem = tensor 83 | temw = conv_1x1 84 | 85 | tensor = tf.nn.relu(tf.nn.bias_add(tf.nn.conv2d(tensorConcat, conv_1x1, strides=[1, 1, 1, 1], padding='SAME'), convb_1x1)) 86 | convId += 1 87 | 88 | conv_w_end = tf.get_variable("conv_%02d_end_w" % (convId), [1, 1, 1, 1], 89 | initializer=tf.contrib.layers.xavier_initializer()) 90 | conv_b_end = tf.get_variable("conv_%02d_end_b" % (convId), [1], initializer=tf.constant_initializer(0)) 91 | weights.append(conv_w_end) 92 | weights.append(conv_b_end) 93 | tensor_end = tf.nn.bias_add(tf.nn.conv2d(tensor, conv_w_end, strides=[1, 1, 1, 1], padding='SAME'), conv_b_end) 94 | 95 | # tensor_end = tf.add(input_tensor, tensor_end) 96 | # return tensor_end, [weights, input_tensor, tensorMin3, tem, temw, tensor_end] 97 | return tensor_end, weights -------------------------------------------------------------------------------- /rhcnn/RHCNN_B_qp37/RHCNN_B_qp37_550.ckpt.data-00000-of-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/rhcnn/RHCNN_B_qp37/RHCNN_B_qp37_550.ckpt.data-00000-of-00001 -------------------------------------------------------------------------------- /rhcnn/RHCNN_B_qp37/RHCNN_B_qp37_550.ckpt.index: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/rhcnn/RHCNN_B_qp37/RHCNN_B_qp37_550.ckpt.index -------------------------------------------------------------------------------- /rhcnn/RHCNN_B_qp37/RHCNN_B_qp37_550.ckpt.meta: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/rhcnn/RHCNN_B_qp37/RHCNN_B_qp37_550.ckpt.meta -------------------------------------------------------------------------------- /rhcnn/RHCNN_I_qp37/RHCNN_I_qp37_589.ckpt.data-00000-of-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/rhcnn/RHCNN_I_qp37/RHCNN_I_qp37_589.ckpt.data-00000-of-00001 -------------------------------------------------------------------------------- /rhcnn/RHCNN_I_qp37/RHCNN_I_qp37_589.ckpt.index: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/rhcnn/RHCNN_I_qp37/RHCNN_I_qp37_589.ckpt.index -------------------------------------------------------------------------------- /rhcnn/RHCNN_I_qp37/RHCNN_I_qp37_589.ckpt.meta: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/rhcnn/RHCNN_I_qp37/RHCNN_I_qp37_589.ckpt.meta -------------------------------------------------------------------------------- /rhcnn/RHCNN_P_qp37/RHCNN_ldp_qp37_566.ckpt.data-00000-of-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/rhcnn/RHCNN_P_qp37/RHCNN_ldp_qp37_566.ckpt.data-00000-of-00001 -------------------------------------------------------------------------------- /rhcnn/RHCNN_P_qp37/RHCNN_ldp_qp37_566.ckpt.index: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/rhcnn/RHCNN_P_qp37/RHCNN_ldp_qp37_566.ckpt.index -------------------------------------------------------------------------------- /rhcnn/RHCNN_P_qp37/RHCNN_ldp_qp37_566.ckpt.meta: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/rhcnn/RHCNN_P_qp37/RHCNN_ldp_qp37_566.ckpt.meta -------------------------------------------------------------------------------- /sefcnn/SEFCNN.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | import numpy as np 3 | #PATCH_SIZE = (35, 35) 4 | 5 | def seblock(temp_tensor, convId, weights): 6 | t_tensor1 = None 7 | t_tensor2 = None 8 | t_tensor3 = None 9 | conv_secondID = 0 10 | 11 | 12 | for i in range(3): 13 | if(i==0): 14 | conv_w = tf.get_variable("conv_%02d_%02d_w" % (convId,conv_secondID), [3,3,64,64], initializer=tf.random_normal_initializer(stddev=np.sqrt(2.0/9/64))) 15 | conv_b = tf.get_variable("conv_%02d_%02d_b" % (convId,conv_secondID), [64], initializer=tf.constant_initializer(0)) 16 | weights.append(conv_w) 17 | weights.append(conv_b) 18 | #t_tensor1 = batch_norm_layer(t_tensor1, TRAIN_PHASE, "conv_%02d_%02d" % (convId,conv_secondID)) #bn,scale 19 | t_tensor1 = tf.nn.relu(tf.nn.bias_add(tf.nn.conv2d(temp_tensor, conv_w, strides=[1,1,1,1], padding='SAME'), conv_b)) 20 | elif(i==1): 21 | conv_w = tf.get_variable("conv_%02d_%02d_w" % (convId,conv_secondID), [3,3,64,64], initializer=tf.random_normal_initializer(stddev=np.sqrt(2.0/9/64))) 22 | conv_b = tf.get_variable("conv_%02d_%02d_b" % (convId,conv_secondID), [64], initializer=tf.constant_initializer(0)) 23 | weights.append(conv_w) 24 | weights.append(conv_b) 25 | #t_tensor1 = batch_norm_layer(t_tensor1, TRAIN_PHASE, "conv_%02d_%02d" % (convId,conv_secondID)) #bn,scale 26 | t_tensor1 = tf.nn.relu(tf.nn.bias_add(tf.nn.conv2d(t_tensor1, conv_w, strides=[1,1,1,1], padding='SAME'), conv_b)) 27 | else: 28 | # no relu 29 | conv_w = tf.get_variable("conv_%02d_%02d_w" % (convId,conv_secondID), [3,3,64,64], initializer=tf.random_normal_initializer(stddev=np.sqrt(2.0/9/64))) 30 | conv_b = tf.get_variable("conv_%02d_%02d_b" % (convId,conv_secondID), [64], initializer=tf.constant_initializer(0)) 31 | weights.append(conv_w) 32 | weights.append(conv_b) 33 | #t_tensor1 = batch_norm_layer(t_tensor1, TRAIN_PHASE, "conv_%02d_%02d" % (convId,conv_secondID)) #bn,scale 34 | t_tensor1 = tf.nn.bias_add(tf.nn.conv2d(t_tensor1, conv_w, strides=[1,1,1,1], padding='SAME'), conv_b) 35 | conv_secondID += 1 36 | 37 | #Now t_tensor1 has passed through 3x3,64 three times 38 | 39 | #fetch t_tensor2 by globel average pooling 40 | #t_tensor2 = tf.nn.avg_pool(t_tensor1, ksize=[1, PATCH_SIZE[0], PATCH_SIZE[1], 1], strides=[1,1,1,1], padding='SAME') 41 | t_tensor2 = tf.reduce_mean(t_tensor1,[1, 2]) 42 | t_tensor2 = tf.reshape(t_tensor2, [t_tensor1.shape[0], 1, 1, t_tensor1.shape[-1]]) 43 | 44 | 45 | #1x1_down 46 | conv_w = tf.get_variable("conv_%02d_%02d_w" % (convId,conv_secondID), [1,1,64,64], initializer=tf.random_normal_initializer(stddev=np.sqrt(2.0/64))) 47 | conv_b = tf.get_variable("conv_%02d_%02d_b" % (convId,conv_secondID), [64], initializer=tf.constant_initializer(0)) 48 | conv_secondID += 1 49 | weights.append(conv_w) 50 | weights.append(conv_b) 51 | #relu 52 | t_tensor2 = tf.nn.relu(tf.nn.bias_add(tf.nn.conv2d(t_tensor2, conv_w, strides=[1,1,1,1], padding='SAME'), conv_b)) 53 | 54 | #1x1_up 55 | conv_w = tf.get_variable("conv_%02d_%02d_w" % (convId,conv_secondID), [1,1,64,64], initializer=tf.random_normal_initializer(stddev=np.sqrt(2.0/64))) 56 | conv_b = tf.get_variable("conv_%02d_%02d_b" % (convId,conv_secondID), [64], initializer=tf.constant_initializer(0)) 57 | conv_secondID += 1 58 | weights.append(conv_w) 59 | weights.append(conv_b) 60 | #sigmoid 61 | t_tensor2 = tf.sigmoid(tf.nn.bias_add(tf.nn.conv2d(t_tensor2, conv_w, strides=[1,1,1,1], padding='SAME'), conv_b)) 62 | 63 | #gain t_tensor3 after 1x1,64 ,no relu 64 | conv_w = tf.get_variable("conv_%02d_%02d_w" % (convId,conv_secondID), [1,1,64,64], initializer=tf.random_normal_initializer(stddev=np.sqrt(2.0/64))) 65 | conv_b = tf.get_variable("conv_%02d_%02d_b" % (convId,conv_secondID), [64], initializer=tf.constant_initializer(0)) 66 | conv_secondID += 1 67 | weights.append(conv_w) 68 | weights.append(conv_b) 69 | t_tensor3 = tf.nn.bias_add(tf.nn.conv2d(temp_tensor, conv_w, strides=[1,1,1,1], padding='SAME'), conv_b) 70 | #t_tensor3 = batch_norm_layer(t_tensor3, TRAIN_PHASE, "conv_%02d_%02d" % (convId,conv_secondID)) #bn,scale 71 | 72 | #t_tensor2 * t_tensor1 + t_tensor3 73 | t_tensor1 = tf.nn.relu( tf.add(tf.multiply(t_tensor1, t_tensor2), t_tensor3)) 74 | 75 | return t_tensor1 76 | 77 | def model(input_tensor): 78 | #VDSR15_SE_VDSR 79 | # with tf.device("/gpu:0"): 80 | weights = [] 81 | tensor = None 82 | convId = 0 83 | 84 | conv_00_w = tf.get_variable("conv_%02d_w" % (convId), [3,3,1,64], initializer=tf.random_normal_initializer(stddev=np.sqrt(2.0/9))) 85 | conv_00_b = tf.get_variable("conv_%02d_b" % (convId), [64], initializer=tf.constant_initializer(0)) 86 | convId += 1 87 | weights.append(conv_00_w) 88 | weights.append(conv_00_b) 89 | tensor = tf.nn.relu(tf.nn.bias_add(tf.nn.conv2d(input_tensor, conv_00_w, strides=[1,1,1,1], padding='SAME'), conv_00_b)) 90 | 91 | for i in range(14): 92 | conv_w = tf.get_variable("conv_%02d_w" % (convId), [3,3,64,64], initializer=tf.random_normal_initializer(stddev=np.sqrt(2.0/9/64))) 93 | conv_b = tf.get_variable("conv_%02d_b" % (convId), [64], initializer=tf.constant_initializer(0)) 94 | convId += 1 95 | weights.append(conv_w) 96 | weights.append(conv_b) 97 | tensor = tf.nn.relu(tf.nn.bias_add(tf.nn.conv2d(tensor, conv_w, strides=[1,1,1,1], padding='SAME'), conv_b)) 98 | 99 | tensor = seblock(tensor, convId, weights) 100 | convId += 1 101 | '''for i in range(7): 102 | conv_w = tf.get_variable("conv_%02d_w" % (convId), [3,3,64,64], initializer=tf.random_normal_initializer(stddev=np.sqrt(2.0/9/64))) 103 | conv_b = tf.get_variable("conv_%02d_b" % (convId), [64], initializer=tf.constant_initializer(0)) 104 | convId += 1 105 | weights.append(conv_w) 106 | weights.append(conv_b) 107 | tensor = tf.nn.relu(tf.nn.bias_add(tf.nn.conv2d(tensor, conv_w, strides=[1,1,1,1], padding='SAME'), conv_b))''' 108 | 109 | conv_w = tf.get_variable("conv_%02d_w" % (convId), [3,3,64,1], initializer=tf.random_normal_initializer(stddev=np.sqrt(2.0/9/64))) 110 | conv_b = tf.get_variable("conv_%02d_b" % (convId), [1], initializer=tf.constant_initializer(0)) 111 | convId += 1 112 | weights.append(conv_w) 113 | weights.append(conv_b) 114 | tensor = tf.nn.bias_add(tf.nn.conv2d(tensor, conv_w, strides=[1,1,1,1], padding='SAME'), conv_b) 115 | 116 | tensor = tf.add(tensor, input_tensor) 117 | 118 | 119 | return tensor, weights 120 | -------------------------------------------------------------------------------- /sefcnn/VDSR12.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | import numpy as np 3 | 4 | def model(input_tensor): 5 | # with tf.device("/gpu:0"): 6 | weights = [] 7 | tensor = None 8 | convId = 0 9 | conv_00_w = tf.get_variable("conv_%02d_w" % (convId), [3,3,1,64], initializer=tf.random_normal_initializer(stddev=np.sqrt(2.0/9))) 10 | conv_00_b = tf.get_variable("conv_%02d_b" % (convId), [64], initializer=tf.constant_initializer(0)) 11 | convId += 1 12 | weights.append(conv_00_w) 13 | weights.append(conv_00_b) 14 | tensor = tf.nn.relu(tf.nn.bias_add(tf.nn.conv2d(input_tensor, conv_00_w, strides=[1,1,1,1], padding='SAME'), conv_00_b)) 15 | 16 | for i in range(10): 17 | conv_w = tf.get_variable("conv_%02d_w" % (convId), [3,3,64,64], initializer=tf.random_normal_initializer(stddev=np.sqrt(2.0/9/64))) 18 | conv_b = tf.get_variable("conv_%02d_b" % (convId), [64], initializer=tf.constant_initializer(0)) 19 | convId += 1 20 | weights.append(conv_w) 21 | weights.append(conv_b) 22 | tensor = tf.nn.relu(tf.nn.bias_add(tf.nn.conv2d(tensor, conv_w, strides=[1,1,1,1], padding='SAME'), conv_b)) 23 | 24 | conv_w = tf.get_variable("conv_%02d_w" % (convId), [3,3,64,1], initializer=tf.random_normal_initializer(stddev=np.sqrt(2.0/9/64))) 25 | conv_b = tf.get_variable("conv_%02d_b" % (convId), [1], initializer=tf.constant_initializer(0)) 26 | convId += 1 27 | weights.append(conv_w) 28 | weights.append(conv_b) 29 | tensor = tf.nn.bias_add(tf.nn.conv2d(tensor, conv_w, strides=[1,1,1,1], padding='SAME'), conv_b) 30 | 31 | tensor = tf.add(tensor, input_tensor) 32 | return tensor, weights 33 | -------------------------------------------------------------------------------- /sefcnn/VDSR16.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | import numpy as np 3 | 4 | def model(input_tensor): 5 | # with tf.device("/gpu:0"): 6 | weights = [] 7 | tensor = None 8 | convId = 0 9 | conv_00_w = tf.get_variable("conv_%02d_w" % (convId), [3,3,1,64], initializer=tf.random_normal_initializer(stddev=np.sqrt(2.0/9))) 10 | conv_00_b = tf.get_variable("conv_%02d_b" % (convId), [64], initializer=tf.constant_initializer(0)) 11 | convId += 1 12 | weights.append(conv_00_w) 13 | weights.append(conv_00_b) 14 | tensor = tf.nn.relu(tf.nn.bias_add(tf.nn.conv2d(input_tensor, conv_00_w, strides=[1,1,1,1], padding='SAME'), conv_00_b)) 15 | 16 | for i in range(14): 17 | conv_w = tf.get_variable("conv_%02d_w" % (convId), [3,3,64,64], initializer=tf.random_normal_initializer(stddev=np.sqrt(2.0/9/64))) 18 | conv_b = tf.get_variable("conv_%02d_b" % (convId), [64], initializer=tf.constant_initializer(0)) 19 | convId += 1 20 | weights.append(conv_w) 21 | weights.append(conv_b) 22 | tensor = tf.nn.relu(tf.nn.bias_add(tf.nn.conv2d(tensor, conv_w, strides=[1,1,1,1], padding='SAME'), conv_b)) 23 | 24 | conv_w = tf.get_variable("conv_%02d_w" % (convId), [3,3,64,1], initializer=tf.random_normal_initializer(stddev=np.sqrt(2.0/9/64))) 25 | conv_b = tf.get_variable("conv_%02d_b" % (convId), [1], initializer=tf.constant_initializer(0)) 26 | convId += 1 27 | weights.append(conv_w) 28 | weights.append(conv_b) 29 | tensor = tf.nn.bias_add(tf.nn.conv2d(tensor, conv_w, strides=[1,1,1,1], padding='SAME'), conv_b) 30 | 31 | tensor = tf.add(tensor, input_tensor) 32 | return tensor, weights 33 | -------------------------------------------------------------------------------- /sefcnn/model/VDSR12_qp22/VDSR12_qp22_420.ckpt.data-00000-of-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/VDSR12_qp22/VDSR12_qp22_420.ckpt.data-00000-of-00001 -------------------------------------------------------------------------------- /sefcnn/model/VDSR12_qp22/VDSR12_qp22_420.ckpt.index: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/VDSR12_qp22/VDSR12_qp22_420.ckpt.index -------------------------------------------------------------------------------- /sefcnn/model/VDSR12_qp22/VDSR12_qp22_420.ckpt.meta: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/VDSR12_qp22/VDSR12_qp22_420.ckpt.meta -------------------------------------------------------------------------------- /sefcnn/model/VDSR16_qp27/VDSR16_qp27_389.ckpt.data-00000-of-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/VDSR16_qp27/VDSR16_qp27_389.ckpt.data-00000-of-00001 -------------------------------------------------------------------------------- /sefcnn/model/VDSR16_qp27/VDSR16_qp27_389.ckpt.index: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/VDSR16_qp27/VDSR16_qp27_389.ckpt.index -------------------------------------------------------------------------------- /sefcnn/model/VDSR16_qp27/VDSR16_qp27_389.ckpt.meta: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/VDSR16_qp27/VDSR16_qp27_389.ckpt.meta -------------------------------------------------------------------------------- /sefcnn/model/VDSRx21_SE_qp40_B/VDSRx21_SE_qp40_576.ckpt.data-00000-of-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/VDSRx21_SE_qp40_B/VDSRx21_SE_qp40_576.ckpt.data-00000-of-00001 -------------------------------------------------------------------------------- /sefcnn/model/VDSRx21_SE_qp40_B/VDSRx21_SE_qp40_576.ckpt.index: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/VDSRx21_SE_qp40_B/VDSRx21_SE_qp40_576.ckpt.index -------------------------------------------------------------------------------- /sefcnn/model/VDSRx21_SE_qp40_B/VDSRx21_SE_qp40_576.ckpt.meta: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/VDSRx21_SE_qp40_B/VDSRx21_SE_qp40_576.ckpt.meta -------------------------------------------------------------------------------- /sefcnn/model/VDSRx21_SE_qp40_I/VDSRx21_SE_qp40_I_set2K+1238_394.ckpt.data-00000-of-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/VDSRx21_SE_qp40_I/VDSRx21_SE_qp40_I_set2K+1238_394.ckpt.data-00000-of-00001 -------------------------------------------------------------------------------- /sefcnn/model/VDSRx21_SE_qp40_I/VDSRx21_SE_qp40_I_set2K+1238_394.ckpt.index: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/VDSRx21_SE_qp40_I/VDSRx21_SE_qp40_I_set2K+1238_394.ckpt.index -------------------------------------------------------------------------------- /sefcnn/model/VDSRx21_SE_qp40_I/VDSRx21_SE_qp40_I_set2K+1238_394.ckpt.meta: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/VDSRx21_SE_qp40_I/VDSRx21_SE_qp40_I_set2K+1238_394.ckpt.meta -------------------------------------------------------------------------------- /sefcnn/model/VDSRx21_SE_qp40_P/VDSRx21_SE_qp40_P_2618_129.ckpt.data-00000-of-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/VDSRx21_SE_qp40_P/VDSRx21_SE_qp40_P_2618_129.ckpt.data-00000-of-00001 -------------------------------------------------------------------------------- /sefcnn/model/VDSRx21_SE_qp40_P/VDSRx21_SE_qp40_P_2618_129.ckpt.index: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/VDSRx21_SE_qp40_P/VDSRx21_SE_qp40_P_2618_129.ckpt.index -------------------------------------------------------------------------------- /sefcnn/model/VDSRx21_SE_qp40_P/VDSRx21_SE_qp40_P_2618_129.ckpt.meta: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/VDSRx21_SE_qp40_P/VDSRx21_SE_qp40_P_2618_129.ckpt.meta -------------------------------------------------------------------------------- /sefcnn/model/gradient_Model/VDSRx15_SE_qp37_I_2K_High_Grad_147.ckpt.data-00000-of-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/gradient_Model/VDSRx15_SE_qp37_I_2K_High_Grad_147.ckpt.data-00000-of-00001 -------------------------------------------------------------------------------- /sefcnn/model/gradient_Model/VDSRx15_SE_qp37_I_2K_High_Grad_147.ckpt.index: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/gradient_Model/VDSRx15_SE_qp37_I_2K_High_Grad_147.ckpt.index -------------------------------------------------------------------------------- /sefcnn/model/gradient_Model/VDSRx15_SE_qp37_I_2K_High_Grad_147.ckpt.meta: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/gradient_Model/VDSRx15_SE_qp37_I_2K_High_Grad_147.ckpt.meta -------------------------------------------------------------------------------- /sefcnn/model/sefcnn_B_qp32/sefcnn_B_qp32_500.ckpt.data-00000-of-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/sefcnn_B_qp32/sefcnn_B_qp32_500.ckpt.data-00000-of-00001 -------------------------------------------------------------------------------- /sefcnn/model/sefcnn_B_qp32/sefcnn_B_qp32_500.ckpt.index: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/sefcnn_B_qp32/sefcnn_B_qp32_500.ckpt.index -------------------------------------------------------------------------------- /sefcnn/model/sefcnn_B_qp32/sefcnn_B_qp32_500.ckpt.meta: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/sefcnn_B_qp32/sefcnn_B_qp32_500.ckpt.meta -------------------------------------------------------------------------------- /sefcnn/model/sefcnn_B_qp37/sefcnn_B_qp37_368.ckpt.data-00000-of-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/sefcnn_B_qp37/sefcnn_B_qp37_368.ckpt.data-00000-of-00001 -------------------------------------------------------------------------------- /sefcnn/model/sefcnn_B_qp37/sefcnn_B_qp37_368.ckpt.index: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/sefcnn_B_qp37/sefcnn_B_qp37_368.ckpt.index -------------------------------------------------------------------------------- /sefcnn/model/sefcnn_B_qp37/sefcnn_B_qp37_368.ckpt.meta: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/sefcnn_B_qp37/sefcnn_B_qp37_368.ckpt.meta -------------------------------------------------------------------------------- /sefcnn/model/sefcnn_I_qp32/sefcnn_I_qp32_599.ckpt.data-00000-of-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/sefcnn_I_qp32/sefcnn_I_qp32_599.ckpt.data-00000-of-00001 -------------------------------------------------------------------------------- /sefcnn/model/sefcnn_I_qp32/sefcnn_I_qp32_599.ckpt.index: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/sefcnn_I_qp32/sefcnn_I_qp32_599.ckpt.index -------------------------------------------------------------------------------- /sefcnn/model/sefcnn_I_qp32/sefcnn_I_qp32_599.ckpt.meta: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/sefcnn_I_qp32/sefcnn_I_qp32_599.ckpt.meta -------------------------------------------------------------------------------- /sefcnn/model/sefcnn_I_qp37/sefcnn_I_qp37_model_570.ckpt.data-00000-of-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/sefcnn_I_qp37/sefcnn_I_qp37_model_570.ckpt.data-00000-of-00001 -------------------------------------------------------------------------------- /sefcnn/model/sefcnn_I_qp37/sefcnn_I_qp37_model_570.ckpt.index: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/sefcnn_I_qp37/sefcnn_I_qp37_model_570.ckpt.index -------------------------------------------------------------------------------- /sefcnn/model/sefcnn_I_qp37/sefcnn_I_qp37_model_570.ckpt.meta: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/sefcnn_I_qp37/sefcnn_I_qp37_model_570.ckpt.meta -------------------------------------------------------------------------------- /sefcnn/model/sefcnn_P_qp37/sefcnn_P_qp37_588.ckpt.data-00000-of-00001: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/sefcnn_P_qp37/sefcnn_P_qp37_588.ckpt.data-00000-of-00001 -------------------------------------------------------------------------------- /sefcnn/model/sefcnn_P_qp37/sefcnn_P_qp37_588.ckpt.index: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/sefcnn_P_qp37/sefcnn_P_qp37_588.ckpt.index -------------------------------------------------------------------------------- /sefcnn/model/sefcnn_P_qp37/sefcnn_P_qp37_588.ckpt.meta: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/sefcnn/model/sefcnn_P_qp37/sefcnn_P_qp37_588.ckpt.meta -------------------------------------------------------------------------------- /trainSet/trainSet&testSet.md: -------------------------------------------------------------------------------- 1 | # This document describes in detail the train and test sets we use. 2 | --- 3 | ## Train set : 4 | ### Intra block of all QPs: 5 | - 800 high definition high resolution images of train data in [DIV2K](https://data.vision.ee.ethz.ch/cvl/DIV2K/) 6 | ### Inter block of QP22 and QP27: 7 | - 800 high definition high resolution images of train data in [DIV2K](https://data.vision.ee.ethz.ch/cvl/DIV2K/) 8 | ### Inter block of QP32 and QP37? 9 | - 800 high definition high resolution images of train data in [DIV2K](https://data.vision.ee.ethz.ch/cvl/DIV2K/) 10 | - Some frames in the following 332 sequences: 11 | 12 | | id | sequence | frame | 13 | | --- | :------------------------------------------------------: | ---------------: | 14 | | 1 | 2Rally_1920x1080_30_orig_01_GOP02.yuv | 10,30,50,...,190 | 15 | | 2 | 2Rally_1920x1080_30_orig_01_GOP03.yuv | 10,30,50,...,190 | 16 | | 3 | 2Rally_1920x1080_30_orig_01_GOP04.yuv | 10,30,50,...,190 | 17 | | 4 | 2Rally_1920x1080_30_orig_01_GOP05.yuv | 10,30,50,...,190 | 18 | | 5 | 2Rally_1920x1080_30_orig_01_GOP06.yuv | 10,30,50,...,190 | 19 | | 6 | 2Rally_1920x1080_30_orig_01_GOP07.yuv | 10,30,50,...,190 | 20 | | 7 | 2Rally_1920x1080_30_orig_01_GOP08.yuv | 10,30,50,...,190 | 21 | | 8 | 2Rally_1920x1080_30_orig_01_GOP09.yuv | 10,30,50,...,190 | 22 | | 9 | 2Rally_1920x1080_30_orig_01_GOP10.yuv | 10,30,50,...,190 | 23 | | 10 | 2Rally_1920x1080_30_orig_01_GOP11.yuv | 10,30,50,...,190 | 24 | | 11 | 3fjords_1920x1080_30_orig_01_GOP01.yuv | 10,30,50,...,190 | 25 | | 12 | 3fjords_1920x1080_30_orig_01_GOP02.yuv | 10,30,50,...,190 | 26 | | 13 | 3fjords_1920x1080_30_orig_01_GOP03.yuv | 10,30,50,...,190 | 27 | | 14 | 5costa_1920x1080_30_orig_01_GOP01.yuv | 10,30,50,...,190 | 28 | | 15 | 5costa_1920x1080_30_orig_01_GOP02.yuv | 10,30,50,...,190 | 29 | | 16 | 5costa_1920x1080_30_orig_01_GOP03.yuv | 10,30,50,...,190 | 30 | | 17 | 5costa_1920x1080_30_orig_01_GOP04.yuv | 10,30,50,...,190 | 31 | | 18 | 5costa_1920x1080_30_orig_01_GOP05.yuv | 10,30,50,...,190 | 32 | | 19 | 6hongkong_1920x1080_30_orig_01_GOP01.yuv | 10,30,50,...,190 | 33 | | 20 | 6hongkong_1920x1080_30_orig_01_GOP02.yuv | 10,30,50,...,190 | 34 | | 21 | 6hongkong_1920x1080_30_orig_01_GOP03.yuv | 10,30,50,...,190 | 35 | | 22 | 6hongkong_1920x1080_30_orig_01_GOP04.yuv | 10,30,50,...,190 | 36 | | 23 | 6hongkong_1920x1080_30_orig_01_GOP05.yuv | 10,30,50,...,190 | 37 | | 24 | 6hongkong_1920x1080_30_orig_01_GOP06.yuv | 10,30,50,...,190 | 38 | | 25 | 7_1920x1080_30_orig_01_GOP01.yuv | 10,30,50,...,190 | 39 | | 26 | 7_1920x1080_30_orig_01_GOP02.yuv | 10,30,50,...,190 | 40 | | 27 | 7_1920x1080_30_orig_01_GOP03.yuv | 10,30,50,...,190 | 41 | | 28 | 7_1920x1080_30_orig_01_GOP04.yuv | 10,30,50,...,190 | 42 | | 29 | 7_1920x1080_30_orig_01_GOP05.yuv | 10,30,50,...,190 | 43 | | 30 | 7_1920x1080_30_orig_01_GOP06.yuv | 10,30,50,...,190 | 44 | | 31 | 7_1920x1080_30_orig_01_GOP07.yuv | 10,30,50,...,190 | 45 | | 32 | 8americanfootball_1920x1080_30_orig_01_GOP01.yuv | 10,30,50,...,190 | 46 | | 33 | 8americanfootball_1920x1080_30_orig_01_GOP02.yuv | 10,30,50,...,190 | 47 | | 34 | 8americanfootball_1920x1080_30_orig_01_GOP03.yuv | 10,30,50,...,190 | 48 | | 35 | 8americanfootball_1920x1080_30_orig_01_GOP04.yuv | 10,30,50,...,190 | 49 | | 36 | 8americanfootball_1920x1080_30_orig_01_GOP05.yuv | 10,30,50,...,190 | 50 | | 37 | 8americanfootball_1920x1080_30_orig_01_GOP06.yuv | 10,30,50,...,190 | 51 | | 38 | 8americanfootball_1920x1080_30_orig_01_GOP07.yuv | 10,30,50,...,190 | 52 | | 39 | 10AsianFusion_1920x1080_30_orig_01_GOP01.yuv | 10,30,50,...,190 | 53 | | 40 | 10AsianFusion_1920x1080_30_orig_01_GOP02.yuv | 10,30,50,...,190 | 54 | | 41 | 10AsianFusion_1920x1080_30_orig_01_GOP03.yuv | 10,30,50,...,190 | 55 | | 42 | 10AsianFusion_1920x1080_30_orig_01_GOP04.yuv | 10,30,50,...,190 | 56 | | 43 | 10AsianFusion_1920x1080_30_orig_01_GOP05.yuv | 10,30,50,...,190 | 57 | | 44 | 10AsianFusion_1920x1080_30_orig_01_GOP06.yuv | 10,30,50,...,190 | 58 | | 45 | 11skateboarding_1920x1080_30_orig_01_GOP07.yuv | 10,30,50,...,190 | 59 | | 46 | 11skateboarding_1920x1080_30_orig_01_GOP09.yuv | 10,30,50,...,190 | 60 | | 47 | 11skateboarding_1920x1080_30_orig_01_GOP10.yuv | 10,30,50,...,190 | 61 | | 48 | 11skateboarding_1920x1080_30_orig_01_GOP12.yuv | 10,30,50,...,190 | 62 | | 49 | 11skateboarding_1920x1080_30_orig_01_GOP13.yuv | 10,30,50,...,190 | 63 | | 50 | 12redrockvol3_1920x1080_50_orig_01_GOP01.yuv | 10,30,50,...,190 | 64 | | 51 | 12redrockvol3_1920x1080_50_orig_01_GOP03.yuv | 10,30,50,...,190 | 65 | | 52 | 12redrockvol3_1920x1080_50_orig_01_GOP04.yuv | 10,30,50,...,190 | 66 | | 53 | 12redrockvol3_1920x1080_50_orig_01_GOP05.yuv | 10,30,50,...,190 | 67 | | 54 | 13redrockvol2_1920x1080_50_orig_01_GOP01.yuv | 10,30,50,...,190 | 68 | | 55 | 13redrockvol2_1920x1080_50_orig_01_GOP02.yuv | 10,30,50,...,190 | 69 | | 56 | 13redrockvol2_1920x1080_50_orig_01_GOP03.yuv | 10,30,50,...,190 | 70 | | 57 | 13redrockvol2_1920x1080_50_orig_01_GOP04.yuv | 10,30,50,...,190 | 71 | | 58 | 13redrockvol2_1920x1080_50_orig_01_GOP05.yuv | 10,30,50,...,190 | 72 | | 59 | 13redrockvol2_1920x1080_50_orig_01_GOP06.yuv | 10,30,50,...,190 | 73 | | 60 | 13redrockvol2_1920x1080_50_orig_01_GOP07.yuv | 10,30,50,...,190 | 74 | | 61 | 13redrockvol2_1920x1080_50_orig_01_GOP08.yuv | 10,30,50,...,190 | 75 | | 62 | 13redrockvol2_1920x1080_50_orig_01_GOP09.yuv | 10,30,50,...,190 | 76 | | 63 | 14airacrobatics_1920x1080_50_orig_01_GOP01.yuv | 10,30,50,...,190 | 77 | | 64 | 14airacrobatics_1920x1080_50_orig_01_GOP02.yuv | 10,30,50,...,190 | 78 | | 65 | 14airacrobatics_1920x1080_50_orig_01_GOP03.yuv | 10,30,50,...,190 | 79 | | 66 | 16raptors_1920x1080_50_orig_01_GOP01.yuv | 10,30,50,...,190 | 80 | | 67 | 16raptors_1920x1080_50_orig_01_GOP02.yuv | 10,30,50,...,190 | 81 | | 68 | 16raptors_1920x1080_50_orig_01_GOP03.yuv | 10,30,50,...,190 | 82 | | 69 | 16raptors_1920x1080_50_orig_01_GOP04.yuv | 10,30,50,...,190 | 83 | | 70 | 16raptors_1920x1080_50_orig_01_GOP05.yuv | 10,30,50,...,190 | 84 | | 71 | 18ANIMALS_1920x1080_50_orig_01_GOP01.yuv | 10,30,50,...,190 | 85 | | 72 | 18ANIMALS_1920x1080_50_orig_01_GOP02.yuv | 10,30,50,...,190 | 86 | | 73 | 18ANIMALS_1920x1080_50_orig_01_GOP04.yuv | 10,30,50,...,190 | 87 | | 74 | 18ANIMALS_1920x1080_50_orig_01_GOP05.yuv | 10,30,50,...,190 | 88 | | 75 | 18ANIMALS_1920x1080_50_orig_01_GOP06.yuv | 10,30,50,...,190 | 89 | | 76 | 18ANIMALS_1920x1080_50_orig_01_GOP07.yuv | 10,30,50,...,190 | 90 | | 77 | 18ANIMALS_1920x1080_50_orig_01_GOP08.yuv | 10,30,50,...,190 | 91 | | 78 | 18ANIMALS_1920x1080_50_orig_01_GOP09.yuv | 10,30,50,...,190 | 92 | | 79 | 18ANIMALS_1920x1080_50_orig_01_GOP10.yuv | 10,30,50,...,190 | 93 | | 80 | 18ANIMALS_1920x1080_50_orig_01_GOP11.yuv | 10,30,50,...,190 | 94 | | 81 | 18ANIMALS_1920x1080_50_orig_01_GOP12.yuv | 10,30,50,...,190 | 95 | | 82 | 18ANIMALS_1920x1080_50_orig_01_GOP13.yuv | 10,30,50,...,190 | 96 | | 83 | aspen_832x480_.yuv | 10,30,50,...,190 | 97 | | 84 | aspen_1920x1080_.yuv | 10,30,50,...,190 | 98 | | 85 | beach_1920x1080_25.yuv | 10,30,50,...,190 | 99 | | 86 | blue-sky_1920x1080_25.yuv | 10,30,50,...,190 | 100 | | 87 | Cactus-ProRes-420p-GOP01_1920x1080_.yuv | 10,30,50,...,190 | 101 | | 88 | city_1280x720_60.yuv | 10,30,50,...,190 | 102 | | 89 | cont-6-GOP01_1920x1080_.yuv | 10,30,50,...,190 | 103 | | 90 | cont-6-GOP02_1920x1080_.yuv | 10,30,50,...,190 | 104 | | 91 | controlled-burn_832x480_.yuv | 10,30,50,...,190 | 105 | | 92 | controlled-burn_1920x1080_.yuv | 10,30,50,...,190 | 106 | | 93 | crew_1280x720_60.yuv | 10,30,50,...,190 | 107 | | 94 | crowd-run_832x480_.yuv | 10,30,50,...,190 | 108 | | 95 | ducks_1920x1080_.yuv | 10,30,50,...,190 | 109 | | 96 | ducks-take-off_832x480_.yuv | 10,30,50,...,190 | 110 | | 97 | Flowervase_832x480_30.yuv | 10,30,50,...,190 | 111 | | 98 | Foreman-ProRes-420p_1920x1080_.yuv | 10,30,50,...,190 | 112 | | 99 | gipsrestat_1280x720_.yuv | 10,30,50,...,190 | 113 | | 100 | harbour_1280x720_60.yuv | 10,30,50,...,190 | 114 | | 101 | into-tree_832x480_.yuv | 10,30,50,...,190 | 115 | | 102 | LiquidAssets-blackfish-original_1920x1080_420p_1080p.yuv | 10,30,50,...,190 | 116 | | 103 | LiquidAssets-boats-original_1920x1080_420p_1080p.yuv | 10,30,50,...,190 | 117 | | 104 | LiquidAssets-camofish-original_1920x1080_420p_1080p.yuv | 10,30,50,...,190 | 118 | | 105 | LiquidAssets-diver2-original_1920x1080_420p_1080p.yuv | 10,30,50,...,190 | 119 | | 106 | LiquidAssets-diver4-original_1920x1080_420p_1080p.yuv | 10,30,50,...,190 | 120 | | 107 | LiquidAssets-eels-original_1920x1080_420p_1080p.yuv | 10,30,50,...,190 | 121 | | 108 | male150_1920x1080_1080p.yuv | 10,30,50,...,190 | 122 | | 109 | mobcal-ter_1280x720_50.yuv | 10,30,50,...,190 | 123 | | 110 | Netflix-Aerial_1920x1080_.yuv | 10,30,50,...,190 | 124 | | 111 | Netflix-Aerial_2048x1080_60fps_420.yuv | 10,30,50,...,190 | 125 | | 112 | Netflix-BarScene_2048x1080_60fps_420.yuv | 10,30,50,...,190 | 126 | | 113 | Netflix-Boat_1920x1080_.yuv | 10,30,50,...,190 | 127 | | 114 | Netflix-Boat_2048x1080_60fps_420.yuv | 10,30,50,...,190 | 128 | | 115 | Netflix-BoxingPractice_2048x1080_60fps_420.yuv | 10,30,50,...,190 | 129 | | 116 | Netflix-Crosswalk_1920x1080_.yuv | 10,30,50,...,190 | 130 | | 117 | Netflix-Crosswalk_2048x1080_60fps_420.yuv | 10,30,50,...,190 | 131 | | 118 | Netflix-Dancers_2048x1080_60fps_420.yuv | 10,30,50,...,190 | 132 | | 119 | Netflix-DinnerScene_2048x1080_60fps_420.yuv | 10,30,50,...,190 | 133 | | 120 | Netflix-DrivingPOV_1280x720_.yuv | 10,30,50,...,190 | 134 | | 121 | Netflix-DrivingPOV_2048x1080_60fps_420.yuv | 10,30,50,...,190 | 135 | | 122 | Netflix-FoodMarket_1920x1080_.yuv | 10,30,50,...,190 | 136 | | 123 | Netflix-Narrator_2048x1080_60fps_420.yuv | 10,30,50,...,190 | 137 | | 124 | Netflix-PierSeaside_1920x1080_.yuv | 10,30,50,...,190 | 138 | | 125 | Netflix-PierSeaside_2048x1080_60fps_420.yuv | 10,30,50,...,190 | 139 | | 126 | Netflix-RitualDance_2048x1080_60fps_420.yuv | 10,30,50,...,190 | 140 | | 127 | Netflix-RollerCoaster_1280x720_.yuv | 10,30,50,...,190 | 141 | | 128 | Netflix-RollerCoaster_2048x1080_60fps_420.yuv | 10,30,50,...,190 | 142 | | 129 | Netflix-SquareAndTimelapse_1920x1080_.yuv | 10,30,50,...,190 | 143 | | 130 | Netflix-SquareAndTimelapse_2048x1080_60fps_420.yuv | 10,30,50,...,190 | 144 | | 131 | Netflix-Tango_2048x1080_60fps_420.yuv | 10,30,50,...,190 | 145 | | 132 | Netflix-ToddlerFountain_2048x1080_60fps_420.yuv | 10,30,50,...,190 | 146 | | 133 | Netflix-TunnelFlag_1920x1080_.yuv | 10,30,50,...,190 | 147 | | 134 | Netflix-TunnelFlag_2048x1080_60fps_420.yuv | 10,30,50,...,190 | 148 | | 135 | Netflix-WindAndNature_2048x1080_60fps_420.yuv | 10,30,50,...,190 | 149 | | 136 | niklas_640x360_.yuv | 10,30,50,...,190 | 150 | | 137 | old-town-cross_1920x1080_50.yuv | 10,30,50,...,190 | 151 | | 138 | onedarkfinal_1920x1080_1080p.yuv | 10,30,50,...,190 | 152 | | 139 | park-joy_1920x1080_1080p50.yuv | 10,30,50,...,190 | 153 | | 140 | parkrun-ter_1280x720_50.yuv | 10,30,50,...,190 | 154 | | 141 | pedestrian-area_1920x1080_25.yuv | 10,30,50,...,190 | 155 | | 142 | riverbed_1920x1080_25.yuv | 10,30,50,...,190 | 156 | | 143 | rush-field-cuts_832x480_.yuv | 10,30,50,...,190 | 157 | | 144 | rush-field-cuts_1920x1080_.yuv | 10,30,50,...,190 | 158 | | 145 | rush-hour_1920x1080_.yuv | 10,30,50,...,190 | 159 | | 146 | rush-hour_1920x1080_25.yuv | 10,30,50,...,190 | 160 | | 147 | shields_640x360_.yuv | 10,30,50,...,190 | 161 | | 148 | shields-ter_1280x720_50.yuv | 10,30,50,...,190 | 162 | | 149 | simo_1920x1080_.yuv | 10,30,50,...,190 | 163 | | 150 | snow-mnt_832x480_.yuv | 10,30,50,...,190 | 164 | | 151 | snow-mnt_1920x1080_.yuv | 10,30,50,...,190 | 165 | | 152 | speed-bag_640x360_.yuv | 10,30,50,...,190 | 166 | | 153 | speed-bag_832x480_.yuv | 10,30,50,...,190 | 167 | | 154 | stockholm-ter_1280x720_5994.yuv | 10,30,50,...,190 | 168 | | 155 | taishan_1920x1080_25.yuv | 10,30,50,...,190 | 169 | | 156 | tears-of-steel1_832x480_.yuv | 10,30,50,...,190 | 170 | | 157 | tears-of-steel2_832x480_.yuv | 10,30,50,...,190 | 171 | | 158 | touchdown-pass_1920x1080_.yuv | 10,30,50,...,190 | 172 | | 159 | tractor_1920x1080_25.yuv | 10,30,50,...,190 | 173 | | 160 | training_1920x1080_.yuv | 10,30,50,...,190 | 174 | | 161 | videoSRC002_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 175 | | 162 | videoSRC003_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 176 | | 163 | videoSRC004_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 177 | | 164 | videoSRC006_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 178 | | 165 | videoSRC007_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 179 | | 166 | videoSRC008_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 180 | | 167 | videoSRC009_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 181 | | 168 | videoSRC010_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 182 | | 169 | videoSRC011_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 183 | | 170 | videoSRC012_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 184 | | 171 | videoSRC013_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 185 | | 172 | videoSRC014_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 186 | | 173 | videoSRC015_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 187 | | 174 | videoSRC016_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 188 | | 175 | videoSRC017_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 189 | | 176 | videoSRC018_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 190 | | 177 | videoSRC019_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 191 | | 178 | videoSRC020_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 192 | | 179 | videoSRC021_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 193 | | 180 | videoSRC022_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 194 | | 181 | videoSRC023_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 195 | | 182 | videoSRC024_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 196 | | 183 | videoSRC026_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 197 | | 184 | videoSRC027_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 198 | | 185 | videoSRC028_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 199 | | 186 | videoSRC030_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 200 | | 187 | videoSRC031_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 201 | | 188 | videoSRC032_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 202 | | 189 | videoSRC033_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 203 | | 190 | videoSRC034_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 204 | | 191 | videoSRC035_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 205 | | 192 | videoSRC036_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 206 | | 193 | videoSRC037_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 207 | | 194 | videoSRC038_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 208 | | 195 | videoSRC039_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 209 | | 196 | videoSRC040_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 210 | | 197 | videoSRC041_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 211 | | 198 | videoSRC042_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 212 | | 199 | videoSRC043_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 213 | | 200 | videoSRC044_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 214 | | 201 | videoSRC045_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 215 | | 202 | videoSRC046_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 216 | | 203 | videoSRC047_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 217 | | 204 | videoSRC048_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 218 | | 205 | videoSRC049_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 219 | | 206 | videoSRC050_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 220 | | 207 | videoSRC051_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 221 | | 208 | videoSRC052_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 222 | | 209 | videoSRC053_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 223 | | 210 | videoSRC054_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 224 | | 211 | videoSRC055_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 225 | | 212 | videoSRC056_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 226 | | 213 | videoSRC057_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 227 | | 214 | videoSRC058_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 228 | | 215 | videoSRC059_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 229 | | 216 | videoSRC060_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 230 | | 217 | videoSRC061_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 231 | | 218 | videoSRC062_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 232 | | 219 | videoSRC063_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 233 | | 220 | videoSRC064_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 234 | | 221 | videoSRC065_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 235 | | 222 | videoSRC066_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 236 | | 223 | videoSRC067_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 237 | | 224 | videoSRC068_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 238 | | 225 | videoSRC069_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 239 | | 226 | videoSRC070_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 240 | | 227 | videoSRC071_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 241 | | 228 | videoSRC072_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 242 | | 229 | videoSRC073_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 243 | | 230 | videoSRC074_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 244 | | 231 | videoSRC075_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 245 | | 232 | videoSRC076_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 246 | | 233 | videoSRC077_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 247 | | 234 | videoSRC078_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 248 | | 235 | videoSRC079_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 249 | | 236 | videoSRC080_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 250 | | 237 | videoSRC081_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 251 | | 238 | videoSRC082_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 252 | | 239 | videoSRC085_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 253 | | 240 | videoSRC086_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 254 | | 241 | videoSRC087_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 255 | | 242 | videoSRC088_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 256 | | 243 | videoSRC089_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 257 | | 244 | videoSRC090_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 258 | | 245 | videoSRC091_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 259 | | 246 | videoSRC093_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 260 | | 247 | videoSRC094_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 261 | | 248 | videoSRC095_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 262 | | 249 | videoSRC096_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 263 | | 250 | videoSRC097_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 264 | | 251 | videoSRC098_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 265 | | 252 | videoSRC099_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 266 | | 253 | videoSRC100_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 267 | | 254 | videoSRC102_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 268 | | 255 | videoSRC104_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 269 | | 256 | videoSRC105_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 270 | | 257 | videoSRC106_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 271 | | 258 | videoSRC107_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 272 | | 259 | videoSRC109_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 273 | | 260 | videoSRC110_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 274 | | 261 | videoSRC111_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 275 | | 262 | videoSRC112_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 276 | | 263 | videoSRC113_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 277 | | 264 | videoSRC114_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 278 | | 265 | videoSRC115_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 279 | | 266 | videoSRC116_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 280 | | 267 | videoSRC118_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 281 | | 268 | videoSRC119_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 282 | | 269 | videoSRC120_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 283 | | 270 | videoSRC121_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 284 | | 271 | videoSRC122_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 285 | | 272 | videoSRC123_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 286 | | 273 | videoSRC124_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 287 | | 274 | videoSRC125_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 288 | | 275 | videoSRC127_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 289 | | 276 | videoSRC128_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 290 | | 277 | videoSRC130_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 291 | | 278 | videoSRC131_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 292 | | 279 | videoSRC132_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 293 | | 280 | videoSRC134_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 294 | | 281 | videoSRC135_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 295 | | 282 | videoSRC137_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 296 | | 283 | videoSRC138_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 297 | | 284 | videoSRC139_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 298 | | 285 | videoSRC140_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 299 | | 286 | videoSRC141_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 300 | | 287 | videoSRC142_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 301 | | 288 | videoSRC143_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 302 | | 289 | videoSRC144_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 303 | | 290 | videoSRC145_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 304 | | 291 | videoSRC146_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 305 | | 292 | videoSRC147_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 306 | | 293 | videoSRC148_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 307 | | 294 | videoSRC149_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 308 | | 295 | videoSRC150_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 309 | | 296 | videoSRC151_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 310 | | 297 | videoSRC152_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 311 | | 298 | videoSRC153_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 312 | | 299 | videoSRC154_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 313 | | 300 | videoSRC155_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 314 | | 301 | videoSRC156_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 315 | | 302 | videoSRC158_640x360_30_qp_00_420p.yuv | 10,30,50,...,190 | 316 | | 303 | videoSRC159_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 317 | | 304 | videoSRC160_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 318 | | 305 | videoSRC161_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 319 | | 306 | videoSRC162_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 320 | | 307 | videoSRC163_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 321 | | 308 | videoSRC164_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 322 | | 309 | videoSRC165_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 323 | | 310 | videoSRC166_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 324 | | 311 | videoSRC167_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 325 | | 312 | videoSRC168_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 326 | | 313 | videoSRC169_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 327 | | 314 | videoSRC170_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 328 | | 315 | videoSRC171_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 329 | | 316 | videoSRC172_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 330 | | 317 | videoSRC173_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 331 | | 318 | videoSRC174_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 332 | | 319 | videoSRC175_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 333 | | 320 | videoSRC176_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 334 | | 321 | videoSRC177_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 335 | | 322 | videoSRC178_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 336 | | 323 | videoSRC179_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 337 | | 324 | videoSRC180_640x360_24_qp_00_420p.yuv | 10,30,50,...,190 | 338 | | 325 | vidyo1_1280x720_60.yuv | 10,30,50,...,190 | 339 | | 326 | vidyo3_1280x720_60.yuv | 10,30,50,...,190 | 340 | | 327 | vidyo4_1280x720_60.yuv | 10,30,50,...,190 | 341 | | 328 | vtc1nw_720x480_.yuv | 10,30,50,...,190 | 342 | | 329 | washdc_720x480_.yuv | 10,30,50,...,190 | 343 | | 330 | west-wind-easy_832x480_.yuv | 10,30,50,...,190 | 344 | | 331 | west-wind-easy_1920x1080_.yuv | 10,30,50,...,190 | 345 | | 332 | x2_1920x1080_.yuv | 10,30,50,...,190 | 346 | 347 | 348 | ## TestSet: 349 | - 18 sequences in HEVC common test conditions: 350 | 351 | | class | sequence | resolution | frame | 352 | | ----- | :-------------: | ---------: | ----: | 353 | | A | Traffic | 2560x1600 | 0~50 | 354 | | A | PeopleOnStreet | 2560x1600 | 0~50 | 355 | | B | Kimono | 1920x1080 | 0~50 | 356 | | B | ParkScene | 1920x1080 | 0~50 | 357 | | B | Cactus | 1920x1080 | 0~50 | 358 | | B | BQTerrace | 1920x1080 | 0~50 | 359 | | B | BasketballDrive | 1920x1080 | 0~50 | 360 | | C | RaceHorses | 832x480 | 0~50 | 361 | | C | BQMall | 832x480 | 0~50 | 362 | | C | PartyScene | 832x480 | 0~50 | 363 | | C | BasketballDrill | 832x480 | 0~50 | 364 | | D | RaceHorses | 416x240 | 0~50 | 365 | | D | BQSquare | 416x240 | 0~50 | 366 | | D | BlowingBubbles | 416x240 | 0~50 | 367 | | D | BasketballPass | 416x240 | 0~50 | 368 | | E | FourPeople | 1280x720 | 0~50 | 369 | | E | Johnny | 1280x720 | 0~50 | 370 | | E | KristenAndSara | 1280x720 | 0~50 | 371 | 372 | -------------------------------------------------------------------------------- /vrcnn/model/QP22.caffemodel: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/vrcnn/model/QP22.caffemodel -------------------------------------------------------------------------------- /vrcnn/model/QP27.caffemodel: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/vrcnn/model/QP27.caffemodel -------------------------------------------------------------------------------- /vrcnn/model/QP32.caffemodel: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/vrcnn/model/QP32.caffemodel -------------------------------------------------------------------------------- /vrcnn/model/QP37_B.caffemodel: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/vrcnn/model/QP37_B.caffemodel -------------------------------------------------------------------------------- /vrcnn/model/QP37_I.caffemodel: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/vrcnn/model/QP37_I.caffemodel -------------------------------------------------------------------------------- /vrcnn/model/QP37_P.caffemodel: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/IVC-Projects/cnn_In-loop_filter/693931c43c4861f4cf86fc01c9d8a6acfd3bb759/vrcnn/model/QP37_P.caffemodel -------------------------------------------------------------------------------- /vrcnn/vrcnn_test.prototxt: -------------------------------------------------------------------------------- 1 | name: "DRRN_B1U9_20C128" 2 | input: "data" 3 | input_dim: 1 4 | input_dim: 1 5 | input_dim: 200 6 | input_dim: 200 7 | 8 | layer { 9 | name: "conv1" 10 | type: "Convolution" 11 | bottom: "data" 12 | top: "conv1" 13 | param { 14 | lr_mult: 1 15 | } 16 | param { 17 | lr_mult: 0.1 18 | } 19 | convolution_param { 20 | num_output: 64 21 | kernel_size: 5 22 | stride: 1 23 | pad: 2 24 | weight_filler { 25 | type:"msra" 26 | } 27 | bias_filler { 28 | type: "constant" 29 | value: 0 30 | } 31 | } 32 | } 33 | layer { 34 | name: "relu1" 35 | type: "ReLU" 36 | bottom: "conv1" 37 | top: "conv1" 38 | } 39 | layer { 40 | name: "unit1/3x3" 41 | type: "Convolution" 42 | bottom: "conv1" 43 | top: "unit1/3x3" 44 | param { 45 | lr_mult: 1 46 | } 47 | param { 48 | lr_mult: 0.1 49 | } 50 | convolution_param { 51 | num_output: 32 52 | kernel_size: 3 53 | pad: 1 54 | stride: 1 55 | weight_filler { 56 | type:"msra" 57 | } 58 | bias_filler { 59 | type: "constant" 60 | value: 0 61 | } 62 | } 63 | } 64 | layer { 65 | name: "relu_unit1/3x3" 66 | type: "ReLU" 67 | bottom: "unit1/3x3" 68 | top: "unit1/3x3" 69 | } 70 | layer { 71 | name: "unit1/5x5" 72 | type: "Convolution" 73 | bottom:"conv1" 74 | top: "unit1/5x5" 75 | param { 76 | lr_mult: 1 77 | } 78 | param { 79 | lr_mult: 0.1 80 | } 81 | convolution_param { 82 | num_output: 16 83 | kernel_size: 5 84 | pad:2 85 | stride:1 86 | weight_filler { 87 | type:"msra" 88 | } 89 | bias_filler { 90 | type: "constant" 91 | value: 0 92 | } 93 | } 94 | } 95 | layer { 96 | name: "unit1_relu/5x5" 97 | type: "ReLU" 98 | bottom:"unit1/5x5" 99 | top: "unit1/5x5" 100 | } 101 | layer { 102 | name: "unit1/concat" 103 | type: "Concat" 104 | bottom: "unit1/5x5" 105 | bottom: "unit1/3x3" 106 | top: "unit1/concat" 107 | concat_param { 108 | axis: 1 109 | } 110 | } 111 | 112 | #unit1 ends: 48 channels 113 | layer { 114 | name: "unit2/3x3" 115 | type: "Convolution" 116 | bottom: "unit1/concat" 117 | top: "unit2/3x3" 118 | param { 119 | lr_mult: 1 120 | } 121 | param { 122 | lr_mult: 0.1 123 | } 124 | convolution_param { 125 | num_output: 16 126 | kernel_size: 3 127 | pad: 1 128 | stride: 1 129 | weight_filler { 130 | type:"msra" 131 | } 132 | bias_filler { 133 | type: "constant" 134 | value: 0 135 | } 136 | } 137 | } 138 | layer { 139 | name: "relu_unit2/3x3" 140 | type: "ReLU" 141 | bottom: "unit2/3x3" 142 | top: "unit2/3x3" 143 | } 144 | layer { 145 | name: "unit2/1x1" 146 | type: "Convolution" 147 | bottom:"unit1/concat" 148 | top: "unit2/1x1" 149 | param { 150 | lr_mult: 1 151 | } 152 | param { 153 | lr_mult: 0.1 154 | } 155 | convolution_param { 156 | num_output: 32 157 | kernel_size: 1 158 | pad:0 159 | stride:1 160 | weight_filler { 161 | type:"msra" 162 | } 163 | bias_filler { 164 | type: "constant" 165 | value: 0 166 | } 167 | } 168 | } 169 | layer { 170 | name: "unit2_relu/1x1" 171 | type: "ReLU" 172 | bottom:"unit2/1x1" 173 | top: "unit2/1x1" 174 | } 175 | layer { 176 | name: "unit2/concat" 177 | type: "Concat" 178 | bottom: "unit2/1x1" 179 | bottom: "unit2/3x3" 180 | top: "unit2/concat" 181 | } 182 | #unit2 ends: 48 channels 183 | layer { 184 | name: "conv2" 185 | type: "Convolution" 186 | bottom: "unit2/concat" 187 | top: "conv2" 188 | param { 189 | lr_mult: 1 190 | } 191 | param { 192 | lr_mult: 0.1 193 | } 194 | convolution_param { 195 | num_output: 1 196 | pad: 1 197 | kernel_size: 3 198 | weight_filler { 199 | type:"msra" 200 | } 201 | bias_filler { 202 | type: "constant" 203 | value: 0 204 | } 205 | } 206 | } 207 | layer 208 | { 209 | name:"res" 210 | type: "Eltwise" 211 | bottom:"conv2" 212 | bottom:"data" 213 | top:"res" 214 | 215 | } 216 | --------------------------------------------------------------------------------