首页 > 其他分享 >openH264官方项目wiki的编码解码指导的阅读笔记

openH264官方项目wiki的编码解码指导的阅读笔记

时间：2022-12-28 17:01:25浏览次数：74  
标签：wiki 19 OpenH264 解码 sDstParseInfo crop h264 main openH264

目录

参考资料

  1. 官方项目：https://github.com/cisco/openh264
  2. 官方wiki文档：解码：https://github.com/cisco/openh264/wiki/UsageExampleForDecoder
  3. 编码：https://github.com/cisco/openh264/wiki/UsageExampleForEncoder
  4. 其他人的demo：https://github.com/ibaoger/OpenH264Demo/blob/master/src/OpenH264Demo.cpp OpenH264Demo/OpenH264Demo.cpp at master · ibaoger/OpenH264Demo
  5. 网上的demo，实际上就是官方项目/test/decoder/DecUT_ParseSyntax.cpp：https://cpp.hotexamples.com/examples/-/ISVCDecoder/DecodeParser/cpp-isvcdecoder-decodeparser-method-examples.html
  6. https://blog.csdn.net/bjrxyz/article/details/52918879#comments_20802556 openh264使用指南_太上绝情的博客-CSDN博客_libopenh264

前言

我搜索的时候是有搜到一些demo的，不过很多都用c++封装，或者是在安卓上跑；
自己调试过程中遇到一些细节没理解的坑，在这里记录下

C语言调用

我自己写了一个c语言的，openH264的头文件是支持C语言调用方式，只是看起来有点绕，理解了就好
例如pSvcDecoder->Initialize(&sDecParam);
要写成 (*pSvcDecoder)->Initialize(pSvcDecoder, &sDecParam);

编码

基本上照着官方的指导就可以写出demo，并且能正确编码
要求源文件的yuv是yuv420p，也就是videoFormatI420
官方文档也写了，SEncParamBase是基本的编码参数，用它编译就是编译AVC
改用SEncParamExt和相关接口做初始化或者设置，则编码可以选择SVC，目前就看到空间分层和时间分层的选项
层数最多是4
空间分层的二进制比较复杂，要看H.264协议的SVC扩展章节（附录G）也许还不一定看得懂（？
时间分层似乎只是利用了nal_ref_idc的2bit，所以结构和AVC好像没差别
编码出来的视频ffplay都能放，空间分层的视频分辨率会比原视频小

解码

解码比较坑，照着wiki写，会发现解码不出来
wiki的usage实在是太简单了
而且还有错误

空间分配

sDstParseInfo.pDstBuff = new unsigned char[PARSE_SIZE]; //In Parsing only, allocate enough buffer to save transcoded bitstream for a frame

这个注释也许以前是对的，现在应当是不需要自己分配空间了
说起来参考链接6也坑到我一次，我只看到前面说“需要提前分配好数组空间”，没看到后半句“但是实际的每个指针指向的内存不用分配”

DecodeParser使用送入大小限制

我摸索出来的方法是先用DecodeParser先解码，然后再把分好的NALU往DecodeFrameNoDelay送
官方文档的Parser和Decoder我各创建了一个
但是一直是报错失败

点击查看日志 
main WelsGetDecoderCapability: profile-idc:66 profile-iop:224 level-idc:32 imaxmbps:216000 imaxfs:5120 imaxcpb:20000 imaxdpb:20480 imaxbr:20000 bredundant-pic-cap:0
main ./1080_h264_30_6s.h264.yuv_svc_3layers_spatial.h264 filesize:3709385
main DecodeParser failed ret:18(unknown)
main DecodeFrameNoDelay failed ret:4(error bitstreams(maybe broken internal frame) the decoder cared)

我就把编码那边demo的设置日志等级搬过来用

点击查看日志 
10:44:41 ~/my-projects/openh264 master ?21 > LD_LIBRARY_PATH=./out/x86/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH ./openh264_dec_test ./1080_h264_30_6s.h264.yuv_svc_3layers_spatial.h264 ./1080_h264_30_6s.h264.yuv_svc_3layers_spatial.h264.yuv

main WelsGetDecoderCapability: profile-idc:66 profile-iop:224 level-idc:32 imaxmbps:216000 imaxfs:5120 imaxcpb:20000 imaxdpb:20480 imaxbr:20000 bredundant-pic-cap:0
main ./1080_h264_30_6s.h264.yuv_svc_3layers_spatial.h264 filesize:3709385
[OpenH264] this = 0x0x55612d49a690, Info:WelsRequestMem(): memory alloc size = 1920 * 1088, ref list size = 3
[OpenH264] this = 0x0x55612d49a690, Info:SyncPictureResolutionExt(), overall memory usage: 43757159 bytes
[OpenH264] this = 0x0x55612d49a690, Info:decode failed, failure type:18 

main DecodeParser failed ret:18(unknown)
[OpenH264] this = 0x0x55612d49c270, Warning:Invalid slice type(0) in IDR picture. 
[OpenH264] this = 0x0x55612d49c270, Warning:Invalid slice type(0) in IDR picture. 
[OpenH264] this = 0x0x55612d49c270, Info:WelsRequestMem(): memory alloc size = 480 * 272, ref list size = 3
[OpenH264] this = 0x0x55612d49c270, Info:SyncPictureResolutionExt(), overall memory usage: 13583167 bytes
[OpenH264] this = 0x0x55612d49c270, Info:DecodeFrameConstruction(): will output first frame of new sequence, 480 x 272, crop_left:0, crop_right:0, crop_top:0, crop_bottom:1, ignored error packet:0.
[OpenH264] this = 0x0x55612d49c270, Info:DecodeFrameConstruction():New sequence detected, but freezed, correct MBs (510) out of whole MBs (510).
[OpenH264] this = 0x0x55612d49c270, Warning:slice type too large (143) at first_mb(0)
[OpenH264] this = 0x0x55612d49c270, Warning:slice type too large (159) at first_mb(0)

看不懂，我想着先把SVC解码换成AVC解码

点击查看日志 
10:55:20 ~/my-projects/openh264 master ?21 > LD_LIBRARY_PATH=./out/x86/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH ./openh264_dec_test ./1080_h264_30_6s.h264.yuv.h264  ./1080_h264_30_6s.h264.yuv.h264.yuv                                                                      INT
main WelsGetDecoderCapability: profile-idc:66 profile-iop:224 level-idc:32 imaxmbps:216000 imaxfs:5120 imaxcpb:20000 imaxdpb:20480 imaxbr:20000 bredundant-pic-cap:0
main ./1080_h264_30_6s.h264.yuv.h264 filesize:1427959
[OpenH264] this = 0x0x55cdf6a5e690, Info:WelsRequestMem(): memory alloc size = 1920 * 1088, ref list size = 3
[OpenH264] this = 0x0x55cdf6a5e690, Info:SyncPictureResolutionExt(), overall memory usage: 34083761 bytes
[OpenH264] this = 0x0x55cdf6a5e690, Info:DecodeFrameConstruction(): will output first frame of new sequence, 1920 x 1088, crop_left:0, crop_right:0, crop_top:0, crop_bottom:4, ignored error packet:0.
[OpenH264] this = 0x0x55cdf6a5e690, Info:DecodeFrameConstruction(): current NAL num (131) exceeds permitted num (130). Will expand
main DecodeParser size:0x55cdf71795f0 20 iNalNum:178 pDstBuff=0x7fa8f4c1d020 wxh:1920x1080 in_BsTime:0 out_BsTime:0
[OpenH264] this = 0x0x55cdf6a60270, Info:WelsRequestMem(): memory alloc size = 1920 * 1088, ref list size = 3
[OpenH264] this = 0x0x55cdf6a60270, Info:SyncPictureResolutionExt(), overall memory usage: 30998878 bytes
[OpenH264] this = 0x0x55cdf6a60270, Info:DecodeFrameConstruction(): will output first frame of new sequence, 1920 x 1088, crop_left:0, crop_right:0, crop_top:0, crop_bottom:4, ignored error packet:0.
[OpenH264] this = 0x0x55cdf6a60270, Info:DecodeFrameConstruction(): will output first frame of new sequence, 1920 x 1088, crop_left:0, crop_right:0, crop_top:0, crop_bottom:4, ignored error packet:0.
[OpenH264] this = 0x0x55cdf6a60270, Info:DecodeFrameConstruction(): will output first frame of new sequence, 1920 x 1088, crop_left:0, crop_right:0, crop_top:0, crop_bottom:4, ignored error packet:0.
[OpenH264] this = 0x0x55cdf6a60270, Info:DecodeFrameConstruction(): will output first frame of new sequence, 1920 x 1088, crop_left:0, crop_right:0, crop_top:0, crop_bottom:4, ignored error packet:0.
[OpenH264] this = 0x0x55cdf6a60270, Info:DecodeFrameConstruction(): will output first frame of new sequence, 1920 x 1088, crop_left:0, crop_right:0, crop_top:0, crop_bottom:4, ignored error packet:0.
[OpenH264] this = 0x0x55cdf6a60270, Info:DecodeFrameConstruction(): will output first frame of new sequence, 1920 x 1088, crop_left:0, crop_right:0, crop_top:0, crop_bottom:4, ignored error packet:0.

看上去好像没问题了，结果不知道为什么解码是花屏
我又把视频换成了时间分层的SVC

点击查看日志 
10:57:57 ~/my-projects/openh264 master ?22 > LD_LIBRARY_PATH=./out/x86/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH ./openh264_dec_test ~/Videos/1080_h264_30_6s.h264 ./1080_h264_30_6s.h264_openh264dec.yuv                                                                      12s
main WelsGetDecoderCapability: profile-idc:66 profile-iop:224 level-idc:32 imaxmbps:216000 imaxfs:5120 imaxcpb:20000 imaxdpb:20480 imaxbr:20000 bredundant-pic-cap:0
main /home/xys/Videos/1080_h264_30_6s.h264 filesize:10485760
[OpenH264] this = 0x0x55d330745690, Warning:Max AU size exceeded. Allowed size = 7077888, current size = 10485760
[OpenH264] this = 0x0x55d330745690, Info:ResetDecoder(), context error code is 4
[OpenH264] this = 0x0x55d330745690, Info:CWelsDecoder::init_decoder(), openh264 codec version = openh264 default: 1.4, ParseOnly = 1
[OpenH264] this = 0x0x55d330745690, Info:CWelsDecoder::UninitDecoderCtx(), openh264 codec version = openh264 default: 1.4.
[OpenH264] this = 0x0x55d330745690, Info:CWelsDecoder::UninitDecoder(), verify memory usage (0 bytes) after free..
[OpenH264] this = 0x0x55d330745690, Info:eVideoType: 1
main DecodeParser failed ret:16384(out of memory due to new request)
[OpenH264] this = 0x0x55d330747270, Warning:Max AU size exceeded. Allowed size = 7077888, current size = 10485760
[OpenH264] this = 0x0x55d330747270, Info:ResetDecoder(), context error code is 4
[OpenH264] this = 0x0x55d330747270, Info:CWelsDecoder::init_decoder(), openh264 codec version = openh264 default: 1.4, ParseOnly = 0
[OpenH264] this = 0x0x55d330747270, Info:CWelsDecoder::UninitDecoderCtx(), openh264 codec version = openh264 default: 1.4.
[OpenH264] this = 0x0x55d330747270, Info:CWelsDecoder::UninitDecoder(), verify memory usage (0 bytes) after free..
[OpenH264] this = 0x0x55d330747270, Info:eVideoType: 1
main DecodeFrameNoDelay failed ret:16384(out of memory due to new request)

我终于注意到了Max AU size exceeded. Allowed size = 7077888, current size = 10485760这一行。我原本送入Parser的大小是整个文件（10M），超了……
修改一次送入的大小为512000，确保至少一帧完整帧能解析出来……（要不我下次换成一次送入1M算了）

DecodeParser 的使用处理

接下来还是错误

点击查看日志 
16:44:14 ~/my-projects/openh264 master ?23 > LD_LIBRARY_PATH=./out/x86/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH ./openh264_dec_test ~/Videos/1080_h264_30_6s.h264 ./1080_h264_30_6s.h264_openh264dec.yuv
main WelsGetDecoderCapability: profile-idc:66 profile-iop:224 level-idc:32 imaxmbps:216000 imaxfs:5120 imaxcpb:20000 imaxdpb:20480 imaxbr:20000 bredundant-pic-cap:0
main /home/xys/Videos/1080_h264_30_6s.h264 filesize:10485760
[OpenH264] this = 0x0x55bcfbe48690, Warning: max_num_ref_frames exceeds level limits!
[OpenH264] this = 0x0x55bcfbe48690, Info:WelsRequestMem(): memory alloc size = 1920 * 1088, ref list size = 8
[OpenH264] this = 0x0x55bcfbe48690, Info:SyncPictureResolutionExt(), overall memory usage: 34527850 bytes
[OpenH264] this = 0x0x55bcfbe48690, Info:DecodeFrameConstruction(): will output first frame of new sequence, 1920 x 1088, crop_left:0, crop_right:0, crop_top:0, crop_bottom:4, ignored error packet:0.
main DecodeParser pDstBuff=0x7fc443892020 wxh:1920x1080 in_BsTime:0 out_BsTime:0 iNalNum:28 NalLenArray:[32 9 466 70 69 69 69 69 73 69 69 69 69 74 69 69 69 69 76 69 69 69 69 76 69 70 70 70 ]
[OpenH264] this = 0x0x55bcfbe4a270, Warning: max_num_ref_frames exceeds level limits!
[OpenH264] this = 0x0x55bcfbe4a270, Info:WelsRequestMem(): memory alloc size = 1920 * 1088, ref list size = 8
[OpenH264] this = 0x0x55bcfbe4a270, Info:SyncPictureResolutionExt(), overall memory usage: 51730590 bytes
[OpenH264] this = 0x0x55bcfbe4a270, Info:DecodeFrameConstruction(): will output first frame of new sequence, 1920 x 1088, crop_left:0, crop_right:0, crop_top:0, crop_bottom:4, ignored error packet:0.
[OpenH264] this = 0x0x55bcfbe4a270, Warning:DecodeCurrentAccessUnit() failed (468766) in frame: 6 uiDId: 0 uiQId: 0
[OpenH264] this = 0x0x55bcfbe4a270, Debug:returned error from decoding:[0x7271e]
[OpenH264] this = 0x0x55bcfbe4a270, Info:decode failed, failure type:4 
main DecodeFrameNoDelay failed ret:4(error bitstreams(maybe broken internal frame) the decoder cared)
main DecodeFrameNoDelay sDstBufInfo: Status:0 InBsTime:0 OutYuvTime:0 SysMemBuf:0x0 fmt:unknown stride[0 0] pDst:(nil) (nil) (nil) pData:(nil) (nil) (nil)
[OpenH264] this = 0x0x55bcfbe48690, Info:decode failed, failure type:4 
main DecodeParser failed ret:4(error bitstreams(maybe broken internal frame) the decoder cared)
main DecodeFrameNoDelay failed ret:4(error bitstreams(maybe broken internal frame) the decoder cared)
main DecodeFrameNoDelay sDstBufInfo: Status:0 InBsTime:0 OutYuvTime:0 SysMemBuf:0x0 fmt:unknown stride[0 0] pDst:(nil) (nil) (nil) pData:(nil) (nil) (nil)
[OpenH264] this = 0x0x55bcfbe48690, Warning:DecodeCurrentAccessUnit() failed (468766) in frame: 6 uiDId: 0 uiQId: 0
[OpenH264] this = 0x0x55bcfbe48690, Debug:returned error from decoding:[0x7271e]
[OpenH264] this = 0x0x55bcfbe48690, Warning:referencing pictures lost due frame gaps exist, prev_frame_num: 5, curr_frame_num: 7
[OpenH264] this = 0x0x55bcfbe48690, Debug:returned error from decoding:[0x50433]
main DecodeParser failed ret:22(unknown)
main DecodeFrameNoDelay failed ret:18(unknown)
main DecodeFrameNoDelay sDstBufInfo: Status:0 InBsTime:0 OutYuvTime:0 SysMemBuf:0x0 fmt:unknown stride[0 0] pDst:(nil) (nil) (nil) pData:(nil) (nil) (nil)
main DecodeParser failed ret:4(error bitstreams(maybe broken internal frame) the decoder cared)
main DecodeFrameNoDelay failed ret:4(error bitstreams(maybe broken internal frame) the decoder cared)
main DecodeFrameNoDelay sDstBufInfo: Status:0 InBsTime:0 OutYuvTime:0 SysMemBuf:0x0 fmt:unknown stride[0 0] pDst:(nil) (nil) (nil) pData:(nil) (nil) (nil)
main DecodeParser failed ret:4(error bitstreams(maybe broken internal frame) the decoder cared)

我接下来都在这边的错误打转了一天
换了视频从AVC再换回SVC（因为上面日志里的AVC视频用其他H264查看工具发现有错误信息，于是找了openH264自己编码的空间分层H264视频）

此时的我的代码差不多是

点击查看代码 
    while (iParsePos + iParseStep <= iSize) {
        //for Parsing only; iSize shouldn''t be larger than Allowed size
        // [OpenH264] this = 0x0x55c92af4d690, Warning:Max AU size exceeded. Allowed size = 7077888, current size = 10485760
        printf("%s iSize:%d iParsePos:%d iDecodePos:%d iParseStep:%d\n", __func__, iSize, iParsePos, iDecodePos, iParseStep);

        ret = (*pSvcParser)->DecodeParser(pSvcParser, pBuf + iParsePos, iParseStep, &sDstParseInfo);
        if (ret) {
            printf("%s DecodeParser failed ret:%d(%s)\n", __func__, ret, decoding_state_tostring(ret));
        } else {
            printf("%s DecodeParser pDstBuff=%p wxh:%dx%d in_BsTime:%llu out_BsTime:%llu iNalNum:%d NalLenArray:[",
                __func__,
                sDstParseInfo.pDstBuff,
                sDstParseInfo.iSpsWidthInPixel,
                sDstParseInfo.iSpsHeightInPixel,
                sDstParseInfo.uiInBsTimeStamp,
                sDstParseInfo.uiOutBsTimeStamp,
                sDstParseInfo.iNalNum);
            iParseTmp = 0;
            for (int i = 0; i < sDstParseInfo.iNalNum; i++) {
                printf("%d ", sDstParseInfo.pNalLenInByte[i]);
                iParseTmp += sDstParseInfo.pNalLenInByte[i];
                iParsePos += sDstParseInfo.pNalLenInByte[i];
            }
            fwrite(sDstParseInfo.pDstBuff, 1, iParseTmp, parsed_file);
            printf("]\n");
        }

        //for Parsing only, sDstParseInfo can be used for, e.g., HW decoding
        // if (sDstBufInfo.iNalNum > 0){
        // if (sDstParseInfo.iNalNum > 0){   // this is right?
        //     Hardware decoding sDstParseInfo;
        // }

        for (int i = 0; i < sDstParseInfo.iNalNum; i++) {
            //for Decoding only
            //or iRet = DecodeFrame2(pBuf, iSize, pData, &sDstBufInfo);
            ret = (*pSvcDecoder)->DecodeFrameNoDelay(pSvcDecoder, pBuf + iDecodePos, sDstParseInfo.pNalLenInByte[i], pData, &sDstBufInfo);
            if (ret != 0) {
                printf("%s DecodeFrameNoDelay:%d size:%d failed ret:%d(%s)\n",
                    __func__, i, sDstParseInfo.pNalLenInByte[i], ret, decoding_state_tostring(ret));
            } else {
                //for Decoding only, pData can be used for render.
                printf("%s DecodeFrameNoDelay:%d size:%d sDstBufInfo: Status:%d InBsTime:%llu OutYuvTime:%llu"
                    " SysMemBuf:%dx%d fmt:%s stride[%d %d] pDst:%p %p %p pData:%p %p %p\n",
                    __func__,
                    i,
                    sDstParseInfo.pNalLenInByte[i],
                    sDstBufInfo.iBufferStatus,             ///< 0: one frame data is not ready; 1: one frame data is ready
                    sDstBufInfo.uiInBsTimeStamp,     ///< input BS timestamp
                    sDstBufInfo.uiOutYuvTimeStamp,     ///< output YUV timestamp, when bufferstatus is 1
                    sDstBufInfo.UsrData.sSystemBuffer.iWidth,
                    sDstBufInfo.UsrData.sSystemBuffer.iHeight,
                    video_format_type_tostring(sDstBufInfo.UsrData.sSystemBuffer.iFormat),
                    sDstBufInfo.UsrData.sSystemBuffer.iStride[0],
                    sDstBufInfo.UsrData.sSystemBuffer.iStride[1],
                    sDstBufInfo.pDst[0],
                    sDstBufInfo.pDst[1],
                    sDstBufInfo.pDst[2],
                    pData[0],
                    pData[1],
                    pData[2]);
                if (sDstBufInfo.iBufferStatus == 1) {
                    fwrite(pData[0], 1, 1920 * 1080, out_file);
                    fwrite(pData[1], 1, 1920 * 1080 / 2, out_file);
                    fwrite(pData[2], 1, 1920 * 1080 / 2, out_file);
                }
                iDecodePos += sDstParseInfo.pNalLenInByte[i]; // need the right position
            }
        }

        //no-delay decoding can be realized by directly calling DecodeFrameNoDelay(), which is the recommended usage.
        //no-delay decoding can also be realized by directly calling DecodeFrame2() again with NULL input, as in the following.
        //In this case, decoder would immediately reconstruct the input data. This can also be used similarly for Parsing only.
        //Consequent decoding error and output indication should also be considered as above.
        //  iRet = DecodeFrame2(NULL, 0, pData, &sDstBufInfo);
        //  judge iRet, sDstBufInfo.iBufferStatus ...

        usleep(33000);
    }

我一直看不懂日志里错误信息，错误返回486766是什么玩意，换成16进制也不懂，因为代码里没这个。。
直到把前一天加的DecodeFrameNoDelay打印结果拿出来又看一遍，多分析了一下

点击查看日志 
20:43:26 ~/my-projects/openh264 #v2.3.0 ?23 > LD_LIBRARY_PATH=./out/x86/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH ./openh264_dec_test ./1080_h264_30_6s.h264.yuv_svc_3layer_temporal.h264 ./1080_h264_30_6s.h264.yuv_svc_3layer_temporal.h264.yuv
main WelsGetDecoderCapability: profile-idc:66 profile-iop:224 level-idc:32 imaxmbps:216000 imaxfs:5120 imaxcpb:20000 imaxdpb:20480 imaxbr:20000 bredundant-pic-cap:0
main ./1080_h264_30_6s.h264.yuv_svc_3layer_temporal.h264 filesize:4296071
main iSize:4296071 iParsePos:0 iParseStep:512000
[OpenH264] this = 0x0x556f041d1690, Info:WelsRequestMem(): memory alloc size = 1920 * 1088, ref list size = 4
[OpenH264] this = 0x0x556f041d1690, Info:SyncPictureResolutionExt(), overall memory usage: 29139414 bytes
[OpenH264] this = 0x0x556f041d1690, Info:DecodeFrameConstruction(): will output first frame of new sequence, 1920 x 1088, crop_left:0, crop_right:0, crop_top:0, crop_bottom:4, ignored error packet:0.
main DecodeParser pDstBuff=0x7fb5371b2020 wxh:1920x1080 in_BsTime:0 out_BsTime:0 iNalNum:50 NalLenArray:[20 8 6164 20 26 25 39 199 205 19 220 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 53 23002 216 29301 28378 40189 32428 58207 4447 54330 26824 35538 18409 47093 2705 41290 28752 ]
main DecodeFrameNoDelay:0 size:20 sDstBufInfo: Status:0 InBsTime:0 OutYuvTime:0 SysMemBuf:0x0 fmt:unknown stride[0 0] pDst:(nil) (nil) (nil) pData:(nil) (nil) (nil)
main DecodeFrameNoDelay:1 size:8 sDstBufInfo: Status:0 InBsTime:0 OutYuvTime:0 SysMemBuf:0x0 fmt:unknown stride[0 0] pDst:(nil) (nil) (nil) pData:(nil) (nil) (nil)
[OpenH264] this = 0x0x556f041d3270, Info:WelsRequestMem(): memory alloc size = 1920 * 1088, ref list size = 4
[OpenH264] this = 0x0x556f041d3270, Info:SyncPictureResolutionExt(), overall memory usage: 32628638 bytes
[OpenH264] this = 0x0x556f041d3270, Info:DecodeFrameConstruction(): will output first frame of new sequence, 1920 x 1088, crop_left:0, crop_right:0, crop_top:0, crop_bottom:4, ignored error packet:0.
main DecodeFrameNoDelay:2 size:6164 sDstBufInfo: Status:1 InBsTime:0 OutYuvTime:0 SysMemBuf:1920x1080 fmt:I420 stride[1984 992] pDst:0x7fb535aa8840 0x7fb535ccae30 0x7fb535d56630 pData:0x7fb535aa8840 0x7fb535ccae30 0x7fb535d56630
main DecodeFrameNoDelay:3 size:20 sDstBufInfo: Status:1 InBsTime:0 OutYuvTime:0 SysMemBuf:1920x1080 fmt:I420 stride[1984 992] pDst:0x7fb535662840 0x7fb535884e30 0x7fb535910630 pData:0x7fb535662840 0x7fb535884e30 0x7fb535910630
main DecodeFrameNoDelay:4 size:26 sDstBufInfo: Status:1 InBsTime:0 OutYuvTime:0 SysMemBuf:1920x1080 fmt:I420 stride[1984 992] pDst:0x7fb53521c840 0x7fb53543ee30 0x7fb5354ca630 pData:0x7fb53521c840 0x7fb53543ee30 0x7fb5354ca630
main DecodeFrameNoDelay:5 size:25 sDstBufInfo: Status:1 InBsTime:0 OutYuvTime:0 SysMemBuf:1920x1080 fmt:I420 stride[1984 992] pDst:0x7fb535eee840 0x7fb536110e30 0x7fb53619c630 pData:0x7fb535eee840 0x7fb536110e30 0x7fb53619c630
main DecodeFrameNoDelay:6 size:39 sDstBufInfo: Status:1 InBsTime:0 OutYuvTime:0 SysMemBuf:1920x1080 fmt:I420 stride[1984 992] pDst:0x7fb535662840 0x7fb535884e30 0x7fb535910630 pData:0x7fb535662840 0x7fb535884e30 0x7fb535910630
main DecodeFrameNoDelay:7 size:199 sDstBufInfo: Status:1 InBsTime:0 OutYuvTime:0 SysMemBuf:1920x1080 fmt:I420 stride[1984 992] pDst:0x7fb535eee840 0x7fb536110e30 0x7fb53619c630 pData:0x7fb535eee840 0x7fb536110e30 0x7fb53619c630

日志不全，实际上是解码出了49帧的，和Parser个数一样；

点击查看日志 
main iSize:4296071 iParsePos:478525 iParseStep:512000
[OpenH264] this = 0x0x556f041d1690, Warning:DecodeCurrentAccessUnit() failed (459800) in frame: 24 uiDId: 0 uiQId: 0
[OpenH264] this = 0x0x556f041d1690, Debug:returned error from decoding:[0x70418]
[OpenH264] this = 0x0x556f041d1690, Debug:reference picture introduced by this frame is lost during transmission! uiTId: 0
[OpenH264] this = 0x0x556f041d1690, Debug:returned error from decoding:[0x433]
[OpenH264] this = 0x0x556f041d1690, Debug:reference picture introduced by this frame is lost during transmission! uiTId: 0
……
[OpenH264] this = 0x0x556f041d1690, Debug:returned error from decoding:[0x433]
[OpenH264] this = 0x0x556f041d1690, Info:decode failed, failure type:6 

main DecodeParser failed ret:6(unknown)
main iSize:4296071 iParsePos:478525 iParseStep:512000
……
[OpenH264] this = 0x0x556f041d1690, Debug:returned error from decoding:[0x433]
main DecodeParser failed ret:2(layer lost at reference frame with temporal id 0)
main iSize:4296071 iParsePos:478525 iParseStep:512000
[OpenH264] this = 0x0x556f041d1690, Debug:reference picture introduced by this frame is lost during transmission! uiTId: 0

除了调整我解码和解析的一些逻辑bug外，我还试过其他情况：Parse的步长是固定512000，就是说虽然第一次parse完才到478525位置，但是我第二次Parse保持在512000开始，解析下一个500KB（代码里就是修改iParsePos增加的地方），日志是这样的

点击查看日志 
16:34:29 ~/my-projects/openh264 #v2.3.0 ?25 > LD_LIBRARY_PATH=./out/x86/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH ./openh264_dec_test ./1080_h264_30_6s.h264.yuv_svc_3layer_temporal.h264 ./1080_h264_30_6s.h264.yuv_svc_3layer_temporal.h264.yuv
main WelsGetDecoderCapability: profile-idc:66 profile-iop:224 level-idc:32 imaxmbps:216000 imaxfs:5120 imaxcpb:20000 imaxdpb:20480 imaxbr:20000 bredundant-pic-cap:0
main ./1080_h264_30_6s.h264.yuv_svc_3layer_temporal.h264 filesize:4296071
main iSize:4296071 iParsePos:0 iDecodePos:0 iParseStep:512000
[OpenH264] this = 0x0x5639c5fc3870, Info:WelsRequestMem(): memory alloc size = 1920 * 1088, ref list size = 4
[OpenH264] this = 0x0x5639c5fc3870, Info:SyncPictureResolutionExt(), overall memory usage: 29139414 bytes
[OpenH264] this = 0x0x5639c5fc3870, Info:DecodeFrameConstruction(): will output first frame of new sequence, 1920 x 1088, crop_left:0, crop_right:0, crop_top:0, crop_bottom:4, ignored error packet:0.
main DecodeParser pDstBuff=0x7f71579ca020 wxh:1920x1080 in_BsTime:0 out_BsTime:0 iNalNum:50 NalLenArray:[20 8 6164 20 26 25 39 199 205 19 220 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 53 23002 216 29301 28378 40189 32428 58207 4447 54330 26824 35538 18409 47093 2705 41290 28752 ]
main iSize:4296071 iParsePos:512000 iDecodePos:0 iParseStep:512000
[OpenH264] this = 0x0x5639c5fc3870, Warning:DecodeCurrentAccessUnit() failed (459800) in frame: 24 uiDId: 0 uiQId: 0
[OpenH264] this = 0x0x5639c5fc3870, Debug:returned error from decoding:[0x70418]
[OpenH264] this = 0x0x5639c5fc3870, Debug:reference picture introduced by this frame is lost during transmission! uiTId: 0
[OpenH264] this = 0x0x5639c5fc3870, Debug:returned error from decoding:[0x433]
[OpenH264] this = 0x0x5639c5fc3870, Debug:reference picture introduced by this frame is lost during transmission! uiTId: 0
[OpenH264] this = 0x0x5639c5fc3870, Debug:returned error from decoding:[0x433]

而我想如果第一次Parse完，openH264不帮我缓存前一次没处理完的东西，第二次我就要从上一次Parse成功的地方继续（iParsePos代码改成增加NAL长度），日志好像没什么变化

点击查看日志 
16:35:25 ~/my-projects/openh264 #v2.3.0 ?25 > LD_LIBRARY_PATH=./out/x86/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH ./openh264_dec_test ./1080_h264_30_6s.h264.yuv_svc_3layer_temporal.h264 ./1080_h264_30_6s.h264.yuv_svc_3layer_temporal.h264.yuv
main WelsGetDecoderCapability: profile-idc:66 profile-iop:224 level-idc:32 imaxmbps:216000 imaxfs:5120 imaxcpb:20000 imaxdpb:20480 imaxbr:20000 bredundant-pic-cap:0
main ./1080_h264_30_6s.h264.yuv_svc_3layer_temporal.h264 filesize:4296071
main iSize:4296071 iParsePos:0 iDecodePos:0 iParseStep:512000
[OpenH264] this = 0x0x5603bb23f870, Info:WelsRequestMem(): memory alloc size = 1920 * 1088, ref list size = 4
[OpenH264] this = 0x0x5603bb23f870, Info:SyncPictureResolutionExt(), overall memory usage: 29139414 bytes
[OpenH264] this = 0x0x5603bb23f870, Info:DecodeFrameConstruction(): will output first frame of new sequence, 1920 x 1088, crop_left:0, crop_right:0, crop_top:0, crop_bottom:4, ignored error packet:0.
main DecodeParser pDstBuff=0x7fb44edca020 wxh:1920x1080 in_BsTime:0 out_BsTime:0 iNalNum:50 NalLenArray:[20 8 6164 20 26 25 39 199 205 19 220 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 53 23002 216 29301 28378 40189 32428 58207 4447 54330 26824 35538 18409 47093 2705 41290 28752 ]
main iSize:4296071 iParsePos:478525 iDecodePos:0 iParseStep:512000
[OpenH264] this = 0x0x5603bb23f870, Warning:DecodeCurrentAccessUnit() failed (459800) in frame: 24 uiDId: 0 uiQId: 0
[OpenH264] this = 0x0x5603bb23f870, Debug:returned error from decoding:[0x70418]
[OpenH264] this = 0x0x5603bb23f870, Debug:reference picture introduced by this frame is lost during transmission! uiTId: 0
[OpenH264] this = 0x0x5603bb23f870, Debug:returned error from decoding:[0x433]
[OpenH264] this = 0x0x5603bb23f870, Debug:reference picture introduced by this frame is lost during transmission! uiTId: 0
[OpenH264] this = 0x0x5603bb23f870, Debug:returned error from decoding:[0x433]
[OpenH264] this = 0x0x5603bb23f870, Debug:reference picture introduced by this frame is lost during transmission! uiTId: 0

而后面的Parse结果都是错误……
我还想过把Parse的结果拿出来看，挺正常的
我卡了一天才开始考虑，会不会这个问题和DecodeFrameNoDelay无关，我先把解码的代码去掉去看
确实是如此；但是还是找不到原因

我甚至还去看FlushFrame有什么用……C函数那边没注释，C++有，好像是如果有完全不一样的视频流送过来，要先用FlushFrame清空所有原来缓存的视频信息

找不到原因后我想着，干脆直接找DecodeParser的demo怎么用，于是搜到了相关链接5
……好家伙这不就是openH264/test/里面自己的代码吗
赶紧看源码，除了找到的链接外，其他test目录下调用源码的地方都是类似的写法：

    iRet |= m_pDec->DecodeParser (pBuf + iBufPos, iSliceSize, &m_sParserBsInfo);
    iRet |= m_pDec->DecodeParser (NULL, 0, &m_sParserBsInfo);
    if (eDecCase == CorrectParseOnly) {
      EXPECT_TRUE (iRet == dsErrorFree || iRet == dsFramePending);
    }

不懂为什么DecodeParser后面都要送一个空指针进去……姑且试一下；while循环内代码改成这样

点击查看代码 
        ret = (*pSvcParser)->DecodeParser(pSvcParser, pBuf + iParsePos, iParseStep, &sDstParseInfo);
        if (ret) {
            printf("%s DecodeParser failed ret:%d(%s)\n", __func__, ret, decoding_state_tostring(ret));
        } else {
            printf("%s DecodeParser pDstBuff=%p wxh:%dx%d in_BsTime:%llu out_BsTime:%llu iNalNum:%d NalLenArray:[",
                __func__,
                sDstParseInfo.pDstBuff,
                sDstParseInfo.iSpsWidthInPixel,
                sDstParseInfo.iSpsHeightInPixel,
                sDstParseInfo.uiInBsTimeStamp,
                sDstParseInfo.uiOutBsTimeStamp,
                sDstParseInfo.iNalNum);
            iParseTmp = 0;
            for (int i = 0; i < sDstParseInfo.iNalNum; i++) {
                printf("%d ", sDstParseInfo.pNalLenInByte[i]);
                iParseTmp += sDstParseInfo.pNalLenInByte[i];
                // iParsePos += sDstParseInfo.pNalLenInByte[i];
            }
            fwrite(sDstParseInfo.pDstBuff, 1, iParseTmp, parsed_file);
            printf("]\n");
        }
        iParsePos += iParseStep;

        ret = (*pSvcParser)->DecodeParser(pSvcParser, NULL, 0, &sDstParseInfo);
        if (ret) {
            printf("%s DecodeParser twice failed ret:%d(%s)\n", __func__, ret, decoding_state_tostring(ret));
        } else {
            printf("%s DecodeParser twice pDstBuff=%p wxh:%dx%d in_BsTime:%llu out_BsTime:%llu iNalNum:%d NalLenArray:[",
                __func__,
                sDstParseInfo.pDstBuff,
                sDstParseInfo.iSpsWidthInPixel,
                sDstParseInfo.iSpsHeightInPixel,
                sDstParseInfo.uiInBsTimeStamp,
                sDstParseInfo.uiOutBsTimeStamp,
                sDstParseInfo.iNalNum);
            iParseTmp = 0;
            for (int i = 0; i < sDstParseInfo.iNalNum; i++) {
                printf("%d ", sDstParseInfo.pNalLenInByte[i]);
                iParseTmp += sDstParseInfo.pNalLenInByte[i];
            }
            // fwrite(sDstParseInfo.pDstBuff, 1, iParseTmp, parsed_file);
            printf("]\n");
        }

日志没啥变化

点击查看日志 
16:38:32 ~/my-projects/openh264 #v2.3.0 ?25 > gcc -o openh264_dec_test ./openh264_dec_test.c -I./out/x86/include -L./out/x86/lib/x86_64-linux-gnu -lopenh264                                                                                                               INT
16:40:47 ~/my-projects/openh264 #v2.3.0 ?25 > LD_LIBRARY_PATH=./out/x86/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH ./openh264_dec_test ./1080_h264_30_6s.h264.yuv_svc_3layer_temporal.h264 ./1080_h264_30_6s.h264.yuv_svc_3layer_temporal.h264.yuv
main WelsGetDecoderCapability: profile-idc:66 profile-iop:224 level-idc:32 imaxmbps:216000 imaxfs:5120 imaxcpb:20000 imaxdpb:20480 imaxbr:20000 bredundant-pic-cap:0
main ./1080_h264_30_6s.h264.yuv_svc_3layer_temporal.h264 filesize:4296071
main iSize:4296071 iParsePos:0 iDecodePos:0 iParseStep:512000
[OpenH264] this = 0x0x5582ccbab870, Info:WelsRequestMem(): memory alloc size = 1920 * 1088, ref list size = 4
[OpenH264] this = 0x0x5582ccbab870, Info:SyncPictureResolutionExt(), overall memory usage: 29139414 bytes
[OpenH264] this = 0x0x5582ccbab870, Info:DecodeFrameConstruction(): will output first frame of new sequence, 1920 x 1088, crop_left:0, crop_right:0, crop_top:0, crop_bottom:4, ignored error packet:0.
main DecodeParser pDstBuff=0x7f76b0b38020 wxh:1920x1080 in_BsTime:0 out_BsTime:0 iNalNum:50 NalLenArray:[20 8 6164 20 26 25 39 199 205 19 220 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 53 23002 216 29301 28378 40189 32428 58207 4447 54330 26824 35538 18409 47093 2705 41290 28752 ]
[OpenH264] this = 0x0x5582ccbab870, Warning:DecodeCurrentAccessUnit() failed (459800) in frame: 24 uiDId: 0 uiQId: 0
[OpenH264] this = 0x0x5582ccbab870, Debug:returned error from decoding:[0x70418]
[OpenH264] this = 0x0x5582ccbab870, Info:decode failed, failure type:4 

main DecodeParser twice failed ret:4(error bitstreams(maybe broken internal frame) the decoder cared)
main iSize:4296071 iParsePos:990525 iDecodePos:0 iParseStep:512000
[OpenH264] this = 0x0x5582ccbab870, Debug:reference picture introduced by this frame is lost during transmission! uiTId: 0
[OpenH264] this = 0x0x5582ccbab870, Debug:returned error from decoding:[0x433]
[OpenH264] this = 0x0x5582ccbab870, Debug:reference picture introduced by this frame is lost during transmission! uiTId: 0

这个是Parse固定500KB整数倍的，然后我又套回刚才那个第二次Parse从合适位置Parse
终于解决了

点击查看代码 
        ret = (*pSvcParser)->DecodeParser(pSvcParser, pBuf + iParsePos, iParseStep, &sDstParseInfo);
        if (ret) {
            printf("%s DecodeParser failed ret:%d(%s)\n", __func__, ret, decoding_state_tostring(ret));
        } else {
            printf("%s DecodeParser pDstBuff=%p wxh:%dx%d in_BsTime:%llu out_BsTime:%llu iNalNum:%d NalLenArray:[",
                __func__,
                sDstParseInfo.pDstBuff,
                sDstParseInfo.iSpsWidthInPixel,
                sDstParseInfo.iSpsHeightInPixel,
                sDstParseInfo.uiInBsTimeStamp,
                sDstParseInfo.uiOutBsTimeStamp,
                sDstParseInfo.iNalNum);
            iParseTmp = 0;
            for (int i = 0; i < sDstParseInfo.iNalNum; i++) {
                printf("%d ", sDstParseInfo.pNalLenInByte[i]);
                iParseTmp += sDstParseInfo.pNalLenInByte[i];
                iParsePos += sDstParseInfo.pNalLenInByte[i];
            }
            fwrite(sDstParseInfo.pDstBuff, 1, iParseTmp, parsed_file);
            printf("]\n");
        }
        // iParsePos += iParseStep;

        ret = (*pSvcParser)->DecodeParser(pSvcParser, NULL, 0, &sDstParseInfo);
        if (ret) {
            printf("%s DecodeParser twice failed ret:%d(%s)\n", __func__, ret, decoding_state_tostring(ret));
        } else {
            printf("%s DecodeParser twice pDstBuff=%p wxh:%dx%d in_BsTime:%llu out_BsTime:%llu iNalNum:%d NalLenArray:[",

日志：

点击查看日志 
16:43:32 ~/my-projects/openh264 #v2.3.0 ?25 > LD_LIBRARY_PATH=./out/x86/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH ./openh264_dec_test ./1080_h264_30_6s.h264.yuv_svc_3layer_temporal.h264 ./1080_h264_30_6s.h264.yuv_svc_3layer_temporal.h264.yuv
main WelsGetDecoderCapability: profile-idc:66 profile-iop:224 level-idc:32 imaxmbps:216000 imaxfs:5120 imaxcpb:20000 imaxdpb:20480 imaxbr:20000 bredundant-pic-cap:0
main ./1080_h264_30_6s.h264.yuv_svc_3layer_temporal.h264 filesize:4296071
main iSize:4296071 iParsePos:0 iDecodePos:0 iParseStep:512000
[OpenH264] this = 0x0x55b0fe164870, Info:WelsRequestMem(): memory alloc size = 1920 * 1088, ref list size = 4
[OpenH264] this = 0x0x55b0fe164870, Info:SyncPictureResolutionExt(), overall memory usage: 29139414 bytes
[OpenH264] this = 0x0x55b0fe164870, Info:DecodeFrameConstruction(): will output first frame of new sequence, 1920 x 1088, crop_left:0, crop_right:0, crop_top:0, crop_bottom:4, ignored error packet:0.
main DecodeParser pDstBuff=0x7f9eb33a2020 wxh:1920x1080 in_BsTime:0 out_BsTime:0 iNalNum:50 NalLenArray:[20 8 6164 20 26 25 39 199 205 19 220 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 53 23002 216 29301 28378 40189 32428 58207 4447 54330 26824 35538 18409 47093 2705 41290 28752 ]
[OpenH264] this = 0x0x55b0fe164870, Warning:DecodeCurrentAccessUnit() failed (459800) in frame: 24 uiDId: 0 uiQId: 0
[OpenH264] this = 0x0x55b0fe164870, Debug:returned error from decoding:[0x70418]
[OpenH264] this = 0x0x55b0fe164870, Info:decode failed, failure type:4 

main DecodeParser twice failed ret:4(error bitstreams(maybe broken internal frame) the decoder cared)
main iSize:4296071 iParsePos:478525 iDecodePos:0 iParseStep:512000
main DecodeParser pDstBuff=0x7f9eb33a2020 wxh:1920x1080 in_BsTime:0 out_BsTime:0 iNalNum:16 NalLenArray:[46449 19878 37386 3968 55623 24878 34701 22278 50854 3049 41082 29826 51157 20420 38868 5260 ]
[OpenH264] this = 0x0x55b0fe164870, Warning:DecodeCurrentAccessUnit() failed (459800) in frame: 32 uiDId: 0 uiQId: 0
[OpenH264] this = 0x0x55b0fe164870, Debug:returned error from decoding:[0x70418]
main DecodeParser twice failed ret:4(error bitstreams(maybe broken internal frame) the decoder cared)
main iSize:4296071 iParsePos:964202 iDecodePos:0 iParseStep:512000
main DecodeParser pDstBuff=0x7f9eb33a2020 wxh:1920x1080 in_BsTime:0 out_BsTime:0 iNalNum:16 NalLenArray:[53122 27778 35640 17956 45053 5564 43836 26730 47300 18898 37412 5537 55790 29589 34734 18594 ]
[OpenH264] this = 0x0x55b0fe164870, Warning:DecodeCurrentAccessUnit() failed (12) in frame: 40 uiDId: 0 uiQId: 0
[OpenH264] this = 0x0x55b0fe164870, Debug:returned error from decoding:[0xc]

我终于看到了第二次Parse的结果了……
现在来看为啥要NULL传一次？它也没说吧……不好像还真说了

//for Decoding only
 iRet = DecodeFrameNoDelay(pBuf, iSize, pData, &sDstBufInfo);
 //or
 iRet = DecodeFrame2(pBuf, iSize, pData, &sDstBufInfo);
 //for Parsing only
 iRet = DecodeParser(pBuf, iSize, &sDstParseInfo);
 //decode failed
 If (iRet != 0){
     RequestIDR or something like that.
 }
 //for Decoding only, pData can be used for render.
 if (sDstBufInfo.iBufferStatus==1){
     output pData[0], pData[1], pData[2];
 }
//for Parsing only, sDstParseInfo can be used for, e.g., HW decoding
 if (sDstBufInfo.iNalNum > 0){
     Hardware decoding sDstParseInfo;
 }
 //no-delay decoding can be realized by directly calling DecodeFrameNoDelay(), which is the recommended usage.
 //no-delay decoding can also be realized by directly calling DecodeFrame2() again with NULL input, as in the following. In this case, decoder would immediately reconstruct the input data. This can also be used similarly for Parsing only. Consequent decoding error and output indication should also be considered as above.
 iRet = DecodeFrame2(NULL, 0, pData, &sDstBufInfo);
 judge iRet, sDstBufInfo.iBufferStatus ...

……在DecodeFrame2里有这个用法，说是立刻重建数据
然后在长得不行的注释里提到了DecodeParser也可以这么用（cnblogs这里刚好还原了和github一样的情况，没有wrap自动换行）
还有就是DecodeFrameNoDelay可以直接接DecodeFrameNoDelay调用。。。所以我解码那边没出问题算误打误撞
我去
这算我没读懂用例么……唉

写下笔记，这笔记也是乱乱的，就这样吧

标签：wiki,19,OpenH264,解码,sDstParseInfo,crop,h264,main,openH264
From： https://www.cnblogs.com/SendBoringBackToNoWhere/p/17010529.html

相关文章

赞助商

阅读排行

网站主页关于我们联系我们网站地图

本网站内容转载自其他媒体，侵权联系[admin##ips99.com]。

Copyright © 2020-2023 IPS99 版权所有 IPS99