Trying to deinterlace with Bob NVEnc; job crashes

[shijan]

After some struggle i am ok with current logic. No need deinterlacer - set input to progressive. So if this makes things more stable and easier to code and if it produce less less bugs - i can live with it.

But from user side it is slightly confused and non intuitive to understand it first time. So removed "Use deinterlacer" option needs some decent explanation in release notes and tooltip manual. Otherwise it will be endless amount of same question from forum users.

[antoniu200]

@antoniu200: will send you a link to a dev version to test in a few minutes.
If add 'none' and 'CUVID (FFMPEG)' as deinterlacers.
Note that to use  'CUVID (FFMPEG)' Filtering->Support need to be set to 'no XSynth'.
I tested it with a few clips and it seem to work, but I'm pretty sure there are some bugs created by these two changes.

Take as much time as you need. No rush, by me.
I will also test it and try and break it. I'm no genius at that, but sometimes I do manage to trigger quite obscure bugs.

As an opinion from a less experienced programmer: honestly, I don't think anybody will ever find every bug in their program before they release it. But a good, working release can be done quite simply by not rushing and thinking thoroughly what it is that you're adding or changing.

After some struggle i am ok with current logic. No need deinterlacer - set input to progressive. So if this makes things more stable and easier to code and if it produce less less bugs - i can live with it.

But from user side it is slightly confused and non intuitive to understand it first time. So removed "Use deinterlacer" option needs some decent explanation in release notes and tooltip manual. Otherwise it will be endless amount of same question from forum users.

That was what I thought as well.

[antoniu200]

I just noticed your PM. First bug I see is that it forces you to drop one field.

I'll check more in-depth tomorrow.

[Selur]

I just noticed your PM. First bug I see is that it forces you to drop one field.

That's how the deinterlacer works.
Either:
a. you use 'bob' and one frame per frame is returned
or
b. you use 'adaptive' and then by default, if the frame is marked as interlaced in the input, the first-field get's dropped or by setting '-drop_second_field' you drop the second field. If the field is marked progressive in the input it not deinterlaced.

XXXX_cuvid AVOptions:
  -deint             <int>        .D.V....... Set deinterlacing mode (from 0 to 2) (default weave)
     weave           0            .D.V....... Weave deinterlacing (do nothing)
     bob             1            .D.V....... Bob deinterlacing
     adaptive        2            .D.V....... Adaptive deinterlacing
  -gpu               <string>     .D.V....... GPU to be used for decoding
  -surfaces          <int>        .D.V....... Maximum surfaces to be used for decoding (from 0 to INT_MAX) (default 25)
  -drop_second_field <boolean>    .D.V....... Drop second field when deinterlacing (default false)
  -crop              <string>     .D.V....... Crop (top)x(bottom)x(left)x(right)
  -resize            <string>     .D.V....... Resize (width)x(height)

see: "ffmpeg -h decoder=XXXX_cuvid"
Only thing that Hybrid does not support is 'weave', since it does not seem logical to use 'deint' and then not deinterlace.

Cu Selur

[antoniu200]

I just noticed your PM. First bug I see is that it forces you to drop one field.

That's how the deinterlacer works.
Either:
a. you use 'bob' and one frame per frame is returned
or
b. you use 'adaptive' and then by default, if the frame is marked as interlaced in the input, the first-field get's dropped or by setting '-drop_second_field' you drop the second field. If the field is marked progressive in the input it not deinterlaced.

That's quite not true. The CUVID adaptive is not the same thing as NVEnc adaptive.

NVEncC adaptive works by outputting one frame for 2 fields.
CUVID adaptive works by outputting one frame for 1 field.

Please test this behavior for yourself with any interlaced video. The videos I sent here clearly show how CUVID adaptive works.

Original - interlaced
CUVID Adaptive Deinterlaced

It says here:

adaptive        2            .D.V....... Adaptive deinterlacing

Note how it doesn't say

• normal ... standard 60i → 30p interleave cancellation.
  
• adaptive ... same as normal
  

Like it did here: https://github.com/rigaya/NVEnc/blob/mas...ace-string

---

I managed to break it: I saved the config for an mpeg2 input file, loaded an h264 input file and clicked on "Add to queue and start queue". It started the job using the wrong decoder (mpeg2_cuvid for h264 file).

The suggested fix: don't save the decoder used for the deinterlacer, but just the deinterlacer. Let the program always choose the deinterlacer.

One more thing to keep in mind: for mpeg2_cuvid, it is required that the final expected framerate be specified through FFmpeg, by using the option "-r <output_framerate>", basically overriding the input framerate. In my cases, I would need to add "-r 50", right before mpeg2_cuvid. I would advise using this option for all cuvid decoders, because testing each of them whether or not they require it is very time consuming.

[Selur]

CUVID adaptive works by outputting one frame for 1 field.

Got any documentation about this?
Because normally adaptive deinterlacing means:
Deinterlaces looks at the input frame:
- in case it's marked as progressive, the frame is kept.
- in case it's marked as interlaced, two field will be converted into one
Deinterlacing while creating one progressive frame per field is normally called bob deinterlacing.

Also:

-drop_second_field <boolean>    .D.V....... Drop second field when deinterlacing (default false)

wouldn't make any sense when the adaptive deinterlacer wasn't a same-frame-rate-deinterlacer which creates one progressive frame for two fields.

-> Unless you can show me actual documentation I'm not starting changing things.

One more thing to keep in mind: for mpeg2_cuvid, it is required that the final expected framerate be specified through FFmpeg, by using the option "-r <output_framerate>", basically overriding the input framerate. In my cases, I would need to add "-r 50", right before mpeg2_cuvid. I would advise using this option for all cuvid decoders, because testing each of them whether or not they require it is very time consuming.

Documentation?


Reading https://trac.ffmpeg.org/ticket/6971 sounds to me like current cuvid is buggy for quite some time,...

Cu Selur

[Selur]

I managed to break it: I saved the config for an mpeg2 input file, loaded an h264 input file and clicked on "Add to queue and start queue". It started the job using the wrong decoder (mpeg2_cuvid for h264 file).

Can't reproduce.

I started Hybrid, loaded an MPEG-2 input file, configured the deinterlacer to CUVID. Saved the configuration as defaults. Loaded an H.264 input file and the decoder properly changed to h264_cuvid.

I then restarted Hybrid. Deinterlacer was still 'CUVID (FFmpeg)' with Decoder mpeg2_cuvid, I then loaded an H.264 input file, again the decoder was properly changed to d264_cuvid.

-> Can you give more details in what you did?

---
About adaptive deinterlacing: What is the difference between adaptive and bod if both return one frame per field?

Cu Selur

[Selur]

Okay, I had some time to test, both
"-deint 1" and "-deint 2" output one frame per field.
and '-dropf_second_field 1' could be used to get to same-frame-rate deinterlacing.

-> Question is: Where is the difference between bob and adaptive? Seems like they both do the same thing.

Cu Selur

[antoniu200]

I managed to break it: I saved the config for an mpeg2 input file, loaded an h264 input file and clicked on "Add to queue and start queue". It started the job using the wrong decoder (mpeg2_cuvid for h264 file).

Can't reproduce.

-> Can you give more details in what you did?

Sorry, forgot to mention that after the h264 input was loaded and the defaults were set, I restored the defaults. This is why I think defaults should not retain the decoder, but only the deinterlacer. Hybrid should always decide the decoder based on the input and never the user.

---

About adaptive deinterlacing: What is the difference between adaptive and bod if both return one frame per field?

Sorry I have no documentation on this. The only way I can tell this is how it's supposed to function is by looking at output videos.

The difference between Bob and Adaptive is bob is the casual, usual bobber; whereas Adaptive is a Vector Adaptive deinterlacing method (deduced from the artifacts it creates with some videos used for testing).

Here are the sources I used to determine this: https://www.avsforum.com/threads/hd-1080...r.1157287/

---

Also, not sure CUVID is the broken part here:

ffmpeg -r 50 -c:v mpeg2_cuvid -deint adaptive -i ~/00001.ts -sn -an -qp 22 -c:v h264_nvenc -y out.mkv

Works fine for me on release/3.4, release/4.0 and master.

Note the -r 50 as input option, for the file which is originally 25 fps. You need to manually specify that, as ffmpeg.c is not up to the challenge of changing the fps from inside of a decoder.

ffmpeg.c just does not support a decoder changing the framerate of a video. Making it so it does is a major undertaking.
The only way around it is to use the deinterlacer via API from an application that supports it.

Another useful quote:

After further testing, -r 50 on input works fine if input is a correct encoded file, starting from keyframe etc etc. If input is a live stream (http or udp multicast), even if it starts at keyframe, with -r 50 as input any input error will immediately result in audio/video sync error. -r 50 on input completely breaks the audio/video sync mechanism on input (I guess because it kind of breaks timestamps). So it is not a good solution, it works for some workloads (file transcoding) but not for live transcoding.

[Selur]

About the frame rate adjustment: That would only be needed in case the encoder is also ffmpeg based when piping the output to an encoder, the frame rate ist written by the encoder.

Okay, I probalby will change teh cuvid support to only offer:
a. bob (this will do -deint 2)
a. same-rate (this will do -deint 2 -drop_second_frame)

I will also look at, that in case ffmpeg is the encoder (and no piping takes place) the output frame rate will be specified.

Sorry, forgot to mention that after the h264 input was loaded and the defaults were set, I restored the defaults. This is why I think defaults should not retain the decoder, but only the deinterlacer. Hybrid should always decide the decoder based on the input and never the user.

Okay, loading the defaults after loading the source will always cause problems, since most of the setting collected will be lost,...
-> probably not going to changing that.

Cu Selur

Ps.: I'm still not sure what config you load after loading/restore after loading the source.