My encode result is a green messed up video

[Eamon]

Hi Cu Selur,

The last time I posted here was on this problem. Firstly, I want to thank you for all the help you provided last year.

This time, I have a similar scenario but a very different problem. I am once again using an AVS script. It loads into Hybrid with no problem. But the end result is this:


The original video looks like this:


I've tried the video on Media Player Classic and VLC Player and it shows the same green result. So I don't believe this to be a playback issue.

This is the script I used:

LoadPlugin("C:\Users\User\Desktop\encode\ffms2.dll")
LoadPlugin("C:\Users\User\Desktop\encode\VSFilter.dll")
LoadPlugin("C:\Users\User\Desktop\encode\VSFilterMod.dll")

FFVideoSource("C:\Users\User\Desktop\encode\VID.mkv")

TextSub("C:\Users\User\Desktop\encode\Hardcoding.ass")
TextSubMod("C:\Users\User\Desktop\encode\Karaoke.ass")

return last

There are two filters because the two different .ass files use different libraries for special effects. The green problem has nothing to do with the filters as I've taken both out of the script and tried it again resulting in the same green messed up video. I even changed ffms2.dll with LSMASHSource.dll and replaced 'FFVideoSource' in the script with 'LWLibavVideoSource' and the exact same problem occurs.

However, one thing I have found, is when I changed the source video to something else, the problem did not occur. This makes me wonder if Hybrid has trouble with the properties of the video? Here's what mediainfo says of VID.mkv:

General
Unique ID                      : 337104770900286713744679338367114178536 (0xFD9C046BA0A738F5180E372ACA626FE8)
Complete name                  : C:\Users\User\Desktop\encode\VID.mkv
Format                         : Matroska
Format version                 : Version 4
File size                      : 298 MiB
Duration                       : 23 min 35 s
Overall bit rate               : 1 767 kb/s
Movie name                     : VID
Encoded date                   : UTC 2020-04-03 00:44:32
Writing application            : mkvmerge v35.0.0 ('All The Love In The World') 64-bit
Writing library                : libebml v1.3.9 + libmatroska v1.5.2

Video
ID                             : 1
Format                         : AVC
Format/Info                    : Advanced Video Codec
Format profile                 : High 10@L5
Format settings                : CABAC / 15 Ref Frames
Format settings, CABAC         : Yes
Format settings, Reference fra : 15 frames
Codec ID                       : V_MPEG4/ISO/AVC
Duration                       : 23 min 35 s
Bit rate                       : 1 766 kb/s
Width                          : 1 280 pixels
Height                         : 720 pixels
Display aspect ratio           : 16:9
Original display aspect ratio  : 16:9
Frame rate mode                : Constant
Frame rate                     : 23.976 (24000/1001) FPS
Color space                    : YUV
Chroma subsampling             : 4:2:0
Bit depth                      : 10 bits
Scan type                      : Progressive
Bits/(Pixel*Frame)             : 0.080
Stream size                    : 298 MiB (100%)
Title                          : [H264]
Writing library                : x264 core 128 r2216+698 b3c1019
Encoding settings              : cabac=1 / ref=16 / deblock=1:0:0 / analyse=0x3:0x133 / me=umh / subme=10 / psy=1 / fade_compensate=0.00 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=32 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=-4 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=0 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / fgo=0 / bframes=12 / b_pyramid=1 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=23 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=16.0000 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Language                       : Japanese
Default                        : Yes
Forced                         : No

Any idea what the issue is?Please, read the 'Infos needed to fix&reproduce bugs,..'-sticky before you post about a problem.

[Selur]

a. You are using 32bit Filters right? (Hybrid used Avisynth+ 32bit)
b. The source should not really matter if you feed the Avisynth script to Hybrid since Hybrid then itself doesn't open the source, your script does.
c. Does the Avisynth Preview work properly?
Would need a debug output level 9 of the analysis and the job generation to know more.

Cu Selur

Ps.: if you create a small sample of your source and you can reproduce the issue with that sample and share it (+ the subitles) with me I can try to reproduce it here.

[Eamon]

a. Yep, only 32bit filters.
b. Yes, I understand. The strange issue is when I change the source, there is no problem. Which makes me think that this issue is something to do with the source vid.
c. I assume you mean in the Filtering tab right? When you change Support > Avisynth (default was Vapoursynth) and then I click Avisynth Preview. A new window opens up, and it says Preparing Environment for --Input. But then nothing else occurs. When I try to move the blue thing or press forward etc.. the entire window disappears.

I'll PM you the files I've used.

[Selur]

c. I assume you mean in the Filtering tab right? When you change Support > Avisynth (default was Vapoursynth) and then I click Avisynth Preview. A new window opens up, and it says Preparing Environment for --Input. But then nothing else occurs. When I try to move the blue thing or press forward etc.. the entire window disappears.

which means that something went wrong, usually the log should show where the problem was.

Cu Selur

[Selur]

Found the problem.
Hybrid doesn't know how to handle 10bit Avisynth input atm.
-> I'll look into it.

Cu Selur

[Eamon]

Nice. Will await the fix

[Selur]

Send you a link to dev version via PM which adds support for high bit depth Avisynth scripts which hopefully fixes your problem.

Cu Selur

[Eamon]

Okay, I installed your dev version. I got a new error.

When I open the AVS script in Hybrid, I get a message saying: Vapoursynth - unsupported color sampling: YV12 with bit depth: 2.

When I try to run the encode in Jobs, it automatically crashes upon start. Same thing happens when I change Support to 'Avisynth.'

[Selur]

Strange worked fine here.
And yes you can't use Vapoursynth while feeding Hybrid with an Avisynth script.
-> will need a debug output level 9 to know what's not working for you

[Eamon]

And yes you can't use Vapoursynth while feeding Hybrid with an Avisynth script.

Hmm, strange. Because remember I said in a few posts above that when I changed source, it worked okay with no errors - and I didn't change Support from Vapoursynth to Avisynth.

I've sent you the debug output level 9 via PM.