Selur's Little Message Board

Hi there!

I have been using Hybrid for some time for upscaling with a lot of sucess.
But I have not been able to convert/upscale this file for the life of me. I tried to convert it to h264 in Adobe Media Encoder, but i still cannot get it to work in hybrid

Here is the original file.
https://drive.google.com/file/d/1hvZKmJx...sp=sharing

Here is the AME converted file
https://drive.google.com/file/d/1o_HKR3Q...sp=sharing

Here is the debug Hybrid log
https://drive.google.com/file/d/1SkdPMxv...sp=sharing

Please help! Tysm

NVEnc reports: "y4m: failed to parse y4m header."
which means that either something is wrong with the Vapoursynth script used or something is interfering with the encoding.

The encoding call used:
seems fine.

The script:
-> The letterboxing is wrong. AddBorders can't be used with negative values and the gui should not allow this.

I tried reproducing this but I can't find a click order to get this.
Only way this might work is if one loads a settings preset after loading a source.
-> I need a step-by-step to guide to reproduce this.

Cu Selur

(15.04.2022, 07:34)Selur Wrote: [ -> ]The script:

Code:

# Imports
import os
import sys
import ctypes
# Loading Support Files
Dllref = ctypes.windll.LoadLibrary("C:/Program Files/Hybrid/64bit/vsfilters/Support/libfftw3f-3.dll")
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
# Import scripts folder
scriptPath = 'C:/Program Files/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# Loading Plugins
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/SharpenFilter/AWarpSharp2/libawarpsharp2.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/SharpenFilter/CAS/CAS.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/GrainFilter/RemoveGrain/RemoveGrainVS.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/GrainFilter/AddGrain/AddGrain.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/NEO_FFT3DFilter/neo-fft3d.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/DFTTest/DFTTest.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/EEDI3m.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/ResizeFilter/nnedi3/NNEDI3CL.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/libmvtools.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/temporalsoften.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/scenechange.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# Import scripts
import edi_rpow2
import havsfunc
# source: 'G:\6002688.mp4'
# current color space: YUV420P8, bit depth: 8, resolution: 720x480, fps: 29.97, color matrix: 470bg, yuv luminance scale: limited, scanorder: bottom field first
# Loading G:\6002688.mp4 using LibavSMASHSource
clip = core.lsmas.LibavSMASHSource(source="G:/6002688.mp4")
# Setting color matrix to 470bg.
clip = core.std.SetFrameProps(clip, _Matrix=5)
clip = clip if not core.text.FrameProps(clip,'_Transfer') else core.std.SetFrameProps(clip, _Transfer=5)
clip = clip if not core.text.FrameProps(clip,'_Primaries') else core.std.SetFrameProps(clip, _Primaries=5)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# making sure frame rate is set to 29.970
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
# setting field order to what QTGMC should assume (bottom field first)
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=1)
# Deinterlacing using QTGMC
clip = havsfunc.QTGMC(Input=clip, Preset="Slow", TFF=False, opencl=True) # new fps: 29.97
# make sure content is preceived as frame based
clip = core.std.SetFieldBased(clip, 0)
clip = clip[::2]
# denoising using DFTTest
clip = core.dfttest.DFTTest(clip=clip)
# contrast sharpening using CAS
clip = core.cas.CAS(clip=clip, sharpness=0.850)
clip = havsfunc.EdgeCleaner(c=clip)
# resizing using NNEDI3CL
clip = edi_rpow2.nnedi3cl_rpow2(clip=clip, rfactor=4, nns=4) # 2880x1920
# adjusting resizing
clip = core.fmtc.resample(clip=clip, w=1964, h=1080, kernel="lanczos", interlaced=False, interlacedd=False)
# letterboxing 1964x1080 to 1920x1080
clip = core.std.AddBorders(clip=clip, left=-22, right=-22, top=0, bottom=0)
# adjusting output color from: YUV420P16 to YUV420P8 for NVEncModel
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, range_s="limited")
# set output frame rate to 29.970fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
# Output
clip.set_output()

-> The letterboxing is wrong. AddBorders can't be used with negative values and the gui should not allow this.

Selur, thanks for the quick response!
I was able to upscale the h264 converted file on AME when I untick the Letterbox option.

I have a saved XML preset to resize SD files.
https://drive.google.com/file/d/1U6jHHRa...sp=sharing

I open Hybrid, apply the XML profile, open the original mpeg SD file 6002688.mpeg
https://drive.google.com/file/d/1hvZKmJx...R7gbV/view

and Hybrid gets stuck at about 98%, as seen on the log I shared earlier:
https://drive.google.com/file/d/1SkdPMxv...IGCxr/view

Best regards from Rio,

What I did was:

• load your profile
  
• adjust the paths in your profile
  
• save the profile again.
  
• restart Hybrid.
  
• load your profile.
  
• load the source you provided.
  
• check the Crop/Resize tab
  -> everything seems fine (Target resolution shows 1920x1080, same as Letterbox)
  
• check the Vapoursynth Filter Preview
  -> everything seems fine
  
  Code:

  clip = edi_rpow2.nnedi3cl_rpow2(clip=clip, rfactor=4, nns=4) # 2880x1920
# adjusting resizing
clip = core.fmtc.resample(clip=clip, w=1920, h=1080, kernel="lanczos", interlaced=False, interlacedd=False)
  no AddBorders.
  

-> You must be doing something different than I am as I can't reproduce the problem with the steps you mentioned.

I tried to reproduce this by:

• starting Hybrid
  
• apply the preset
  
• loading a 640x480 source (PAR 1:1)
  -> resize&letterbox get adjusted to 1964x1080
  since Hybrid is configured to get the target height to 1080. This results in a resize/target height of 1964, which also increases the letterbox (since letterbox needs to be >= target resolution)
  
• then I load your source
  target size is 1920x1080, letterbox is 1964x1080.
  
  Code:

  clip = core.fmtc.resample(clip=clip, w=1920, h=1080, kernel="lanczos", interlaced=False, interlacedd=False)
# contrast sharpening using CAS
clip = core.cas.CAS(clip=clip, sharpness=0.850)
# letterboxing 1920x1080 to 1964x1080
clip = core.std.AddBorders(clip=clip, left=22, right=22, top=0, bottom=0)
  script looks 'fine' no negative AddBorders values.
  

Sorry, but I can't fix the issue as I have no clue how to reproduce the issue.
-> Please check the steps and provide a step-by-step (<-!important!), on how to reproduce the issue. (negative letterbox values)
I'll will also write some code to add some save guards, which will do some additional checks.


Cu Selur

Dear Selur,

I have again the same problem wiht Hybrid gets stuck at about 98%, as seen on the log I shared earlier.
https://drive.google.com/file/d/1SkdPMxv...IGCxr/view

I have also recorded a video of the problem:
https://drive.google.com/file/d/1BfNX-PF...sp=sharing

Here is my step by step, done on the video:
1- Open Hybrid
2- Reset all settings
3- Apply the profile I created
https://drive.google.com/file/d/1U6jHHRa...AKYnz/view
4- Open a 4x3 SD mpeg file
https://drive.google.com/file/d/1DzLjFS2...sp=sharing
5- Start the job

And then hybrid gets stuck at 98,13%.
I also put task manager on the recorded steps so you can see that Hybrid's CPU usage goes from 100% to about 8% but nothing changes.

Maybe I need to delete my profile and start over?
Please let me know if you need further info! Thank you for this support and software.

Quote:I have again the same problem wiht Hybrid gets stuck at about 98%, as seen on the log I shared earlier.

Debug output the encodind is aborted due to "NVEnc output: y4m: failed to parse y4m header."
Looking at the Vapoursynth script:
I see no issue. -> Does the Vapoursynth Preview work without issues?
(input isn't the file you posted below)

-> will try to reproduce the issue here.

Cu Selur

I can reproduce the issue (which has nothing todo with the debug output), but atm. I have no clue whats causing it.
-> will look at it.

Cu Selur

Okay, there is something wrong with the input file.
When I open it in Hybrid, open the Vapoursynth preview and try to jump to a position near the end Preview freezes.
-> this is why the encoding stopy, Vapoursynth doesn't close the preview.

Seems like the d2vwitch&d2vsource can't handle the file properly.
When I tell Hybrid to use DGDecNV ("Filtering->Vappursynth->Misc->Use DGDeNV when available") and use the Preview it works fine, encoding works fine additionally enabling "Prefer LWLibAVSource over DG" and setting 'Libav hardware decoding' to 1 preview and encoding works fine too.
-> Seems like the libav software decoder can't properly handle the source.

Cu Selur

(27.04.2022, 16:20)Selur Wrote: [ -> ]Seems like the d2vwitch&d2vsource can't handle the file properly.
When I tell Hybrid to use DGDecNV ("Filtering->Vappursynth->Misc->Use DGDeNV when available") and use the Preview it works fine, encoding works fine additionally enabling "Prefer LWLibAVSource over DG" and setting 'Libav hardware decoding' to 1 preview and encoding works fine too.
-> Seems like the libav software decoder can't properly handle the source.

Cu Selur

Fantastic! This worked, thank you so much.
I created a profile only for this problematic MPEG files with these options enabled. ^^

I see no harm in keeping DGDecNV enabled by default. (It's not enabled by default in Hybrid since it only works with NVIDIA cards.)