11.04.2022, 11:42
I wasn't sure if this should be categorized as a Bug or just Help, so I'm leaving the title without a prefix until I know which it is.
Hello everyone, I'm new here but been using Hybrid for a few years now.
I wanted to see if I could finally get a bug I've had for ages with Hybrid potentially figured out.
I'll be quite comprehensive with this in hopes to eliminate the need for too many follow-up questions.
First, let's cover the things asked for in the sticky.
Next, I guess I'll give you a rundown of what I'm running and what my primary use for Hybrid is and how I do things.
My Hardware:
Intel i7 8700k (8th Gen)
NVIDIA GTX 1060 6GB
16GB 3200MHz Memory
Samsung 980 NVMe 1TB
OS: Windows 11 Enterprise
Tools I use:
I typically encode anime, and I usually do this from raw BluRay rips (M2TS files).
However, I sometimes have to use files that have already been re-encoded as well (MKV files) due to availability issues.
I'm usually re-encoding videos for use in a VR game (VRChat) where video playback within the game has some requirements. The video players are designed to syncronize the video playback to everyone in the room and people can be located all over the world and have varying bandwidths, ping, and hardware, not to mention there are limitations in how Unity (the engine the game uses) handles videos, and the player can break for everyone if it detects issues with syncing it properly to everyone in the instance simultaneously... so these videos need to be encoded to some strict limitations and requirements to ensure that they can be delivered without playback issues arising.
I've learned that worldwide synchronized video playback is a huge pain in the ass where you need to encode to strict low-bandwidth & decode standards and use a good CDN that has good replication across servers hosted globally. But getting to watch things with my internet friends is worth it.
Anyway, the first step of the process is finding acceptable subtitles and remuxing them into the file. Sometimes the included (usually PGS) file that comes with the BluRay is decent enough in which case I'll just slap the M2TS directly into Hybrid. However I typically prefer well-stylized and better translated versions which I have to get from elsewhere, in which case I have to use MKVToolNix to remux them together properly (sometimes I need to add positive or negative delays to the subtitles to get them to sync properly with the video).
Point being, what I slap into Hybrid is either the raw M2TS from BDRip, or an MKV I've created because I wanted a different subtitle than the included subs.
Audio I typically encode from PCM or in rarer cases DTSHD-MA or FLAC. I would only encode from anything else if I couldn't find something lossless.
How I Do Things
When watching movies in VRChat, resolutions above 720p aren't terribly meaningful unless you're using a gigantic screen or standing way too close to the screen, so the first thing I do to push required bitrate down is scale my (usually 1920x1080) source down 1280x720 (or an equivalent if aspect isn't 16:9). This saves on both bitrate and encode/decode immensely.
I assume Bicubic Spline is the best downscale filter available without getting into Avisynth/Vapoursnyth madness, although I've tried to flip between Lanczos and Bicubic Spline and it looks to me like Lanczos might be ever so slightly sharper but BSpline doesn't have as much ringing maybe? Idk, at this point I've just stuck with BSpline because it's default (default is usually default for a reason, so that's what I do if I can't really tell the difference between two things).
For encoding I use CRF 23 because I want consistent quality-level moreso than a strict bitrate (although I monitor it to ensure the bitrate stays below 2000kbps on average... usually CRF23 pushes it down closer to 1000-1500 when dealing with anime. I really just want to tell the encoder to "encode to this quality level, keep the bitrate as low as you can within this quality level, and you're allowed to use up to this much bitrate maximum", so I think a capped CRF where I could specify a hard maximum bitrate (I'd say 2000kbps) is what I'd actually like to use but I don't see that as an option in Hybrid. But yeah, CRF23 and most of my encodes come out in the 1000kbps to 2000kbps average bitrate range.
Video Encode Settings
Codec: x264
AVC Profile: Base (Unity seems to prefer Base)
Level: 3.1 (I have "Minimize AVC level for mbps and bitrate" enabled)
Entropy Coding: CAVLC
Preset: Veryslow
Tune: fastdecode
Crop/Resize Settings
Resize: 1280x720 (or equivalent)
Resize Method: Bicubic Spline
Audio Encode Settings
Codec: AAC
Bitrate: 320kbps
Channels: Stereo (downmixed from surround if need be)
Sample Rate: 48KHz (VRC doesn't behave with anything else)
Subtitle Settings:
Embed Subtitles Into Frame (required)
Container Settings:
Type: MP4 or MKV (usually MP4, MKV is only for if I need to do some more edits after encoding)
Special Settings: Interleave 500ms
Filter Support:
No XSynth
Config View:
Post Work:
After output, typically either the job is done (movies), or if its a series I'll use MP4Box to concatenate several outputs into a single large MP4 file (a single monolithic file instead of multiple separate files is better for the player as well as the viewing experience). As far as I can tell Hybrid doesn't have support for concatenation so I use MP4Box and MKVToolNix for this sort of thing. So usually just something like "MP4Box -add 01.mp4 -cat 02.mp4 -cat 03.mp4 -cat 04.mp4 -cat 05.mp4 -cat 06.mp4 Output.mp4" in CMD and I'm done.
My Issue
The issue arises during encoding, where when it gets around to the point where it needs to encode, it just crashes with the following error message:
Now here's the strange part -- simply changing the filter support to Avisynth appears to fix the issue, although with some caveats.
Typically I prefer to use the "no XSynth" support mode because I'm not really using anything in Vapoursynth or Avisynth.
The crash behavior in no XSynth mode is that it'll try to encode video stream before crashing partway through.
Switching to Vapoursnyth will fix it sometimes, but if it doesn't then the crash behavior is that it'll simply look at the video stream and crash immediately, like it won't even bother trying.
It will typically throw the error:
Switching to AviSynth, it almost never crashes, and will seemingly encode the video just fine... with the caveat that some subtitles won't be in the right typeface, I guess some part of the subtitle extraction process goes wrong with AviSynth. It also sometimes will throw alert dialogues about fonts when it starts to encode the video... but I think I've confirmed that subtitles aren't the issue and that this occurs whether or not subtitles are even present, Avisynth either not properly extracting or supporting subtitles seems to be a separate issue of its own.
To make matters more wonky, the crashes are also inconsistent across a batch of files. What I mean by that is, within one set of M2TS or MKV files all extracted from the same disc of the same BluRay, or a batch of files all encoded the same way, only some will crash the encoder while others won't... even though as far as I can tell all the files are all untouched direct rips or all encoded the same way, which makes no sense to me. So I just kinda roll the dice every time I feed a file into Hybrid, hoping it doesn't crash or complain, which if I didn't know better I'd say it does it at random, but I know there must be a reason.
This is an issue I've dealt with for years now, kinda just hoping an eventual update to Hybrid would fix it but it never has, so I guess it's time for me to just finally say something and either figure out what I'm doing wrong, or draw attention to the issue if there really is a bug somewhere. All this time I've just kinda lived with it, and told Hybrid to encode all files without pausing for crashes and at the end of it I retry all failed files with AviSynth and just live with whatever fuck-ups it does to the subtitles in those files.
I figured reporting an issue that is intermittent and inconsistent like this probably won't yield any concrete answers... but worth a shot anyway.
Conclusion
Anyway, just figured I'd introduce myself and what the context for what I wanna do is, as well as present this age-old problem of mine.
If anyone has any other advice about things I could do better or you see anything I'm clearly doing wrong here etc, do let me know. I think I may eventually post to another part of the forum to ask about things I could maybe do with AviSynth or Vapoursynth to improve my end results (as long as they don't make the files bigger or more difficult to decode, or increase encode time by too much). Things that can be done to improve downscaling anime would be especially helpful. It's worth noting that I'm a GUI-dweller who doesn't really do great with purely commandline based programs (why I'm using Hybrid in the first place), but fiddling with this stuff has kinda forced me down that road regardless.
Oh and Selur, I'm not sure if you'd remember but I was the guy who was e-mailing you about SVP support a few years back, I believe you actually added Overwrite option to the interframe interpolation in AviSynth because of me wanting to use my specific SVP settings in Hybrid back then, I used it quite a bit so thanks a lot for that.
Hello everyone, I'm new here but been using Hybrid for a few years now.
I wanted to see if I could finally get a bug I've had for ages with Hybrid potentially figured out.
I'll be quite comprehensive with this in hopes to eliminate the need for too many follow-up questions.
First, let's cover the things asked for in the sticky.
- Q: What Hybrid version did you use?
- A: Windows 11 Enterprise Version 10.0.22000 Build 22000 (though this issue existed on Windows 10 as well).
- Q: What did happen? If an error message came up what did it exactly say?
- A: "CRASHED Helper1, exitCode: -1073741819" in No XSynth or "Crashed with exit status 0" in Vapoursynth
- Q: What did you do to get to the bug?
- A: I think the rest of this post pretty well covers that I guess.
- Q: Debug output would be helpful.
- A: I've included debug logs for NoXSynth, VapourSynth, and AviSynth scenarios (AviSynth works but with the caveat of fonts sometimes being the wrong face or spacing).
- Q: A small sample, which allows the reprocudtion of the problem would be nice.
- A: I'll find a way to include these if you really need one, we're dealing with large files here so unless you're fine with downloading a 5GB+ file I host on GDrive or something I'd need to figure out a way to get the size down by altering the file in such a way that the problem is preserved.
Next, I guess I'll give you a rundown of what I'm running and what my primary use for Hybrid is and how I do things.
My Hardware:
Intel i7 8700k (8th Gen)
NVIDIA GTX 1060 6GB
16GB 3200MHz Memory
Samsung 980 NVMe 1TB
OS: Windows 11 Enterprise
Tools I use:
- Hybrid
- MKVToolNix
- MP4Box
I typically encode anime, and I usually do this from raw BluRay rips (M2TS files).
However, I sometimes have to use files that have already been re-encoded as well (MKV files) due to availability issues.
I'm usually re-encoding videos for use in a VR game (VRChat) where video playback within the game has some requirements. The video players are designed to syncronize the video playback to everyone in the room and people can be located all over the world and have varying bandwidths, ping, and hardware, not to mention there are limitations in how Unity (the engine the game uses) handles videos, and the player can break for everyone if it detects issues with syncing it properly to everyone in the instance simultaneously... so these videos need to be encoded to some strict limitations and requirements to ensure that they can be delivered without playback issues arising.
I've learned that worldwide synchronized video playback is a huge pain in the ass where you need to encode to strict low-bandwidth & decode standards and use a good CDN that has good replication across servers hosted globally. But getting to watch things with my internet friends is worth it.
Anyway, the first step of the process is finding acceptable subtitles and remuxing them into the file. Sometimes the included (usually PGS) file that comes with the BluRay is decent enough in which case I'll just slap the M2TS directly into Hybrid. However I typically prefer well-stylized and better translated versions which I have to get from elsewhere, in which case I have to use MKVToolNix to remux them together properly (sometimes I need to add positive or negative delays to the subtitles to get them to sync properly with the video).
Point being, what I slap into Hybrid is either the raw M2TS from BDRip, or an MKV I've created because I wanted a different subtitle than the included subs.
Audio I typically encode from PCM or in rarer cases DTSHD-MA or FLAC. I would only encode from anything else if I couldn't find something lossless.
How I Do Things
When watching movies in VRChat, resolutions above 720p aren't terribly meaningful unless you're using a gigantic screen or standing way too close to the screen, so the first thing I do to push required bitrate down is scale my (usually 1920x1080) source down 1280x720 (or an equivalent if aspect isn't 16:9). This saves on both bitrate and encode/decode immensely.
I assume Bicubic Spline is the best downscale filter available without getting into Avisynth/Vapoursnyth madness, although I've tried to flip between Lanczos and Bicubic Spline and it looks to me like Lanczos might be ever so slightly sharper but BSpline doesn't have as much ringing maybe? Idk, at this point I've just stuck with BSpline because it's default (default is usually default for a reason, so that's what I do if I can't really tell the difference between two things).
For encoding I use CRF 23 because I want consistent quality-level moreso than a strict bitrate (although I monitor it to ensure the bitrate stays below 2000kbps on average... usually CRF23 pushes it down closer to 1000-1500 when dealing with anime. I really just want to tell the encoder to "encode to this quality level, keep the bitrate as low as you can within this quality level, and you're allowed to use up to this much bitrate maximum", so I think a capped CRF where I could specify a hard maximum bitrate (I'd say 2000kbps) is what I'd actually like to use but I don't see that as an option in Hybrid. But yeah, CRF23 and most of my encodes come out in the 1000kbps to 2000kbps average bitrate range.
Video Encode Settings
Codec: x264
AVC Profile: Base (Unity seems to prefer Base)
Level: 3.1 (I have "Minimize AVC level for mbps and bitrate" enabled)
Entropy Coding: CAVLC
Preset: Veryslow
Tune: fastdecode
Crop/Resize Settings
Resize: 1280x720 (or equivalent)
Resize Method: Bicubic Spline
Audio Encode Settings
Codec: AAC
Bitrate: 320kbps
Channels: Stereo (downmixed from surround if need be)
Sample Rate: 48KHz (VRC doesn't behave with anything else)
Subtitle Settings:
Embed Subtitles Into Frame (required)
Container Settings:
Type: MP4 or MKV (usually MP4, MKV is only for if I need to do some more edits after encoding)
Special Settings: Interleave 500ms
Filter Support:
No XSynth
Config View:
x264 --preset slower --profile baseline --level 3.1 --ref 5 --sync-lookahead 18 --partitions i4x4,p4x4,p8x8,b8x8 --merange 24 --vbv-maxrate 42000 --vbv-bufsize 14000 --sar 1:1 --qpfile GENERATED_QP_FILE --no-deblock --non-deterministic --range tv --colormatrix bt709 --demuxer raw --input-res 1280x720 --input-csp i420 --input-range tv --input-depth 8 --fps 24000/1001 --output-depth 8 --output "C:\Users\Shiguri\Videos\Hybrid\Temp\01.264" -
Post Work:
After output, typically either the job is done (movies), or if its a series I'll use MP4Box to concatenate several outputs into a single large MP4 file (a single monolithic file instead of multiple separate files is better for the player as well as the viewing experience). As far as I can tell Hybrid doesn't have support for concatenation so I use MP4Box and MKVToolNix for this sort of thing. So usually just something like "MP4Box -add 01.mp4 -cat 02.mp4 -cat 03.mp4 -cat 04.mp4 -cat 05.mp4 -cat 06.mp4 Output.mp4" in CMD and I'm done.
My Issue
The issue arises during encoding, where when it gets around to the point where it needs to encode, it just crashes with the following error message:
CRASHED Helper1, exitCode: -1073741819
Now here's the strange part -- simply changing the filter support to Avisynth appears to fix the issue, although with some caveats.
Typically I prefer to use the "no XSynth" support mode because I'm not really using anything in Vapoursynth or Avisynth.
The crash behavior in no XSynth mode is that it'll try to encode video stream before crashing partway through.
Switching to Vapoursnyth will fix it sometimes, but if it doesn't then the crash behavior is that it'll simply look at the video stream and crash immediately, like it won't even bother trying.
It will typically throw the error:
Crashed with exit status 0
Switching to AviSynth, it almost never crashes, and will seemingly encode the video just fine... with the caveat that some subtitles won't be in the right typeface, I guess some part of the subtitle extraction process goes wrong with AviSynth. It also sometimes will throw alert dialogues about fonts when it starts to encode the video... but I think I've confirmed that subtitles aren't the issue and that this occurs whether or not subtitles are even present, Avisynth either not properly extracting or supporting subtitles seems to be a separate issue of its own.
To make matters more wonky, the crashes are also inconsistent across a batch of files. What I mean by that is, within one set of M2TS or MKV files all extracted from the same disc of the same BluRay, or a batch of files all encoded the same way, only some will crash the encoder while others won't... even though as far as I can tell all the files are all untouched direct rips or all encoded the same way, which makes no sense to me. So I just kinda roll the dice every time I feed a file into Hybrid, hoping it doesn't crash or complain, which if I didn't know better I'd say it does it at random, but I know there must be a reason.
This is an issue I've dealt with for years now, kinda just hoping an eventual update to Hybrid would fix it but it never has, so I guess it's time for me to just finally say something and either figure out what I'm doing wrong, or draw attention to the issue if there really is a bug somewhere. All this time I've just kinda lived with it, and told Hybrid to encode all files without pausing for crashes and at the end of it I retry all failed files with AviSynth and just live with whatever fuck-ups it does to the subtitles in those files.
I figured reporting an issue that is intermittent and inconsistent like this probably won't yield any concrete answers... but worth a shot anyway.
Conclusion
Anyway, just figured I'd introduce myself and what the context for what I wanna do is, as well as present this age-old problem of mine.
If anyone has any other advice about things I could do better or you see anything I'm clearly doing wrong here etc, do let me know. I think I may eventually post to another part of the forum to ask about things I could maybe do with AviSynth or Vapoursynth to improve my end results (as long as they don't make the files bigger or more difficult to decode, or increase encode time by too much). Things that can be done to improve downscaling anime would be especially helpful. It's worth noting that I'm a GUI-dweller who doesn't really do great with purely commandline based programs (why I'm using Hybrid in the first place), but fiddling with this stuff has kinda forced me down that road regardless.
Oh and Selur, I'm not sure if you'd remember but I was the guy who was e-mailing you about SVP support a few years back, I believe you actually added Overwrite option to the interframe interpolation in AviSynth because of me wanting to use my specific SVP settings in Hybrid back then, I used it quite a bit so thanks a lot for that.