[BUG] Subtitle title tag punctation causes tag truncation and crashes hybrid

[rinzero]

Using Hybrid v2024.03.18.1 64-bit on Windows 10 x64, I'm seeing subtitle metadata title tags truncated in the output file. The stream itself is fine, just the metadata appears truncated. Using ffprobe to compare TAG:title lines:

Source file (the first two are from audio streams, the second two from subtitle streams):
TAG:title=Commentary by Sir Roger Moore
TAG:title=Commentary by director Lewis Gilbert, prod. designer Ken Adam, co-writer Christopher Wood and Michael G. Wilson
TAG:title=Commentary by Sir Roger Moore
TAG:title=Commentary by director Lewis Gilbert, prod. designer Ken Adam, co-writer Christopher Wood and Michael G. Wilson

Output file:
TAG:title=Commentary by Sir Roger Moore
TAG:title=Commentary by director Lewis Gilbert, prod. designer Ken Adam, co-writer Christopher Wood and Michael G. Wilson
TAG:title=Commentary by Sir Roger Moore
TAG:title=Commentary by director Lewis Gilbert

I noticed that the truncation happened at the first comma, so I made a test file:

Source subtitle tags:
TAG:title=Long subtitle stream title tag with no punctuation to see if there is a string length limit or if there is some other problem with string processing
TAG:title=Short title, with comma
TAG:title=Test me: áåōü .<> /? ;':" []{} \| `~!@#$%^&*()-_=+

The truncated title tags were visible in the Subtitle Queue on the Subtitle tag. The second tag truncated to "Short title" and the third tag truncated to "Test me: áåōü .<> /? ;':" []{} \| `~!@#$"

That test file crashed the 04_muxing step. The log tab showed:

Error: The file '[]{}' could not be opened for reading: open file error.
-> 2024-04-08@10_41_35_0210_04_muxing crashed: Error: The file '[]{}' could not be opened for reading: open file error.
Aborting '2024-04-08@10_41_35_0210_04_muxing' due to: Error: The file '[]{}' could not be opened for reading: open file error.

So I dropped the punctuation test tag and tried hybrid again with just the first two tags, and that gave the (expected) truncated output:

TAG:title=Long subtitle stream title tag with no punctuation to see if there is a string length limit or if there is some other problem with string processing
TAG:title=Short title

I tested that same file with Handbrake and ffmpeg, and in both cases all three tags copied to the output without issue. So I'm guessing at some point hybrid puts the subtitle metadata in a filename or command parameter, and special characters make things implode? If so, what's the allowed character set (or list of prohibited characters)?

[Selur]

Will look at it tomorrow, to late today.
No clue why it's crashing without looking at the source.
It's probably some issue in the muxing call and some needed escaping.
=> a debug output would help (read the sticky), since for example I have no clue what muxer is called with what settings from your description.


Cu Selur

[rinzero]

Debug output and sample file attached.

[Selur]

Thanks will fix it and report back after work.

Cu Selur

[Selur]

Not sure I get this fixed today as mkvmerge has it's own escaping and I'm not sure how to tackle this:

F:\Hybrid\64bit\mkvmerge.exe --ui-language de --output ^"G:\Output\test.mkv^" --language 0:en --display-dimensions 0:1920x1080 --language 1:en --track-name 1:Stereo --language 2:en --track-name ^"2:Long subtitle stream title tag with no punctuation to see if there is a string length limit or if there is some other problem with string processing^" --language 3:es --track-name ^"3:Short title, with comma^" --language 4:fr --track-name ^"4:Test me: áåōü .^<^> /? ;':\^" []{} \^| `~^!@#$^%^^^&*^(^)-_=+^" ^"^(^" ^"C:\Users\Selur\Desktop\tag_punctuation_test.mkv^" ^"^)^" --title ^"Ministry of Propaganda - Theatrical Archive - Looking for Some Thrills-trailer^" --track-order 0:0,0:1,0:2,0:3,0:4

using ^ as escaple element is rather uncommon.

Cu Selur

[Selur]

Okay, after some testing and coding, I got to the following conclusion.
There are multiple ways to fix this:
a. escape like mkvtoolnix for this breaks tons of later code in Hybrid
b. disallow

[]()%!^\"<>&|

inside titles and simply remove them when titles get passed through.
=> I'll probably go with option 2, but I'll have to do some testing whether all there characters need to be removed.

Cu Selur

[Selur]

Send you a link to a dev version for testing, via pm.
Let me know how it behaves.

Cu Selur

[rinzero]

I tested with the new version of Hybrid and it completes with the expected character filtering. See 2024.04.10.1_test1_HybridDebugOutput.

I also tested a bit more by adding a % after "Short" in the second title tag, and removing the % and " from the third title tag (see tag_punctuation_test2):

TAG:title=Short% title, with comma
TAG:title=Test me: áåōü .<> /? ;': []{} \| `~!@#$^&*()-_=+

2024.03.18.1 gave (see 2024.03.18.1_test2_HybridDebugOutput):

TAG:title=Short
TAG:title=Test me: áåōü .<> /? ;': []{} \| `~!@#$^&*()-_=+

and 2024.04.10.1 gave (see 2024.04.10.1_test2_HybridDebugOutput):

TAG:title=Short title, with comma
TAG:title=Test me: áåōü . /? ;': []{} \ `~@#$*-_=+

So it looks like % also causes the truncation, and it was the " in the title that broke mkvmerge

[Selur]

Okay, that seems to be 'fixed' then.

[rinzero]

It does! Thanks for getting to this so quickly