[BUG] Hybrid CLI hangs on non-ASCII file name

[potejan]

Hello,

Opening non-ASCII files from GUI works fine, but if passed as argument via CLI, they get interpreted as '????', and that hangs the GUI.

Examples:

& "C:\Program Files\Hybrid\Hybrid.exe" '.\ГГГГГ.avi'
& "C:\Program Files\Hybrid\Hybrid.exe" '.\ГГГГГ\test.avi'

If I were to make an educated guess, the issue is probably in converting the argv C-strings to QStrings.

Thanks!

[Selur]

Problem is probalby related to your local code page and the general Windows settings.
Would need more details about your system to reproduce it. Especially what code pages your system uses in the command line.

[potejan]

I am running Windows 11 using English (United States) set everywhere. chcp from cmd.exe returns code page: 437 (United States). I tried the new Terminal app, as well as Powershell and Command Prompt, all experience the same issue.

[Selur]

Does the problem also occure if you just call Hybrid (without chaining it behind something else)?
Btw. why are you doing that?
(I suspect the problem is related to that Hybrid uses utf-8 for everything,..)

[potejan]

If you run the GUI by double-clicking on the Hybrid EXE, it handles non-ASCII characters fine. Only the CLI struggles. Maybe the GUI code page is UTF8 while the CLI inherits the terminal one which is US?

The AVIs I am processing are named according to their event, and they are written in Cyrillic

---

Btw, I tried to set the code page to UTF8 via: 

chcp 65001

But it did not change anything for Hybrid...

UTF8 most definitely supports Cyrillic, I feel like something just isn't able to properly convert the input string from Cyrillic to UTF8.

[Selur]

When using QApplication::arguments(), I get it displayed correctly in the GUI (so no ???), but QFile::exists(..) fails and thus Hybrid assumes the file doesn't exist.
->I'm looking into it.

Cu Selur

[Selur]

int main(int argc, char* argv[])
{
  QApplication application(argc, argv);

  for(int i = 1; i < argc; ++i) {
    std::cerr << "exists(1):" << QFile::exists(QFile::decodeName(argv[i])) << std::endl;
    std::cerr << "exists(2):" << QFile::exists(QFile::encodeName(argv[i])) << std::endl;
    std::cerr << "exists(3):" << QFile::exists(QFile::fromLocal8Bit(argv[i])) << std::endl;
    std::cerr << "exists(4):" << QFile::exists(QFile::fromUtf8(argv[i])) << std::endl;
    std::cerr << "exists(5):" << QFile::exists(QFile::fromLatin1(argv[i])) << std::endl;
    std::cerr << "exists(6):" << QFile::exists(argv[i]) << std::endl;
  }
  QStringList arguments = application.arguments(); // when I use this the gui properly displays the name
  QString arg;
  for(int i = 1; i < argc; ++i) {
    arg = arguments.at(i);
    std::cerr << "exists(7):" << QFile::exists(QFile::encodeName(arg)) << std::endl;
    std::cerr << "exists(8):" << QFile::exists(arg) << std::endl;
    std::cerr << "exists(9):" << QFileInfo(arg).exists() << std::endl;
  }

  return 0;
}

returns 0 for each of these tests :/
-> no clue how to get that working, may be I look at it again over the weekend, but I'm out of ideas.

Cu Selur

[potejan]

Very interesting. What string encoding method is Hybrid compiled to use? If I recall correctly, there's 3 types: single-byte, multi-byte, and unicode. Maybe this has something to do.

Ref: https://docs.microsoft.com/en-us/cpp/atl...w=msvc-170

What does QFile.fileName() print? Does that work?

If you can send me a tiny sample app with the code you just mentioned above, I can try to reverse engineer/debug it to see the exact memory that gets compared and see how it differs.

Thanks

[Selur]

Seems like atm.

cl -c -nologo -Zc:wchar_t -FS -Zc:rvalueCast -Zc:inline -Zc:throwingNew /std:c++17 -bigobj /Zc:__cplusplus -permissive- -O2 -MD /Zc:__cplusplus -utf-8 -W3 -w34100 -w34189 -w44996 -w44456 -w44457 -w44458 -wd4577 -wd4467 -EHsc -DUNICODE -D_UNICODE -DWIN32 -D_ENABLE_EXTENDED_ALIGNED_STORAGE -DWIN64 -DTARGET="\"Hybrid\"" -DBUILDVERSION=\"2022.07.15.1\" -DWINORIGINALFILENAME="\"Hybrid.exe\"" -DWINCOMPANYNAME="\"Selurs Software\"" -DWINFILEVERSION=2022,07,15 -DWINFILEDESCRIPTION="\"Hybrid Encoder Gui\"" -DWINLEGALCOPYRIGHT="\"Copyright 2021 by Selur\"" -DWINLEGALTRADEMARKS1="\"All Rights Reserved\"" -DBUILDDATE=\"20220715\" -DWINPRODUCTVERSION=0,2,4,0 -DNDEBUG -DQT_NO_DEBUG -DQT_SVG_LIB -DQT_WIDGETS_LIB -DQT_MULTIMEDIA_LIB -DQT_GUI_LIB -DQT_XML_LIB -DQT_CONCURRENT_LIB -DQT_NETWORK_LIB -DQT_CORE_LIB -I. -IC:\Qt\6.3.1\msvc2019_64\include -IC:\Qt\6.3.1\msvc2019_64\include\QtSvg -IC:\Qt\6.3.1\msvc2019_64\include\QtWidgets -IC:\Qt\6.3.1\msvc2019_64\include\QtMultimedia -IC:\Qt\6.3.1\msvc2019_64\include\QtGui -IC:\Qt\6.3.1\msvc2019_64\include\QtXml -IC:\Qt\6.3.1\msvc2019_64\include\QtConcurrent -IC:\Qt\6.3.1\msvc2019_64\include\QtNetwork -IC:\Qt\6.3.1\msvc2019_64\include\QtCore -Irelease -IuiHeaders -I/include -IC:\Qt\6.3.1\msvc2019_64\mkspecs\win32-msvc -Forelease\ @C:\Users\Selur\AppData\Local\Temp\avsCAS.obj.13944.5125.jom

Seems like Qt by default uses:
" -Zc:wchar_t" and "-DUNICODE -D_UNICODE"

Cu Selur

[i990049]

shanaencoder and xmedia recode work fine with Unicode.Do you have the same batch process problem on your
system? 

https://forum.selur.net/thread-2474-page-2.html