Feature Suggestions Regarding Automatic Filename Generation - Printable Version +- Selur's Little Message Board (https://forum.selur.net) +-- Forum: Hybrid - Support (https://forum.selur.net/forum-1.html) +--- Forum: Problems & Questions (https://forum.selur.net/forum-3.html) +--- Thread: Feature Suggestions Regarding Automatic Filename Generation (/thread-153.html) |
Feature Suggestions Regarding Automatic Filename Generation - Nuihc88 - 17.11.2017 As i've been doing a lot of test encodes lately, which mostly just consists of tediously copy-pasting Encoding settings and filesizes to text files for comparison in WinMerge, i've come up with some ideas that would make this sort of thing a whole lot faster...
RE: Feature Suggestions Regarding Automatic Filename Generation - Selur - 17.11.2017 My two cents: Regarding 1.: This request should go to the x265 developers not me. Regarding 2: Not going to happen. Intermediate files did at one point contain lots of meta data which causes lots of problems on Windows due to lack of UTF-8 support through a bunch of tools and some Windows combinations. To avoid those problems Hybrid now should use generated names based on the job ID and this is the way it will stay. Regarding 3: or example: 'test#{2}' would become 'test#01' unless it already exists, in which case it'll become 'test#02' as opposed to just replacing the older file. If something like this is wanted it should cover all eventualities and I need to know whether this is a feature or the file name generation or of the job processing. To clarify: During File generation: Existence of file XY would be generated upon job creation. So if you run a job multiple times output would be overwritten. (atm. 'Generate' option adds a '_new' whenever the file name already exists <- this I would remove) During job processing: During job processing a Hybrid would check the last 'rename' (or if no 'rename' subjob exists the last processing call to check whether the output exists and adjust the call accordingly - this could be a lot of work.) Regarding 4: *.chp (Chapter files) and *.xml (Matroska tag files) should both be created inside the temp folder, not the source folder (unless the source is the temp folder. + both should get deleted unless Hybrid is told to keep intermediate files (or the processing is aborted or crashed) Tested both here and for me the *.chp and the *.xml file both get created inside the temp folder and get deleted during normal processing. -> If this isn't the case for you, share details and I might be able to fix it. Cu Selur RE: Feature Suggestions Regarding Automatic Filename Generation - Nuihc88 - 17.11.2017 (17.11.2017, 06:47)Selur Wrote: Regarding 1.:I requested it here since this information is mostly already available through Hybrid and the files it produces as long as mkvinfo & mediainfo are up to date. It just takes a while to gather it all manually; except for PSNR & SSIM, as i'm still not sure how to find those values after setting their Metrics on at 'x265 > Misc'. (17.11.2017, 06:47)Selur Wrote: Regarding 2:I don't think i got the purpose and method of this suggestion properly across... It would only need to rename the intermediate files (for later reference), after the encoding job is done and full file has been generated. It would only need to use metadata for files supported through mkvinfo & mediainfo. Having a standard source format to read should deal with compatibility problems. If a specified field is not found, it should display nothing and it would be run at the point these files are normally deleted. Readout forwarded to Filename could be limited to supported characters and everything else replaced with something readable. The real benefit from Suggestions 1 & 2 would be that it would be possible to arrange files by PSNR & SSIM in any file explorer as well as keep track of trends in how .265(, etc.) file's bitrates, sizes and other values correlate with set parameters, essentially turning any file browser into a dynamic comparison chart. (17.11.2017, 06:47)Selur Wrote: What should happen if there already is a file named 'test#2' ?What i intend here is that it would always choose the next untaken numerical value and if it reaches #99, it'll go to #100, even if zero padding is set to only 2 digits. (17.11.2017, 06:47)Selur Wrote: If something like this is wanted it should cover all eventualities and I need to know whether this is a feature or the file name generation or of the job processing.I would describe it as a quick filesystem check and name correction happening about the same time the job is generating tagging info's, before intermediate files get put into a container.
(17.11.2017, 06:47)Selur Wrote: *.chp (Chapter files) and *.xml (Matroska tag files) should both be created inside the temp folder, not the source folder (unless the source is the temp folder .I chose to keep the intermediate (.265 & .aac) files as a workaround, since Hybrid overwrites the .mkv test files i'm generating (as '_new' seems to only kick in occasionally, perhaps due to some condition i'm unaware of; most of my test files only differ by one parameter). I then check their metadata with MediaTab (which internally uses mediainfo). (17.11.2017, 06:47)Selur Wrote: Tested both here and for me the *.chp and the *.xml file both get created inside the temp folder and get deleted during normal processing.This is because i used 'enc\' filename prefix as a workaround in order to get a dynamic subfolder under the source folder. All other temp files get directed to 'enc\' where i want them. I realize i could just set static output and temp paths, but i want to keep my encodes close to my source files. This way i don't get my intermediates from different sources mixed up, nor do i have to jump between tabs as much. So the point of suggestion 4 is about having full support for dynamic directories for all generated files. RE: Feature Suggestions Regarding Automatic Filename Generation - Selur - 17.11.2017 Quote:I requested it here since this information is mostly already available through Hybrid and the files it produces as long as mkvinfo & mediainfo are up to date. It just takes a while to gather it all manually; except for PSNR & SSIM, as i'm still not sure how to find those values after setting their Metrics on at 'x265 > Misc'.Tell Hybrid to create report files ('Config->Internals->Create report file') and you will see something like this: x264 (1st pass bitrate) processing started... Problem with generic tags is most applications don't support them. Most containers like mp4 and mkv, support a few official tags. (For mkv see https://matroska.org/technical/specs/tagging/index.html.) And libraries like MediaInfo only support these (not sure if MediaInfo really supports all official mkv tags). -> So in your case it won't help at all if I would add additional mkv tags since MediaInfo won't detect them and report them. Quote:This is because i used 'enc\' filename prefix as a workaround in order to get a dynamic subfolder under the source folder.Okay, this feature isn't used as intended and I do not plan to write any workaround for that. Quote:So the point of suggestion 4 is about having full support for dynamic directories for all generated files.So a different file directory for each type of generated file. -> Sorry, that's not happening. Quote:I chose to keep the intermediate (.265 & .aac) files as a workaround, since Hybrid overwrites the .mkv test files i'm generating (as '_new' seems to only kick in occasionally, perhaps due to some condition i'm unaware of; most of my test files only differ by one parameter).The '_new' will only checked and added when 'Generate' gets triggered. For example: You load a file (or multiple files) into Hybrid and 'Generate' is enabled, Hybrid will check for each file whether a name with it's name exists and will add '_new' in case it does. So assuming you now add a job for the file(s) you added, change your encoding settings and create another job: a. generate won't get triggered b. no additional '_new' would get added, even if 'Generate' was triggered since the output file doesn't exist. c. if you do stuff like changing the output folder through file name generation like you did, Hybrid will look inside the wrong folder. Quote:I then check their metadata with MediaTab (which internally uses mediainfo).Did some reading up and tested a few things, seems like MediaInfo nowadays supports custom tagging (it even shows non standard tags). I theoretically could write an option which would add separate tags for any wanted data. To add PSNR and SSIM results the output of the encoder would need to be parsed and then custom tags would need to be added. (Different code for each multiplexer.) -> will think about it in case some other users also see a usefulness in this option. Cu Selur RE: Feature Suggestions Regarding Automatic Filename Generation - Nuihc88 - 17.11.2017 (17.11.2017, 19:25)Selur Wrote: Tell Hybrid to create report files ('Config->Internals->Create report file') and you will see something like this:Thanks, this is something i've been looking for, for a while now. (17.11.2017, 19:25)Selur Wrote: -> So in your case it won't help at all if I would add additional mkv tags since MediaInfo won't detect them and report them. SmileI haven't actually asked for more tags as such though, but for copying of metadata to output filename. e.g. 'output_filename_bitrate=12345Kbps_size=987654bytes' or 'output_filename_crf=24;qcomp=0.8;pbratio=1.2.mkv'. The exact syntax wouldn't matter either, if ';' or '=' were to cause problems, underscore would do too. (17.11.2017, 19:25)Selur Wrote: Okay, this feature isn't used as intended and I do not plan to write any workaround for that.Understandable, but besides the point. I only use it because it's currently closer to what i want than the official implementation for setting directories. (17.11.2017, 19:25)Selur Wrote: So a different file directory for each type of generated file. -> Sorry, that's not happening.If i understand you correctly, nope. What i mean is that all intermediate files get put into the same subfolder, that's location is dynamic in relation to either the source or output directory. There is no need to separate each filetype. Implementations i would find practical would look something like: 'source_folder/encodes/*' with or without the output file, 'output_folder/intermediaries/*' with or without the output file or a new folder named after the output filename (without an extension) such as 'source_folder/output_filename/*' or 'output_folder/output_filename/*'. (17.11.2017, 19:25)Selur Wrote: The '_new' will only checked and added when 'Generate' gets triggered.This is consistent with my observations. The way this check works is probably good enough for casual encoding jobs, but when testing and comparing various encoding settings it becomes very inpractical, hence the need to keep the intermediaries, put them in a separate folder, group by type & arrange folder by date. The extra .chp & .xml files in source folder are a rather minor griavance by comparison. (17.11.2017, 19:25)Selur Wrote: c. if you do stuff like changing the output folder through file name generation like you did, Hybrid will look inside the wrong folder.Due to points a. and b., knowing this doesn't really make a difference though, hence Suggestion 4. RE: Feature Suggestions Regarding Automatic Filename Generation - Selur - 17.11.2017 Btw. if overwriting is a real problem enable 'noOverwriting' through the misc.ini. see: [INFO] *hidden* Hybrid options,... This way Hybrid should never overwrite an output file. Cu Selur RE: Feature Suggestions Regarding Automatic Filename Generation - Selur - 18.11.2017 Intermediate files go to the temp folder, unless you tell Hybrid that the temp folder should be the same as the output folder by either selecting: 'Config->Output->Container Settings->Genarl Settings->Keep intermediate files' or setting: 'Config->Path->Adjust->Temp path to output path'. Adding another folder which would be used instead of the temp folder seems wrong. That is what the temp folder is for. -> Adding an option which allows to name a folder that should be used as temp relative to the output folder might solve the problem, but in case the relatively specified temp folder wouldn't exist Hybrid would abort the job creation. ----------------- About adding data to the output name, it's the same problem as using tags.
RE: Feature Suggestions Regarding Automatic Filename Generation - Nuihc88 - 18.11.2017 (17.11.2017, 23:27)Selur Wrote: Btw. if overwriting is a real problem enable 'noOverwriting' through the misc.ini.Ok, ran some tests and this is somewhat better in that regard, but on the downside it results in very long filenames quite quickly. I'll leave this enabled for now, though. (18.11.2017, 01:01)Selur Wrote: Adding another folder which would be used instead of the temp folder seems wrong. That is what the temp folder is for.It would only need to be created when 'Keep intermediate files' is set. (18.11.2017, 01:01)Selur Wrote: -> Adding an option which allows to name a folder that should be used as temp relative to the output folder might solve the problem, but in case the relatively specified temp folder wouldn't exist Hybrid would abort the job creation.This would in itself be acceptable for this type of usage scenario, automatically checking for and creating that folder on fly would be even better. There are however other usage scenarios where setting dynamic paths would also be useful. For example if user were to run Hybrid in portable mode from an USB-stick (as i do) and then move it to another computer... Without dynamic paths you'll need to set these paths manually each time and remember to make sure each directory you set has enough free space available, rather than just checking the free space on the source file's drive. (18.11.2017, 01:01)Selur Wrote: About adding data to the output name, it's the same problem as using tags.I guess this will be more of a problem for PSNR & SSIM readouts? In my experience MediaInfo at least seems fairly consistent in it's readouts. Either way, personally i'd probably implement this using regular expression text searches, like this: Cache MediaInfo output and search for metadata text readout matching: '[linebreak]File size: ' & copy contents from there on until next [linebreak] is found and remove any spaces. If no matches are found output nothing, if multiple matches are found output longest string by default, if multiple same length strings are found use the first one by default. This way you wouldn't need to separately add support for each possible tag. It might also be useful to allow user to select one of the other matches and specify a limiter for string length. You could thereby let users specify custom searches using some RegEx like syntax such as:'filename_size=[File size: ]_rate=[{2}Bit rate: (3)]' With '{2}' being one possible implementation for selecting the second match and '(3)' being one possible implementation of a custom limiter for cutting off excess text where necessary. Example output in this case would be something like: 'filename_size=12345bytes_rate=123'+.extension Since this would be a feature for advanced users, when something breaks, normally they'd be able to find out why and correct it for themselves. PS. Adding PSNR & SSIM to custom tags and/or filename would certainly be a nice feature, but even MediaInfo readouts can be very useful by themselves. Copying recent changes to 'Encoding settings:' over to filename could be somewhat tricky as well, but probably also the most useful. Personally i would solve the Encoding settings parsing problem with something like this:
RE: Feature Suggestions Regarding Automatic Filename Generation - Selur - 18.11.2017 Quote:It would only need to be created when 'Keep intermediate files' is set.Argh. I would prefer to not create folders in Hybrid if at all possible. ----------------------------------------------------------------------------------------------- Quote:Personally i would solve the Encoding settings parsing problem with something like this:This sounds more like something that should be done in a separate tool. Like a small program (or script) which you feed with a file and a bunch of parameter. Something like: Quote: For better understanding of what I mean with this. Here a few examples: Example 1: MediaInfoRenamer --Inform="Video;%Width%,%Height%,%BitRate%,%FrameRate%" --Separator=","--Merger=" " "z:\Output\myfile.mkv" a. call: mediainfo --Inform="Video;%Width%,%Height%,%BitRate%,%FrameRate%" "z:\Output\myfile.mkv" 640,352,722134,25.000 b. split the output using the separator:
c. join that data using the merger: 640 352 722134 25.000 d. rename the file from: "z:\Output\myfile.mkv" "z:\Output\myfile 640 352 722134 25.000.mkv" Example 2: MediaInfoRenamer --Inform="Video;%Width/String%#%Height/String%#%BitRate/String%#%FrameRate/String%" --Separator="#" --Merger="_" "z:\Output\myfile.mkv" a. call: mediainfo --Inform="Video;%Width/String%#%Height/String%#%BitRate/String%#%FrameRate/String%" "z:\Output\myfile.mkv" 640 pixels#352 pixels#722 kb/s#25.000 FPS b. split the output using the separator:
c. join that data using the merger: 640 pixels_352 pixels_722 kb/s_25.000 FPS d. try to rename the file from: "z:\Output\myfile.mkv" "z:\Output\myfile_640 pixels_352 pixels_722 kb/s_25.000 FPS.mkv" Note: The following character are reserved and thus will cause problems:
Example 3: MediaInfoRenamer --Inform="Video;Width %Width/String%#Height %Height/String%#%Bitrate BitRate/String%#Framerate %FrameRate/String%" --Separator="#" --Merger="_" "z:\Output\myfile.mkv" a. call: mediainfo --Inform="Video;%Width/String%#%Height/String%#Bitrate(kBit pro sec) %BitRate%#Framerate(fps) %FrameRate%" "z:\Output\myfile.mkv" Width 640 pixels#Height 352 pixels#Bitrate(kBit pro sec) 722#Framerate(fps) 25.000 b. split the output using the separator:
c. join that data using the merger: Width 640 pixels_Height 352 pixels_Bitrate 722_Framerate(fps) 25.000 d. rename the file from: "z:\Output\myfile.mkv" "z:\Output\myfile_640 pixels_352 pixels_722_Framerate(fps) 25.000.mkv" Cu Selur RE: Feature Suggestions Regarding Automatic Filename Generation - Nuihc88 - 18.11.2017 (18.11.2017, 07:54)Selur Wrote: This sounds more like something that should be done in a separate tool.Hmm... It would also need a way of being pointed towards user's baseline reference for Encoding settings or it won't be able to shorten the output, as well as towards the correct '*_Report.txt' file for PSNR & SSIM parsing if/when implemented here. Other than that this looks good. |