Feature Request: Dynamic TensorRT models and build options

[Selur]

Okay, that does not answer my questions clearly.
But after some reading,...

a. min_shapes in vsmlrt is the min resolution of the content that the trained engine can handle.
b. max_shapes in vsmlrt is the max resolution of the content that the trained engine can handle.
c. opt_shapes in vsmlrt is the resolution of the content that the engine really gets trained.
The more min&max differ from opt the less reliable/efficient is the model.

So if the contents you apply a model to only have just a few resolutions, you get better results training the model for each resolution.
If the resolutions of your content, are around a few fixed resolutions, creating multiple engine files around those fixed resolutions is a good idea.
Using something like:

min_shapes = (720, 480)       # smallest resolution
opt_shapes = (1920, 1080)     # most frequent resolution
max_shapes = (3840, 2160)     # highest resolution
static_shape = False

might be 'okay', if you want to only create one engine, but the min and max resolution will be less efficient than individual models.
If you use tiling, note that the tiling size is the resolution, so if you always use 256x256 tiling it does not make sense to use dynamic shapes.
So if you have a model that requires a specific min resolution it does not make sense to use a min_shapes value below this resolution.
Would be interresting to read about some tests, experiences about how 'much' diversion is 'okay' between opt<>min opt<>max before one really should better create a separate engine file.

Cu Selur

Ps.: send you link to a test version => let me know whether that works as expected.