Suggestions on video optimizing quality

[TheSashmo]

My commands are pretty straight forward, I try not to add too many options but here is my encode command:

-t decklink:connection=sdi:device=4:codec=UYVY -c libavcodec:codec=H.264:encoder=libx264:bitrate=20000000:preset=veryfast:subsampling=420:x264-params=intra-refresh=1:rc-lookahead=4 -s embedded --audio-codec=OPUS:sample_rate=48000:bitrate=256k --audio-capture-format channels=16

Yes I know this is 4:2:0 but honestly the best quality I have seen without overloading a machine is using that conversion, this does lead to some colour banding in solid images but I see occasional pulsing in static shots, and at times, see this video clip, you can see in the scene changes the frame glitches out, here is a still of it:

Here is the short video:
https://www.dropbox.com/scl/fi/8an0cx59jf9ea8klvdkh4/IMG_8897.mov?rlkey=elv12cd6hnbeornfvhsjgh55s&dl=0

What are your suggested encodings, or even suggested decode options?

[MartinPulec]

Hi, the glitches - it looks to me rather that is the network loss - is it possible for you to check receiver output/logs?

Altogether, it is a bit tough to deal with unspecified content. I've tried your parameters with this dataset, namely concatenating Jockey and ReadySetGo HEVC files. Recorded using --record and resulting .h264 files again concatenated, the result is here. Can you look at it and check if something of your issues can be seen there as well? The movement seem a bit choppy but I think it is in the source as well:

cat Jockey_3840x2160_120fps_420_8bit_HEVC_RAW.hevc ReadySetGo_3840x2160_120fps_420_8bit_HEVC_RAW.hevc > /tmp/test.hevc
uv --playback=/tmp/test.hevc -c libavcodec:codec=H.264:encoder=libx264:bitrate=20000000:preset=veryfast:subsampling=420:x264-params=intra-refresh=1:rc-lookahead=4 --record=/tmp

Occasional pulsing in static shots - so you are getting it even in paused video? I've tried with a single frame, and uv -t testcard:size=3840x2160:00000001.yuv:still, which seem ok to me (without pulsation).

When looking to your command, I don't see anything much harmful, although it can have been shorter (eg. rc-lookahead AFAIK does nothing if B-frames are not in use, which aren't by default, etc.). If you do not insist on fixed bit-rate, I believe that it is better to use CRF, especially if there is some motion.

Another idea - if you can tolerate higher bit-rate variance, you can try to set --param lavc-rc-buffer-size-factor= to higher values than the default 2.5, let say 10 or 20 and see if it makes some difference? Other idea - since these compression are done through libavcodec, if you had some recorded content (you can eventually use UG --record), you can try to find out command-line parameters for FFmpeg that will look ok and we can then check if this can be used through UltraGrid.

[alatteri]

Does increasing this value add to latency?

Another idea - if you can tolerate higher bit-rate variance, you can try to set --param lavc-rc-buffer-size-factor= to higher values than the default 2.5, let say 10 or 20

[TheSashmo]

@MartinPulec @alatteri the issue is I don't see any errors on the input or the output logs, no audio or video under runs. I'll be working with it extensively this week so I will have more information soon. Thanks for the input!

[TheSashmo]

@MartinPulec @alatteri

Still seeing the glitches. Could it be because my source is interlaced? I don't see any more errors come in looking at the cumulative logs.

Command:
--control-port=2615 --audio-filter controlport_stats --capture-filter preview:path=/tmp/encode_thumbnail_615.socket:target_size=854x480 -t decklink:connection=sdi:device=0:mode=Hi59 -c libavcodec:codec=H.264:encoder=libx264:bitrate=15000000:disable_intra_refresh:x264-params=rc-lookahead=5:tune=zerolatency,fastdecode:coder=cabac -s embedded --audio-codec=OPUS:sample_rate=48000:bitrate=256k --audio-capture-format channels=16 -f V:rs:120:180 -f A:mult:2 -m 1316 -P 21550 172.18.20.3

Encoder:
[lavcd aud.] Using audio encoder: libopus
[lavcd aud.] Using audio encoder: libopus
[lavcd aud.] Using audio encoder: libopus
[lavcd aud.] Setting bit rate to: 256000 bps
[lavc] Using codec: H.264, encoder: libx264
[lavc] Setting bitrate to 15.0 Mbps.
[lavc] Setting codec parameter preset to veryfast
[lavc] Setting codec parameter tune to zerolatency,fastdecode
[lavc] Auto-enabling interlaced DCT which may not be supported by HW decoders.
[lavc] Use ':disable_interlaced_dct' or ':safe' to disable.
[lavc] Setting thread count to 0, type: (other)
[lavc] Setting codec parameter tune to zerolatency,fastdecode
[lavc] Setting codec parameter x264-params to rc-lookahead=5
[lavcd aud.] Setting bit rate to: 256000 bps
[lavcd aud.] Setting bit rate to: 256000 bps
[lavcd aud.] Setting bit rate to: 256000 bps
[lavcd aud.] Setting bit rate to: 256000 bps
[lavcd aud.] Setting bit rate to: 256000 bps
[lavcd aud.] Setting bit rate to: 256000 bps
[lavcd aud.] Setting bit rate to: 256000 bps
[lavcd aud.] Setting bit rate to: 256000 bps
[lavcd aud.] Setting bit rate to: 256000 bps
[lavcd aud.] Setting bit rate to: 256000 bps
[lavcd aud.] Setting bit rate to: 256000 bps
[lavcd aud.] Setting bit rate to: 256000 bps
[lavcd aud.] Setting bit rate to: 256000 bps
[lavcd aud.] Setting bit rate to: 256000 bps
[lavcd aud.] Setting bit rate to: 256000 bps
[lavc libx264 @ 0x7f278c001100] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 AVX512
[lavc libx264 @ 0x7f278c001100] profile High 4:2:2, level 4.0, 4:2:2, 8-bit
[lavc] Selected pixfmt: yuv422p
[lavc] Selected pixfmt has subsampling 4:2:2, which is usually not supported by hw. decoders
[lavc] Use ':subs=420' or ':safe' to disable.
[transmit] FEC symbol size: 1134, symbols per packet: 1, payload size: 1134
[transmit] FEC symbol size: 375, symbols per packet: 3, payload size: 1125
[transmit] FEC symbol size: 469, symbols per packet: 2, payload size: 938

Decoder:
Video dec stats (cumulative): 929 total / 909 disp / 20 drop / 0 corr / 0 miss FEC noerr/OK/NOK: 928/1/0
Audio dec stats (cumulative): 8005 played / 8005 total audio frames
Video dec stats (cumulative): 1859 total / 1839 disp / 20 drop / 0 corr / 0 miss FEC noerr/OK/NOK: 1858/1/0
Audio dec stats (cumulative): 9556 played / 9556 total audio frames
Video dec stats (cumulative): 2789 total / 2769 disp / 20 drop / 0 corr / 0 miss FEC noerr/OK/NOK: 2788/1/0
Audio dec stats (cumulative): 11106 played / 11106 total audio frames
Video dec stats (cumulative): 3719 total / 3699 disp / 20 drop / 0 corr / 0 miss FEC noerr/OK/NOK: 3718/1/0
Audio dec stats (cumulative): 12584 played / 12679 total audio frames
Video dec stats (cumulative): 4649 total / 4629 disp / 20 drop / 0 corr / 0 miss FEC noerr/OK/NOK: 4648/1/0
Audio dec stats (cumulative): 14134 played / 14229 total audio frames
Video dec stats (cumulative): 5579 total / 5559 disp / 20 drop / 0 corr / 0 miss FEC noerr/OK/NOK: 5578/1/0
Audio dec stats (cumulative): 15610 played / 15802 total audio frames
Video dec stats (cumulative): 6509 total / 6489 disp / 20 drop / 0 corr / 0 miss FEC noerr/OK/NOK: 6508/1/0
Audio dec stats (cumulative): 17161 played / 17353 total audio frames

[TheSashmo]

So here is what I think I came up with. Thanks to @alatteri for suggestions, I think I have it dialed in now, but I need to try other sources on it.

--control-port=2615 --audio-filter controlport_stats --capture-filter preview:path=/tmp/encode_thumbnail_615.socket:target_size=854x480 -t decklink:connection=sdi:device=0:mode=Hi59 -c libavcodec:codec=H.264:encoder=libx264:bitrate=25000000:x264-params=vbv-bufsize=50000\:vbv-maxrate=25000 -s embedded --audio-codec=OPUS:sample_rate=48000:bitrate=256k --audio-capture-format channels=16 -f V:rs:120:180 -f A:mult:2 -m 1460 -P 21550 172.18.20.3

[MartinPulec]

@alatteri

Does increasing this value add to latency?

no. It allows just bigger individual frames (at the expense of the others). But, similarly to other non-standard options, it helps just in specific cases - in this case, because intra-refresh is in force, maybe it can help when there are scene cuts.

@TheSashmo
Still seeing the glitches. Could it be because my source is interlaced?

Usually no, lib264 has (still) support for interlaced video encode and it handles interlacing quite decently (we still sometime use it for interlaced signal). You can try to disable the support with disable_interlaced_dct to see the difference - it will likely perform significantly worse when there is much motion; it can slightly reduce compression load for static picture.

[TheSashmo]

@MartinPulec I'm still having issues. I can confirm its not packet loss, I'm on my own private lan and I still a line of banding at random times. I even added SRT on it to eliminate the thoughts that it was a packet issue, and I have 0 packet loss and 0 packet retransmission. Everything in the logs are clean, I even increased the bitrate to 30Mbps and its the same thing. I can't seem to replicate the exact cause of the problem, and its not even just scene changes. There is tons of CPU left over I am barely using 40% cpu.

[TheSashmo]

Could the issue be coming from the multiplier?

uv --param window-title=ultragird_decode_process_633 --control-port=2633 --audio-filter controlport_stats -d multiplier decklink:device=3:drift_fix#preview:path=/tmp/decode_thumbnail_633.socket:hq:rate_limit=2:target_size=854x480 -r embedded --audio-delay=130 -P 21948 127.0.0.1

[TheSashmo]

I have two encodes running almost identical, and one with intra refresh on, and one with intra refresh off, the one with it set to off definitely has more issues, the issue is getting better, but its still not 100% clean. I still have multiplier off in the decoder.

[alatteri]

can you try it with the most basic command for the receiver unit to eliminate the other stuff.

uv -d decklink

[TheSashmo]

@alatteri my most simple command that I can run, drift fix on or off makes no difference.

uv -control-port=2633 --audio-filter controlport_stats -d decklink device=3:drift_fix -r embedded --audio-delay=130 -P 21948 127.0.0.1

[MartinPulec]

I'd have another idea - if you have the possibility, you can try to record the compressed content on the receiver with --record=<recording_path>?

If you are using libx264 you'll get individual frames in the specified directory. If you concatenate them (cd <recording_path>; cat *.h264 >out.h264). The resulting file can be played with eg. mpv - can try to replay the file to see if the problem occurs there as well or it is clean? This might give us an information whether the problem is already present in the encoded stream.

[TheSashmo]

@MartinPulec ok thanks I have to find a way to add that programmatically on my side, I guess I can use the control port to activate that.

[TheSashmo]

So in my testing, I can confirm when using multiplier, I get no errors on the decode side, but I can see errors in the video.

Without adding more arguments to record the content, its hard for me to show you the output of the video.

Suggestions? @MartinPulec

[MartinPulec]

I am sorry but without more information I do not have much ideas, to sum up:

1. encoding params doens't seem to cause the problem when I've tried to replicate
2. you are saying that there is no network losses so that it seems that it is hard to tell where later it can occur

If you don't have to possibility to record the sent data, are you able to capture at least the data at the receiver?

uv -d dump[:<directory>]