I am working on a web UI for a robotics project the requires multiple video streams to be viewed in a web client. I have not done much with video before and am having trouble finding resources that explain how I can achive this.
I have been able to get a video stream by using the gstreamer udpsink:
gst-launch-1.0 v4l2src device=/dev/video0 ! image/jpeg, width=320, height=240, framerate=30/1 ! udpsink host=0.0.0.0 port=8123
Then I listen to this stream using a nodejs server that relays the data back to the client. The problems with this is that the image is very small otherwise I get errors from the udpsink saying that the packet size it too big. I understand that gstreamer has rtp payload plugins that solve this issue but I cannot find a way to depayload using nodejs.
The other method that I have tried is to use MediaMTX since it provides easy browser access which works well on localhost but gives considerable latency over network. The gstreamer pipline that I used for this is:
gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! x264enc tune=zerolatency ! rtspclientsink location=rtsp://localhost:8554/mystream
