Chromecast Casting¶

MediaX detects Chromecast devices on the network and casts H.264 video to them. This is a native implementation, so there is no Cast SDK and no other library apart from OpenSSL, which MediaX already uses for SRT encryption, and which is needed only for the TLS connection that the Cast protocol runs over.

Build option: -DBUILD_CAST=ON (the default, needs OpenSSL, not available on Windows).

How it works¶

Encoder (H.264) -> HLS segmenter (TS segments + live playlist) -> HTTP server (this process)
                                                                        ^
                                                                        | GET playlist, segments
Chromecast <--- CASTv2 over TLS: LAUNCH, LOAD <--- CastSender ----------+

1. CastDiscovery asks the network for Chromecasts and keeps a list of the devices that answer.
2. CastSender starts an HTTP server on the local address that routes to the device and cuts the video into segments, one per key frame after the target length (2 seconds by default).
3. It connects to the device, starts the Default Media Receiver and tells it to play the playlist URL. The device then fetches the playlist and segments from this computer. The first segments must exist before the device is told to play, so there is a delay before the picture appears.
4. The control connection is kept alive with the heartbeat channel. If the connection is lost, or somebody else takes over the device, the sender connects again and carries on.

The device has to reach this computer over HTTP, so a firewall must allow the connection from the device. The HTTP server only talks to the device being cast to, it refuses everybody else, and it serves nothing but the playlist and the segments from memory.

Finding Chromecasts¶

#include "cast/cast_discovery.h"

mediax::cast::CastDiscovery discovery;
discovery.RegisterDeviceFoundCallback([](const mediax::cast::CastDevice& device) {
  std::cout << device.friendly_name << " (" << device.model << ") at " << device.address << "\n";
});
discovery.RegisterDeviceLostCallback([](const mediax::cast::CastDevice& device) { /* ... */ });
discovery.Start();

mediax::cast::CastDevice tv;
if (discovery.WaitForDevice("Living Room TV", 5000, &tv)) {
  // The name is not case sensitive. The id or the address work as well
}
auto everything = discovery.GetDevices();

Callbacks run on the discovery thread and must not block. Devices are reported when they join, when what they announce changes and when they go away or stop answering. Speakers and displays that speak the Cast protocol are listed too. CastDevice::SupportsVideo() says whether a device can show video, and CastSender refuses those that cannot.

Casting video¶

#include "cast/cast_sender.h"

mediax::cast::CastSender sender;
sender.SetDevice(tv);
sender.RegisterErrorCallback([](mediax::cast::CastStatus status, const std::string& message) { /* ... */ });
sender.Open();    // starts the HTTP server
sender.Start();   // connects to the device in the background, and again if it is lost
while (capturing) {
  sender.SendAccessUnit(nal_data, nal_size, pts_90khz, is_idr);
}
sender.Close();   // stops the application on the device

SendAccessUnit() takes one access unit as an Annex B byte stream. Parameter sets (SPS and PPS) are repeated in front of key frames that lack them, using the ones seen earlier. Video given before the first key frame is dropped.

To cast a MediaX encoder (this is how test cards are cast), attach it before opening it. The encoder then gives its output to the sender instead of sending RTP:

mediax::rtp::h264::openh264::RtpH264OpenH264Payloader encoder;
encoder.SetStreamInfo(info);
sender.Attach(&encoder);
sender.RegisterKeyFrameRequestCallback([&encoder] { encoder.ForceIdr(); });
encoder.Open();
sender.Open();
sender.Start();
encoder.Start();
// ... encoder.Transmit(rgb_frame) for every frame

Segments end at key frames. When a key frame request callback is registered the sender asks for one as soon as a segment is longer than wanted, and when it is about to start, so the encoder's own key frame interval does not matter. Without the callback the encoder needs a key frame about every 2 seconds, see CastConfig::hls.

Settings¶

SetReconnect(enable, delay_ms) controls the automatic reconnection. WaitForStreaming(timeout_ms) returns once the device plays the stream, GetState() and RegisterStateCallback() follow the connection through kConnecting, kLaunching, kLoading and kStreaming, and GetStatistics() has the frame, segment and HTTP counters.

Casting an RTP stream¶

RtpToCast takes the RTP packets of an H.264 stream and casts the same video without decoding or encoding it, RtpSapToCastRoute does it for a stream announced with SAP and follows the announcement:

mediax::cast::CastDiscovery discovery;
discovery.Start();

mediax::cast::RtpToCastRouteConfig config;
config.session_name = "Camera 1";    // The SAP session to cast
config.device_name = "Living Room TV";
mediax::cast::RtpSapToCastRoute route(config);
route.SetDiscovery(&discovery);
route.Start();

mediax::sap::SapListener::GetInstance().RegisterSapListener(
    "", [&route](mediax::sap::SdpMessage* sdp, void*) { route.OnAnnouncement(*sdp); }, nullptr);
mediax::sap::SapListener::GetInstance().Start();

When the announcement appears the route joins its RTP stream and casts it, when it is deleted the casting stops, and when the device has not been detected yet the route tries again at the next announcement. Only H.264 can be cast, and RTP carries no decode time stamps, so streams with B frames are not supported. After a lost RTP packet the video is held back until the next key frame.

Qt6¶

auto* discovery = new mediax::qt6::QtCastDiscovery(this);
connect(discovery, &mediax::qt6::QtCastDiscovery::deviceFound, this,
        [this](const QVariantMap& device) { combo->addItem(device["name"].toString(), device); });
discovery->start();

auto* sender = new mediax::qt6::QtCastSender(this);
sender->setDevice(combo->currentData().toMap());
connect(sender, &mediax::qt6::QtCastSender::stateChanged, this, [](mediax::qt6::QtCastSender::State) { /* ... */ });
connect(sender, &mediax::qt6::QtCastSender::errorOccurred, this, [](const QString& text) { /* ... */ });
sender->start();                                   // returns at once
sender->sendAccessUnit(nal_bytes, pts, is_idr);    // from any thread

Devices are QVariantMaps with the keys id, name, model, address, port and capabilities, usable from QML. All signals are emitted from the thread the object lives in. attachEncoder() casts a MediaX encoder, and setKeyFrameRequestHandler() lets the encoder be asked for key frames.

Example¶

The cast-list example lists the video devices on the network that can be streamed to (--all adds speakers, --watch keeps running and reports devices as they appear and disappear):

cast-list
cast-list --watch --search=3

The cast-stream example casts an animated test card, encoded with OpenH264. Run at a terminal with nothing on the command line it searches the network and asks which device and which test card to use. Giving --device (or --address) and --card skips the questions, and --nointeractive never asks, which is what a script wants (the first device found and the bouncing ball are then used unless given). -h lists all the options:

cast-stream                                 # asks for the device and the test card
cast-stream --list                          # what is on the network
cast-stream --device="Living Room TV"       # asks for the test card only
cast-stream --device="Living Room TV" --card=cube --nointeractive
cast-stream --address=192.168.1.20 --card=sweep --width=1920 --height=1080
cast-stream -h

The animated cards, all offered by the menu, are ball, sweep, cube, text, pulse, noise and elite (the ships of the game Elite, --ship=0..12 shows one, --ship_seconds sets how long each is shown). smpte is a still picture that can be given with --card.

Troubleshooting¶

Limits¶

• Only H.264 is cast. Devices differ in what else they can play over HLS, and AV1 and H.265 are not covered.
• There is no audio.
• The delay is several seconds. Shorter segments reduce it, down to the limit the device and the network allow.
• The device authentication challenge of the Cast protocol is not made, and the self-signed certificate of the device is not verified, as for every other Cast sender. Devices that insist on authentication, such as those used for protected content, are not supported. The Default Media Receiver does not need it.
• Discovery needs multicast DNS to pass between this computer and the devices. Different subnets and some Wi-Fi guest networks block it, --address and CastSender::SetDevice() reach a device without it.