nerfnet: Streaming Video over nRF24L01

A couple of days ago I published a blog discussing how I used NRF24L01 radios to implement a point-to-point network between two Raspberry Pi computers. I implemented this as a virtual network device and sent packets between the radios.

Since then, I have made numerous improvements to the software and more than tripled the throughput from ~90kbps to nearly 300kbps. These improvements were through a variety of changes that I will cover in this blog post.

Streaming video from one headless Raspberry Pi to another

Thanks to the higher throughput, I was able to implement streaming video using the h264 HEVC video codec and monaural audio using the Opus codec at 32kbps. The result is great, especially when considering the link.

Continue reading to learn more!

Implementation

If you want to know more about the implementation, I recommend that you ready my previous blog post. I will go over the improvements that were made and how I was able to stream video.

The previous implementation was able to transmit network traffic at 90kbps (that's kilobits). Since that blog post, I have improved performance to nearly 300kbps which is high enough to support streaming h.265 video!

Protocol Improvements

The first improvement was to add transaction/sequence IDs to packets. This allows senders/receivers to know which packets have been processed and detect gaps/remove duplicates. This was added to the protobuf protocol.

  // A TxRx request for the network tunnel.
  message NetworkTunnelTxRx {
    // The bytes to send to the secondary radio.
    optional bytes payload = 1;

    // The number of remaining bytes for this packet. Once zero, write out
    // to the tunnel interface.
    optional uint32 remaining_bytes = 2;

    // The ID of this payload.
    optional uint32 id = 3;

    // The ID that this payload is acking from the secondary radio.
    optional uint32 ack_id = 4;
  }

Radio Tuning

The next step was to tune the radio. The main improvement was to ensure that the radio transitioned back to standby mode after transmitting.

  while (!radio_.txStandBy()) {
    LOGI("Waiting for transmit standby");
  }

In addition, small tweaks were made to decrease the interval between retries, decrease the address size from 5 to 3 bytes and use an 8-bit CRC instead of 16-bit.

  CHECK(channel < 128, "Channel must be between 0 and 127");
  CHECK(radio_.begin(), "Failed to start NRF24L01");
  radio_.setChannel(channel);
  radio_.setPALevel(RF24_PA_MAX);
  radio_.setDataRate(RF24_2MBPS);
  radio_.setAddressWidth(3);
  radio_.setAutoAck(1);
  radio_.setRetries(0, 15);
  radio_.setCRCLength(RF24_CRC_8);
  CHECK(radio_.isChipConnected(), "NRF24L01 is unavailable");

Frame Format

The final tweak was to replace protocol buffers with a hand-rolled packet format with only two bytes of overhead per 30 bytes of network traffic.

bool RadioInterface::EncodeTunnelTxRxPacket(
    const TunnelTxRxPacket& tunnel, std::vector<uint8_t>& request) {
  request.resize(kMaxPacketSize, 0x00);
  if (tunnel.id.has_value()) {
    request[0] = tunnel.id.value();
  }

  if (tunnel.ack_id.has_value()) {
    request[0] |= (tunnel.ack_id.value() << 4);
  }

   if (tunnel.payload.size() > kMaxPayloadSize) {
    LOGE("TxRx packet payload is too large");
    return false;
  }

  request[1] = tunnel.bytes_left;
  for (size_t i = 0; i < tunnel.payload.size(); i++) {
    request[2 + i] = tunnel.payload[i];
  }

  return true;
}

The first byte of this packet encodes the transaction ID as a pair of nibbles. These transaction IDs are never, which allows the value 0 in this byte to be used to negotiate a connection reset.

The second byte contains the number of remaining bytes in a network frame, capped to 255. If this value ever drops to 30 or fewer bytes, the frame is complete and written to the network tunnel.

Video Streaming

In order to achieve video streaming, I used the h.265 video codec at a low bit rate. I also downsampled the framerate and size of the stream. Here is the ffmpeg command used to convert the video. 

ffmpeg -i bbb_sunflower_1080p_60fps_normal.mp4 \
    -acodec libopus -b:a 32k -ac 1 \
    -vcodec libx265 -b:v 50k -filter:v fps=fps=15,scale=720:480 \
    bbb_sunflower_1080p_60fps_lowbitrate.mkv

The next step was to setup an RTP stream using VLC.

cvlc -vvv --no-sout-video \
    --sout '#rtp{dst=localhost,port=9000,sdp=rtsp://:8080/test.sdp}' \
    --sout-rtp-caching=5000 \
    bbb_sunflower_1080p_60fps_lowbitrate.mkv

This was received by mplayer and displayed on the framebuffer.

sudo mplayer -ao alsa:device=hw=2.0 \
    -vo fbdev2 -vf scale=720:480 -cache 64000 \
    rtsp://192.168.10.2:8080/test.sdp

Automatic Connection Recovery

The final step to make this project easier to use in the field was to set it up to run as a systemd job. I added support to nerfnet to automatically detect a bad connection and reset it. The following rule is crude, but worked for my experiments.

[Unit]
Description=The nerfnet listener.
After=multi-user.target

[Service]
Type=idle
ExecStart=/home/andrew/Projects/nerfnet/build/nerfnet/net/nerfnet --secondary
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

Range

After all of these tweaks, I took the hardware out to a nearby parking lot and did some range testing. I was able to easily achieve 60 meters and still be able to stream video. This was enough for me to call it a success.

Radio on a lamp standard, roughly 60 meters away

Conclusion

I think this is about as good as it will get. There might be room for further performance improvements in the way that the radio is used, but this was enough for me to call it a success.

Thanks for reading!