Blog

All Blog Posts | Next Post | Previous Post

WebRTC peer-to-peer connection in native applications

Bookmarks:

Monday, July 24, 2023

We just released TMS WEB Core v2.2 last week with added WebRTC support, but did you know TMS FNC WX Pack also has a control for WebRTC? This let's you to create WebRTC-based applications not only for the web but also as native applications. In case you missed our first introduction to WebRTC, you can read about it in our previous post here.

TTMSFNCWXWebRTCCommunication

In the latest release of TMS FNC WX Pack, the new TTMSFNCWXWebRTCCommunication control was added. TTMSFNCWXWebRTCCommunication allows you to create serverless peer-to-peer connection between 2 (or more) peers. You can use this not only for audio and video calls but also screen sharing and data sending!

A small example

Let's explore an example connection for a video call with the demo that is also included in TMS FNC WX Pack. The basic idea is to connect two peers via a signaling server. The signaling server is used until the connection is established between the two peers, in many cases you can simply disconnect after that. In the demo this signaling server is a WebSocket server made with TMS FNC WebSocket, but keep in mind the signaling server could be anything as long as the required offer/answer and candidates can be communicated between the peers.

First we connect to the server, when the connection succeeds we request a unique identifier for ourselves from the server:

procedure TForm5.Button2Click(Sender: TObject);
begin
  if edtSelf.Text <> '' then
  begin
    TMSFNCWebsocketClient1.Port := Round(NumberBox1.Value);
    TMSFNCWebsocketClient1.Connect;
  end;
end;

procedure TForm5.TMSFNCWebsocketClient1Connect(Sender: TObject;
  AConnection: TTMSFNCWebSocketConnection);
begin
  AConnection.Send('{"type": "request-id", "sender": "' + edtSelf.Text + '"}');
  Button2.Enabled := False;
end;

This unique identifier will be used to send messages via the server as the server does not persist information about where each peer is connected, it only forwards messages. This is an implementation we choose for the demo, there are of course other ways to handle this, it all depends on your use-case!

Once we are connected to the server we can see a list of peers that are also connected to the same server. To connect to a peer, we can select one from the list and click the connect button, which will first send a join message through the signaling server. We want to know, if the peer is free:

FPeer := ListBox1.ListItems[ListBox1.ItemIndex].ItemData.Detail;
TMSFNCWebsocketClient1.Send('{"type": "join", "user": "' + FPeer + '", "sender": "' + FID + '"}');

The rest of the code is handling messages arriving from the server. Such as, if our join request was successful, we send our offer:

procedure TForm5.TMSFNCWebsocketClient1MessageReceived(Sender: TObject;
  AConnection: TTMSFNCWebSocketConnection; const aMessage: string);
begin
  //...
  else if msgType = 'join-success' then
  begin
  //Disable controls
  SetControls(False);
  //Add peer connection object
  TMSFNCWXWebRTCCommunication1.AddPeerConnection(FPeer, '<video class="peer-fullscreen" autoplay playsinline></video>', '');

  //Could join to peer, initiate call by creating offer
  TMSFNCWXWebRTCCommunication1.CreateOffer(FPeer);
  end
  else if msgType = 'offer' then
  begin
    //An offer arrived, accept:
    if jsonObject.TryGetValue<TJSONObject>('offer', data) then
      TMSFNCWXWebRTCCommunication1.AcceptOffer(FPeer, data.ToJSON);
  end
  else if msgType = 'answer' then
  begin
    //An answer arrived, accept:
    if jsonObject.TryGetValue<TJSONObject>('answer', data) then
      TMSFNCWXWebRTCCommunication1.AcceptAnswer(FPeer, data.ToJSON);
  end
  else if msgType = 'candidate' then
  begin
    //A candidate arrived, add:
    if jsonObject.TryGetValue<TJSONObject>('candidate', data) then
      TMSFNCWXWebRTCCommunication1.AddIceCandidate(FPeer, data.ToJSON);
  end
  //...
end;

procedure TForm5.TMSFNCWXWebRTCCommunication1Offer(Sender: TObject; AID,
  AOffer: string);
begin
  TMSFNCWebsocketClient1.Send('{"type": "offer", "user": "' + FPeer + '", "offer": ' + AOffer + '}');
end;

When the receiving peer gets an offer, it will accept it and create an answer in return. The answer is then sent to the initiating peer which accepts it. Meanwhile the candidates are also exchanged. And with that, our connection is live and our two cat colleagues can finally have a virtual meeting!

It's worth to mention in this demo we don't wait for the candidates to gather. While this allows a quicker connection, it can be sometimes unreliable if the candidates are sent before it makes sense for the receiving peer to accept them (= before it can generate an answer). To wait until the ICE candidates are gathered and added as part of the offer and answer, you can use the OnOfferWithCandidates and OnAnswerWithCandidates events instead.

Combine web with native

WebRTC is supported by web browsers and that is what makes TTMSFNCWXWebRTCCommunication possible. This also means you can connect to a web application from a native application, as long as you have the proper signaling server to connect the peers! Don't worry, we also have a demo for WEB, and with the proper signaling server you can even connect to/from TWebRTCCommunication included in TMS WEB Core.

Interested?

Even if you don't have TMS FNC WebSocket, we have included precompiled demo applications for both server and client so you can see it live without dependencies! You can also check out our online documentation that goes into further detail about this new control!

Let us know what use cases you have in mind!

Tunde Keller