Professional Documents
Culture Documents
Inspiring WebRTC
Tsahi Levent-Levi
tsahil@bloggeek.me
Tsahi Levent-Levi
Table of Contents
Introduction........................................................................................................................................................... 2
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Tsahi Levent-Levi
Introduction
I have been running interviews with entrepreneurs making use of WebRTC since 2012.
There are already over 60 WebRTC interviews on BlogGeek.me and growing.
I decided to curate 10 inspiring interviews out of that collection and provide them as a
single ebook something you can read in long form format. The idea? Get your juices
going around the endless possibilities open to you with WebRTC.
So find a comfortable place to sit down and pick the interviews that most interest you
Ive enjoyed all of them!
Google -
TokBox -
Talko -
LiveNinja -
Togetheroo -
Jocly -
Jitsi -
CallStats.io -
Peer5 -
10
Lantern -
Tsahi Levent-Levi
What is the most compelling service or demo youve seen done on top of
WebRTC? Something that didnt cross your mind when you started off with
WebRTC at Google.
All the games, augmented reality and the gesture tracking have really surprised the
whole team.
Where do you think WebRTC will have the most impact? Is it pure VoIP? video
conferencing? OTT players in the telco market? Someplace else?
I think the main impact will be to naturally bring in human conversation in normal every
day scenarios without having to go to a specific app. Think of it as being able to drop
in a real time voice / video conversation in any web app and make it look and feel as
natural as a text input field.
Tsahi Levent-Levi
What I see today is startup companies playing with and adopting WebRTC
wholeheartedly and on the other hand, the large players of the industry looking
into it companies like AT&T and Cisco. Where are all the mid-sized and even
large companies? Why arent they doing anything interesting with WebRTC?
WebRTC can impact all sizes of companies and were excited to see companies like
Voxeo a great example of a medium company executing with WebRTC projects.
Youve taken a strategy of desktop first with WebRTC, where in todays world the
tendency is to go mobile first. Why is that?
One reason is that the WebRTC API has undergone several changes at the W3C and
IETF. We decided to use the desktop as an implementation reference to iterate quickly
in development of the standard. Mobile possibilities for WebRTC are incredible!
I understood that Google Talk, Google Voice and Google+ Hangout are planned to
be converged into a single service or platform if you will. I assume that they will
be using WebRTC for that. How are you going to solve the scalable video coding
issue for Hangout while that happens?
WebRTC is still a very new platform it has not fully launched :). Our goal is to create
an open web platform that has a quality standard so high that any system will be able to
adopt it without compromise. First step: launching the basic platform and getting it solid
and stable.
Anything else youd like my readers to be aware of? Other resources that people
interested in WebRTC should be following?
+webrtc on google+ is a good resource. Our discuss list on Google groups as well and
the IETF WebRTC mailing list. All linked from webrtc.org homepage.
Tsahi Levent-Levi
Tsahi Levent-Levi
You are part of Telefnica now. Has that changed anything for you guys (besides
the financial backing)?
Telefnica has been establishing a good track record of acquiring innovative technology
companies while giving them the freedom and room they need to flourish. At a
technical level, weve been able to take advantage of some of Telefnicas in-depth
understanding of networks, real-time QOS and mobile connectivity. But more
importantly, having access to Telefnicas global infrastructure and scale has further
reinforced our ability to provide enterprise-grade service and has accelerated our ability
to innovate around the WebRTC standard. As more enterprises are embracing video
communications as part of their business and service offering, particularly around
WebRTC, this infrastructure, scale and speed is increasingly critical.
I am interested to know what value does Telefnica see in WebRTC and a video
API platform?
Telefnica Digital (the innovation arm of Telefnica) has a mission to put digital at the
core of the company. WebRTC, and video generally, are becoming a major part of the
digital communications world, so it makes sense for Telefnica Digital to have a major
beachhead in the future of digital communications, and to want to incorporate OpenTokbased applications into broader areas of the business. Telefnica has really embraced
our level of innovation and the Silicon Valley thinking we bring to the business.
You started off as a Flash based API company. What made you switch your focus
to WebRTC?
From the very beginning, we designed OpenTok to be protocol-neutral, because we
believed that a then-undefined browser standard would eventually supersede
Flash. Now that WebRTC is here, it has a wide range of advantages over Flash,
starting with increased audio and video quality. For more than a year, we have focused
almost all of our energy on the new standard a focus that is now being rewarded with
the expanding distribution and access to WebRTC. With WebRTC now natively part of
Chrome and Firefox, and our bridging capability for other browsers as well as mobile
devices, it is now a globally viable solution.
Tsahi Levent-Levi
Tsahi Levent-Levi
Can you tell me a bit about the technologies you selected for your back end?
To make OpenTok an enterprise-grade, WebRTC-native service at the browser level,
and in the cloud most of what we need to do is pretty bleeding-edge. The core of our
cloud is based on three scalable WebRTC-native components: Mantis, Rumor and
Symphony. We brought Mantis out into the open a couple of months ago. Mantis is our
cloud-based WebRTC media scaler, which routes, scales and shapes WebRTC traffic,
and was custom-developed in-house. You canread more about Mantis design on our
bog.
In addition to Rumor (discussed more above), we also have a custom-built WebRTC
session coordination server called Symphony which coordinates our distributed
WebRTC cloud. We havent talked about Symphony before, so I guess your Q&A is its
coming-out party! Internally, we use WebRTC compliant protocols such as RTP/RTCP
for media and control. We distribute Mantis, Rumor and Symphony services in data
centers around the globe, and operate it as one cloud.
You started off as a pure video API company, and now it seems that with your
new offerings you are trying to provide a more complete solution. Any reason for
that shift?
We are focused on providing all the pieces that a developer requires to build and
manage the video communications component of a commercial application. Weve had
APIs, widgets, plug-ins, SDKs. Whatever makes life faster and simpler for a developer
is what well provide.
When you added WebRTC into your existing platform what challenges did you
face?
On the web side, we wanted our clients to be fully HTML5 compliant. This entailed
adapting our signaling infrastructure to run over web-sockets so it could be as fast and
highly performing as possible. In addition, we also wanted to interoperate with endpoints
that had no native WebRTC support such as iOS devices, so that required building
custom SDK libraries for those devices. Last, the WebRTC standard doesnt include the
end-to-end functionality needed to deliver real-world use cases. We built out a powerful
cloud WebRTC infrastructure that provides value added services such as multi-party,
intelligent adaptation to varying network conditions and enables scale.
Tsahi Levent-Levi
Tsahi Levent-Levi
10
Tsahi Levent-Levi
11
Talk LIVE or not: Talk when and how you want, LIVE or not, so you maximize
team productivity and minimize disruption. If theres a team call and someone
cant be there, no problem they can listen and reply to the whole conversation,
or just the important moments, when they free up.
Show while you talk: Tap the camera as you speak and others will instantly see
what you see at the moment. Theres no disruption to the flow of the
conversation.
Any network, no network: Mobile networks can be spotty its sad but true.
Whether online (WiFi or cell) or offline in Talko, it doesnt matter. Keep talking
and it will just go when your device reconnects. No need to ever Try Again
later.
Find, replay and share: Calls neednt require tedious note-taking and email
follow-up to get everyone on the same page. Recorded voice in Talko lets you
organize, find, replay and share important conversations and key moments using
flags, tags and bookmarks.
Tsahi Levent-Levi
What was the reason you chose to use WebRTC in a mobile app?
We decided to bet on WebRTC in early 2012, just as we were beginning product
development. While the answer at that time was far less obvious than it would be now,
there were several factors that played into our decision at that time and that have
proven to be tremendously beneficial:
Open source: We were able to stand on the shoulders of some amazing work
that had been done by those in the open source community. When youre a small
team like us, its necessary to be really thoughtful about where you need to build
unique, differentiating technology and where it makes obvious sense to work with
the community. Many companies have open-sourced their WebRTC
implementation at Talko, we forked Googles.
VoIP: WebRTC is built upon the latest and greatest in VoIP and real-time media,
which is crucial for Talko and the scenarios we enable, including allowing users
to communicate both LIVE and not LIVE, with seamless transitions between the
two.
12
Tsahi Levent-Levi
The Opus audio codec is a critical piece to delivering the high quality audio experience
that our users have come to expect in Talko. It allows us to send audio at a very high
bitrate and sample rate without consuming all of a users available bandwidth, and its
FEC algorithms allow us to seamlessly conceal the inevitable packet losses, which will
occur on any mobile Internet connection.
13
Tsahi Levent-Levi
If you had one piece of advice for those thinking of adopting WebRTC, what
would it be?
Theres a good amount of detail on this in our recent blog post, but for those already in
the VoIP/live media space we think that adopting WebRTC is a no-brainer. It represents
not only the latest technology, but also the greatest technology. It is a true best-in-class
VoIP platform, and it is free. Rather than managing your own VoIP stack, you can rely
on WebRTC and focus on building out your application logic.
14
Tsahi Levent-Levi
15
Tsahi Levent-Levi
You have gone off and decided to build Katana from the ground up. Why was
that?
At LiveNinja, we pride ourselves in thinking outside the box. The question is never can
we do this?, but rather how can we do this?. With Katana, we have a very long list of
features listed in our pipeline. We had to be sure we would be able to get through this
list (and account for any other features that came up), without having to rely on any
external dependencies. The only way to truly do that is to build it yourself!
16
Tsahi Levent-Levi
If you had one piece of advice for those thinking of adopting WebRTC, what
would it be?
Dont just build another video chat application. Building a video chat application with
WebRTC is the equivalent of a Hello World app in my eyes. Think of what it is you
want to accomplish and how WebRTC can help you get there. What are you trying to
solve? What kind of experience are you looking to create for your potential users? Let
WebRTC be the stepping-stone that helps you build out your idea. WebRTC is only a
part of the big picture, not what you are selling.
17
Tsahi Levent-Levi
18
Tsahi Levent-Levi
You started from the desktop. When will we see a mobile version?
Adaptablox is being built with an emphasis on accessibility. It is our goal to enable
families to safely connect in a shared virtual space without any friction or specific
hardware requirements. As such we intend to continue developing the core of our
product for the browser while iterating on a native mobile/tablet extension that is
interoperable with that experience.
19
Tsahi Levent-Levi
If you had one piece of advice for those thinking of adopting WebRTC, what
would it be?
When developing your MVP it makes sense to leverage services like TokBox to
experiment with WebRTC. In general, we believe that the path of least resistance is the
best approach to building new products. If we believed that OpenTok couldnt provide
the stability or features that we needed we might consider another solution but so far we
have been pleased with their service.
20
Tsahi Levent-Levi
21
Tsahi Levent-Levi
I dont know about you, but people always say how great
WebRTC is for the gaming industry. And yet, while we have
a gazillion vendors doing plain-old-video-conferencing,
there are no gaming vendors out there. Or at least there
werent.
Jocly is the first vendor to go WebRTC as far as I know,
while the rest are just show cases and demos of what can
be done with WebRTC for games.
Once I found Jocly, I had to have a chat with its CEO,
Michel Gutierrez. Heres what he had to say about Jocly, gaming and WebRTC.
What is Jocly all about?
Jocly is a board-gaming platform running in pure HTML5. The idea is to provide a
number of generic features that apply to a large number of games. This includes a
common artificial intelligence for playing against the computer, support for live and turnbased games between actual people, saving and loading games for analysis, a
developer API for modifying existing games or creating new ones, and many other cool
things.
One of the most remarkable features of Jocly is the use of the WebGL HTML5
technology. This allows viewing and moving around the board in 3D as if you had the
real game craft in front of you. This results in a much better immersion. Most popular
games and all the new ones we develop support the 3D view, but of course all games
are also available in 2D for browsers or devices that do not have WebGL capabilities.
The fact we use only standard Web technologies allows Jocly to run almost anywhere,
desktop, tablets and mobiles.
22
Tsahi Levent-Levi
23
Tsahi Levent-Levi
24
Tsahi Levent-Levi
If you had one piece of advice for those thinking of adopting WebRTC, what
would it be?
If you have ideas, its now! Tomorrow will be too late to play in this arena unless you are
a big one.
If, as a developer, you hesitate putting the technology in place, give it a try, its certainly
much easier than you think.
25
Tsahi Levent-Levi
Now please excuse me I need to go play a game or two see if you can find me
online on Jocly.
26
Tsahi Levent-Levi
27
Tsahi Levent-Levi
An interesting note here is that Jitsi Videobridge is not a service. It is an open source
(LGPL) server application that you can download and run in your own data center /
cloud / premises.
28
Tsahi Levent-Levi
So, you started from a VoIP software client, and now offer a MCU of sorts. Why?
Great question! Often, while working on the client, we were frustrated by the lack of
advanced video features in existing media servers. The thing is that Jitsi could already
host conferences even as a client but doing it on an end-users laptop is rarely a good
thing. You run into all of these bandwidth and reliability issues.
So, we thought, why dont we push all this to the network?
We did and this is how Jitsi Videobridge was born. We did our first release of the bridge
in January 2013. Then, we started adding WebRTC compatibility.
Luckily our friend Tim Panton and the Bouncy Castle team had already taken care of a
DTLS implementation for Java and we used that. We also added ICE support through
our ice4j lib. Then Philipp Hancke came along and started playing with all this and
eventually came up with what later became JitMeet.
Exciting times!
Being an open source vendor how do you make a living out of Jitsi?
We sell development. Weve been doing this for quite a while now. We
founded BlueJimp, the company I work for, in 2009 and we started this model with Jitsi.
It would always go the same way: wed come up with a tech and then people will come
by asking us to either extend it or adapt it to their infrastructure.
Right now it seems that the same is also working out pretty nicely for Jitsi Videobridge.
People have different ideas and different requirements for their services. Open source is
great in that it gives you a head start for free, but sometimes youd turn out to be
missing last 5% of functionality that you need. Thats where we come in with BlueJimp.
We are always considering other options of course, we like changing and evolving, so
far however, this model is working out just fine.
29
Tsahi Levent-Levi
30
Tsahi Levent-Levi
This doesnt mean that we consider XMPP the right choice for every situation. We dont
and I dont believe anyone does. Signalling protocols are just a tool. They are solving a
bunch of problems, so if you are on a path that would put you against such problems it
only makes sense not to reinvent the wheel. Also its not a decision that is taken once
and for all. You can always change if things dont work out. Still, it is much easier to
begin with a tool and then rebuild it your way based on experience, than it is to start
designing from scratch while you dont even know where you are headed.
31
Tsahi Levent-Levi
If you had one piece of advice for those thinking of adopting WebRTC, what
would it be?
Well that would depend on who they are. I definitely wouldnt want a surgeon to
operate on me through a WebRTC app.
To all those who are thinking of adding non-critical RTC features to applications: this
should really be the first thing you look at.
Oh and of course: use trickle ICE!
32
Tsahi Levent-Levi
33
Tsahi Levent-Levi
You went for a hosted monitoring solution. How did the idea come up?
In the last 12 months, while building WebRTC applications, we realized that one of the
challenges apart from building complex features is to measure the performance of the
video calls (or lack of measuring it). At the time, there was no StatsAPI (or webrtcinternals://) and we built a system outside the browser to monitor the performance of the
media flow(s), today we offer this as a component to measure performance of media
flows at a middle-box (media gateway, TURN server or conference bridge).
Meanwhile, many WebRTC developers and service providers faced similar performance
issues and they contacted us. The issues were mainly at two levels: service-level and
endpoint/users-level. Diagnosing at the endpoint or user level can be done locally;
however, the service level needs to be done at a centralized point (in the cloud). By
34
Tsahi Levent-Levi
offering this service as a hosted solution, we take the pain of performance and
diagnosis away from the application developers so that they can focus on building the
product or complex call features. The callstats.io dashboard currently displays service
level engagement, network metrics and user experience, and the possibility to drill down
to diagnose issues with individual users.
Can you share a bit more about the architecture you use for getting the data from
WebRTC clients into your backend?
The communication between an endpoint and the backend is over HTTP/Web sockets.
We provide two components: a client library for WebRTC endpoints and a connection
monitor for middle-boxes. We currently provide callstats.js (a javascript client library)
that communicates with our backend, gathers metrics and exposes an API to send user
events (muting, hangup, etc.). The callstats.js will be accessible at the beginning of
2014.
The backend collects per flow statistics and puts them through an aggregation and
summarization process to produce conference-level statistics (useful for multiparty calls)
and service level statistics. Additionally, the developers can input their application
constraints and let callstats.io derive the best set of optional constraints for each user
based on past experience. In this way, we provide a mechanism for a consistent user
experience for each user.
The backend also produces a dashboard for each customer, from which they can gauge
the performance of their service at various levels. In early-2014, we will release our innetwork monitoring module (as a binary module); it will provide an additional
measurement point besides the endpoints and help with diagnosing potential
performance issues.
35
Tsahi Levent-Levi
If you had one piece of advice for those thinking of adopting WebRTC, what
would it be?
Developers building yet another SDK or video calling application should look at their
competitors and think hard about what or where they can provide value. There are
enough MCUs, SDKs and various video-calling services already out there. Therefore,
developers should try and integrate WebRTC into the flow of their web application, i.e.,
the video call should not necessarily start and end by pressing a button, but the
participants in the call should be provided adequate context, prior and after the call to
make the user experience frictionless.
The first will be callstats.js, it will be released in the coming weeks, developers will be
able to integrate it into their WebRTC application and they will be able to access the
app- and media-level metrics from the dashboard. Complementing the JS will be the
API documentation to help with the integration. Weve had a busy fall and foresee a
hectic spring. Happy New Year!
36
Tsahi Levent-Levi
37
Tsahi Levent-Levi
You decided to focus on the data channel and in the domain of content delivery.
Why is that?
WebRTC has become really popular due to its main API which enables video (or
audio) streaming via P2P. This API is very useful in scenarios where you want to build a
1-to-1 video conference app. But WebRTC has a fantastic core which can actually
transfer any kind of data very efficiently. To fully leverage this core, RTCDataChannel
API was added. It is a more generic API which lets us send raw data, encoded in our
preference. This API is perfect for content delivery and we use it to deliver files, images,
audio, video and also dynamic real time application data.
38
Tsahi Levent-Levi
39
Tsahi Levent-Levi
10.
December 2014
WebRTC offers privacy that is unparalleled. There are many aspects to that. What
Lantern has decided to do with WebRTC to enable people to connect to the internet in
their own terms, and why they selected WebRTC speaks volumes.
For a downloadable app that just needs connectivity, why bother with WebRTC?
WebRTC offers a number of advantages. First, Lantern relies on peer-to-peer
connections to make ordinary users available from their home connections as
proxies/access points, and WebRTCs use of ICE to provide that connectivity across
heterogeneous networks is as good as any implementation out there. In fact, theres a
40
Tsahi Levent-Levi
distinct scarcity of good libraries devoted specifically to NAT and firewall traversal, and
the Lantern team has parceled out specifically that piece from the WebRTC code base
in a library called Natty, available at http://natty.io/.
Beyond simple connectivity, WebRTC also offers cover traffic. Because WebRTC is
integrated into the browser, ICE is now a common protocol seen on the Internet.
Lantern seeks to blend in with this traffic, forcing censors to block ICE if they want to
block Lantern. WebRTC provides what we call cover traffic for Lantern.
Your app. What other technologies do you wrap into it besides WebRTC?
Lantern also uses a pluggable transport architecture that currently includes FTEProxy
(https://fteproxy.org/)
and
domain
fronting
(described
in
more
detail
at https://trac.torproject.org/projects/tor/wiki/doc/meek).
FTEProxy
uses
regular
expressions to specify how traffic over the wire should look, allowing that traffic to
conform to whatever protocol we specify. That again allows Lantern to blend in with
existing protocols, theoretically forcing censors to block things they dont want to block
in order to block Lantern.
On the frontend we use AngularJS and D3.
41
Tsahi Levent-Levi
Why Go?
Were porting a lot to Go for a number of reasons:
1. Its got great concurrency primitives, and we do a lot of concurrency
2. Its got great build-in networking, and we do a lot of networking
3. Its super fast and easy to develop in
4. It has great cross-compilation and relatively small binaries
Were also moving away from Java because the download size is so large, which is
particularly problematic for users in Iran who dont have much bandwidth.
42
Tsahi Levent-Levi
43
Tsahi Levent-Levi
BlogGeek.Me Blog
My blog is the place where the bulk of my writing exists. It is available freely and can be
subscribed via both RSS and email. See https://bloggeek.me
Reports
This is my second report about WebRTC. To find out about other paid reports I have
written, see https://bloggeek.me/reports/
Consulting
I provide consulting services to vendors, especially around VoIP, video conferencing
and WebRTC. See https://bloggeek.me/consulting/
44