WebRTC Browsers n Stacks Implementation differences

Browsers and stack
implementation differences
Dr. Alex Gouaillard

Plan
• global architectures of browsers and stacks
• -- chrome / firefox / webkit
• -- webrtc stand alone, webrtc in chrome,
webrtc in firefox
• - how to trace the different browsers without
recompiling

Chrome Variation
• Wait .. What? Chrome does not use libwebrtc?
Why?

Chrome Variation
• Wait .. What? Chrome does not use libwebrtc?
Why?
– Chrome layering
– Sandbox and per tab security
– Echo canceling
– Special capturers (screen/window/tab)
– Capturer access security
– Pretty much the same applies to all browsers

Chrome Variations
• OK, Chrome does not use libwebrtc? Why
should I Care?
• If you use the webrtc lib in your project, and
expect the same results as chrome, be ready
for big surprises!
– No Screen/windows/tab sharing
– Obsolete mac capturer
– No HTTP PROXY support

Libwebrtc src file layout
• base/
• common_audio/
• common_video/
• libjingle/
• modules/
• p2p/
• sound/
• system_wrappers/
• tools/
• video/
• video_engine/
• voice_engine/
• Too many to list
• Basic signal processing (FFT, window, transforms, …)
• Frame buffer, I420, libyuv, …
• Session management
• See next slide
• ICE Transports: TURN, STUN, DTLS, ….
• Platform specific sound system wrappers
• Platform specific low level wrappers (threads, timer, …)
• Poach and weaponize!!
• Video encoder/decoder/rec and send stream
• Capturer, Channel, Encoder, Sync, REMB, Send
• Channel, DTMF, …

• audio_coding/
• audio_conference_mixer/
• audio_device/
• audio_processing/
• bitrate_controller/
• desktop_capture/
• media_file/
• pacing/
• remote_bitrate_estimator/
• rtp_rtcp/
• utility/
• video_capture/
• video_coding/
• video_processing/
• video_render/
• ACM2, NACK, NETQ, DTMF tone gen., ….
• mixer
• Platform specific
• Noise, typing, gain detection, NS, Echo Cancellation, …
• Bandwidth estimation and bitrate controller
• Desktop capture (abstract only)
• Audio files reading/writing
• Paced sender
• Obvious
• RTP, RTCP, Payloads, FEC, …
• More Audio files utilities, rtp_dump, recorder, player
• Generic and platform dependent capturers (OLD)
• Codecs, encoders and decoders, send and rec, jitter, …
• Video processing and image enhancement
• Generic and platform dependent renderers (OLD)

• Why do I care?
– 1. you might need to go in the code
– 2. you need to know the layout for debugging:
chrome.exe --enable-logging --
vmodule=*/webrtc/*=2,*/libjingle/*=2,*=-2 --no-
sandbox

LibWEBRTC DEPS
• binutils
• Boringssl
• Colorama
• Drmemory
• Expat
• Icu
• instrumented_libraries
• Jsoncpp
• Libjpeg
• libjpeg_turbo
• Libsrtp
• libudev
• libvpx
• Libyuv
• llvm-build
• Nss
• Ocmock
• openmax_dl
• Opus
• Protobuf
• Sqlite
• Syzygy
• Usrsctp
• Yasm
• zlib

Chrome Specific Webrtc Bits
• //src/content/renderer/media/
• //src/third_party/WebKit/public/platform/
• //src/third_party/WebKit/Source/modules/mediastream
• //src/content/browser/media/
• //src/content/browser/media/capture/
• //src/media/capture/video/
• //src/content/browser/renderer_host/media/
• .
• .
• .
• .
• .
• .
• .

// MediaStreamManager is used to open/enumerate media capture devices (video
// supported now). Call flow:
// 1. GenerateStream is called when a render process wants to use a capture
// device.
// 2. MediaStreamManager will ask MediaStreamUIController for permission to
// use devices and for which device to use.
// 3. MediaStreamManager will request the corresponding media device manager(s)
// to enumerate available devices. The result will be given to
// MediaStreamUIController.
// 4. MediaStreamUIController will, by posting the request to UI, let the
// users to select which devices to use and send callback to
// MediaStreamManager with the result.
// 5. MediaStreamManager will call the proper media device manager to open the
// device and let the MediaStreamRequester know it has been done.

Chrome GUM example
capturers / frames / Encoding / ….

Media Stream Manager
(singleton@browser)GUM JS API
(tab/sandbox)
1. Send request

(tab/sandbox)
2. Check if MST is already available

(tab/sandbox)
Security Manager
(source, origin)
3. Check rights
2. Check if MST is already available - NO

(singleton@browser)
Audio
Capturer
Video
Capturer
Security Manager
(source, origin)
GUM JS API
(tab/sandbox)
4. Ask Corresponding capturer
type to start capturing
3. Check rights - OK

(singleton@browser)
Audio
Capturer
Video
Capturer
Security Manager
(source, origin)
A
GUM JS API
(tab/sandbox)
4. Ask Corresponding capturer
type to create one - OK
V
5. Store the MST

(singleton@browser)
Audio
Capturer
Video
Capturer
Security Manager
(source, origin)
A
GUM JS API
(tab/sandbox)
V
6. Trigger callback
Keep feeding frames

(singleton@browser)
Audio
Capturer
Video
Capturer
Security Manager
(source, origin)
A
GUM JS API
(tab/sandbox)
V
NOTE 1: second call for same device with same constraints will directly return the
MST, that allows to share streams across tabs without blocking

(singleton@browser)
Audio
Capturer
Video
Capturer
Security Manager
(source, origin)
A
GUM JS API
(tab/sandbox)
V
NOTE 2: Recently, a second call for the same device but with different constraints
(think simulcast) will indeed return a different resolution. Before it would return the
first resolution asked.

(singleton@browser)
Audio
Capturer
Video
Capturer
Security Manager
(source, origin)
A
GUM JS API
(tab/sandbox)
V
NOTE 3: Not only this allow to share cams across processes, it allows for global echo
cancellation (yes, including the key strokes). Before tabs could cross feed.

Chrome Screensharing 2 steps (1)
(singleton@browser)
Screen/Windows/Tab
Capturer
Security Manager
(source, origin)
Screensharing
(extension)
1
2
3
4
S
5

Chrome Screensharing 2 steps (2)
(singleton@browser)
Screen/Windows/Tab
Capturer
S
GUM JS API
(tab/sandbox)
With ID

Firefox Variation – Desktop 1process

FireFox Arch
• PeerConnection
– PeerConnection.js -- shim translation layer to let us do API adaptation to the C++
– PeerConnectionImpl -- C++ implementation of the PeerConnection interface.
– SIPCC -- handles SDP and media negotiation. Provided by Cisco but not a downstream.
• Media
– Webrtc.org/GIPS -- handles media encoding and decoding. Downstream from Google.
– MediaConduit -- Generic wrapper around Webrtc.org
– MediaPipeline -- Wrapper to hold the MediaConduit, mtransport subsystem, and the SRTP
contexts, as well as interface with MediaStreams.
• Transport
– mtransport -- generic transport subsystem with implementations for ICE, DTLS, etc.
– NSS -- new DTLS stack. Mentioned because we need to land the new version of NSS
– nICEr -- ICE stack; downstream from reSIProcate project
– nrappkit --portable runtime, utility library; downstream from nrappkit.sourceforge.net
• DataChannel
– DataChannel implementation in the DOM
– libsctp -- SCTP implementation; downstream from the BSD SCTP guys

Webkit (safari ?)
Current work:
- Implement datachannel for webkit (yin liu)
- Implement webrtc.org as a back end for
webkit (adam tiamou)
- Implement windows support for OWR

WebRTC Browsers n Stacks Implementation differences

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to WebRTC Browsers n Stacks Implementation differences

Similar to WebRTC Browsers n Stacks Implementation differences (20)

More from Alexandre Gouaillard

More from Alexandre Gouaillard (17)

Recently uploaded

Recently uploaded (20)

WebRTC Browsers n Stacks Implementation differences

Editor's Notes