Mozilla ZAP Media Kit
=====================
Built-in media frame types:
===========================
I) audio/pcm
------------
PCM audio frame.
Interfaces: zapIMediaFrame
zapIMediaFrame::streamInfo :
- ACString "type" : = "audio/pcm"
- unsigned long "sample_rate" : sample rate in Hz
- unsigned long "samples" : number of samples in frame *per channel*
- unsigned long "channels" : number of channels
- ACString "sample_format" : "float32_1" | "float32_32768" | "int16" | "int32"
zapIMediaFrame::data :
"sample_format" == "float32_1" | "float32_32768" | "int32":
(max amplitude = +/-1 | +/- 32768 | +/- 2^31)
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| sample 0 channel 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| sample 0 channel 1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| sample 0 channel n-1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| sample 1 channel 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| sample 'samples-1' channel n-1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
"sample_format" == "int16":
As above but with 16bit samples.
----------------------------------------------------------------------
II) datagram
------------
Generic datagram frame for transmission over a network socket.
Interfaces: zapIMediaFrame,
nsIDatagram (see netwerk/base/public/nsIUDPSocket.idl)
zapIMediaFrame::streamInfo :
- ACString "type" : = "datagram"
zapIMediaFrame::timestamp : not used
zapIMediaFrame::data :
- same as nsIDatagram::data
----------------------------------------------------------------------
III) audio/speex
----------------
Speex encoded audio frame.
Interfaces: zapIMediaFrame
zapIMediaFrame::streamInfo :
- ACString "type" : = "audio/speex"
zapIMediaFrame::data :
- binary data as described in speex specs
----------------------------------------------------------------------
IV) rtp
-------
RTP packet.
Interfaces: zapIRTPFrame (spec of zapIMediaFrame)
zapIMediaFrame::streamInfo :
- ACString "type" : = "rtp"
zapIMediaFrame::timestamp : currently fixed to 0.
zapIMediaFrame::data :
- raw rtp packet data including header
----------------------------------------------------------------------
V) audio/pcmu
-------------
Mu-law encoded pcm audio frame.
Interfaces: zapIMediaFrame
zapIMediaFrame::streamInfo :
- ACString "type" : = "audio/pcmu"
zapIMediaFrame::data :
8 bit pcmu samples.
Framing is currently fixed to 8000Hz, 0.02s, 1 channel
----------------------------------------------------------------------
VI) audio/pcma
--------------
Alpha-law encoded pcm audio frame.
Interfaces: zapIMediaFrame
zapIMediaFrame::streamInfo :
- ACString "type" : = "audio/pcma"
zapIMediaFrame::data :
8 bit pcma samples.
Framing is currently fixed to 8000Hz, 0.02s, 1 channel
----------------------------------------------------------------------
VII) audio/tone
---------------
Telephony tone frame as described in RFC2833.
Interfaces: zapIAudioToneFrame (spec of zapIMediaFrame)
zapIMediaFrame::streamInfo :
- ACString "type" : = "audio/tone"
zapIMediaFrame::data :
- raw audio/tone payload as described in RFC2833 with implied
8000Hz sample rate.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| modulation |T| volume | duration |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0 0 0 0| frequency |0 0 0 0| frequency |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| .... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0 0 0 0| frequency |0 0 0 0| frequency |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
----------------------------------------------------------------------
VIII) audio/telephone-event
---------------------------
Telephone events (e.g. DTMF digits) as defined in RFC2833.
Interfaces: zapITelephoneEventFrame (spec of zapIMediaFrame)
zapIMediaFrame::streamInfo :
- ACString "type" : = "audio/telephone-event"
zapIMediaFrame::data :
- raw audio/telephone-event payload as described in RFC2833 with
implied 8000Hz sample rate.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| event |E|0| volume | duration |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
----------------------------------------------------------------------
IX) stun
--------
A udp datagram containing a RFC3489 STUN request or response.
Interfaces: zapIMediaFrame,
zapIStunMessage (see zap/netutils/src/zapIStunMessage.idl),
nsIDatagram (see netwerk/base/public/nsIUDPSocket.idl)
zapIMediaFrame::streamInfo :
- ACString "type" : = "stun"
zapIMediaFrame::timestamp : fixed to 0.
zapIMediaFrame::data :
- same as zapIStunMessage::serialize()
----------------------------------------------------------------------
X) raw binary data
------------------
Interfaces: zapIMediaFrame
zapIMediaFrame::streamInfo :
- ACString "type" : = "raw"
zapIMediaFrame::timestamp :
usage varies. see generating nodes for more detail.
zapIMediaFrame::data :
- raw binary data
----------------------------------------------------------------------
XI) clock
---------
A clock frame
Interfaces: zapIMediaFrame
zapIMediaFrame::streamInfo :
- ACString "type" := "clock"
- double "clock_cycle" := 1/clock rate in seconds
zapIMediaFrame::timestamp incremented in ms.
zapIMediaFrame::data : empty
======================================================================
======================================================================
Built-in node types:
====================
----------------------------------------------------------------------
1) audioin
----------
Audio source.
Sinks: none
Sources: 1 (active)
Control interfaces: zapIAudioIn
Node parameters:
- zapIAudioDevice "device" (default: default audio in)
- int "buffers" : number of buffer mediating between zmk and device
(default: 2, must be >=2)
- unsigned long "sample_rate" : sample rate in Hz (default: 8000)
- unsigned long "samples" : number of samples in frame *per channel* (default: 160)
- unsigned long "channels" : number of channels (default: 1)
- ACString "sample_format" : "float32_1" | "float32_32768" | "int16" | "int32"
(default: "float32_32768")
Output stream:
audio/pcm frames with
- unsigned long "sample_rate" == corresponding node parameter
- unsigned long "samples" == corresponding node parameter
- unsigned long "channels" == corresponding node parameter
- unsigned long "sample_format" == corresponding node parameter
----------------------------------------------------------------------
2) audioout
-----------
Audio sink.
Sinks: 1 (active)
Sources: 1 (active)
- Clock output that receives clock frames at the sampling rate
Control interfaces: zapIAudioOut
Node parameters:
- zapIAudioDevice "device" (default: default audio out)
- int "buffers" : number of buffer mediating between zmk and device
(default: 4, must be >=2)
- unsigned long "sample_rate" : sample rate in Hz (default: 8000)
- unsigned long "samples" : number of samples in frame *per channel* (default: 160)
- unsigned long "channels" : number of channels (default: 1)
- ACString "sample_format" : "float32_1" | "float32_32768" | "int16" | "int32"
(default: "float32_32768")
Input stream:
audio/pcm frames with
- unsigned long "sample_rate" == corresponding node parameter
- unsigned long "samples" == corresponding node parameter
- unsigned long "channels" == corresponding node parameter
- unsigned long "sample_format" == corresponding node parameter
Output stream:
clock frames with
- double "clock_cycle" : duration of one clock cycle in seconds.
- zapIMediaFrame::timestamp incremented in ms.
----------------------------------------------------------------------
3) audio-mixer
--------------
Audio mixer node, mixes n audio sinks to one audio source. Initial
master volume will be 0dB and the mixer will initially be unmuted.
This is a locking node. Trying to modify the mediagraph while this
node is processing a frame will throw an exception.
Timestamps increase monotonously in sample clock units for every frame
requested. Streambreaks from input streams will not be propagated. If
no input streams are connected, silence frames will be generated.
Control interfaces: zapIAudioMixer
Sinks: n (active)
Sources: 1 (passive)
Node parameters:
- unsigned long "sample_rate" : sample rate in Hz (default: 8000)
- unsigned long "samples" : number of samples in frame *per channel* (default: 160)
- unsigned long "channels" : number of channels (default: 1)
- ACString "sample_format" : "float32_32768" | "float32_1"
(default: "float32_32768")
Input streams:
audio/pcm frames with
- unsigned long "sample_rate" == corresponding node parameter
- unsigned long "samples" == corresponding node parameter
- unsigned long "channels" == corresponding node parameter
- unsigned long "sample_format" == corresponding node parameter
Output stream:
audio/pcm frames with
- unsigned long "sample_rate" == corresponding node parameter
- unsigned long "samples" == corresponding node parameter
- unsigned long "channels" == corresponding node parameter
- unsigned long "sample_format" == corresponding node parameter
----------------------------------------------------------------------
4) buffer
---------
General purpose packet buffer.
Sinks: 1 (passive)
Sources: 1 (passive)
Node parameters:
- unsigned long "max_size" : maximum number of packets to buffer (10)
Must be >= 1.
- unsigned long "min_size" : minimum number of packets that need to be in
the buffer before a packet can be taken out (1).
Must be >= 1.
- unsigned long "lift_count" : whenever the size of the buffer drops to 0,
lift_count packets will be pre-buffered before
the attached sink is allowed to take any packets
out of the buffer (0). A lift_count of 0 has the
same effect as a lift_count of 1.
- unsigned int "drop_count" : number of packets to drop at the front of the buffer
(older packets) when a new packet arrives for a
buffer that has reached its maximum size (0).
If this is zero, no new packet will be added to
a buffer that has reached its maximum size,
i.e. newly arriving packets will be dropped.
Control interfaces: zapIPacketBuffer
Input stream:
any
Output stream:
any
----------------------------------------------------------------------
5) udp-socket
-------------
A UDP socket for sending and receiving datagrams.
Control interfaces:
- zapIUDPSocket
Sinks: 1 (passive)
Sources: 1 (active)
Node parameters:
- nsIUDPSocket "socket" : An initialized udp socket (nsIUDPSocket).
The socket will be closed on removal from
the graph.
- unsigned short "port" : local port of udp socket (default: 0)
- unsigned short "portbase" : bind to first available port >= portbase.
- (XXX ACString "address" currently we bind to 0.0.0.0)
If both "socket" and "port" or "portbase" are given, "port" and
"portbase" will be ignored.
If both "port" and "portbase" are given, "portbase" will be ignored.
If neither "port", "portbase" or "socket" are given, the behaviour
will be as if a "port"==0 had been specified (allocate a random free
port).
Input stream:
datagram frames
Output stream:
datagram frames
----------------------------------------------------------------------
6) udp-socket-pair
------------------
A UDP socket pair (for rtp/rtcp). port_a and port_b will be chosen
such that portbase <= port_a, port_a = 2*n, and port_b = 2*n+1
Control interfaces:
- zapIUDPSocketPair
Sinks: 2
- ACString "name" == "socket-a" (passive)
- ACString "name" == "socket-b" (passive)
Sources: 2
- ACString "name" == "socket-a" (active)
- ACString "name" == "socket-b" (active)
Node parameters:
- unsigned short "portbase" : port at which to start searching for a
free pair (default: 49152)
Input streams:
- as in udp-socket
Output streams:
- as in udp-socket
----------------------------------------------------------------------
7) speex-encoder
----------------
Filter for encoding audio streams into speex streams.
Control interfaces: zapISpeexEncoder
Node parameters:
- unsigned long "sample_rate" : 8000|16000|32000 (default: 8000)
Input stream:
audio/pcm frames with
- unsigned long "sample_rate" == 8000, 16000, or 32000; must match node parameter
- unsigned long "samples" : number of samples in frame *per channel* == 160
- unsigned long "channels" == 1
- unsigned long "sample_format" == "float32_32768"
Output stream:
audio/speex frames
----------------------------------------------------------------------
8) speex-decoder
----------------
Filter for decoding speex streams as encoded using speex-encoder
Control interfaces: zapISpeexDecoder
Node parameters:
-unsigned long "sample_rate" : 8000|16000|32000 (default:8000)
Input stream:
audio/speex frames
Output stream:
audio/pcm frames with
- unsigned long "sample_rate" == 8000, 16000, or 32000; matches node parameter
- unsigned long "samples" == 160
- unsigned long "channels" == 1
- unsigned long "sample_format" == "float32_32768"
----------------------------------------------------------------------
9) speex-rtp-packetizer
-----------------------
Filter for converting speex streams into rtp streams.
Node parameters:
- unsigned short "payload_type" (default: 96)
Input stream:
audio/speex frames
Output stream:
rtp frames (SSRC and sequence number NOT set).
----------------------------------------------------------------------
10) speex-rtp-depacketizer
--------------------------
Filter for converting rtp streams into speex streams.
Input stream:
rtp frames (with assumed speex payload)
Output stream:
audio/speex frames
----------------------------------------------------------------------
11) g711-encoder
----------------
Filter for encoding audio streams into g.711 (pcmu or pcma) streams.
Control interfaces:
Node parameters:
- ACString "type" == "audio/pcmu" | "audio/pcma" (default: "audio/pcmu")
Input stream:
audio/pcm frames with
- unsigned long "sample_rate" == 8000
- unsigned long "samples" == 160
- unsigned long "channels" == 1
- unsigned long "sample_format" == "float32_32768"
Output frames:
audio/pcmu or audio/pcma frames; determined by node parameter
----------------------------------------------------------------------
12) g711-decoder
----------------
Filter for decoding g.711 streams as encoded using g711-encoder
Control interfaces:
Node parameters:
Input stream:
audio/pcmu or audio/pcma frames
Output stream:
audio/pcm frames with
- unsigned long "sample_rate" == 8000
- unsigned long "samples" == 160
- unsigned long "channels" == 1
- unsigned long "sample_format" == "float32_32768"
----------------------------------------------------------------------
13) g711-rtp-packetizer
-----------------------
Filter for converting g.711 streams into rtp streams.
Node parameters:
- unsigned short "payload_type" (default: 0)
Input stream:
audio/pcmu or audio/pcma frames
Output stream:
rtp frames (SSRC and sequence number NOT set).
----------------------------------------------------------------------
14) g711-rtp-depacketizer
-------------------------
Filter for converting rtp streams into g.711 streams.
Node parameters:
- ACString "type" == "audio/pcmu" | "audio/pcma" (default: "audio/pcmu")
Input stream:
rtp frames with assumed g711 payload
Output stream:
audio/pcmu or audio/pcma frames (determined by node parameter)
----------------------------------------------------------------------
15) rtp-session
---------------
Maintains a unicast RTP session.
RTP-session sets the SSRC and sequence number of outbound rtp frames
and generates/processes RTCP.
Node parameters:
(XXX - unsigned long "SSRC" )
Sinks: 3
ACString "name" == "local2remote-rtp" : outbound rtp frames (neutral)
ACString "name" == "remote2local-rtp" : inbound rtp frames (neutral)
ACString "name" == "remote2local-rtcp" : inbound rtcp frames (neutral)
Sources: 3
ACString "name" == "remote2local-rtp" : inbound rtp frames (neutral)
ACString "name" == "local2remote-rtp" : outbound rtp frames (neutral)
ACString "name" == "local2remote-rtcp" : outbound rtcp frames (neutral)
Input streams:
"local-rtp": rtp frames w/o ssrc or seq no.
...XXX
Output streams:
"local2remote-rtp": rtp frames with ssrc and seq number set.
----------------------------------------------------------------------
16) rtp-transmitter
-------------------
Filter for transmitting RTP packets over UDP.
Node parameters:
- ACString "address" : destination address for datagrams
- unsigned short "port" : destination port for datagrams
Input stream:
rtp frames.
Output stream:
datagram frames.
----------------------------------------------------------------------
17) rtp-receiver
----------------
Filter for receiving RTP datagrams.
Input stream:
datagram frames
Output stream:
rtp frames
----------------------------------------------------------------------
18) splitter
------------
Sends one input to several outputs.
XXX The input frames are not cloned! Unexpected things might happen if
a mutable stream (like e.g. rtp) is being split!
Sinks: 1 (passive)
Sources: n (active)
Node parameters:
Input stream:
any
Output stream:
any
----------------------------------------------------------------------
19) rtttl-player
----------------
Audio source for playing RTTTL-encoded ringtones.
RTTTL is the "Ringing Tones Text Transfer Language", a popular format
for Nokia mobile phone ringtones.
Sinks: none
Sources: 1 (passive)
Control interfaces:
Node parameters:
- ACString "rtttl" : The rtttl source code to be played
- boolean "loop" : true: the sample should be looped indefinitely (default: true)
- double "amplitude" : 0.0 <= amplitude <= 1.0 (default: 0.5)
- unsigned long "sample_rate" : sample rate in Hz (default: 8000)
- unsigned long "samples" : == 160
- unsigned long "channels" : number of channels (default: 1)
- ACString "sample_format" : "float32_1" | "float32_32768" | "int16" | "int32"
(default: "float32_32768")
Output stream:
audio/pcm frames (layout determined by source parameters).
----------------------------------------------------------------------
20) ttone->pcm
--------------
Decoder for generating pcm from audio/tone telephony tones (RFC2833).
Sinks: 1 (active)
Sources: 1 (passive)
Control interfaces:
Node parameters:
double "ztlp" : zero dBm0 tranmission level point as a fraction of the
maximum output stream amplitude (Default: 1.0==maximum amplitude
of output stream)
unsigned long "sample_rate" : sample rate in Hz (default: 8000)
unsigned long "samples" : number of samples in frame *per channel* (default: 160)
unsigned long "channels" : number of channels (default: 1)
ACString "sample_format" : "float32_1" | "float32_32768" | "int16" | "int32"
(default: "float32_32768")
Input stream:
audio/tone frames
Output stream:
audio/pcm frames (layout determined by source parameters).
----------------------------------------------------------------------
21) dtmf-generator
------------------
Generates RFC2833 dtmf audio/telephone-event frames from user input.
Will generate a new stream based (timebase=0) for each new synchronous
'run' of digits.
Note: Doesn't generate end frame retransmissions. Frames will be played
without inter-frame pauses; the consumer has to reconstruct the timing from
the frame timestamps.
Sinks: none
Sources: 1 (passive)
Control interfaces: zapIDTMFGenerator
Node parameters:
- double "tone_duration" : duration of tones in s (0.09).
x <= (2^16-1)/sample_rate
- double "pause_duration" : duration of pauses in s (0.07).
x <= (2^16-1)/sample_rate
- double "volume" : volume in dBm0 (-8)
- unsigned long "sample_rate" : XXX 8000Hz hardcoded atm.
- double "max_samples" : maximum number of samples in one frame
seconds (0.02).
1/sample_rate <= x <= (2^16-1)/sample_rate
Output stream:
audio/telephone-event frames WITHOUT end frame retransmissions
----------------------------------------------------------------------
22) tevent->ttone
-----------------
Filter for generating audio/tone frames from audio/telephone-event
frames (RFC2833). Note: This is currently implemented as a passive
filter. One implication is that it might not deal gracefully with end
frame retransmissions that don't stick to the pipeline's packetization
period. Inter-frame pauses will not be converted to packets; the
consumer has to reconstruct the timing from the frame timestamps.
Sinks: 1 (neutral)
Sources: 1 (neutral)
Control interfaces:
Node parameters:
Input stream:
audio/telephone-event frames
Output stream:
audio/tone frames
----------------------------------------------------------------------
23) ttone-generator
-------------------
Generates RFC2833 telephony tone frames from user input
Sinks: none
Sources: 1 (passive)
Control interfaces: zapITToneGenerator
Node parameters:
- unsigned long "sample_rate" : XXX 8000Hz hardcoded atm.
- double "max_frame_duration" : minimum packetization interval in seconds
(default: 0.02s).
1/sample_rate <= x <= (2^16-1)/sample_rate
Output stream:
audio/tone frames
----------------------------------------------------------------------
24) pump
--------
General purpose packet pump. Pumps an unaltered "input" packet to the
output whenever a "clock" frame is received or when prodded through
zapIPacketPump. The timestamp of the frame is NOT modified. Frames
will only be pumped if an output sink is connected.
Sinks: 2
ACString "name" == "input" : pumped input frames; any stream type (active)
ACString "name" == "clock" : reference clock; any stream type (passive)
Sources: 1 (active)
Control interfaces: zapIPacketPump
Output stream:
same as "input" stream
----------------------------------------------------------------------
25) tevent-rtp-packetizer
-------------------------
Filter for converting telephone-event streams into rtp streams.
Node parameters:
- unsigned short "payload_type" (default: 101)
Input stream:
audio/telephone-event frames
Output stream:
rtp frames (SSRC and sequence number NOT set).
----------------------------------------------------------------------
26) stream-merger
-----------------
Sends unaltered packets from several input streams to the output
stream as soon as they arrive.
Sinks: n (passive)
Sources: 1 (active)
Control interfaces:
-
Node parameters:
Input streams:
any
Output stream:
as input streams
----------------------------------------------------------------------
27) timestamp-syncer
--------------------
Adds an offset to the timestamp of input frames, so that the timestamp
of the first frame of an input stream coincides with the last
timestamp seen at the 'timebase' input. Whenever the input stream or
timebase stream changes (i.e. when the streaminfo is different from one
frame to the next), the offset will be recalculated.
Note: Warning: this node mutates frames. XXX maybe we need cloning.
Sinks: 2
- ACString "name" == "input" : input stream (any type) (neutral)
- ACString "name" == "timebase" : timebase stream (any type) (passive)
Sources: 1
Output stream:
as input stream, with frame timestamps adjusted.
----------------------------------------------------------------------
28) rtp-demuxer
---------------
Dispatches input frames from one input to several outputs based on rtp
payload type number.
Sinks: 1 (passive)
Sources: n (active).
Source parameters: ACString "payload_type" == payload type of this
output (no default; parameter MUST be specified)
Input stream:
rtp frames
Output streams:
rtp frames; as input
----------------------------------------------------------------------
29) stream-switch
-----------------
Switches one input stream to one of n outputs.
Sinks: 1 (neutral)
Sources: n (neutral)
Source parameters:
ACString "id" : id by which a particular output can be selected with
zapISwitch::selectOutput(). Must be unique among all sources
Control interfaces: zapIStreamSwitch
Input stream:
any
Output stream:
as input stream
----------------------------------------------------------------------
30) stream-tap
--------------
Similar to a splitter, but one output is passive ("type"=="master")
and the other outputs active ("type"=="tap"), i.e. they receive a
packet whenever the master output requests one.
XXX The input frames are not cloned! Unexpected things might happen if
a mutable stream (like e.g. rtp) is being tapped!
This is a locking node. Trying to modify the mediagraph while this
node is processing a frame will throw an exception.
Sinks: 1 (active)
Sources: n (1 passive , n-1 active)
Source parameters:
passive master source: ACString "type" == "master"
active tap sources: ACString "type" == "tap"
Node parameters:
Input stream:
any
Output stream:
any
----------------------------------------------------------------------
31) speex-audio-processor
-------------------------
Speex acoustic echo canceller and audio preprocessor
Sinks: 2
ACString "name" == "input" (neutral): Audio as captured by the microphone
ACString "name" == "echo" (active): Audio played through speaker
Sources: 1 (neutral): Audio with echo removed
Node parameters:
- boolean "aec" : toggles acoustic echo cancellation (default: false)
- boolean "aec_2_stage" : toggles 2 stage AEC (default: false)
- float "aec_tail" : AEC tail in ms (default: 300)
- boolean "denoise" : toggles denoising (default: false)
- boolean "agc" : toggles automatic gain control (default: false)
- float "agc_level" : (default: 8000)
- boolean "vad" : toggles voice activity detection (default: false)
- boolean "dereverb" : toggles dereverb (default: false)
- float "dereverb_level" : (default: 0.2)
- float "dereverb_decay" : (default: 0.5)
Control interfaces: zapISpeexAudioProcessor
Input streams:
audio/pcm frames with
- unsigned long "sample_rate" == 8000
- unsigned long "samples" == 160
- unsigned long "channels" == 1
- unsigned long "sample_format" == "float32_32768"
Output stream:
audio/pcm frames (as input streams)
----------------------------------------------------------------------
32) audioio
-----------
Combined audio source/sink. Provides lower resource use and better
synchronization than having separate audioin and audioout. The latter
is important for things like echo cancellation, where the audio output
needs to be correlated with the input.
For each sampling period, the audioio node will call "consumeFrame" on
the attached sink before calling "produceFrame" on the attached
source.
Note: Under some circumstances the combined audioio can lead to choppy
audio (observed on Windows XP when other apps using the sound hardware
are active). In these cases using separate audioin/audioout might be
better.
Sinks: 1 (active)
Sources: 2 (active)
Source parameters:
ACString "name" == "ain" : audio in.
ACString "name" == "monitor" : receives every frame played on audioout,
including automatically generated silence
frames.
Control interfaces: zapIAudioIO
Node parameters:
- zapIAudioDevice "input_device" (default: default audio in)
- zapIAudioDevice "output_device" (default: default audio out)
- int "buffers" : number of buffers mediating between zmk and audio hardware
(default: 4, must be >=2)
- unsigned long "sample_rate" : sample rate in Hz (default: 8000)
- unsigned long "samples" : number of samples in frame *per channel* (default: 160)
- unsigned long "channels" : number of channels (default: 1)
- ACString "sample_format" : "float32_1" | "float32_32768" | "int16" | "int32"
(default: "float32_32768")
Input stream:
audio/pcm frames with
- unsigned long "sample_rate" == corresponding node parameter
- unsigned long "samples" == corresponding node parameter
- unsigned long "channels" == corresponding node parameter
- unsigned long "sample_format" == corresponding node parameter
Output stream:
audio/pcm frames with
- unsigned long "sample_rate" == corresponding node parameter
- unsigned long "samples" == corresponding node parameter
- unsigned long "channels" == corresponding node parameter
- unsigned long "sample_format" == corresponding node parameter
----------------------------------------------------------------------
33) stun-client
---------------
Sinks: 1 (passive)
Sources: 1 (active)
Control interfaces: zapIStunClient
Node parameters: -
Input stream: datagram frames | stun frames
Output stream: datagram frames
----------------------------------------------------------------------
34) stun-server
---------------
Sinks: 1 (passive)
Source: 1 (active)
Control interfaces: zapIStunServer
Node parameters:
- ACString "source_addr" : Source address to insert into binding responses.
(no default; mandatory)
- unsigned short "source_port" : Source port to insert into binding
responses. (no default; mandatory)
Input stream: datagram frames | stun frames
Output stream: datagram frames
----------------------------------------------------------------------
35) stun-demuxer
----------------
Dispatches input frames from one input to a stun server, stun client
or other output, depending on the input frame.
Sinks: 1 (passive)
Sources: 3 (active)
Source parameters:
ACString "name" == "stun-req" : stun requests
ACString "name" == "stun-res" : stun responses
ACString "name" == "other" : other udp traffic
Input stream:
udp frames
Output streams:
stun frames ("stun-req"/"stun-res")
udp frames ("other")
----------------------------------------------------------------------
36) filein
----------
Reads a file as a sequence of raw data packets.
Frames will contain between 1 and <= block_size bytes, apart from eof
frames which contain 0 bytes. When the file is looped or seeked, a new
streaminfo will be set. In addition, if 'generate_eof' is set to true,
an eof frame (with the previous streaminfo) will be emitted before
resuming at the new position.
Sinks: none
Sources: 1 (passive)
Control interfaces: zapIFileIn
Node parameters:
- AUTF8String "file_url" : url pointing to the file to read. Must resolve
to a local file. The scheme can be file:/ or
resource:/ (and possibly others).
- unsigned long "block_size" : max size of data packets (default: 8192).
- boolean "loop" : if true, the file will be looped (i.e. seeked to the
beginning once the end has been reached) (default: false)
- boolean "generate_eof" : If true, an empty frame will be generated when
the end of the file is reached or when
a new stream position is set using
'zapIFileIn::Seek'. (default: true)
Output stream:
- raw binary frames (type = "raw"). Size of frame data with be <=
block_size
- timestamps give the byte offset to the beginning of the file
----------------------------------------------------------------------
37) client-sink
---------------
Allows clients to pull media frames using zapIClientSink::pullFrame().
Sinks: 1 (active)
Source: none
Output stream:
as input stream
----------------------------------------------------------------------
38) null-sink
-------------
A passive sink that discards all frames.
Sinks: 1 (passive)
Source: none
Input stream:
any
----------------------------------------------------------------------
39) pcm-ping
------------
A filter emiting a 'pcm ping' for each frame received.
Sinks: 1 (passive)
Sources: 1 (active)
Node parameters:
- unsigned long "sample_rate" : sample rate in Hz (default: 8000)
- unsigned long "samples" : number of samples in frame *per channel* (default: 160)
- unsigned long "channels" : number of channels (default: 1)
- ACString "sample_format" : "float32_1" | "float32_32768" | "int16" | "int32"
(default: "float32_32768")
- double "ping_pitch" : frequency of the ping (in Hz, default: 1760Hz)
Input stream:
any
Output stream:
audio/pcm frames with
- unsigned long "sample_rate" == corresponding node parameter
- unsigned long "samples" == corresponding node parameter
- unsigned long "channels" == corresponding node parameter
- unsigned long "sample_format" == corresponding node parameter
----------------------------------------------------------------------
40) udp-packetizer
------------------
Filter for payloading generic frames onto a UDP frame.
Note that no size check will be made, i.e. the resulting datagram
frames might be larger than the MTU!
Node parameters:
- ACString "address" : destination address for datagrams (no default)
- unsigned short "port" : destination port for datagrams (no default)
- ACString "header" : a header that will be prepended to all datagrams
(default: "")
Input stream:
any frames.
Output stream:
datagram frames.
----------------------------------------------------------------------
41) data-extractor
------------------
Filter for extracting partial data from frames
Node parameters:
- unsigned long "start" : offset into the input frame's data at which to
start extracting. (default: 0)
Input stream:
any frames.
Output stream:
raw binary frames (type = "raw") with timestamps set to 0.
----------------------------------------------------------------------
42) audio-reformatter
---------------------
Node for reformatting (but not resampling!) audio streams.
Sinks: 1 (active)
Sources: 1 (passive)
Control interfaces: zapIAudioReformatter
Node parameters:
- unsigned long "sample_rate" : sample rate in Hz (default: 8000)
- unsigned long "samples" : number of samples in frame *per channel* (default: 160)
- unsigned long "channels" : number of channels (default: 1, must be <=2)
- ACString "sample_format" : "float32_32768" | "float32_1"
(default: "float32_32768")
Input streams:
audio/pcm frames with
- unsigned long "sample_rate" == corresponding node parameter
- unsigned long "channels" : must be <= channels set as node parameter
Output stream:
audio/pcm frames with
- unsigned long "sample_rate" == corresponding node parameter
- unsigned long "samples" == corresponding node parameter
- unsigned long "channels" == corresponding node parameter
- unsigned long "sample_format" == corresponding node parameter
If a streambreak results in a partially output frame, it will be
padded with silence. Any new stream will begin with the next frame.
----------------------------------------------------------------------
43) clock-reducer
-----------------
Node for filtering a fixed proportion of frame. Only every n'th frame
received at the input will be fed to the output, where n =
numerator/denominator.
Sinks: 1 (passive)
Sources: 1 (active)
Node parameters:
- long "numerator" : See description above (default: 1)
- long "denominator" : See description above (default: 1)
Input streams:
any
Output streams:
any
----------------------------------------------------------------------
44) timestamper
---------------
Filter for adding timestamps to frames.
Note: Warning: this node mutates frames. XXX maybe we need cloning.
Input frame n will be timestamped with
TS = offset + (incr_num*n)/incr_denom.
n is initially 0 and reset to 0 after each stream break.
Node parameters:
- unsigned long long "offset" : Timestamp of first frame in input
stream (default: 0)
- long "incr_num" : Increment numerator. See description
above. (default: 1)
- long "incr_denom" : Increment denominator. See description above
(default: 1)
Input stream:
- any with mutable timestamps
Output stream:
- as input stream, with timestamps set according to offset, incr_num,
incr_denom
----------------------------------------------------------------------
45) stream-syncer
-----------------
Synchronizes a stream to a clock stream.
Sinks: 2
- ACString "name" == "input" : input stream (any type) (active)
- ACString "name" == "clock" : clock stream (any type) (passive)
Sources: 1 (active)
Control interfaces: zapIStreamSyncer
Output stream:
as input stream
----------------------------------------------------------------------
46) timestamp-scaler
--------------------
Filter for modifying the timestamps t of input frames to
(numerator*t)/denominator+offset.
Note: Warning: this node mutates frames. XXX maybe we need cloning.
Control interfaces: zapITimestampScaler
Node parameters:
- unsigned long long "offset": Offset to add to timestamps (default: 0)
- unsigned long "numerator" : See description above (default: 1)
- unsigned long "denominator" : See description above (default: 1)
Input stream:
any
Output stream:
any
----------------------------------------------------------------------
47) timestamp-trigger
---------------------
Filter for triggering callbacks when a timestamp is being traversed.
Node parameters:
none
Control interfaces: zapITimestampTrigger
Input stream:
any
Output stream:
same as input stream
----------------------------------------------------------------------
48) burster
-----------
General purpose packet 'burster'. Once started by calling
zapIPacketBurster::start(), moves unaltered packets from the attached
source to the attacked sink (if present) until stopped by calling
zapIPacketBurster::stop(), or until the attached source is removed or
the attached source returns an error for a produceFrame() call.
Sinks: 1 (active)
Sources: 1 (active)
Control interfaces: zapIPacketBurster
Output stream:
same as "input" stream
