| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Current pacat code reads whatever available from STDIN and writes
it directly to the playback stream. A minimal buffer is created
for each read operation; no further reads are then allowed unless
earlier read buffer has been fully consumed by a stream write.
While quite simple, this model breaks upon the new requirements of
writing only frame-aligned data to the stream (commits #1 and #2).
The kernel read syscall can return a length much smaller than the
frame-aligned size requested, leading to invalid unaligned writes.
This can easily be reproduced by choosing a starved STDIN backend:
pacat /dev/random pa_stream_write() failed: EINVAL
echo 1234 | pacat pa_stream_write() failed: EINVAL
or by playing an incomplete WAV file in raw, non-paplay, mode.
So guard against such incomplete kernel reads by writing only in
frame-aligned sizes, while caching any trailing partial frame for
subsequent writes.
Other operation modes are not affected. Non-raw paplay playback is
handled by libsndfile, ensuring complete reads, and recording mode
just writes to the STDOUT fd without any special needs.
CommitReference #1: 22827a5e1e62
CommitReference #2: 150ace90f380
BugLink: https://bugs.freedesktop.org/show_bug.cgi?id=98475
BugLink: https://bugs.freedesktop.org/show_bug.cgi?id=77595
Suggested-by: David Henningsson <diwic@ubuntu.com>
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
Executing below command will not produce any audio:
pacat --channels=3 /dev/urandom
Turns out that pa_stream_write() breaks large audio buffers into
segments of the maximum memblock size available -- a value which
is not necessarily frame aligned.
Meanwhile the server discards any non-aligned client audio, as a
security measure, due to some earlier reported daemon crashes.
Thus divide sent audio to the expected aligned form.
CommitReference-1: 22827a5e1e62
CommitReference-2: 150ace90f380
BugLink: https://bugs.freedesktop.org/show_bug.cgi?id=98475
BugLink: https://bugs.freedesktop.org/show_bug.cgi?id=77595
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
Users reported pacat crashes when playing certain multi-channel
audio. For example:
pacat --channels=2 /dev/zero works
pacat --channels=3 /dev/zero pa_stream_write() failed: EINVAL
pacat --channels=4 /dev/zero works
pacat --channels=5 /dev/zero pa_stream_write() failed: EINVAL
pacat audio playback is pipe-like, from STDIN to PA write stream.
Meanwhile STDIN "ready to read" events got regularly triggered
before the write stream was even created, or at moments where the
stream could not accept any more audio.
In these out-of-sync cases, the write stream could not report the
appropriate buffer lengths it accepts, thus a default of 4K bytes
was chosen -- compatible by luck with some channel counts and
incompatible with others.
Instead of choosing a faulty default in these scenarios, mute the
the STDIN events until the write stream is available & queriable.
BugLink: https://bugs.freedesktop.org/show_bug.cgi?id=98475
Reported-by: Tanu Kaskinen <tanuk@iki.fi>
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
Although such 9.0 clients support memfd transport, they have an
iochannel bug that would break memfd audio if they're run in 32
bit mode over a 64-bit kernel. Influence them to use the POSIX
shared memory model instead.
Also bump the protocol version to exclusively mark such v9.0
libraries. Check commit 451d1d676237c81 for further details.
BugLink: https://bugs.freedesktop.org/show_bug.cgi?id=97769
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
Users reported audio breakage for 32-bit pulse clients connected
to a 64-bit server over memfds. Investigating the issue further,
the problem is twofold:
1. iochannel's file-descriptor passing code is liberal in what it
issues: produced ancillary data object's "data" section exceeds
length field. How such an extra space is handled is a grey area
in the POSIX.1g spec, the IETF RFC #2292 "Advanced Sockets API
for IPv6" memo, and the cmsg(3) manpage.
2. A 64-bit kernel handling of such extra space differs by whether
the app is 64-bit or 32-bit. For 64-bit apps, the kernel
smartly ducks the issue. For 32-bit apps, an -EINVAL is
directly returned; that's due to a kernel CMSG header traversal
bug in the networking stack "32-bit sockets emulation layer".
Compare Linux Kernel's socket.h cmsg_nxthdr() code and the
32-bit emulation layer version of it at net/compat.c
cmsg_compat_nxthdr() for further info. Notice how the former
graciously ignores incomplete CMSGs while the latter _directly_
complains about them -- as of kernel version 4.9-rc5.
(A kernel patch is to be submitted)
Details:
iochannel typically uses sendmsg() for passing FDs & credentials.
>From RFC 2292, sendmsg() control data is just a heterogeneous
array of embedded ancillary objects that can differ in length.
Linguistically, a "control message" is an ancillary data object.
For example, below is a sendmsg() "msg_control" containing two
ancillary objects:
|<---------------------- msg_controllen---------------------->|
| |
|<--- ancillary data object -->|<----- ancillary data object->|
|<------- CMSG_SPACE() ------->|<------- CMSG_SPACE() ------->|
| | |
|<-------- cmsg_len ------->| |<-------- cmsg_len ------->| |
|<------- CMSG_LEN() ------>| |<------- CMSG_LEN() ------>| |
| | | | |
+-----+-----+-----+--+------+--+-----+-----+-----+--+------+--+
|cmsg_|cmsg_|cmsg_|XX|cmsg_ |XX|cmsg_|cmsg_|cmsg_|XX|cmsg_ |XX|
|len |level|type |XX|data[]|XX|len |level|type |XX|data[]|XX|
+-----+-----+-----+--+------+--+-----+-----+-----+--+----+-+--+
^^^^^^^ Ancil Object #1 ^^^^^^^ Ancil Object #2
(control message) (control message)
^
|
+--- sendmsg() "msg_control" points here
Problem is, while passing FDs, iochannel's code try to avoid
variable-length arrays by creating a single cmsg object that can
fit as much FDs as possible:
union {
struct cmsghdr hdr;
uint8_t data[CMSG_SPACE(sizeof(int) * MAX_ANCIL_DATA_FDS)];
} cmsg; ^^^^^^^^^^^^^^^^^^
Most of the time though the number of FDs to be passed is less
than the maximum above, thus "cmsg_len" is set to the _actual_ FD
array size:
cmsg.hdr.cmsg_len = CMSG_LEN(sizeof(int) * nfd);
^^^
This inconsistency tricks the kernel into thinking that we have 2
ancillay data objects instead of one! First cmsg is valid as
intended, but the second is instantly _corrupt_ since it has a
cmsg_len size of 0 -- thus failing kernel's CMSG_OK() tests.
For 32-bit apps on a 32-bit kernel, and 64-bit apps over a 64-bit
one, the kernel's own CMSG header traversal macros just ignore the
second "incomplete" cmsg. For 32-bit apps over a 64-bit kernel
though, the kernel 32-bit socket emulation macros does not forgive
such incompleteness and directly complains of invalid args (due to
a subtle bug).
Avoid this ugly problem, which can also bite us in a pure 64-bit
environment if MAX_ANCIL_DATA_FDS got extended to 5 FDs, by
setting "cmsg_data[]" array size to "cmsg_len".
BugLink: https://bugs.freedesktop.org/show_bug.cgi?id=97769
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
As reported by valrgrind
==30002== Conditional jump or move depends on uninitialised value(s)
==30002== at 0x5CB883C: pa_cmsg_ancil_data_close_fds (pstream.c:193)
==30002== by 0x5CBB161: do_write (pstream.c:759)
==30002== by 0x5CB8B51: do_pstream_read_write (pstream.c:233)
==30002== by 0x5CB8EE8: io_callback (pstream.c:279)
...
The pa_cmsg_ancil_data structure has two main guards:
'creds_valid', which implies that it holds credentials
information, and 'nfd', which implies it holds file descriptors.
When code paths create a credentials ancillary data structure,
they just set the 'nfd' guard to zero. Typically, the rest of
pa_cmsg_ancil_data fields related to fds are _all_ left
_uninitialized_.
pa_cmsg_ancil_data_close_fds() has broken the above contract:
it accesses the new 'close_fds_on_cleanup' flag, which is related
to file descriptors, without checking the 'nfd == 0' guard first.
Fix this inconsistency.
Reported-by: Alexander E. Patrakov <patrakov@gmail.com>
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
Signed-off-by: Arun Raghavan <arun@arunraghavan.net>
|
|
As shown by valgrind
==10615== Conditional jump or move depends on uninitialised value(s)
==10615== at 0x5CC0483: shm_marker_size (shm.c:97)
==10615== by 0x5CC1685: shm_attach (shm.c:381)
==10615== by 0x5CC1990: pa_shm_cleanup (shm.c:453)
==10615== by 0x5CC068E: sharedmem_create (shm.c:150)
...
Solution is to fix the shm_marker_size() signature itself: At
certain code paths like shm_attach(), we don't want to initialize
_any_ field in the passed SHM segment descriptor except after
making sure all error exit conditions have been passed.
Reported-by: Alexander E. Patrakov <patrakov@gmail.com>
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
Signed-off-by: Arun Raghavan <arun@arunraghavan.net>
|
|
Now that all layers in the stack support memfd blocks, add memfd
support for the daemon's global core mempool. Also introduce
"enable-memfd=" daemon argument and configuration option.
For now, memfd support is an opt-in feature to be activated only
when daemon's enable-memfd= is set to yes.
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
Now that all layers in the stack support memfd blocks, add memfd
pools support for client context and audio playback data.
Use such memfd pools by default only if the server signals memfd
support in its connection negotiations.
Also add ability for clients to force-disable memfd transport
through the `enable-memfd=' client configuration option.
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
The advertised alternative, module-udev-detect, is Linux-specific.
BugLink: https://bugs.freedesktop.org/show_bug.cgi?id=94339
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
Both must be in multiples of the stream's sample spec frame size.
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
srbchannel needs fd passing. Otherwise we get the following error
for systems without SCM_CREDENTIALS support:
Code should not be reached at pulsecore/pstream-util.c:95,
function pa_pstream_send_tagstruct_with_fds(). Aborting.
[[ The root cause is that we define HAVE_CREDS only if
SCM_CREDENTIALS is defined, but SCM_CREDENTIALS is a Linux-specific
symbol. Thus HAVE_CREDS is always disabled on Solaris.
And since pulse couples the non-portable creds passing support
with the portable fd passing one, through _35_ places where
HAVE_CREDS is used, a real fix needs a PA redesign -- assuming that
latency on Solaris is something people care about. ]]
BugLink: https://bugs.freedesktop.org/show_bug.cgi?id=94339
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
Per glibc feature_test_macros(7), setting compiler flags to
-std=c11 (or any c* variant like c99) enforces strict ANSI
mode.
Enforcing strict ANSI makes all declarations under _GNU_SOURCE
unavailable. This leads to build warnings in the form of:
warning: implicit declaration of function ‘syscall’
Thus replace -std=c11 with -std=gnu11
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
Now that we have the necessary infrastructure to memexport and
mempimport a memfd memblock, extend that support higher up in the
chain with pstreams.
A PA endpoint can now _transparently_ send a memfd memblock to the
other end by simply calling pa_pstream_send_memblock() – provided
the block's memfd pool was earlier registered with the pstream.
If the pipe does not support memfd transfers, we fall back to
sending the block's full data instead of just its reference.
** Further details:
A single pstream connection usually transfers blocks from multiple
pools including the server's srbchannel mempool, the client's
audio data mempool, and the server's global core mempool.
If these mempools are memfd-backed, we now require registering
them with the pstream before sending any blocks they cover. This
is done to minimize fd passing overhead and avoid fd leaks.
Moreover, to support all these pools without hard-coding their
number or nature in the Pulse communication protocol itself, a new
REGISTER_MEMFD_SHMID command is introduced. That command can be
sent _anytime_ during the pstream's lifetime and is used for
creating on demand SHM ID to memfd mappings.
Suggested-by: David Henningsson <david.henningsson@canonical.com>
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
Color global mempools with a special mark. This special marking
is needed for handling memfd-backed pools.
To avoid fd leaks, memfd pools are registered with the connection
pstream to create an ID<->memfd mapping on both PA endpoints.
Such memory regions are then always referenced by their IDs and
never by their fds, and so their fds can be safely closed later.
Unfortunately this scheme cannot work with global pools since the
registration ID<->memfd mechanism needs to happen for each newly
connected client, and thus the need for a more special handling.
That is, for the pool's fd to be always open :-(
Almost all mempools are now created on a per-client basis. The
only exception is the pa_core's mempool which is still shared
between all clients of the system.
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
To transfer memfd-backed blocks without passing their fd every time,
thus minimizing overhead and avoiding fd leaks, a command is sent
with the memfd fd as ancil data very early on.
This command has an ID that uniquely identifies the memfd region.
Further memfd block references are then exclusively done using this
ID.
This commit implements the details of such 'permanent' mappings on
the receiving end, using memimport segments.
Suggested-by: David Henningsson <david.henningsson@canonical.com>
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
Memfd is a simple memory sharing mechanism, added by the systemd/kdbus
developers, to share pages between processes in an anonymous, no global
registry needed, no mount-point required, relatively secure, manner.
This patch introduces the necessary building blocks for using memfd
shared memory transfers in PulseAudio.
Memfd support shall also help us in laying out the necessary (but not
yet sufficient) groundwork for application sandboxing, protecting PA
from its clients, and protecting clients data from each other.
We plan to exclusively use memfds, instead of POSIX SHM, on the way
forward.
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
pa_shm_create_rw() is responsible for creating two types of memory:
POSIX shared memory and regular malloc()-ed ones.
A third memory type, memfds, will be added later. Thus to add this
extra shared memory type in a sane manner, refactor private memory
allocations into their own static methods.
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
Soon we're going to have three types of memory pools: POSIX shm_open()
pools, memfd memfd_create() ones, and privately malloc()-ed pools.
Thus introduce annotations for the memory types supported and change
pa_mempool_new() into a factory method based on required memory.
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
The PA daemon currently uses a single SHM file for all clients
sending and receiving commands over the low-latency srbchannel
mechanism.
To avoid leaks between clients in that case, and to provide the
necessary ground work later for sandboxing and memfds, create the
srbchannel SHM files on a per-client basis.
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
In future commits, server-wide SHMs will be replaced with per-client
ones that will be dynamically created and freed according to clients
connections open and close.
Meanwhile, current PA design does not guarantee that the per-client
mempool blocks are referenced only by client-specific objects.
Thus reference count the pools and let each memblock inside the pool
itself, or just attached to it, increment the pool's refcount upon
allocation. This way, per-client mempools will only be freed when no
further component in the system holds any references to its blocks.
DiscussionLink: https://goo.gl/qesVMV
Suggested-by: Tanu Kaskinen <tanuk@iki.fi>
Suggested-by: David Henningsson <david.henningsson@canonical.com>
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
The daemon `shm-size-bytes' configuration value was read, and then
directly used, for creating the initial server-wide SHM files.
This is fine for now, but soon, such server-wide SHMs will be replaced
with per-client SHM files that will be dynamically created and deleted
according to clients open and close. Thus, appropriately cache this
configuration value.
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
sd_journal_send() implicitly add fields for the source file,
function name and code line from where it's invoked. As code
location fields CODE_FILE, CODE_LINE and CODE_FUNC are handled
by PA's log module, we do not want the automatic values
supplied by the sd_journal API.
Without suppressing these, both the actual log event source
and the call to sd_journal_send() will be logged:
$ journalctl -b -f -o json-pretty
[...]
CODE_FILE : [ pulsecore/log.c, pulsecore/module.c ],
CODE_LINE : [ 505, 181 ],
MESSAGE : Failed to load module module-gconf
CODE_FUNC : [ pa_log_levelv_meta, pa_module_load ],
[...]
(Commit log adapted from abrt libreport commit d1eaae97f0287f)
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
Now that we have memory usage benchmarks collected at our disposal,
introduce a gnuplot script to plot the newest version.
To avoid scaling issues, memory is plotted in a "double y axis" form,
with VM usage on the left, and dirty RSS memory usage on the right.
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
Add shell script to sample PulseAudio memory usage while increasing
the number of connected 'paplay' clients over time.
Linux kernel /proc/$PID/smaps Private and Shared_Dirty fields are used
to accurately measure the total size of used dirty pages over time.
This shall be useful for benchmarking the PA daemon's memory while
introducing new features like per-client SHM access and memfds.
Also add an empty benchmarks-collection directory 'benchmarks/'. All
output from the benchmarking tools shall be saved in this place, with
timestamps and symbolic links to the newest versions.
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
Otherwise the important configure script error messages get buried
by the "make clean" output.
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
Commit 262bdae0330e used symbols which are only available if systemd
support was compiled in. Fix by using the appropriate #ifdef guards.
Also document the resulting PULSE_LOG_JOURNAL environment variable
behavior if systemd journal support was not compiled in.
[Diwic: changed wording slightly.]
Reported-by: David Henningsson <david.henningsson@canonical.com>
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
By introducing such an environment variable, applications using the
PA client libraries can configure these libraries to send their logs
directly to the journal.
While client libraries journal logging can be indirectly achieved
using PULSE_LOG_SYSLOG, this pollutes the journal. Meta data gets
replicated twice: once in the journal meta fields and once in the
syslog(3) plain-text message itself.
For attaching any backtraces, also introduce the PA-specific journal
meta field PULSE_BACKTRACE. This is the recommend journal practice
instead of appending any furuther data to the logging message itself.
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
|
Document how to modify the client libraries logging behvaior
using any of the PA-specific environment variables.
Using the PULSE_LOG_* environment variables makes debugging
and tracing PA applications quite easy, thus the need for an
official documentation text.
Signed-off-by: Ahmed S. Darwish <darwish.07@gmail.com>
|
