Title
#users-public
j

Jakub Kubík

09/20/2022, 8:39 AM
Hello! We have recently stumbled upon another issue with the python-questdb-client. From our empirical testing, it seems that when sending a buffer and awaiting reply, the thread executing the action is not switched away from (not exactly busy waiting, but does not appear to the OS as blocked by I/O). Is this a known issue?
Adam Cimarosti

Adam Cimarosti

09/20/2022, 11:58 AM
In Python you call the
flush
method:
sender.flush()
In Cython, we call the C method:
if not line_sender_flush(self._impl, c_buf, &err):
This is implemented in Rust, and it just calls the
flush()
function:
sender.flush(buffer)
This is implemented as so:
pub fn flush(&mut self, buf: &mut Buffer) -> Result<()> {
        self.flush_and_keep(buf)?;
In turn, flush and keep just does a
write_all
to the socket:
if let Err(io_err) = self.conn.write_all(bytes) {
The write-all will switch depending on encryption on or off and call
write
repeatedly until it's all written out:
impl io::Write for Connection {
        fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
            match self {
                Self::Direct(sock) => sock.write(buf),
                Self::Tls(stream) => stream.write(buf)
            }
        }

        ...
    }
Assuming no encryption, this directly calls the
socket2::Socket::write
library function. In turn it's implemented as so:
impl Write for Socket {
        fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
            self.send(buf)
        }
In turn:
pub fn send(&self, buf: &[u8]) -> io::Result<usize> {
        self.send_with_flags(buf, 0)
    }
In turn:
pub fn send_with_flags(&self, buf: &[u8], flags: c_int) -> io::Result<usize> {
        sys::send(self.as_raw(), buf, flags)
    }
Here
sys
is the library's OS-specific dispatcher. Assuming this is a Unix-like OS:
pub(crate) fn send(fd: Socket, buf: &[u8], flags: c_int) -> io::Result<usize> {
        syscall!(send(
            fd,
            buf.as_ptr().cast(),
            min(buf.len(), MAX_BUF_LEN),
            flags,
        ))
        .map(|n| n as usize)
    }
That macro is this one:
macro_rules! syscall {
        ($fn: ident ( $($arg: expr),* $(,)* ) ) => {{
            #[allow(unused_unsafe)]
            let res = unsafe { libc::$fn($($arg, )*) };
            if res == -1 {
                Err(std::io::Error::last_os_error())
            } else {
                Ok(res)
            }
        }};
    }
The
libc::$fn($($arg, )*)
is the important part here as it just calls the libc implementation of
send
.
pub fn send(socket: ::c_int, buf: *const ::c_void, len: ::size_t, flags: ::c_int) -> ::ssize_t;