IRC Sonic pi repl

1. A what?

The idea is that you can interact with a bot over irc, sending sonic-pi code and it will play that code live in the radio. Sonic pi is "Sonic Pi is a live coding environment based on Ruby, originally designed to support both computing and music lessons in schools, developed by Sam Aaron in the University of Cambridge Computer Laboratory in collaboration with Raspberry Pi Foundation". https://sonic-pi.net/

If you want to try it out it is running at irc.dot.org.es, 6667/6697-ssl on the channel #radio (type !help there). You can hear it at https://radio.dot.org.es/playground.ogg

2. Setting up sonic pi on a VPS

2.1. Basic setup

Set up on a arm64 (aarch64) oracle vps (that you can get for free and it's pretty cool!) using an arch linux container.

yay -S sonic-pi jack2 sonic-pi-tool darkice pulseaudio pulseaudio-jack alsa alsa-tools alsa-plugins jack-example-tools

On ubuntu the packages have similar names. I know jack is jackd2, sonic-pi-tool has to be installed manually from https://github.com/emlyn/sonic-pi-tool/ (it is just a single python file with some pypi dependencies and we only use to launch sonic-pi from cli). Darkice is for live streaming to the icecast server and is available on ubuntu repos. The rest might not be a problem.

You might also need the snd-dummy modules:

modprobe snd-dummy

When you have all set up properly aplay -L should return a list like:

default
    Playback/recording through the PulseAudio sound server
null
    Discard all samples (playback) or generate zero samples (capture)
samplerate
    Rate Converter Plugin Using Samplerate Library
speexrate
    Rate Converter Plugin Using Speex Resampler
jack
    JACK Audio Connection Kit
oss
    Open Sound System
pulse
    PulseAudio Sound Server
upmix
    Plugin for channel upmix (4,6,8)
vdownmix
    Plugin for channel downmix (stereo) with a simple spacialization
dummy

Also ls /dev/snd/ should be like:

$ ls /dev/snd/
by-path  controlC0  controlC2  pcmC0D0c  pcmC0D0p  pcmC2D0c  pcmC2D0p  pcmC2D1c  pcmC2D1p  seq  timer

If you are going to use pulseaudio edit /etc/puse/default.pa adding:

load-module module-null-sink sink_name=0
load-module module-jackdbus-detect channels=2

If you plan to use a ubuntu vps for this also install the qt platform plugin xcb to launch sonic pi:

apt install libxinerama1 libxcb-util1

You have to run sonic-pi at least once before being able to launch it with sonic-pi-tool start-server. Or maybe just manually create the log file that python script wants if you get the error FileNotFoundError: [Errno 2] No such file or directory: '/home/mattf/.sonic-pi/log/server-output.log':

mkdir -p /home/mattf/.sonic-pi/log/
touch /home/mattf/.sonic-pi/log/server-output.log

Remember that jack must be always running in order for sonic-pi to work.

2.2. Lxd container setup

If you are setting this up on a lxd container, besides of doing all above also inside the container, you will need some modifications to make realtime scheduling works. It is easier if it is a privileged container. You can edit the config with sudo EDITOR=nvim lxc config edit arch. Here is mine

architecture: aarch64
config:
  image.architecture: arm64
  image.description: Archlinux current arm64 (20220325_14:16)
  image.os: Archlinux
  image.release: current
  image.requirements.secureboot: "false"
  image.serial: "20220325_14:16"
  image.type: squashfs
  image.variant: default
  limits.kernel.nofile: "200000"
  raw.lxc: |-
    lxc.mount.entry = /dev/fuse dev/fuse none bind,create=file,optional
    lxc.mount.auto=cgroup:rw
    lxc.cgroup.devices.allow = c 116:* rwm
    lxc.mount.entry = /dev/snd dev/snd none bind,optional,create=dir
  security.nesting: "true"
  security.privileged: "true"



    (...)

The raw.lxc part is the relevant part. On arch it is necessary to install alsa-plugins in order for aplay -L show any output.

2.3. Confirming things work

You might try to launch jack now, either with qjackctl or with jack -d dummy, then launch sonic-pi. If you run into an error like: No such file or directory - jack_connect you are probably missing jack-example-tools as described here: https://github.com/sonic-pi-net/sonic-pi/issues/2994

The archwiki might also help in case of other problems https://wiki.archlinux.org/title/PulseAudio/Troubleshooting https://wiki.archlinux.org/title/JACK_Audio_Connection_Kit.

The way set it up on archlinux was using pulseaudio and jack on top, but you might not need them both in fact, just jack is required for sonic pi and a broadcasting software like darkice can use jack even though i haven't tested that. I just found it more comfortable to have both.

If sonic-pi launched great! If not, maybe try the lxd approach, I couldn't really get this working on ubuntu as well ;).

3. Broadcasting sonic pi

Considering you already have a shoutcast or icecast2 server to stream, here is the setup I've done for streaming audio from sonic-pi.

At first I logged into a graphical environment into the container to set things up using pavucontrol and qjackctl. It is possible to set this up from the tty using tools like pacmd and whatever you can use for jack but if you can afford it, doing from the GUI is far much easier and faster in this case.

So my goal is to use darkice to stream sonic-pi's audio to an icecast2 instance. I can use commands like sonic-pi-tool run-file song.rb or sonic-pi-tool evan "play :c5" to test the stream. Darkice will be capturing the audio output from sonic pi and streaming it. So first create a darkice config like:

[general]
duration        = 0
bufferSecs      = 5
reconnect       = yes
realtime        = yes
rtprio          = 3

[input]
device          = pulse # Maybe you just use jack?
sampleRate      = 44100
bitsPerSample   = 16
channel         = 2

[icecast2-0]
bitrateMode     = cbr
format          = vorbis
bitrate         = 128
server          = radio.dot.org.es
port            = 8000
password        = passwordherehahaha
mountPoint      = playground.ogg
name            = Sonic Pi
description     = Sonic Pi REPL
url             = https://radio.dot.org.es
genre           = radio
public          = yes

Again you might want to change the input device to jack if you don't want to use pulseaudio. In my cause I opted to use pulseaudio to also stream from other tools that aren't compatible with jack.

Another thing to notice! This is a vorbis encoded stream. This means that silence results in no flux of data being transmitted, which the majority of players and browsers don't know how to handle it. To circumvent this I will be using a comfort noise generated with:

sox -c1 -n result.wav synth 10 sin 25000 vol 1

And you can play it in loop with something like:

mpv -loop result.wav

This should be inaudible but for some reason there is a perceptible noise on the stream still. I haven't found a way to fix that but if you adjust the volumes correctly it will become unnoticeable. It just has to be enough so that there is no silence on the stream and the players weirdly start playing some random cache or simply don't know what to do (because they think the server stopped streaming).

You could also simply stream in another format that doesn't have this problem, but maybe it would have bigger latency then?

3.1. Start broadcasting

Launch pulseaudio if you are using it:

pulseaudio --start

Then the jack daemon:

jackd -d dummy

Play the comfort noise:

mpv -loop result.wav

Start broadcasting:

darkice -c darkice.conf

And play something!

live_loop :bass do
  play :c2
  sleep 0.5
  play :c2
  sleep 0.5
  play :e2
  sleep 0.5
  play :f2
  sleep 0.5
end
sonic-pi-tool run-file song.rb

4. The bot

I then created a irc bot using my own irc python library: https://github.com/matheusfillipe/ircbot I stole what matters from sonic-pi-tool source code:

import collections
import html
import logging
import os
import re
import socket
import sys
import time

from oscpy.client import OSCClient
from oscpy.server import OSCThreadServer

SERVER_OUTPUT = "~/.sonic-pi/log/server-output.log"


logger = logging.getLogger()


class Server:
    """Represents a running instance of Sonic Pi."""

    preamble = '@osc_server||=SonicPi::OSC::UDPServer.new' + \
               '({},use_decoder_cache:true) #__nosave__\n'

    def __init__(self, host, cmd_port, osc_port, send_preamble, verbose):
        self.client_name = 'SONIC_PI_TOOL_PY'
        self.host = host
        self._cmd_port = cmd_port
        self._cached_cmd_port = None
        self.osc_port = osc_port
        # fix for https://github.com/repl-electric/sonic-pi.el/issues/19#issuecomment-345222832
        self.send_preamble = send_preamble
        self._cmd_client = None
        self._osc_client = None

    def get_cmd_port(self):
        return self._cmd_port

    def cmd_client(self):
        if self._cmd_client is None:
            self._cmd_client = OSCClient(self.host, self.get_cmd_port(),
                                         encoding='utf8')
        return self._cmd_client

    def osc_client(self):
        if self._osc_client is None:
            self._osc_client = OSCClient(self.host, self.osc_port,
                                         encoding='utf8')
        return self._osc_client

    def get_preamble(self):
        if self.send_preamble:
            return Server.preamble.format(self.get_cmd_port())
        return ''

    def send_cmd(self, msg, *args):
        client = self.cmd_client()
        logger.info("Sending command to {}:{}: {} {}"
                    .format(self.host, self.get_cmd_port(), msg,
                            ', '.join(repr(v) for v in (self.client_name,) + args)))
        client.send_message(msg, (self.client_name,) + args)

    def send_osc(self, path, args):
        def parse_val(s):
            try:
                return int(s)
            except ValueError:
                pass
            try:
                return float(s)
            except ValueError:
                pass
            if len(s) > 1 and s[0] == '"' and s[-1] == '"':
                return s[1:-1]
            return s

        client = self.osc_client()
        parsed = [parse_val(s) for s in args]
        logger.info("Sending OSC message to {}:{}: {} {}"
                    .format(self.host, self.osc_port, path,
                            ', '.join(repr(v) for v in parsed)))
        client.send_message(path, parsed)

    def check_if_running(self):
        cmd_listening = Server.port_in_use(self.get_cmd_port())
        logger.info("The command port ({}) is {}in use".format(self.get_cmd_port(),
                                                               "" if cmd_listening else "not "))
        osc_listening = Server.port_in_use(self.osc_port)
        logger.info("The OSC port ({}) is {}in use".format(self.osc_port,
                                                           "" if osc_listening else "not "))
        osc_listening = True
        if cmd_listening and osc_listening:
            logger.info("Sonic Pi is running, and listening on port {} for commands and {} for OSC"
                        .format(self.get_cmd_port(), self.osc_port), True)
            return 0
        elif not cmd_listening and not osc_listening:
            logger.info("Sonic Pi is not running", True)
            return 1
        else:
            logger.info("Sonic Pi is not running properly, or there's an issue with the port numbers",
                        True)
            return 2

    def stop_all_jobs(self):
        self.send_cmd('/stop-all-jobs')

    def run_code(self, code):
        self.send_cmd('/run-code', self.get_preamble() + code)

    def start_recording(self):
        self.send_cmd('/start-recording')

    def stop_and_save_recording(self, path):
        self.send_cmd('/stop-recording')
        self.send_cmd('/save-recording', path)

    @staticmethod
    def port_in_use(port):
        with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sock:
            try:
                sock.bind(('127.0.0.1', port))
            except OSError:
                return True
        return False

    @staticmethod
    def determine_command_port():
        try:
            with open(os.path.expanduser(SERVER_OUTPUT)) as f:
                for line in f:
                    m = re.search('^Listen port: *([0-9]+)', line)
                    if m:
                        return int(m.groups()[0])
        except FileNotFoundError:
            pass

    @staticmethod
    def handle_log_info(style, msg):
        msg = "=> {}".format(msg)
        logger.info(msg)
        logger.info()

    @staticmethod
    def handle_multi_message(run, thread, time, n, *msgs):
        msg = "{{run: {}, time: {}}}".format(run, time)
        logger.info(msg)
        for i in range(n):
            typ, msg = msgs[2 * i: 2 * i + 2]
            for j, line in enumerate(msg.splitlines()):
                if i < n - 1:
                    prefix = "  ├─ " if j == 0 else "  │"
                else:
                    prefix = "  └─ " if j == 0 else "   "
                logger.info(f"{prefix}, {line}, {typ}")
        logger.info()

    @staticmethod
    def handle_runtime_error(run, msg, trace, line_num):
        lines = html.unescape(msg).splitlines()
        prefix = "Runtime Error: "
        for line in lines:
            logger.debug(f"{prefix=} {line=}")
            prefix = ""
        logger.debug(html.unescape(trace))

    @staticmethod
    def handle_syntax_error(run, msg, code, line_num, line_s):
        logger.error("Error: " + html.unescape(msg))
        prefix = "[Line {}]: ".format(line_num) if line_num >= 0 else ""
        logger.error(f"{prefix=}, {code=}")

    def follow_logs(self):
        try:
            server = OSCThreadServer(encoding='utf8')
            server.listen(address='127.0.0.1', port=4558, default=True)
            server.bind('/log/multi_message', self.handle_multi_message)
            server.bind('/multi_message', self.handle_multi_message)
            server.bind('/log/info', self.handle_log_info)
            server.bind('/info', self.handle_log_info)
            server.bind('/error', self.handle_runtime_error)
            server.bind('/syntax_error', self.handle_syntax_error)
            while True:
                time.sleep(1)
        except Exception as e:
            return e


def eval_stdin(server: Server):
    server.run_code(sys.stdin.read())


def eval_file(server: Server, path):
    server.run_code(path.read())


def osc(server: Server, path, args):
    server.send_osc(path, args)

I probably could still remove more from there though.

And in the case of the lxd container I redirected the port since I wanted to run the bot in the host. Sonic pi listens on port 4557 udp.

socat UDP-LISTEN:4557,fork,bind=$CONTAINER_IP UDP:127.0.0.1:4557

That is forwarding udp packets coming from the container interface to its localhost, which is the address sonic-pi binds by default. You can see the container ip with lxc list from the host or with ip a within the container.

With that in mind I just had to write a simple and dumb repl using the server.run_code method. Here is a simplified version of the bot's code:

import re

from sonic_pi import NoteNotFound
from sonic_pi import Server as PiServer
from sonic_pi import convert_to_notes

from IrcBot.bot import Color, IrcBot, Message, utils

sonic_pi_users = {}
sonic_pi_history = {}

@utils.regex_cmd_with_messsage(r"^(.+)$")
def all_msgs(args: re.Match, msg: Message):
    if msg.nick not in sonic_pi_users or msg.message.strip().startswith(PREFIX):
        return
    sonic_pi_users[msg.nick].append(args[1])
def convert(*args):
    transpose = 0
    if args[0].isdigit():
        octave = int(args[0])
        if args[1][0] in ["+", "-"] and args[1][1:].isdigit():
            transpose = int(args[1])
            letters = "".join(args[2:])
        else:
            letters = "".join(args[1:])
    else:
        octave = 4
        if args[0][0] in ["+", "-"] and args[0][1:].isdigit():
            transpose = int(args[0])
            letters = "".join(args[1:])
        else:
            letters = "".join(args)
        logger.debug(f"{octave=} {letters=}")
        return ", ".join(convert_to_notes(letters, octave, transpose))


@utils.arg_command("pi", "Toggles sonic pi repl", f"{PREFIX}pi [command]- https://sonic-pi.net/tutorial.html")
async def pi(bot: IrcBot, args: re.Match, msg: Message):
    args = utils.m2list(args)
    if args:
        sonic_pi_users[msg.nick] = [" ".join(args)]

    if msg.nick in sonic_pi_users:
        # append to history
        if msg.nick in sonic_pi_history:
            sonic_pi_history[msg.nick].extend(sonic_pi_users[msg.nick])
        else:
            sonic_pi_history[msg.nick] = sonic_pi_users[msg.nick]

        # Apply template
        for i, line in enumerate(deepcopy(sonic_pi_users[msg.nick])):
            for match in re.findall(r"\$\{([^}]+?)\}", line):
                try:
                    replace = convert(*match.split(" "))
                    sonic_pi_users[msg.nick][i] = sonic_pi_users[msg.nick][i].replace(
                        "${" + match + "}", replace)
                    logger.debug(
                        f"Applying template at {i=} {line=} {match=}, {replace=} {sonic_pi_users[msg.nick][i]=}")
                except NoteNotFound as e:
                    await reply(bot, msg, error(f"Could not find note '{e}'"))
                    del sonic_pi_users[msg.nick]
                    return

        await reply(bot, msg, "Your Sonic Pi repl is now off. Sending code to sonic pi...")
        server.run_code("\n".join(sonic_pi_users[msg.nick]))
        del sonic_pi_users[msg.nick]
        return

    sonic_pi_users[msg.nick] = []
    await reply(bot, msg, f"Your Sonic Pi repl is now live at: {SONIC_PI_LIVE_URL}. Type {PREFIX}pi to turn it off and evaluate your code.")


@utils.arg_command("convert", "Convert keyboard characters into sonic pi notes", f"{PREFIX}convert [octave] [±transpose] <letters>  -  Only qwerty layout")
async def cmd_convert(bot: IrcBot, args: re.Match, msg: Message):
    args = utils.m2list(args)
    if not args:
        await reply(bot, msg, error("You need to specify an octave an a string to convert"))
        return
    if args[0].isdigit() and (int(args[0]) < 0 or int(args[0]) > 10):
        await reply(bot, msg, error("The first argument must be between 0 and 10"))
        return

    try:
        res = convert(*args)
        await reply(bot, msg, res)
    except NoteNotFound as e:
        await reply(bot, msg, error(f"Could not find note for '{e}'"))


@utils.arg_command("pstop", "Stops sonic pi audio")
async def stop(bot: IrcBot, args: re.Match, msg: Message):
    server.stop_all_jobs()
    await reply(bot, msg, "Stopping audio...")


@utils.arg_command("paste", "Pastes your sonic pi code and clears your history")
async def pipaste(bot: IrcBot, args: re.Match, msg: Message):
    if msg.nick not in sonic_pi_history:
        await reply(bot, msg, error("You need to turn on your sonic pi repl first. Use {}pi".format(PREFIX)))
        return
    await reply(bot, msg, paste("\n".join(sonic_pi_history[msg.nick])))
    del sonic_pi_history[msg.nick]


@utils.arg_command("read", "Read code from ix.io paste (or any raw text url)")
async def readurl(bot: IrcBot, args: re.Match, msg: Message):
    try:
        server.run_code(read_paste(args[1]))
    except Exception as e:
        await reply(bot, msg, error("Failed to read paste: ") + str(e))
    await reply(bot, msg, "Code has been read and sent!")


# And you might want to define all those constants somewhere
if __name__ == "__main__":
    utils.setLogging(LOG_LEVEL, LOGFILE)
    bot = IrcBot(HOST, PORT, NICK, CHANNELS, PASSWORD, use_ssl=PORT == 6697,
                 dcc_host=DCC_HOST, dcc_ports=DCC_PORTS, dcc_announce_host=DCC_ANNOUNCE_HOST)
    bot.runWithCallback(onconnect)

You can see the full code for this bot at: https://github.com/matheusfillipe/mpd_irc_bot but it does much more than being a sonic pi repl now. It manages the whole radio playlist queue allowing users to submit music with url's, youtube links or with the irc DCC file transfer protocol.

5. Conclusion

It was a nice experiment but sonic-pi, despite being a fantastic tool for playing around on your desktop, is not really well forged for the server setup. It relies on too many stuff and chances of something going wrong are high. I went with it for this experiment because there isn't a musical language with an easier to understand language and just as well as documented. There is overtone: https://github.com/overtone/overtone but come on, who even knows clojure! Also, sonic-pi comes with so many preset samples that I didn't even consider creating a bot command to allow users to use their own samples.

I imagine it would be nice to have a real web app using a setup like this(https://in-thread.sonic-pi.net/t/sonic-pi-in-a-web-browser/1563/5)! Not sure how to reduce the latency though. I am having like 6 seconds of latency with this jack -> darkice -> icecast setup. Here is an interesting similar project: https://github.com/merongivian/negasonic



terminal

Comments


Be the first to comment!

    ﹏﹏﹏﹏﹏﹏﹏﹏﹏﹏﹏﹏﹏﹏﹏﹏﹏﹏﹏﹏﹏﹏﹏﹏﹏﹏﹏﹏

    IRC Sonic pi repl

    Author: Matheus Fillipe

    Creation Date: 2022-05-11 Wed 00:00

    Modified: 2022-07-15 Fri 02:40

    View this page on github

    Emacs 28.1 (Org mode 9.5.2)