IRC Sonic pi repl
Table of Contents
1. A what?
The idea is that you can interact with a bot over irc, sending sonic-pi code and it will play that code live in the radio. Sonic pi is "Sonic Pi is a live coding environment based on Ruby, originally designed to support both computing and music lessons in schools, developed by Sam Aaron in the University of Cambridge Computer Laboratory in collaboration with Raspberry Pi Foundation". https://sonic-pi.net/
If you want to try it out it is running at irc.dot.org.es, 6667/6697-ssl on the channel #radio (type !help there). You can hear it at https://radio.dot.org.es/playground.ogg
2. Setting up sonic pi on a VPS
2.1. Basic setup
Set up on a arm64 (aarch64) oracle vps (that you can get for free and it's pretty cool!) using an arch linux container.
yay -S sonic-pi jack2 sonic-pi-tool darkice pulseaudio pulseaudio-jack alsa alsa-tools alsa-plugins jack-example-tools
On ubuntu the packages have similar names. I know jack is jackd2, sonic-pi-tool has to be installed manually from https://github.com/emlyn/sonic-pi-tool/ (it is just a single python file with some pypi dependencies and we only use to launch sonic-pi from cli). Darkice is for live streaming to the icecast server and is available on ubuntu repos. The rest might not be a problem.
You might also need the snd-dummy modules:
modprobe snd-dummy
When you have all set up properly aplay -L
should return a list like:
default Playback/recording through the PulseAudio sound server null Discard all samples (playback) or generate zero samples (capture) samplerate Rate Converter Plugin Using Samplerate Library speexrate Rate Converter Plugin Using Speex Resampler jack JACK Audio Connection Kit oss Open Sound System pulse PulseAudio Sound Server upmix Plugin for channel upmix (4,6,8) vdownmix Plugin for channel downmix (stereo) with a simple spacialization dummy
Also ls /dev/snd/
should be like:
$ ls /dev/snd/ by-path controlC0 controlC2 pcmC0D0c pcmC0D0p pcmC2D0c pcmC2D0p pcmC2D1c pcmC2D1p seq timer
If you are going to use pulseaudio edit /etc/puse/default.pa
adding:
load-module module-null-sink sink_name=0 load-module module-jackdbus-detect channels=2
If you plan to use a ubuntu vps for this also install the qt platform plugin xcb to launch sonic pi:
apt install libxinerama1 libxcb-util1
You have to run sonic-pi
at least once before being able to launch it with sonic-pi-tool start-server
. Or maybe just manually create the log file that python script wants if you get the error FileNotFoundError: [Errno 2] No such file or directory: '/home/mattf/.sonic-pi/log/server-output.log'
:
mkdir -p /home/mattf/.sonic-pi/log/ touch /home/mattf/.sonic-pi/log/server-output.log
Remember that jack must be always running in order for sonic-pi to work.
2.2. Lxd container setup
If you are setting this up on a lxd container, besides of doing all above also inside the container, you will need some modifications to make realtime scheduling works. It is easier if it is a privileged container. You can edit the config with sudo EDITOR=nvim lxc config edit arch
. Here is mine
architecture: aarch64 config: image.architecture: arm64 image.description: Archlinux current arm64 (20220325_14:16) image.os: Archlinux image.release: current image.requirements.secureboot: "false" image.serial: "20220325_14:16" image.type: squashfs image.variant: default limits.kernel.nofile: "200000" raw.lxc: |- lxc.mount.entry = /dev/fuse dev/fuse none bind,create=file,optional lxc.mount.auto=cgroup:rw lxc.cgroup.devices.allow = c 116:* rwm lxc.mount.entry = /dev/snd dev/snd none bind,optional,create=dir security.nesting: "true" security.privileged: "true" (...)
The raw.lxc
part is the relevant part. On arch it is necessary to install alsa-plugins
in order for aplay -L
show any output.
2.3. Confirming things work
You might try to launch jack now, either with qjackctl or with jack -d dummy
, then launch sonic-pi. If you run into an error like: No such file or directory - jack_connect
you are probably missing jack-example-tools
as described here: https://github.com/sonic-pi-net/sonic-pi/issues/2994
The archwiki might also help in case of other problems https://wiki.archlinux.org/title/PulseAudio/Troubleshooting https://wiki.archlinux.org/title/JACK_Audio_Connection_Kit.
The way set it up on archlinux was using pulseaudio and jack on top, but you might not need them both in fact, just jack is required for sonic pi and a broadcasting software like darkice can use jack even though i haven't tested that. I just found it more comfortable to have both.
If sonic-pi launched great! If not, maybe try the lxd approach, I couldn't really get this working on ubuntu as well ;).
3. Broadcasting sonic pi
Considering you already have a shoutcast or icecast2 server to stream, here is the setup I've done for streaming audio from sonic-pi.
At first I logged into a graphical environment into the container to set things up using pavucontrol
and qjackctl
. It is possible to set this up from the tty using tools like pacmd
and whatever you can use for jack but if you can afford it, doing from the GUI is far much easier and faster in this case.
So my goal is to use darkice to stream sonic-pi's audio to an icecast2 instance. I can use commands like sonic-pi-tool run-file song.rb
or sonic-pi-tool evan "play :c5"
to test the stream. Darkice will be capturing the audio output from sonic pi and streaming it. So first create a darkice config like:
[general] duration = 0 bufferSecs = 5 reconnect = yes realtime = yes rtprio = 3 [input] device = pulse # Maybe you just use jack? sampleRate = 44100 bitsPerSample = 16 channel = 2 [icecast2-0] bitrateMode = cbr format = vorbis bitrate = 128 server = radio.dot.org.es port = 8000 password = passwordherehahaha mountPoint = playground.ogg name = Sonic Pi description = Sonic Pi REPL url = https://radio.dot.org.es genre = radio public = yes
Again you might want to change the input device to jack if you don't want to use pulseaudio. In my cause I opted to use pulseaudio to also stream from other tools that aren't compatible with jack.
Another thing to notice! This is a vorbis encoded stream. This means that silence results in no flux of data being transmitted, which the majority of players and browsers don't know how to handle it. To circumvent this I will be using a comfort noise generated with:
sox -c1 -n result.wav synth 10 sin 25000 vol 1
And you can play it in loop with something like:
mpv -loop result.wav
This should be inaudible but for some reason there is a perceptible noise on the stream still. I haven't found a way to fix that but if you adjust the volumes correctly it will become unnoticeable. It just has to be enough so that there is no silence on the stream and the players weirdly start playing some random cache or simply don't know what to do (because they think the server stopped streaming).
You could also simply stream in another format that doesn't have this problem, but maybe it would have bigger latency then?
3.1. Start broadcasting
Launch pulseaudio if you are using it:
pulseaudio --start
Then the jack daemon:
jackd -d dummy
Play the comfort noise:
mpv -loop result.wav
Start broadcasting:
darkice -c darkice.conf
And play something!
live_loop :bass do play :c2 sleep 0.5 play :c2 sleep 0.5 play :e2 sleep 0.5 play :f2 sleep 0.5 end
sonic-pi-tool run-file song.rb
4. The bot
I then created a irc bot using my own irc python library: https://github.com/matheusfillipe/ircbot I stole what matters from sonic-pi-tool source code:
import collections import html import logging import os import re import socket import sys import time from oscpy.client import OSCClient from oscpy.server import OSCThreadServer SERVER_OUTPUT = "~/.sonic-pi/log/server-output.log" logger = logging.getLogger() class Server: """Represents a running instance of Sonic Pi.""" preamble = '@osc_server||=SonicPi::OSC::UDPServer.new' + \ '({},use_decoder_cache:true) #__nosave__\n' def __init__(self, host, cmd_port, osc_port, send_preamble, verbose): self.client_name = 'SONIC_PI_TOOL_PY' self.host = host self._cmd_port = cmd_port self._cached_cmd_port = None self.osc_port = osc_port # fix for https://github.com/repl-electric/sonic-pi.el/issues/19#issuecomment-345222832 self.send_preamble = send_preamble self._cmd_client = None self._osc_client = None def get_cmd_port(self): return self._cmd_port def cmd_client(self): if self._cmd_client is None: self._cmd_client = OSCClient(self.host, self.get_cmd_port(), encoding='utf8') return self._cmd_client def osc_client(self): if self._osc_client is None: self._osc_client = OSCClient(self.host, self.osc_port, encoding='utf8') return self._osc_client def get_preamble(self): if self.send_preamble: return Server.preamble.format(self.get_cmd_port()) return '' def send_cmd(self, msg, *args): client = self.cmd_client() logger.info("Sending command to {}:{}: {} {}" .format(self.host, self.get_cmd_port(), msg, ', '.join(repr(v) for v in (self.client_name,) + args))) client.send_message(msg, (self.client_name,) + args) def send_osc(self, path, args): def parse_val(s): try: return int(s) except ValueError: pass try: return float(s) except ValueError: pass if len(s) > 1 and s[0] == '"' and s[-1] == '"': return s[1:-1] return s client = self.osc_client() parsed = [parse_val(s) for s in args] logger.info("Sending OSC message to {}:{}: {} {}" .format(self.host, self.osc_port, path, ', '.join(repr(v) for v in parsed))) client.send_message(path, parsed) def check_if_running(self): cmd_listening = Server.port_in_use(self.get_cmd_port()) logger.info("The command port ({}) is {}in use".format(self.get_cmd_port(), "" if cmd_listening else "not ")) osc_listening = Server.port_in_use(self.osc_port) logger.info("The OSC port ({}) is {}in use".format(self.osc_port, "" if osc_listening else "not ")) osc_listening = True if cmd_listening and osc_listening: logger.info("Sonic Pi is running, and listening on port {} for commands and {} for OSC" .format(self.get_cmd_port(), self.osc_port), True) return 0 elif not cmd_listening and not osc_listening: logger.info("Sonic Pi is not running", True) return 1 else: logger.info("Sonic Pi is not running properly, or there's an issue with the port numbers", True) return 2 def stop_all_jobs(self): self.send_cmd('/stop-all-jobs') def run_code(self, code): self.send_cmd('/run-code', self.get_preamble() + code) def start_recording(self): self.send_cmd('/start-recording') def stop_and_save_recording(self, path): self.send_cmd('/stop-recording') self.send_cmd('/save-recording', path) @staticmethod def port_in_use(port): with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sock: try: sock.bind(('127.0.0.1', port)) except OSError: return True return False @staticmethod def determine_command_port(): try: with open(os.path.expanduser(SERVER_OUTPUT)) as f: for line in f: m = re.search('^Listen port: *([0-9]+)', line) if m: return int(m.groups()[0]) except FileNotFoundError: pass @staticmethod def handle_log_info(style, msg): msg = "=> {}".format(msg) logger.info(msg) logger.info() @staticmethod def handle_multi_message(run, thread, time, n, *msgs): msg = "{{run: {}, time: {}}}".format(run, time) logger.info(msg) for i in range(n): typ, msg = msgs[2 * i: 2 * i + 2] for j, line in enumerate(msg.splitlines()): if i < n - 1: prefix = " ├─ " if j == 0 else " │" else: prefix = " └─ " if j == 0 else " " logger.info(f"{prefix}, {line}, {typ}") logger.info() @staticmethod def handle_runtime_error(run, msg, trace, line_num): lines = html.unescape(msg).splitlines() prefix = "Runtime Error: " for line in lines: logger.debug(f"{prefix=} {line=}") prefix = "" logger.debug(html.unescape(trace)) @staticmethod def handle_syntax_error(run, msg, code, line_num, line_s): logger.error("Error: " + html.unescape(msg)) prefix = "[Line {}]: ".format(line_num) if line_num >= 0 else "" logger.error(f"{prefix=}, {code=}") def follow_logs(self): try: server = OSCThreadServer(encoding='utf8') server.listen(address='127.0.0.1', port=4558, default=True) server.bind('/log/multi_message', self.handle_multi_message) server.bind('/multi_message', self.handle_multi_message) server.bind('/log/info', self.handle_log_info) server.bind('/info', self.handle_log_info) server.bind('/error', self.handle_runtime_error) server.bind('/syntax_error', self.handle_syntax_error) while True: time.sleep(1) except Exception as e: return e def eval_stdin(server: Server): server.run_code(sys.stdin.read()) def eval_file(server: Server, path): server.run_code(path.read()) def osc(server: Server, path, args): server.send_osc(path, args)
I probably could still remove more from there though.
And in the case of the lxd container I redirected the port since I wanted to run the bot in the host. Sonic pi listens on port 4557 udp.
socat UDP-LISTEN:4557,fork,bind=$CONTAINER_IP UDP:127.0.0.1:4557
That is forwarding udp packets coming from the container interface to its localhost, which is the address sonic-pi binds by default. You can see the container ip with lxc list
from the host or with ip a
within the container.
With that in mind I just had to write a simple and dumb repl using the server.run_code
method. Here is a simplified version of the bot's code:
import re from sonic_pi import NoteNotFound from sonic_pi import Server as PiServer from sonic_pi import convert_to_notes from IrcBot.bot import Color, IrcBot, Message, utils sonic_pi_users = {} sonic_pi_history = {} @utils.regex_cmd_with_messsage(r"^(.+)$") def all_msgs(args: re.Match, msg: Message): if msg.nick not in sonic_pi_users or msg.message.strip().startswith(PREFIX): return sonic_pi_users[msg.nick].append(args[1]) def convert(*args): transpose = 0 if args[0].isdigit(): octave = int(args[0]) if args[1][0] in ["+", "-"] and args[1][1:].isdigit(): transpose = int(args[1]) letters = "".join(args[2:]) else: letters = "".join(args[1:]) else: octave = 4 if args[0][0] in ["+", "-"] and args[0][1:].isdigit(): transpose = int(args[0]) letters = "".join(args[1:]) else: letters = "".join(args) logger.debug(f"{octave=} {letters=}") return ", ".join(convert_to_notes(letters, octave, transpose)) @utils.arg_command("pi", "Toggles sonic pi repl", f"{PREFIX}pi [command]- https://sonic-pi.net/tutorial.html") async def pi(bot: IrcBot, args: re.Match, msg: Message): args = utils.m2list(args) if args: sonic_pi_users[msg.nick] = [" ".join(args)] if msg.nick in sonic_pi_users: # append to history if msg.nick in sonic_pi_history: sonic_pi_history[msg.nick].extend(sonic_pi_users[msg.nick]) else: sonic_pi_history[msg.nick] = sonic_pi_users[msg.nick] # Apply template for i, line in enumerate(deepcopy(sonic_pi_users[msg.nick])): for match in re.findall(r"\$\{([^}]+?)\}", line): try: replace = convert(*match.split(" ")) sonic_pi_users[msg.nick][i] = sonic_pi_users[msg.nick][i].replace( "${" + match + "}", replace) logger.debug( f"Applying template at {i=} {line=} {match=}, {replace=} {sonic_pi_users[msg.nick][i]=}") except NoteNotFound as e: await reply(bot, msg, error(f"Could not find note '{e}'")) del sonic_pi_users[msg.nick] return await reply(bot, msg, "Your Sonic Pi repl is now off. Sending code to sonic pi...") server.run_code("\n".join(sonic_pi_users[msg.nick])) del sonic_pi_users[msg.nick] return sonic_pi_users[msg.nick] = [] await reply(bot, msg, f"Your Sonic Pi repl is now live at: {SONIC_PI_LIVE_URL}. Type {PREFIX}pi to turn it off and evaluate your code.") @utils.arg_command("convert", "Convert keyboard characters into sonic pi notes", f"{PREFIX}convert [octave] [±transpose] <letters> - Only qwerty layout") async def cmd_convert(bot: IrcBot, args: re.Match, msg: Message): args = utils.m2list(args) if not args: await reply(bot, msg, error("You need to specify an octave an a string to convert")) return if args[0].isdigit() and (int(args[0]) < 0 or int(args[0]) > 10): await reply(bot, msg, error("The first argument must be between 0 and 10")) return try: res = convert(*args) await reply(bot, msg, res) except NoteNotFound as e: await reply(bot, msg, error(f"Could not find note for '{e}'")) @utils.arg_command("pstop", "Stops sonic pi audio") async def stop(bot: IrcBot, args: re.Match, msg: Message): server.stop_all_jobs() await reply(bot, msg, "Stopping audio...") @utils.arg_command("paste", "Pastes your sonic pi code and clears your history") async def pipaste(bot: IrcBot, args: re.Match, msg: Message): if msg.nick not in sonic_pi_history: await reply(bot, msg, error("You need to turn on your sonic pi repl first. Use {}pi".format(PREFIX))) return await reply(bot, msg, paste("\n".join(sonic_pi_history[msg.nick]))) del sonic_pi_history[msg.nick] @utils.arg_command("read", "Read code from ix.io paste (or any raw text url)") async def readurl(bot: IrcBot, args: re.Match, msg: Message): try: server.run_code(read_paste(args[1])) except Exception as e: await reply(bot, msg, error("Failed to read paste: ") + str(e)) await reply(bot, msg, "Code has been read and sent!") # And you might want to define all those constants somewhere if __name__ == "__main__": utils.setLogging(LOG_LEVEL, LOGFILE) bot = IrcBot(HOST, PORT, NICK, CHANNELS, PASSWORD, use_ssl=PORT == 6697, dcc_host=DCC_HOST, dcc_ports=DCC_PORTS, dcc_announce_host=DCC_ANNOUNCE_HOST) bot.runWithCallback(onconnect)
You can see the full code for this bot at: https://github.com/matheusfillipe/mpd_irc_bot but it does much more than being a sonic pi repl now. It manages the whole radio playlist queue allowing users to submit music with url's, youtube links or with the irc DCC file transfer protocol.
5. Conclusion
It was a nice experiment but sonic-pi, despite being a fantastic tool for playing around on your desktop, is not really well forged for the server setup. It relies on too many stuff and chances of something going wrong are high. I went with it for this experiment because there isn't a musical language with an easier to understand language and just as well as documented. There is overtone: https://github.com/overtone/overtone but come on, who even knows clojure! Also, sonic-pi comes with so many preset samples that I didn't even consider creating a bot command to allow users to use their own samples.
I imagine it would be nice to have a real web app using a setup like this(https://in-thread.sonic-pi.net/t/sonic-pi-in-a-web-browser/1563/5)! Not sure how to reduce the latency though. I am having like 6 seconds of latency with this jack -> darkice -> icecast setup. Here is an interesting similar project: https://github.com/merongivian/negasonic
Comments
Be the first to comment!