Alloy 0.10.0 graphical log streaming is live

You can now get some insight into what Alloy is thinking by watching a live graphical representation of its logs as it plays on Tiltyard.

I decided to do this primarily as a way to better understand my player and UCT. I had moved Alloy to run on a dedicated computer so it can run continuously, but this made it difficult to view information directly from the program. Streaming this publicly makes it easier to see regardless of where I am, and it also allows others to watch this information and gather their own insights about game AI.

By moving from a few lines of logging at a time to a full screen of graphical information, I can also afford to include a wider array of information useful for debugging or for understanding my player's weaknesses.

I now expect to do a lot of tweaking to make the UI useful and easy for a trained eye to parse quickly, so I won't explain all the current aspects of the UI. (Two things I should point out are Alloy's role index in the top row -- that's zero-indexed, so 0 is the first player in the game, 1 is the second, and so on -- and the E[v] graph in the top left that tracks Alloy's current expected value.)

Instead, I want to explain today the technical aspects of how I did this for anyone interested in applying this to their own players.

The player's backend process generates logs in a structured format, taking advantage of a library I wrote called Escape Rope. This makes it relatively easy to go back and forth between objects and "ropes" (each rope is a string or a list of ropes) and between ropes and individual strings. Those strings are then placed in the logs in lieu of custom, non-machine-readable log lines.

The UI is displayed by a Java program in a separate process that reads these logs as they come in using the Apache Commons IO Tailer class. (Warning: Before version 2.5 of the library, Tailer does not correctly handle files containing any real variety of characters, such as the outputs of Escape Rope.) It uses one master log file to determine when a new game has started, so it can switch to new games automatically. The log lines are converted back into their original objects for easy handling.

The display itself is just a giant JPanel with a custom paint() method. As it doesn't need to accept any user input, I can get away with using fixed coordinates and low-level graphical commands in lieu of actual Swing components, which are difficult to arrange. (This is also likely easier on the CPU.)

This is all running on a Linux box, so as far as I can tell there's only one good option for streaming it: OBS Studio. (I haven't found any libraries that would allow me to directly stream video to a website from Java.) Unfortunately, this is GUI-based, and I'm trying to run this on a headless machine that I can restart and control remotely. Fortunately, I can do the required setup once with a monitor attached, and do subsequent runs from the command line using the --startstreaming argument. The one non-obvious aspect of my OBS setup is that I reduced the frame rate from 30 FPS to 10 FPS to try to reduce my bandwidth consumption.

Now we get into the really "fun" technical difficulties. Take my explanations below with a grain of salt, and my solutions with a shakerful, because I don't know nearly enough about these aspects of the operating system to speak with any authority.

First is an easy one: OBS Studio requires OpenGL 3.2 or higher, which generally requires a graphics card. No big problem there: I repurposed an old graphics card from another machine for that purpose. (And by "old", I mean my main desktop now has a shiny new graphics card.) By default the machine will use an open-source driver called Nouveau for Nvidia cards, which only supports earlier versions of OpenGL; but after installing the official drivers from the nvidia-current package, it can handle OBS Studio just fine. (The mesa-utils package includes the glxinfo command, which is very useful for determining the driver in use and the supported OpenGL version.)

Now we start running into a bunch of problems that are harder to solve, because our operating system is too clever and won't load the resources needed for graphical programs if it doesn't think anyone is going to use them.

First that means installing a bunch of software that isn't included in the Ubuntu Server distribution by default. That includes, at the minimum, xorg and a window manager of some kind. I used fluxbox, but I wouldn't be surprised if all this were easier just using Gnome and GDM, e.g. by just starting from an Ubuntu Desktop installation. (I hear that allows you to set up automatic login, which bypasses some of the later steps here.)

Next we have to trick the system into thinking it is hooked up to a monitor. It appears a software-only solution for this exists, but as I was having problems beyond that, I just got a professionally made dummy monitor plug. (It's actually quite possible to make one of these yourself, and is one of the cooler hacks I've seen to fix a software problem.)

Now we have to actually get the X server running. Unfortunately, it doesn't want you to start it (via the startx command) over SSH, and will fail if you try to do so. Since I'm too lazy to stick an actual keyboard and monitor into the machine every time I restart it, instead I have the computer's startup scripts do this for me. This is fraught with additional problems, as the X server has its own permissions system that will cause failures if it is run by the wrong user -- and the way I'm running things on startup will run it as the wrong user. But I did find a way to hack around this.

So I have a script in the /etc/init.d/ folder called startx-custom, copied from "skeleton" and with the following line at the end (you'll also need to chmod a+x the file to make it executable):

XAUTHORITY=/home/<my username>/.Xauthority startx

Then, to have this run during normal startup (runlevel 5), I have a symlink to that script in /etc/rc5.d/ named S05startx-custom. (Naming it S05 has it run after all the other scripts in that folder, which are called S04 or earlier in my case.)

I mentioned that running startx here causes the X server and fluxbox to be run by root. (This could possibly cause security concerns, but I'm not really worried about this particular server.) There's an "X authority" file that the server and any applications that want to run against it need to agree on, which is usually the problem in this case; the X server will pick some random file that's inaccessible to my user if it is run as root. As you can see in the script above, I instead have it use ~/.Xauthority for that purpose, which is what it would be if my user were running it, and so all my applications will use it. This leaves me with one, much more solvable problem: After startup, the ~/.Xauthority file will be inaccessible to my user due to being owned by root. This is solved by a simple manual command: sudo chown <my username> ~/.Xauthority. (This doesn't work when I stick it in the startup script for some reason, but I don't mind typing in one extra line over SSH.)

After that, I can run my programs in screen sessions by telling them the X display to use:

DISPLAY=:0 ./gradlew runViz
DISPLAY=:0 obs --startstreaming

(If I had an actual monitor hooked up to the computer, this would cause the programs to pop up on that monitor.)

So with those steps, I was able to get the stream up and running from a headless Linux box, offering more insights into my new player's flaws and shortcomings. Hopefully I can use it to improve the player a bit before I start sinking time into my next bright idea.