Brief notes on the mess of scripts involved for Coyo YouTube Post Notifications

I don't exactly know which audience I should write this for, so I'll try just giving some high level notes and anyone curious can follow deeper if they want. If you have any questions feel free to bring them up. Anyway, the context behind this post is that because YouTube won't do notifications of community posts for some reason (though it does do notifications of member posts...) I decided to solve the problem for myself. This initially resulted in a quick and dirty script here.

All the script does is this:

1. Open up a new instance of my Firefox browser, in "headless" mode so that it doesn't actually show up in my task bar and interfere with any other windows open on my desktop
2. Navigate to Coyo's community tab
3. Check to see if there's any new message or not from last time it checked.
• If there was a new message, notify about it.

I have the script arbitrarily set to run once every five minutes, so there's not too much delay between a post being made and seeing it. This is done with a simple bash loop: while true; do sbcl --script coyo_yt_posts.lisp; sleep 300; done But it could just as well be done by modifying the Lisp code instead, or setting a cron job. (Edit: the latest version of the script on gist now has a loop in the Lisp code and is a bit more robust to some errors.) I could also compile the script into an EXE binary, but there's no need here. The way it checks if a post is new or not is also stupidly basic: it just saves the contents to a local file, and compares against that each time, overwriting it with the new post if it's different.

Since it occurred to me that perhaps other Coyodachi might want these notifications too, I put up the page you likely came here from. I looked into how web push notifications work, decided it's not really worth the effort to implement them for this unless there's a demand, and instead opted for a simple email system. This also resulted in modifying the above-linked code to add an additional step: notify the web page of a new notification so it can send out emails about it. That's what the commented out dex:post ... stuff is for.

I love Common Lisp, so I try to program everything I can in that language these days, but this domain's already set up to trivially run PHP so I figured why not make sure I can still write PHP if I have to. It was a good reminder for why I don't enjoy the exercise anymore. Anyway, the resulting set of code can be found here.

All this hacky PHP script does is actually 4 things packed in one. (A better design would split things up.) By default, it just serves the information on the normal page and lets users enter an email address to sign up. The request to add the email address goes to the same page, so that's the second of its responsibilities. I do have a MySQL database running, but for fun and ease I instead decided to use a Sqlite database to keep track of saved email addresses.

The third thing this script is responsible for is to receive signals from my local Lisp script, which tells it (using a secret) a new post has been made and that it can send out the emails, which it proceeds to do. In each email is an unsubscribe link, so the script also handles unsubscribing if that unique and unguessable link is visited.

Since I'll be somewhat surprised if more than one or two people besides myself even use this, I haven't put much effort into designing things professionally. Email validation is as basic as checking for an '@', and there's currently no anti-abuse measures (hopefully there won't need to be). Still, it seems to work alright, and didn't take much effort.

This code, hacky as it is, is free software you can do whatever you want with. With the exception of the email template in EasyMail.php which is MIT-licensed, all this code is released into the public domain as described by the Unlicense. The EasyMail script I included just for the sake of completion -- it's just a fancy wrapper around PHP's mail() function, I've been using it unmodified for many years now. I modified it a bit for this project as I discovered it did not handle Japanese/emoji characters well. The easiest approach was to just base64-encode the email body.

Behind the scenes, PHP is calling out to a Common Lisp binary I made whose source you can find here. All that binary does is talk to Amazon's simple email service which does the actual real work of sending emails.

If you want to try running things yourself, starting with just the local script coyo_yt_posts.lisp you will need:

1. A Common Lisp implementation -- I recommend and have only tested with SBCL (you want the amd64 bit version for your OS)
2. Have it setup with QuickLisp for loading libraries
3. Selenium Server running -- I recommend the Java version
4. If you don't modify the code, it expects to use Firefox (though it can use Chrome just fine if you do modify), but that means you need to have GeckoDriver somewhere that Selenium can find it, i.e. on your PATH
5. If you're not on Linux, you'll need to modify the "notify-send" command to something else applicable for your platform

If you want to run the PHP service (it can also be done locally, you don't need to pay for web hosting) you'll need to setup PHP which is beyond the scope of this blog. A common setup is still the "LAMP" server, so google that. Edit the scripts to have consistent secrets and URLs (uncommenting the dex:post from the Lisp code) and you'll have what I have.

A final closing note, I'm very much a fan of the Bruce Lee philosophy applied to all sorts of learning:

"Research your own experience; absorb what is useful, reject what is useless and add what is essentially your own."
-- Bruce Lee, Wisdom for the Way

This mess makes use of Common Lisp, simple text files, bash loops, using an external program for desktop notification sending, using the JVM for Selenium, PHP, Sqlite, HTML, CSS... It may seem intimidating! But it's just the result of my own experience. You can shed any part of it and do it some other way if you want. There is something to be said for doing things in a homogeneous environment (per my preference, Lisp all the way) and professional environments are strongly biased that way. But there's a ton of useful software out there and if all you're interested in is using it, even from even more software rather than manually, you don't have to spend much time learning how, and often no time learning how it actually works. Frequently this is better than rewriting it yourself.

---

Posted on 2023-07-01 by Jach

Tags: lisp, php, programming, services, vtuber

Permalink: https://www.thejach.com/view/id/417

Trackback URL: https://www.thejach.com/view/2023/7/brief_notes_on_the_mess_of_scripts_involved_for_coyo_youtube_post_notifications

Back to the top