Time series statistics are tricky

In one of my last projects at BigCo, I had to implement a bunch of metrics monitoring and alerting for our services. This was years ago now but every so often I think back on it and how it drove me a bit nutty... Here's a short write up of some of the problems I remember, from just considering one of its aspects: metrics on a single API endpoint request.

People seem to think statistics like "average requests per minute" or "p99 response times" are straightforward metrics that can be pulled with a simple query. But they can be quite complex, and the results can be very misleading depending on how the events and queries have been defined.

So again, starting with something simple, we just want a requests counter for a single endpoint. How you define this counter changes how you interpret its data. One approach is to emit an event each time the endpoint is hit, logging every single request. Alternatively, you could maintain an asynchronous counter that emits its value at fixed intervals (every minute, say), incrementing only when new requests come in.

See Full Post and Comments

Thanksgiving Approaches

I was listening to a bit of my grandma's funeral service recording. Not yet ready for my mom's, but as next month is the 10 year anniversary, I'll do hers then. My mom only did a short segment for grandma, to read a poem, that apparently she read every Thanksgiving at grandma's? But I don't remember that, or the poem. My memory is so shot on these sorts of things. It's embarrassing.

The poem title is "Family Ties", a quick search found its text here and written by Allie J. Hilder in 1954. Though my mom spoke a few differences: "golden memories" instead of "olden moments", which I think is better due to the "silver" later on. And she used "Lord" instead of "God".

Anyway, here's the complete text, using my mom's pair of differences rather than the link's text. Maybe I'll have to bring the tradition back, and at least read it to myself. Alas, I didn't really take up the caring about family bits of personality which my grandmothers on both sides had so much in abundance.

See Full Post and Comments

Happenings

Been in a slump the last several months. Going to try an experiment, that I suspect will fail, of writing down daily or at least weekly blurbs here of things I've been up to, like things I've been learning or doing. Or if nothing else, maybe some food I've been eating or music I've been listening to. In short, a public journal, but without trying to be super detailed -- but maybe some entries will be TMI anyway and I suppose I'd rather go for more details than fewer if the thing is triggering some thoughts. (Some days have been written in a sort of micro-blogging style. Not logging everything, but more than I thought I would... Need to try and keep things past-tense though, only update what I actually did, not what I'm currently doing or planning.) All entries will go in this post in reverse-chronological order (so updates are at the top -- note that "today's" updates may not be complete when you first see them). We'll see how long I keep it up. Made up a few terms for my own book-keeping on micro-goals, they might translate to something like "drank a liter of water" or "read a page in a book" or whatever, sometimes same word for different things.

9/14/24 - 9/15/24

Some programming. Some stupid arguments online. Some new games.

9/13/24

More system maintenance. Finished Risen 3.

See Full Post and Comments

Japan Trip 4 and 5

Previously.

I went to Japan again a couple more times. But nothing too exciting to report here. The 4th time was indeed primarily for the reason alluded to in the previous post, seeing my favorite idol have her birthday concert. I ended up helping with the preparations, too, which was a fun experience. I had a good time, but I was there for a while, basically from December 4th through January 16th. (Originally Jan 11th but I extended my stay for an extra weekend to see her a few more times... The plane ticket change didn't increase my fare.) So about 6 weeks. I was happy to be going home.

Of course, on the day I flew out, I heard that her group was disbanding with a final graduation event in a month. My friend is funny, he said when I got home, "So I guess you'll be going back soon right? You'll regret it if you don't!" He was right, of course. So I went back.

See Full Post and Comments

Possible errors and downtime

I've migrated my blog and other things over to OVH, leaving Amazon's EC2 behind. The DNS switch has been made, though my local machine keeps trying to talk to the old server I have shut down to encourage not caching the old IP. I also had to upgrade to PHP 8.2 and recreate my database. I've fixed a few errors so far, but there might be more. Anyway, just a warning as the title says in case anyone is experiencing the issues.

See Full Post and Comments

Parting is such sweet sorrow

One of the few Shakespeare lines I like.

Wrapping up a trip soon. I'm ready to head back home and resume something more resembling the hikikimori lifestyle.

It was a good trip, I'll write up a bit more about it later. Part of me wants to stay, but it's time.

See Full Post and Comments

Leaving a mark

Most people in all of human history didn't leave much of a mark. The greatest, longest lasting contribution for most of them was simply having children.

So it's actually rather arrogant, but goes well if you have the ambition, to try and make it your life goal to leave a big mark somehow. This drive isn't a bad thing, without it much progress would not have been made.

But one shouldn't feel bad if one lacks the drive. We don't all need to save the world, we don't all need to fill our every waking hour trying to make some difference. It's ok if you do no better than the average human ancestor in the last few hundred thousand years -- it's ok to do rather worse, too. And if you don't plan to have kids, or don't see that happening for other reasons, you probably will do worse. Fine.

See Full Post and Comments

Older Posts