Results

Inclination matters

March 6

Postrel’s argument about dynamism, evolutionary processes, and dispersed knowledge has direct relevance to technological change. She suggests that technological progress, much like biological evolution, is unpredictable and emerges through decentralized innovation rather than central planning, mirroring Hayek’s insight that no single entity can have enough knowledge to foresee or control all the variables involved in invention and innovation.

~ Lynne Kiesling, from Progress and Its Enemies

slip:4usupo6.

For a long time—perhaps all of my life until just a few years ago—I would have said I was completely on team Continuous Forward Progress; I would have said that change is good simply because it implies forward progress. But now I’m really seeing the value of “if this isn’t nice, I don’t know what is.” In most situations, I’m now feeling that leaving something just as it is would be great. The article is making a point about people who are against change on principle (although I’d wager most such people haven’t thought of it as a principle) and that’s not me either. I’m perfectly fine with change. But this is nice too.

ɕ

Incredible

May 13, 2024

The linked post is an Apple-specific, nerdy deep dive related to technical details in recording. In the specifics it’s about people ripping on Apple for certain claims about something being “shot on iPhone.”

I much prefer the other way of looking at this same rig, which is that it is incredible that this entire professional workflow is being funneled through a tiny sensor on basically the same telephone I have in my pocket right now.

~ Nick Heer from, How Apple Shot ‘Let Loose’

slip:4upoli1.

Heer is so spot-on here. Hear! Hear! I love this sentiment. When I take a moment to mentally zoom out, I’m knocked out by the incomprehensibly-advanced super-computers which are now everywhere. If this isn’t nice, I don’t know what is.

ɕ

Recognizing

April 29, 2024

There is a phrase I like to trot out: “Any day at the crag.” And you need two details for this to make sense: First, that a crag is any outdoors place where one goes to climb rocks. Second, the part left unsaid is that any day at the crag is better than any other day. Thus, any day at the crag. Yes, despite the litany of things that normal people would list as negatives at the crag.

To be clear, by “appreciate” I don’t mean “enjoy” exactly, although you might also enjoy the experience. I mean “recognizing the unique or worthy qualities” of the experience itself: the texture of it, the aesthetics of it, the heft of it, the heat of it, the poetry of it, the poignancy of it — whatever strikes your sensibilities when you pay attention to what’s happening.

~ David Cain from, Appreciate What Happens, as a Rule

slip:4uraae2.

And yet, despite that litany, the density of “if this isn’t nice, I don’t know what is” moments is much higher, any day at the crag. Dappled sunlight. A cool breeze in the shade. A view. Friends. Food tastes better. Sleep is less troubled. You don’t have to climb (or go to a crag), but you do have to find your something.

ɕ

Control

June 23, 2023

At one point I made a serious attempt at gratitude journaling. I learned that the many moments of delight I encounter on any given day don’t stick in my memory. In those moments I am aware of the experience (I really do think “if this isn’t nice, I don’t know what is” often) but it drifts into the forgotten realms. I don’t randomly have thoughts like, “hey Craig, remember that delightful thing from that other moment?” I have to intentionally pause to make a space for those thoughts to appear.

Humans by nature have a strong desire to control and predict. We want to know what happens at the end of the story, and we focus on those things we can measure and easily influence.

~ Cierra Martin from, «https://www.gapingvoid.com/blog/2022/11/21/more-dancing-less-box-checking/»

slip:4ugabo2.

Can one remove a desire? This desire which I definitely have, has not abated through familiarity (something which I believe can cause a desire to fade.) I think the only way is to connect the many experiences which did not go through my control-it desire, and led to happiness. “See brain, we got to delight and there was not even an attempt at control there.” Pausing in moments of delight, as it were, to ask, “well… how did I get here?“

ɕ

Work ethic

May 24, 2023

Hedonistic adaptation ensures that I continuously cycle back and forth between, “If this isn’t nice I don’t know what is!” and rage-quitting all my self-assigned should’s. Two things help temper my intemperance: Journalling provides me with some—albeit subjective—perspective, and reading about the reality of people’s actual lives and work ethics relaxes my self-criticism.

A truer answer would have been that he was fiercely private and deeply caring. He often let other people talk, entering a conversation with a single considered sentence. He didn’t smile unless he was really pleased, and his biggest laugh was a small chuckle. His eyes would squeeze shut and his head would tip back and after he chuckled, he would look at you with delight.

~ Alison Fairbrother from, Lessons in Writing and Life from My Grandfather, E.L. Doctorow

slip:4ulile1.

Without Internet cheating, I can’t name a single one of E. L. Doctorow’s works. But I have this diffuse idea that he is (was?) A Real Writer. Someone who got things written, and maybe had a few ideas worth sharing. Darn it if reading Fairbrother’s piece didn’t tug at the ‘ol heart strings, and I might have gotten something briefly stuck in the corner of one of my eyes.

But I came away with a recalibration: I now have this diffuse idea that he was A Real Writer, got things written, had a few ideas worth sharing, and it was made possible by his family and his work ethic.

(Also, maybe help this autodidact out by hitting reply and telling me one thing of Doctorow’s I must read.)

ɕ

Human collaborators

May 14, 2023

And so we did the math, and it was really at the same time that I had lost [my idea] that she had gotten [her idea]. And we like to think that the idea jumped from my mind to hers during our little kiss that we had when we met. That’s our magical thinking around it. But it’s — there is no explanation for that other than the one that I’ve always abided by, which is that ideas are conscious and living, and they have will, and they have great desire to be made, and they spin through the cosmos, looking for human collaborators.

~ Elizabeth Gilbert from, The Muse Strikes Again

slip:4usete5.

Obviously that’s not how any of it really works. But it is a sublime, inspiring idea! I know that if I focus (or worse, fixate) on where some idea came from it’s easy to lose the delight of the overall thing. This cosmic perspective from Gilbert reminds me to simply take things and run with them. If I can. If this isn’t nice, I don’t know what is.

If I can’t run with it, well, that’s okay too. It is simply okay. But, if I still need some self-convincing, that cosmic perspective gives me the comfort I need to let go.

ɕ

I probably need to work on this

March 19, 2023

My life is always better when I treat myself as if I were someone I care about.

~ Hugh Hollowell from, «https://www.soverybeautiful.org/how-we-treat-ourselves/»

I’m really good at digging in and schlepping through the hard work. I’m really good at figuring out how to make three strange pieces fit together so these four people can make some progress on those five incompatible goals. Lift heavy things. Break a sweat. Get shit done. Go above and beyond. Get this letter to Garcia. Abuse English.

Know what I suck at? Treating myself as if I were someone I care about. Can I say, “no, thank you,” to some opportunity because I’m already overwhelmed? Can I take a nap in my hammock, without first spending significant time weighing the merits of giving in to passing out from exhaustion, versus just. work. a little. more. Can I choose to go do that fun thing with my friends, when my weekly plan says I should get some peak heart-rate workout time today? I’m often heard preaching about self-care, taking time to look back and think, “if this isn’t nice…” but, can I actually do those things?

ɕ

Stingy with positive reinforcement

August 5, 2022

Here’s something I’ve noticed about myself: If I read something great, I’ll sometimes write a short comment like “This was amazing, you’re the best!” Then I’ll stare at it for 10 seconds and decide that posting it would be lame and humiliating, so I delete it go about my day. But on the rare occasions that I read something that triggers me, I get a strong feeling that I have important insights. Assuming that I’m not uniquely broken in this way, it explains a lot.

~ “Dynomight” from, So you’re thinking about writing on the internet

slip:4udyie1.

I too have this tendency. In recent years I’ve been actively working on my own version of “See something. Say something.” as part of my changes to achieve results. My version is that nice things must be said out loud. No more sitting on the positive thoughts; Yes, I need to squish my incessant critical commentary. Dial that down, please. But I also need to practice letting out the good stuff too. Nice shirt. Smooth movement. This food is delicious. It’s so insanely comfortable here. Thank you for making this come together. If this isn’t nice, I don’t know what is.

ɕ

Grit

July 11, 2022

(Part 72 of 72 in series, My Journey)

Don’t let ease tempt you. Don’t fall for its false promises. What you gain in ease, you lose in meaning. What you gain in ease, you lose in excellence.

~ Hugh MacLeod from, «https://www.gapingvoid.com/blog/2022/07/08/follow-the-yellowbri-road-to-greatness/»

slip:4ugabo3.

This topic came up today in an outdoor Parkour class. Being outside, training, sweating, and overcoming challenges with friends old and new is always a treat. (“If this isn’t nice…“)

ɕ

Maybe I should walk back?

November 2, 2021

As autumn settles in where I am, I’ve been looking ahead to winter with longer nights, brisk days, etc.. I also looked back at the shape I’ve been in in years past. I’m not lamenting, “if only I had my youth back.” Rather, just thinking about health, movement, and what would be the minimum effective dosage of some exercise to move me in the direction I want. (That DuckDuckGo link should make you wonder why a medical-sounding phrase is used most relating to exercise not medicine, and strength training in particular.)

Sometimes—by which I mean any time running comes up—I say that running is both the best thing for me, and the form of activity I hate most. Both of which are untrue. What’s actually best for me is zone-2 aerobic exercise and that’s sometimes what I get when I run. It’s best for me, because that is the main driver of base fitness until you get well up into being a competent athlete. But usually, being quite over-weight at the moment, any running drives my heart-rate above the surprisingly low/slow zone-2. The second part about hating it is also untrue. It turns out that one time—the one single day apparently—that I was ever in shape, I enjoyed running. I was walking, the weather was beautiful, and I had an irresistible urge to run, (and so I did.) But, literally, that happened once.

Anyway. It’ll suffice to say: I spent a few weeks recently thinking about going full-on nerd with zone-2 training. To do it right requires planning, scheduling, and—sources vary—between 150 to 180 minutes exercising each week. And warm-up and cool-down time are not included in those weekly times. Honestly, the deal-breaker was I’m seriously pissed at FitBit, (and their watches are useless without a FitBit account,) and I refuse to spend many-hundreds on an Apple watch. Also, my $30 Timex is nicer, for my definition of “nicer.”

My thinking continued, and eventually I thought: I should just walk back from Mordor.

…except this time I’m not going to bother trying to track the actual mileage. Just walk as many days as I can. Listen to some podcasts some of the time. And basically just stroll along thinking, “If this isn’t nice…“

ɕ

Last vestiges

September 4, 2021

It can be hard to say no. It means refusing someone, and often it means denying yourself instant gratification. The rewards of doing this are uncertain and less tangible. I call decisions like this “first-order negative, second-order positive.” Most people don’t take the time to think through the second-order effects of their choices. If they did, they’d realize that freedom comes from the ability to say no.

~ Shane Parrish from, Break the Chain: Stop Being a Slave

slip:4ufobe1.

I think the “slavery” [to things, to money, to “more”] metaphor is inappropriate, but philosophers from Epictetus and earlier have been using it, so it’s entrenched. “Freedom” is mentioned in the pull-quote, and the metaphor also appears in the article. None the less, it connects a few different ideas together and gives good guidance if you’re new to the ideas. (Or if you could use a wee refresher.)

For me, the last vestiges of the yearning—as Wu Hsin put it—is the yearning for experiences. I am quite often restless. I often joke: “I do not idle well.” In my series on parkour-travel I even mentioned the idea of, when spare time exists, move towards the next scheduled-thing, and kill time there. I believe this yearning springs from my bias to action. As a counter-practice, I like to pause—often seemingly randomly—to remind myself: If this isn’t nice, I don’t know what is.

That phrase can get tossed around lightly, but there’s deep wisdom in it. Once I understand that this is in fact nice, right now, then when I realize that I wasn’t—just then, in the moment—feeling how nice it is… then the second part of the phrase has power: I don’t know what is. Put another way:

If I know what is nice, then this is.

ɕ

Becoming a Supple Leopard

July 10, 2023

…is both the title of a book, and a thing I’d very much like to do. What’s stopping me?

It’s not genetics, because that only sets the boundary parameters. Sure, I’ll never literally be a leopard. But the set of genes I’ve been dealt seem pretty choice. Bonus, I can even change my genetic expression. So genetics is not what’s holding me back.

There are two things holding me back: My mindset and knowledge.

Mindset — I like to think of it like this: See this body? This is the body which results from all my choices and my mindset up to this moment. I don’t want a different body so that I can do this or that. (Well, I do but that’s exactly the problem.) Instead, I need to make better decisions. Here are a few ways that I use to steer my life…

  • “I’m not currently able to do that. To do that, I would first need to work on this, strengthen this, and learn this other skill.” (Never simply, “I can’t do that.“)
  • “That isn’t a priority for me now.” (Never simply, “I don’t have time for that.“) Saying, “Sleep isn’t a priority,” or “Healthy eating isn’t a priority,” sorts my mindset out quickly.
  • “I am the sort of person who…” …is barefoot, until I have a reason to add things to my feet. …goes to bed early and regularly. …enjoys spending time preparing healthy meals. …is willing to say that isn’t a priority so that I can have a larger yes for things which are important to me.

Knowledge — There are many things which are a priority for me. Learning everything about each of the fields of human biology, physiology, kinesiology, nutrition, etc. is not a priority. I’ve made great strides in figuring out solutions to many of my problems, but it’s too enormous of a knowledge space for me to learn everything in every field.

Years ago (h/t Jesse!) I first saw a copy of Kelly Starrett’s book Becoming a Supple Leopard. It was an impressive book, and was well recommended. But I was still at a place in my journey where I wanted to carve my own path, and went on my way trying to figure everything out on my own. But no more!

Recently (h/t Andrew!) I was gifted a big, beautiful 2nd edition of the book. Which dovetails nicely with my no longer wanting to figure everything out on my own. So I’ve been diving into Starrett’s Becoming a Supple Leopard.

The third and most notable problem with our current thinking is that it continues to be based on a model that prioritizes task completion above everything else. It’s a sort of one-or-zero, task-done-or-not, weight-lifted-or-not, distance-swum-or-not mentality. This is like saying, “I deadlifted 500 pounds, but I herniated a disc,” or, “I finished a marathon, but I wore a hole in my knee.” Imagine this sort of ethic spilling over into the other aspects of your life: “Hey, I made you some toast! But I burned down the house.”

~ Kelly Starrett from, Becoming a Supple Leopard

I’m still reading the entire book-worth of information in the first part of the book. Plus, the middle parts are an encyclopedic compendium of gargantuan proportions with hundreds of mobility exercises. I skimmed through all of it, and resigned myself to never being able to try, let alone learn, all of them in a systematic fashion. Instead, in the back of the book there is a 14-day system for cherry-picking things to do, and that is the thing I’m digging into. In fact, I expect I’ll simply repeat the 14-day thing (changing what specific activities I’m picking) until I become bored or a supple leopard.

To make that a little easier, I made this PDF so I could print and write directly on it:

BASL-14-day-Overhaul-85×11-1Download

ɕ

Who we are

May 1, 2023

The Stoics believed that, in the end, it’s not about what we do, it’s about who we are when we do it. They believed that anything you do well is noble, no matter how humble or impressive, as long as it’s the right thing. That greatness is up to you—it’s what you bring to everything you do.

~ Ryan Holiday from, Discipline is Destiny: 25 Habits That Will Guarantee You Success

slip:4urydi1.

Depending on where you are on your own journey, this could be the greatest 25-item list you’ve ever seen, or it could be 24 items of hogwash. How great is that? For me, it’s the one about being kind to oneself which I need most to let sink in farther. Every absolute rule, every simple guideline, and every pithy virtue becomes problematic when taken to the extreme. It’s almost as if *gasp* life is complicated, and I’m a complex person.

I feel like I’m living in the negative. My life isn’t a passing timeline of “this is nice” punctuated with some stuff that qualifies as work, chores, and maintency-things. Instead, I feel like any time I’m in a span of “this is nice”, I’m on borrowed time. It’s is always “this is nice, but…” followed by something I feel I should be doing just as soon as I’m done loafing. It’s as if my personal demon is relaxing, just out of sight at the bar as I loaf here on the veranda, but still dutifully keeping track of exactly how long I’ve been loafing. I continuously feel like things will go better for me (in the way mobsters would say that) if I choose to stop loafing rather than waiting to see how long I can get away with it. That’s not healthy and thus my awareness of the need for self-kindness.

ɕ

What actually is the problem

April 21, 2022

Every obstacle that we normally think of as a problem to be fixed … every “flaw” in ourselves or others that we judge as something to be fixed … what if we can pause, find stillness, and get curious instead of trying to fix?

~ Leo Babauta from, Stillness & Curiosity

slip:4uzeeo1.

Any day that Babauta gets me thinking is a good day. (If that isn’t nice, I don’t know what is.) I’ve gotten pretty durn good at the “pause”, and the “find stillness”, parts. I also believe I have the “wait but why” curiosity bit figured out, since it has always been with me. It’s that “trying to fix” part upon which I’m perpetually stuck. And I get “particularly stuck”— “particularly stuck” aren’t the right words… if I could find the right words or word, I would use it instead. “Ensnared” is close. Or, have you ever gotten caught by a single thorn while out walking or hiking? That one thorn isn’t going to do too much damage if you stop quickly. In an instant, that one thorn becomes the laser focus of all of my attention. I really feel like I should be able to find the right word to fix that sentence.

Well, that’s curious.

ɕ

Inconsistent yet persistent

February 7, 2022

TK is an all-around mover, a dancer, rock climber, traceusse and earned her degree in athletic training. In addition to her movement practices, TK is a certified authentic Tantra instructor, teaching holistic healing of body, mind, spirit and sex. TK considers herself a sex activist and is the founder of LoveCraft, a sexual coaching and empowerment collective.

Tantra was the obvious place to begin since we were surely going to end up talking about tantric sex. My fear was that most people’s—myself included—knowledge of Tantra would be something to do with the artist, Sting. We immediately agreed that leaving the world only knowing about “men in linen pants” would be a disservice. “Tantra means, literally, to weave light and sound with form, the light being visualizations of your chakras in your body, sound being chants that you’re making, and then the form being your body, your physical body. That’s it, in a nutshell. The way that often looks is meditating. The way a lot of people do that is they’ll meditate and then have sex; they’ll meditate during sex; they’ll meditate on their own without any sex. Yeah, that’s kind of that, which means nothing, right? It’s like a, ‘Cool, and then what?’ which is what got me into having a coach.” — ~ TK from, ~4’40”

(more…)

How does one take notes…

December 27, 2021

…when the goal isn’t to end up with a pile of notes?

There are many scenarios where, over time, I do want to end up with a collection of notes. This is straight forward; start taking notes, and keep them somewhere. Bonus points if you review them, or use them as reference, or do anything with them.

But what if I have a scenario where I want to “do a better job” but I don’t care at all about the notes themselves. Suppose you have a regularly scheduled recurring meeting, but you don’t need a historical collection of notes. In fact, suppose you don’t actually need notes, but you think: It would be nice to know what we did last time, so we can follow-up next time.

And so I’m thinking this would be easy. I’ll just have a pile of notes (physical, digital, whatever) and I’ll go through them and … wait, what, actually? Recopy them? gag, that’s tedious. How many do I keep? How long do I keep the old ones? Here’s what I came up with…

I’m working in a single digital document. I have a heading, “Ongoing,” at the top that has the big things we currently have on our radar. The list has some dates with notes; “Oct 2020 — started that big project” and similar things.

Next I have a heading, “Jan 5, 2022” with the date of our next scheduled meeting. When that meeting arrives, I start by doing something very weird: I add “9876543210” on the line below the heading. Then I take simple bullet-point notes under that heading. “We discussed the foo bazzle widget needs defranishizing,” and similar items. Before our meeting ends, I add a heading for the date of the NEXT meeting, ABOVE this meeting’s heading. This pushes the heading and notes down the page a bit.

Then I continue reading. The heading just below this meeting’s, is the date of our last meeting. Just below the heading is “9876543210”, which I put there when we had that meeting. I delete the “9” from the front. I read my notes from the meeting. I may even edit them. Sometimes things that were obvious then, don’t seem so obvious a week later.

Then I continue reading. The next heading is the one from two meetings ago. Just below it is “876543210” — think about that, if it’s not obvious that last week, I read this part and already removed the “9”. So this week, I remove the “8.” Read the notes.

I work my way down each of the historical dates. Snipping a lead number, off the front of the line after each heading. 7. 6. 5. etc.

At the very end of the document, I find a heading that is from 11 meetings ago. Below the heading is “0” — because I’ve looked at these notes 9, 8, 7, 6, etc deleting a digit each time. These notes are now quite old. In fact, they should be irrelevant after 11 meetings. If they are not, I figure out what I have to add to “Ongoing” (the very topmost heading)… or perhaps I put a note under the coming meetings heading (just below “Ongoing”.)

It sounds wonky, but it’s magic. One digital document, you can skim the entire thing right in any of the meetings. You can search in the document. I can be sure I’m not forgetting things, but I can be sure I’m not making a huge collection of crap I’m never going to look at again.

Care to guess where that delete-a-digit each time comes from? It’s an idea from book printing. When they used to set type (physical lead type in trays) they would put “1 2 3 4 5 6 7 8 9 10” (or other orderings of the numbers) in the cover plate. Then print the book. What printing? This one is “1” Next printing? …they’d just chip off the “1” and print “2 3 4 5…” in the book… second printing. They still print those weird sequences of digits in digitally printed books. I believe this one is a second edition, 3rd printing…

ɕ

Looking back

December 26, 2020

I’ve started thinking about a touch phrase for 2021. (2020’s was the superbly helpful, “Get less done.”) As part of the thinking, I was browsing the old blog, and wondered when I last missed a daily post. That turns out to be November 18, 2019. It’s simply a random day with nothing posted.

For a few weeks leading up to that 18th, there’s a post on each day. But October of 2019 is Swiss cheese—actually, it’s more hole than cheese. But early to mid 2019 things look mostly solidly-posted. ¯\_(ツ)_/¯ I also know that in the very beginning of this blog I wasn’t even intending to post daily; In the beginning it was just a place for me to put things that I felt I needed some place to put. Unsurprisingly, character by character this blog was built like a drifting sand dune. In 2021 this blog will turn 10. Hello World was posted on August 13, 2011. If I continue, and I see no reason to stop, post number 3,000 should appear in late December 2021.

I bring this up because this time of year is traditionally a time for wrapping things up, and striking out anew, perhaps with a fresh start or a new commitment, into the new year. meh. That never works for me. But you know what has been working well—year-round, not just during this completely arbitrary calendar roll-over point—

LOOK BACK!

Look back at some of the things you’ve accomplished or experienced and think: “Well if that isn’t nice, I don’t know what is.” Seriously. I’m not going to end this post on a, “but if you don’t like what you see…”. No.

Take some time during the arbitrary end-of-the-year machinations to look back and think:

Well if that isn’t nice, I don’t know what is.

ɕ

40 knots in the freezing Atlantic

May 6, 2020

The point is, you’re basically this walking, lumbering habit machine. And these habits — a.k.a. your identity — have been built up over the course of decades of living and breathing, laughing and loving, succeeding and failing, and through the years, they have built up a cruising speed of 40 knots or so in the freezing Atlantic. And if you want to change them — that is, change your identity, how you perceive yourself or how you adapt to the world — well you better slam that steering wheel to the side and be ready to hit a couple icebergs, because ships this big don’t turn so well.

~ Mark Manson from, Shut Up and Be Patient

slip:4umabe1.

I’ve been saying, “big ship, little rudder,” for a long time about my own attempts to change course. I’m certain I’m right about myself, but it’s reassuring to hear other people say they see this about themselves too. Wether or not Manson is your cup ‘o tea, it’s nice to hear things that confirm your assessment. “That looks like a shark, right? That’s a shark; we should get out of the water, right?”

The reason—not “one of the reasons”, but the actual, single, I’m-not-kidding, this is really why, reason—I tell the truth is that it helps other people form a good model of their world. Sound bonkers? …try this:

You know what it looks like, and/or sounds like, when a car is approaching along a roadway. You have a model of your world that, I hope, predicts that being struck by said car would be Very Bad. So you instinctively adjust your actions—get out of the way that is—when you see or hear a car approaching. That’s you using a model of reality; in this case a really good model that almost always works. Your model isn’t perfect: The driver could swerve to avoid you, and end up hitting you. In which case your model of the world has failed; you should have stood stationary, in the street to avoid being hit.

Where did you get that model?

What if I had arranged it so that every car you encountered as a child was driven by a confidant of mine. I had them all swerve to avoid you, and I taught you that cars will avoid you. You’d have a very different model! …and you’d agree I had done some SERIOUS lying to you!

Does my definition of True make sense now? I’ll say something if it will help you build a better model of the world. One can try this test on everything; it always works perfectly to tell you what is morally correct [in the context of speaking]. Anyway, I digress.

So I’ve been saying to myself, “big ship, small rudder,” and here I have an external bit of evidence from Manson that my model is correct. *shrugs* Sorry, this is what happens when you peek into my head.

ɕ

Content-encoding gzip, plus HTTP range requests, equals bad mojo

October 19, 2019

This is going to be long. You’ve been warned.

Act 1: Wherein our hero is oblivious to the trouble

Years ago, early in the life of my podcast, someone waved their phone at me and said, “sometimes the podcast playback jumps back to the beginning, and then I cannot skip or scrub forward to resume where I was.” I shrugged. What’s one problem report for Google Podcasts on Android, particularly since this was early days for Google’s Podcasts app.

Curious.

Aside: “scrub” is audio lingo for manually sliding through audio. On apps, there is usually a small grab-here marker at the current play position. If you touch that and slide, the audio will scrub along until the time where you let go. (Versus “skip” buttons which jump forward or backward a set amount of time.)

I have a vague memory of some later time, where some automated analysis of our podcast feed reported that we “don’t support range requests.” I initially ignored this, but made a note. One day—months later—I looked up what an [HTTP] range request is, and verified that our web site does in fact support range requests.

Curioser.

A week ago, I got another problem report. From an Android user with the Google Podcast app. First off, it’s no longer early days for that app, so I’m less inclined to just “blame the app” when someone speaks up. Second, the callout was more thorough. This person had tried several different of our episodes, (all of which exhibited the problem,) and they had verified that some other podcasts they subscribe to did not exhibit this problem.

“Curioser and curioser,” said Alice.

Our podcast is self-hosted. We run a virtual instance of CentOS on BlueHost, with Apache and WordPress, with the Seriously Simple Podcasting (SSP) plugin producing our podcast RSS feed. Seriously, none of that is simple. But it does mean that we have a tremendous amount of control—if we want to look under the hood. (Stop here. Take 4 minutes to watch that if you’ve never seen Mike’s New Car.)

Act 2: Wherein our hero heads into the belly of the beast

I asked the person who waved their problem at me years ago, “hey, uh, do you still see that problem?” (Yes they do.) …and I reached out to James Cridland at podnews.net and he verified that he too sees this behavior with my podcast files. …and he pointed out that he was seeing, (it’s not clear exactly what tool he used—but it doesn’t matter for this story,) content-type: gzip for the media file that we were serving.

Wait, wat.

Why am I serving a compressed (i.e., gzip’d) version of an MP3 file? That’s already a file containing compressed data; It probably increases in size when you gzip it. Not to mention the CPU cycles wasted gzip’ing the many-megabyte sized files for each reqeust.

Next the voice in the back of my head started pointing out that HTTP range requests—where the web client (in this story a podcast player app) can ask for a specific range of bytes from a resource—sure feels like the sort of thing that might be related to pulling down some of a file now, and then more of the file later after you’ve listened to it for half an hour. Maybe if we didn’t support range requests that would mess up skipping and scrubbing? But wait, no, I checked two years ago, (and I just rechecked,) that we support range requests. So what the heck?! Is the problem related to compression, to range requests, the combination, or something else?

Spock mode on. Start checking everything methodically. When you’ve eliminated all other possibilities, whatever remains, must be the case.

What if we don’t actually support range requests on our media files? So I started digging into how Seriously Simple Podcasting (SSP) handles the actual feeding out of files.

Aside: I know enough about Apache and PHP to know that just because Apache supports range requests on files (“here’s 100 bytes from that MP3 you asked for…”) doesn’t mean that a PHP program would necessarily be able to answer a range request. Spoiler: It’s very hard to support a range request programmatically in PHP. So I need to know what exactly—Apache or SSP, which is just a pile of PHP code—actually feeds the media file?

So I posted on the SSP support forum…

I’m trying to troubleshoot a problem reported with the Google Podcasts player on Android. (I’ve one reporting user and I cannot personally reproduce the problem.) In the process, I went down a rabbit hole looking into HTTP range requests.

I’m wondering: If the the SSP plugin is serving the MP3 audio files via PHP (which would require the PHP code to implement supporting range requests) or if, after a redirection from the stats-collection URL, it let’s my web server (Apache) just send out the static file (in which case Apache itself handles range requests.)

One of the devs responded:

Do you perhaps have a URL describing HTTP range requests and how they relate to serving files behind the PHP redirect, so that I can understand how it could be causing the problem? From the cursory review I’ve done, it would appear we should update the plugin to support range requests, would you agree?

Which startled me both because range requests are apparently more obscure than I was thinking they are, and that this was a very nice olive branch from a developer right out of the gate. Anyway. I was already really doubting that SSP was causing this problem, so I put on my big-boy detective pants and dug deeper.

Leading me to post:

…I think it’s not actually a problem [with SSP], but I wanted to double-check my analysis with someone familiar with the code.

For range requests, it’s RFC7233 — but before you even bother looking at that. I think the answer is that SSP doesn’t handle the serving of the audio file via PHP, but rather leaves that to the underlying web server. (In my case, that’s Apache, which handles range requests of static assets.)

Straight from my RSS feed, I have (for example) <enclosure url="https://moversmindset.com/podcast-download/4734/062-chris-and-shirley-darlington-rowat-serendipity-family-and-relationships.mp3" length="29493071" type="audio/mpeg"></enclosure> and if I fetch that URL, I get SSP doing a redirection. Here I’m asking Curl to get me a range of bytes:

Craigs-iMac:~ craig$ curl -I --range 500-600 https://moversmindset.com/podcast-download/4734/062-chris-and-shirley-darlington-rowat-serendipity-family-and-relationships.mp3
HTTP/1.1 302 Found
Date: Fri, 11 Oct 2019 14:55:48 GMT
Server: Apache
Pragma: no-cache
Expires: 0
Cache-Control: must-revalidate, post-check=0, pre-check=0
Robots: none
X-Redirect-By: WordPress
Set-Cookie: PHPSESSID=6576b49ab4d78ab7628bb05a727805dd; path=/
Location: https://moversmindset.com/wp-content/uploads/2019/10/MM_62_Chris_and_Shirley.mp3
Content-Type: text/html; charset=UTF-8

Aside: -I with curl says just give me the headers for a response for the requested resource. Not the actual resource. The 302 HTTP status, combined with the Location: header is standard web-speak for a web server saying, “please go get this resource instead.” Critically this is a 302 which is a “temporary” redirect, not a 301 which is a “permanent” redirect. With 302, if you want this resource again or more of it with another range request, you should ask for it again at the original URL. Versus with a 301, where you should not ask again, you should use the new location going forward with any subsequent requests. tl;dr: 302 + Location is what I expected to see.

…that curl request gives me a standard redirection. As expected(!) since SSP wants to track statistics. That new 302 location is a direct-link into the WP assets storage. When I curl that, making a range request again, it works perfectly. (Apache is happy to give me the 101 bytes I’m asking for.) Below is both the headers-only (-I in Curl) and a full fetch….

Craigs-iMac:~ craig$ curl -I --range 500-600 https://moversmindset.com/wp-content/uploads/2019/10/MM_62_Chris_and_Shirley.mp3
HTTP/1.1 206 Partial Content
Date: Fri, 11 Oct 2019 14:58:33 GMT
Server: Apache
Last-Modified: Sun, 06 Oct 2019 14:51:01 GMT
Accept-Ranges: bytes
Content-Length: 101
Vary: Accept-Encoding
Content-Range: bytes 500-600/29493071
Content-Type: audio/mpeg
Craigs-iMac:~ craig$ curl --range 500-600 https://moversmindset.com/wp-content/uploads/2019/10/MM_62_Chris_and_Shirley.mp3 > ./foo
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   101  100   101    0     0    389      0 --:--:-- --:--:-- --:--:--   388
Craigs-iMac:~ craig$ ls -alh ./foo
-rw-r--r--  1 craig  staff   101B Oct 11 10:58 ./foo

Aside: First a bunch of headers saying that I would receive [if I actually asked] 101 bytes of content-length, and then an actual request where I end up with 101 bytes in a file on my computer. tl;dr: everything as expected.

So I think the answer is that SSP doesn’t interfere with HTTP range requests. And that means the problem I’m trying to solve can’t be caused by my site not correctly answering range requests.

At this point, I folded my arms with one of those “hurumph” noises. Then I thought of something: Ya’ know, since it’s Apache that is going to feed me that MP3 file, it would totally be able to change its behavior based on what the web client, (aka, the podcast player app, Google Podcasts,) said it would accept as a response.

Aside: The Web is a conversation between web clients and web servers. Every request—and there can be hundreds of requests to show you one page—starts with the client asking for a resource and listing the types of responses it will accept. Think: am I wanting an image resource, an audio file, a blob of HTML, etc. Also, what types of encoding of those resources can the client understand. tl;dr: No more tl;dr’s here. We’re in the belly of the beast now.

So how do I tell curl to manipulate the encodings it should tell the server it would accept. Answer: By adding a header via the -H flag.

So reviewing: Here’s a normal ask for the headers for a specific media file. This isn’t a range request, this is just an ask for the headers for an entire resource:

Craigs-iMac:~ craig$  curl -I https://moversmindset.com/wp-content/uploads/2019/10/MM_62_Chris_and_Shirley.mp3
HTTP/1.1 200 OK
Date: Thu, 17 Oct 2019 13:09:27 GMT
Server: Apache
Last-Modified: Sun, 06 Oct 2019 14:51:01 GMT
ETag: "220031-1c2074f-5943f1052a358"
Accept-Ranges: bytes
Content-Length: 29493071
Vary: Accept-Encoding
Content-Type: audio/mpeg

That’s exactly what I expect: If I actually asked for the resource, I’d get about 30 megabytes of content back.

And what would happen if I tell curl (note the -H argument on this one) to tell the server that I’d be happy with a gzip’d response:

Craigs-iMac:~ craig$ curl -H "Accept-Encoding: gzip" -I https://moversmindset.com/wp-content/uploads/2019/10/MM_62_Chris_and_Shirley.mp3
HTTP/1.1 200 OK
Date: Thu, 17 Oct 2019 13:11:03 GMT
Server: Apache
Last-Modified: Sun, 06 Oct 2019 14:51:01 GMT
ETag: "220031-1c2074f-5943f1052a358-gzip"
Accept-Ranges: bytes
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Type: audio/mpeg

Oh shit. It would send me a gzip encoded version of my MP3 file. And critically, it doesn’t tell me how big that would be—no Content-length is given—because the server would have to actually compress it with gzip to see how big it would actually be.

Aside: If you know about Apache’s ability to serve out pre-compressed versions of files—so you have the .mp3 and the .mp3.gz files laying on disk ready to go—if you know about that, then you don’t need to read any of this article. I was tempted to set that up just to have Content-length and the gzip encoding header in the shot because you would have noticed. ;)

So IF the client . . . say for example, oh, I don’t know, the Google Podcasts app maybe? . . . happens to mention that it could accept a gzip’d response, then boy-howdy a gzip’d response our Apache would send.

…and that’s a problem why? Because it turns out that you cannot combine compression (any sort, not just gzip) content encoding with range requests. The long version is this Stack Overflow thread, Is it possible to send HTTP response using GZIP and byte ranges at the same time? The short answer is, no, because even if I wanted to waste my time compressing it just to give you 100 bytes out of the middle, it’s not possible for you to then uncompress those 100 bytes. All modern compression algorithms work on streams of data—you have to start decompressing from the first byte.

Aside: The next version of HTTP addresses this because it turns out that being able to have range requests on resources compressed in flight would be very useful.

Ok smart guy, what happens if you try to make a range request and accept compression?

Craigs-iMac:~ craig$ curl --range 500-600 -H "Accept-Encoding: gzip" -I https://moversmindset.com/wp-content/uploads/2019/10/MM_62_Chris_and_Shirley.mp3
HTTP/1.1 206 Partial Content
Date: Sat, 19 Oct 2019 01:58:16 GMT
Server: Apache
Last-Modified: Sun, 06 Oct 2019 14:51:01 GMT
ETag: "220031-1c2074f-5943f1052a358"
Accept-Ranges: bytes
Content-Length: 101
Content-Range: bytes 500-600/29493071
Content-Type: audio/mpeg

Honestly? That’s not what I expected. I was expecting some sort of actual error from the server.

But nope, that’s a perfectly happy, 101 bytes—or it would be if hadn’t specified I just wanted the headers—out of the full 30 megabytes-or-so, and it wouldn’t be compressed. This confuses the hell out of me because it’s exactly what you’d want. The app asked for something we can’t do so we skip the compression part—the app said it would accept compression, not that it demands compression.

Oh, who cares. Compressing MP3 files—especially live on the fly each time they are served—is totally the wrong thing to be doing. Let’s just stop that and hope the problem goes away.

Aside: You thought the wizard behind the curtain always figures it out? I’ve got some bad news for you sunshine, Pink isn’t well, he stayed back at the hotel…

Act 3: Wherein our hero vanquishes the problem by typing four characters

Still in Spock mode, let’s describe the actual problem…

  1. There’s crazy-level complexity with compression and range requests
  2. It’s not clear what exactly the Google Podcast app is requesting—I don’t have any Android devices and I’d have to capture TCP data from the network to even find out
  3. So the app makes some sort of request…
  4. …and the server responds
  5. Making scrub and skip not work.

Oh, well that’s perfectly clear then, isn’t it?

Seriously, screw this. I’m just turning off compression of MIME type “audio/mpeg” files. (MP3 files are an example of MIME type “audio/mpeg”.)

Aside: Yes, I said MIME—the Multipart Internet Mail Extension specification is how we ended up classifying what something is on the web. Major type “audio”, minor type “mpeg”. I know, this stuff is bonkers… it’s just turtles all the way down.

Since Apache does not compress things by default, all I have to do is find where the “DEFLATE”—that’s really what it’s called, gzip is one way of “deflating” files—output filter is assigned to handle files of MIME type “audio/mpeg.” That’s actually easy to do if you are fluent in Apache.

I’m an Apache configuration file wizard. I’ve been using Apache since it was spun off from something else in—I had to look it up—1995. Trivia: It was “a patchy server” cobbled together from some open-source work done—never mind, go read it on Wikipedia. ANYWAY.

I read over the entire Apache configuration, it’s complicated in files that include other files with nesting and logic and lions and tigers and bears… but it’s all perfectly clear and straight-forward to me…

…and yet I can’t see why it would EVER decide to apply the DEFLATE output filter to an audio/mpeg file.

Strike one. Off with the kid gloves.

If I can’t figure out where it’s turned on and remove that, the next best thing to do is to just add a rule that turns if off. But when I went to do that, I found that whoever designed the system architecture had already stumbled upon this mess. (Compressing media files is wrong-headed.) They already had a rule, which confesses the sin of wacky configuration by saying, “yo! for these media files, knock it off with the DEFLATE output filter!”

SetEnvIfNoCase Request_URI .(?:gif|jpg|png|ico|zip|gz|mp4|flv)$ no-gzip

As soon as I saw this, I was like, “you’re kidding me, right?” That says if the thing being requested ends with a period followed by any of those file extensions, then set an environment flag telling the DEFLATE module not to gzip.

…and “mp3” is not listed.

So I cursed like a sailor, threw my hands up in the air, and added “mp3|” to that string right after “mp4|”, restarted Apache, checked with James, and the problem is fixed.

omfg where’s my Tylenol?

ɕ