Sysadmin

There is only discipline

September 18, 2023

I often mention the false sense of urgency that I experience. I have lots of ideas, sure, but it’s more than the frequent appearance of those endless new opportunities. It’s more so the sense that anything I’m already working on, I could do just a little bit better. There’s a pessimistic paranoia that old, greying system administrators develop; they look both ways even when crossing one-way streets. All of that combines within me. I’m not sure if all that striving leads me to feel there’s a scarcity of time and opportunity, or vice versa— I have a sense of scarcity, which leads to the sense of urgency and incessant striving.

Schopenhauer’s pessimism is based on two kinds of observation. The first is an inward-looking observation that we aren’t simply rational beings who seek to know and understand the world, but also desiring beings who strive to obtain things from the world. Behind every striving is a painful lack of something, Schopenhauer claims, yet obtaining this thing rarely makes us happy. For, even if we do manage to satisfy one desire, there are always several more unsatisfied ones ready to take its place. Or else we become bored, aware that a life with nothing to desire is dull and empty. If we are lucky enough to satisfy our basic needs, such as hunger and thirst, then in order to escape boredom we develop new needs for luxury items, such as alcohol, tobacco or fashionable clothing. At no point, Schopenhauer says, do we arrive at final and lasting satisfaction. Hence one of his well-known lines: ‘life swings back and forth like a pendulum between pain and boredom’.

~ David Bather Woods from, https://aeon.co/essays/for-schopenhauer-happiness-is-a-state-of-semi-satisfaction

slip:4uaeea20.

For five months I’ve had a single sticky-note on my monitor which reads, “There are no miracles. There is only discipline.” It’s a strikingly clear guide star. I believe that a disciplined person knows not only when to strive, but also when to ignore an idea, when to pause for the time being, and when to rejuvenate.

Most often that sticky-note triggers my thinking about living a balanced discipline. I see the note (it’s unfortunately only on my monitor, but should be added to the interior of my eyelids) and then I notice if I’m feeling harried, or if I’m striving… Why? Is this thing I’m doing, or that thing I feel I should be doing, actually urgent? And how—get clear here, Craig—did this or that even get to be the thing I’m doing, the thing on my radar, on my to-do list, on my to-should list… What would it be like, to simply be?

ɕ

Caution: Tulpa

December 31, 2021

I’ve recently made a startling discovery: Maybe there really is a tulpa in my head.

First, I’ve said for many years that my brain is broken. (Yes, I am aware I have terrible self-talk.) Here’s why I call it broken: I am literally unable to NOT see problems. I notice an endless onslaught of things that, in my opinion, could be improved. I don’t mean, “that sucks, I wish it could be better.” No, I mean, “that sucks and it’s obvious this way would be better and if you’d just let me get started . . . ” Adderall might help, I suppose.

Everyone loves that I get stuff done, and try to make things better. But unless you have this same problem, I’d imagine it’s hard to understand how this is debilitating. I am aware that this is recursive—I see my own brain as a broken process that I feel I should repair. All I can say is that you should be happy, and thank your fave diety if that’s your thing, that you don’t understand. Because to understand is to have the problem, and you do. not. want. this. problem.

Second, I’ve also said for many years that, “the remainder cannot go into the computer.” I’m referring to a endless source of struggle in programming and systems administration; Computers are exact, and the real world—with its real people, real problems, and things which really are subjective shades of gray—is not. So programmers and systems administrators factor, in the mathematical sense of finding factors which when multiplied give you the original, reality into the computers. And when factoring reality, there is always a remainder. That remainder shows up when you find your software does something weird. That could be a mistake, but I tell you from experience, it is more often some edge case. Some people had to make choices when they factored.

The result of that second point is that I’ve spent the majority of my life factoring, (and “normalizing” for your math geeks who know about vector spaces,) problems into computers. And then trying to live with the remainders that didn’t go into the computer. The remainders are all in my head. Or on post-it notes on my wall, (back in the day.) Or the remainder is some scheduled item reminding me to check the Foobazzle process to ensure the comboflux has not gone frobnitz. To do that I had to intentionally be pragmatic and logical. And the really scary part is I also learned that the best way to do all of that was to talk to myself—sometimes literally, bat-shit crazy, out loud, but usually very loudly inside my own mind—to discover the smallest, least-worst, remainder that I could manage to live with.

What if those two things were sufficient to create a Tulpa. (I am serious.)

I think there’s a Tulpa in here! (My title is the sign on the front gate.) It is absolutely pragmatic. It knows an alarming amount of detail about things I’ve built, (or maintained, or fixed.) It is cold and calculating. It is terrified that it will forget about one of those details, 2347 will happen, and everyone will run out of ammunition defending their canned goods from the roaming bands of marauders. I definitely don’t “have” the Tulpa. It’s more like discovering there’s an extra person living in your house. Although, I don’t hold hope of banishing this Tulpa, Yoda does make a good point if I’m going to try. So, I should definitely give it a name.

Maybe, Sark?

That is an intriguing idea indeed! Sark, what do you think?

ɕ

To serve man

January 24, 2018

And you know why I do it? I need that help, too. I get tired, angry, upset, emotional, cranky, irritable, frustrated and I need to be reminded from time to time to choose to be the better version of myself. I don’t always succeed. But I want to. And I believe everyone else – for some reasonable statistical value of everyone else – fundamentally does, too.

~ Jeff Atwood from, https://blog.codinghorror.com/to-serve-man-with-software/

slip:4ucoto1.

He had me at the “to serve man” Twighlight Zone reference…

ɕ

End of another era

July 5, 2017

I’ve been dutifully tending equipment in this bay for 15 years… little bit sad that I won’t ever drive here again. On the other hand, 10 years ago I nearly died in a car crash (not my fault) coming here in the middle of the night. There’s an invisible army of system admin who work around the world, every hour of every day. We make every aspect of your modern world function. I’m proud to be one of them, doing my little part.

ɕ

The folly of the unwise

June 6, 2015

When I left, the search for my replacement took a long time. Much longer than I’d have preferred, and to an eventually unsatisfactory conclusion for everyone involved, I believe. I contented myself with the knowledge that my skill set was sufficiently wide in breadth and complex in nature that I was hard to replace. I used this to buoy my ego. ALthough I had sympathy for the people I was leaving, and the one I left in my stead, it felt good to be needed and wanted, and I was proud that I could fill that role like no one else we’d found.

Such is the folly of the unwise, I’m afraid.

~ Matt Simmons from, http://www.standalone-sysadmin.com/blog/2013/02/and-when-you-gaze-long-into-an-infrastructure-the-infrastructure-will-gaze-back-into-you/

Note to self: Hang out with Matt more and listen to what he has to say.

ɕ

Less interrupting please

May 20, 2015

I have seen this happen more times than the number of yaks I’ve shaved. At nearly every job I’ve had, I’ve walked this fine line. I’ve had performance reviews where I’ve been called pushy, aggressive, assertive, abrasive, or bitchy simply for speaking up in a similar manner to that of my male colleagues, and on the other side of things, I’ve been interrupted and spoken over more times than I can count. I’ve worked at places where I was the only one being interrupted (backstory: I’ve been the only woman in a lot of engineering departments), which has bothered me. But I’ve also worked at places where everyone interrupts each other all the time. For a while, I thought that was better. “At least I’m not being spoken over because I’m the only woman; the guys get interrupted too,” I thought to myself. But everyone interrupting everyone else really isn’t that much better.

~ Katherine Daniels from, http://beero.ps/2015/01/13/on-interrupting-interrupt-culture/

For, let’s say, the first half of my life, I was always the one doing the interrupting. As I’ve begun to listen, I now realize how much everyone interrupts everyone else. When I’m relaxed and on my game, I try to have a meta-listening happening so I can tell when to stop talking to keep the conversation working. As best I can manage, when I’m interrupted, I simply stop talking.

But sometimes, just for fun, I like to toss this in quietly while the interrupter is still speaking…

Oh! I’m sorry, did the middle of my sentence interrupt the beginning of yours?

ɕ

The forsaken art of pedagogy

April 8, 2015

In other words, you actually belong to a wider group: you are one of the increasingly commonplace factions of society that takes pride in not bothering to make yourself understood. You feel entitled to let others worry about what you really mean, and even revel in the tribalism of `being in the know’ rather than letting others into your secret world, as if playing the role of an ignorant tourist in a foreign country.

~ Mark Burgess from, http://markburgess.org/blog_pedagogy.html

slip:4umabo1.

ɕ

The third wave of IT engineering

December 22, 2014

Just as human society still envelops everything from pre-agricultural tribes, farming communities, and factory sweat-shops to information-based commerce, in different parts of the globe, so IT today straddles all three `waves’ of development from manual chaos to goal-oriented self-repair in different organizations.

~ Mark Burgess from, http://markburgess.org/blog_cap.html from, http://cfengine.com/company/blog-detail/cfengine-sysadmin-3-0-and-the-third-wave-of-it-engineering/

slip:4umabo3.

ɕ

Programming sucks

December 20, 2014

Every friend I have with a job that involves picking up something heavier than a laptop more than twice a week eventually finds a way to slip something like this into conversation: “Bro,1 you don’t work hard. I just worked a 4700-hour week digging a tunnel under Mordor with a screwdriver.”

System administration sucks too:

… And if these people stop, the world burns. Most people don’t even know what sysadmins do, but trust me, if they all took a lunch break at the same time they wouldn’t make it to the deli before you ran out of bullets protecting your canned goods from roving bands of mutants.

~ Peter Welch from, http://stilldrinking.org/programming-sucks

slip:4usipo1.

hear! hear!

ɕ

DevOps

December 10, 2014

It’s been said by a number of smart people that DevOps is largely founded in an organization’s skillful collaboration and communication, and the culture that results. I agree with that idea, and I also think that it’s one of the reasons why the term DevOps is sometimes difficult to explain, because these are ‘soft’ skills we’re talking about. These aren’t things you can graph or alert on, they only manifest in the resulting product and environment.

~ John Allspaw from, http://www.agileweboperations.com/devops-these-soft-parts

slip:4uaide2.

ɕ

Take back the Internet

May 21, 2014

This is not the Internet the world needs, or the Internet its creators envisioned. We need to take it back.

And by we, I mean the engineering community.

Yes, this is primarily a political problem, a policy matter that requires political intervention.

But this is also an engineering problem, and there are several things engineers can — and should — do.

~ Bruce Schneir from, https://www.schneier.com/blog/archives/2013/09/take_back_the_i.html

slip:4usebo4.

I’d venture that the vast majority of regular, everyday people working in technology related jobs are not actively trying to do evil. People go to work, make the best decisions they can and then go home. If that’s true, then it’s going to be nigh impossible to change the momentum of how things (e.g., NSA surveillance) are going. Because in order for it to change, we need to start thinking bigger.

ɕ

Operable systems

April 21, 2014

Especially for complex, multi-purpose systems, the gap between how things are supposed to work and how they actually work can be quite large. (Ask any police sergeant about the difference between policing in theory and policing in practice!) A primary function of operators is to bridge this gap in ways that result in better rather than worse outcomes. The capacity of systems to be operated is what allows operators to perform this valuable function, sometimes called technical work.

~ Richard Cook from, http://programming.oreilly.com/2013/10/making-systems-operable.html

slip:4uoema1.

More and more I’ve been getting a lot mileage from this idea: Make things easier TO USE, rather than trying to fully automate (i.e., so I don’t have to use them.) One cornerstone to accomplishing that is creating “affordences“.

ɕ

Heartbleed: For want of one nail, the kingdom is lost

April 8, 2014

The Heartbleed OpenSSL problem is big news ( http://heartbleed.com if you’ve been under a rock ). What’s wrong?

In short, Heartbeat allows one endpoint to go “I’m sending you some data, echo it back to me”. It supports up to 64 KiB. You send both a length figure and the data itself. Unfortunately, if you use the length figure to claim “I’m sending 64 KiB of data” (for example) and then only actually send, say, one byte, OpenSSL would send you back your one byte — plus 64 KiB minus one byte of other data from RAM.

Whoops!

~ Matt Nordhoff from, http://security.stackexchange.com/questions/55116/how-exactly-does-the-openssl-tls-heartbeat-heartbleed-exploit-work

So this one, tiny-looking problem brings our entire sand-castle Internet kingdom down. “Secure” web sites turn out aren’t necessarily secure. Worse, they haven’t been secure for some uncertain amount of time. So, anything communicated insecurely, during some uncertain time-frame… is, uh, possibly snooped, stolen, etc. The system admins have to patch the fix in, then redo site certificates, then everything everyone has put to/from those sites, (your login and password for example!) has to all be considered stolen/tainted and has to be reentered.

Bonus: it’s even worse than I’m making it sound: Try this on…

http://security.stackexchange.com/questions/55116/how-exactly-does-the-openssl-tls-heartbeat-heartbleed-exploit-work

http://security.stackexchange.com/questions/55097/can-heartbleed-be-used-to-obtain-memory-from-other-processes

ɕ

Senate steps into the data breach controversy

February 5, 2014

The Senate Judiciary Committee spent the day looking into recent data thefts at Target and Neiman Marcus. Lawmakers know there is a big problem, but they are struggling with what role the federal government should play is creating new standards to safeguard consumer data.

~ Jim Zarroli from, http://www.npr.org/2014/02/04/271591552/senate-steps-into-the-data-breach-controversy

slip:4unose2.

Yeah. I said this before.

ɕ

Knowledge workers as a political class?

February 2, 2014

Other people are recognizing that we work in an important intersection of knowledge and responsibility, too. I came across a presentation from this year’s Chaos Communication Congress in Germany. It was a talk by Jacob Appelbaum and Julian Assange, who were introduced by Sarah Harrison. The name of the talk was SysAdmins of the World Unite.

~ Matt Simmons from, http://www.standalone-sysadmin.com/blog/2013/12/knowledge-workers-as-a-political-class/

ɕ

Repercussions

November 12, 2013

If I were in government right now, I would be leery of starting another big software project. I’d also know that big software projects are going to be necessary as our civilization gets more and more complex. So, if I were in government right now, I’d be thinking about laws to regulate the Software Industry. I’d be thinking about what languages and processes we should force them to use, what auditing should be done, what schooling is necessary, etc. etc. I’d be thinking about passing laws to get this unruly and chaotic industry under some kind of control.

If I were the President right now, I might even be thinking about creating a new Czar or Cabinet position: The Secretary of Software Quality. Someone who could regulate this misbehaving industry upon which so much of our future depends.

Maybe that thought hasn’t occurred to them yet. Maybe. But how many more healthcare.gov debacles will it take before it does?

~ Bob Martin from, http://blog.8thlight.com/uncle-bob/2013/11/12/Healthcare-gov.html

Most people I’ve talked to, (who write software or do systems and network administration,) are in the “I have work to do” camp. They’ve no time to think about professionalization, or standardization, of their field. To which I say:

That’s cool; I understand. No worries! The government will eventually get around to ramming standardization and licensing down your throat. I’m sure that will work out well for us.

If you work in these fields, you should be paying attention. If you wok in network and systems administration, you should be paying attention to LOPSA and Usenix/LISA.

Update:

Feb 2014: Senate Steps Into the Data Breach Controversy

ɕ

Where did the term “Software Engineering” originate?

August 19, 2013

Just in case you thought it was fairly new, it’s probably(*) older than you. Here’s a deep link, to a down-the-rabbit-hole discussion. Seems most sources attribute a 1968 conference, while the author of this message from the Software Craftsmanship group has dug up an ACM article from 1966.

* …and probably significantly older than you since the average age of the entire world population is definitely less than Software Engineering’s 47 years (and counting.)

ɕ

Network Theory applied to altitude sickness

August 10, 2013

They then mapped out the correlations between the various symptoms, creating a network. An increasingly standard tool in network theory these days is cluster detection–the ability to spot parts of a network that are more strongly linked together than others.

~ http://www.technologyreview.com/view/512986/network-theory-approach-reveals-altitude-sickness-to-be-two-different-diseases/

Acute mountain sickness (AMS) is a common problem among visitors at high altitude, and may progress to life-threatening pulmonary and cerebral oedema in a minority of cases. … These results challenge the accepted paradigm that AMS is a single disease process and describe at least two distinct syndromes following acute ascent to high altitude. This approach to analysing symptom patterns has potential utility in other clinical syndromes.

~ http://arxiv.org/abs/1303.6525

ɕ

Keep it simple. Good luck with that.

July 8, 2013

In fact it is so difficult to argue against simplicity that this post won’t even attempt to.  Let’s state emphatically that software should always do only what you need it to do, with the fewest number of steps, and least potential for errors due to complex choices and options.

On the other hand, good luck with that.

~ Steven Sinofsky from, http://blog.learningbyshipping.com/2013/03/19/designing-for-scale-and-the-tyranny-of-choice/

slip:4ulede1.

ɕ