Selecting quotations using Chat-GPT

21 Days Ago

With great power, comes great responsibility. Large language models (LLM) like Chat-GPT are powerful tools. How do I use it responsibly?

I want to find and present great quotations from guests on my podcast episodes. What happens when I try to get Chat-GPT to do it? Following is a really deep dive into exactly what happens, along with my best efforts to work with this power tool in a way which accurately represents what my guests say, while showing them in the best possible lighting.

The transcripts the LLM works from

For this to make any sense, you need to know that I start from a machine-generated transcript. I get them from the recording service, or from another service where I upload audio (for older episodes before machine-generated-from-the-service was available.)

Below is a screenshot.

  • In the left-margin are line numbers. Line 107 and 123 are too long to fit, so my text editor has visually, “soft” wrapped them for ease of reading.
  • The files have time codes in them (the format does vary somewhat too). They have silly amounts of precision: 16:51.57 is 16 minutes and 51.57 seconds. Hours appear in the front as another number with another colon.
  • The LLM understands which person is the guest, because it understands who introduces the show, and introduces the other person.

I break the very long transcript text file into chunks, because there’s a maximum amount of text you can paste into the LLM interface. The screenshot above is from the 2nd or three chunks from my conversation with Martin. The whole chunk is 144 lines and about 8,000 characters.

Imagine having to read through the whole transcript to find the best part to quote. That’s very hard for me to do (nevermind I don’t have the time to do it), but the LLM can do it in a blink. LLMs are tireless and patient.

The prompt

There’s much discussion about “prompt engineering.” It’s an art. The best clues I can give you are: Explain it to a 5-year-old. And, the 5-year-old does not get confused by ordered lists, even if you nest them.

I give the LLM all the transcript chunks. Then I give it this prompt:

Select 5 direct quotations (not from Craig) from the conversation. I prefer longer quotations which include more context. For each quotation you select, do three things: First, show me your selected quotation. Do not rewrite the quotations. You may remove verbal tics such as “ah”, “yeah” and “um”. You must leave the speaker’s false-starts and restarts in place, ending those with an em-dash and a space. Second, show the nearest time from before the selected quotation. Show that time exactly as it appears in the original transcript. Third, show the exact original transcript from which you selected the quotation. For context, show several lines of the original, unedited transcript before and where you selected the quotation.

Oh! Reviewing this post, I even found a problem in the prompt above. Can you see it? Below, you’ll see a complaint about the LLM response. Now I think it’s not an LLM error, but an error in my prompt above. :slight_smile:

(This entire post about quotations is actually just part 2 of a much larger prompt which starts with, “Perform the following 5 tasks. Include a numbered heading before your response for each of these tasks:”)

It spits the result out in one long stream of text. I’ll break it apart…

Quote 1

There are many things that I have to consider as I look at that:

Is it really a good quote, based on what I remember of that conversation? Meh, it’s okay. But that’s why I as it for 5 selections.

I don’t love that it refuses to give me context after the part where it selected the quote. I’ve tried, but after hours of work, I’m done prompt engineering and want to start this post. (As I mentioned above, I think this is because there’s an error, above, in my prompt.)

In this particular conversation, Martin talks a lot about “parkour”—thus “park order” (and many other variations I see a lot)—is just an error in the raw transcript. Ignore that for today.

Most importantly: Is it hallucinating?? Well, it’s easy to use that timestamp. 1 minute 22 seconds is definitely in the 1st chunk… a moment of scrolling…

Here’s the actual, original chunk I uploaded, and the LLM’s output side by side…

Okay, that’s sane. If I was going to pick this quotation, I’d have to work some form of my question into that quote, so his quote has some context…

Moving on, I’m just going to give you the screenshots for each.

Quote 2

Quote 3

Quote 4

Quote 5

Closing thoughts

I use LLMs to write my show notes. Getting a quote or two is just one part of that.

“Write show notes” is not “the work only I can do” (as Seth would say.) And, I simply do not have the time to do show notes from scratch.

Yes, I’ve spent hours today on prompt engineering, but I have 319 more podcast episodes from 2022 and earlier (!) that I want to have show notes for. Those episodes would be better with show notes. A few hours spent here, enable me to copy-and-paste… wait a few minutes (the LLM is not instantaneous) and I have a really good starting point for show notes.

ɕ

Mulligan? Do-over? Re-brand!

23 Days Ago

But, all things considered, this became more and more obviously the right move for me, despite the headaches it would cause in execution. So to hopefully save you the same pain and give you a little look behind the scenes, I wanted to share how I came to this conclusion and executed the rebrand.

~ Jay Clouse, from Behind the rebrand

Changing names comes up pretty often. Most recently in, https://forum.podcaster.community/t/a-question-from-david-nebinski/4356

The article above is a deep dive into one person’s thinking and efforts as part of re-brand. (Although, it has nothing to do with podcasting specifically.)

There are good points in there too about what, exactly, is a brand.

ɕ

Sabbath

24 Days Ago

Seriously? A whole day each week without work? Sitting in the orientation that day, I could not imagine pulling it off. I had spent a decade of my professional life running from event to event, fighting for the time to read and write and reflect.

~ Brenton Dickieson, from https://apilgriminnarnia.com/2016/01/20/sabbath-unplugged/

Therein lies much wisdom. Go readeth thou shall.

I’d like to suggest a layer of nuance be added: One can only achieve a certain “width” of change for that one day each week. You cannot have a completely relaxed day if the other six are maximally frenetic. If you live with the chaos and noise turned up to 11, you cannot turn it down to 2 on a particular day. By “width” I mean you can really only turn it down somewhat; If you’re normally living at an 11, you can only expect to get down to, perhaps, a 7 on your sabbath. If you really want to have a relaxing day of rest, that’s probably down around like a zero on the chaos and noise knob.

So, Sabbath? Certainly. And work regularly—every day—to turn that chaos and noise knob. Live life at 2, and then your day off can be a sublime zero of rest and recovery.

ɕ

Let’s be okay with being a little nervous

25 Days Ago

There are a bunch of insights in this little conversation with Ira Glass:

I’m ambitious! I want the stories to be special and I want the interviews to be special. The nervousness is my fear that they won’t be, and my awakeness to how hard it might be to get it to work. If you have any ambition, you march into the interview with a battle plan. You have this theory about what’s going to happen with this other person but you really have not the best idea if it’s going to work. Interviewing is an art form that so depends on the soul of the other person and also on how the two of you interact. ~ Ira Glass, from Ira Glass

I particularly like that one about nervousness as a sign that we are doing something right as podcast creators. What’s the hard work? What’s the part that makes us a little nervous? Exactly.

ɕ

Oh, good

26 Days Ago

I’ve never worked a day in my life! I don’t! These days I get up every morning and say, “Oh good, another day!”

~ Ridley Scott, from https://the-talks.com/interview/ridley-scott/

I hope the key-phrase in there is, “these days.” Because I definitely do not get up every morning thinking that. Presuming I ever reach that point of “oh good”, the challenge for me will be not piling more “should”s onto myself and forcing myself backwards. We shall see.

ɕ

Labyrinthine and awkward

27 Days Ago

But my very first memory was the Jim Henson film, Labyrinth. This has to be, without doubt, the creepiest Muppet film ever. I reacted strongly to the film as a child: frustration, intrigue, terror, revulsion, surprise, delight, and an awkward boyhood crush on Jennifer Connelly as Sarah, the heroine. It was a powerful film, not least because of David Bowie as Jareth. Evil and allure, Bowie is my archetypal Goblin King.

~ Brenton Dickieson, from https://apilgriminnarnia.com/2016/01/11/echoes-in-the-labyrinth/

I agree with this entire post. As in: OMGYESPLEASE jumping up flipping my desk. It’s now been 8 years since Bowie died… and I very nearly rewatched Labyrinth just because of this post. “Evil and allure” in every direction, indeed. Muppets and kidnapped children. Beauty and horror.

ɕ

AI, but in the other direction

28 Days Ago

This recent bite of news from Podnews.net reminded me of a discussion I recently had with Tracy Hazzard…

Is a new Google service flooding podcast apps with spam? Calling them “a threat to the podcasting community”, the podcast directory Listen Notes has made a NotebookLM Detector, to spot shows made by Google’s NotebookLM. So far, it’s detected more than 280 shows which have been made using the AI tool. “Notebook LM has made it easier to mass-produce low-quality, fake content”, says Listen Notes founder Wenbin Fang; though The Spectator’s Sean Thomas suggests that AI may “make the podcast bro irrelevant”.

~ Podnews Oct 8, 2024 issue

As podcasters we’re focused on one direction; call it the “forward” direction with increasing amounts of refinement and care…

  1. we record an episode
  2. we do post-production
  3. we write show notes
  4. we write blog posts or in-depth articles based on the episode
  5. we write based on themes we find running through several of our episodes

Each of us puts a lot of effort into that work, in that “forward” direction.

Tracy and I were talking about using AI to generate podcasts by going in the other direction. What if we took our own work, and used AI to generate new podcasts?

If it was done well, the AI could generate great podcasts, in my voice— me doing host-on-mic, from the things I have written.

That’s an interesting idea…

ɕ

My mind is an emulation

October 20

My mind is [apparently] a poor emulation of every movie ever created.

I mean, can you reliably tell whether you are an actual human in base reality or an upload/simulation?

Or to extend this even further:

Is there any text conversation that somebody could have with you that would convince you that you are actually a machine?

What a weapon that would be. What a cursed SMS that would be to receive.

IF YOU’RE READING THIS, YOU’VE BEEN IN A COMA FOR ALMOST 20 YEARS NOW. WE’RE TRYING A NEW TECHNIQUE. WE DON’T KNOW WHERE THIS MESSAGE WILL END UP IN YOUR DREAM, BUT WE HOPE WE’RE GETTING THROUGH.

~ Matt Webb, from Turing test variations

First off, there are three different things I wanted to quote and three different directions. Go read the thing. I wish I had written it. Hello? Could the programmers running my brain in the simulation please drop the upgrade soon?! I digress.

As I read the above (the bit I quoted) I had a few thoughts…

One — There’s an idea—I recall it being called a “scissors” or a “shears”—which breaks a mind (human, but the discussion was also about an AI’s mind) once you have the idea. I mean: There’s a discussion of whether or not there can exist such a scissors. I’m sure I’ve read about this; I would have swore I blogged about it. But I can’t even find the discussion on the Internet. What I have found is discussion about the discussion with references to the discussion being deleted and moderator-blocked for a few years. Apparently, because if such a scissors actually exists . . . *bonk* Therefore, my first thought after reading the quoted bit above is that I think that “scissors” once broke my brain, and caused an emulator crash. And the information was mostly erased before I was restarted.

Two — Have you seen the film, The Thirteenth Floor? No spoiler! Go watch.

Three — Making this connection, as I read, made the hair stand up on my arms. Have you see the film, Prince of Darkness? No Spoiler! Go watch.

ɕ

Living the dream

October 19

Most of us are “living the dream”—living, that is, the dream we once had for ourselves. We might be married to the person we once dreamed of marrying, have the children and job we once dreamed of having, and own the car we once dreamed of buying. But thanks to hedonic adaptation, as soon as we find ourselves living the life of our dreams, we start taking that life for granted. Instead of spending our days enjoying our good fortune, we spend them forming and pursuing new, grander dreams for ourselves.

~ William B. Irvine

slip:4a590.

A guide to the good life

October 18

I wrote this book with the following question in mind: If the ancient Stoics had taken it upon themselves to write a guidebook for twenty-first-century individuals—a book that would tell us how to have a good life—what might that book have looked like? The pages that follow are my answer to this question.

~ William B. Irvine, from A Guide to the Good Life

slip:4a2gu1a.

There are a lot of books and ideas that get put forward when talking about Stoicism. This book is the best place to start. I wish I had found it much closer to when it was written in 2009. I wish it had been written 30 years sooner, and that I’d found it back then.

The Stoics’ interest in logic is a natural consequence of their belief that man’s distinguishing feature is his rationality. Logic is, after all, the study of the proper use of reasoning.

Ibid p33

[…] when the Stoics counsel us to live each day as if it were our last, their goal is not to change our activities but to change our state of mind as we carry out those activities. In particular, they don’t want us to stop thinking about or planning for tomorrow; instead they want us, as we think about and plan for tomorrow, to remember to appreciate today.

Ibid p71

[W]hen Stoics contemplate their own death, it si not because they long for death but because they want to get the most out of life. As we have seen, someone who thinks he will live forever si far more likely to waste his days than someone who fully understands that his days are numbered, and one way to gain this understanding is periodically to contemplate his own death.

Ibid p200

ɕ

Generously reposting our work

October 16

The social media practice called “Throwback Thursdays” came up here a few weeks ago. At the time, I mentioned that I’ve done this in the past—but usually only very randomly. I’ve recently started being systematic about reposting my older work, on Thursdays.

I think this is a generous gift to our audiences. There are so many great podcasts out there, that we shouldn’t expect our listeners to notice our work after just one, initial “here I made this.” (Plus it’s a fun walk down memory lane.)

The hardest part is quickly picking which of our episodes to post about…

There’s a handy tool for that. This little web page will kick out a random number for you, https://www.calculator.net/random-number-generator.html

Then on Thursdays, just go through your usual social media posting process.

Don’t forget to tag it “TBT” — usually just adding a “#TBT” at the bottom gets the job done.

ɕ

The person before you

October 14

Be as open as possible to the person in front of you. Listen and observe so closely that you can follow up on a sigh, a shake of the head, a change in tone of voice. Take a chance on reflecting your observations back to them without judgment: “You looked amused just now. What’s that about?” Ask the question about feelings that occurs to you, the one you might normally suppress.

Allow silence.

And see what happens. I think you may be astonished at the result.

~ Elaine Appleton Grant, from Great storytelling is built on revealing interviews. Are yours falling flat?

In conversation generally, and in our work as podcasters, is there anything more fundamental than being open?

ɕ

The Work

October 12

After reading the first “book” in Steven Pressfield’s The War of Art, I’m inspired to define, for me specifically: What is “the work?”

In a specific moment, on a specific day, when I feel that odd uneasiness, I will not try to identify the specific form Resistance is taking. Instead as Pressfield mentions on page 12:

[…] We can use this. We can use it as a compass. We can navigate by Resistance, letting it guide us to that calling or action that we must follow before all others.

Rule of thumb: The more important a call or action is to our soul’s evolution, the more Resistance we will feel toward pursuing it.

As a real example of my own experience, Resistance’s compass guides me towards watching sci-fi entertainment (“a harmless relaxation,” I think after working in my yard for hours). So the opposite would be to… organize and streamline my writing environment and processes so tomorrow I can write more easily! No. My hiding in preparation and perfection is just another form of Resistance.

The best way for me is to look at all the possible things I could do, rather than follow Resistance’s compass. Then boil that down to a list of positive, actionable, directions in which I can sit down and work.

In a specific moment, on a specific day, when I feel that odd uneasiness, I can glance at my list and simply do a bit of The Work.

To defeat Resistance I can simply sit down, and do a little bit of any of the following…

  • do guest outreach for podcasting
  • write the next issue of 7 for Sunday
  • write the next Open + Curious article
  • write new blog posts

ɕ

PS: I’ve listed “do guest outreach” because—for me—once I do that consistently for a few weeks, all the rest of the podcasting process follows automatically.

Lake Seneca

October 11

Just a lake-shore.

During a short camping stay, I had the delightful chance to sit between paddling around.

Words don’t really do the feeling justice. I spent decades sailing (beginning in the womb). For many years we went every weekend to the nearest lakes. Lakes, rivers, the Caribbean even. My dad was really into it. Once, my father excitedly got us to the lake for the first, Spring-sailing outing of a season (think: fr-fr-fr-frigid water, rubber wetsuits, die-hard sailors) only to discover the entire lake was STILL FROZEN. Too soon, dad. Too soon. Much fun. Endless stories.

ɕ

What is intention?

October 10

Friend and fellow podcaster Angie Flynn-McIver published a book, Before You Say Anything, in 2021…

Your intention should be something that really lights you up. You need to feel it in your chest, in your gut. It is the verb that connects you to your audience, the umbilical cord that takes your energy to them and brings theirs back to you. Intention is action.

~ Angie Flynn-McIver

slip:4a1471.

Another quote from the book, was a seed for a recent post to Open + Curious…

Active, not passive

Little Box of Quotes — For those who don’t know: The quote at the top is now included in my collection, which I call the Little Box of Quotes — number 1,471 to be exact. There’s a daily email available with just exactly one quote, and nothing more. For several years it was also a daily podcast which to more than 1,000 episodes (still available wherever you listen). All free, just because I like to create things that make the world a better place.

ɕ

Changing the name of a podcast?

October 5

My podcasting-friend, and coach, David Nebinski has a question for podcasters over on his Substack…

Should I change the name of the podcast? I have gone back and forth on changing the name… What do you think? Reply back and let me know!

~ David Nebinski, from Should I change the name…

I don’t have a login for Substack (an intentional choice) so I cannot reply there. ¯\_(ツ)_/¯

I have changed the name of a podcast. The Movers Mindset show had a very different name in the beginning. For me, I realized I was always spending time explaining the meaning of the name— and I realized I wanted to spend that time explaining the show itself rather than the odd name I had.

So I think that’s one reason to change a name: When the name just doesn’t fit.

When people ask me this, I direct them first to Eric Nuzum’s classic question: Can you describe your show in 12 words? …10 words?

Our show titles don’t have be our one (or 2 or 3 even) word answer to that question. But I think they have to at least not detract from describing our show.

ɕ

Appearance on the Visible Voice podcast

October 4

Mary Chan is a skilled hostess with a gift for great conversation. Her show is laser-focused on helping other podcasters grow and excel. She recently invited me to join her— …actually, stop reading. Go listen!

Even if we’re lucky enough to “do what we love,” work tends to have tedious aspects we don’t like. Craig Constantine is a passion podcaster who, through the creation of thousands of episodes and rigorous reflection, has built a workflow that simplifies every step of his method, from prep to publication.

~ Mary Chan, from The Podcaster’s Guide to a Visible Voice

Simplify Your Workflow to Keep Your Podcasting Passion Alive with Craig Constantine – Episode 84

ɕ

Can you?

October 4

…paddle a canoe.

On a whim, I’ve recently being trying to get into canoeing. I’m surprised to discover that I’m quite good at paddling a canoe… but I’m not sure when/how I ever learned. Seriously. This strikes me as very strange.

ɕ