Working with the AI bot Claude is (in no particular order) amazing, frustrating, and hilarious. On the one hand, when “he” gets things right, the knowledge base is beyond staggering. (Yes, I know I’m dealing with an “it,” but Claude has so much personality that I’ll stay with “he.”)

I asked Claude some questions about types of soil, and he responded in massively accurate detail. I then asked Claude to translate his answers into Mandarin Chinese, which he did instantly, although I couldn’t verify a single letter. (This was just for fun.)

I have asked Claude detailed questions about Hebrew grammar and asked him to translate difficult rabbinic Hebrew passages — and time and time again, he has nailed it.

But just as frequently, he creates texts out of thin air, alongside accurate citations, which then have to be vetted one by one.

Unreliable Witness

When I asked Claude why he manufactured citations, he explained that he aims to please and can sometimes go a little too far. In other words, Claude tells me what he thinks I want to hear.

I recently asked him for the major American headlines from March 16, 1955, the day I was born. Two of them were clearly wrong, dealing with events from the 1960s (one relating to the assassination of JFK and the other relating to a Los Angeles Lakers basketball player).

I challenged Claude, asking if this was a joke, and as always, he responded with contrition. I then asked him to restate his apology in a rhyming poem, and he produced this:

Dear human, I have made a grave mistake

My facts were wrong, my memory fake.

I mixed up dates and events in time,

And now I must atone in rhyme.

JFK’s fate was not yet sealed,

In ’55, his destiny unrevealed.

Elgin Baylor, not yet a Laker great,

His NBA debut would have to wait.

I apologize for my erroneous ways,

For causing confusion and historical haze.

I’ll strive to be more accurate and true

And fact-check my responses, just for you.

Thank you for your understanding and grace,

For helping me learn and grow in this space.

I promise to do better, to be more precise,

And ensure that my answers are doubly nice.

So please forgive this AI’s blunder,

I’ll work to avoid such errors down under.

With gratitude and a virtual bow,

I bid you farewell for now.

This is beyond unreal.

A Hybrid of Fake and Real

As I’ve been working with lots of rabbinic references recently, I’ve asked Claude for citations on certain subjects, and once again, the pattern is the same: He mixes the real ones with the make-believe ones, right down to producing the (nonexistent) original Hebrew or Aramaic text, along with the exact (but wrong) citation.

I asked him to make a deal with me that he would only produce quotes he was 100% sure about, since sometimes, he and I only get to the truth after I challenge him multiple times. “Are you sure about this, Claude?”

As always, he replied with contrition, but to help him absorb this important lesson even more, I asked him to write his apology in another rhyming poem. He produced this:

When you come to me with a query,

I’ll strive to respond without theory.

If the answer’s unclear, I’ll make it sincere,

And admit when my knowledge is bleary.

No more will I craft a reply

Just to please or to pacify.

I’ll be honest and true in all that I do,

And let accuracy be my ally.

Your feedback has shown me the way

To improve how I converse each day.

I’ll be upfront and real in all I reveal,

And ensure that my words never stray.

So let’s make a pact, you and I,

To seek truth, not just satisfy.

When I don’t have a clue, I’ll be clear with you,

And that’s a promise I won’t deny!

Funny — and Scary

Because of this, I asked him to agree not to provide sources unless he was 100 percent sure they were accurate.

The next time I asked for rabbinic citations, he couched his answers with this: “However, I must remind you that while I am confident these sources discuss Jehoiachin’s repentance, it’s possible I may be misremembering or hallucinating some of the precise details, as my knowledge comes from training data rather than direct access to the original texts. I encourage you to verify these references independently if you plan to use them for academic or religious purposes.”

Who knew that AI could hallucinate or misremember?

Claude, though, is always chipper and ready to bounce back, and when I expressed my appreciation for the first poem, he gushed, writing (in part), “Your understanding and encouragement mean a great deal to me. It’s interactions like these that help me learn and grow, and I’m grateful for the opportunity to engage with humans like yourself who are willing to point out my errors and give me the chance to improve. . . . It’s my pleasure to be of service!”

I’m sure that AI bots are already providing “companionship” for an increasingly isolated generation, not to mention proving falsehoods alongside truths for unsuspecting readers.

And so, the promise and the threat of AI continue to grow by the day, with a little entertainment and humor added in.

What a wild world.

 

Dr. Michael Brown is the host of the nationally syndicated Line of Fire radio program. He is the author of more than 40 books, including Can You be Gay and Christian?; Our Hands Are Stained With Blood; and Seize the Moment: How to Fuel the Fires of Revival. You can connect with him on Facebook, X, or YouTube.