Manny's Reviews > Echoes of Solitude: a short story by o1-preview

I've spent most of the last week experimenting with o1-preview, aka "Strawberry", OpenAI's new language model. It is based on Chain-of-Thought reasoning and reinforcement learning, and seems to represent another large jump in AI capabilities.

The first thing I did was to check how well o1-preview did at playing Tic-Tac-Toe, a notorious stumbling block for previous Large Language Models. In sharp contrast to GPT-4, o1-preview played the game perfectly with no special instructions at all. In the CoT trace, I could see it spotting all my threats and blocking them.

I then integrated o1-preview into C-LARA, our open source language learning platform. Again, it did much better than GPT-4. The difference in quality was particularly noticeable on the task of glossing English in Ukrainian, a challenging language pair we have been investigating, with limited success, for some time.

This is our most substantial test yet. GPT-4 is good at writing short texts, but my experience is that it has trouble composing anything longer than about 750 words and keeping things coherent. Here, I asked o1-preview to write a 3000 word science-fiction story and again it had no problems. We went through an outline and four drafts, where at each stage I suggested large rewrites. It kept up effortlessly, with the CoT summary showing it thinking sensibly about the issues and responding to my ideas.

We have just posted a report on ResearchGate, here, with the text of the story and full details of how it was written. We'd love to know what people think.
_________________

Taylor, in their kind review, compares o1-preview's story to the early Asimov. That makes me John W. Campbell Jr.

Campbell was not a very nice man but a terrific editor, so I'm happy to be standing in for him!