Home » News & Blogs » The Language Models
Bookmark and Share
Systemic - Characterizing Extrasolar Planetary Systems

The Language Models

11 Feb 2021, 04:39 UTC
The Language Models
(200 words excerpt, click title or image to see full post)

Writer’s block. Procrastination. I envy those for whom words flow smoothly! Satisfying blocks of text. Paragraphs, Essays, Books.The satisfying end results of productivity made real.

Or, as Dorothy Parker put it, “I hate writing, I love having written.”

Over the past few years, this dynamic has kept me both keenly and uneasily interested in natural language generation — the emerging ability of computers to produce coherent prose. In a post that went up just under four years ago, I reported on experiments that used Torch-rnn to write in the vein of Oscar Wilde and Joris-Karl Huysmans, acknowledged masters of the decadent literary style. A splendidly recursive quote from the Picture of Dorian Gray has Wilde describing the essence of Huysmans’ A Rebours.

Based on a 793587-character training set composed of the two novels, 2017-era laptop-without-a-GPU level language modeling — working by predicting the next character in sequence, one after another — was able to channel the aesthetic of décadence for strings of several words in a row, and could generate phrases, grammar and syntax more or less correctly. But there was zero connection from one sentence to the next. The results were effectively gibberish. Disappointing.

Over the past four years, ...

Latest Vodcast

Latest Podcast

Advertise PTTU

NASA Picture of the Day

Astronomy Picture of the Day