“Reading gives us someplace to go when we have to stay where we are.” ― Mason Cooley
Generative Pre-trained Transformers are all the rage. Recent language models, such as GPT-3, generate written text of such quality that is often difficult to differentiate from a text written by a human. In May 2020 OpenAI a San Francisco-based artificial intelligence research lab published the paper about GTP-3 and hundreds of media outlets have written about the system and its capabilities: The New York Times published an op-ed about it , Guardian published an article written by the machine.
Over the past couple of years as I have experimented with NLP algorithms I have seen the evolution from simplified bag of words to drastic contextual word representations that allowed for leapfrogging in in human language translation. As I played with these algorithms my main use case was the desire for the computer to comprehensively summarize things for me, yet without losing specific details. However, here are the additional interesting use cases:
text to command - you write plane English and algorithm will translate it into computer language
classification - given high volume of information, come up with good taxonomy to classify content (beyond generic categories)
“summarization -“ aka “summarize for a 2nd grader - take a complicated text (think legal papers) and translate it into a big body of work.
But it seems like GPT-2 and GPT-3 have made huge progress in these areas as well as tackled additional ones such as conversation (Q&A/Chatbots) as well as variations (given a prompt machine completes the sentence). But there are lots of skeptics about the robustness and usefulness of transformers. One of the most vocal critics is Gary Marcus (There is an amazing debate of Yann LeCun vs. Gary Marcus about the nature of Artificial Intelligence. I am definitely team Gary). He makes an argument that machines do not learn about the world - they learn about the text and how people use words in relation to other words. and hence no matter how impressive are the results, they remain “spotty and pointillistic, arguably useful and certainly impressive, but never reliable”. He points out the necessity for biological, physical, social and commonsense reasoning when dealing with text and these are the intricacies that unfortunately can not be grasped from the high volume of tests.
More data makes for a better, more fluent approximation to language; it does not make for trustworthy intelligence.
He is not alone, here is additional skepticism on transformers:
* Douglas Summers-Stay gave this methaphor : “GPT is odd because it doesn’t ‘care’ about getting the right answer to a question you put to it. It’s more like an improv actor who is totally dedicated to their craft, never breaks character, and has never left home but only read about the world in books. Like such an actor, when it doesn’t know something, it will just fake it. You wouldn’t trust an improv actor playing a doctor to give you medical advice.” Douglas Summers-Stay
* nostalgebraist compared GPT-3 to a 10-year-old prodigy: “No, my 10-year-old math prodigy hasn’t proven any new theorems, but she /can/ get a perfect score on the math SAT in under 10 minutes. Isn’t that groundbreaking?”
As OpenAI is gearing up for commercial applications I started examining how I would feel about linguistic augmentation of humans. Overall I think I am a proponent of human augmentation - getting marginal advantage with the help of genetic enhancement and algorithmic thinking should be considered a natural extension of using technology as an aid in our lives. However, I am uncertain how I would feel getting emails from my friends, knowing that they did not necessarily write them and just fed “bullet points” to the machine.
Further reading:
These are Louis Vuitton Book trunks. /During an era where the paperback books did not exist, transporting one’s books as a bibliophile or writer was a complicated step. Gaston-Louis Vuitton, a lover of literature, decided to take charge of this problem in order to allow everyone to carry their books during their trip, or conversely, to bring back rare editions from the other side of the world. I have seen those in 2017 when LV did a pop up exhibit in FiDi. Sorry, no kindle can compare to traveling in style with these.
More here. Also found 17th-century traveling library case but not as impressive
On the similar note - this year I have finished 90 books. As everyone will be doing `end of year` book recommendation lists, here are a couple from me:
* any works by Vaclav Smil - the man has shaped how we think about energy
“Gotham: A history of New York City to 1898” monumental work that is meticulously researched. I would not stop recommending it to anyone who is interested in New York or history or how we shape our world - for those who don’t like reading it comes as 67 hours of an audiobook.
Richard Hamming’s “You and Your Research” is a mantra to approach one’s work.
* And everyone should be happy that I have discovered Roy Peter Clark - he made my writing much more digestible.
Most of my books this year were ordered at McNelly Jackson South Str Seaport and it was a highlight during pandemic to get emails from them that “your order has arrived to the store”. I will get dressed up and walk to pick up my books in the bookstore. Instead of boxes I would carry them in paper bags. Sometimes they would be heavy. And no, I did not miss getting my books from Amazon….
“Almost every essayist concedes that the concept of dignity remains slippery and ambiguous. In fact, it spawns outright contradictions at every turn. We read that slavery and degradation are morally wrong because they take someone’s dignity away. But we also read that nothing you can do to a person, including enslaving or degrading him, can take his dignity away. We read that dignity reflects excellence, striving, and conscience, so that only some people achieve it by dint of effort and character. We also read that everyone, no matter how lazy, evil, or mentally impaired, has dignity in full measure. Several essayists play the genocide card and claim that the horrors of the twentieth century are what you get when you fail to hold dignity sacrosanct. But one hardly needs the notion of “dignity” to say why it’s wrong to gas six million Jews or to send Russian dissidents to the gulag.” Steven Pinker on dignity “The stupidity of dignity”
“poison cabinet”- is a hollowed out book from the 16th century. In the pages’ place are eleven drawers of varying sizes with meticulous labels, each spelling out which plant each drawer contained.
Monogamous vs. Polygamous Reading Do you read one book at a time? Or do you have two or more going at once — say, a novel, a nonfiction book and a collection of short stories? Alison Morris, has identified several species of polygamists. They include: the Whimsical, who dips in and out of multiple books at once; the Placebound, who reads several books simultaneously but each in its own locale — at home, at work, on the subway; and the Noncompetitive, who reads one fiction book and one non-fiction book at the same time, or some similar combination. There’s no wrong way to read. Some bibliophiles devour multiple books at once while others savor a single book at a time. I am still making up my mind…
I have been reorganizing the articles and found this gem of outfits to work from home . Published in December 2018 . Little did she know that they will be so handy, oh so handy in 2020…