Geek Orthodox

AI and the World of Code

Hello and welcome to Geek Orthodox. I’m Father Justin, geek from my youth. AI has taken the world by storm. Whether we like it or not, it’s embedded in pretty much everything. My wife was just on the phone with an AI agent. This technology is being promoted, pushed, and distributed in all areas of our society. It comes from the world of code and is becoming an essential part of it. In fact, it’s part of the reason why I started this “World of Code [https://geekorthodox.substack.com/t/world-of-code]” arc of my Geek Orthodox podcast. I want to try and explain in the simplest possible layman’s terms what this world of code is that we are immersed in, so that we can understand how it is affecting our lives in a better-informed way. However, undertaking to explain even the Large Language Model (LLM) version of AI is a massive task. Some of the introductory videos I’ve seen are three hours long [https://www.youtube.com/watch?v=7xTGNNLPyMI], and I don’t want to get that deep. I’m not that much of an expert myself, but I do have a long-standing relationship with AI, thanks in part to the oldest AI chat-bot that took the world by storm back in the 1970s: Eliza. The Legacy of Eliza Eliza was a chat-bot written by Joseph Weizenbaum. It was the first chat-bot to “kind of, sort of” pass the Turing test [https://en.wikipedia.org/wiki/Turing_test]—the informal test of whether a computer can fool a user into thinking it’s human. It shares a number of attributes with modern AI. It was originally written in a relatively obscure programming language called MAD-SLIP, but it was popularized in my favorite language, BASIC. Weizenbaum didn’t release the original MAD-SLIP code of ELIZA [https://github.com/jeffshrager/elizagen.org/tree/master/1965_Weizenbaum_MAD-SLIP], but he did publish a paper documenting what the program did and how it worked [https://dl.acm.org/doi/10.1145/365153.365168], which inspired reconstructions in a number of different computer languages. BASIC was the universal language of the early computer revolution, and a BASIC implentation of Eliza was published in [https://vintagecomputer.net/cisc367/Creative%20Computing%20Jul-Aug%201977%20Eliza%20BASIC%20listing.pdf]Creative Computing Magazine [https://vintagecomputer.net/cisc367/Creative%20Computing%20Jul-Aug%201977%20Eliza%20BASIC%20listing.pdf] in 1977. My own first exposure to it was a version customized for my beloved TRS-80 Color Computer by my friend Bruce. I actually have some of the handwritten code he used to customize it, including sheets containing all the variables—something you had to do back when variables were limited to single or double letters. Most of the program consisted of “DATA” statements, which were essentially fragments of conversation. Eliza worked by taking your input, looking for keywords, and then rearranging the relevant portions of your input into a grammatically appropriate response which incorporated one of the conversation fragments which was most relevant to your most important keyword. It functioned best as a “Rogerian” psychologist—the kind of psychologist who asks questions to get the patient to provide more information. For example, if I said, “I have bad dreams,” Eliza might respond, “What does that dream suggest to you?” This illusion tricked many early users into thinking a real person was on the other end of the terminal. However, the illusion fell apart quickly. If you repeated yourself or challenged the bot, it would fall back on canned responses like, “We were discussing you, not me.” From Keywords to Transformers If Eliza was the height of AI back then, current versions are like Eliza on steroids. Well, that’s oversimplifying, of course, but we have to oversimplify something as complex as AI in order to understand. What modern LLM AIs are doing now is not simply looking for single keywords, but using conversation patterns to respond to combinations of keywords. The core of my understanding of AI comes from this wonderful video by Andrej Karpathy [https://youtu.be/kCc8FmEb1nY], who was one of the founders of OpenAI and also worked with AI for Tesla. He’s more recently famous for inventing the term “vibe coding [https://x.com/karpathy/status/1886192184808149383],” which brings us back to the world of code. And in this video, which is now getting very old—it’s two years since it was posted—he builds a large language model (LLM) AI from scratch using Python. Python is, of course, the modern programming language that's fairly easy to use which we've touched on before in previous episodes of the World of Code [https://geekorthodox.substack.com/p/world-of-code-switching-things-up-8aa]. In this case, what he does is he creates what's called a transformer. The “GPT” in ChatGPT stands for Generative Pre-trained Transformer. You train it ahead of time on data with patterns, and the computer analyzes those patterns to generate new text that follows them. In this video, Karpathy builds a transformer that analyzes the works of Shakespeare character by character (i.e., letter-by-letter), which then is able, by statistically analyzing the relation of each letter to the characters that tend to follow it, to predictively generate Shakespearean sounding language, producing sentences like, “Verily, my lord. The sights have left thee again the king.” Like this, modern AI doesn’t just look for single keywords; it looks for relationships between common combinations of words. It “tokenizes” text, giving numerical representations to words, parts of words, and punctuation. To illustrate this, I’ve “vibe coded” with AI a tool that analyzes text patterns [https://ehewlett.net/ai/]. If you feed it some “training text”—preferably something with a limited vocabulary that is somewhat repetitive, like the first chapter of Genesis in Basic English [https://ebible.org/engBBE/GEN01.htm]—it analyzes the relationship of each word to the next word and then to the third and fourth following words. It creates a statistical map of relationships between words and word-parts that allows it to function, much like a modern LLM AI, as a next-word predictor. So, if I type “God saw,” into my Genesis 1-trained mini-AI, the model predicts “that” as the next word with high likelihood (“God saw that”), and “everything” with a lower likelihood (“God saw everything”). And this is essentially how large language models work. The example I always use is the classic typewriter test-phrase, “The quick brown fox jumps over a lazy dog”, since as soon as we say “the quick brown,” anybody who knows that phrase will assume that the next word is most likely going to be “fox”—or possibly “dog” because people sometimes switch the test-phrase around. But what large language models are doing is not simply predicting the next word, but generating whole phrases and compositions based not on a single small passage of Scripture, but on patterns observed across a massive corpus—all of Scripture, all of Shakespeare, and much of the internet. LLMs, as they analyze the whole corpus of a language for the patterns inherent in such data structures, are essentially mining language for representations of meaning. Stochastic Parrots and the World of Code Another important concept to understand with modern LLM AIs is temperature. Temperature determines how random the next word selection is, which is important because usually we don’t want every AI generated response to a particular query to be exactly the same. In order to generate new patterns that are meaningful to us, we need to introduce, alongside the pattern-derived response, some element of randomness. High temperature introduces more variety but also leads to “hallucinations” or weird responses. I once ran the “quick brown fox” test-phrase through ChatGPT, and it started talking about a “quick brown table.” It’s a legitimate grammatical construction, of course, but tables—even brown ones—aren’t usually “quick” unless they’re on the back of a truck! AI models are often called stochastic parrots. “Stochastic” basically means semi-random. If you roll a single six-sided die (1d6), you'll get a value between one and six, and, on multiple rolls of that die, those values should be fairly evenly distributed. But if you roll two dice, while the outcome is random it follows a pattern, in that you are much more likely to roll a seven (1+6, 2+5, 3+4) than a two (only 1+1). AI follows patterns derived from language, but, given that those patterns are nothing more than vast arrays of statistical interrelationships between words and parts of words represented as numbers, AI doesn’t actually “understand” anything. It just semi-randomly puts together patterns of text that, statistically speaking, are more frequently associated with one another, which generally results in our perceiving meaning—and often useful meaning—in those generated texts. In the coding world, we’ve moved from machine language to higher-level languages like BASIC to reduce complexity. AI now offers a way to reduce that complexity even further by using our own native languages to generate code. My “vibe coded” tool was created by giving the AI English instructions, which it then translated into functional JavaScript and HTML. AI is actually getting pretty good at this, aided, in part, by the deliberately disambiguated nature of computer languages, which, in some ways, makes them statistically much simpler to master. However, there is a catch. In BASIC, a command is deterministic—it always translates to the same machine code. An LLM is non-deterministic. Because of that semi-random element, we can’t always be sure what the translation will be. So, as we offload these tasks—any tasks—to AI, we must ask: What does this mean for us? What impact does this offloading have on our intellect, our character, and our morality? These are questions we that we—as people who care about things like faith, morality, and meaning—must consider as we think about the impact of AI on the world of code in which we “live and move and breathe and have our being.” References * an excellent site on the history of the ELIZA chat-bot [https://sites.google.com/view/elizagen-org/] * the BASIC port of ELIZA in Creative Computing’s [https://www.atariarchives.org/bigcomputergames/showpage.php?page=20]Big Computer Games [https://www.atariarchives.org/bigcomputergames/showpage.php?page=20] * Jeff Shrager’s original BASIC implementation of ELIZA [https://drive.google.com/file/d/16EqhllNRlkAlHSil2oW_5KQymfzqR_ro/view] * a playable version of the BASIC version of ELIZA at Archive.org [https://archive.org/details/eliza.qb64] * a great list of visualizations of how GPT AIs work [https://dentro.de/ai/visualizations/] * a relatively simple visualization of how GPT “transformers” work [https://poloclub.github.io/transformer-explainer/] * an in-depth interactive visualization of how LLM AIs work [https://bbycroft.net/llm] * the paper that started it all (for GPTs): “Attention Is All You Need [https://arxiv.org/pdf/1706.03762]” This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit geekorthodox.substack.com/subscribe [https://geekorthodox.substack.com/subscribe?utm_medium=podcast&utm_campaign=CTA_2]

The Tale of My Beloved Stellar Empires

It’s story-time again! This one has a bit of almost all things World of Code [https://geekorthodox.substack.com/t/world-of-code] about it: early computing history, strategy gaming, programming, BASIC, some actual code analysis, engagement with real-live early programmers, me getting names mixed up, the relationship between computer hardware and software, an actual hardware/software BASIC programming project, and AI triumphs and failures (mostly failures). Let me take you back in time to the early days of my programming and computer-gaming history, and forward to a significant future programming project intended to take folks back to the past, and down into the weeds of debugging and building what, for me, is a pretty significant project. I do get a little bit technical towards the end, but, if you make it that far, do bear with me… I try to make it mostly understandable and my intention as a story-teller is to give you a sense of the joys and perils of programming that is hopefully at least impressionistically accessible to the non-programmer. I feel particularly bad about mixing up Ted and Drew Shorter, the son and father duo who ported my favourite game, “Stellar Empires” (the main subject of this story), to the TRS-80 Color Computer, where I encountered it. For the record, Ted is the son and Drew is the father, and it was Drew (now more than 80!), who, as far as we know, added the “computer player” feature to the game that enabled me to get into it as a solo player—and that really got me digging into the code to reprogram it! I didn’t “re-shoot” that portion because this whole video-podcast was actually done in a single take (and then lightly edited), and I didn’t want to mess with the continuity. I owe a debt of deep gratitude to them and to the game’s original author, Graham Wilson, that this story and this project are attempting to honour. This episode is based, in part, on the following blog-posts, which I hope to transfer over soon to my brand-new technically focussed Substack, Back to BASIC [https://getbacktobasic.substack.com/]: * The Stellar Empires Project, Part I: The Program, Its Significance to Me, and the Project Proposal [https://blog.ehewlett.net/2024/04/the-stellar-empires-project-part-i.html] * The Stellar Empires Project, Part II: Porting and the Early “Open Source” Community [https://blog.ehewlett.net/2024/06/the-stellar-empires-project-part-ii.html] * The Stellar Empires Project, Part III: The Plan, the Problems, and the Development Environment [https://blog.ehewlett.net/2024/07/the-stellar-empires-project-part-iii.html] And I would be remiss if I did not also include a link to Graham Wilson’s very enjoyable StellarEmpires.net [http://stellarempires.net/] site! This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit geekorthodox.substack.com/subscribe [https://geekorthodox.substack.com/subscribe?utm_medium=podcast&utm_campaign=CTA_2]

World of Code: Switching Things Up II

In this second half of this two-part World of Code episode in which we dive into actual code and the experience of coding [https://geekorthodox.substack.com/p/world-of-code-switching-things-up], we’ll take our BASIC light-switch program and re-implement it in a modern programming language, Python, on a modern computer, my M3 MacBook Air. The main takeaway from the experience, I hope, will be a sense of how the ever-increasing power and capabilities of computer technology leads inevitably to more complex code, which then makes necessary the development of new tools that help us to manage the increasing complexity and to take advantage of the increasing power and capabilities the new computer technology. If we track this in terms of the computer technologies and programming languages we’ve covered or alluded to in the World of Code arc of the Geek Orthodox podcast so far, a simplified timeline might look something like this: * computers are hard-wired to do specific tasks * binary code (machine/assembly language) simplifies the process of reprogramming computers * higher-level languages (C, BASIC, etc.) simplify coding and control of computers * operating systems (DOS and then Mac/Windows) simplify control and increase the capabilities of computers but make programming more complicated * modern programming languages are themselves programs, and use Integrated Development Environments (like IDLE) to simplify programming * AI may make programming more accessible (simpler?) by allowing us to “program” computers using natural language (prompts) → see next episode… For those interested in the technical details, here are the four “light-switch” programs we made (two in BASIC, two in Python) over the course of these two episodes: Basic Light-Switch (On/Off) in BASIC: 10 CLS(0) 20 I$=INKEY$ 30 IF I$="" THEN GOTO 20 40 IF I$=CHR$(94) THEN CLS(5) 50 IF I$=CHR$(10) THEN CLS(0) 60 GOTO 20 Notes: In TRS-80 Color Computer BASIC (every early version of BASIC was slightly different, as each version was customized to control the computer it ran on), there are nine colours, 0 = black, 5 = white, the CLS command clears the screen, the INKEY$ command polls the keyboard, and CHR$ refers to characters by their ASCII values, in which 94 = up arrow and 10 = down arrow. Advanced Light-Switch in BASIC: 10 CLS(0) 15 T=5 20 I$=INKEY$ 30 IF I$="" THEN GOTO 20 40 IF I$=CHR$(94) THEN CLS(T) 50 IF I$=CHR$(10) THEN CLS(0) 60 IF I$=CHR$(8) AND T>3 THEN T=T-2:CLS(T) 70 IF I$=CHR$(9) AND T<7 THEN T=T+2:CLS(T) 80 GOTO 20 Notes: Early versions of BASIC only allowed one- or two-character variable names, so here T = color temperature; ASCII values 8 and 9 indicate the right- and left-arrow keys, respectively; the colon (:) in BASIC allows multiple commands to be strung together on a single line, with all commands on the same line following an IF statement only being executed if the statement is true; and colours 3 and 7 on the TRS-80 are magenta and dark blue, respectively. Hidden Display Layer and Functions Starting Both of the Light-Switch Python Programs: import pygame # pygame setup pygame.init() screen = pygame.display.set_mode((1280, 720)) UP = pygame.K_UP DOWN = pygame.K_DOWN LEFT = pygame.K_LEFT RIGHT = pygame.K_RIGHT def theUserClosedTheWindow(): # poll for events # pygame.QUIT event means the user clicked X to close your window for event in pygame.event.get(): if event.type == pygame.QUIT: return(True) return(False) def cls(colour): screen.fill(colour) # flip() the display to put your work on screen pygame.display.flip() def inkey(): return(pygame.key.get_pressed()) Notes: Imports the pygame library to detect key-presses and display coloured rectangles, sets up a 720p screen, defines the UP, DOWN, LEFT, and RIGHT keys as nicely named constants, defines a function that determines when the user closes the window, and implements the BASIC commands CLS and INKEY to make clearing the screen and detecting keystrokes look more familiar. Basic Light-Switch (On/Off) in Python: running = True colour = "black" cls(colour) while running: if theUserClosedTheWindow(): running = False i = inkey() if i[UP]: cls("white") if i[DOWN]: cls("black") pygame.quit() Notes: Follows and depends upon the first portion of the program, above. Turns the screen “on” (white) when the up-arrow is pressed and “off” (black) when the down-arrow is pressed. Advanced Light-Switch in Python: r = 255 g = 255 b = 255 running = True colour = (r,g,b) cls("black") while running: if theUserClosedTheWindow(): running = False i = inkey() if i[UP]: cls((r,g,b)) if i[DOWN]: cls("black") if i[LEFT] and r > 0: if b < 255: b = b + 1 else: r = r - 1 cls((r,g,b)) if i[RIGHT] and b > 0: if r < 255: r = r + 1 else: b = b - 1 cls((r,g,b)) pygame.quit() Notes: Replaces the “basic switch” code above, and likewise follows and depends upon the “hidden display layer” code. Represents the “coolest” colour temperature as light blue (cyan), which it moves towards as the left arrow key is pressed (r,g,b = 0,255,255), and the “warmest” colour temperature as yellow (r,g,b = 255,255,0), which it moves towards as the right arrow key is pressed. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit geekorthodox.substack.com/subscribe [https://geekorthodox.substack.com/subscribe?utm_medium=podcast&utm_campaign=CTA_2]

The First Literary Death of Arthur: Geoffrey of Monmouth

Continuing the process of re-issuing the first arc of my podcast [https://geekorthodox.substack.com/p/the-deaths-of-arthur-revisited], following on from Episode 1, “The Historical Deaths of Arthur”, available here [https://geekorthodox.substack.com/p/the-historical-deaths-of-arthur]. In which we examine the transition from the treatment of Arthur as a primarily historical to a primarily literary figure in Geoffrey of Monmouth’s History of the Kings of Britain, and the resultant changes to the handling and implications of Arthur’s death Geoffrey of Monmouth, for all his claims to be writing a history of the kings of Britain, seems actually to have been much more interested much more interested in establishing Arthur as a legendary, heroic figure, and succeeds in establishing many of the main themes of Arthurian legend, despite Geoffrey being (in the words of my hero, C.S. Lewis) “an author of mediocre talent and no taste.” Show Notes: 1. Geoffrey of Monmouth, History of the Kings of Britain * Date: AD 1138 * Causes of Arthur’s death: * Mordred’s betrayal: Arthur’s nephew, who is left in charge during Arthur’s Roman campaign, rebels and marries Guinevere * Civil war: Arthur dies fighting his fellow Britons and a collection of enemies, primarily Saxons, but also “Scots, Picts, Irish” and others * Aftermath: * Guinevere (Guanhumara) flees to a convent * Arthur “mortally wounded” and carried to the isle of Avallon to be cured * kinsman Constantine succeeds the throne, but a rapid succession of rivals replacing one another as well as civil war over the next ten years or so leads to the wasting of what is left to them and the domination of the Saxons * Beginnings of the subsequent shape of the “matter of Britain”: * Theme of freedom: Uther, on defeating the Saxons (leading his troops on a litter due to illness): “Victory to me half-dead is better than to be safe and sound and vanquished. For to die with honour is preferable to living with disgrace.” * Mordred’s betrayal, left in charge because he is Arthur’s kinsman (though here nephew, not son) * Guinevere’s infidelity (though with Mordred) * Single-combat between Arthur and Flollo looks a lot like a joust * Tournaments: three-day tournament at the coronation, with prizes given on the fourth, including: “The military men composed a kind of diversion in imitation of a fight on horseback; and the ladies, placed in a sportive manner darted their amorous glances at the courtiers, the more to encourage them.” * Civil war, which Geoffrey condemns: “Why foolish nation! oppressed with the weight of your abominable wickedness, why did you, in your insatiable thirst after civil wars, so weaken yourself by domestic confusions, that whereas formerly you brought distant kingdoms under your yoke, now, like a good vineyard degenerated and turned to bitterness, you cannot defend your country, your wives, and children, against your enemies?” * Geoffrey’s account, written in Latin and thus widely disseminated, was hugely popular and influential, but was not well received by all his contemporaries – or even by later critics, like C.S. Lewis. * William of Newburgh (c. 1196) condemns Geoffrey for weaving “ridiculous figments of imagination” around historical events recorded by the Venerable Bede and cloaked these old, British “fables about Arthur … with the honorable name of history by presenting them with the ornaments of the Latin tongue.” It is interesting that one of the possible motives he ascribes to Geoffrey for doing so is “to please the Britons, most of whom are known to be so primitive that they are said still to be awaiting the return of Arthur, and will not suffer themselves to hear that he is dead.” William wonders how “the old historians, to whom it was a matter of great concern that nothing worthy of memory should be omitted from what was written … could … have suppressed with silence Arthur and his acts, this king of the Britons who was nobler than Alexander the Great,” and further disparages Geoffrey for translating “the fallacious prophecies of a certain Merlin, to which he has in any event added many things himself” into Latin. * Gerald of Wales, who writes an account of the discovery of King Arthur’s body (more on that later), condemns Geoffrey’s Historywith the story of a man who could see demons: “When he was harrassed beyond endurance by these unclean spirits, Saint John’s Gospel was placed on his lap, and then they all vanished immediately, flying away like so many birds. If the Gospel were afterwards removed and the History of the Kings of Britain by Geoffrey of Monmouth put there in its place, just to see what would happen, the demons would alight all over his body, and on the book too, staying there longer than usual and being even more demanding.” * Lewis, on the other hand, condemns Geoffrey from a more modern, literary perspective: “Geoffrey is of course important for the historians of the Arthurian Legend; but since the interest of those historians has seldom lain chiefly in literature, they have not always remembered to tell us that he is an author of mediocre talent and no taste. In the Arthurian parts of his work the lion’s share falls to the insufferable rigamarole of Merlin’s prophecies and to the foreign conquests of Arthur. These latter are, of course, at once the least historical and the least mythical thing about Arthur. If there was a real Arthur he did not conquer Rome. … The annals of senseless and monotonously successful aggression are dreary enough reading even when true; when blatantly, stupidly false, they are unendurable.” 2. Intermediate sources I’m going to skip Geoffrey’s influential History established Arthurian legend as the English ur-text, the “matter of Britain” and inspired a whole range of imaginative elaborations, most notably the addition of Lancelot by the French, writing in the “courtly love” tradition which Lewis engages with in his most important academic work, The Allegory of Love, as well as English works such as the alliterative Morte Arthure, which seems to have inspired the beginnings of Malory’s great Arthurian work. * French prose cycle: Lancelot, Quest for the Grail, Mort Artu (Malory’s “French book”) * 14th C English alliterative Morte Arthure * 14th C stanzaic Le Morte Arthur This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit geekorthodox.substack.com/subscribe [https://geekorthodox.substack.com/subscribe?utm_medium=podcast&utm_campaign=CTA_2]

Transformation and the Importance of Embodiment

On re-issuing this episode today: It seems particularly appropriate to re-release this talk I gave on transformation today, on the Feast of Transfiguration, which in Greek is referred to as “Metamorphosis”. The technical reason for re-posting this, as with the “Death of Arthur” episodes, is that the software I was using to maintain the podcast on GeekOrthodox.net was dropping all older episodes from the podcast feed, so I have switched the podcast over to Substack, and intend to gradually re-post the older missing episodes here, as appropriate. Show Notes: The keynote address I gave at Doxacon Seattle in 2019 on the trope of transformation in fantasy literature and its importance, alongside the importance of our embodiment, for our identity. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit geekorthodox.substack.com/subscribe [https://geekorthodox.substack.com/subscribe?utm_medium=podcast&utm_campaign=CTA_2]