{"version":"https://jsonfeed.org/version/1","title":"Micro.blog - Thomas Lodato","home_page_url":"https://micro.blog","feed_url":"https://micro.blog/posts/deptofthomas","_microblog":{"about":"https://micro.blog/about/api","id":"134888","username":"deptofthomas","bio":"Dept of Thomas is the writing and thinking of Thomas Lodato. Thomas is a individual contributor in the Innovation Studio at Mozilla focused on generative AI.","pronouns":"he/him","is_following":false,"is_you":false,"following_count":2,"discover_count":0},"author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://avatars.micro.blog/avatars/2023/35/134888.jpg"},"items":[{"id":"34286474","content_html":"<p>Follow up: they are the same resolution (72 DPI)</p>\n<div class=\"microblog_collection\">\n<img src=\"https://cdn.micro.blog/photos/1000x/https%3A%2F%2Fdeptofthomas.micro.blog%2Fuploads%2F2024%2Fscreenshot-2024-03-26-at-4.12.30pm.png\" width=\"600\" height=\"455\" alt=\"\" loading=\"lazy\"><img src=\"https://cdn.micro.blog/photos/1000x/https%3A%2F%2Fdeptofthomas.micro.blog%2Fuploads%2F2024%2Fscreenshot-2024-03-26-at-4.12.40pm.png\" width=\"600\" height=\"230\" alt=\"\" loading=\"lazy\"><img src=\"https://cdn.micro.blog/photos/1000x/https%3A%2F%2Fdeptofthomas.micro.blog%2Fuploads%2F2024%2Fdalle-2024-03-26-16.10.27create-a-pixelated-retro-style-image-of-a-yellow-f.webp\" width=\"600\" height=\"600\" alt=\"\" loading=\"lazy\"><img src=\"https://cdn.micro.blog/photos/1000x/https%3A%2F%2Fdeptofthomas.micro.blog%2Fuploads%2F2024%2Fdalle-2024-03-26-16.10.22generate-a-simple-image-of-a-yellow-face-with-a-sh.webp\" width=\"600\" height=\"600\" alt=\"\" loading=\"lazy\">\n</div>\n","summary":"","url":"https://deptofthomas.micro.blog/2024/03/26/follow-up-they.html","date_published":"2024-03-26T23:14:54+00:00","author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://cdn.micro.blog/photos/96/https%3A%2F%2Favatars.micro.blog%2Favatars%2F2023%2F35%2F134888.jpg","_microblog":{"username":"deptofthomas"}},"_microblog":{"date_relative":"2024-03-26 23:14","date_timestamp":1711494894,"is_favorite":false,"is_bookmark":false,"is_deletable":false,"is_conversation":false,"is_linkpost":false,"is_mention":false,"note":"","syndication":[]}},{"id":"34285977","content_html":"<p>When asked to make an image lower res(olution), ChatGPT refuses because of built-in quality standards. I imagine this is something that could be “fixed,” but it points out that while we can create seemingly anything with models, doing so also flattens expression in unforeseen ways.</p>\n<img src=\"https://cdn.micro.blog/photos/1000x/https%3A%2F%2Fdeptofthomas.micro.blog%2Fuploads%2F2024%2Fscreenshot-2024-03-26-at-3.57.39pm.png\" width=\"600\" height=\"200\" alt=\"\" loading=\"lazy\">\n","summary":"","url":"https://deptofthomas.micro.blog/2024/03/26/when-asked-to.html","date_published":"2024-03-26T23:06:07+00:00","author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://cdn.micro.blog/photos/96/https%3A%2F%2Favatars.micro.blog%2Favatars%2F2023%2F35%2F134888.jpg","_microblog":{"username":"deptofthomas"}},"_microblog":{"date_relative":"2024-03-26 23:06","date_timestamp":1711494367,"is_favorite":false,"is_bookmark":false,"is_deletable":false,"is_conversation":false,"is_linkpost":false,"is_mention":false,"note":"","syndication":[]}},{"id":"31084430","content_html":"This series of attempts to get Google’s Gemini model depict a horse riding an astronaut is great, as it is the article by Gary Marcus explaining why it is so hard to get an image generation model to do so. Similar to my experience trying to code with a foundation model, the last mile is really tedious and ... <a href=\"https://deptofthomas.micro.blog/2024/02/15/this-series-of.html\">deptofthomas.micro.blog</a>","summary":"","url":"https://deptofthomas.micro.blog/2024/02/15/this-series-of.html","date_published":"2024-02-15T23:15:49+00:00","author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://cdn.micro.blog/photos/96/https%3A%2F%2Favatars.micro.blog%2Favatars%2F2023%2F35%2F134888.jpg","_microblog":{"username":"deptofthomas"}},"_microblog":{"date_relative":"2024-02-15 23:15","date_timestamp":1708038949,"is_favorite":false,"is_bookmark":false,"is_deletable":false,"is_conversation":false,"is_linkpost":false,"is_mention":false,"note":"","syndication":[]}},{"id":"27511896","content_html":"<p>My colleague Mike Heavers made a nice overview of llamafile. llamafile allows you to run a #chatbot on your own computer with a single file. If you aren’t a developer, Mike’s video makes the set up v easy to understand.</p>\n<p>Watch the overview here: <a href=\"https://www.youtube.com/watch?v=GjP7y3AiFWc\">https://www.youtube.com/watch?v=GjP7y3AiFWc</a></p>\n","summary":"","url":"https://deptofthomas.micro.blog/2023/12/07/my-colleague-mike.html","date_published":"2023-12-07T20:44:33+00:00","author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://cdn.micro.blog/photos/96/https%3A%2F%2Favatars.micro.blog%2Favatars%2F2023%2F35%2F134888.jpg","_microblog":{"username":"deptofthomas"}},"_microblog":{"date_relative":"2023-12-07 20:44","date_timestamp":1701981873,"is_favorite":false,"is_bookmark":false,"is_deletable":false,"is_conversation":false,"is_linkpost":false,"is_mention":false,"note":"","syndication":["https://mozilla.social/@tlodato/111541084412201346"]}},{"id":"27218627","content_html":"My colleague Stephen Hood and MIECO fellow Justine Tunney released llamafile this week. “llamafile lets you turn large language model (LLM) weights into executables” so that you can run a model on various operating systems without installing anything. Big win for giving access to models!\n\nBlogpost explaini... <a href=\"https://deptofthomas.micro.blog/2023/12/01/my-colleague-stephen.html\">deptofthomas.micro.blog</a>","summary":"","url":"https://deptofthomas.micro.blog/2023/12/01/my-colleague-stephen.html","date_published":"2023-12-01T17:50:32+00:00","author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://cdn.micro.blog/photos/96/https%3A%2F%2Favatars.micro.blog%2Favatars%2F2023%2F35%2F134888.jpg","_microblog":{"username":"deptofthomas"}},"_microblog":{"date_relative":"2023-12-01 17:50","date_timestamp":1701453032,"is_favorite":false,"is_bookmark":false,"is_deletable":false,"is_conversation":false,"is_linkpost":false,"is_mention":false,"note":"","syndication":["https://mozilla.social/@tlodato/111506426476062440"]}},{"id":"26783700","content_html":"<p>A lot of the reporting about Sam Altman’s return to OpenAI buries the fact that both of the women on the board were replace by men. The only board member who remains from the previous batch is a man. Assuming Brockman and Sutskever are still on it, here is what the board looks like now.</p>\n<img src=\"https://cdn.micro.blog/photos/1000x/https%3A%2F%2Fdeptofthomas.micro.blog%2Fuploads%2F2023%2Fscreenshot-2023-11-22-at-9.39.00am.png\" width=\"600\" height=\"303\" alt=\"\" loading=\"lazy\">\n","summary":"","url":"https://deptofthomas.micro.blog/2023/11/22/a-lot-of.html","date_published":"2023-11-22T17:43:01+00:00","author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://cdn.micro.blog/photos/96/https%3A%2F%2Favatars.micro.blog%2Favatars%2F2023%2F35%2F134888.jpg","_microblog":{"username":"deptofthomas"}},"_microblog":{"date_relative":"2023-11-22 17:43","date_timestamp":1700674981,"is_favorite":false,"is_bookmark":false,"is_deletable":false,"is_conversation":false,"is_linkpost":false,"is_mention":false,"note":"","syndication":["https://mozilla.social/@tlodato/111455436184126517"]}},{"id":"26746555","content_html":"<p>After writing a post on <a href=\"https://deptofthomas.micro.blog/2023/10/06/show-your-work.html\">how LLMs answer math questions</a> (tl;dr: answering math questions is not the same as doing math), I found <a href=\"https://open.substack.com/pub/garymarcus/p/math-is-hard-if-you-are-an-llm-and\">this post by Gary Marcus on the topic</a>. The chart he created of expected results from a calculator cracked me up (screenshot here).</p>\n<img src=\"https://cdn.micro.blog/photos/1000x/https%3A%2F%2Fdeptofthomas.micro.blog%2Fuploads%2F2023%2Fscreenshot-2023-11-21-at-2.27.03pm.png\" width=\"600\" height=\"226\" alt=\"\" loading=\"lazy\">\n","summary":"","url":"https://deptofthomas.micro.blog/2023/11/21/after-writing-a.html","date_published":"2023-11-21T22:37:29+00:00","author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://cdn.micro.blog/photos/96/https%3A%2F%2Favatars.micro.blog%2Favatars%2F2023%2F35%2F134888.jpg","_microblog":{"username":"deptofthomas"}},"_microblog":{"date_relative":"2023-11-21 22:37","date_timestamp":1700606249,"is_favorite":false,"is_bookmark":false,"is_deletable":false,"is_conversation":false,"is_linkpost":false,"is_mention":false,"note":"","syndication":["https://mozilla.social/@tlodato/111450931677510267"]}},{"id":"26588935","content_html":"<p>From Nadia Eghbal’s “Working in Public: the making and maintenance of open source software”: of the four types of open source projects, Eghbal notes that stadiums (high user growth, low contributor growth) are “increasingly commonplace” despite how many narrativize OSS.</p>\n<img src=\"https://cdn.micro.blog/photos/1000x/https%3A%2F%2Fdeptofthomas.micro.blog%2Fuploads%2F2023%2F72cc4c4a78.jpg\" width=\"450\" height=\"600\" alt=\"\" loading=\"lazy\">\n","summary":"","url":"https://deptofthomas.micro.blog/2023/11/18/from-nadia-eghbals.html","date_published":"2023-11-18T17:17:59+00:00","author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://cdn.micro.blog/photos/96/https%3A%2F%2Favatars.micro.blog%2Favatars%2F2023%2F35%2F134888.jpg","_microblog":{"username":"deptofthomas"}},"_microblog":{"date_relative":"2023-11-18 17:17","date_timestamp":1700327879,"is_favorite":false,"is_bookmark":false,"is_deletable":false,"is_conversation":false,"is_linkpost":false,"is_mention":false,"note":"","syndication":["https://mozilla.social/@tlodato/111432688123290402"]}},{"id":"26434453","content_html":"<p>A colleague shared a project to run an LLM on iOS. The project sounds rad; the name is not (and maybe NSFW). In any case, I decided to do some work for the community with a prompt.</p>\n<img src=\"https://cdn.micro.blog/photos/1000x/https%3A%2F%2Fdeptofthomas.micro.blog%2Fuploads%2F2023%2Fscreenshot-2023-11-15-at-9.13.37am.png\" width=\"600\" height=\"355\" alt=\"\" loading=\"lazy\">\n","summary":"","url":"https://deptofthomas.micro.blog/2023/11/15/a-colleague-shared.html","date_published":"2023-11-15T17:20:15+00:00","author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://cdn.micro.blog/photos/96/https%3A%2F%2Favatars.micro.blog%2Favatars%2F2023%2F35%2F134888.jpg","_microblog":{"username":"deptofthomas"}},"_microblog":{"date_relative":"2023-11-15 17:20","date_timestamp":1700068815,"is_favorite":false,"is_bookmark":false,"is_deletable":false,"is_conversation":false,"is_linkpost":false,"is_mention":false,"note":"","syndication":["https://mozilla.social/@tlodato/111415710113476125"]}},{"id":"26347582","content_html":"<p>I decided to explore Replit’s AI coding feature and in the process made a thing:</p>\n<p><a href=\"https://replit.com/@deptofthomas/Do-I-Need-AI-For-That?v=1\">Do I need AI for that?</a> is a tool to determine if you need AI for that.</p>\n<p>(Write up of coding with AI forthcoming.)</p>\n","summary":"","url":"https://deptofthomas.micro.blog/2023/11/13/i-decided-to.html","date_published":"2023-11-13T22:52:56+00:00","author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://cdn.micro.blog/photos/96/https%3A%2F%2Favatars.micro.blog%2Favatars%2F2023%2F35%2F134888.jpg","_microblog":{"username":"deptofthomas"}},"_microblog":{"date_relative":"2023-11-13 22:52","date_timestamp":1699915976,"is_favorite":false,"is_bookmark":false,"is_deletable":false,"is_conversation":false,"is_linkpost":false,"is_mention":false,"note":"","syndication":["https://mozilla.social/@tlodato/111405693750147046"]}},{"id":"26222324","content_html":"<p>Simon Wardley (part of the <a href=\"https://future.mozilla.org/mieco/\">Mozilla Internet Ecosystem (MIECO) program</a>) shared some <a href=\"https://medium.com/@swardley/why-open-source-ai-matters-a46a7d23ad0e\">thoughts on open source AI models.</a> Building on the idea that training data are source code for foundation models, he argues that open source models exist only when those data (not just weights) are open to inspection.</p>\n","summary":"","url":"https://deptofthomas.micro.blog/2023/11/10/simon-wardley-part.html","date_published":"2023-11-11T02:52:04+00:00","author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://cdn.micro.blog/photos/96/https%3A%2F%2Favatars.micro.blog%2Favatars%2F2023%2F35%2F134888.jpg","_microblog":{"username":"deptofthomas"}},"_microblog":{"date_relative":"2023-11-11 02:52","date_timestamp":1699671124,"is_favorite":false,"is_bookmark":false,"is_deletable":false,"is_conversation":false,"is_linkpost":false,"is_mention":false,"note":"","syndication":["https://mozilla.social/@tlodato/111389646985817604"]}},{"id":"26206414","content_html":"<p>Notes on multimodal models, AI for infrastructure, fine tuning, and soft prompting: <a href=\"https://deptofthomas.micro.blog/2023/11/10/notes-on-multimodal.html\">deptofthomas.micro.blog</a></p>","summary":"","url":"https://deptofthomas.micro.blog/2023/11/10/notes-on-multimodal.html","date_published":"2023-11-10T19:06:22+00:00","author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://cdn.micro.blog/photos/96/https%3A%2F%2Favatars.micro.blog%2Favatars%2F2023%2F35%2F134888.jpg","_microblog":{"username":"deptofthomas"}},"_microblog":{"date_relative":"2023-11-10 19:06","date_timestamp":1699643182,"is_favorite":false,"is_bookmark":false,"is_deletable":false,"is_conversation":false,"is_linkpost":true,"is_mention":false,"note":"","syndication":["https://mozilla.social/@tlodato/111387815720812636"]}},{"id":"26113302","content_html":"<p>[Field Report] Silicon Valley Generative AI Paper Reading Event: RWKV <img src=\"https://micro.blog/photos/50/https://deptofthomas.micro.blog/uploads/2023/b2ef6d1ec5.jpg\" width=\"20\" height=\"20\" class=\"mini_thumbnail\" alt=\"\"> : <a href=\"https://deptofthomas.micro.blog/2023/11/08/field-report-silicon.html\">deptofthomas.micro.blog</a></p>","summary":"","url":"https://deptofthomas.micro.blog/2023/11/08/field-report-silicon.html","date_published":"2023-11-08T22:03:18+00:00","author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://cdn.micro.blog/photos/96/https%3A%2F%2Favatars.micro.blog%2Favatars%2F2023%2F35%2F134888.jpg","_microblog":{"username":"deptofthomas"}},"_microblog":{"date_relative":"2023-11-08 22:03","date_timestamp":1699480998,"is_favorite":false,"is_bookmark":false,"is_deletable":false,"is_conversation":false,"is_linkpost":true,"is_mention":false,"note":"","syndication":["https://mozilla.social/@tlodato/111377186966939461"]}},{"id":"26067215","content_html":"<p>More for less: OpenAI's larger and cheaper context window: <a href=\"https://deptofthomas.micro.blog/2023/11/07/more-for-less.html\">deptofthomas.micro.blog</a></p>","summary":"","url":"https://deptofthomas.micro.blog/2023/11/07/more-for-less.html","date_published":"2023-11-08T00:21:59+00:00","author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://cdn.micro.blog/photos/96/https%3A%2F%2Favatars.micro.blog%2Favatars%2F2023%2F35%2F134888.jpg","_microblog":{"username":"deptofthomas"}},"_microblog":{"date_relative":"2023-11-08 00:21","date_timestamp":1699402919,"is_favorite":false,"is_bookmark":false,"is_deletable":false,"is_conversation":false,"is_linkpost":true,"is_mention":false,"note":"","syndication":["https://mozilla.social/@tlodato/111372069872209194"]}},{"id":"26051422","content_html":"<p>Casey Newton’s <a href=\"https://open.substack.com/pub/platformer/p/how-openai-is-building-a-path-toward\">latest post</a> about OpenAI’s agents (“GPTs”) is fascinating esp. the way making an agent requires no coding but using a chat interface. This is how, in many cases, chatbots have been doing some tasks: creating code unbeknownst to the user. Now it’s a feature.</p>\n<img src=\"https://cdn.micro.blog/photos/1000x/https%3A%2F%2Fdeptofthomas.micro.blog%2Fuploads%2F2023%2Fimage.png\" width=\"337\" height=\"600\" alt=\"\" loading=\"lazy\">\n","summary":"","url":"https://deptofthomas.micro.blog/2023/11/07/casey-newtons-latest.html","date_published":"2023-11-07T17:27:14+00:00","author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://cdn.micro.blog/photos/96/https%3A%2F%2Favatars.micro.blog%2Favatars%2F2023%2F35%2F134888.jpg","_microblog":{"username":"deptofthomas"}},"_microblog":{"date_relative":"2023-11-07 17:27","date_timestamp":1699378034,"is_favorite":false,"is_bookmark":false,"is_deletable":false,"is_conversation":false,"is_linkpost":false,"is_mention":false,"note":"","syndication":["https://mozilla.social/@tlodato/111370439195000074"]}},{"id":"25738604","content_html":"<p>Notes on Eric Jang's \"How to Understand ML Papers Quickly\": <a href=\"https://deptofthomas.micro.blog/2023/10/31/notes-on-eric.html\">deptofthomas.micro.blog</a></p>","summary":"","url":"https://deptofthomas.micro.blog/2023/10/31/notes-on-eric.html","date_published":"2023-11-01T00:10:30+00:00","author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://cdn.micro.blog/photos/96/https%3A%2F%2Favatars.micro.blog%2Favatars%2F2023%2F35%2F134888.jpg","_microblog":{"username":"deptofthomas"}},"_microblog":{"date_relative":"2023-11-01 00:10","date_timestamp":1698797430,"is_favorite":false,"is_bookmark":false,"is_deletable":false,"is_conversation":false,"is_linkpost":true,"is_mention":false,"note":"","syndication":["https://mozilla.social/@tlodato/111332388494594771"]}},{"id":"25547632","content_html":"<p>Some reading from this week on AI: <a href=\"https://deptofthomas.micro.blog/2023/10/27/some-reading-from.html\">deptofthomas.micro.blog</a></p>","summary":"","url":"https://deptofthomas.micro.blog/2023/10/27/some-reading-from.html","date_published":"2023-10-27T23:16:45+00:00","author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://cdn.micro.blog/photos/96/https%3A%2F%2Favatars.micro.blog%2Favatars%2F2023%2F35%2F134888.jpg","_microblog":{"username":"deptofthomas"}},"_microblog":{"date_relative":"2023-10-27 23:16","date_timestamp":1698448605,"is_favorite":false,"is_bookmark":false,"is_deletable":false,"is_conversation":false,"is_linkpost":true,"is_mention":false,"note":"","syndication":["https://mozilla.social/@tlodato/111309527930165589"]}},{"id":"25491272","content_html":"<p><a href=\"https://ai-guide.future.mozilla.org/\">The Mozilla AI Guide is now live!</a></p>\n<p>The AI Guide is an MVP resource we created for new and existing developers who want to learn about generative AI, especially about local models, open-source software, and responsible use. Check it out, share your feedback, and look for more updates soon!</p>\n","summary":"","url":"https://deptofthomas.micro.blog/2023/10/26/the-mozilla-ai.html","date_published":"2023-10-26T20:41:27+00:00","author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://cdn.micro.blog/photos/96/https%3A%2F%2Favatars.micro.blog%2Favatars%2F2023%2F35%2F134888.jpg","_microblog":{"username":"deptofthomas"}},"_microblog":{"date_relative":"2023-10-26 20:41","date_timestamp":1698352887,"is_favorite":false,"is_bookmark":false,"is_deletable":false,"is_conversation":false,"is_linkpost":false,"is_mention":false,"note":"","syndication":["https://mozilla.social/@tlodato/111303258104980149"]}},{"id":"25348434","content_html":"<p>Am I doing this right? Three factors that make learning with and about LLMs difficult: <a href=\"https://deptofthomas.micro.blog/2023/10/23/am-i-doing.html\">deptofthomas.micro.blog</a></p>","summary":"","url":"https://deptofthomas.micro.blog/2023/10/23/am-i-doing.html","date_published":"2023-10-23T22:41:14+00:00","author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://cdn.micro.blog/photos/96/https%3A%2F%2Favatars.micro.blog%2Favatars%2F2023%2F35%2F134888.jpg","_microblog":{"username":"deptofthomas"}},"_microblog":{"date_relative":"2023-10-23 22:41","date_timestamp":1698100874,"is_favorite":false,"is_bookmark":false,"is_deletable":false,"is_conversation":false,"is_linkpost":true,"is_mention":false,"note":"","syndication":["https://mozilla.social/@tlodato/111286739207942468"]}},{"id":"25339313","content_html":"<p>TIL from <a href=\"https://arxiv.org/abs/2305.15425\">this paper</a> that although LLMs can translate text from one language to another, the cost of doing so varies greatly, thereby making LLMs more costly within some languages because of how words are tokenized.</p>\n<img src=\"https://cdn.micro.blog/photos/1000x/https%3A%2F%2Fdeptofthomas.micro.blog%2Fuploads%2F2023%2Fscreenshot-2023-10-23-at-11.59.35am.png\" width=\"600\" height=\"369\" alt='Screenshot of an academic paper that is titled: Language Model Tokenizers Introduce Unfairness Between Languages\". The authors are:  Aleksandar Petrov, Emanuele La Malfa, Philip H.S. Torr, Adel Bibi. The abstract reads: Recent language models have shown impressive multilingual performance, even when not explicitly trained for it. Despite this, there are concerns about the quality of their outputs across different languages. In this paper, we show how disparity in the treatment of different languages arises at the tokenization stage, well before a model is even invoked. The same text translated into different languages can have drastically different tokenization lengths, with differences up to 15 times in some cases. These disparities persist even for tokenizers that are intentionally trained for multilingual support. Character-level and byte-level models also exhibit over 4 times the difference in the encoding length for some language pairs. This induces unfair treatment for some language communities in regard to the cost of accessing commercial language services, the processing time and latency, as well as the amount of content that can be provided as context to the models. Therefore, we make the case that we should train future language models using multilingually fair subword tokenizers. ' loading=\"lazy\">\n","summary":"","url":"https://deptofthomas.micro.blog/2023/10/23/til-from-this.html","date_published":"2023-10-23T19:04:03+00:00","author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://cdn.micro.blog/photos/96/https%3A%2F%2Favatars.micro.blog%2Favatars%2F2023%2F35%2F134888.jpg","_microblog":{"username":"deptofthomas"}},"_microblog":{"date_relative":"2023-10-23 19:04","date_timestamp":1698087843,"is_favorite":false,"is_bookmark":false,"is_deletable":false,"is_conversation":false,"is_linkpost":false,"is_mention":false,"note":"","syndication":["https://mozilla.social/@tlodato/111285885192758897"]}},{"id":"25017324","content_html":"<p>I love getting sassed by ChatGPT (GPT-3.5) to just Google something with the attitude of “you know the tech under here is pretty advanced, right?” Also, it doesn’t mean use something “like Google”–it means Google because Bing is not good.</p>\n<img src=\"https://cdn.micro.blog/photos/1000x/https%3A%2F%2Fdeptofthomas.micro.blog%2Fuploads%2F2023%2Fscreenshot-2023-10-16-at-2.32.47pm.png\" width=\"600\" height=\"233\" alt=\"\" loading=\"lazy\">\n","summary":"","url":"https://deptofthomas.micro.blog/2023/10/16/i-love-getting.html","date_published":"2023-10-16T21:41:27+00:00","author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://cdn.micro.blog/photos/96/https%3A%2F%2Favatars.micro.blog%2Favatars%2F2023%2F35%2F134888.jpg","_microblog":{"username":"deptofthomas"}},"_microblog":{"date_relative":"2023-10-16 21:41","date_timestamp":1697492487,"is_favorite":false,"is_bookmark":false,"is_deletable":false,"is_conversation":false,"is_linkpost":false,"is_mention":false,"note":"","syndication":["https://mozilla.social/@tlodato/111246867905714241"]}},{"id":"24875074","content_html":"<p><a href=\"https://micro.blog/deptofthomas\">@deptofthomas</a> <a href=\"https://overcast.fm/+m_rrYUsu8/49:05\">here is the episode</a></p>\n","url":"https://micro.blog/deptofthomas/24875074","date_published":"2023-10-13T15:34:11+00:00","author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://cdn.micro.blog/photos/96/https%3A%2F%2Favatars.micro.blog%2Favatars%2F2023%2F35%2F134888.jpg","_microblog":{"username":"deptofthomas"}},"_microblog":{"date_relative":"2023-10-13 15:34","date_timestamp":1697211251,"is_favorite":false,"is_bookmark":false,"is_deletable":false,"is_conversation":true,"is_linkpost":false,"is_mention":true}},{"id":"24875027","content_html":"<p>Catching up in “Hard Fork”: this segment on prompt engineering with Riley Goodside is pretty fascinating esp because it sounds a lot more like animal training than pen testing</p>\n","summary":"","url":"https://deptofthomas.micro.blog/2023/10/13/catching-up-in.html","date_published":"2023-10-13T15:33:10+00:00","author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://cdn.micro.blog/photos/96/https%3A%2F%2Favatars.micro.blog%2Favatars%2F2023%2F35%2F134888.jpg","_microblog":{"username":"deptofthomas"}},"_microblog":{"date_relative":"2023-10-13 15:33","date_timestamp":1697211190,"is_favorite":false,"is_bookmark":false,"is_deletable":false,"is_conversation":true,"is_linkpost":false,"is_mention":false,"note":"","syndication":["https://mozilla.social/@tlodato/111228432772923144"]}},{"id":"24745468","content_html":"<p>I am working on a bit of writing about the challenge of learning about LLMs by using them, and here is a prime example. The image is a screencap of <a href=\"https://huggingface.co/google/flan-t5-base\">Google’s flan-t5-base model on HuggingFace</a>. The diagram gives an answer to a question that when demoed live does not produce the expected answer.</p>\n<img src=\"https://cdn.micro.blog/photos/1000x/https%3A%2F%2Fdeptofthomas.micro.blog%2Fuploads%2F2023%2Fscreenshot-2023-10-10-at-12.32.00-pm.png\" width=\"600\" height=\"202\" alt=\"\" loading=\"lazy\">\n","summary":"","url":"https://deptofthomas.micro.blog/2023/10/10/i-am-working.html","date_published":"2023-10-10T22:14:10+00:00","author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://cdn.micro.blog/photos/96/https%3A%2F%2Favatars.micro.blog%2Favatars%2F2023%2F35%2F134888.jpg","_microblog":{"username":"deptofthomas"}},"_microblog":{"date_relative":"2023-10-10 22:14","date_timestamp":1696976050,"is_favorite":false,"is_bookmark":false,"is_deletable":false,"is_conversation":false,"is_linkpost":false,"is_mention":false,"note":"","syndication":["https://mozilla.social/@tlodato/111213023644444406"]}},{"id":"24682529","content_html":"Oh, and one more:\nWhen using PAL chain the result translated the plus sign (+) into the word “plus” to explain the final answer without mentioning anything about the addition operation.\nEntering new  chain...\ndef solution():\n    'What is 2+2? Explain how you got an answer.'\n    2+2 is 4. This is because 2+... <a href=\"https://deptofthomas.micro.blog/2023/10/09/oh-and-one.html\">deptofthomas.micro.blog</a>","summary":"","url":"https://deptofthomas.micro.blog/2023/10/09/oh-and-one.html","date_published":"2023-10-09T15:53:35+00:00","author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://cdn.micro.blog/photos/96/https%3A%2F%2Favatars.micro.blog%2Favatars%2F2023%2F35%2F134888.jpg","_microblog":{"username":"deptofthomas"}},"_microblog":{"date_relative":"2023-10-09 15:53","date_timestamp":1696866815,"is_favorite":false,"is_bookmark":false,"is_deletable":false,"is_conversation":false,"is_linkpost":false,"is_mention":false,"note":"","syndication":["https://mozilla.social/@tlodato/111205863773198666"]}},{"id":"24572857","content_html":"<p>Appendix to \"Show Your Work\": <a href=\"https://deptofthomas.micro.blog/2023/10/06/appendix-to-show.html\">deptofthomas.micro.blog</a></p>","summary":"","url":"https://deptofthomas.micro.blog/2023/10/06/appendix-to-show.html","date_published":"2023-10-06T23:49:25+00:00","author":{"name":"Thomas Lodato","url":"https://deptofthomas.micro.blog/","avatar":"https://cdn.micro.blog/photos/96/https%3A%2F%2Favatars.micro.blog%2Favatars%2F2023%2F35%2F134888.jpg","_microblog":{"username":"deptofthomas"}},"_microblog":{"date_relative":"2023-10-06 23:49","date_timestamp":1696636165,"is_favorite":false,"is_bookmark":false,"is_deletable":false,"is_conversation":false,"is_linkpost":true,"is_mention":false,"note":"","syndication":["https://mozilla.social/@tlodato/111190747860584599"]}}]}