<?xml version="1.0" encoding="UTF-8"?>
<rss  xmlns:atom="http://www.w3.org/2005/Atom" 
      xmlns:media="http://search.yahoo.com/mrss/" 
      xmlns:content="http://purl.org/rss/1.0/modules/content/" 
      xmlns:dc="http://purl.org/dc/elements/1.1/" 
      version="2.0">
<channel>
<title>Lipping Off</title>
<link>https://lippingoff.com/posts/</link>
<atom:link href="https://lippingoff.com/posts/index.xml" rel="self" type="application/rss+xml"/>
<description></description>
<generator>quarto-1.10.19</generator>
<lastBuildDate>Tue, 04 Mar 2025 00:00:00 GMT</lastBuildDate>
<item>
  <title>My developer tools: 2025 edition</title>
  <link>https://lippingoff.com/posts/2025-03-04-devtools-25/</link>
  <description><![CDATA[ 




<p>I have stewed on like a dozen blog posts for quite some time, and a good chunk of them boil down to <em>write about that tool you love to use</em> or *write about that tool you <del>hate</del> <strong><em>do not love</em></strong> to use.* These ideas stayed in the pot for a long, long time (so long I forgot I had even drafted some of these posts!). Rather than serve overcooked stew, let’s start fresh with some small plates. Today I’ll just focus on the positive - I could use more of that.</p>
<section id="developer-tools-i-love-to-use" class="level1">
<h1>Developer tools I love to use</h1>
<section id="d2" class="level2">
<h2 class="anchored" data-anchor-id="d2"><a href="https://d2lang.com/"><code>d2</code></a></h2>
<p>If Marie Kondo were a programmer, <code>d2</code> would be her favorite developer tool. This program sparks so much joy.</p>
<p><code>d2</code> is a text-based diagramming program that is easier and faster to write and prettier to look at than all its competitors. Don’t believe me? The team behind <code>d2</code> also <a href="https://text-to-diagram.com/">put together a site</a> for head-to-head comparisons with alternatives.</p>
<p>I’ve used it in my <a href="https://github.com/zachlipp/object-detection-service/blob/main/README.md">READMEs</a>, I use it at work, I use it whenever I can. I cannot endorse it enough. Here’s an example: If I want to create the SIP call flow from <a href="https://www.rfc-editor.org/rfc/rfc3261.html">RFC 3261</a>, I can do it in a matter of minutes.</p>
<p>Without any custom styling, I can write a sequence diagram like this:</p>
<pre class="d2"><code>direction: right
shape: sequence_diagram

a: Alice
ap: atlanta.com proxy
bp: biloxi.com proxy
b: Bob

a -&gt; ap: INVITE F1
ap -&gt; bp: INVITE F2
ap -&gt; a: 100 Trying F3
bp -&gt; b: INVITE F4
bp -&gt; ap: 100 Trying F5
b -&gt; bp: 180 Ringing F6
bp -&gt; ap: 180 Ringing F7
ap -&gt; a: 180 Ringing F8
b -&gt; bp: 200 OK F9
bp -&gt; ap: 200 OK F10
ap -&gt; a: 200 OK F11
a -&gt; b: ACK F12
a &lt;-&gt; b: Media Session
b -&gt; a: BYE F13
a -&gt; b: 200 OK F14</code></pre>
<p>Which generates this diagram (yes, it’s a bit tall): <img src="https://lippingoff.com/posts/2025-03-04-devtools-25/sip.png" style="max-height:900px"></p>
<p>It’s pretty obvious how the code works, and even without any custom styling, the diagrams look great. Flat code like this also plays great in version control, which is a primary driver for text-based diagramming tools in the first place. Give it a spin sometime - If you don’t want to install the binary, you can always use <a href="https://play.d2lang.com/">the WASM version</a>.</p>
</section>
<section id="neovim" class="level2">
<h2 class="anchored" data-anchor-id="neovim"><a href="https://neovim.io/">Neovim</a></h2>
<p>It’s probably telling that I originally started this section with <em>“Look, I’m not trying to argue.”</em> But that’s not entirely true: I <strong>am</strong> trying to argue…with myself as of a few months ago.</p>
I’ve been a regular Neovim user for something like 6 years. During most of that time, I suggested those newer to coding choose an IDE instead. I disagree now. A properly-managed Neovim configuration is a joy to extend and tweak, and it’s <em>so much faster</em> than VSCode.<sup>1</sup>
<p></p>
<p>What do I mean by <em>properly configured</em>?</p>
<ol type="1">
<li><strong>Use Lua</strong>. You can do it, I promise. (Shoutout to <a href="https://alanza.xyz">Robbie</a> for encouraging me to do it myself!)</li>
<li><strong>Set up <a href="https://lazy.folke.io/"><code>lazy.nvim</code></a></strong>. I’ve tried a few Neovim plugin managers; this one is the best.</li>
<li><strong>Set up some <a href="https://microsoft.github.io/language-server-protocol/">language server protocol implementations</a></strong>. With LSP implementations like <a href="https://github.com/microsoft/pyright"><code>pyright</code></a>, you get most of the interactivity of an IDE right in your vim session. This takes some upfront cost to configure, but once you have them, developing is a breeze. This is so essential I think it supersedes tinkering around with any other plugins.</li>
<li><strong>Tinker around with other plugins!</strong> Come on, you knew this was coming. Right now, I mostly just use <code>conform</code> for file formatting. Unlike with my old <code>vimscript</code>-based configuration, though, it is so simple for me to try out new plugins that I can mix and match in a matter of minutes with only a few lines of code.</li>
</ol>
<p>For a simple example of how to set <code>lazy.nvim</code> and manage LSP implementations and other packages, see <a href="https://github.com/zachlipp/dotfiles/tree/main/neovim/.config/nvim">my dotfiles</a> or search around for other reference implementations.</p>
</section>
<section id="obsidian" class="level2">
<h2 class="anchored" data-anchor-id="obsidian"><a href="https://obsidian.md/">Obsidian</a></h2>
I have tried dozens of note-taking apps over the years and nothing supplanted pen-and-paper for me. Obsidian changes that. The notes are markdown, the notes are data, the notes seamlessly link to each other. I feel like the sky is the limit here and that I am barely above sea level. The plugin system is breathtaking. Do you know how hard it is to get someone who just pitched Neovim to recommend an Electron app?<sup>2</sup>
<p></p>
<p>The bar is damn high. Obsidian raises it. Oh, and <a href="https://obsidian.md/blog/free-for-work/">it’s now free for professional use, too</a>.</p>
<p>The plugin ecosystem is more important here than it is for me with Neovim. Right now, I use a few plugins together to maintain a <code>daily</code> note directory, where each day gets its own types of notes: <code>meetings</code>, <code>coding</code>, <code>messages</code>, and <code>documents</code>. I love using folders to organize these; I find it much clearer to have one note per meeting than to try to collate and classify my thoughts while I’m recording them.</p>
<p>The big idea here is to use the <code>tasks</code> plugin to surface all <code>TODO</code> items from any of these documents into one place. This creates a sort of dashboard for my thoughts and words. For now, it looks like this: <img src="https://lippingoff.com/posts/2025-03-04-devtools-25/vault.png" class="img-fluid"></p>
<p>If this sounds interesting to you, I’ve published <a href="https://github.com/zachlipp/example_vault/tree/main">an example vault</a> that you can try out for yourself (once you install the required plugins).</p>
<p>I will say I’m definitely not an expert here yet. I want to get better at linking information conceptually once I’ve actually recorded it.</p>
</section>
<section id="uv" class="level2">
<h2 class="anchored" data-anchor-id="uv"><a href="https://docs.astral.sh/uv/"><code>uv</code></a></h2>
<p><code>uv</code> makes the worst parts of Python painless. It is blazing fast, simple, and excellent. <code>uv venv</code> and <code>uv pip install</code> are two of my most-run commands. Use it, cherish it, spread the good news to your friends and family. <a href="https://www.bitecode.dev/p/a-year-of-uv-pros-cons-and-should">Here’s a great article</a> if you’re looking for a place to start.</p>
</section>
<section id="pytest" class="level2">
<h2 class="anchored" data-anchor-id="pytest"><a href="https://docs.pytest.org/en/stable/"><code>pytest</code></a></h2>
<p>I was years into writing Python before I wrote my first <code>pytest</code> test. It was years after that I learned how to use <code>pdb</code>. I did not know, until a month ago, that these two things can go great together. Run <code>pytest --pdb</code> to launch an interactive debugger within your test session. You can even set a lazy breakpoint in your code without imports with a quick <code>assert False</code>. This is incredible, and plays so well to Python’s strengths. A few <code>dir()</code> calls here and there and I can write tests an order of magnitude faster than I could before.</p>
<p>I also want to mention how many powerhouse <a href="https://docs.pytest.org/en/stable/reference/fixtures.html">fixtures</a> come baked right into the framework: <code>caplog</code> for testing logging, <code>capsys</code> for testing other side effects, and <code>tmpdir</code> for making temporary directories have become some of my most-used tools.</p>
</section>
<section id="ripgrep-and-fd" class="level2">
<h2 class="anchored" data-anchor-id="ripgrep-and-fd"><a href="https://github.com/BurntSushi/ripgrep"><code>ripgrep</code></a> and <a href="https://github.com/sharkdp/fd"><code>fd</code></a></h2>
<p>When I started programming professionally, I never imagined using the command line. I pictured a solitary, well-defined codebase that I’d primarily work with through one lanugage-specific IDE. I have seen enough now that I don’t think code like that exists.</p>
<p>Using the command line can feel intimidating. I think that’s arguably intentional: We’re reliant on dozens of tools built decades ago with minimal, if not hostile, user interfaces. There’s work being done to make this a better experience for everyone, though. Hopping between directories and getting your bearings is made so much easier with the implausibly fast <code>ripgrep</code>. <code>ripgrep</code> searches all the files under your current directory for a given string, just like <code>grep -R</code>. It’s a powerful tool for finding uses of classes, hunting down an error, and even searching your notes. Start using it and you won’t stop.</p>
<p><code>fd</code> also gets a mention here too, as basically <em>ripgrep for files</em>. It deserves bonus points because it is <em>so much easier</em> to use than vanilla <code>find</code>. I often find myself dancing between the two, spitting out quick searches so fast that I might as well be in a fine-tuned IDE.</p>
</section>
<section id="jq-yq-and-difftastic" class="level2">
<h2 class="anchored" data-anchor-id="jq-yq-and-difftastic"><a href="https://jqlang.org/"><code>jq</code></a>, <a href="https://mikefarah.gitbook.io/yq"><code>yq</code></a>, and <a href="https://github.com/Wilfred/difftastic"><code>difftastic</code></a></h2>
<p>If you find yourself working with JSON, you won’t regret learning some basic <code>jq</code>. <code>jq</code> is a query system for JSON objects, providing a <em>lot</em> of functionality for working with JSON without firing up a programming language. <code>yq</code> is, in my opinion, primarily useful for converting YAML into a reasonable format (JSON). Use <code>yq -r &lt;file.yaml&gt; -o j | jq</code> to make sense of the world’s strangest templating language.</p>
<p>I want to mention <code>difftastic</code> here, too, because I’m often using it in conjunction with <code>jq --sort-keys</code> to compare highly nested data - JSON, YAML, etc. This is a really powerful workflow that will save you from writing a lot of repetitive scripts. Also, <code>difftastic</code> makes for much better <a href="https://difftastic.wilfred.me.uk/git.html"><code>git</code> diffs</a>.</p>
</section>
<section id="liftosaur" class="level2">
<h2 class="anchored" data-anchor-id="liftosaur"><a href="https://www.liftosaur.com/"><code>Liftosaur</code></a></h2>
<p>Okay, okay, okay. I know this post is about “developer tools” I use in 2025, but I have to end with my weight training app. Liftosaur is a <strong><em>weight tracking app for coders.</em></strong> Features include:</p>
<ul>
<li>A beautiful, simple syntax for structuring workout splits and progressions</li>
<li><a href="https://github.com/astashov/liftosaur">Open source code</a></li>
<li>The option to self-host backups</li>
<li>A cute dinosaur mascot(!) <img src="https://www.liftosaur.com/images/logo.svg" class="img-fluid" width="80"></li>
</ul>
<p>Cuteness aside, you can see the website for the great UI of this app. I’m going to focus here on the ability to program workouts.</p>
<p>Here’s a subset of my current split, presented as code (do NOT pay attention to the weights please 🙈)</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb2" style="background: #f1f3f5;"><pre class="sourceCode txt code-with-copy"><code class="sourceCode default"><span id="cb2-1"># Week 1</span>
<span id="cb2-2">## Upper #1</span>
<span id="cb2-3">Seated Row, Leverage Machine / 3x8-10 / 110lb / warmup: 1x10 45lb / progress: lp(5lb)</span>
<span id="cb2-4">// Use small bar, overhead. Lean into it. Get stable before you start the lift</span>
<span id="cb2-5">Triceps Extension, Cable / 2x8-10 / 57.5lb / warmup: 1x5 20lb / progress: lp(5lb)</span>
<span id="cb2-6"></span>
<span id="cb2-7">## Leg #1</span>
<span id="cb2-8">Seated Leg Curl / 3x8-10 / 160lb / warmup: 1x10 80lb / progress: lp(5lb)</span>
<span id="cb2-9">Hip Abductor, Cable / 3x10-12 / 20lb / warmup: 1x10 10lb / progress: lp(5lb)</span>
<span id="cb2-10">Seated Leg Press / 1x4, 1x6, 1x8 / 240lb / warmup: 2x10 150lb / progress: lp(5lb)</span>
<span id="cb2-11">// 2-3s negative</span>
<span id="cb2-12">Leg Extension / 3x10-12 / 140lb / warmup: 1x10 100lb / progress: lp(5lb)</span></code></pre></div></div>
<p>Each line represents an exercise, with optional exercise text included in a comment syntax <code>//</code> above each exercise.</p>
<p>Within an exercise, you have the exercise name (including machine, if applicable), the sets and reps, the weight, and subsequent sections for the warmup sets and progression options. If you’re more ambitious, you can code your own progressions and reps from scratch:</p>
<pre><code>// Progression example
Overhead Press / 2x5 67.5lb, 1x5+ 67.5lb / warmup: 1x5 45% / progress: custom(increase: 2.5lb) {~
  if (completedReps[ns] &gt;= reps[ns]) {
    if (completedReps[ns] &gt;= 10) {
      weights += state.increase * 2
    } else {
      weights += state.increase
    }
  } else if (completedReps[ns] &lt; reps[ns]) {
    weights = weights[ns] * 0.9
  }
~}

// Reps example
Reverse Fly, Leverage Machine / 3x5 / 20lb / update: custom() {~
  if (setIndex == 1) {
       reps = 4
    } else if (setIndex == 2) {
      // 3-3+ reps
       sets(2, 2, 3, 3, 1, weights[1], 0, 0, 0)
    }
~}</code></pre>
<p>This is a hacker’s playground, but unlike many of the other tools I use, I would recommend this app to non-coders, too. There’s a fully-featured GUI for editing workouts if you don’t want to use code, and there are so many great quality of life features: support for different equipment (with different weights available), support for different gyms with different equipment, support for custom exercises, and the killer feature: <strong>Automatic plate math calculations.</strong> After a few leg sets, I’m approximating a <a href="https://www.youtube.com/watch?v=8CPijQ_LLGQ">1 Intelligence Fallout Run</a>, and therefore this feature has probably saved me hours over the last year.</p>
<p>This is yet another largely open source, largely solo developer project, so show it some love. I love this app so much I am trying to get the developer to support <a href="https://github.com/astashov/liftosaur/discussions/251">referrals</a> so he can better understand how much I (specifically) love this app.</p>
</section>
<section id="request" class="level2">
<h2 class="anchored" data-anchor-id="request">Request</h2>
<p>That’s all I’ve got for now! If you read this far and wouldn’t mind thinking of me just a little bit more, could you reach out and let me know what you thought? It took me a <em>lot</em> of tries to write this, and to be blunt, I could use some encouragement to keep trying.</p>
</section>
<section id="footnotes" class="level2">
<h2 class="anchored" data-anchor-id="footnotes">Footnotes</h2>
<p id="f1">
<sup>1</sup>I tried to come up with a quantitative answer here, but I couldn’t. <code>hyperfine</code> is a great tool to test program execution speed, but <em>my editor is open and can accept inputs</em> is not something I can easily tie to a program ending. Claude led me to look into <code>osascript</code>, which is close, but VSCode boots the Electron app almost a second before it actually accepts inputs. If anyone wants to look at this, let me know!
</p>
<p id="f2">
<sup>2</sup>I am aware of <a href="https://github.com/epwalsh/obsidian.nvim"><code>obsidian.nvim</code></a>. I don’t get it.
</p>


</section>
</section>

 ]]></description>
  <guid>https://lippingoff.com/posts/2025-03-04-devtools-25/</guid>
  <pubDate>Tue, 04 Mar 2025 00:00:00 GMT</pubDate>
</item>
<item>
  <title>Comment-delimited Jupyter cells in neovim</title>
  <link>https://lippingoff.com/posts/2024-10-04-molten/</link>
  <description><![CDATA[ 




<section id="introduction" class="level2">
<h2 class="anchored" data-anchor-id="introduction">Introduction</h2>
<p>I spent a few minutes last night trying to run a Jupyter notebook in a free-tier Google Colab account. It was a rude reminder of why I paid for a better tier in the past. As these things tend to go, I decided I’d rather spend an order of magnitude more time coming up with a local alternative than resubscribe.</p>
<p>I have appreciated VSCode’s ability to execute Python files as if they are Jupyter notebooks - that is, based on a comment delimiter, executing virtual Jupyter cells within a valid Python script. For example:</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb1" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb1-1"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># I am a valid Python script!</span></span>
<span id="cb1-2"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># ---</span></span>
<span id="cb1-3"><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">print</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"I am a Jupyter cell! I can execute in whatever order I want"</span>)</span>
<span id="cb1-4"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># ---</span></span>
<span id="cb1-5"><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">print</span>(<span class="ss" style="color: #20794D;
background-color: null;
font-style: inherit;">f"I am another Jupyter cell! This is </span><span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span>mood<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span><span class="ss" style="color: #20794D;
background-color: null;
font-style: inherit;">"</span>)</span>
<span id="cb1-6"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># ---</span></span>
<span id="cb1-7">mood <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"great"</span></span>
<span id="cb1-8"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># ---</span></span></code></pre></div></div>
<p>This feels to me like the best of both worlds: The adaptability of notebooks without sacrificing simple version control. Because I’m <del>a stubborn old dog that doesn’t want to learn new tricks</del> <strong><em>privacy-conscious</em></strong>, I wanted a tool like this within neovim. I got the <a href="https://github.com/benlubas/molten-nvim"><code>molten.nvim</code></a> package to support this sort of flow after some tinkering. Here’s a demo (<a href="https://asciinema.org/a/XQY1dNkzL9RyUWP5mf5uy4vgf">view a larger version with syntax highlighting here</a>):</p>
<p><img src="https://lippingoff.com/posts/2024-10-04-molten/demo.gif" style="max-width: 100%; max-height: 100%"></p>
</section>
<section id="usage" class="level2">
<h2 class="anchored" data-anchor-id="usage">Usage</h2>
<p>With <code>\mi</code>, I start a Jupyter kernel, and with <code>\m</code> I run a cell. Moving the cursor out of the cell hides the displayed output, and moving the cursor back into a cell displays it again. If you’d like to try it for yourself, I’ve made a minimal implementation <a href="https://gist.github.com/zachlipp/da4a94237c3b61622510026b9bb0ba72">here</a> and provided some instructions for setting up the environment.</p>
</section>
<section id="how-it-works" class="level2">
<h2 class="anchored" data-anchor-id="how-it-works">How it works</h2>
<p>The key functionality for this already exists in <code>molten.nvim</code> in the command <code>:MoltenEvaluateRange</code>. The rest of this work comes in two parts: Verifying <code>molten.nvim</code> can work and then calculating the values for <code>:MoltenEvaluateRange</code>.</p>
<section id="what-do-you-mean-can-work" class="level3">
<h3 class="anchored" data-anchor-id="what-do-you-mean-can-work">What do you mean <em>can</em> work?</h3>
<p>Before we can run Jupyter, we need a virtual environment. Operating systems are rolling out <a href="https://peps.python.org/pep-0668/">PEP 668</a> support by default, which prohibits users from installing Python packages at the system level. (Some containers are even starting to enforce this, see <a href="https://github.com/joachimbbp/openvdb_docker/pull/1">here</a> for more details). There are a lot of ways to create virtual environments - I’m using <code>uv venv</code> for now, but this approach is adaptable to whatever method you prefer. Once you have a virtual environment set up, you’ll also need to install <code>molten.nvim</code>’s Python dependencies: <code>pynvim</code>, <code>jupyter_client</code>, and (to make new kernels) <code>ipykernel</code>.</p>
<p>Second, we need a Jupyter kernel…sort of. Because we install these three packages, we get to do something cool. Usually <code>molten.nvim</code> prompts users to select a Jupyter kernel by hand. <strong><em>Boring!</em></strong> Instead, we can create and use the kernel on the fly based on the user’s virtual environment. Refreshingly true to its name, the kernel is indeed quite small. Each kernel is just a small text file pointing to other programs. We can view the location of our kernels with <code>jupter kernelspec list</code>, and, without loss of generality, see how one is constructed:</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb2" data-hl_lines="[&quot;3&quot;],linenostart=1,linenos=inline" style="background: #f1f3f5;"><pre class="sourceCode json code-with-copy"><code class="sourceCode json"><span id="cb2-1"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb2-2"> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"argv"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">[</span></span>
<span id="cb2-3">  <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"/Users/zlipp/git/molten-demo/.venv/bin/python3"</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-4">  <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"-Xfrozen_modules=off"</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-5">  <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"-m"</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-6">  <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"ipykernel_launcher"</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-7">  <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"-f"</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-8">  <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"{connection_file}"</span></span>
<span id="cb2-9"> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">]</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-10"> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"display_name"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"molten-demo"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-11"> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"language"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"python"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb2-12"> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"metadata"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb2-13">  <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">"debugger"</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">:</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">true</span></span>
<span id="cb2-14"> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb2-15"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">}</span></span></code></pre></div></div>
<p>Notice that third line: The kernel is running code from the virtual environment we created for this project. That’s all there is to it! Given a virtual environment with the minimal dependencies installed, we can start executing code in asynchronous chunks, still get all the benefits of version control, and save a few bucks along the way.</p>
</section>
<section id="about-the-lua" class="level3">
<h3 class="anchored" data-anchor-id="about-the-lua">About the Lua</h3>
<p>As I mentioned, I wrote <a href="https://gist.github.com/zachlipp/da4a94237c3b61622510026b9bb0ba72#file-init-lua">some Lua</a> to figure out which lines to send to <code>:MoltenEvaluateRange</code>. I’m pretty proud of the quick-failing error handling and how the cells break on comments; that is, the comment delimiters aren’t considered to be parts of cells themselves.</p>
<p>One more thing I’ll note here: I started writing this script using ChatGPT, winding up with something close to useful after a few iterations. That said, you are not seeing that code now. Without exaggeration, I have rewritten <strong>every</strong> line it generated. This is the first time I have ever written Lua.</p>
<p>I’m skeptical of the claim that coding alongside LLMs is especially transformative; this feels exactly like copying and pasting from StackOverflow.</p>
</section>
</section>
<section id="next-steps" class="level2">
<h2 class="anchored" data-anchor-id="next-steps">Next steps</h2>
<p>In the medium term, I’d like neovim to fully set up environments in new directories; but I hit enough bumps trying to code it now that I’m going to wait on it. Eventually I’d like to support containerized and remote kernels, too. That said, this is my first ever Lua script! I’m impressed. Now that I’ve written it and plenty about it, I can get back to writing PyTorch…the Python framework deep learning engineers wrote to avoid writing Lua. Oy.</p>


</section>

 ]]></description>
  <guid>https://lippingoff.com/posts/2024-10-04-molten/</guid>
  <pubDate>Fri, 04 Oct 2024 00:00:00 GMT</pubDate>
</item>
<item>
  <title>A minimal nonsense guide to using highlight.js with Anki</title>
  <link>https://lippingoff.com/posts/2024-08-20-anki-syntax-highlighting/</link>
  <description><![CDATA[ 




<p>I’ve been using the flashcard program Anki religiously to study programming concepts. It’s been even less exciting than it sounds because I haven’t had syntax highlighting; all my code has looked like this:</p>
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><img src="https://lippingoff.com/posts/2024-08-20-anki-syntax-highlighting/leetcode_hard.png" class="img-fluid figure-img"></p>
<figcaption>leetcode hard png</figcaption>
</figure>
</div>
<p>After a few fits and starts, I figured out a way to highlight code blocks that doesn’t rely on Anki’s add-on system, which isn’t supported by its mobile apps. This post is more synthesis than genesis - I found a lot of useful information online, but nothing that put everything together. Here’s my attempt to do so.</p>
<section id="the-easiest-way-to-get-started" class="level2">
<h2 class="anchored" data-anchor-id="the-easiest-way-to-get-started">The easiest way to get started</h2>
<p>If you are just looking to get <code>highlight.js</code> working with an Anki collection; I have a treat for you. <a href="highlight-js-bundle.colpkg">Here’s</a> a massive 151KB Anki export (half the size of one of the images in this post) containing a single card…and all the files you need to highlight your code. To change your theme, open the card template editor (for example, from studying a card -&gt; <code>Edit</code> -&gt; <code>Cards...</code>, click <code>Back Template</code>, and modify <code>&lt;link rel="stylesheet" href="_nord.css"&gt;</code> to refer to the name of another highlight.js stylesheet (see <a href="https://highlightjs.org/demo">the demo</a> for inspiration).</p>
<p>If you want to learn more about how this works, read on.</p>
</section>
<section id="configuring-anki-to-use-highlight.js-the-hard-way" class="level2">
<h2 class="anchored" data-anchor-id="configuring-anki-to-use-highlight.js-the-hard-way">Configuring Anki to use <code>highlight.js</code>: The hard way</h2>
<ul>
<li>Download <a href="https://highlightjs.org/">highlight.js</a>. You’ll likely only need <code>highlight.min.js</code> and the themes you want to use (as a tall, pale dude from the Midwest, I’m all-in on <code>nord.css</code>). See the documentation to verify your target language is supported.</li>
<li>You’ll need to add <code>highlight.min.js</code> and your css file to your Anki installation’s <code>collection.media</code> directory. On Mac, this is located at <code>~/Library/Application Support/Anki2/User 1/collection.media</code>. <code>User 1</code> here is my Anki username and the default username in Anki; this will change if you have updated your username in Anki. By default, Anki will consider these files unused media if a user uses <code>Tools &gt; Check Media</code>, and it will therefore suggest we remove them. We can prevent this by prefixing the files with an underscore. I suggest renaming them to <code>_highlight.min.js</code> and <code>_nord.css</code> (or the like).</li>
<li>Create a new file in the <code>collection.media</code> named something like <code>_my_highlight.js</code>. This file should include the following:</li>
</ul>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb1" style="background: #f1f3f5;"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb1-1"><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">document</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">querySelectorAll</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'pre code'</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">forEach</span>((block) <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">=&gt;</span> {</span>
<span id="cb1-2">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// Replace HTML &lt;br&gt; with newline characters</span></span>
<span id="cb1-3">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// https://stackoverflow.com/a/37815205/4738478</span></span>
<span id="cb1-4">    block<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">innerHTML</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> block<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span><span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">innerHTML</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">replace</span>(<span class="ss" style="color: #20794D;
background-color: null;
font-style: inherit;">/&lt;br</span><span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">\s*\/?</span><span class="ss" style="color: #20794D;
background-color: null;
font-style: inherit;">&gt;/gi</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'</span><span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">\n</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb1-5">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// Use highlight.js</span></span>
<span id="cb1-6">    hljs<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">highlightElement</span>(block)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb1-7">})<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span></code></pre></div></div>
<ul>
<li>Open the Anki card template where you want to enable syntax highlighting (I get there with <code>Browse -&gt; Cards</code>. Edit the card to include the following:</li>
</ul>
<pre><code>&lt;link rel="stylesheet" href="_nord.css"&gt;
&lt;script&gt;
    if (typeof hljs === "undefined") {
        var script = document.createElement('script');
        script.src = "_highlight.min.js";
        script.async = false;
        document.head.appendChild(script);
    }

    var script = document.createElement('script');
    script.src = '_my_highlight.js';
    script.async = false;
    document.head.appendChild(script);
    document.head.removeChild(script);
&lt;/script&gt;

&lt;pre&gt;
&lt;code&gt;
{{MyCodeField}}
&lt;/code&gt;
&lt;/pre&gt;</code></pre>
<p>Where <code>{MyCodeField}</code> is the name of the code field on your card. (Mine is <code>{Solution}</code>). Why does this work? I’m still figuring that out.</p>
<p>You should now see your rendered code in your card preview.</p>
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><img src="https://lippingoff.com/posts/2024-08-20-anki-syntax-highlighting/leetcode_medium.png" class="img-fluid figure-img"></p>
<figcaption>leetcode_medium dot png</figcaption>
</figure>
</div>
<ul>
<li>Close your Anki app to sync your changes.</li>
</ul>
<p>Congrats, that’s it! This gives you: - Syntax highlighting in whatever theme you want - Automatic language discovery through <code>highlight.js</code>. (Languages can be set manually with <code>class="code-python"&gt;</code> if you prefer) - Mac, web, and iOS support, at least (if any Android Anki-ers (Ankis? Ank-ers?) read this, I’d love to see if it works for you, too) - Offline availability. You have to love some local JavaScript and CSS. - Your data preserved. You don’t have to modify your card data to use this highlighting (unlike some add-ons). Remove this code, and your card is just as it was before.</p>
</section>
<section id="useful-links" class="level2">
<h2 class="anchored" data-anchor-id="useful-links">Useful links</h2>
<ul>
<li><a href="https://www.reddit.com/r/Anki/comments/bk82ov/comment/emf8q6c/">Loading highlight JS with Anki</a> Note: This comment is almost everything I needed, but I had to replace the <code>&lt;br&gt;</code> tags in my code with newlines and you may have to, too.</li>
<li><a href="https://forums.ankiweb.net/t/how-to-include-external-files-in-your-template-js-css-etc-guide">Loading external JS with Anki</a></li>
<li><a href="https://ankiweb.net/shared/info/1415523481">Anki add-ons not supported on mobile</a></li>
<li><a href="https://addon-docs.ankiweb.net/console-output.html#console-output">Running the Anki QT app with logs</a></li>
<li><a href="https://forums.ankiweb.net/t/this-piece-of-javascript-does-not-seem-to-work-hint-button/9627/4">Anki + the DOM</a></li>
</ul>


</section>

 ]]></description>
  <guid>https://lippingoff.com/posts/2024-08-20-anki-syntax-highlighting/</guid>
  <pubDate>Tue, 20 Aug 2024 00:00:00 GMT</pubDate>
</item>
<item>
  <title>Visualizing language model sampling</title>
  <link>https://lippingoff.com/posts/2024-05-31-sampling/</link>
  <description><![CDATA[ 




<p>I have been reading, implementing, and playing with <a href="https://arxiv.org/abs/1904.09751">The Curious Case of Neural Text Degeneration</a>, a wonderful paper on sampling from language models. I want to give some background on why we sample from language models, how we can sample from them, and why nucleus sampling (or top-p sampling), introduced in this paper, works so well.</p>
<section id="how-do-language-models-work" class="level2">
<h2 class="anchored" data-anchor-id="how-do-language-models-work">How do language models work?</h2>
<p>Consider a language model as a black box, much the way we might use the ChatGPT website. We send in words; we get out words.</p>
<p><img src="https://lippingoff.com/posts/2024-05-31-sampling/llm_app.png" class="img-fluid"></p>
<p>We translate our question by <strong>encoding</strong> it, or converting it to a sequence of numbers. The model outputs more numbers, which are <strong>decoded</strong> back to tokens.</p>
<details>
<summary>
ℹ️ A quick aside on terms
</summary>
Encoding and decoding are overloaded terms. While they do refer to translating language (or other data) to and from model space, they also refer to translating language (like these words!) to bytes the computer can understand and back. To make matters worse, models based on the Transformer architecture - like GPT2, analyzed in this paper, and ChatGPT - contain large blocks called <b>encoders</b> and <b>decoders</b>. Yuck.
</details>
<p><img src="https://lippingoff.com/posts/2024-05-31-sampling/encode_decode.png" class="img-fluid"></p>
<p>Sampling is concerned with the decoding part of this diagram. Models generally work in the following manner: Given some inputs, do a bunch of math, get some arbitrary scores as outputs.</p>
<p><img src="https://lippingoff.com/posts/2024-05-31-sampling/logits.png" class="img-fluid"></p>
<p>On their own, these scores are meaningless. We often normalize them using a function called <strong>softmax</strong> which let’s us interpret them as probabilities.</p>
<details>
<summary>
ℹ️ What does <i>interpret them as probabilities</i> mean?
</summary>
The softmax function sets all of our values to be between 0 and 1, with the constraint that all of these numbers sum to 1. This is also true of probabilities. That said, there aren’t constraints to calibrate these probabilities: A token with a softmax output of <code>0.66</code> may not actually be correct 66% of the time. I’m most comfortable calling these <b>probabilities</b>, <b>psuedo-probabilities</b>, though some people use <b>confidence scores</b>. I use <b>probabilities</b> throughout this piece as the visualization app’s sampling is properly calibrated.
</details>
<p><img src="https://lippingoff.com/posts/2024-05-31-sampling/softmax.png" class="img-fluid"></p>
</section>
<section id="choosing-a-prediction" class="level2">
<h2 class="anchored" data-anchor-id="choosing-a-prediction">Choosing a prediction</h2>
<p>Let’s step away from the ChatGPT example for a second. Consider a classification problem: An app that identifies what’s in a photo. How do we go from a model generating probabilities to a prediction? That’s simple enough: Given an array of probabilities, pick the biggest one.</p>
<p><img src="https://lippingoff.com/posts/2024-05-31-sampling/vision.png" class="img-fluid"></p>
<p>It’s tempting to think this approach will work for our language model, too. We have one big difference to worry about, though: Language models consume their own outputs. In the case of GPT2 (analyzed in this paper), we can output any one of 50,257 <strong>tokens</strong> (which are sometimes entire words, sometimes parts of words, and sometimes characters like <code>.</code>.</p>
<p>To generate an answer like <code>print("Hello world!")</code> in our example above, the model does soemthing this: - <code>How do I write hello world in Python?</code> -&gt; <code>\r</code> (new paragraph) - <code>How do I write hello world in Python?\r</code> -&gt; <code>print</code> - <code>How do I write hello world in Python?\rprint</code> -&gt; <code>(</code> - <code>How do I write hello world in Python?\rprint</code> -&gt; <code>("Hello</code> - <code>How do I write hello world in Python?\rprint("Hello</code> -&gt; <code>,</code></p>
<p>…And so on.</p>
<p>Each output we select when decoding becomes the input to find the next entry in the sequence. We call these types of models <strong>autoregressive</strong>.</p>
</section>
<section id="motivating-sampling" class="level2">
<h2 class="anchored" data-anchor-id="motivating-sampling">Motivating sampling</h2>
<p>What happens if a language model maximize scores like a classification model? The model will often wind up in a loop, where the most probable predictions lead us back to a token we’ve already generated, which leads us back…and so on. From Figure 1 of the paper:</p>
<blockquote class="blockquote">
<p>The Universidad Nacional Autónoma de México (UNAM/Universidad Nacional Autónoma de México/Universidad Nacional Autónoma de México/Universidad Nacional Autónoma de México/Universidad Nacional Autónoma de…</p>
</blockquote>
<p>We want this model to generate probable, meaningful langauge. One of the fun things the paper does is start with the question: What does the distribution of words in language usually look like?</p>
<p>The authors compare actual written text with what a score-maximizing approach would predict, and the results look nothing alike as shown in Figure 2:</p>
<p><img src="https://lippingoff.com/posts/2024-05-31-sampling/beam.png" class="img-fluid"></p>
<p>Real language is pretty random! It follows that for generated text to look like real langauge, the decoder should randomly consider other options than just the most likely ones. We call this <strong>sampling</strong>.</p>
</section>
<section id="sampling-methods" class="level2">
<h2 class="anchored" data-anchor-id="sampling-methods">Sampling methods</h2>
<section id="random-sampling" class="level3">
<h3 class="anchored" data-anchor-id="random-sampling">Random sampling</h3>
<p>The simplest sampling method is sampling according to computed probabilities. For example, a model outputs some score, and the softmax function converts this score to <code>0.66</code>. We can use a function to generate this token 66% of the time. Let’s do exactly that!</p>
<p>In the application below, you can choose between two prompts: - <code>"The greatest basketball player of all time is..."</code> - <code>"The car is..."</code></p>
<p>I made up some possible next words for these prompts, and I made up some scores for those words. The first prompt has a high-variance output (a peaked softmax) and the second prompt a low-variance output (a flat softmax). While these initial scores are arbitrary, this app computes sampling functions and transformations in real time.</p>
<p>There are <strong>two ways</strong> to interact with this app: - You can change the prompt with the radio buttons - You can draw samples with the <strong><em>Generate the next token!</em></strong> button. (Note: Greedy sampling would always select the biggest number(s) here.)</p>
<div class="rotate-holder">
<div class="rotate"></div>
<img class="rotate-icon" src="https://lippingoff.com/posts/2024-05-31-sampling/rotate-smartphone.png">
<p class="instructions rotate">Please rotate your phone to view this app</p>
<iframe loading="lazy" class="p5" src="sketch/index.html?level=0">
</iframe>
<a class="rotate license" href="https://www.flaticon.com/free-icons/mobile-phone" title="mobile phone icons">Mobile phone icons created by Freepik - Flaticon</a>
</div>
<p>In practice, sampling totally randomly causes generations to diverge too much to be useful. Researchers have suggested some modifications to improve this.</p>
</section>
<section id="temperature" class="level3">
<h3 class="anchored" data-anchor-id="temperature">Temperature</h3>
<p>We may add a parameter called <strong>temperature</strong> to modify the distribution of computed probablities. Temperature (<img src="https://latex.codecogs.com/png.latex?t">) is a scaling constant that is set to a non-negative integer. Temperatures between 0 and 1 exagerate differences in the results, while temperatures larger than 1 downplay these differences.</p>
<p>Let’s add a <strong>third</strong> input to the app: a slider for temperature.</p>
<div class="rotate-holder">
<div class="rotate"></div>
<img class="rotate-icon" src="https://lippingoff.com/posts/2024-05-31-sampling/rotate-smartphone.png">
<p class="instructions rotate">Please rotate your phone to view this app</p>
<iframe loading="lazy" class="p5" src="sketch/index.html?level=1">
</iframe>
<a class="rotate license" href="https://www.flaticon.com/free-icons/mobile-phone" title="mobile phone icons">Mobile phone icons created by Freepik - Flaticon</a>
</div>
<p>Temperature can help shift softmax distributions around, but in practice we don’t know the shape of these distributions for each input or prmopt. People tend to set temperature by guessing and checking for some results.</p>
</section>
<section id="top-k-sampling" class="level3">
<h3 class="anchored" data-anchor-id="top-k-sampling">Top-K sampling</h3>
<p>What if we want to be a little more extreme than temperature, and <strong>only</strong> consider our most-likely predictions? This is called <strong>Top-K</strong> sampling, and can improve sampling.</p>
<p>Top-k sampling takes the top scores at each step, sets everything else to 0, and rescales those top k values them with another softmax transformation to generate a new set of probabilities. Put another way, we prevent the model from generating anything other than, for example, the 5 most likely tokens.</p>
<p>I’ve added a <strong>fourth</strong> input to this app: A text box for inputting a value for K. Try it out with each distribution type for 2, 3, and 4.</p>
<div class="rotate-holder">
<div class="rotate"></div>
<img class="rotate-icon" src="https://lippingoff.com/posts/2024-05-31-sampling/rotate-smartphone.png">
<p class="instructions rotate">Please rotate your phone to view this app</p>
<iframe loading="lazy" class="p5" src="sketch/index.html?level=2">
</iframe>
<a class="rotate license" href="https://www.flaticon.com/free-icons/mobile-phone" title="mobile phone icons">Mobile phone icons created by Freepik - Flaticon</a>
</div>
<p>Okay, now follow my lead here. Set temperature equal to one, K equal to 3, and swap between our two prompts.</p>
<p>While we may be happy to only consider the top 3 predictions for the peaked (basketball) prompt, this leaves out fine predictions for the flat (car) prompt. The top 3 predictions for the basketball prompt are responsible for <code>99.62%</code> of our probabilities, but the top 3 in the car prompt gives us a much smaller <code>63%</code>.</p>
<p>This is precisely the problem. We don’t actually know the shape of these distributions and a value of K that is appropriate for one distribution may be inappropriate for another.</p>
<p>What if we could infer approprate values of K from the data? This motivates nucleus sampling.</p>
</section>
<section id="top-p-nucleus-sampling" class="level3">
<h3 class="anchored" data-anchor-id="top-p-nucleus-sampling">Top-P (nucleus) sampling</h3>
<p>Let’s return to the paper directly: &gt; The key intuition of Nucleus Sampling is that the vast majority of probability mass at each time step is concentrated in the nucleus, a small subset of the vocabulary that tends to range between one and a thousand candidates. Instead of relying on a fixed top-k, or using a temperature parameter to control the shape of the distribution without sufficiently suppressing the unreliable tail, we propose sampling from the top-p portion of the probability mass, expanding and contracting the candidate pool dynamically.</p>
<p>One note here on scale: The authors are testing the model GPT2 in this paper, which has a fixed vocabulary of 50,257 tokens. Therefore these neighborhoods (or nuclei) of probable candidates are at most <code>1000/50257~1.99%</code> of the possible results.</p>
<p>I find it more helpful to think of nucleus sampling as <strong>top-p sampling</strong>. Similar to top-K, let’s set unlikely tokens to 0 before we sample; but unlike a fixed top-K, let’s consider a dynamic threshold based on the sum of computed probabilities.</p>
<p>I’ve added a fifth and final input to the app: A slider for setting <strong>top-p</strong>. I’ve tried to visualize how top-p is computed by displaying our cumulative sum of computed probabilities as well as where p lands in this space.</p>
<div class="rotate-holder">
<div class="rotate"></div>
<img class="rotate-icon" src="https://lippingoff.com/posts/2024-05-31-sampling/rotate-smartphone.png">
<p class="instructions rotate">Please rotate your phone to view this app</p>
<iframe loading="lazy" class="p5" src="sketch/index.html?level=3">
</iframe>
<a class="rotate license" href="https://www.flaticon.com/free-icons/mobile-phone" title="mobile phone icons">Mobile phone icons created by Freepik - Flaticon</a>
</div>
<p>The authors find top-p sampling to perform very well with <code>p=0.95</code>. They do not scale temperature in their nucleus sampling experiments, so this saves a ton of effort in manual tuning.</p>
</section>
</section>
<section id="closing" class="level2">
<h2 class="anchored" data-anchor-id="closing">Closing</h2>
<p>That’s what I’ve got! I enjoyed writing this a lot. Getting these examples to work in JavaScript looks nothing like the PyTorch (or other deep learning framework) code, so I feel like I learned this twice. If you’re still curious about this space, I highly recommend reading the paper yourself, I skipped a lot of the good parts.</p>


</section>

 ]]></description>
  <guid>https://lippingoff.com/posts/2024-05-31-sampling/</guid>
  <pubDate>Fri, 31 May 2024 00:00:00 GMT</pubDate>
</item>
<item>
  <title>Caching data with Docker</title>
  <link>https://lippingoff.com/posts/2021-02-20-docker-data-caching/</link>
  <description><![CDATA[ 




<p>In a welcome change of pace, I had an idea last week.</p>
<p>I am quite impressed with Garret Dash Nelson’s <a href="http://viewshed.matinic.us/2018/01/13/1139">snowpiles visualization</a>, and I wanted to reproduce it given our shocking winter. The project consists of the following:</p>
<ul>
<li>A Python script to pull snowfall maps from the NOAA</li>
<li>Some gdal functions to reproject the maps into the Albers projection and convert the snowfall to hillshades</li>
<li>Some imagemagick calls to crop the maps and convert them to gifs</li>
<li>A final imagemagick call to create an animated gif</li>
</ul>
<p>This looks like a typical data science pipeline: retrieving data, transforming it, and then doing what you actually want with it. In the interest of time and conscientiousness to the NOAA’s servers, I wanted to build the pipeline off a local sample of maps before running it on everything at once.</p>
<p>I knew I needed a local data cache and I was already using Docker. Here enters my idea: <strong>Can I use Docker layers to cache these maps?</strong></p>
<section id="hey-it-works" class="level2">
<h2 class="anchored" data-anchor-id="hey-it-works">Hey, it works!</h2>
<p>Yes, Docker’s cache works for just fine for data. Here’s the map:</p>
<p><a href="https://github.com/zachlipp/snowpiles/blame/b2bb3913ce7db0dde0ea4fd51a56a0e7bbeab524/Dockerfile">And here’s</a> the relevant part of my Dockerfile:</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb1" style="background: #f1f3f5;"><pre class="sourceCode dockerfile code-with-copy"><code class="sourceCode dockerfile"><span id="cb1-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">COPY</span> download_snow_maps.sh /home</span>
<span id="cb1-2"></span>
<span id="cb1-3"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">WORKDIR</span> home</span>
<span id="cb1-4"></span>
<span id="cb1-5"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">RUN</span> <span class="ex" style="color: null;
background-color: null;
font-style: inherit;">./download_snow_maps.sh</span></span>
<span id="cb1-6"></span>
<span id="cb1-7"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">COPY</span> snowpiles.sh .</span>
<span id="cb1-8"></span>
<span id="cb1-9"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">ENTRYPOINT</span> <span class="ex" style="color: null;
background-color: null;
font-style: inherit;">./snowpiles.sh</span></span></code></pre></div></div>
<p>The <code>COPY/RUN</code> layer pairing just for <code>download_snow_maps.sh</code> ensures Docker’s cache of the downloaded maps won’t get discarded if other files in the directory change. This let me debug and eventually finish the actual map generation code (<code>snowpiles.sh</code>) without pulling more maps. As well, the actual <a href="https://github.com/zachlipp/snowpiles/blob/b2bb3913ce7db0dde0ea4fd51a56a0e7bbeab524/download_snow_maps.sh#L7">data pulling code</a> could now be wonderfully ignorant:</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb2" style="background: #f1f3f5;"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb2-1"><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">TODAY</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">$(</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">date</span> <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">-I</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">)</span></span>
<span id="cb2-2"><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">PAST</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">$(</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">date</span> <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">-I</span> <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">-d</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">$TODAY</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;"> - 4 months"</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">)</span></span>
<span id="cb2-3"><span class="ex" style="color: null;
background-color: null;
font-style: inherit;">...</span></span>
<span id="cb2-4"><span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">until</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">[[</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">$PAST</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">==</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">$TODAY</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">]];</span> <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">do</span></span>
<span id="cb2-5"><span class="ex" style="color: null;
background-color: null;
font-style: inherit;">...</span></span></code></pre></div></div>
<p>I used a stale cache of data until I wanted to finish the map, then I just rebuilt the Docker image without cache (<code>docker build --no-cache</code>) to produce the final animation.</p>
</section>
<section id="oh-god-it-works-how" class="level2">
<h2 class="anchored" data-anchor-id="oh-god-it-works-how">Oh God, it works how?!</h2>
<p>In retrospect, maybe “wonderfully ignorant” code isn’t laudable. Problems include:</p>
<ul>
<li><p>The builds are particularly ugly. Builds shouldn’t run scripts that fetch data, especially long-running and potentially intrusive ones.</p></li>
<li><p>Caching the data pull obfuscates when I last updated the map - <em>I can’t just rely on when I last ran the container!</em> Instead, I have to use <code>docker history</code> to find the layer created by executing <code>./download_snow_maps.sh</code>. Atrocious.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb3" style="background: #f1f3f5;"><pre class="sourceCode default code-with-copy"><code class="sourceCode default"><span id="cb3-1">➜ docker history snowpiles | grep 'download_snow'</span>
<span id="cb3-2">4e6770bac83f   5 days ago     /bin/sh -c ./download_snow_maps.sh              183MB</span></code></pre></div></div></li>
<li><p>We know our data source regularly updates (daily in this case) and that every update of source data alters the produced visual. This means <strong>our result depends on our build time</strong>. I am one <code>docker system prune</code> away from a different result, and colleagues would be guaranteed one unless we synchonize when we build. I cannot repent enough.</p></li>
<li><p>Similarly, this is a foolish way to store this data. The build either use the existing data or starts over. A much more sensible approach to this problem is to only fetch and transform maps we don’t already have.</p></li>
</ul>
<p>Caching data with Docker is clearly not a long-term solution. But what about a short-term one?</p>
</section>
<section id="lesson-learned-solve-one-problem-at-a-time" class="level2">
<h2 class="anchored" data-anchor-id="lesson-learned-solve-one-problem-at-a-time">Lesson learned: Solve one problem at a time</h2>
<p>Reproducibility and efficiency are important goals for a project but <em>only if you actually finish the damn project.</em> Before I thought to use a Docker layer as a cache, I was trying to reason around caching in my initial code… while also figuring out the file names for NOAA’s maps, how to convert maps when <code>ESPG</code> codes won’t parse, and the appropriate delay between frames for the final animation. I found this exhausting. Had I not used this naive cache, I might have burned myself out writing loops in bash - a terrible way to go. It’s not that the caching code I describe above is wildly sophisticated (<a href="https://github.com/zachlipp/snowpiles/pull/1/">see for yourself</a>); it’s that it’s something else to worry about.</p>
<p>Caching steps in a data pipeline as layers in a Docker image is noxious and foul. Build times lengthen, data age hides itself in the dark corners of the Docker CLI (who even uses <code>docker history</code>?), reproducibility goes out the window, and all for <strong>a “cache” that cannot update</strong> - it exists as-is or it is rebuilt from scratch. No project should do this indefinitely, and no shared project should do it at all.</p>
<p>And yet it separated my concerns. I finished something I started (a rare treat), I reasoned about it, and I fixed the parts I didn’t like. I don’t know if that would have happened if I spent longer staring at bash conditions.</p>


</section>

 ]]></description>
  <guid>https://lippingoff.com/posts/2021-02-20-docker-data-caching/</guid>
  <pubDate>Sat, 20 Feb 2021 00:00:00 GMT</pubDate>
</item>
<item>
  <title>Cute math trick and some tips for proofs</title>
  <link>https://lippingoff.com/posts/2019-07-14-softmax-trick/</link>
  <description><![CDATA[ 




<section id="cute-math-trick-and-some-tips-for-proofs" class="level1">
<h1>Cute math trick and some tips for proofs</h1>
<p>I had the privilege of attending <a href="https://icml.cc/Conferences/2019">ICML 2019</a> with some colleagues last month, and I’ve started working through some of the papers that stood out to me. First on the docket: <a href="https://arxiv.org/abs/1905.10964"><em>Combating Label Noise in Deep Learning Using Abstention</em>.</a> Key idea: When classifying <img src="https://latex.codecogs.com/png.latex?k"> variables, create a <img src="https://latex.codecogs.com/png.latex?k+1"> category, codify the extra class as “choosing to abstain”, and then train your model to abstain explicitly rather than deriving abstention after the fact based on classification scores.</p>
<section id="the-trick" class="level2">
<h2 class="anchored" data-anchor-id="the-trick">The trick</h2>
<p>I was hitting some issues with the very first formula in the paper (I’m new to research if that wasn’t obvious), and I wanted to see how the authors coded it up.</p>
<p>The formula</p>
<div>
<img src="https://latex.codecogs.com/png.latex?%7BL%7D(x_j)%20=%20(1%20-%20p_%7Bk+1%7D)(-%5Csum%5E%7Bk%7D_%7Bi=1%7D%20t_i%20%5Clog%20%5Cfrac%7Bp_i%7D%7B1%20-%20p_%7Bk+1%7D%7D)%20+%20%5Calpha%20%5Clog%20%5Cfrac%7B1%7D%7B1%20-%20p_%7Bk+1%7D%7D">
</div>
<p>Gets <a href="https://github.com/thulas/dac-label-noise/blob/542f3cf6442e2095cf1be7215797da5c32c1728a/dac_loss.py#L122">translated to</a></p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb1" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb1-1">loss <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> (<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1.</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> p_out_abstain)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span>h_c <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> \ <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">self</span>.alpha_var<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span>torch.log(<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1.</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> p_out_abstain)</span></code></pre></div></div>
<p>This seems okay (log transform on the rightmost term but whatever), but what is <a href="https://github.com/thulas/dac-label-noise/blob/master/dac_loss.py#L76"><code>h_c</code></a>?</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb2" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb2-1">h_c <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> F.cross_entropy(input_batch[:,<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>:<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>],target_batch,<span class="bu" style="color: null;
background-color: null;
font-style: inherit;">reduce</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">False</span>)</span></code></pre></div></div>
<p>Huh? What happened to $ $? It took me a few tries to convince myself, but these are actually equivalent.</p>
<p>Even after I was convinced, I had to chew on the proof for a bit. I revisited the problem after a long weekend off, and it’s pretty slick.</p>
<div>
<p><img src="https://latex.codecogs.com/png.latex?%0A%5Cbegin%7Bequation%7D%0A%20%20%5Cbegin%7Bsplit%7D%0A%20%20%5Cfrac%7Bp_i%7D%7B1%20-%20p_%7Bk+1%7D%7D%0A%20%20%20%20%20%20%20&amp;=%20%5Cfrac%7B%5Cfrac%7Be%5Ev%7D%7B%5Csum%5E%7Bk+1%7D_%7Bi=1%7De%5Ev%7D%7D%7B1%20-%20p_%7Bk+1%7D%7D%20&amp;&amp;%20%5Ctext%7Bgiven%20ouput%20%7D%20v%20%5Ctext%7B,%20softmax%20function%7D%20%5C%5C%5C%5C%0A%20%20%20%20%20%20%20&amp;=%20%5Cfrac%7Be%5Ev%7D%7B%5Csum%5E%7Bk+1%7D_%7Bi=1%7De%5Ev%7D%20%5Ccdot%20%5Cfrac%7B1%7D%7B1%20-%20p_%7Bk+1%7D%7D%20%5C%5C%5C%5C%0A%20%20%20%20%20%20%20&amp;=%20%5Cfrac%7Be%5Ev%7D%7B%5Csum%5E%7Bk+1%7D_%7Bi=1%7De%5Ev%7D%20%5Ccdot%20%5Cfrac%7B1%7D%7B%5Csum%5E%7Bk%7D_%7Bj=1%7Dp_j%7D%20&amp;&amp;%20%5Cquad%5Ctext%7B(1)%20%20%20%7D%20%5C%5C%5C%5C%0A%20%20%20%20%20%20%20%20&amp;=%20%5Cfrac%7Be%5Ev%7D%7B%5Csum%5E%7Bk+1%7D_%7Bi=1%7De%5Ev%7D%20%5Ccdot%20%5Cfrac%7B1%7D%7B%5Csum%5E%7Bk%7D_%7Bj=1%7D(%5Cfrac%7Be%5Ev%7D%7B%5Csum%5E%7Bk+1%7D_%7Bi=1%7De%5Ev%7D)%7D%20%5C%5C%5C%5C%0A%20%20%20%20%20%20%20&amp;=%20%5Cfrac%7Be%5Ev%7D%7B%5Csum%5E%7Bk+1%7D_%7Bi=1%7De%5Ev%7D%20%5Ccdot%20%5Cfrac%7B%5Csum%5E%7Bk+1%7D_%7Bi=1%7De%5Ev%7D%7B%5Csum%5E%7Bk%7D_%7Bj=1%7De%5Ev%7D%20%5C%5C%5C%5C%0A%20%20%20%20%20%20%20&amp;=%20%5Cfrac%7Be%5Ev%7D%7B%5Csum%5E%7Bk%7D_%7Bj=1%7De%5Ev%7D%20%5C%5C%5C%5C%0A%20%20%20%20%20%20%20&amp;=%20p_j%20&amp;&amp;%20%5Ctext%7Bby%20softmax%20definition%7D%0A%20%20%5Cend%7Bsplit%7D%0A%5Cend%7Bequation%7D%0A"></p>
</div>
<p>This means that</p>
<div>
<p><img src="https://latex.codecogs.com/png.latex?%0A%5Cbegin%7Bequation%7D%0A%20%20%5Cbegin%7Bsplit%7D%0A%20%20%20%20%5Ctext%7Bh_c%7D%0A%20%20%20%20%20%20&amp;=%20-%5Csum%5E%7Bk%7D%5C_%7Bi=1%7D%20t_i%20%5Clog%20%5Cfrac%7Bp_i%7D%7B1%20-%20p_%7Bk+1%7D%7D%20%5C%5C%5C%5C%0A%20%20%20%20%20%20&amp;=%20-%5Csum%5E%7Bk%7D%5C_%7Bi=1%7D%20t_i%20%5Clog%20p_i%20%5C%5C%5C%5C%0A%20%20%20%20%20%20&amp;=%20%5Ctext%7Bcross_entropy%7D(v,%20t)%20&amp;&amp;%20%5Ctext%7Bfor%20outputs,%20targets%20%7D%20v,%5Cspace%20t%0A%20%20%5Cend%7Bsplit%7D%0A%5Cend%7Bequation%7D%0A"></p>
</div>
<p>Pretty cool stuff, but definitely deserved a comment!</p>
</section>
<section id="the-tips" class="level2">
<h2 class="anchored" data-anchor-id="the-tips">The tips</h2>
<p>As mentioned above, it’s been a few years since I tried a proof (and to be honest, I don’t think I ever successfully did this much series - apologies to my Calc II teacher). Here are some tips I have for future me the next time I try this, maybe someone else can find them useful, too.</p>
<ul>
<li><strong>Demonstrate, then prove.</strong> When I originally saw <code>h_c</code>, I assumed it was a typo or mistake in the calculation. It wasn’t until I’d shown for myself that the trick works in practice that I could approach demonstrating why it worked</li>
<li><strong>A little understanding goes a long way.</strong> I spent many hours on this little proof. If I had to give a “eureka” moment, it’d definitely be understanding what <img src="https://latex.codecogs.com/png.latex?1%20-%20p_%7Bk%20+%201%7D"> represents (See <img src="https://latex.codecogs.com/png.latex?%5Ctext%7B(1)%7D"> in the proof.). It wasn’t until I asked myself out lout <em>Well, what is this?</em> and answered with <em>The remaining probability mass of the sum of the first k probabilities</em> that I saw a path through the proof.</li>
<li><strong>Don’t stress subscripts at first, just focus on what’s a scalar and what’s a vector.</strong> This might be a little too specific, but I found labeling the dimension of each term immensely helpful. My first work through of this proof eschewed subscripts entirely Instead, I just noted when each value represented a scalar or a vector. I haven’t used many vector-valued functions in proofs, and this explicit labeling approach helped me a lot more than pining for intuition. (Note: This technique was especially helpful, I think, because <img src="https://latex.codecogs.com/png.latex?p_i"> is overloaded in the paper - see my introduction of <img src="https://latex.codecogs.com/png.latex?p_k"> in the proof).</li>
</ul>


</section>
</section>

 ]]></description>
  <guid>https://lippingoff.com/posts/2019-07-14-softmax-trick/</guid>
  <pubDate>Sun, 14 Jul 2019 00:00:00 GMT</pubDate>
</item>
</channel>
</rss>
