Index on Michał Góral Recent content in Index on Michał Góral Hugo -- pl-PL Fri, 29 Mar 2019 12:00:00 +0000 RSS and Other Site News Sat, 30 May 2020 00:00:00 +0000 <p>I think I fixed RSS feed and now it includes new <a href="">projects</a> section which I added few days ago. There are now 2 main feeds:</p> <ul> <li><a href="">main feed</a> which has all content available;</li> <li><a href="">blog feed</a> which has only blog posts, so it works like main feed had worked previously.</li> </ul> <p>It&rsquo;s also possible to subscribe to individual tags, projects etc. How to do that is described on a separate <a href="">RSS info page</a>. RSS for all projects is a bit empty though, because it doesn&rsquo;t list all leaf projects recursively. Maybe I&rsquo;ll fix that sometime later.</p> <p>I also added <a href="">archive page</a> which simply lists all blog posts grouped by year in which they were published.</p> <p>I like the results and that my website starts feeling personal again. It&rsquo;s nice to tinker a bit here and there for half an hour every evening before sleep finally falls on me. It amazes me how much Hugo, a static page generator which is a backbone for my site, can be bent to my will.</p> Niesulice Thu, 28 May 2020 00:00:00 +0000 <p>From time to time we come back to <a href="">Niesulice</a>, a small village over Niesłysz lake. I think it was circa 2005 when we discovered this place with a group of friends. Without a car and with outdated summer bus schedules we almost had to bribe bus driver to drop us as close as he could without breaking his schedule, and we still had over an hour of walk in the middle of nowhere. I love that memory.</p> <h2 id="2015">2015</h2> <figure class="center"> <a href=""> <img src="" alt="A picture of a tree growing from a trunk in the middle of a lake" /> </a> <figcaption><p>This is probably one of my favourite photos. There's nothing impossible for life which will flourish even in most dire circumstances. Here: a small tree tries to grow from a trunk in the middle of a lake.</p></figcaption> </figure> <figure class="center"> <a href=""> <img src="" alt="A picture of a broken bus stop sign" /> </a> </figure> <figure class="center"> <a href=""> <img src="" alt="A picture of sunset" /> </a> </figure> <figure class="center"> <a href=""> <img src="" alt="A picture of people playing volleyball with 2 cans of beer in the foreground" /> </a> </figure> <figure class="center"> <a href=""> <img src="" alt="A picture of sky with trees around" /> </a> </figure> <figure class="center"> <a href=""> <img src="" alt="A picture of a woman holding a paper cup of coffee" /> </a> </figure> Exotic Animals in Kaszuby Tue, 26 May 2020 00:00:00 +0000 <p>During our vacations in Kaszuby (part of Poland) in 2019, we went to <em>Park Edukacyjny Zoo Egzotyczne Kaszuby</em><sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>, which is a kind of mini-zoo, besieged by school and family trips. Photos were taken on July 23, 2019.</p> <figure class="center"> <a href=""> <img src="" alt="A picture of a spider" /> </a> </figure> <figure class="center"> <a href=""> <img src="" alt="A picture of a spider" /> </a> </figure> <figure class="center"> <a href=""> <img src="" alt="A picture of a spider" /> </a> </figure> <figure class="center"> <a href=""> <img src="" alt="A picture of lizard" /> </a> </figure> <figure class="center"> <a href=""> <img src="" alt="A picture of cameleon" /> </a> </figure> <figure class="center"> <a href=""> <img src="" alt="A picture of cameleon" /> </a> </figure> <section class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1" role="doc-endnote"> <p><a href="">OpenStreetMap</a> <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </section> Your Pain Is Bigger Than Mine Wed, 20 May 2020 00:00:00 +0000 <p>Few months ago I almost entirely stopped listening to Polish Radio Trójka. It happened when Wojciech Mann finally quit due to firing Anna Gacek. Or rather due to forcing her to resign. I adored his programs and felt that this radio won&rsquo;t be ever the same without his legendary voice.</p> <p>Unfortunately Mr Mann wasn&rsquo;t the first one to quit. In the last few years we saw many others resigning, mostly because of the political atmosphere in a country and in public media. Thankfully for Trójka, it attracted talents like no other radio station and even though some people left, many others remained. But it has changed.</p> <p>It shocked me and saddened when I heard that last Saturday the most recognised music chart in Poland, Lista Przebojów Programu Trzeciego, was censored because <a href="">the winner</a> critcizes leader of the ruling party, Jarosław Kaczyński. After this event Marek Niedźwiedzki (author and host of Lista Przebojów), Piotr Metz (Music Director) and Marcin Kydryński quit. President of Trójka asked Bartosz Gil, who had helped to count the votes, to sign a statement saying that chart was manipulated by Mr Niedźwiedzki. He refused and was immediately suspended. Today Piotr Stelmach quit. Piotr Baron refused to co-host any next Lista Przebojów, but he&rsquo;ll likely quit in June as well, after he&rsquo;s back from vacations. The list grows with every transmitted program when another reporters say their goodbyes. Now it&rsquo;s hard to say who&rsquo;s left.</p> <p>While I don&rsquo;t believe government or any high-level politician had anything to do with it, the situation shows the sad state of public media in Poland. It is owned and managed by people who fear other people sitting above them on the ladder of power, and will do anything to please them &ndash; sometimes autonomously, sometimes not. Ultimately the goal is to please the single person on high top of that ladder.</p> <p>It hurts me to think what Trójka has turned into and I&rsquo;m afraid that it won&rsquo;t recover anytime soon. Regime methods don&rsquo;t fit very well to the station with such history, which was a place of rest and a breath of fresh air for its listeners, even before 1989.</p> <p>I wish well to all the reporters who decided to quit and to those who stay despite all of things happening right now. I hope that one day we will hear you once again on better, free frequencies.</p> My First Bread Wed, 15 Apr 2020 00:00:00 +0000 <p>I baked a bread. No, food shortages caused by people overbuying and overeating during COVID-19 lockdown didn&rsquo;t force me to start making my own food, but maybe thanks to this pandemy I finally have some time to do so.</p> <p>Frankly it&rsquo;s all on my wife. Although I always wanted to try it, I was too lazy to learn about all the details. So when she brought this topic to the table I thought that maybe it&rsquo;s now or never: she can learn everything and then she can teach me. Of course it was harder than that as I refused being taught because I like discovering things myself. A paradox. It&rsquo;s not easy to bear with me.</p> <p>So we ordered yeast and two types of flour (rye and wheat). In times of pandemy it was harder than it should be, but not impossible for a determined couple. Recipe on the other hand was easy to find - there are thousands of them on the internet. Now, after 2 weeks since our first bread, we have roughly 10 of them on our account. It means that we&rsquo;re still amateurs, but very fulfilled ones. Plus every single of our breads tasted far better than the ones we bought before.</p> <p>Baking is easier than I thought, especially amateur version which uses yeast instead of sourdough (but we&rsquo;ll get there, don&rsquo;t worry). It takes about 30 minutes of making a dough, then few hours until it proves, but it depends on the kind of bread, then 30-50 minutes of baking. We even started baking 2 breads at a time because otherwise the one huge loaf was too hard to cut even with our biggest knife and it was quickly getting old and stale.</p> <p>After few initial trials we ordered 25 kg of flour. It was unexpectedly delivered in a huge white sack - looking exactly like you probably imagine huge white sack of flour. Let me tell you: 25 kg is no joke. It is big, it is heavy and it spills flour every time we try to move it. Every time I look at it I think about Settlers (the game) - these small, poor workers carrying white little bags around the village, without a single word, not even a curse. Yeah, bullshit.</p> <p>So, bread is now officially checked-off from my list. Buns are next.</p> <p> <figure class="center"> <a href=""> <img src="" /> </a> <figcaption><p>Dough.</p></figcaption> </figure> <figure class="center"> <a href=""> <img src="" /> </a> <figcaption><p>We use aluminum foil to prevent dough from sticking to the glass bowl. And to save it from the influence of 5G.</p></figcaption> </figure> <figure class="center"> <a href=""> <img src="" /> </a> <figcaption><p>Bread in the middle of baking.</p></figcaption> </figure> <figure class="center"> <a href=""> <img src="" /> </a> <figcaption><p>Final result.</p></figcaption> </figure> <figure class="center"> <a href=""> <img src="" /> </a> </figure> </p> Anker SoundCore Sat, 11 Apr 2020 00:00:00 +0000 <p>I don&rsquo;t write reviews very often and I don&rsquo;t remember the last time I wrote anything longer than a tweet-long comment buried between other similar comments. I think they were mostly negative because that&rsquo;s what people generally do: they leave negative reviews when they&rsquo;re angry on a product or a service and want to share their anger with the whole world. Not this time.</p> <figure class="center"> <a href=""> <img src="" /> </a> <figcaption><p>Front of Anker SoundCore Bluetooth Speaker</p></figcaption> </figure> <p>Sometime last year we decided, together with my wife, that we need a bluetooth speaker. We hoped to listnen to the music in the garden and in the kitchen where we have nothing else to generate background noise. We hesitated to spend a lot because we weren&rsquo;t sure if we were going to like it and use it, so we set our upper limit to 100-150 zł<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>.</p> <p>My first thought was to buy one of JBL&rsquo;s models because I had just bought their earphones and was fairly happy with them<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup>. I did some research first and to my surprise most of friendly internet people agreed that there are better options in that price range and Anker SoundCore (first version) was often on top of their lists.</p> <p>Few quick messages later between me and my wife and I placed an order for our new bluetooth speaker. It arrived later that week and immediately conquered our hearts.</p> <h2 id="it-doesnt-swim">It doesn&rsquo;t swim</h2> <p>First things first: it doesn&rsquo;t swim. Every review says that. Apparently there&rsquo;s market for this feature but I never felt the urge to swim with my speaker. Maybe if I were younger or spent more time over the water&hellip;</p> <p>However, this fact has an implication: the speaker isn&rsquo;t waterproof. Which might be a problem if weather changes suddenly in the middle of BBQ party. I&rsquo;m yet to find myself in that situation, but I guess I don&rsquo;t throw enough parties.</p> <h2 id="sound-is-amazing">Sound is amazing</h2> <p>Jokes aside, Anker SoundCore isn&rsquo;t Hi-Fi, but its capabilities are truly amazing for the price and size. Think about it: it is 165 mm wide, 45 mm tall and has two 3 W speakers. How big can they be? 1&rdquo;, maybe 2&rdquo;? I wasn&rsquo;t familiar with speakers that small so I expected something slightly better than smartphone farting and SoundCore outgrew all of my expectations.</p> <p>Sound is clear and deep but lacks some details which can be heard on higher end equipment. You probably won&rsquo;t catch the sound of fingers sliding over the strings of cellos in the back of symphony orchestra. I&rsquo;m fine with that. After all we&rsquo;re talking about portable speaker, designed to accompany people on the trips and in places where would be inconvenient to use anything else. But even with these constraints in mind, it subjectively performs better than many bigger and more expensive speakers that I had encountered.</p> <p>I didn&rsquo;t expect any bass at all, but here it is. Not super deep but just enough satisfy. I thought that middles will be flat and trebles squeaky, but they&rsquo;re clear. Volume can be quite loud at maximum, but probably not enough to summon your elderly neighbours to come and knock at your door. Overall the sound experience is very pleasant.</p> <h2 id="battery-is-incredible">Battery is incredible</h2> <p>Everyone says that, but I must repeat it: battery is everlasting. Officially Anker promises 24 hours of play time, but that&rsquo;s a lie. Battery lasts much longer.</p> <p>SoundCore reports its battery status but after constantly seeing over 90% of remaining charge I started to think that it is glitched and kept recharging it anyway every few days. However, last month I won a lottery ticket to do some house painting over the weekend. So I turned on the music, as all profesional painters do, and started to paint. After 11 or 12 hours of non-stop work I discharged SoundCore to 60%. I think I literally gasped when I saw that number.</p> <h2 id="buttons-are-awful">Buttons are awful</h2> <p>Buttons are located on top of the speaker. There are 5 of them: power on/off, play/pause, volume up and down and bluetooth pairing. They have near to no tactile feedback which Anker replaced with rather shitty, cheap rubber feel. Because of that I tend to push buttons down really hard for really long time, just to be sure.</p> <p>Bluetooth and power buttons are the worst because they must be held down to activate. For example, according to the manual, to pair with a new device you have to hold down the bluetooth button for over a second. I never held it shorter than at least 4 seconds simply because I don&rsquo;t trust it. Same with power button - it must be held down to turn the device off. At least in this case we get a clear feedback that it worked (music stops playing) so it can be released as soona as possible.</p> <h2 id="soundcore-is-being-replaced-with-inferior-brother">SoundCore is being replaced with inferior brother</h2> <p>Since we bought it, Anker released second version of SoundCore which is arguably prettier and waterproof (finally, yay!). Too bad for Anker, it also gathered much more negative reviews for its sound<sup id="fnref:3"><a href="#fn:3" class="footnote-ref" role="doc-noteref">3</a></sup>. It seems that you can&rsquo;t have everything: speakers either swim or sound good.</p> <p>If you&rsquo;re thinking about buying a small and cheap bluetooth speaker, this might be the good time, when new SoundCores are still available online. I love mine and wouldn&rsquo;t replace it with any other speaker. Even my 3.5 years old son demands it loudably when he plays music from my wife&rsquo;s iPod, and he&rsquo;s the most sincere reviewer.</p> <section class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1" role="doc-endnote"> <p>$30-$40 <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:2" role="doc-endnote"> <p>Since then I have bought Shure SE215 IEM and quickly learned how completely clueless I was about earphones quality. <a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:3" role="doc-endnote"> <p>I never touched it so I can neither confirm nor deny them. <a href="#fnref:3" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </section> Working on TWC Formatting Wed, 01 Apr 2020 00:00:00 +0000 <p>Last year I started writing <a href="">TWC</a>, a TUI wrapper for TaskWarrior, great command line todo manager. I wrote it because I couldn&rsquo;t find any similar interactive applications and because I really like Taskwarrior, but quite dislike its tabular form of tasks presentation.</p> <p>TWC groups tasks in agendas. They are basically task filters displayed together on a single page/tab. I called each of them a <em>block</em>. Each block has separate definition of how it should render tasks on the screen. This definition is called <em>formatting string</em>.</p> <h2 id="html-era">HTML Era</h2> <p>Prior to 0.9 release users defined formatting strings as a mixture of HTML-like markup and <a href="">Python format mini-language</a>. TWC allowed passing two kinds of HTML tags:</p> <ul> <li>definitions of styles (like background and foreground colors);</li> <li>pre-defined tags with special semantics of adding or changing text wrapped by them: <ul> <li><code>&lt;sr&gt;</code> to surround non-empty text;</li> <li><code>&lt;ind&gt;</code> to change non-empty text to some other text (indicator).</li> </ul> </li> </ul> <p>Below is an example of such string, prettified for the needs of this article. In real life it would be wrapped in quotes, assigned to the variable and with line endings escaped.</p> <div class="highlight"><pre class="chroma"><code class="language-html" data-lang="html"><span class="p">&lt;</span><span class="nt">comment</span><span class="p">&gt;</span> <span class="p">&lt;</span><span class="nt">sr</span> <span class="na">right</span><span class="o">=</span><span class="s">&#34; &#34;</span><span class="p">&gt;</span> <span class="p">&lt;</span><span class="nt">sr</span> <span class="na">left</span><span class="o">=</span><span class="s">&#34;[&#34;</span> <span class="na">right</span><span class="o">=</span><span class="s">&#34;]&#34;</span><span class="p">&gt;</span> <span class="p">&lt;</span><span class="nt">ind</span> <span class="na">value</span><span class="o">=</span><span class="s">&#34;A&#34;</span><span class="p">&gt;</span>{annotations}<span class="p">&lt;/</span><span class="nt">ind</span><span class="p">&gt;</span> <span class="p">&lt;</span><span class="nt">ind</span> <span class="na">value</span><span class="o">=</span><span class="s">&#34;D&#34;</span><span class="p">&gt;</span>{due}<span class="p">&lt;/</span><span class="nt">ind</span><span class="p">&gt;</span> <span class="p">&lt;</span><span class="nt">ind</span> <span class="na">value</span><span class="o">=</span><span class="s">&#34;S&#34;</span><span class="p">&gt;</span>{scheduled}<span class="p">&lt;/</span><span class="nt">ind</span><span class="p">&gt;</span> <span class="p">&lt;/</span><span class="nt">sr</span><span class="p">&gt;</span> <span class="p">&lt;/</span><span class="nt">sr</span><span class="p">&gt;</span> <span class="p">&lt;</span><span class="nt">sr</span> <span class="na">right</span><span class="o">=</span><span class="s">&#34; &#34;</span><span class="p">&gt;</span>{id}<span class="p">&lt;/</span><span class="nt">sr</span><span class="p">&gt;</span> <span class="p">&lt;/</span><span class="nt">comment</span><span class="p">&gt;</span> {description} <span class="p">&lt;</span><span class="nt">sr</span> <span class="na">left</span><span class="o">=</span><span class="s">&#34; &#34;</span><span class="p">&gt;&lt;</span><span class="nt">info</span><span class="p">&gt;</span>{tags}<span class="p">&lt;/</span><span class="nt">info</span><span class="p">&gt;&lt;/</span><span class="nt">sr</span><span class="p">&gt;</span> </code></pre></div><p>It should display something like <code>[AD] 5 Paint the room @home:errands</code>, but of course it didn&rsquo;t because HTML parser was buggy. We&rsquo;ll get back to this later.</p> <h2 id="replacing-html-with-dsl">Replacing HTML with DSL</h2> <p>TWC 0.9 introduced a huge change: instead of HTML users can pass a formatted list of items.</p> <p>Single item now looks like this: <code>name:style:fmt:</code>. with <em>style</em> and <em>fmt</em> both being optionals. Multiple items are separated by a comma or plus sign. Choice of separator is important because they have different meanings:</p> <ul> <li><code>,</code> produces a space between two items;</li> <li><code>+</code> glues items together.</li> </ul> <p>Users can pass item&rsquo;s style name after a colon and Python formatting after another one. Similar to sed, style and formatting strings must be terminated with a colon. This one is the biggest quirk which I couldn&rsquo;t get rid of.</p> <p>And here&rsquo;s an example of items definition that actually displays what we intended to with that overly verbose HTML-like thing before:</p> <div class="highlight"><pre class="chroma"><code class="language-txt" data-lang="txt">[flags:comment:%a%d%s:],id:comment:,description,tags:info: </code></pre></div><h2 id="why-not-html">Why not HTML</h2> <p>HTML formatting was single thing which I disliked the most in TWC. So I finally gave it a little thought and after few days concluded that HTML might not be the best choice for TWC formatting strings.</p> <p>User-wise it&rsquo;s hard to read and reason about because of its verbose markup. Configuration files looked horrible because of it. <code>&lt;sr&gt;</code> tag was the worst. Because it had <em>surround-if-not-empty</em> semantics, it was actually the only way to conditionally separate optionally empty items, which in turn made each each formatting string a never-ending river of <code>&lt;sr&gt;'s</code>. Because many blocks usually need slightly different formatting, this horrible pattern was repeated over and over again. Add completely non-standard HTML tags, unfamiliar to the users who now have to constantly check documentation (I, the author, had to check it from time to time) and disaster is ready.</p> <p>Code-wise situation was not much better. It required HTML parser. This alone should say everything, but let&rsquo;s elaborate on that thought, shall we?</p> <p>Parser was implemented on top of <a href="">html.parser</a> module. It wasn&rsquo;t particularily long and complex, its test suite was more than satisfactionary, to the point that I was quite confident modifying it. It covered many uttely stupid corner cases, but that&rsquo;s fine as long as they&rsquo;re tested. And yet, after 2 days of work I somehow wasn&rsquo;t able to fix one silly bug with nesting of particular tags.</p> <p>See, HTML parser had to do a little more than just parse HTML. It had to parse it in particular way, compatible with <em>formatted text</em>, which is data format understood by <a href="">Python Prompt Toolkit</a>, a library which TWC uses for drawing of the interface. Basically it&rsquo;s a list of tuples, but it must be kept in rather simple, flat form, because otherwise Prompt Toolkit fails to render it miserably.</p> <p>So what could I do with a DOM tree which is nested by definition? I defined each tag as a Python class that, when it was appropriate, received parts of <em>formatted text</em>, which could be modified, replaced and recreated. To keep things simple each such entity had only a limited knowledge of a small part of formatting string and nothing else. So it had to deduct what to do from a presence (or lack of presence) of arbitrary strings and from a number of tuples it received &mdash; all of that to flatten some HTML. No surprise that this is the part where bugs occured.</p> <h2 id="alternative">Alternative</h2> <p>So the alternative to HTML which I figured out is a custom domain-specific language. It was designed from scratch<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup> with ease of use in mind. It&rsquo;s, I think, opinionated (a popular trend that I observe recently which stands in opposition to generic, but complex programs):</p> <ul> <li>it deliberately drops some of the features of HTML like tags nesting or allowing drop-in style definitions and replaces them with simpler, more limited counterparts;</li> <li>it also creates some arbitrary conventions like removing spaces between empty items.</li> </ul> <p>But even though, it remains quite fleximple without losing readability. At least it&rsquo;s more readable than HTML counterpart, I think that we all agree on that. A little writing gives a good effect and actually invites users to experiment with it. Users shouldn&rsquo;t need the documentation to jump in.</p> <p>Code-wise it&rsquo;s implemented on top of a simple regular expression and even simpler tokenization. You know: split on commas, partition on colons&hellip; Its simplicity works not only for end-users, but also for implementation. There are not many things to break. That&rsquo;s another win.</p> <p>Is it perfect? No. It still has a field for improvements. For example I&rsquo;d like to implement a kind of &ldquo;intelligent&rdquo; mistake fixing, especially for missing terminating colon - a common mistake I make all the time. I tried to fix it before the release, but I failed, as lack of that colon introduces many interesting corner cases. Like, what about datetime string formatting which typically uses a colon to separate hours from minutes (<code>due::%Y-%m-%d %H:%M:</code>)? What about spaces? What surrounding characters?</p> <p>I hope to figure it out one day. End of story.</p> <section class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1" role="doc-endnote"> <p>But influenced by <a href="">Weechat</a>. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </section> I Blocked Bots Fri, 28 Feb 2020 00:00:00 +0000 <p>Yesterday I started blocking crawlers via robots.txt. I was forced to because it started getting out of hand. I&rsquo;ll observe traffic for a few days and will block them on level of HTTP server or fail2ban/iptables if it turns out that they don&rsquo;t respect robots.txt, contrary to the claims of the biggest offenders.</p> <p>This is mostly low-profile site. Me, my family and maybe some of my friends are doing at most 500-600 requests a day, a little more if I&rsquo;m tinkering with server.</p> <figure class="center"> <a href=""> <img src="" /> </a> <figcaption><p>14 days goaccess log</p></figcaption> </figure> <p>There are 2 main contributors to the increased traffic: <a href="">MJ12</a> and <a href="">SemRush</a>. Above is part of goaccess&rsquo; report from last 14 days. It is clear that these 2 bots alone do over 11000 requests a day, which is insane for a site known by maybe 10 people on Earth. It&rsquo;s nowhere near the capacity of the server, but it translates to 2 GB of transferred data every month. Wasted 2 GB, because those bots give me nothing. I don&rsquo;t even think I&rsquo;m serving that many pages in total, including some dynamically generated.</p> <p>Cherry on top: I&rsquo;m not the only one who hates these bots. <a href="">Wikipedia</a> hates MJ12 especially as well. And <a href="">others</a> <a href="">too</a>.</p> <h2 id="update-2020-03-03">Update: 2020-03-03</h2> <figure class="center"> <a href=""> <img src="" /> </a> </figure> <p>After few days it seems that robots.txt trick worked as things calmed down. I had to add robots.txt to every subdomain though, some disallowing all bots and some merely delaying requests (via non-standard <em>Crawl-Delay</em> directive). Still, there are more requests than I&rsquo;d expect, but I have to evaluate those before saying anything else.</p> Synchronization Sat, 15 Feb 2020 00:00:00 +0000 <p>I synchronize a lot of things. My personal knowledge base, calendar (via vdirsyncer), Qalculate exchange rates, ledger and budget repository, dotfiles, e-mail, tasks&hellip; How do I keep up with all of those things?</p> <p>As a systemd user, it&rsquo;s natural for me to set up systemd timers for some of those repetitive tasks. Unfortunately creating a timer for each small task is quite frustrating due to very explicit nature of system timers architecture: one have to create a service unit, timer unit, connect them, enable them, start them, think about their dependencies&hellip; That&rsquo;s too much work!</p> <p>So I created a single shell script (and also set up systemd timer for it) which launches all registered sync functions in parallel, with help of great <a href="">GNU Parallel</a>. It looks like this:</p> <div class="highlight"><pre class="chroma"><code class="language-bash" data-lang="bash"><span class="cp">#!/bin/bash </span><span class="cp"></span> sync_cal<span class="o">()</span> <span class="o">{</span> <span class="nb">echo</span> <span class="s2">&#34;----- [ </span><span class="si">${</span><span class="nv">FUNCNAME</span><span class="p">[0]</span><span class="si">}</span><span class="s2"> ] -----&#34;</span> <span class="nb">set</span> -e vdirsyncer sync khal2rem -o <span class="nv">$HOME</span>/.khal.rem touch ~/.reminders <span class="nb">set</span> +e <span class="o">}</span> sync_qalc<span class="o">()</span> <span class="o">{</span> <span class="nb">echo</span> <span class="s2">&#34;----- [ </span><span class="si">${</span><span class="nv">FUNCNAME</span><span class="p">[0]</span><span class="si">}</span><span class="s2"> ] -----&#34;</span> <span class="nb">set</span> -e qalc &lt; &lt;<span class="o">(</span><span class="nb">echo</span> <span class="s2">&#34;exrates&#34;</span><span class="o">)</span> <span class="nb">set</span> +e <span class="o">}</span> sync_tasks<span class="o">()</span> <span class="o">{</span> <span class="nb">echo</span> <span class="s2">&#34;----- [ </span><span class="si">${</span><span class="nv">FUNCNAME</span><span class="p">[0]</span><span class="si">}</span><span class="s2"> ] -----&#34;</span> <span class="nb">set</span> -e <span class="nb">local</span> <span class="nv">h</span><span class="o">=</span><span class="s2">&#34;</span><span class="k">$(</span>hostname<span class="k">)</span><span class="s2">&#34;</span> <span class="k">if</span> <span class="o">[[</span> <span class="s2">&#34;</span><span class="nv">$h</span><span class="s2">&#34;</span> <span class="o">==</span> <span class="s2">&#34;hostname_behind_a_proxy&#34;</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then</span> proxychains task sync <span class="k">else</span> task sync <span class="k">fi</span> <span class="nb">set</span> +e <span class="o">}</span> <span class="nv">fns</span><span class="o">=()</span> register<span class="o">()</span> <span class="o">{</span> <span class="nb">export</span> -f <span class="nv">$1</span> <span class="nv">fns</span><span class="o">+=(</span><span class="s2">&#34;</span><span class="nv">$1</span><span class="s2">&#34;</span><span class="o">)</span> <span class="o">}</span> register sync_cal register sync_qalc register sync_tasks <span class="c1"># Handle arguments passed to the script</span> <span class="k">if</span> <span class="o">[[</span> <span class="nv">$#</span> -gt <span class="m">0</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then</span> <span class="nv">fns</span><span class="o">=()</span> <span class="k">for</span> fn in <span class="s2">&#34;</span><span class="nv">$@</span><span class="s2">&#34;</span><span class="p">;</span> <span class="k">do</span> <span class="nv">fns</span><span class="o">+=(</span><span class="s2">&#34;sync_</span><span class="si">${</span><span class="nv">fn</span><span class="si">}</span><span class="s2">&#34;</span><span class="o">)</span> <span class="k">done</span> <span class="k">fi</span> parallel -j0 <span class="s1">&#39;{}&#39;</span> ::: <span class="s2">&#34;</span><span class="si">${</span><span class="nv">fns</span><span class="p">[@]</span><span class="si">}</span><span class="s2">&#34;</span> </code></pre></div><p>I called it <em>mg-sync</em> (I prefix all of my personal scripts with <em>mg-</em>, which makes them easier to find with shell tab-completion. Recently I even wrote a wrapper which instead of e.g. <em>mg-sync</em> allows me to write <em>mg sync</em>, the same way as e.g. Git works). Thanks to this script I can run in parallel all registered functions (when it&rsquo;s called with no arguments) or only selected ones (when it&rsquo;s called with arguments).</p> <p>Even though it&rsquo;s short and concise, there are few things to remember. First, each Bash function must be exported (<code>export -f</code>), otherwise it will be invisible to Parallel. Second, GNU Parallel spawns a separate job for each function (<code>-j0</code>), where full command of each job is a single argument from the argument list following triple colon. I am also explicit about command substitution (<code>{}</code>), but Parallel would work even without it: <code>parallel -j0 ::: &quot;${fns[@]}&quot;</code>.</p> <p>I like this a lot because it handles parallelism for me, waits for each ran command and correctly reports command errors. My systemd timer runs this script every few hours, but I configured a shortcut key which starts systemd service immediately if I need to quickly synchronize some data.</p> Cross References in Notes Wed, 12 Feb 2020 00:00:00 +0000 <p>My <a href="">notes</a> are continuous struggle. Because I render my notes with Hugo, which is a fabulous static site generator, I kept links in Hugo-friendly form: <code>[foo]({{&lt; ref &quot;; &gt;}})</code>. Unfortunately this is only understood by Hugo and sometimes I want to read and navigate notes from my phone. But fear no more - I fixed it!</p> <p>I create Makefiles for almost all of my projects. They&rsquo;re great: they provide easy to remember facade to complex tasks and have shell completion for their targets, so choosing what to run usually boils down to a little tabbing through the list. How does it relates to having and not having Hugo-like references? What should we do to have our cake and eat it? Add a little preprocessing in Makefile of course!</p> <p>First of all I needed to convert back all of my Hugo-like refs to ordinary markdown links. I used some magic sed for it which worked for ~95% of all original refs the rest I fixed manually. It was one time thing and I don&rsquo;t remember the exact regular expression unfortunately, but it was something like <code>sed -i -E -e 's/{{&lt; ref &quot;(; &gt;}}/\1/g'</code>. Unhandled cases? <code>relref</code> shortcode (obviously, I just HAD to use it at least once somewhere) and several refs in a single line (<code>.+</code> is a greedy match).</p> <p>Next, I wanted Hugo to understand that any links to <em>.md</em> files are in fact cross-references, which means that we must somehow replace bare file name with a shortcode, without changing &ndash; without even touching a working copy (because who likes when his text editor screams at you that file has changed when it hasn&rsquo;t).</p> <p>First I thought that I&rsquo;ll be able to use these fancy <a href="">pipelines</a> which Hugo uses to pre-process assets, but apparently they&rsquo;re special snowflakes, specialized for minifying Javascript and compiling SASS to CSS. Not good. So I decided to do the simplest thing ever: copy the whole content aside, run sed through it and use it to generate my site.</p> <div class="highlight"><pre class="chroma"><code class="language-make" data-lang="make"><span class="nv">PUBDIR</span> <span class="o">:=</span> /var/www/docs <span class="nv">CONTENT_TMP</span> <span class="o">:=</span> /tmp/docs_content_tmp <span class="nf">build</span><span class="o">:</span> @rm -rf <span class="s2">&#34;</span><span class="k">$(</span>PUBDIR<span class="k">)</span><span class="s2">&#34;</span> @mkdir -p <span class="s2">&#34;</span><span class="k">$(</span>PUBDIR<span class="k">)</span><span class="s2">&#34;</span> @rm -rf <span class="s2">&#34;</span><span class="k">$(</span>CONTENT_TMP<span class="k">)</span><span class="s2">&#34;</span> cp -r content <span class="s2">&#34;</span><span class="k">$(</span>CONTENT_TMP<span class="k">)</span><span class="s2">&#34;</span> sed -i -E<span class="se">\ </span><span class="se"></span> -e <span class="s1">&#39;s,]\(([^).]\),]({{&lt; ref &#34;\1&#34; &gt;}}),g&#39;</span> <span class="se">\ </span><span class="se"></span> -e <span class="s1">&#39;s,]\((files/[^)]+)\),]({{&lt; file &#34;\1&#34; &gt;}}),g&#39;</span> <span class="se">\ </span><span class="se"></span> <span class="sb">`</span>find <span class="s2">&#34;</span><span class="k">$(</span>CONTENT_TMP<span class="k">)</span><span class="s2">&#34;</span> -name <span class="s2">&#34;*.md&#34;</span> -type f<span class="sb">`</span> hugo -d <span class="s2">&#34;</span><span class="k">$(</span>PUBDIR<span class="k">)</span><span class="s2">&#34;</span> -c <span class="s2">&#34;</span><span class="k">$(</span>CONTENT_TMP<span class="k">)</span><span class="s2">&#34;</span> @rm -rf <span class="s2">&#34;</span><span class="k">$(</span>CONTENT_TMP<span class="k">)</span><span class="s2">&#34;</span> <span class="nf">.PHONY</span><span class="o">:</span> <span class="n">build</span> </code></pre></div><p>I hate that sed, but it does the job and probably it&rsquo;s not the worst one I&rsquo;ve ever written. And hey, it even works for several links in a single line, which wasn&rsquo;t that trivial (see that <code>[^)]</code> block? That&rsquo;s a hack for catching all unicode characters in a non-greedy way, which is generally unsupported by BRE and ERE used by GNU tools. Or it depends on locale. Or something else obscure enough that I don&rsquo;t want to think about it).</p> <p>Second <code>-e</code> script additionally detects file attachments, which are stored inside each repo in <em>files</em> directory, and wraps them with a custom <code>{{&lt; file &gt;}}</code> shortcode which handles conversions to absolute paths. So for example image <code>![foo](files/bar.jpg)</code> gets converted to <code>![foo]({{&lt; &quot;files/bar.jpg&quot; &gt;}})</code>, which in turn is generated to <code>&lt;img src=&quot;http://localhost/docs/0/files/bar.jpg&quot; /&gt;</code>.</p> <p>Shortcode implementation is one-liner:</p> <div class="highlight"><pre class="chroma"><code class="language-txt" data-lang="txt">{{-/* printf &#34;%s/%s/%s&#34; .Site.BaseURL .Page.Section (.Get 0) */-}} </code></pre></div><p>I like this approach a lot because finally content of my notes is independent from any external system/renderer. And now they work rather nicely with <a href="">Markor</a>.</p> <p>One thing which I&rsquo;ll for sure improve in future: I won&rsquo;t blindly copy the <em>whole</em> content directory, but only Markdown files. All other files (&ldquo;attachments&rdquo;) will be symlinked. It should save a little disk space and a little time spent on unnecessary copying.</p> Officially I hate my notes Mon, 03 Feb 2020 00:00:00 +0000 <p>Recently I came across an interesting way of working with personal notes called <a href="">Zettelkasten</a>. I resisted it for a while, but finally jumped the bandwagon and converted my whole knowledge database to zettels, which is a German word for small post-it notes, usually kept in a box.</p> <p>I am plaintext junkie. New fancy formats come and go and basic plaintext files have stayed with me for my whole life. When I started the process my whole knowledge database was kept in roughly 20-30 unordered pages, with strong &ldquo;unmaintained wiki&rdquo; vibes. A month later I have ~150 notes in 2 &ldquo;notebooks&rdquo;, 3 scripts which help me handling renaming of my notes and synchronizing them, <a href="">Hugo</a> theme, <a href="">ctags language definition</a>, freaking custom git-submodules replacement, a few custom Vim functions and mappings on top of that and strong WTF vibes. Everything resides comfortably (yeah, right) in 3 or 4 different git repositories, which is probably the biggest downside of my setup.</p> <p>And I hate it. And then I love it. And then I hate it again. But ultimately I love it.</p> <p>The hatred comes probably from my anxiety which I get when I start looking at it. It is complex and hacky and most probably would call it unnecessary or &ldquo;you don&rsquo;t have anything to do with your free time, don&rsquo;t you&rdquo;. But at the end of the day it just works for me. It even works on my phone terminal, which is a great achievement of modern-day technology, but not very useful as I consume my notes differently on laptop and phone. *Nively put, but as young Cerro said to King Vridank on their wedding night: &ldquo;does it have any practical uses?&quot;<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup></p> <p>Now it&rsquo;s so easy to take the notes that I actually started taking them. And thanks to Hugo, when I&rsquo;m not in a plaintext mood and want to just watch these not-so-beautiful images I embed from time to time to some of my notes, I can just generate my custom not-very-unpretty static webpage in a matter of milliseconds, copy them to <em>/var/www</em>, where I have Python&rsquo;s simple HTTP server<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup> constantly running on port 9999, type that 9999 in a web browser and bam! watch my glorious notes.</p> <h2 id="structure">Structure</h2> <p>I store notes in Git, which is the only system I wholeheartedly trust to help me recover from my stupidity. I believe that my SSD will die sooner than any of my Git repos lose integrity and thanks to its decentralized nature I usually have copies on 2, 3 or 4 different, geographically distant devices. I already deleted some of my notes by accident but thanks to Git they&rsquo;re versioned, so it was trivial to recover them.</p> <p>In the past, when I keptn notes synchronized with syncthing I sometimes run to synchronization conflicts. <em>These weren&rsquo;t that bad.</em> Bad was when empty copy of single file overrided copies with content and OF COURSE I haven&rsquo;t thought about enabling Syncthing&rsquo;s file versioning.</p> <p>My notes are stored in several notebooks and each one of those is a separate Git repo which contains notes only, optionally attachments. All notebooks are bundled inside a parent Git repo with Hugo theme and configuration files. It looks like this:</p> <div class="highlight"><pre class="chroma"><code class="language-text" data-lang="text">./ &lt;parent repo&gt; content/ &lt;main page of generated site&gt; 0/ &lt;personal git repo&gt; files/ &lt;attachments&gt; other/ &lt;work git repo&gt; files/ &lt;attachments&gt; </code></pre></div><p>Sounds like a job for <a href="">Git Submodules</a>, right? Yeah, almost. There are 3 reasons why I don&rsquo;t want submodules:</p> <ol> <li>They complicate the workflow of notes syncing because I have to remember to set-up branch-tracking submodules, update them and then update them (commit + push) in parent repo.</li> <li>I want to have <em>conditional</em> submodules. Why? Because I want to keep my work-related notes only on one laptop and specifically keep them as far as I can from my phone to avoid any IP theft accusations.</li> <li>My work forces me to connect to internet via a proxy, which only opens ports 80 and 443, so connecting to git via ssh is no-go at work but it is my preferred method on any other machine.</li> </ol> <p>So I created <em>csync</em> script which reads submodule definitions from a file named <em>modules</em>. It is a very simple text file which contains 3 columns: <em>hostname</em>, <em>local path</em> and <em>repo URL</em>. <em>csync</em> iterates this file and when hostname matches with machine&rsquo;s hostname, it synchronizes URL under the requested path. For example:</p> <div class="highlight"><pre class="chroma"><code class="language-text" data-lang="text">work content/0 https://personal.git any content/0 ssh://personal.git </code></pre></div><p>And <em>csync</em>:</p> <div class="highlight"><pre class="chroma"><code class="language-bash" data-lang="bash"><span class="cp">#!/bin/bash </span><span class="cp"></span> <span class="nv">synchronized</span><span class="o">=()</span> <span class="k">while</span> <span class="nv">IFS</span><span class="o">=</span><span class="s1">&#39; &#39;</span> <span class="nb">read</span> -r host dir url <span class="k">do</span> <span class="c1"># skip if host doesn&#39;t match; &#34;any&#34; is special host which always matches</span> <span class="k">if</span> <span class="o">[[</span> <span class="s2">&#34;</span><span class="k">$(</span>hostname<span class="k">)</span><span class="s2">&#34;</span> !<span class="o">=</span> <span class="s2">&#34;</span><span class="nv">$host</span><span class="s2">&#34;</span> <span class="o">]]</span> <span class="o">&amp;&amp;</span> <span class="o">[[</span> <span class="s2">&#34;</span><span class="nv">$host</span><span class="s2">&#34;</span> !<span class="o">=</span> <span class="s2">&#34;any&#34;</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then</span> <span class="k">continue</span> <span class="k">fi</span> <span class="c1"># skip if directory was already synchronized</span> <span class="k">if</span> <span class="o">[[</span> <span class="s2">&#34; </span><span class="si">${</span><span class="nv">synchronized</span><span class="p">[@]</span><span class="si">}</span><span class="s2"> &#34;</span> <span class="o">=</span>~ <span class="s2">&#34; </span><span class="nv">$dir</span><span class="s2"> &#34;</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then</span> <span class="k">continue</span> <span class="k">fi</span> <span class="k">if</span> <span class="o">[[</span> ! -e <span class="s2">&#34;</span><span class="nv">$dir</span><span class="s2">&#34;</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then</span> git clone <span class="s2">&#34;</span><span class="nv">$url</span><span class="s2">&#34;</span> <span class="s2">&#34;</span><span class="nv">$dir</span><span class="s2">&#34;</span> <span class="o">&amp;&amp;</span> <span class="nv">synchronized</span><span class="o">+=</span><span class="s2">&#34;</span><span class="nv">$dir</span><span class="s2">&#34;</span> <span class="k">else</span> <span class="nb">pushd</span> <span class="s2">&#34;</span><span class="nv">$dir</span><span class="s2">&#34;</span> &gt; /dev/null <span class="c1"># Better use git-sync instead; read note below!</span> git pull --rebase --autostash <span class="o">&amp;&amp;</span> <span class="nv">synchronized</span><span class="o">+=</span><span class="s2">&#34;</span><span class="nv">$dir</span><span class="s2">&#34;</span> <span class="k">if</span> <span class="o">[</span> -n <span class="s2">&#34;`git status --porcelain`&#34;</span> <span class="o">]</span><span class="p">;</span> <span class="k">then</span> git add -A <span class="o">&amp;&amp;</span> git commit -m <span class="s2">&#34;sync from `hostname`&#34;</span> <span class="o">&amp;&amp;</span> git push <span class="k">fi</span> <span class="nb">popd</span> &gt; /dev/null <span class="k">fi</span> <span class="k">done</span> &lt; <span class="s2">&#34;modules&#34;</span> </code></pre></div><p><em>2020-02-10: Instead of manual git operations as presented above, which are far from being perfect (I once ended with a conflict report instead of file), I switched to the superior <a href="">git-sync</a> script.</em></p> <h2 id="cross-references">Cross References</h2> <p><em>2020-02-12: I changed my approach for Cross References. You can read about it in my <a href="">next post</a></em>.</p> <p>One of central ideas of Zettelkasten is creating and maintaining a graph of connections between notes. One way to do it is by tagging each note. I use literal tags for this purpose: ctags, which I already described in my <a href="">other post</a>. Because 95% of them live in notes&rsquo; frontmatters, they are also understood by Hugo, which correctly renders them the to HTML.</p> <p>Notes cross referencing is equally important. Markdown allows creating hyperlinks in form of <code>[descr](</code>. It&rsquo;s super handy when I read it in Vim, because I can place cursor over <code></code> part, press <code>gf</code> and Vim opens that file for me. That&rsquo;s all blows and whistles, but URLs written this way don&rsquo;t go along with Hugo because they&rsquo;re interpreted as relative paths. That&rsquo;s why I use <a href="">shortcodes for cross references</a> provided by Hugo:</p> <div class="highlight"><pre class="chroma"><code class="language-text" data-lang="text">[descr]({{&lt; ref &#34;; &gt;}}) </code></pre></div><p>I combine them with <a href="">UltiSnips</a>, because shortcode syntax is a little too much writing for me:</p> <div class="highlight"><pre class="chroma"><code class="language-text" data-lang="text">snippet ref &#34;Hugo reference&#34; [$1]({{&lt; ref &#34;$2&#34; &gt;}})$0 endsnippet </code></pre></div><p>Additionally I created a markdown-local mapping which helps me with <code>gf</code> part:</p> <div class="highlight"><pre class="chroma"><code class="language-vim" data-lang="vim"><span class="c">&#34; after/ftplugin/markdown.vim</span><span class="err"> </span><span class="err"></span><span class="nx">nnoremap</span> <span class="p">&lt;</span><span class="nx">buffer</span><span class="p">&gt;</span> <span class="p">&lt;</span><span class="nx">cr</span><span class="p">&gt;</span> <span class="nx">f</span>&#34;<span class="nx">gf</span><span class="err"> </span></code></pre></div><h2 id="permalinks">Permalinks</h2> <p>Cross-linking cannot work without permalinks, period. Niklas Luhmann, who initially developed the system, assigned subsequent letters and numbers, indicating positions of notes in a box (like <em>21/3d7</em>, <em>21/3a1</em>). Today people usually recommend using some kind of timestamp for note IDs, which I tried and hated. After a quick brainstorm I invented my own format: <em>some-title-{8-digits-of-UUID}</em>. For example: <em></em>, <em></em> etc. Pretty unconventional, I say, but I had good reasons to do it this way:</p> <ol> <li>File names must start with human readable string (note topic). That&rsquo;s because sometimes I want to browse them from the phone (with limited search capabilities) or with a file manager (usually some kind of netrw clone to quickly go to the <em>next</em> note on the topic).</li> <li>Files must be unique. I don&rsquo;t want conflicts when I create 2 notes on a single topic on 2 different devices.</li> <li>File names must not be too long or unreadable. Timestamps look horrible, but UUIDs have a nice property of being a mix between letters and numbers which is easier for eye in my opinion.</li> </ol> <p>That&rsquo;s why I use first 8 hexadecimal digits of generated random UUID. It allows storing up to 16<sup>8</sup> notes of the same topic which is unique enough for me<sup id="fnref:3"><a href="#fn:3" class="footnote-ref" role="doc-noteref">3</a></sup>. Even if I run into a collision (I doubt it), it will be spotted by me so I can just generate next short UUID.</p> <p>Of course I don&rsquo;t create those UUIDs by hand. I wrote a vim function which does that for me. For example I can write <code>:Note foo</code> and it&rsquo;ll create a file named <em>0/aaa-{short-uuid}</em> (<em>0</em> is my default notebook, but the function detects if I specified any other notebook). When available, it uses <code>uuidgen</code> and Python <code>uuid.uuid4()</code> as a fallback.</p> <p>To ease referencing newly created notes in other notes I also created a command which yanks basename of current buffer: <em>YankNoteName</em>. The command name itself doesn&rsquo;t look very friendly, but actually is the only user command on my system which starts with <em>Y</em>, so I only have to type <code>:Y&lt;Tab&gt;&lt;CR&gt;</code> to call it.</p> <p>I admit that I&rsquo;m no vimscript pro and that I hate every moment spent writing vimscript, so the code might be below any standards. But hey, it works so let&rsquo;s call it a day!</p> <div class="highlight"><pre class="chroma"><code class="language-vim" data-lang="vim"><span class="c">&#34; 99notes.vim</span><span class="err"> </span><span class="err"></span><span class="nx">func</span><span class="p">!</span> <span class="nx">s</span>:<span class="nx">shortuuid</span><span class="p">()</span><span class="err"> </span><span class="err"></span> <span class="k">let</span> <span class="nx">l</span>:<span class="nx">uuid</span> <span class="p">=</span> <span class="s2">&#34;&#34;</span><span class="err"> </span><span class="err"></span> <span class="k">if</span> <span class="nx">executable</span><span class="p">(</span><span class="s1">&#39;uuidgen&#39;</span><span class="p">)</span><span class="err"> </span><span class="err"></span> <span class="k">let</span> <span class="nx">l</span>:<span class="nx">uuid</span><span class="p">=</span><span class="nx">system</span><span class="p">(</span><span class="s1">&#39;uuidgen&#39;</span><span class="p">)</span>[:<span class="m">-2</span>]<span class="err"> </span><span class="err"></span> <span class="k">else</span><span class="err"> </span><span class="err"></span><span class="k">python</span> <span class="o">&lt;&lt;</span> ENDPY <span class="kn">import</span> <span class="nn">vim</span> <span class="kn">from</span> <span class="nn">uuid</span> <span class="kn">import</span> <span class="n">uuid4</span> <span class="n">vim</span><span class="o">.</span><span class="n">command</span><span class="p">(</span><span class="s2">&#34;let l:uuid = &#39;</span><span class="si">%s</span><span class="s2">&#39;&#34;</span><span class="o">%</span> <span class="nb">str</span><span class="p">(</span><span class="n">uuid4</span><span class="p">()))</span> ENDPY<span class="err"> </span><span class="err"></span> <span class="k">endif</span><span class="err"> </span><span class="err"> </span><span class="err"></span> <span class="k">let</span> <span class="nx">l</span>:<span class="nx">spl</span> <span class="p">=</span> <span class="nx">split</span><span class="p">(</span><span class="nx">l</span>:<span class="nx">uuid</span><span class="p">,</span> <span class="s1">&#39;-&#39;</span><span class="p">)</span><span class="err"> </span><span class="err"></span> <span class="nx">return</span> <span class="nx">tolower</span><span class="p">(</span><span class="nx">l</span>:<span class="nx">spl</span>[<span class="m">0</span>]<span class="p">)</span><span class="err"> </span><span class="err"></span><span class="k">endfunction</span><span class="err"> </span><span class="err"> </span><span class="err"></span><span class="nx">func</span><span class="p">!</span> <span class="nx">custom</span>#<span class="m">99</span>notes#<span class="nx">edit</span><span class="p">(</span><span class="nx">name</span><span class="p">)</span><span class="err"> </span><span class="err"></span> <span class="k">let</span> <span class="nx">l</span>:<span class="nx">uuid</span> <span class="p">=</span> <span class="nx">s</span>:<span class="nx">shortuuid</span><span class="p">()</span><span class="err"> </span><span class="err"></span><span class="c"> </span><span class="c"> &#34; assign note to a default notebook</span><span class="err"> </span><span class="err"></span> <span class="k">let</span> <span class="nx">l</span>:<span class="nx">default_notebook</span> <span class="p">=</span> <span class="s2">&#34;0/&#34;</span><span class="err"> </span><span class="err"></span> <span class="k">let</span> <span class="nx">l</span>:<span class="nx">name</span> <span class="p">=</span> <span class="nx">a</span>:<span class="nx">name</span> <span class="p">=~</span> <span class="s1">&#39;.\+\/.\+&#39;</span> ? <span class="nx">a</span>:<span class="nx">name</span> : <span class="nx">l</span>:<span class="nx">default_notebook</span> . <span class="nx">a</span>:<span class="nx">name</span><span class="err"> </span><span class="err"></span> <span class="k">let</span> <span class="nx">l</span>:<span class="nx">fname</span> <span class="p">=</span> <span class="nx">expand</span><span class="p">(</span><span class="s2">&#34;~/docs/content/&#34;</span><span class="p">)</span> . <span class="nx">l</span>:<span class="nx">name</span> . <span class="s2">&#34;-&#34;</span> . <span class="nx">l</span>:<span class="nx">uuid</span> . <span class="s2">&#34;.md&#34;</span><span class="err"> </span><span class="err"></span> <span class="nx">exec</span> <span class="s2">&#34;e &#34;</span> . <span class="nx">l</span>:<span class="nx">fname</span><span class="err"> </span><span class="err"></span> <span class="nx">exec</span> <span class="s2">&#34;keepalt r &#34;</span> . <span class="nx">expand</span><span class="p">(</span><span class="s2">&#34;~/docs/archetypes/;</span><span class="p">)</span><span class="err"> </span><span class="err"></span> <span class="nx">exec</span> <span class="s2">&#34;normal ggdd&#34;</span><span class="err"> </span><span class="err"></span><span class="nx">endfunc</span><span class="err"> </span><span class="err"> </span><span class="err"></span><span class="nx">func</span><span class="p">!</span> <span class="nx">custom</span>#<span class="m">99</span>notes#<span class="nx">copynotename</span><span class="p">()</span><span class="err"> </span><span class="err"></span> <span class="k">let</span> @<span class="s2">&#34; = expand(&#34;</span>%:<span class="nx">t</span>:<span class="nx">r</span>&#34;<span class="p">)</span><span class="err"> </span><span class="err"></span><span class="nx">endfunc</span><span class="err"> </span><span class="err"> </span><span class="err"></span><span class="nx">command</span><span class="p">!</span> <span class="p">-</span><span class="nx">nargs</span><span class="p">=</span><span class="m">1</span> <span class="nx">Note</span> <span class="nx">call</span> <span class="nx">custom</span>#<span class="m">99</span>notes#<span class="nx">edit</span><span class="p">(&lt;</span><span class="nx">q</span><span class="p">-</span><span class="nx">args</span><span class="p">&gt;)</span><span class="err"> </span><span class="err"></span><span class="nx">command</span><span class="p">!</span> <span class="nx">YankNoteName</span> <span class="nx">call</span> <span class="nx">custom</span>#<span class="m">99</span>notes#<span class="nx">copynotename</span><span class="p">(&lt;</span><span class="nx">f</span><span class="p">-</span><span class="nx">args</span><span class="p">&gt;)</span><span class="err"> </span></code></pre></div><p>But even with all of that sometimes I might have to rename a note. For example, when I create a note from <a href="">Markor</a>. That&rsquo;s when <em>rename</em> script comes into play. It not only renames files, but also searches and replaces all their back-references:</p> <div class="highlight"><pre class="chroma"><code class="language-bash" data-lang="bash"><span class="cp">#!/bin/bash </span><span class="cp"></span> <span class="nv">from</span><span class="o">=</span><span class="nv">$1</span> <span class="nv">to</span><span class="o">=</span><span class="nv">$2</span> <span class="nv">frombase</span><span class="o">=</span><span class="si">${</span><span class="nv">from</span><span class="p">##*/</span><span class="si">}</span> <span class="c1"># remove largest prefix</span> <span class="nv">tobase</span><span class="o">=</span><span class="si">${</span><span class="nv">to</span><span class="p">##*/</span><span class="si">}</span> <span class="nv">dir</span><span class="o">=</span><span class="s2">&#34;</span><span class="k">$(</span>dirname <span class="s2">&#34;</span><span class="nv">$from</span><span class="s2">&#34;</span><span class="k">)</span><span class="s2">&#34;</span> <span class="k">for</span> cf in content/*<span class="p">;</span> <span class="k">do</span> <span class="k">if</span> <span class="o">[[</span> ! -d <span class="s2">&#34;</span><span class="nv">$cf</span><span class="s2">&#34;</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then</span> <span class="k">continue</span> <span class="k">fi</span> <span class="nv">files</span><span class="o">=</span><span class="k">$(</span>git -C <span class="s2">&#34;</span><span class="nv">$cf</span><span class="s2">&#34;</span> ls-files <span class="s2">&#34;*.md&#34;</span><span class="k">)</span> <span class="nb">pushd</span> <span class="s2">&#34;</span><span class="nv">$cf</span><span class="s2">&#34;</span> &gt; /dev/null sed -i <span class="s2">&#34;s/</span><span class="nv">$frombase</span><span class="s2">/</span><span class="nv">$tobase</span><span class="s2">/g&#34;</span> <span class="nv">$files</span> <span class="nb">popd</span> &gt; /dev/null <span class="k">done</span> git -C <span class="s2">&#34;</span><span class="nv">$dir</span><span class="s2">&#34;</span> mv <span class="s2">&#34;</span><span class="nv">$frombase</span><span class="s2">&#34;</span> <span class="s2">&#34;</span><span class="nv">$tobase</span><span class="s2">&#34;</span> </code></pre></div><p>But manual searching of files that don&rsquo;t conform to my filename spec is still too much work to do, so I have <em>bulkrename</em> script which fixes my laziness. Well, not exactly fixes laziness itself, but rather its consequences. It takes a note&rsquo;s title and creates a correct file name from it, with spaces replaced by dashes and all.</p> <div class="highlight"><pre class="chroma"><code class="language-bash" data-lang="bash"><span class="cp">#!/bin/bash </span><span class="cp"></span> <span class="nv">skip</span><span class="o">=(</span><span class="s2">&#34;content/;</span> <span class="s2">&#34;content/0/;</span><span class="o">)</span> <span class="k">for</span> dir in content/*<span class="p">;</span> <span class="k">do</span> <span class="k">if</span> <span class="o">[[</span> ! -d <span class="s2">&#34;</span><span class="nv">$dir</span><span class="s2">&#34;</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then</span> <span class="k">continue</span> <span class="k">fi</span> <span class="nv">files</span><span class="o">=</span><span class="k">$(</span>git -C <span class="s2">&#34;</span><span class="nv">$dir</span><span class="s2">&#34;</span> ls-files <span class="s2">&#34;*.md&#34;</span><span class="k">)</span> <span class="k">for</span> f in <span class="nv">$files</span><span class="p">;</span> <span class="k">do</span> <span class="nv">oldf</span><span class="o">=</span><span class="s2">&#34;</span><span class="nv">$dir</span><span class="s2">/</span><span class="nv">$f</span><span class="s2">&#34;</span> <span class="c1"># skip explicitly listed files</span> <span class="k">if</span> <span class="o">[[</span> <span class="s2">&#34; </span><span class="si">${</span><span class="nv">skip</span><span class="p">[@]</span><span class="si">}</span><span class="s2"> &#34;</span> <span class="o">=</span>~ <span class="s2">&#34; </span><span class="nv">$oldf</span><span class="s2"> &#34;</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then</span> <span class="c1"># check if array contains a value</span> <span class="nb">echo</span> <span class="s2">&#34;Skipping </span><span class="nv">$oldf</span><span class="s2"> (excluded)&#34;</span> <span class="k">continue</span> <span class="k">fi</span> <span class="c1"># skip files which already match a filename pattern (</span> <span class="c1"># where shortuuid are first 8 hexadecimal numbers od full UUID</span> <span class="k">if</span> <span class="o">[[</span> <span class="s2">&#34;</span><span class="nv">$f</span><span class="s2">&#34;</span> <span class="o">=</span>~ ^.+-<span class="o">[</span>a-f0-9<span class="o">]{</span>8<span class="o">}</span>.md$ <span class="o">]]</span><span class="p">;</span> <span class="k">then</span> <span class="nb">echo</span> <span class="s2">&#34;Skipping </span><span class="nv">$oldf</span><span class="s2"> (name correct)&#34;</span> <span class="k">continue</span> <span class="k">fi</span> <span class="nv">shortuuid</span><span class="o">=</span><span class="k">$(</span>uuidgen <span class="p">|</span> sed <span class="s1">&#39;s/-.*//&#39;</span><span class="k">)</span> <span class="nv">title</span><span class="o">=</span><span class="s2">&#34;</span><span class="k">$(</span>awk -F <span class="s1">&#39;[ ().?!,/-]+&#39;</span> <span class="s1">&#39;/^title:/ {for(i=2;i&lt;NF;i++) printf &#34;%s-&#34;, tolower($i); print tolower($NF)}&#39;</span> <span class="s2">&#34;</span><span class="nv">$oldf</span><span class="s2">&#34;</span><span class="k">)</span><span class="s2">&#34;</span> <span class="nv">newf</span><span class="o">=</span><span class="s2">&#34;</span><span class="nv">$dir</span><span class="s2">/</span><span class="nv">$title</span><span class="s2">-</span><span class="nv">$shortuuid</span><span class="s2">.md&#34;</span> <span class="nb">echo</span> <span class="s2">&#34;Renaming </span><span class="nv">$oldf</span><span class="s2"> -&gt; </span><span class="nv">$newf</span><span class="s2">&#34;</span> ./rename <span class="s2">&#34;</span><span class="nv">$oldf</span><span class="s2">&#34;</span> <span class="s2">&#34;</span><span class="nv">$newf</span><span class="s2">&#34;</span> <span class="k">done</span> <span class="k">done</span> </code></pre></div><p>One thing it lacks is a progress bar. HAHA. HAHA. HA. And now I&rsquo;m thinking about implementing one (which shouldn&rsquo;t be that hard). Thank you very much, brain, I love you too, that&rsquo;s exactly what my notes setup lacks: a progress bar in a script which will be used maybe once a year or less. Oh boy, have you seen these nice progress bars that recent pytest versions have? I think they even have CONCURRENT progress bars in pytest-xdist! Maybe I can get one too? Because I certainly can run this script concurrently!</p> <section class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1" role="doc-endnote"> <p>Andrzej Sapkowski - The Lady of the Lake <a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:2" role="doc-endnote"> <p><code>python3 -m http.server 9999</code> <a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:3" role="doc-endnote"> <p>Trivia: it&rsquo;d take over 136 years to create that many notes if a note was created every second. <a href="#fnref:3" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </section> Don't change alternate file in Vim Wed, 08 Jan 2020 00:00:00 +0000 <p>It seems that in Vim <code>:read &lt;file&gt;</code> command (used for inserting other file&rsquo;s contents below the cursor in current buffer) changes alternate file to the inserted one (you can jump between current and alternate file with <code>C-^</code> or <code>C-6</code>, which is super handy). It&rsquo;s not obvious and in my opinion it breaks the rule of least astonishment, because I totally didn&rsquo;t ask Vim to do anything outside of a current buffer (and buffer list shown by <code>:ls</code> isn&rsquo;t even changed and might not contain that file at all). This behaviour complicated my workflow for quickly adding new notes from templates.</p> <p>Fortunately (TIL) there&rsquo;s easy fix &ndash; or rather workaround &ndash; for this. We can prefix any Vim command with <code>:keepalt</code> and this command won&rsquo;t change alternate file in any way. So <code>:r &lt;file&gt;</code> becomes <code>:keepalt r &lt;file&gt;</code>. Super handy and good to know, especially for making custom functions, plugins etc.</p> Fixed RSS Sat, 04 Jan 2020 00:00:00 +0000 <p>For a moment I broke <a href="">RSS feed</a> and then I broke it again by updating it with all previous posts. I updated Hugo to a new version due Markdown rendering issues in the <a href="">last</a> post and it seems that Hugo team introduced quite a lot backward incompatible changes in recent versions of Hugo. Anyway, RSS feed should be fixed now and it should display 15 last articles. Sorry for inconvenience!</p> Ctags For Markdown Notes Fri, 03 Jan 2020 00:00:00 +0000 <p>I keep my notes in plaintext Markdown files. Previously I used vimwiki for managing them but I felt that it was somehow bloated and its Markdown engine slightly differed from the more typical engine used in Markor (which is an Android app which I sometimes use to view and edit notes on the phone). Since I switched away from vimwiki to plain Markdown, there was one thing I missed: tags.</p> <p>Generally I tag my notes and since recently I use 2 types of tags: inline tags which resemble org-mode style of tagging (<em>:tag1:tag2:tag3:</em>), and document-wide tags which are placed in YAML Front Matter of my Markdown files. Typical note looks like this:</p> <div class="highlight"><pre class="chroma"><code class="language-markdown" data-lang="markdown">--- title: Some title tags: [foo, bar] --- <span class="k">-</span> Lorem ipsum :baz:blah: <span class="k">-</span> dolor sit amet :consecteur: </code></pre></div><p>Vimwiki impements its custom way of hand-generating tags file for org-mode style tags. Thanks to it I can jump in vim directly to the place where selected tag is found. For example with <code>:tjump /&lt;tab&gt;</code> I can easily search through all tags in all of my notes.</p> <p>Today I managed to replicate the way of generating tags for my notes, thanks to <a href="">extensibility</a> of ctags implementation that I use: <a href="">Universal Ctags</a>.</p> <p>Long story short, we have to create a file in <em>~/.ctags.d</em> with the following content:</p> <div class="highlight"><pre class="chroma"><code class="language-txt" data-lang="txt">--langdef=notes --languages=notes --kinddef-notes=t,tag,tags --kinddef-notes=d,doctag,doctags --_tabledef-notes=main --_tabledef-notes=frontmatter --_tabledef-notes=fmtags --_mtable-regex-notes=main/---//{tenter=frontmatter} --_mtable-regex-notes=main/:([a-zA-Z][a-zA-Z0-9]*):/\1/t/{mgroup=1}{_advanceTo=1end} --_mtable-regex-notes=main/.// --_mtable-regex-notes=frontmatter/^tags: *//{tenter=fmtags} --_mtable-regex-notes=frontmatter/---//{tleave} --_mtable-regex-notes=frontmatter/.// --_mtable-regex-notes=fmtags/([a-zA-Z][a-zA-Z0-9]*)/\1/d/ --_mtable-regex-notes=fmtags/\]//{tleave} --_mtable-regex-notes=fmtags/://{tleave} --_mtable-regex-notes=fmtags/---//{tleave}{_advanceTo=0start} --_mtable-regex-notes=fmtags/.// </code></pre></div><p>This defines a new regex parser for files with <em>.md</em> extension. This parser is aware of its context (i.e. whether it is inside Front Matter or not) and works like this:</p> <figure class="center"> <a href=""> <img src="" /> </a> <figcaption><p>Ctags table stack</p></figcaption> </figure> <ol> <li> <p>Start in <em>main</em> table (context). This is generic context able to capture org-mode style tags. Thanks to <code>{_advanceTo=1end}</code> directive, a single colon between tags acts as a separator (i.e. will be the start character for the next iteration of ctags regex engine).</p> </li> <li> <p>When we&rsquo;re inside main context, look out for 3 dashes. When they&rsquo;re found, push <em>frontmatter</em> context on top of the stack. This context itself acts as intermediate context: it will pop itself when another 3 dashes occur or it will push <em>fmtags</em> context when <em>tags:</em> text is found.</p> </li> <li> <p>When in <em>fmtags</em> context, any alphanumeric string will be matched as a tag. This automatically handles two forms of YAML&rsquo;s lists: one-line inside brackets, and multiline:</p> <div class="highlight"><pre class="chroma"><code class="language-yaml" data-lang="yaml"><span class="k">tags</span><span class="p">:</span><span class="w"> </span><span class="p">[</span>foo<span class="p">,</span><span class="w"> </span>bar<span class="p">]</span><span class="w"> </span><span class="w"></span><span class="k">tags</span><span class="p">:</span><span class="w"> </span><span class="w"> </span>- foo<span class="w"> </span><span class="w"> </span>- bar<span class="w"> </span></code></pre></div></li> <li> <p><em>fmtags</em> context pops when 3 dashes or colon (some other metadata) or list closing bracket is found. 3 dashes mean end of Front Matter and thanks to <code>{_advanceTo=0start}</code> after the pop they&rsquo;re found again by <em>frontmatter</em> table, which can in turn pop again and bring us back to <em>main</em> context.</p> </li> <li> <p>All tags must start with a letter (and not a number) to avoid interpreting time as a tag (e.g. 12:00:13 AM).</p> </li> </ol> <p>Ctags regex parser uses <a href="">Extended Regular Expression</a> syntax for regular expressions as defined by POSIZ and implemented by glibc. Unfortunately it has many limitations. For example, my tag-matching regular expression (<code>([a-zA-Z][a-zA-Z0-9]*)</code>) fails to match non-ASCII characters, which is kinda braindead in the age of Unicode. For now I changed some of my Polish tags to their English names, but I&rsquo;ll keep on searching for more elegant solution.</p> Migadu Thu, 21 Nov 2019 00:00:00 +0000 <p>I just read on Mastodon Drew DeVault&rsquo;s <a href="">recommendation</a> of <a href="">Migadu</a>, which is a small, independent e-mail provider from Switzerland that looks too good to be real. Never heard of them before but I got curious and created a free account. I haven&rsquo;t switched yet (I must re-configure DNS), but I already had a pleasant experience. I checked their website and haven&rsquo;t seen anything about server-side mail filtering so I reached their support about that. About 15 minutes later I was replied by Miki from Migadu that they already implemented Sieve backend, but need few more weeks to finish their management app.</p> <p>Currently I&rsquo;m happy FastMail customer, but I&rsquo;d like to migrate to a service that doesn&rsquo;t charge me per-mailbox. I&rsquo;d love my wife to have mail address in our home domain, but double paying for that sucks. Especially that our son will probably need a mail in future as well. Migadu has 3 paid plans based on number of mails you send every day. Unless I&rsquo;m a part of heavy discussion on some issue tracker, I usually send 2-3 mails a month so today&rsquo;s mail to Migadu almost filled my monthly quota. :)</p> <p>So I&rsquo;ll probably migrate myself in the following days as I don&rsquo;t use server-side filters THAT hard (with Mutt mail management is easy), but it&rsquo;s a no-go for my wife. Well, good luck Migadu, I&rsquo;ll be checking your changelog daily!</p>