2021-12-02 (permalink)

Advent of Code: [2021/02](https://adventofcode.com/2018/day/2)

It seems like the submarine can take a series of commands like `forward 1`, `down 2`, or `up 3`:


`forward X` increases the horizontal position by X units.

`down X` increases the depth by X units.

`up X` decreases the depth by X units.


Note that since you're on a submarine, down and up affect your depth, and so they have the opposite result of what you might expect.




Calculate the horizontal position and depth you would have after following the planned course. What do you get if you multiply your final horizontal position by your final depth?

OK, let's start by parsing our input, i.e. the list of instructions, into `<direction,unit>` pairs:

- CL-PPCRE:REGISTER-GROUPS-BIND to parse our instructions using regexps (we could have also used SPLIT-SEQUENCE)
- AS-KEYWORD to convert a string into a _keyword_, i.e. `:forward`, `:up`, and `:down`
(defun parse-instructions (data) (mapcar #'parse-instruction data))

(defun parse-instruction (string)
  (cl-ppcre:register-groups-bind ((#'as-keyword dir) (#'parse-integer delta))
      ("(\\w+) (\\d+)" string)
    (cons dir delta)))
(defun dir (i) (car i))
(defun delta (i) (cdr i))
Now to get the answer for part two:

- Iterate over the instructions
- If `:forward`, we move our horizontal position
- If `:up`, we reduce our depth
- If `:down`, we increase our depth
- Last, we return the product of our horizontal position and depth
(defun part1 (instructions &aux (horiz 0) (depth 0))
  (dolist (i instructions (* horiz depth))
    (ecase (dir i)
      (:forward (incf horiz (delta i)))
      (:up (decf depth (delta i)))
      (:down (incf depth (delta i))))))
For part 2 instead, the `:up` and `:down` instructions seem to assume a different meaning:

In addition to horizontal position and depth, you'll also need to track a third value, aim, which also starts at 0. The commands also mean something entirely different than you first thought:


- `down X` increases your aim by X units.

- `up X` decreases your aim by X units.

- `forward X` does two things:


- It increases your horizontal position by X units.

- It increases your depth by your aim multiplied by X.


Again note that since you're on a submarine, down and up do the opposite of what you might expect: "down" means aiming in the positive direction.




Using this new interpretation of the commands, calculate the horizontal position and depth you would have after following the planned course. What do you get if you multiply your final horizontal position by your final depth?

Nothing fancy, let's just implement this:

- Iterate over the instructions
- If `:forward`, we move our horizontal position and descend / ascend based on the current aim
- If `:up`, we increase our aim
- If `:down`, we decrease our aim
- Last, we return the product of our horizontal position and depth
(defun part2 (instructions &aux (horiz 0) (depth 0) (aim 0))
  (dolist (i instructions (* horiz depth))
    (ecase (dir i)
      (:forward (incf horiz (delta i)) (decf depth (* (delta i) aim)))
      (:up (incf aim (delta i)))
      (:down (decf aim (delta i))))))
Final plumbing:
(define-solution (2021 02) (instructions parse-instructions)
  (values (part1 instructions) (part2 instructions)))

(define-test (2021 02) (1648020 1759818555))
And that's it:
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.002 seconds of real time
  0.002600 seconds of total run time (0.001565 user, 0.001035 system)
  150.00% CPU
  6,082,778 processor cycles
  327,536 bytes consed
PS. My [original solution](https://topaz.github.io/paste/#XQAAAQD2AgAAAAAAAAAUGQimgx+p6PZhRh7uIO5BVe7pcMkiG6eOV22j2MYKuFs91c+ybCSTgFinLxYMDBog9+B+g/uA1iuHBXT5kPl5FbfHsjTt/UH6LeDhwtxS++LtE/cfL47hqbZilPQVUp7nWL834NkL1xo124ObhgTHG4Vaq0sDvXEWYVT61bdI9njY95YkUyYD8jeF6mdT+45tB+epQP+OgkkUyyWgSGXG3mXlBPAuGtfQ2d6F6X56fu266chYGK/yaubPY1GuajRiidCeYijfEAPoeMsgyIZcp6U4hgNM0D1M6vPzyThdMrcJtAwVid8BqQUE8ebzylpWag4uZPUQgHKdoZTgteopbArgU34gzK34OzvfXyvwL063kO+uz3rYxgePqqnFxROOQdSe0tseMzpTFeHv+lLiE+efMBoId4Ao8BsmGov79aT/8r1yLg==) parsed instructions into COMPLEX numbers (e.g. `down 4` into `#C(0 -4)`); this way, for part 1 all I had to do was summing all the deltas together; for part 2 instead, I had to check if the delta was horizontal or vertical, and adjust the aim accordingly.  I figured a more readable solution would be better, hence the rewrite.

2021-12-01 (permalink)

Advent of Code: [2021/01](https://adventofcode.com/2018/day/1)

- An elf tripped, and accidentally sent the sleight keys flying into the ocean
- All of a sudden we found ourselves inside a submarine which has an antenna that hopefully we can use to track the keys
- The submarine performs a sonar sweep of the nearby sea floor (our input)

For example, suppose you had the following report:













This report indicates that, scanning outward from the submarine, the sonar sweep found depths of 199, 200, 208, 210, and so on.

We already have an utility function to parse a list of integers, PARSE-INTEGERS, so there isn't much we have to do here, except for calling it:
(parse-integers (uiop:read-file-lines "src/2021/day01.txt"))
Now, here is what we are tasked to do for part 1:

The first order of business is to figure out how quickly the depth increases, just so you know what you're dealing with - you never know if the keys will get carried into deeper water by an ocean current or a fish or something.


To do this, count **the number of times a depth measurement increases** from the previous measurement. (There is no measurement before the first measurement.) [...]

Nothing crazy; we simply do as told:
(defun part1 (numbers)
  (loop for (a b) on numbers
        when b count (< a b)))
For part 2 instead:

Instead, consider sums of a three-measurement sliding window. [...]

Here we can re-use PART1 completely, except that we need to pass to it a different list of numbers, each representing the sum of a three-measurement sliding window; again, LOOP/ON makes for a very readable solution:
(defun part2 (numbers)
    (loop for (a b c) on numbers
          when c collect (+ a b c))))
Final plumbing:
(define-solution (2021 01) (numbers parse-integers)
  (values (part1 numbers) (part2 numbers)))

(define-test (2021 01) (1557 1608))
And that's it:
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.001 seconds of real time
  0.001252 seconds of total run time (0.000647 user, 0.000605 system)
  100.00% CPU
  2,873,667 processor cycles
  163,808 bytes consed

2021-11-26 (permalink)

Vlime: Make swank_repl a prompt buffer

(This was posted on [vlime/pull/55](https://github.com/vlime/vlime/pull/55))

Alright, since these changes got merged into @phmarek's fork, and since _that_ branch is the one I am using these days, I would like to provide some feedback about this; also, please note that I am running this with Vim, so expect some of the following feedback not be relevant to Neovim users.

I have no number to back this up, but I noticed that forms generating some non-trivial output are more likely to cause Vim to freeze if the REPL is made a prompt buffer than when it's not; don't take me wrong, t's a bit sluggish anyway, with or without these changes, but more so with the prompt buffer than without.

I have a [project](https://github.com/iamFIREcracker/adventofcode) with quite a few packages, and every time I touch the main .asd file and QL:QUICKLOAD it, Vim becomes very sluggish.

**Stealing focus**
The currently focused window seems to be losing focus every time a new line of output is added to the prompt buffer; this causes all sort of issues, especially if you were in insert mode when the focus got stolen.

I was messing around with a Web application of mine, and had the client fire a HEAD request every second (request which got logged by the Web server on the standard output, i.e. on the REPL buffer); that alone seemed to make editing in a different buffer impossible (e.g. missing characters, unable to move around the buffer), to the point that I had to disable the polling on the client!

I am not exactly sure what's going on, and I would not be surprised if this wasn't somehow related to the performance issue above; however, the moment I disabled this PR's changes, it all become responsive again.

**Current package**
While Vim itself might be the cause of the two problems mentioned above, this one is most definitely a bug with the current implementation.  Previously, forms typed into `vlime_input` buffer would be evaluated inside the package specified in the closest preceding IN-PACKAGE form; for example, imagine you had the following buffer open:
1: (in-package :cl-user)
5: (in-package :foo)
Opening up an input buffer (i.e. `<localleader>si`) from line 2, and typing `*package*` into it would cause `#<PACKAGE "COMMON-LISP-USER">` to be logged to the REPL buffer; opening up the input buffer from line number 6 instead, and typing in the same would output `#<PACKAGE "FOO">`.  Well, with this pull request, it would always output `#<PACKAGE "COMMON-LISP-USER">`.

**Only works while in insert mode (minor, opinionated)**
I am not sure if this is Vim's fault or if the plugin's implementation, but evaluation from the prompt buffer only seems to work while in insert mode, i.e. pressing enter while in insert mode seems to be the only way to trigger the evaluation to happen.  This unfortunately clashes with some of my settings / how I am used to work; I can work around this of course, re-wire my muscle memory, but I wanted to flag this anyway.

I use [delimitMate](https://github.com/Raimondi/delimitMate) to keep my parens balanced (yes, it's a poor man's solution, but gets the job done most of the times); this means that if I want to type in the form:
          1         2
(format t "Hello world")
By the time I type the closing quote at position 22, the closing parenthesis would be there already, so I would just leave insert mode and expect the form to be evaluated; however, as soon as I leave insert mode, a new line is added to the prompt buffer and then I am forced to type it all again, the closing parenthesis included, and then press `Enter`.

**Empty prompt (opinionated)**
Maybe it's just me, but the first time I entered insert mode while in the prompt buffer I started wondering what that initial space was for.  I thought it was a bug, having something to do with Vlime wrongly calculating the indentation level; but then I looked at the code and realized that that was the way we told the prompt buffer to look like.

I think we should either set this to an empty string, or maybe the classic `> ` or `* `, as that would make it super clear what's going on.

**Next steps**
I am OK with tinkering with this a bit more, but here is what I think we should do:

- Understand if the performance issue mentioned above, as well as the focus stealing one, are getting experienced by everybody, i.e. Vim and Neovim users, or if it's just one man's problem (me...sigh); this will help us understand if switching to a prompt buffer for the REPL is a sound idea or not
- If it was, maybe put everything behind a config flag, so people can enable / disable it, at least until it's a bit more stable (not everybody wants to deal with this)
- Work around some of the usability issues mentioned above, like setting the current package, or changing the prompt string, etc.

Let me know what you guys think about this.


2021-11-25 (permalink)

Building a _local_ system with :TRIVIAL-BUILD

(This was posted on [ceramic/trivial-build/issues/2](https://github.com/ceramic/trivial-build/issues/2))

Currently, there are at least two different ways of dealing with locally defined systems, i.e. systems defined in the working directory:

1. Create a symlink inside ~/quicklisp/local-projects (or ~/common-lisp), pointing to the current directory
2. Add the current directory to ASDF:\*CENTRAL-REPOSITORY\* (`(pushnew '*default-pathname-defaults* asdf:*central-registry*)`)

Now, with the first option, everything works as expected: you call TRIVIAL-BUILD:BUILD, it spawns a new CL image, it runs ASDF:LOAD-SYSTEM in it, and since the current directory is inside ~/quicklisp/local-projects (or ~/common-lisp), your system will be loaded just fine.

Unfortunately the current implementation does not seem to play nicely with the second option, i.e. updating ADSF:\*CENTRAL-REPOSITORY\*; that's because the newly spawn CL image, the one in which ASDF:LOAD-SYSTEM is called, has ASDF:\*CENTRAL-REPOSITORY\* in its default state, i.e. without our customizations.

Is there any interest in supporting this use case, or instead you prefer to push people to use option 1 instead?  Because if there was, here is how I _patched_ LOAD-AND-BUILD-CODE (note the line right above the ASDF:LOAD-SYSTEM call):

- I serialize the current value of ASDF:*CENTRAL-REGISTRY*, in reverse order, so while loading them later with PUSH, the original order is preserved; note: this happens on the host image, i.e. the one where :CERAMIC is run
- For each entry, I push it back into ASDF:*CENTRAL-REGISTRY* -- this instead is run on the final app image, the newly spawned one
(defun load-and-build-code (system-name entry-point binary-pathname)
  "Return a list of code strings to eval."
   "(setf *debugger-hook* #'(lambda (c h) (declare (ignore h)) (uiop:print-condition-backtrace c) (uiop:quit -1)))"
   (format nil "(dolist (entry '~W) (push entry asdf:*central-registry*))" (reverse asdf:*central-registry*))
   (format nil "(asdf:load-system :~A)" system-name)
   (format nil "(setf uiop:*image-entry-point* #'(lambda () ~A))"
   (format nil "(uiop:dump-image ~S :executable t
  #+sb-core-compression :compression #+sb-core-compression t)"
Let me know if you see any problems with this and most importantly if you find this somewhat interesting, in which case, I would not mind creating a pull request for this.


2021-11-23 (permalink)

Advent of Code: [2018/10](https://adventofcode.com/2018/day/10)

In today's problem, we are given a sky map: a list of stars, characterized by their position and velocity:
position=< 9,  1> velocity=< 0,  2>
position=< 7,  0> velocity=<-1,  0>
position=< 3, -2> velocity=<-1,  1>
We are also told, that after some time, stars would align to form a message; that message will be the answer to our part 1.

Let's start by parsing the sky map; we are going to define a custom structure, STAR, having one slot for the star position and one for its velocity; we are also going to store all these stars into a simple list:
(defstruct (star (:conc-name nil)) pos vel)

(defun parse-sky (data)
  (mapcar #'parse-star data))

(defun parse-star (string)
  (destructuring-bind (x y vx vy)
      (mapcar #'parse-integer (cl-ppcre:all-matches-as-strings "-?\\d+" string))
    (make-star :pos (list x y) :vel (list vx vy))))
Now, how can to figure out when the stars are aligned? My idea is to take a look at the bounding box containing all the stars: as the velocity of the stars does not change, I would expect the size of this box to become smaller and smaller, until it starts getting bigger again.  Well, the state of the sky when the bounding box is the smallest, is probably the one in which all the stars are aligned; so dumping it on the standard output would hopefully show us the message!

FIND-MESSAGE accepts the sky map as input, and keeps on moving stars (see SKY-NEXT) minimizing the bounding box (I am using the area of the bounding box, SKY-AREA, as a proxy for its size); lastly, SKY-MESSAGE is responsible for analyzing the end sky map, and print `#` when the stars are:
(defun find-message (input)
  (loop :for sky-prev = input :then sky
        :for sky = input :then (sky-next sky)
        :when (> (sky-area sky) (sky-area sky-prev))
        :return (sky-message sky-prev)))

(defun sky-next (sky) (mapcar #'star-move sky))

(defun star-move (star)
  (with-slots (pos vel) star
    (make-star :pos (mapcar #'+ pos vel) :vel vel)))

(defun sky-area (sky)
  (destructuring-bind ((x-min x-max) (y-min y-max)) (sky-box sky)
    (* (- x-max x-min) (- y-max y-min))))

(defun sky-message (sky)
  (destructuring-bind ((x-min x-max) (y-min y-max)) (sky-box sky)
    (let ((output (loop :repeat (1+ (- y-max y-min))
                        :collect (make-string (1+ (- x-max x-min)) :initial-element #\Space))))
      (loop :for star :in sky :for (x y) = (pos star)
            :for row = (nth (- y y-min) output)
            :do (setf (aref row (- x x-min)) #\#))
      (with-output-to-string (s)
        (format s "~%~{~a~^~%~}" output)))))

(defun sky-box (sky)
  (loop :for star :in sky :for (x y) = (pos star)
        :minimize x :into x-min
        :maximize x :into x-max
        :minimize y :into y-min
        :maximize y :into y-max
        :finally (return (list (list x-min x-max) (list y-min y-max)))))
Let's give this a go:
(setq input (parse-sky (uiop:read-file-lines "src/2018/day10.txt")))
> (find-message input)
#####   #    #  #####      ###   ####   #       #####   ######
#    #  #    #  #    #      #   #    #  #       #    #  #
#    #  #    #  #    #      #   #       #       #    #  #
#    #  #    #  #    #      #   #       #       #    #  #
#####   ######  #####       #   #       #       #####   #####
#    #  #    #  #           #   #  ###  #       #       #
#    #  #    #  #           #   #    #  #       #       #
#    #  #    #  #       #   #   #    #  #       #       #
#    #  #    #  #       #   #   #   ##  #       #       #
#####   #    #  #        ###     ### #  ######  #       ######"
Good!  Now part 2:

Impressed by your sub-hour communication capabilities, the Elves are curious: exactly how many seconds would they have needed to wait for that message to appear?

We should be able to answer this quite easily, by slightly tweaking FIND-MESSAGE:

- Keep track of how much time has elapsed
- And return it, as second value (the call to 1- is to offset the fact that the first iteration is a dummy one, given that `sky-prev` and `sky` are the same)
(defun find-message (input)
  (loop :for time :from 0
        :for sky-prev = input :then sky
        :for sky = input :then (sky-next sky)
        :when (> (sky-area sky) (sky-area sky-prev))
        :return (values (sky-message sky-prev) (1- time))))
Final plumbing:
(define-solution (2018 10) (sky parse-sky)
  (find-message sky))

(define-test (2018 10) ("
#####   #    #  #####      ###   ####   #       #####   ######
#    #  #    #  #    #      #   #    #  #       #    #  #
#    #  #    #  #    #      #   #       #       #    #  #
#    #  #    #  #    #      #   #       #       #    #  #
#####   ######  #####       #   #       #       #####   #####
#    #  #    #  #           #   #  ###  #       #       #
#    #  #    #  #           #   #    #  #       #       #
#    #  #    #  #       #   #   #    #  #       #       #
#    #  #    #  #       #   #   #   ##  #       #       #
#####   #    #  #        ###     ### #  ######  #       ######" 10831))
And that's it:
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.440 seconds of real time
  0.416292 seconds of total run time (0.406274 user, 0.010018 system)
  [ Run times consist of 0.029 seconds GC time, and 0.388 seconds non-GC time. ]
  94.55% CPU
  1,012,629,537 processor cycles
  309,925,952 bytes consed

Advent of Code: [2018/7](https://adventofcode.com/2018/day/7)

Given a list of steps and requirements about which steps are blocked by which other step or steps, we are tasked to find the order in which the steps are going to be completed.

Step C must be finished before step A can begin.
Step C must be finished before step F can begin.
Step A must be finished before step B can begin.
Step A must be finished before step D can begin.
Step B must be finished before step E can begin.
Step D must be finished before step E can begin.
Step F must be finished before step E can begin.
Visually, these requirements look like this:
 /    \      \
C      -->D----->E
 \           /
And the order in which the steps are going to be completed is: `CABDFE` (in the event of multiple steps being unblocked, we should select based on the lexicographical order).

We are going to parse our input into an association list, mapping from steps to their dependencies:
(defun parse-instructions (data)
  (let (rez)
    (loop :for string :in data
          :for (step . dep) = (parse-instruction string)
          :for step-entry = (assoc step rez)
          :for dep-entry = (assoc dep rez)
          :if step-entry :do (push dep (cdr step-entry))
          :else :do (push (cons step (list dep)) rez)
          :unless dep-entry :do (push (cons dep nil) rez))

(defun parse-instruction (string)
  (cl-ppcre:register-groups-bind ((#'parse-char dep step))
      ("Step (\\w) must be finished before step (\\w) can begin." string)
    (cons step dep)))
Feeding the example above into PARSE-INSTRUCTIONS, should result in the following alist:
((#\E #\F #\D #\B)
 (#\D #\A)
 (#\B #\A)
 (#\F #\C)
 (#\A #\C))
Now, to find the order in which the different steps are completed, we are going to proceed as follows:

- We sort the instructions, placing entries with the fewest number of dependencies (ideally, 0) first, or lexicographically in case of ties (see SORT-INSTRUCTIONS, and INSTRUCTION<)
- We pop the first item -- this is the next step to complete
- We unblock any other instruction that might be depending on the step we just completed (see REMOVE-DEPENDENCY)
- We do this until we run out of instructions
(defun part1 (instructions &aux completed)
    (if (null instructions)
      (return (format nil "~{~C~}" (reverse completed)))
      (destructuring-bind ((step) . rest) (sort-instructions instructions)
        (setf completed (cons step completed)
              instructions (remove-dependency rest step))))))

(defun sort-instructions (instructions)
  (sort (copy-seq instructions) #'instruction<))

(defun instruction< (i1 i2)
  (destructuring-bind (step1 . deps1) i1
    (destructuring-bind (step2 . deps2) i2
      (or (< (length deps1) (length deps2))
          (and (= (length deps1) (length deps2))
               (char< step1 step2))))))

(defun remove-dependency (instructions dependency)
  (loop :for (step . deps) :in instructions
        :collect (cons step (remove dependency deps))))
Things are going to be a tiny bit more complicated for part 2:

- Each step takes a certain amount of time to complete: 60 seconds, plus a certain amount, depending on the step name (1 second for `A`, 2 seconds for `B`, ...)
- We are not alone in executing these instructions -- there are 4 elves helping us out, which means multiple steps can run in parallel

We are going to solve this by simulation; to do this, we are going to define a new structure, WORKERS, with the following slots:

- `'BUSY-FOR`, an array as big as the number of workers we have, to keep track of how much time each worker needs to complete the assigned step
- `'BUSY-WITH`, another array, this time keeping track of the step each worker is currently busy with
(defstruct (workers (:conc-name nil)
                    (:constructor make-workers%))

(defun make-workers (n)
  (make-workers% :busy-for (make-array n :initial-element 0)
                 :busy-with (make-array n :initial-element nil)))
With this, the skeleton of our solutions is pretty simple: we initialize our WORKERS state, and until there are workers..._working_...we do another tick of our simulation; when done, we simply return the elapsed time:
(defparameter *workers-count* 5)

(defun part2 (instructions)
  (let ((workers (make-workers *workers-count*))
        (time 0))
      (setf instructions (workers-tick workers instructions))
      (if (every #'null (busy-with workers))
        (return time)
        (incf time)))))
Let's now take a look at the meat of this, WORKERS-TICK:

- We let all the workers progress on their assigned step, and keep track of the completed ones (as well as the workers that finished them)
- We keep the workers busy until there are workers available and unblocked instructions
- Finally, we return the updated list of instructions -- i.e. instructions we could not assign because either we run out of workers, or because active workers were still working on some dependencies
(defun workers-tick (workers instructions)
  (with-slots (busy-for busy-with) workers
    (flet ((try-release-workers ()
             (loop :for i :from 0
                   :for left :across busy-for
                   :for step :across busy-with
                   :if (> left 1) :do (decf (aref busy-for i))
                   :else :collect i :and :do
                   (setf instructions (remove-dependency instructions step)
                         (aref busy-for i) 0
                         (aref busy-with i) nil)))
           (try-assign-work (available)
             (loop :while (and available instructions) :do
                   (let ((i (first available)))
                     (setf instructions (sort-instructions instructions)
                           (aref busy-for i) 0
                           (aref busy-with i) nil)
                     (destructuring-bind ((step . deps) . rest) instructions
                       (if (> (length deps) 0)
                         (setf (aref busy-for i) (step-time step)
                               (aref busy-with i) step
                               available (rest available)
                               instructions rest)))))))
      (let ((available (try-release-workers)))
        (try-assign-work available)

(defparameter *step-duration* 60)

(defun step-time (step)
  (+ *step-duration* (1+ (- (char-code step) (char-code #\A)))))
Final plumbing:
(define-solution (2018 7) (instructions parse-instructions)
  (values (part1 instructions) (part2 instructions)))

(define-test (2018 7) ("GRTAHKLQVYWXMUBCZPIJFEDNSO" 1115))
And that's it:
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.005 seconds of real time
  0.005701 seconds of total run time (0.004362 user, 0.001339 system)
  120.00% CPU
  13,260,235 processor cycles
  2,686,944 bytes consed

2021-11-22 (permalink)

Advent of Code: [2018/12](https://adventofcode.com/2018/day/12)

Today we are given a bunch of pots, each containing (or not) plants; and some notes, describing how these pots change over time, based on the state of the adjacent pots:

The pots are numbered, with 0 in front of you. To the left, the pots are numbered -1, -2, -3, and so on; to the right, 1, 2, 3.... Your puzzle input contains a list of pots from 0 to the right and whether they do (#) or do not (.) currently contain a plant, the initial state. (No other pots currently contain plants.) For example, an initial state of #..##.... indicates that pots 0, 3, and 4 currently contain plants.


Your puzzle input also contains some notes you find on a nearby table: someone has been trying to figure out how these plants spread to nearby pots. Based on the notes, for each generation of plants, a given pot has or does not have a plant based on whether that pot (and the two pots on either side of it) had a plant in the last generation. These are written as LLCRR => N, where L are pots to the left, C is the current pot being considered, R are the pots to the right, and N is whether the current pot will have a plant in the next generation. For example:


- A note like ..#.. => . means that a pot that contains a plant but with no plants within two pots of it will not have a plant in it during the next generation.

- A note like ##.## => . means that an empty pot with two plants on each side of it will remain empty in the next generation.

- A note like .##.# => # means that a pot has a plant in a given generation if, in the previous generation, there were plants in that pot, the one immediately to the left, and the one two pots to the right, but not in the ones immediately to the right and two to the left.

Here is how our input is going to look like:
initial state: #..#.#..##......###...###

...## => #
..#.. => #
.#... => #
.#.#. => #
.#.## => #
.##.. => #
.#### => #
#.#.# => #
#.### => #
##.#. => #
##.## => #
###.. => #
###.# => #
####. => #
Anyways, as usual, let's get this input parsed; we are going to parse this into a list of 2 elements: in the first one, we are going to store the initial state of the pots; in the second, we are going to store all the notes describing how pots change over time.  I am going to be using a HASH-SET for both the initial state of the pots, as well as the set of rules; for the initial state, I am going to simply keep track of the positions of the pots that contain a plant; for the list of notes instead, I am only going to be storing the ones causing a plant to grow, or to stay unchanged (i.e. I am going to skip all the notes with a `.` on the left-hand side of the note...more to this later):
(defun parse-input (data)
  (list (parse-pots (first data)) (parse-notes (cddr data))))

(defun parse-pots (string)
  (let ((pots (make-hset nil)))
    (loop :for i :from 0
          :for ch :across (subseq string 15)
          :when (eq ch #\#) :do (hset-add i pots))

(defun parse-notes (data)
  (let ((notes (make-hset nil :test 'equal)))
    (loop :for string :in data
          :for from = (subseq string 0 5) :for to = (char string 9)
          :when (eq to #\#) :do (hset-add from notes))

(defun pots (input) (first input))
(defun notes (input) (second input))
Now, for part 1, we are asked to evolve our array of pots over time, and see how things look after 20 generations:

After one generation, only seven plants remain. The one in pot 0 matched the rule looking for ..#.., the one in pot 4 matched the rule looking for .#.#., pot 9 matched .##.., and so on.


In this example, after 20 generations, the pots shown as # contain plants, the furthest left of which is pot -2, and the furthest right of which is pot 34. Adding up all the numbers of plant-containing pots after the 20th generation produces 325.


After 20 generations, what is the sum of the numbers of all pots which contain a plant?

As with most of the Game of Life problems, we are going to _evolve_ our state a fixed number of times (i.e. `iterations`), and then inspect the final state to come up withe the challenge's answer:

- We always start anew inside POTS-NEXT, as if all the pots were empty; this is the reason why we don't care about the notes resulting in the central pot to become empty, but only the ones resulting in pots with a plant in it
- As we keep track of plants using a HASH-SET, and not an array, we need to find the min / max positions first, and then iterate for all the values in between
- The `(- ... 4)` and `(+ ... 4)` is to make sure we catch changes happening on the _edge_
- SURROUNDING-POTS returns the state of the pots around a certain given position (this mostly makes up for the fact that we are using a HASH-SET instead of an array)
- SUM-PLANT-POSITIONS simply sums the positions of the pots with a plant (i.e. the answer to our problem)
(defun part1 (input iterations)
  (destructuring-bind (pots notes) input
    (dotimes (i iterations)
      (setf pots (pots-next pots notes)))
    (sum-plant-positions pots)))

(defun pots-next (pots notes)
  (let ((next (make-hash-table))
        (min (find-min (hset-values pots)))
        (max (find-max (hset-values pots))))
    (loop :for i :from (- min 4) :to (+ max 4)
          :for key = (surrounding-pots pots i)
          :when (hset-contains-p key notes) :do (hset-add i next))

(defun surrounding-pots (pots i)
  (let ((string (make-string 5 :initial-element #\.)))
    (loop :for j :from (- i 2) :to (+ i 2)
          :for k :from 0
          :when (hset-contains-p j pots)
          :do (setf (char string k) #\#))

(defun sum-plant-positions (pots)
  (loop :for i :being :the :hash-keys :of pots :sum i))
Let's now take a look at part 2:

After fifty billion (50000000000) generations, what is the sum of the numbers of all pots which contain a plant?

What?!  There has to be a loop involved...

I fed this to my FLOYD loop detector utility function, but unfortunately that kept on spinning forever:
(let ((input (parse-input (uiop:read-file-lines "src/2018/day12.txt"))))
  (flet ((next (pots)
           (pots-next pots (notes input))))
    (floyd #'next (pots input))
    (error "Never gonna give you up!")))
Well, as it turns out, after a few generations the array of pots assume a certain configuration; a configuration which keeps on sliding to the right hand side, over and over again:
120: #.##...#.##.##.##.##.##.##...
121: .#.##...#.##.##.##.##.##.##..
122: ..#.##...#.##.##.##.##.##.##.
124: ...#.##...#.##.##.##.##.##.##
So how can we detect that?  Well, we could change SUM-PLANT-POSITIONS to count _relatively_ to the minimum position of a pot with a plant, instead of 0; and then tell FLOYD to use that to detect a loop:
(defun sum-plant-positions (pots &optional (center 0))
  (loop :for i :being :the :hash-keys :of pots :sum (- i center)))

> (let ((input (parse-input (uiop:read-file-lines "src/2018/day12.txt"))))
    (flet ((next (pots)
             (pots-next pots (notes input)))
           (key (pots)
             (let ((min (find-min (hset-values pots))))
               (loop :for i :being :the :hash-keys :of pots :sum (- i min)))))
        (floyd #'next (pots input) :key #'key)
        (list 'LOOP 'DETECTED)))
Once we found our loop, all is left to do is figure out the number of cycles left to be done, how much SUM-PLANT-POSITIONS increases at each cycle, and then do some math to put together our solution:
(defun part2 (input iterations)
  (flet ((next (pots)
           (pots-next pots (notes input)))
         (key (pots)
           (let ((min (find-min (hset-values pots))))
             (sum-plant-positions pots min))))
    (destructuring-bind (cycles-at cycle-size pots-new)
        (floyd #'next (pots input) :key #'key)
      (let* ((base (sum-plant-positions pots-new))
             (pots-new-next (pots-next pots-new (notes input)))
             (per-cycle-increment (- (sum-plant-positions pots-new-next) base))
             (cycles (- iterations cycles-at)))
        (assert (= cycle-size 1))
        (+ base (* per-cycle-increment cycles))))))
Final plumbing:
(define-solution (2018 12) (input parse-input)
  (values (part1 input 20) (part2 input 50000000000)))
And that's it:
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.060 seconds of real time
  0.108866 seconds of total run time (0.107327 user, 0.001539 system)
  181.67% CPU
  139,759,679 processor cycles
  33,689,536 bytes consed

2021-11-19 (permalink)

Advent of Code: [2018/18](https://adventofcode.com/2018/day/18)

Game of life kind of problem today:

On the outskirts of the North Pole base construction project, many Elves are collecting lumber.


The lumber collection area is 50 acres by 50 acres; each acre can be either open ground (.), trees (|), or a lumberyard (#). You take a scan of the area (your puzzle input).


Strange magic is at work here: each minute, the landscape looks entirely different. In exactly one minute, an open acre can fill with trees, a wooded acre can be converted to a lumberyard, or a lumberyard can be cleared to open ground (the lumber having been sent to other projects).

This is how our input is going to look like:
The only difference is that our input is going to be bigger, i.e. 50x50 instead of 10x10; let's parse this into 2D array:
(defun parse-area (data)
  (let ((rows (length data))
        (cols (length (first data))))
    (make-array (list rows cols) :initial-contents data)))
Before taking a look at the rules governing how the area updates, minute after minute, let's see what exactly are we asked to do:

After 10 minutes, there are 37 wooded acres and 31 lumberyards. Multiplying the number of wooded

acres by the number of lumberyards gives the total resource value after ten minutes: 37 * 31 =



What will the total resource value of the lumber collection area be after 10 minutes?

So we need to:

- Figure out how the area is going to look like after `10` minutes
- Count the number of acres with trees
- Count the number of acres with lumber
- Multiply the two together
(defun part1 (area iterations)
  (dotimes (i iterations)
    (setf area (area-tick area)))
  (resource-value area))
To see how a given area is going to look like, after a minute we will have to:

- For each row
- For each column in the current row
- Inspect the surrounding 8 acres and count which ones are open, which ones have trees, and which ones have lumber
- Update the state of the current acre accordingly

Note: all the updates happen at the same time, so we need to make sure the original area is not updated as we process it.

As for the update rules instead:

- An open acre will become filled with trees if three or more adjacent acres contained trees. Otherwise, nothing happens.

- An acre filled with trees will become a lumberyard if three or more adjacent acres were lumberyards. Otherwise, nothing happens.

- An acre containing a lumberyard will remain a lumberyard if it was adjacent to at least one other lumberyard and at least one acre containing trees. Otherwise, it becomes open.

All of the above, i.e. rules plus algorithm, nicely translates into the following, long-winded, function:
(defun area-next (area)
  (let* ((rows (array-dimension area 0))
         (cols (array-dimension area 1))
         (next (make-array (list rows cols))))
    (loop :for r :below rows :do
          (loop for c :below cols :do
                (loop :with open = 0 :and trees = 0 :and lumby = 0
                      :for rr :from (max (1- r) 0) :upto (min (1+ r) (1- rows)) :do
                      (loop :for cc :from (max (1- c) 0) :upto (min (1+ c) (1- cols))
                            :unless (and (= rr r) (= cc c)) :do
                            (ecase (aref area rr cc)
                              (#\. (incf open))
                              (#\| (incf trees))
                              (#\# (incf lumby))))
                      :finally (setf (aref next r c)
                                     (ecase (aref area r c)
                                       (#\. (if (>= trees 3) #\| #\.))
                                       (#\| (if (>= lumby 3) #\# #\|))
                                       (#\# (if (and (>= lumby 1) (>= trees 1))
                                              #\# #\.)))))))
At this point, all is left to do is calculating the resource value of a given area:
(defun resource-value (area)
  (loop :for i :below (array-total-size area) :for ch = (row-major-aref area i)
        :count (eq ch #\|) :into trees
        :count (eq ch #\#) :into lumby
        :finally (return (* trees lumby))))
Now, off to part 2:

What will the total resource value of the lumber collection area be after 1000000000 minutes?


Well, it turns out this _game_ is cyclic, i.e. after some time a given area configuration will keep on re-appearing over and over again; this means all we have to do is:

- Figure out when it starts cycling
- How big the cycle is
- How many iterations are left to be done if we remove the cycles

Easy peasy, especially if you already bumped into a similar problem before, and had an implementation for the [Floyd-Marshall](https://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm) algorithm handy:
(defun part2 (area)
  (destructuring-bind (cycles-at cycle-size area-new)
      (floyd #'area-next area :test 'equalp)
    (let ((remaining (nth-value 1 (floor (- 1000000000 cycles-at) cycle-size))))
      (part1 area-new remaining))))
Final plumbing:
(define-solution (2018 18) (area parse-area)
  (values (part1 area 10) (part2 area)))

(define-test (2018 18) (549936 206304))
And that's it:
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.877 seconds of real time
  0.826163 seconds of total run time (0.811340 user, 0.014823 system)
  94.18% CPU
  2,018,921,962 processor cycles
  47,953,424 bytes consed

2021-11-18 (permalink)

Advent of Code: [2018/23](https://adventofcode.com/2018/day/23)

This problem's input is a list of _nanobots_, each representing a 3-d location, and a radius (more to this later):
pos=<0,0,0>, r=4
pos=<1,0,0>, r=1
pos=<4,0,0>, r=3
pos=<0,2,0>, r=1
pos=<0,5,0>, r=3
pos=<0,0,3>, r=1
pos=<1,1,1>, r=1
pos=<1,1,2>, r=1
pos=<1,3,1>, r=1
We are going to put this into a structure with two slots:

- The 3-d location, stored as a list
- The communication radius
(defstruct (nanobot (:type list)
  pos r)
(defun x (pos) (first pos))
(defun y (pos) (second pos))
(defun z (pos) (third pos))

(defun parse-nanobots (data)
  (mapcar #'parse-nanobot data))

(defun parse-nanobot (string)
  (cl-ppcre:register-groups-bind ((#'parse-integer x y z r))
      ("pos=<(-?\\d+),(-?\\d+),(-?\\d+)>, r=(\\d+)" string)
    (make-nanobot :pos (list x y z) :r r)))
Now, let's take a look at the ask for part 1:

Find the nanobot with the largest signal radius. How many nanobots are in range of its signals?

OK, easy enough:
(defun part1 (bots)
  (let ((strongest (find-max bots :key #'r)))
    (count-if (partial-1 #'nanobot-contains-p strongest (pos _)) bots)))

(defun nanobot-contains-p (nb pos)
  (<= (manhattan-distance (pos nb) pos) (r nb)))
Now, things are going to get messy with part 2:

Find the coordinates that are in range of the largest number of nanobots. What is the shortest manhattan distance between any of those points and 0,0,0?

OK, let's figure out what the bounding box of all the bots is, and then look for the point with the highest number of bots in range; except, the search space might be bigger that we thought:
> (let ((bots (parse-nanobots (uiop:read-file-lines "src/2018/day23.txt"))))
    (loop :for bot :in bots :for pos = (pos bot)
          :minimize (x pos) :into x-min
          :maximize (x pos) :into x-max
          :minimize (y pos) :into y-min
          :maximize (y pos) :into y-max
          :minimize (z pos) :into z-min
          :maximize (z pos) :into z-max
          :finally (return (list x-min x-max y-min y-max z-min z-max))))
(-151734092 226831435 -103272011 144820743 -114031252 186064394)
That, and the fact that there will be a lot of overlapping (not only are bots well spread out into the 3d space, but they also have huge communication radius too):
> (let ((bots (parse-nanobots (uiop:read-file-lines "src/2018/day23.txt"))))
    (loop :for bot :in bots
          :minimize (r bot) :into r-min
          :maximize (r bot) :into r-max
          :finally (return (list r-min r-max))))
(49770806 99277279)
We are going to need something clever than this, but what exactly?  Well, I originally solved this by [random walking the search space and drilling into the areas with the highest number of bots in range](https://github.com/iamFIREcracker/adventofcode/commit/75abd836d5502df6307c5c77ccc24b7d91d679e4#diff-e25508cbdc3137033eb7cc9d1a6d6c5f0a278b331eca80162a8defb74d5379a7L1255); however, that was not always guaranteed to work (yes, I got lucky for my second star), and after taking a good look at [r/adventofcode](https://reddit.com/r/adventofcode) and even reading the AoC creator [commenting](https://www.reddit.com/r/adventofcode/comments/aa9uvg/day_23_aoc_creators_logic/) on what he thought people should be implementing to solve this, I decided to give the "Octree scan" solution a try.

The basic idea is simple, and it relies on the recursive 3-d space subdivision logic that [octrees](https://en.wikipedia.org/wiki/Octree) implement when efficiently organizing 3-d objects in the space; what follows is an overview of the algorithm that we are going to implement to solve this:

- We start with the biggest bounding box possible, and we count the number of bots in range (all)
- Then we split this box in 8 equal parts, and count the number of bots in range of each
- We keep on doing this until the cubes becomes 3-d points
- We are going to need a way to prune non optimal branches as soon as possible, or chances are we are going to get stuck in this loop for quite some time (more to this later)

Let's start off by defining a BOX structure and create one instance covering for all the bots in our input:
(defstruct (box (:type list)
                (:constructor make-box%))
  range-x range-y range-z)

(defun make-box (x-min x-max y-min y-max z-min z-max)
  (let* ((box (make-box% :range-x (list x-min x-max)
                         :range-y (list y-min y-max)
                         :range-z (list z-min z-max)))
         (volume (box-volume box)))
    (assert (or (= volume 0) (= (log volume 2) (floor (log volume 2)))))

(defun bots-box (bots)
  (loop :for bot :in bots :for pos = (pos bot)
        :minimize (x pos) :into x-min
        :maximize (x pos) :into x-max
        :minimize (y pos) :into y-min
        :maximize (y pos) :into y-max
        :minimize (z pos) :into z-min
        :maximize (z pos) :into z-max
        :finally (return (let* ((largest (max (- x-max x-min)
                                              (- y-max y-min)
                                              (- z-max z-min)))
                                (side (make-pow2 largest)))
                           (make-box x-min (+ x-min side)
                                     y-min (+ y-min side)
                                     z-min (+ z-min side))))))

(defun make-pow2 (number)
  (expt 2 (ceiling (log number 2))))
(Note: we decide to make sure the box is actually a cube, and that each side is a power of 2, so we know the splitting will always be _even_)

Now, while waking through the search space, we would like to pick our next cube to analyze based on:

- Number of nanobots in range of the box: the higher, the better, as that's what we want to maximize, eventually
- Volume of the current box: the smaller, the better, as we want to converge to a solution (optimal or not) as soon as possible so we can use that to prune branching
- Distance from origin: again, the smaller, the better

We are going to create a new structure, for this, STATE, and customize its constructor as follows:
(defstruct (state (:type list)
                  (:constructor make-state%)
  box num-nanobots volume distance)

(defun make-state (bots box)
  (let ((x (caar box))
        (y (caadr box))
        (z (caaddr box)))
    (make-state% :box box
                 :num-nanobots (count-in-range bots box)
                 :volume (box-volume box)
                 :distance (manhattan-distance (list x y z) (list 0 0 0)))))
There are a lot of helpers function used here, so let's take a look at their implementations; COUNT-IN-RANGE first:
(defun count-in-range (nanobots box)
  (count-if (partial-1 #'nanobot-overlaps-p _ box) nanobots))
It relies on another helper function, NANOBOT-OVERLAPS-P, which all it does is calculating how far the nanobot is, from the box, and return T if that distance (i.e. it's a Manhattan distance) is smaller or equal than the nanobot radius itself:
(defun nanobot-overlaps-p (nb box)
  (loop :with distance = 0
        :for (v-min v-max) :in box
        :for v :in (pos nb)
        :if (< v v-min) :do (incf distance (- v-min v))
        :else if (> v v-max) :do (incf distance (- v v-max))
        :never (> distance (r nb))))
Lastly, calculating one box's volume should be pretty straightforward:
(defun box-volume (box)
  (destructuring-bind ((x-min x-max) (y-min y-max) (z-min z-max)) box
    (* (- x-max x-min) (- y-max y-min) (- z-max z-min))))
Let's now take a look at the meat of this whole thing:

- We create the bounding box
- Initialize a sorted queue with it
- We start popping items from it until it's empty
- Now, until we have found _one_ solution (it could be the optimal one, or a sub-optimal one), or so long as the current box has a chance of containing a better solution (i.e. by having more nanobots than our current best solution)
- We split the box in 8, add those cubes to the priority queue, rinse and repeat
- Now, if we cannot split the cube further (i.e. it's a 3-d point), then we check for the number of nanobots covering this location, and if better than our best solution so far, we update that and carry on
(defun part2 (bots)
  (let ((box (bots-box bots))
    (loop :with queue = (make-hq :predicate #'state-better-p)
          :initially (hq-insert queue (make-state bots box))
          :until (hq-empty-p queue)
          :for state = (hq-pop queue)
          :do (when (or (not best) (>= (num-nanobots state) (num-nanobots best)))
                (if (box-point-p (box state))
                  (if (or (not best) (state-better-p state best))
                    (setf best state))
                  (dolist (subbox (box-subdivisions (box state)))
                    (hq-insert queue (make-state bots subbox)))))
          :finally (return (distance best)))))
Easy uh?! Yeah...

Let's take a look at those functions that we have never seen before; let's begin with STATE-BETTER-P, and as a reminder, this is the logic that we are going to try to implement:

- Number of nanobots in range of the box: the higher, the better, as that's what we want to maximize, eventually
- Volume of the current box: the smaller, the better, as we want to converge to a solution (optimal or not) as soon as possible so we can use that to prune branching
- Distance from origin: again, the smaller, the better
(defun state-better-p (state other)
  (destructuring-bind (num-nanobots volume distance) (rest state)
    (destructuring-bind (o-num-nanobots o-volume o-distance) (rest other)
      (or (> num-nanobots o-num-nanobots)
          (and (= num-nanobots o-num-nanobots)
               (or (< volume o-volume)
                   (and (= volume o-volume)
                        (< distance o-distance))))))))
BOX-POINT-P instead, checks if the current box is actually a point (i.e. each range is `0` units wide):
(defun box-point-p (box)
  (loop :for (min max) :in box
        :always (= min max)))
BOX-SUBDIVISIONS finally, takes care of taking a box / cube, and generating all its 8 subdivisions; one thing to note here: we are going to check for the volume of the box first, and if it's `1`, it means we cannot split it any further, and we will go ahead and return 8 new boxes, each covering for one vertex of the cube; otherwise, we will find the mid point of each range, and create sub-cubes accordingly:
(defun box-subdivisions (box)
  (unless (box-point-p box)
    (if (= (box-volume box) 1)
      (loop :for (x y z) :in (box-vertices box)
            :collect (make-box x x y y z z))
      (destructuring-bind ((x-min x-max) (y-min y-max) (z-min z-max)) box
        (let ((x-mid (+ x-min (floor (- x-max x-min) 2)))
              (y-mid (+ y-min (floor (- y-max y-min) 2)))
              (z-mid (+ z-min (floor (- z-max z-min) 2))))
          (loop :for (x-min x-max) :on (list x-min x-mid x-max)
                :when x-max :append
                (loop :for (y-min y-max) :on (list y-min y-mid y-max)
                      :when y-max :append
                      (loop :for (z-min z-max) :on (list z-min z-mid z-max)
                            :when z-max :collect (make-box x-min x-max
                                                           y-min y-max
                                                           z-min z-max)))))))))

(defun box-vertices (box)
  (destructuring-bind (x-range y-range z-range) box
    (loop :for x :in x-range :append
          (loop :for y :in y-range :append
                (loop :for z :in z-range :collect (list x y z))))))
And that should be pretty much it.

Final plumbing:
(define-solution (2018 23) (bots parse-nanobots)
  (values (part1 bots) (part2 bots)))

(define-test (2018 23) (433 107272899))
Et voila'!
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.132 seconds of real time
  0.115931 seconds of total run time (0.105885 user, 0.010046 system)
  87.88% CPU
  305,851,279 processor cycles
  1,768,384 bytes consed

2021-11-12 (permalink)

To get started with Sketch (the Common Lisp graphic framework, inspired by the Processing language), you will have to install SDL2 and some of its add-ons.

On a Mac, the most convenient way to do this is by using Homebrew:
brew install sdl2
brew install sdl2_image
brew install sdl2_ttf
With that done, you should now be able to load Sketch's examples just fine:
(ql:quickload :sketch)
(ql:quickload :sketch-examples)
(make-instance 'sketch-examples:brownian)
Note: as mentioned on this issue, [OSX Can't run hello world example #29](https://github.com/vydd/sketch/issues/29), when trying this out on SBCL, you will have to wrap the MAKE-INSTANCE call inside SDL2:MAKE-THIS-THREAD-MAIN, or most likely you will be presented with the following:
debugger invoked on a SDL2::SDL-ERROR in thread
#<THREAD "SDL2 Main Thread" RUNNING {10056843D3}>:
  SDL Error: NSWindow drag regions should only be invalidated on the Main Thread!

The current thread is not at the foreground,
SB-THREAD:RELEASE-FOREGROUND has to be called in #<SB-THREAD:THREAD "main thread" waiting on: #<WAITQUEUE Anonymous condition variable {10056847B3}> {1007C501B3}>
for this thread to enter the debugger.

debugger invoked on a SB-SYS:INTERACTIVE-INTERRUPT @7FFF5DB4C58E in thread
#<THREAD "main thread" RUNNING {1007C501B3}>:
  Interactive interrupt at #x7FFF5DB4C58E.
restarts (invokable by number or by possibly-abbreviated name):
  0: [CONTINUE] Return from SB-UNIX:SIGINT.
  1: [ABORT   ] Exit debugger, returning to top level.
Alternatively, you can always switch to [CCL](https://ccl.clozure.com/) (another Common Lisp implementation) when playing with Sketch, as that one does not seem to be suffering from the same thread-related issue that SBCL does instead.

2021-11-10 (permalink)

Advent of Code: [2018/20](https://adventofcode.com/2018/day/20)

We find ourselves into an area made up entirely of rooms and doors; the thing is, the Elves don't have a map for this area; instead, they hand us over a list of instructions we can use to build the map ourselves:

- `^` is the "start-of-instructions" marker
- `$` is the "end-of-instructions" marker
- `N` means you can go north (i.e. a door exists between the _current_ room, and the one _north_ to it)
- `E` means you can go east
- `S` means you can go south
- `W` means you can go west
- `(` marks a _fork_, meaning that multiple sub-paths can be started from the current room
- `|` marks the end of a sub-path
- `)` marks the end of the fork

A couple of examples will hopefully help understanding this; this simple list of instructions, `^WNE$`, can be used to build up the following map:
This longer instructions list instead, ` ^ENWWW(NEEE|SSE(EE|N))$`, can be used to create the following map:
Before figuring out how to create the map from the instructions, it's probably best to take a look at what we are asked to do for part 1, so here it goes:

What is the largest number of doors you would be required to pass through to reach a room? That is, find the room for which the shortest path from your starting location to that room would require passing through the most doors; what is the fewest doors you can pass through to reach it?

OK, it seems like we are going to have to actually navigate through this map.  Let's parse the list of instructions into a HASH-SET of locations (i.e. COMPLEX numbers), referring to where the doors are; to do so, we are going to use a _stack_ of current positions, so we can keep track of every fork we step foot on; in particular:

- We skip the `^`
- If the current char is `$`, we stop and return the set of doors we went through
- If the current char is `(`, a new fork begins, so push the current position into the stack
- If the current char is `|`, a sub-path just ended, so let's go back to the room of the most recent fork
- If the current char is `)`, we visited all the sub-paths of a fork, so pop an item from the stack, and start from there
- Otherwise, i.e. one of `NESW`, move into the adjacent room and record the door we used to get there
(defun parse-doors (data &aux (string (first data)) (doors (make-hset nil)))
  (loop :with pos = 0 :with pos-stack = (list pos)
        :for ch :across string
        :do (ecase ch
              ((#\^) nil)
              ((#\$) (return doors))
              ((#\() (push pos pos-stack))
              ((#\)) (setf pos (pop pos-stack)))
              ((#\|) (setf pos (first pos-stack)))
              ((#\N) (hset-add (+ pos (complex 0 1)) doors) (setf pos (+ pos (complex 0 2))))
              ((#\E) (hset-add (+ pos (complex 1 0)) doors) (setf pos (+ pos (complex 2 0))))
              ((#\S) (hset-add (- pos (complex 0 1)) doors) (setf pos (- pos (complex 0 2))))
              ((#\W) (hset-add (- pos (complex 1 0)) doors) (setf pos (- pos (complex 2 0))))))
All is left to do now, is to explore the complete map, figure out which room is the furthest away from us, in terms of number of doors to go through, and then return that number of doors; we can do this with an unbounded BFS:

- start point: `0,0` (or `(complex 0 0)`)
- no goal state / function -- it will stop when there are no more states to process
- from a given location, you can move into the adjacent room if a door exists in between -- see NEIGHBORS
(defparameter *deltas* '(#C(0 1) #C(1 0) #C(0 -1) #C(-1 0)))

(defun neighbors (doors pos)
  (loop :for d :in *deltas*
        :for door-pos = (+ pos d)
        :when (hset-contains-p door-pos doors)
        :collect (+ pos d d)))

(let ((cost-so-far (nth-value 3 (bfs 0 :neighbors (partial-1 #'neighbors doors)))))
  (reduce #'max (hash-table-values cost-so-far)))
For part 2, instead:

Okay, so the facility is big.


How many rooms have a shortest path from your current location that pass through at least 1000 doors?

(let ((cost-so-far (nth-value 3 (bfs 0 :neighbors (partial-1 #'neighbors doors)))))
  (count-if (partial-1 #'>= _ 1000) (hash-table-values cost-so-far)))
Let's wire everything together:
(define-solution (2018 20) (doors parse-doors)
  (let ((cost-so-far (nth-value 3 (bfs 0 :neighbors (partial-1 #'neighbors doors)))))
      (reduce #'max (hash-table-values cost-so-far))
      (count-if (partial-1 #'>= _ 1000) (hash-table-values cost-so-far)))))

(define-test (2018 20) (3835 8520))
And that's it:
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.029 seconds of real time
  0.028120 seconds of total run time (0.018474 user, 0.009646 system)
  96.55% CPU
  67,925,979 processor cycles
  6,870,096 bytes consed

2021-11-09 (permalink)

Advent of Code: [2018/06](https://adventofcode.com/2018/day/6)

Today, we are given a bunch of coordinates (i.e. X,Y locations), representing what could be either be dangerous safe places to land on -- and yes, we are falling.

This is a sample input:
1, 1
1, 6
8, 3
3, 4
5, 5
8, 9
We are going to be parsing this into _tagged_ coordinates, i.e. coordinates with a tag information attached to it (more to this later); as usual, we would be representing coordinates as COMPLEX numbers:
(defvar *tag* 0)

(defun parse-points (data)
  (let ((*tag* 0))
    (mapcar #'parse-point data)))

(defun parse-point (string)
  (cl-ppcre:register-groups-bind ((#'parse-integer col row))
      ("(\\d+), (\\d+)" string)
    (make-point (incf *tag*) row col)))

(defun make-point (tag row col) (cons tag (make-coord row col)))
(defun tag (coord) (car coord))
(defun (setf tag) (value coord) (setf (car coord) value))
(defun coord (point) (cdr point))

(defun make-coord (row col) (complex col (- row)))
(defun row (coord) (- (imagpart coord)))
(defun col (coord) (realpart coord))
For part 1, we are going to assume these to be _dangerous_ coordinates, and we are going to try and find which one is the furthest away (in terms of Manhattan distance) from all of the others.  How?

Using only the Manhattan distance, determine the area around each coordinate by counting the number of integer X,Y locations that are closest to that coordinate (and aren't tied in distance to any other coordinate).


Your goal is to find the size of the largest area that isn't infinite.

Here is how we are going to solve this:

- We find the bounding box, i.e. the smallest rectangle containing all the coordinates of the input -- see BOUNDING-BOX
- For each X,Y point of this bounding box
- We find the input coordinate which is _closest_ to that, being careful to ignore any point which is equally distant to 2 or more coordinates -- see CLOSEST-POINT
- Then if the we are _not_ on the border (see ON-THE-BORDER-P), we mark the location with the tag of the closest input coordinate (we are not actually marking it, but rather counting, but you get the idea)
- Finally, we find the input coordinate with the highest number of _closest_ locations

In code:
(defun part1 (points &aux (bbox (bounding-box points)))
  (destructuring-bind (row-min col-min row-max col-max) bbox
    (let ((sizes (make-hash-table)))
      (loop :for row :from row-min :to row-max :do
            (loop :for col :from col-min :to col-max
                  :for test := (make-coord row col)
                  :for closest := (closest-point test points)
                  :do (if (on-the-border-p test bbox)
                        (setf (gethash (tag closest) sizes) most-negative-fixnum)
                        (incf (gethash (tag closest) sizes 0)))))
      (loop :for size :being :the :hash-values :of sizes :using (hash-key tag)
            :when tag :maximize size))))

(defun bounding-box (points)
  (loop :for p :in points
        :for row := (row (coord p)) :for col := (col (coord p))
        :minimizing row :into row-min
        :maximizing row :into row-max
        :minimizing col :into col-min
        :maximizing col :into col-max
        :finally (return (list row-min col-min row-max col-max))))

(defun closest-point (test points)
  (let (closest closest-distance)
    (dolist (p points)
      (let ((distance (manhattan-distance test (coord p))))
        (when (or (not closest-distance) (<= distance closest-distance))
          (if (or (not closest-distance) (< distance closest-distance))
            (setf closest (list p) closest-distance distance)
            (push p closest)))))
    (when (= (length closest) 1)
      (first closest))))

(defun on-the-border-p (point bbox)
  (destructuring-bind (row-min col-min row-max col-max) bbox
    (or (= (row point) row-min) (= (row point) row-max)
        (= (col point) col-min) (= (col point) col-max))))
For part 2, we are going to assume these input locations to be _safe_ to land on instead, so we are going to try and find the region near as many coordinates as possible:

For example, suppose you want the sum of the Manhattan distance to all of the coordinates to be less than 32. For each location, add up the distances to all of the given coordinates; if the total of those distances is less than 32, that location is within the desired region.


What is the size of the region containing all locations which have a total distance to all given coordinates of less than 10000?

- For each X,Y location inside the same bounding box as part 1
- Sum all the distances with respect of all the input coordinates -- see TOTAL-DISTANCE
- Increment a counter if this sum is less than 10000
(defun part2 (points &aux (bbox (bounding-box points)))
  (destructuring-bind (row-min col-min row-max col-max) bbox
    (loop :for row :from row-min :to row-max :summing
          (loop :for col :from col-min :to col-max
                :for test := (make-coord row col)
                :for total-distance := (total-distance test points)
                :counting (< total-distance 10000)))))

(defun total-distance (test points)
  (loop :for p :in points
        :summing (manhattan-distance test (coord p))))
Final plumbing:
(define-solution (2018 6) (points parse-points)
  (values (part1 points) (part2 points)))

(define-test (2018 6) (3894 39398))
And that's it:
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.661 seconds of real time
  0.618545 seconds of total run time (0.585166 user, 0.033379 system)
  93.65% CPU
  1,521,383,260 processor cycles
  14,120,976 bytes consed

2021-11-08 (permalink)

Advent of Code: [2018/04](https://adventofcode.com/2018/day/4)

December 1st is right around the corner, and we better get in "advent-of-code"-shape real quick; and what better way to do that than re-working some of my oldest solutions, i.e. the ones from 2018, when I was just trying to get started with Common Lisp?!  2018, day 4 it is!

For today's challenge we are tasked to sneak inside a supply closet so we can fix our suit; however, the closet is protected by a guard, so we need to figure out our way in without getting caught.  Lucky us, someone has been keeping tabs on guards shift (it's sleeping schedule, our input), and by the look of it it appears guards would fall asleep quite often, so hopefully we can figure out the best time to get in.

This is the kind of input that we are given:
[1518-11-01 00:00] Guard #10 begins shift
[1518-11-01 00:05] falls asleep
[1518-11-01 00:25] wakes up
[1518-11-01 00:30] falls asleep
[1518-11-01 00:55] wakes up
[1518-11-01 23:58] Guard #99 begins shift
[1518-11-02 00:40] falls asleep
[1518-11-02 00:50] wakes up
[1518-11-03 00:05] Guard #10 begins shift
[1518-11-03 00:24] falls asleep
[1518-11-03 00:29] wakes up
[1518-11-04 00:02] Guard #99 begins shift
[1518-11-04 00:36] falls asleep
[1518-11-04 00:46] wakes up
[1518-11-05 00:03] Guard #99 begins shift
[1518-11-05 00:45] falls asleep
[1518-11-05 00:55] wakes up
We are going to parse all this into an ALIST having the guard ID as index; the value would be another ALIST having the minute as index (note: for each day, we are told to analyze the `00:00`-`00:59` time range only); the value, would be the list of days in which the specific guard fell asleep for the specific minute.  The example above would be parsed into the following:
((99 (54 "1518-11-05") (53 "1518-11-05") (52 "1518-11-05") (51 "1518-11-05")
  (50 "1518-11-05") (39 "1518-11-04") (38 "1518-11-04") (37 "1518-11-04")
  (36 "1518-11-04") (49 "1518-11-05" "1518-11-02")
  (48 "1518-11-05" "1518-11-02") (47 "1518-11-05" "1518-11-02")
  (46 "1518-11-05" "1518-11-02") (45 "1518-11-05" "1518-11-04" "1518-11-02")
  (44 "1518-11-04" "1518-11-02") (43 "1518-11-04" "1518-11-02")
  (42 "1518-11-04" "1518-11-02") (41 "1518-11-04" "1518-11-02")
  (40 "1518-11-04" "1518-11-02"))
 (10 (28 "1518-11-03") (27 "1518-11-03") (26 "1518-11-03") (25 "1518-11-03")
  (54 "1518-11-01") (53 "1518-11-01") (52 "1518-11-01") (51 "1518-11-01")
  (50 "1518-11-01") (49 "1518-11-01") (48 "1518-11-01") (47 "1518-11-01")
  (46 "1518-11-01") (45 "1518-11-01") (44 "1518-11-01") (43 "1518-11-01")
  (42 "1518-11-01") (41 "1518-11-01") (40 "1518-11-01") (39 "1518-11-01")
  (38 "1518-11-01") (37 "1518-11-01") (36 "1518-11-01") (35 "1518-11-01")
  (34 "1518-11-01") (33 "1518-11-01") (32 "1518-11-01") (31 "1518-11-01")
  (30 "1518-11-01") (24 "1518-11-03" "1518-11-01") (23 "1518-11-01")
  (22 "1518-11-01") (21 "1518-11-01") (20 "1518-11-01") (19 "1518-11-01")
  (18 "1518-11-01") (17 "1518-11-01") (16 "1518-11-01") (15 "1518-11-01")
  (14 "1518-11-01") (13 "1518-11-01") (12 "1518-11-01") (11 "1518-11-01")
  (10 "1518-11-01") (9 "1518-11-01") (8 "1518-11-01") (7 "1518-11-01")
  (6 "1518-11-01") (5 "1518-11-01")))
Doing the actual parsing turned out way hairier than I originally anticipated, but whatever; anyways, high level:

- Sort entries lexicographically (yes, the input is out of order)
- Recursively
- Parse guard ID -- PARSE
- Parse "falls-asleep" time and day -- FILL-IN-SCHEDULE
- Parse "wakes-up" time and day -- FILL-IN-SCHEDULE
- For each minute in the time range
- Update the schedule accordingly -- ADD-ASLEEP-TIME-DAY
(defun parse-schedule (data)
  (let (schedule)
    (labels ((parse (remaining)
               (let ((guard (parse-schedule-guard (pop remaining))))
                 (when guard (fill-in-schedule guard remaining))))
             (fill-in-schedule (guard remaining)
               (if (not (parse-schedule-timeday (first remaining)))
                 (parse remaining)
                 (let ((falls-asleep (parse-schedule-timeday (pop remaining)))
                       (wakes-up (parse-schedule-timeday (pop remaining))))
                   (loop :with day = (cdr falls-asleep)
                         :for time :from (car falls-asleep) :below (car wakes-up)
                         :do (add-asleep-time-day guard time day))
                   (fill-in-schedule guard remaining))))
             (add-asleep-time-day (guard time day)
               (let ((entry (assoc guard schedule)))
                 (unless entry
                   (setf entry (list guard))
                   (push entry schedule))
                 (let ((st-entry (assoc time (sleeptable entry))))
                   (unless st-entry
                     (setf st-entry (list time))
                     (push st-entry (sleeptable entry)))
                   (push day (days st-entry))))))
      (parse (sort (copy-seq data) #'string<)))

(defun parse-schedule-guard (string)
  (cl-ppcre:register-groups-bind ((#'parse-integer guard))
      ("Guard #(\\d+) begins shift" string)

(defun parse-schedule-timeday (string)
  (cl-ppcre:register-groups-bind (date (#'parse-integer time))
      ("(\\d{4}-\\d{2}-\\d{2}) 00:(\\d\\d)] (falls asleep|wakes up)" string)
    (cons time date)))
The rest is a bunch of selectors for writing / pulling data into / out of the schedule:
(defun guard (schedule-entry) (car schedule-entry))
(defun sleeptable (schedule-entry) (cdr schedule-entry))
(defun (setf sleeptable) (value schedule-entry) (setf (cdr schedule-entry) value))

(defun minute (sleeptable-entry) (car sleeptable-entry))
(defun days (sleeptable-entry) (cdr sleeptable-entry))
(defun (setf days) (value sleeptable-entry) (setf (cdr sleeptable-entry) value))
Now, let's get to the juicy part:

Strategy 1: Find the guard that has the most minutes asleep. What minute does that guard spend asleep the most?

We find the schedule entry with the highest count of minutes asleep first; then, in that entry, we look for the _minute_ entry in the sleep table with the highest count of days; finally do the multiplication:
(defun part1 (schedule)
  (let ((entry (find-max schedule :key #'minutes-asleep)))
    (* (guard entry) (sleepiest-minute entry))))

(defun minutes-asleep (entry)
  (reduce #'+ (sleeptable entry)
          :key #'(lambda (e) (length (cdr e)))))

(defun sleepiest-minute (entry)
  (multiple-value-bind (st num-days-asleep)
      (find-max (sleeptable entry)
                :key #'(lambda (st)
                        (length (days st))))
    (values (minute st) num-days-asleep)))
For part 2 instead:

Strategy 2: Of all guards, which guard is most frequently asleep on the same minute?

We find the entry with the highest number of of days associated for any given minute (note: we don't want the minute but the number of days the guard feel asleep for that given minute, that's why we are using NTH-VALUE here); then we call SLEEPIEST-MINUTE on that; we do multiplication last:
(defun part2 (schedule)
  (let ((entry (find-max schedule
                         :key #'(lambda (entry)
                                 (nth-value 1 (sleepiest-minute entry))))))
    (* (guard entry) (sleepiest-minute entry))))
Final plumbing:
(defun part2 (schedule)
  (let ((entry (find-max schedule
                         :key #'(lambda (entry)
                                 (nth-value 1 (sleepiest-minute entry))))))
    (* (guard entry) (sleepiest-minute entry))))
And that's it:
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.005 seconds of real time
  0.005837 seconds of total run time (0.004618 user, 0.001219 system)
  120.00% CPU
  13,727,746 processor cycles
  720,224 bytes consed

2021-11-07 (permalink)

Abusing Common Lisp feature expressions to comment things out! (another follow up) @vim @commonlisp

The simple syntax rules that I mentioned inside one of my previous [.plan entry](https://matteolandi.net/plan.html#day-2021-10-29) turned out to break quite easily in case of nested parentheses; so I decided to try a different approach instead, and after playing with Vim's syntax regions a bit, I came up with the following:
syntax region lispExcludedFormWithNil
            \ matchgroup=lispExcludedContentStart
            \ start="#+nil ("
            \ skip="|.\{-}|"
            \ matchgroup=lispExcludedContentStop
            \ end=")"
            \ contains=@lispBaseListCluster
highlight link lispExcludedContentStart lispComment
highlight link lispExcludedContentStop lispComment
With this I was able to deal with nested parentheses just fine; however, if I there were `highlight` commands defined for any of the elements contained inside the `@lispBaseListCluster` cluster (and of course there are!), then Vim would end up coloring those elements; what I wanted instead, was the whole region to look like a `lispComment`.  This is where I got stuck; this is where I started questioning the whole approach altogether, and where I ended up asking for [help](https://www.reddit.com/r/vim/comments/qmn052/override_colors_based_on_the_surrounding_syntax/) on the /r/vim sub-reddit.

Anyways, long story short, one reply made me realize that I did not actually need all the definitions included inside the `@lispBaseListCluster` cluster, especially if then I would need to figure out a way to mute their colors; so what I did instead, was creating a new, simpler but recursive syntax region, to cater for my _excluded_ lists:
syntax region lispExcludedList
            \ contained
            \ matchgroup=lispExcludedListOpenParen
            \ start="("
            \ skip="|.\{-}|"
            \ matchgroup=lispExcludedListCloseParen
            \ end=")"
            \ contains=lispExcludedList
highlight link lispExcludedList lispComment
highlight link lispExcludedListOpenParen lispComment
highlight link lispExcludedListCloseParen lispComment
I then updated my original `lispExcludedFormWithNil` definition to _contain_ `lispExcludedList` instead of the `@lispBaseListCluster` cluster:
syntax region lispExcludedFormWithNil
            \ matchgroup=lispExcludedContentStart
            \ start="#+nil ("
            \ skip="|.\{-}|"
            \ matchgroup=lispExcludedContentStop
            \ end=")"
            \ contains=lispExcludedList
highlight link lispExcludedFormWithNil lispComment
highlight link lispExcludedContentStart lispComment
highlight link lispExcludedContentStop lispComment
And that's it; with this, it all started to work as expected:
#+nil (list 1 (list 2 3) 4)
                          `- `lispComment` syntax up until this position

#+nil (list 1 (list 2 (list 3 4) 5))
                                   `- `lispComment` syntax up until this position
If I then were to add `lispExcludedFormWithNil` to the `@lispBaseListCluster`:
syn cluster lispBaseListCluster add=lispExcludedFormWithNil
Then this would work even in case the _excluded_ expression was found nested inside another expression / list:
(list #+nil (list 1 (list 2 (list 3 4) 5)) 6)
                                         ^ ^
                                         | |
                                         | `- this is highlighed as `lispNumber`
                                         `- `lispComment` syntax up until this position

So how is my Lisp syntax file looking?  There you go:

" Excluded forms {{{

syntax region lispExcludedList
            \ contained
            \ matchgroup=lispExcludedListOpenParen
            \ start="("
            \ skip="|.\{-}|"
            \ matchgroup=lispExcludedListCloseParen
            \ end=")"
            \ contains=lispExcludedList

highlight link lispExcludedList lispComment
highlight link lispExcludedListOpenParen lispComment
highlight link lispExcludedListCloseParen lispComment
highlight link lispExcludedContentStart lispComment
highlight link lispExcludedContentStop lispComment

function! s:createLispExcludedSyntaxRegion(regionName, regionPrefix) abort "{{{
    execute 'syntax region' a:regionName
                \ 'matchgroup=lispExcludedContentStart'
                \ 'start="' . a:regionPrefix . '("'
                \ 'skip="|.\{-}|"'
                \ 'matchgroup=lispExcludedContentStop'
                \ 'end=")"'
                \ 'contains=lispExcludedList'
    execute 'highlight link' a:regionName 'lispComment'
    execute 'syntax cluster lispBaseListCluster add=' . a:regionName
endfunction " }}}

" ... with NIL {{{

syntax match lispExcludedElementWithNil /\v\#\+nil [^(`][^ ]+/
highlight link lispExcludedElementWithNil lispComment
syn cluster lispBaseListCluster add=lispExcludedElementWithNil

call s:createLispExcludedSyntaxRegion("lispExcludedFormWithNil", "#+nil ")
call s:createLispExcludedSyntaxRegion("lispExcludedQuotedFormWithNil", "#+nil '")
call s:createLispExcludedSyntaxRegion("lispExcludedQuasiQuotedFormWithNil", "#+nil `")
call s:createLispExcludedSyntaxRegion("lispExcludedSharpsignedDotFormWithNil", "#+nil #\.")

" }}}
" ... with #:_description_ {{{

syntax match lispExcludedElementWithDescription /\v\#\+\#:[^ ]+ [^(`][^ ]+/
highlight link lispExcludedElementWithDescription lispComment
syn cluster lispBaseListCluster add=lispExcludedElementWithDescription

call s:createLispExcludedSyntaxRegion("lispExcludedFormWithDescription", "#+#:[^ ]\\+ ")
call s:createLispExcludedSyntaxRegion("lispExcludedQuotedFormWithDescription", "#+#:[^ ]\\+ '")
call s:createLispExcludedSyntaxRegion("lispExcludedQuasiQuotedFormWithDescription", "#+#:[^ ]\\+ `")
call s:createLispExcludedSyntaxRegion("lispExcludedSharpsignedDotFormWithDescription", "#+#:[^ ]\\+ #\.")

" }}}

" }}}
PS. Yes, I created a script function, `createLispExcludedSyntaxRegion`, to make the process of creating new syntax regions a tiny bit less hairy.

2021-11-05 (permalink)

(Trying to) Speed up Common Lisp Replit REPLs startup time -- or how to both install Quicklisp and cache *.fasl files in the current directory @commonlisp @replit

All of my Common Lisp Replit REPLs have the following defined inside '.replit-files/init.lisp' (that's the script I am loading when starting up SBCL):
(let ((quicklisp-init (merge-pathnames "quicklisp/setup.lisp"
  (if (probe-file quicklisp-init)
      (load quicklisp-init)
        (load (merge-pathnames "quicklisp.lisp" *load-truename*))
        (funcall (find-symbol "INSTALL" (find-package "QUICKLISP-QUICKSTART"))))))
This pretty much translates to checking if '~/quicklisp/setup.lisp' already exists; if it does, it means we already got Quicklisp set up, so we go ahead and load that file up; otherwise, we load 'quicklisp.lisp' from the local directory (it's a copy of the file available on [quicklisp.org](https://www.quicklisp.org/beta/)), and then install Quicklisp altogether.

After this, with Quicklisp installed, all is left to do is loading up our app (the following is an example from a [recent REPL of mine](https://github.com/iamFIREcracker/cl-face-mask-ce-verifier/blob/master/build/replit.lisp)):
(ql:quickload '("face-mask-ce-verifier-web" "swank-tunnel"))
(fmcv-web:start :web-interface "")
Now, as you can imagine, the following two main activities will be slowing down the REPL startup time:

1. Bootstrapping Quicklisp, i.e. downloading it, installing it, fetching the latest dist
2. Loading / compiling the application system, and all its dependencies

Now, doing this once is perfectly fine, but can we configure things so that we don't have to do this over and over again, when Replit decides your REPL needs to be put to sleep or moved off to a new instance?

The answer to that question is of course: "yes!"; basically all we have to do is make sure that all the artifacts which were dynamically generated and which were needed by the REPL, are saved somewhere inside the REPL workspace (yes, your REPL workspace will pretty much always be kept around).

Alright; let's start off by changing our Quicklisp install script as follows:
;;; Quicklisp (installed in the current directory)
(let ((quicklisp-init (merge-pathnames ".quicklisp/setup.lisp" *default-pathname-defaults*)))
  (if (probe-file quicklisp-init)
      (load quicklisp-init)
        (load (merge-pathnames "quicklisp.lisp" *load-truename*))
        (funcall (find-symbol "INSTALL" (find-package "QUICKLISP-QUICKSTART"))
                 :path (merge-pathnames ".quicklisp/" *default-pathname-defaults*)))))
Here, we look for Quicklisp's setup script inside '.quicklisp/setup.lisp' instead of the canonical '~/quicklisp/setup.lisp' (note how we are using *DEFAULT-PATHNAME-DEFAULTS* instead of `(user-homedir-pathname)`); we also pass `:path (merge-pathnames ".quicklisp/" *default-pathname-defaults*)` to `QUICKLISP-QUICKSTART:INSTALL`, effectively telling it to install everything inside the local directory, '.quicklisp'.  Nice!

Now, to change the location where ADFS stores compiled files to make sure it's in the current workspace, we will have to play with ADFS's [Configuration DSL](https://common-lisp.net/project/asdf/asdf/Output-Configuration-DSL.html) a bit; it took me some Googling around ([1](https://gitlab.common-lisp.net/asdf/asdf/-/issues/22), [2](https://www.mail-archive.com/asdf-devel@common-lisp.net/msg06372.html), [3](https://stackoverflow.com/questions/25323394/asdf-output-redirection), and [4](https://www.reddit.com/r/lisp/comments/23azqf/need_help_setting_asdf_cache_output_directory/)) and quite a few tries before I got this to work; but ultimately, this is what I came up with:
;;; Locally cached FASLs
(let ((cache-dir (merge-pathnames ".common-lisp/" *default-pathname-defaults*)))
    `(:output-translations (t (,cache-dir :implementation))
Again, you can read more about the DSL in the link above, but in general:

1. Using `:implementation` forces ASDF to use separate directories, inside '.common-lisp', based on the current Lisp implementation; this way, should we change the REPL to use CCL instead of SBCL, it would compile everything from scratch, instead of trying to load the artifacts generated while we were using the other implementation
2. Using `:disable-cache` tells ASDF to stop caching compiled files under '$XDG_CONFIG_HOME/.common-lisp'; this is the behavior that the previous rule is meant to override, so disabling this will hopefully make sure the default caching mechanism won't get in our way
3. Using `:ignore-inherited-configuration` tells ASDF to ignore any other _inherited_ rule; now, I am not entirely sure what this is for, but ASDF was complaining that I should use either `:inherit-configuration` or `:ignore-inherited-configuration` when calling `INITIALIZE-OUTPUT`, so here it is

Anyways, restart the REPL, and you should now finally see the '.quicklisp' and '.common-lisp' local directories being put to good use:
~/Face-mask-CE-marking-verifier$ ls .quicklisp/
asdf.lisp     dists       quicklisp   tmp
client-info.sexp  local-projects  setup.lisp

~/Face-mask-CE-marking-verifier$ ls .common-lisp/
And that's it!  Until the next time...

2021-11-03 (permalink)

TIL: You cannot use plain HTTP with a .dev domain; Google will force a HTTPS redirect! https://ma.ttias.be/chrome-force-dev-domains-https-via-preloaded-hsts/

2021-11-02 (permalink)

* Add new page for `lg` to matteolandi.net

+ Common Lisp / Replit: if we install dependencies inside ./ instead of ~/quicklisp, would Replit cache them between restarts?

? The new version of Vlime (the one supporting JSON protocol) made the REPL editable, but it's a little bit cumbersome to interact with it -- you got to press enter while in insert mode, and if you leave insert mode early, that's it... no way to undo that

Today I woke up to realize that I could not deploy anything to my VPN anymore:
task path: /Users/matteolandi/Workspace/matteolandi.net/system/ansible/roles/matteolandi-index/tasks/main.yml:4
fatal: [matteolandi.net]: FAILED! => {"changed": false, "msg": "Error pulling matteolandi.net:5000/matteolandi-index - code: None message: unable to ping re
gistry endpoint https://matteolandi.net:5000/v0/\nv2 ping attempt failed with error: Get https://matteolandi.net:5000/v2/: x509: certificate has expired or
is not yet valid\n v1 ping attempt failed with error: Get https://matteolandi.net:5000/v1/_ping: x509: certificate has expired or is not yet valid"}
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

matteolandi.net            : ok=2    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0
Figured it had something to do with with the recent [Let's Encrypt's root certificate getting expired](https://techcrunch.com/2021/09/21/lets-encrypt-root-expiry), so I went on and upgraded my OS certificates (yeah, that plus all the other packages I had not upgraded in a while):
yum update -y
Then I tried to re-deploy again; it did not succeed, but at least the error message was different:
task path: /Users/matteolandi/Workspace/matteolandi.net/system/ansible/roles/docker-login/tasks/main.yml:1
fatal: [matteolandi.net]: FAILED! => {"changed": false, "msg": "Failed to import the required Python library (Docker SDK for Python: docker (Python >= 2.7)
or docker-py (Python 2.6)) on centos-512mb-fra1-01.localdomain's Python /usr/bin/python. Please read module documentation and install in the appropriate loc
ation. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpret
er, for example via `pip install docker` or `pip install docker-py` (Python 2.6). The error was: cannot import name certs"}
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

matteolandi.net            : ok=33   changed=1    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0
First I upgraded `pip`:
pip install --upgrade pip
Then uninstalled a bunch of borked packages:
pip uninstall docker requests urllib3
And re-installed them again:
pip install --upgrade urllib3 requests docker
And after this, it all went back to normal... Crisis averted!

2021-10-31 (permalink)

* While releasing `cg`, the created GH release is named `refs/tags/...` -- turns out I was setting the `name` input to `github.ref`: https://github.com/iamFIREcracker/cg/commit/c0c3ef5b17be40872c67709f445dcbc66c1936c2

+ migrate `ap` to GitHub actions -- see `cg`

+ migrate `adventofcode` to GitHub actions -- see `cg`

+ migrate `lg` to GitHub actions -- see `cg`

? migrate `plan-convert` to GitHub actions -- see `cg`

? migrate `xml-emitter` to GitHub actions -- see `cg`

2021-10-29 (permalink)

Abusing Common Lisp feature expressions to comment things out! (a follow up) @vim @commonlisp

I posted a link to my previous [.plan entry](https://matteolandi.net/plan.html#day-2021-10-23) on [/r/Common_Lisp](https://www.reddit.com/r/Common_Lisp/comments/qfgci7/abusing_common_lisp_feature_expressions_to/) the other day, and a couple of things came out.

First off, an evil person could add `:nil` to *FETURES* and break hell loose:

#+nil is not the best way of doing this because, in theory, an evil person can put :nil in *features* and then all hell will break lose. Use #+(or) instead, which is immune to this sort of trickery.

Uh oh...
> (or #+nil "WTF?!" "Hello, world!")
"Hello, world!"

> (pushnew :nil *features*)

> (or #+nil "WTF?!" "Hello, world!")

> (setf *features* (delete :nil *features*))
Then someone else suggested to use [unintenred symbols](http://www.lispworks.com/documentation/HyperSpec/Body/02_dhe.htm) instead, as that will not only keep you safe from the previously mentioned problem, but also give you the opportunity to _document_ why the specific form / element got intentionally excluded:

I use #+#:buggy or any other word to give a tiny piece of documentation as to why it’s commented out. Sometimes I’ll write #+#:pedagogical-implementation to provide a pedagogical, easy-to-understand version of an expression that has been made obscure through optimizations.

Again, let's try this out
> (or #+#:excluded "WTF?!" "Hello, world!")
"Hello, world!"

> (pushnew #:excluded *features*)
;     (LET* ((#:ITEM #:EXCLUDED))
; caught WARNING:
;   undefined variable: #:EXCLUDED
; compilation unit finished
;   Undefined variable:
;     #:EXCLUDED
;   caught 1 WARNING condition

debugger invoked on a UNBOUND-VARIABLE @5351833B in thread
#<THREAD "main thread" RUNNING {1001548143}>:
  The variable #:EXCLUDED is unbound.
restarts (invokable by number or by possibly-abbreviated name):
  0: [CONTINUE   ] Retry using #:EXCLUDED.
  1: [USE-VALUE  ] Use specified value.
  2: [STORE-VALUE] Set specified value and use it.
  3: [ABORT      ] Exit debugger, returning to top level.

((LAMBDA ()))
   source: (LET* ((#:ITEM #:EXCLUDED))
0] 3

> (pushnew :excluded *features*)

> (or #+#:excluded "WTF?!" "Hello, world!")
"Hello, world!"

> (setf *features* (delete :excluded *features*))
Alright then, I think I like _uninterned symbol_ approach better, so I am going to update my Vim files as follows:
diff --git a/.vim/vimrc b/.vim/vimrc
index 69c5db4..6ca6151 100644
--- a/.vim/vimrc
+++ b/.vim/vimrc
diff --git a/.vim/after/syntax/lisp.vim b/.vim/after/syntax/lisp.vim
index 5a2859c..cdf3c03 100644
--- a/.vim/after/syntax/lisp.vim
+++ b/.vim/after/syntax/lisp.vim
@@ -855,7 +921,7 @@ function! s:vim_sexp_mappings() " {{{
         " Check if it's a #+nil marker
         execute "normal \<Plug>(sexp_move_to_prev_element_head)"
         normal yW
-        if @@ =~? "#+nil"
+        if @@ =~? "#+#:"
             let should_add_comment = 0
         let @@ = reg_save
@@ -864,7 +930,7 @@ function! s:vim_sexp_mappings() " {{{
         " Do the needed
         if should_add_comment
             " Insert the comment marker, and move back to the element / form
-            execute "normal! i#+nil "
+            execute "normal! i#+#:excluded "
             execute "normal \<Plug>(sexp_move_to_next_element_head)"
             " Move back, and the delete forward
diff --git a/.vim/after/syntax/lisp.vim b/.vim/after/syntax/lisp.vim
index 5a2859c..cdf3c03 100644
--- a/.vim/after/syntax/lisp.vim
+++ b/.vim/after/syntax/lisp.vim
@@ -11,3 +11,11 @@ syn cluster lispBaseListCluster add=lispExcludedForm
 syntax match lispExcludedElement /\v\#\+nil [^(][^ ]+/
 highlight link lispExcludedElement lispComment
 syn cluster lispBaseListCluster add=lispExcludedElement
+syntax match lispExcludedFormWithDescription /\v\#\+\#:[^ ]+ \([^)]+\)/
+highlight link lispExcludedFormWithDescription lispComment
+syn cluster lispBaseListCluster add=lispExcludedFormWithDescription
+syntax match lispExcludedElementWithDescription /\v\#\+\#:[^ ]+ [^(][^ ]+/
+highlight link lispExcludedElementWithDescription lispComment
+syn cluster lispBaseListCluster add=lispExcludedElementWithDescription
Right on!

2021-10-27 (permalink)

? take a look at how mongosh implemented _awaiting_ for promises without the explicit `await` statement -- https://github.com/mongodb-js/mongosh/tree/main/packages/async-rewriter2#next-gen-async-rewriter

? CL/1AM: push tests into a custom special variable, say *MY-TESTS*, and bind 1AM:*TESTS* to *MY-TESTS* when actually running the tests (this way only the current project's tests will be executed)

2021-10-23 (permalink)

Abusing Common Lisp feature expressions to comment things out! @vim @commonlisp

Today I realized that I can easily comment out a form by placing a `#+nil` in front of it, and this turned out to be quite useful and productive especially when throwing things out at the REPL.

Let's take a look at an example (this was taken from a [recent](https://github.com/iamFIREcracker/cl-face-mask-ce-verifier) small project of mine):
(hunchentoot:define-easy-handler (index :uri "/") (q)
  (let ((bodies (and q (fmcv:nb-search q))))
    (with-page (:title "Face Mask CE Verifier")
The bit to focus your attention to is the second line, which in words, pretty much translates to:

when `q`, the request the query string is non-NIL, call FMCV:NB-SEARCH with it, and assign the result to `bodies`; otherwise, `bodies` is initialized with nil

Now, while working on this, I realized FMCV:NB-SEARCH could sometimes take up to 3 seconds to return, and since there wasn't much I could do about it (i.e. it was the external service the function was calling behind the scenes, which was taking so long to reply), I decide to make the whole development experience a tiny bit more enjoyable by introducing a little bit of caching:

- call the API once
- save the response somewhere
- change the logic to read from the cached response until I was done with everything else
(defvar *cached-bodies* (fmcv:nb-search "00"))

(hunchentoot:define-easy-handler (index :uri "/") (q)
  (let ((bodies (and q *cached-bodies*)))
    (with-page (:title "Face Mask CE Verifier")
The above just works, 100%; however, it also makes it difficult to quickly switch between implementations.  For example, should I want to hit the external API again, I would have to put the FMCV:NB-SEARCH function call back in, and that would be a lot of typing /s

This is where [feature expressions](http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec/Body/sec_24-1-2-1.html) could come to the rescue:

- Keep both *cached-bodies* and the FMCV:NB-SEARCH call in place
- Want to use the cached response?  Place a `#+nil` in front of the FMCV:-NB-SEARCH call
(hunchentoot:define-easy-handler (index :uri "/") (q)
  (let ((bodies (and q *cached-bodies* #+nil (fmcv:nb-search q))))
    (with-page (:title "Face Mask CE Verifier")
- Want to hit the external service instead?  Place a `#+nil` in front of *cached-bodies*
(hunchentoot:define-easy-handler (index :uri "/") (q)
  (let ((bodies (and q #+nil *cached-bodies* (fmcv:nb-search q))))
    (with-page (:title "Face Mask CE Verifier")
Note: `#+` is a reader macro that operates on the next two expressions found in the stream; if the first expression evaluates to non-NIL, then the macro would emit the second expression; otherwise, it would swallow it.  Now, we are always feeding the macro with a NIL as first expression, so basically we are telling it to swallow the second expression, always!

Pretty cool uh?!

_All the Vim haters should stop reading here_

I was so excited by this that I ended up creating two Vim mappings to automate this:

- `gce` to add (or remove) a `#+nil` in front of the current element (_element_ in the [vim-sexp](https://github.com/guns/vim-sexp) sense)
- `gcf` to add (or remove) a `#+nil` in front of the current form
" Comment out the current form / element
function! ToggleCommonLispComment(toggle_form) abort "{{{
    " Move at the start of the current element / form; if on an open paren
    " already, do nothing
    if getline('.')[col('.') - 1] != "("
        let motion = ""
        if a:toggle_form
            let motion = "\<Plug>(sexp_move_to_prev_bracket)"
            let motion = "\<Plug>(sexp_move_to_prev_element_head)"
        execute "normal " . motion

    " Figure out if we need to add a comment, or remove the existing one
    let should_add_comment = 1
    let save_cursor = getcurpos()
    let reg_save = @@
    " Move back to the previous element
    " Yank it
    " Check if it's a #+nil marker
    execute "normal \<Plug>(sexp_move_to_prev_element_head)"
    normal yW
    if @@ =~? "#+nil"
        let should_add_comment = 0
    let @@ = reg_save
    call setpos('.', save_cursor)

    " Do the needed
    if should_add_comment
        " Insert the comment marker, and move back to the element / form
        execute "normal! i#+nil "
        execute "normal \<Plug>(sexp_move_to_next_element_head)"
        " Move back, and the delete forward
        execute "normal \<Plug>(sexp_move_to_prev_element_head)"
        normal dW
" }}}
nmap <silent><buffer>   gce  :call ToggleCommonLispComment(0)<CR>
nmap <silent><buffer>   gcf  :call ToggleCommonLispComment(1)<CR>
It's a bit hack-y to be honest, but whatever!

Also, you can even change the relevant syntax file (e.g. ~/.vim/after/syntax/lisp.vim) and make these regions look like comments:
syntax match lispExcludedForm /\v\#\+nil \([^)]+\)/
highlight link lispExcludedForm lispComment
syn cluster lispBaseListCluster add=lispExcludedForm

syntax match lispExcludedElement /\v\#\+nil [^(][^ ]+/
highlight link lispExcludedElement lispComment
syn cluster lispBaseListCluster add=lispExcludedElement
What a joy!

Until next time...

2021-10-22 (permalink)

? Can I get a Replit instance, open a ssh connection to my VPS, so that I can then use my VPS as jump server to connect back into the REPL?

2021-10-20 (permalink)

How to verify the CE marking on your face mask?!

First off, what does the CE marking mean? From [Wikipedia](https://en.wikipedia.org/wiki/CE_marking):

The CE mark on a product indicates that the manufacturer or importer of that product affirms its compliance with the relevant EU legislation and the product may be sold anywhere in the European Economic Area (EEA). It is a criminal offence to affix a CE mark to a product that is not compliant or offer it for sale.

It continues:

The marking does not indicate EEA manufacture or that the EU or another authority has approved a product as safe or conformant. The EU requirements may include safety, health, and environmental protection. If stipulated in any EU product legislation, assessment by a Notified Body or manufacture according to a certified production quality system may be required. Where relevant, the CE mark is followed by the registration number of the notified body involved in conformity assessment.

So, for example, when you read `CE-2233` on a product, `CE` represents the CE marking (duh), while `2233` refers to ID of the notified body involved in conformity assessment.

Now off to the verification part.

Note: the steps listed below will help you verify that the notified body involved with the assessment is..._legit_, i.e. can actually issue the specific conformity assessment certificate; however, these won't help with verifying that the specified notified body had indeed run a conformity assessment on a given product (i.e. the product seller could lie about this, but as the Wikipedia article states, this is considered a criminal offense and by consequence, and published by law).

- To sell face masks in the UE, you need the CE marking
- For this matter, notified bodies need to be competent with the '2016/425
  Personal protective equipment' legislation
- They also need to be entitled to issue specific conformity assessment modules for 'Equipment providing respiratory system protection' products (e.g. face masks)
- Luckily for us, an online database exists,
  [NANDO](https://ec.europa.eu/growth/tools-databases/nando/index.cfm), storing
  all this information about all the registered bodies

So, without further ado, navigate to: [Nando](https://ec.europa.eu/growth/tools-databases/nando/index.cfm) > [Free search](https://ec.europa.eu/growth/tools-databases/nando/index.cfm?fuseaction=search.main).

Take the CE numerical value (i.e. the `2233` part of `CE-2233`), populate the `Keyword On Notified body number` field with it, and press the `Search` button.

Now `Ctrl+F` for the CE numerical value (i.e. the `2233` part of `CE-2233`), and if you are lucky enough, you should find a match inside the `Body type` column:

NB 2233 [GÉPTESZT Termelőeszközöket Felülvizsgáló és Karbantartó Kft.](https://ec.europa.eu/growth/tools-databases/nando/index.cfm?fuseaction=directive.nb&body_type=NB&refe_cd=NANDO_INPUT_166341) Hungary

Click on the link in the second column to open up the page listing addresses, contact details, and all the legislations that the notified body has scoped to assess and certify against.

Body :


GÉPTESZT Termelőeszközöket Felülvizsgáló és Karbantartó Kft.

Jablonka u. 79.

1037 Budapest

Country : Hungary


Phone : +36 1 250 3531

Fax : +36 1 250 3531


Email : gepteszt@gepteszt.hu

Website : www.gepteszt.hu


Notified Body number : 2233


Last approval date : 13/04/2010






- Regulation (EU) 2016/425 Personal protective equipment [HTML](https://ec.europa.eu/growth/tools-databases/nando/index.cfm?fuseaction=notification.html&ntf_id=309049&version_no=1) [PDF](https://ec.europa.eu/growth/tools-databases/nando/index.cfm?fuseaction=notification.pdf&dir_id=155501&ntf_id=309049)

Search for the `Personal protective equipment` entry inside the Legislations table, and click on the link labeled with `HTML`.  On this page you will find all the information the notified body has been certified for.

You should first confirm that there is a entry for `Equipment providing respiratory system protection`; without this, even the PPE certification notified body cannot engage in CE certification activities of protective masks).  Then, you will have to confirm the "conformity assessment" modules that the notified body is entitled to issue; in particular we are looking for:

- [Module B: EU type-examination](https://support.ce-check.eu/hc/en-us/articles/360019298431-Conformity-Assessment-Module-B)
- [Module C2: Supervised product checks at random intervals](https://support.ce-check.eu/hc/en-us/articles/360019507611-Conformity-Assessment-Module-C2-)
- [Module D: Quality assurance of the production process](https://support.ce-check.eu/hc/en-us/articles/360019191712-Conformity-Assessment-Modules-D-D1)

Note: that manufacturers can only enter the EU market legally after obtaining a Module B + Module C2 certificate or a Module B + Module D certificate



- Equipment providing respiratory system protection

- Protective Equipment against falls from heights




- EU type-examination

- Quality assurance of the production process

- Supervised product checks at random intervals


Articles / Annexes


- Annex V

- Annex VIII

- Annex VII

By the looks of the information shown above, it appears that this notified body have all it takes to properly run conformity assessments against face masks...Hurray!


- [Guide to using NANDO website to identify Notified Bodies PPE](https://yewtu.be/watch?v=ceCAceuDOhE)
- [Mascherine Ffp2, ecco i codici certificati CE non a norma e lo strumento di verifica Ue](https://quifinanza.it/info-utili/video/mascherine-ffp2-covid-codici-certificati-ce-non-a-norma/471686/)
- [How to Verify CE Certification?](https://verifyfull.com/how-to-verify-ce-certification%EF%BC%9F/)

2021-10-15 (permalink)

Today I fucking figured out how to make Vim's * / # not suck! @vim
" Star search plugin {{{
" Few things that this plugin does, and that others might not:
" 1. Works in both normal, and visual mode
" 2. Plays nice with smartcase search -- vim's builtin * / # mappings do not
"    support that
" 3. Does not automatically jump to the _first_ next occurrence; instead, moves
"    the cursor to the beginning of the current occurrence so that if for
"    whatever reason you decided to swap search direction with `N`, you would
"    be taken to the first previous occurrence (and not to the beginning of the
"    current word
" Normal mode:
" - Save the word under the cursor -- running `expand('<cword>')` while
"   searching actually cuases the cursor to move around one char forward, and
"   if you happened to start from the end of the word, that
"   'move-one-char-ahead' behavior would actually move the cursor over to the
"   next word, causing `expand('<cword>')` to return something unexpected
"   instead
" - Saves the current view (so we can restore it later)
" - Triggers the word-boundary-d search
" - Restores to the previous view -- effectively undoing the first jump
" - Checks if the cursor wasn't already at the beginning of a match, and if
"   not, runs `b` to move at the beginning of the current word
nnoremap <Plug>NormalStarSearchForward
            \ :let star_search_cword = expand('<cword>')<CR>
            \ :let star_search_view = winsaveview()<CR>
            \ /\<<C-R>=escape(star_search_cword, '*\')<CR>\><CR>
            \ :call winrestview(star_search_view)<CR>
            \ :execute "normal! " . (getline('.')[col('.') - 1:] !~# '^' . star_search_cword ? "b" : "")<CR>
nnoremap <Plug>NormalStarSearchBackward
            \ :let star_search_cword = expand('<cword>')<CR>
            \ :let star_search_view = winsaveview()<CR>
            \ ?\<<C-R>=escape(star_search_cword, '*\')<CR>\><CR>
            \ :call winrestview(star_search_view)<CR>
            \ :execute "normal! " . (getline('.')[col('.') - 1:] !~# '^' . star_search_cword ? "b" : "")<CR>
nmap * <Plug>NormalStarSearchForward
nmap # <Plug>NormalStarSearchBackward

" Visual mode:
" - Saves the current view (so we can restore it later)
" - Triggers the search -- without word-boundaries...you can use normal the
"   normal mode alternative for that
" - Restores to the previous view -- effectively undoing the first jump
xnoremap <Plug>VisualStarSearchForward
            \ :<C-U>let star_search_view = winsaveview()<CR>
            \ /<C-R>=escape(GetCurrentSelection(visualmode()), '*\')<CR><CR>
            \ :call winrestview(star_search_view)<CR>
xnoremap <Plug>VisualStarSearchBackward
            \ :<C-U>let star_search_view = winsaveview()<CR>
            \ ?<C-R>=escape(GetCurrentSelection(visualmode()), '*\')<CR><CR>
            \ :call winrestview(star_search_view)<CR>
xmap * <Plug>VisualStarSearchForward
xmap # <Plug>VisualStarSearchBackward

" }}}
There is a lot of copy-pasta in there, but it gets the job done and given the amount of time I have already spent on this today, or in the past even, I think it's time to move on!

? git: switch back to per-branch stashes, but register a pre-push hook to abort if any unpushed commit is tagged as SAVEPOINT -- https://gist.github.com/GUI/676bcb25389cd01d47828fddcf37e1f2

? change Vim/lisp setup so that the REPL is actually started in a different terminal (via Dispatch maybe) and does not take real estate

2021-10-06 (permalink)

* Added GitHub Actions to `cg` -- bye bye TravisCI -- [cg/pull/8](https://github.com/iamFIREcracker/cg/pull/8)

+ See if it's possible to enhance [40ants/setup-lisp](https://github.com/40ants/setup-lisp) to support Windows

2021-09-30 (permalink)

`ScratchThis`, a Vim command to make the current buffer a `scratch` one (see `:h special-buffers` for additional info):
function! s:ScratchThis() abort
    setlocal buftype=nowrite bufhidden=delete noswapfile

command! ScratchThis call s:ScratchThis()

2021-09-15 (permalink)

Nice little vim/fugitive gem: use `!` to open a command prompt with the file under the cursor.
nnoremap ! :! <C-R><C-F><Home><Right>
It's very handy if you want to delete an untracked file (see https://github.com/tpope/vim-fugitive/issues/23).

netrw comes with an identical mapping too!

2021-09-13 (permalink)

* finished reading [Tools for Thought: The History and Future of Mind-Expanding Technology](https://www.amazon.com/Tools-Thought-History-Mind-Expanding-Technology/dp/0262681153/ref=sr_1_1?dchild=1&keywords=tools+for+thought&qid=1631599675&s=books&sr=1-1)

2021-09-10 (permalink)

? book: The Dream Machine https://www.amazon.com/Dream-Machine-M-Mitchell-Waldrop/dp/1732265119

2021-09-08 (permalink)

Notes about reverse engineering Basecamp's Recording pattern

Before we begin: I never used RoR nor programmed in Ruby, so apologies if some of the things mentioned are inaccurate or completely wrong; I simply got interested in the pattern, and tried my best to understand how it could have been implemented.  So bear with me, and of course feel free to suggest edits.

Main concepts:

- Buckets are collections of records, i.e. _recordings_, having different types e.g. `Todo`, `Todoset`
- Recordings are just pointers, pointing to the actual data record, i.e. the _recordables_
- Recordables are immutable records, i.e. every time something changes, a new _recordable_ record is created and the relevant recording is updated to point to it
- Events are created to link together 2 recordables (the current version and the previous one), the recording, the bucket, and the author of the change
- This enables data versioning
- This enables data auditing (at the recording level, at the bucket level, and at the user level)

It all started with this [tweet](https://twitter.com/dhh/status/962156053461843968?lang=en):

Probably the single most powerful pattern in BC3 ❤️. Really should write it up at one point.

This is how a Basecamp URL looks like: `/1234567890/projects/12345/todos/67890`

In it:

- `1234567890` is the account ID -- it identifies a company's instance, it represents a tenant
- `12345` is the bucket ID, pointing to a record whose `bucketable_type` is `Project`
- `67890` is the recording ID, pointing to a record whose `recordable_type` is `Todo`

Bucket is not a project, but links to one (project is the _bucketable_ entity):
class Bucket < ActiveRecord::Base
  belongs_to :account
  delegated_type :bucketable, types: %w[ Project ] # this adds bucketable_id, bucketable_type to the schema ... or `belongs_to :bucketable`
  has_many :recordings, dependent: :destroy
Recording is not a todo, but links to one (todo is the _recordable_ entity):
class Recording < ActiveRecord::Base
  belongs_to :bucket, touch: true
  belongs_to: :creator, class_name: 'Person', default: -> { Current.person }
  delegated_type :recordable, types: %w[ Todo Todolist Todoset Dock ] # this adds recordable_id and recordable_type to the schema

  delegate :account, to: :bucket

module Recordable extend ActiveSupport::Concern
  included do
    has_many :recordings, as: :recordable # same as specified inside `Recording` as `delegated_type`
Recordables are immutable, and do not usually have all the created-at, updated-at metadata...recordings do!

Recordings have parents, and children; it almost becomes a graph database, with a tree structure:

- recording.recordable_type: Todo
- recording.parent.recordable_type: Todolist
- recording.parent.parent.recordable_type: Todoset
- recording.parent.parent.parent.recordable_type: Dock
- recording.parent.parent.parent.children.map(&:recordable_type): Chat::Transcript, Message::Board, Todoset, Schedule, Questionnaire, Vault

The above, i.e. the tree structure, is most likely implemented via the `Tree` concern:
module Recording::Tree
  extend ActiveSupport::Concern

  included do
    belongs_to :parent, class_name: 'Recording', optional: true
    has_many :children, class_name: 'Recording', foreign_key: :parent_id
Note: I don't assume the above to be correct, but it should get the job done; I question whether it would make sense to add a _type_ field on this parent/child relation, but maybe it's not strictly required (i.e. you always know that one todo's parent is a todolist).

As stated above, recordings are pointers, almost like symlinks, and when you update a todo, you don't actually update the record, but create a new one, and update the recording to point to that; since URLs are all keyed-up accordingly (i.e. they use Recordings IDs, and not Recordable ones), the URL does not change but the actual todo the recording point to does.

Controllers then look something like this:
class TodosController < ApplicationController
  def create
    @recording = @bucket.record new_todo

  def update
    @recording.update! recordable: new_todo

    def new_todo
      Todo.new params.require(:todo).permit(:content, :description, :starts_on, :due_on)
While this is how `Bucket.record` looks like:
class Bucket < ActriveRecord::Base
  def record(recordable, children: nil, parent: nil, status: :active, creator: Current.person, **options)
    transaction do
      options.merge!(recordable: recordable, parent: parent, status: status, creator: creator)

      recordings.create!(options).tap do | recording |
        Array(children).each do | child |
          record child, parent: recording, status: status, creator: creator
This enables you to do things like, "Copy message to a new project", way more efficient: previously you had to set up a new job that literally copied the message, the comments, the events onto the new project; while now, all you have to do is set up a new recording pointing to the existing _immutable_ recordable.

For example:

- Project has messages -- messages are recordables
- Messages have comments -- comments are recordables
- When you copy a message from one project to another, you will have to create a recording not just for the message to be copied, but for all its existing comments as well (that's what `children` inside `Bucket.record` is for)

Note: `children` inside `Bucket.record` is expected to be a recordable, and not a recording; this means we will have to pass in all the child recordables, e.g. `message.recording.children.map(&:recordable)`.

Note: `Bucket.record` (at least the version above, which was taken from [shown in the presentation](https://www.youtube.com/watch?v=tmWlm0M6JSk&t=3060s)) does not seem to recurse, which means you can only create recordings for the parent entity and its direct children, i.e. messages and comments, but not `Todoset`s, `Todolist`s _and_ `Todo`s.

Note: is this more efficient, really?  Even though you will not have to copy the immutable records (i.e. the recordables), you will still have to recreate the same recordings tree, right?

After you copied a recording, you should end up with something like this:
>> Recording.find(111111111).recordable.id
=> 33333333333333

>> Recording.find(222222222).recordable.id
=> 33333333333333
Again, two pointers, i.e. recordings, pointing to the same immutable record, i.e. the recordable.

Note: a recordable has many recordings, not one; this is because the same recordable object can be pointed by two different recordings.

This Recording pattern works in tandem with `Event`s:

- They link to the newer version of the recordable, and the previous one
- They belong to a `Recording`: this way you can easily see all the times a recording was updated
- They belong to a `Bucket`: this way you can easily see all the times a bucket was updated
- They belong to a `Person`: this way you can easily see all the changes done by one user
- They have one `Detail`, i.e. a hash containing some additional metadata linked to the current event
class Event < ActiveRecord::Base
  belongs_to: :recording, required: false
  belongs_to: :bucket
  belongs_to: :creator, class_name: 'Person'

  has_one :detail, dependent: :delete, required: true

  include Notifiable...Requested

and with the `Eventable` concern:
module Recording::Eventable
  extends ActiveSupport::Concern
  included do
    has_many :events, dependent: :destroy

    after_create :track_created
    after_update :track_updated
    aftter_update_commit :forget_adoption_tracking, :forget_events

  def track_event(action, recordable_previous: nil, **particulars)
    Event.create! \
      recording: self, recordable: recordable, recordable_previous: recordable_previous,
      bucket: bucket, creator: Current.person, action: action,
      detail: Event::Detail.new(particulars)
`track_event` here is the generic method, but more specific ones exist as well, for more common use cases so we don't have to manually plumb arguments.

We track an event every single time an eventable is created, or updated:
    after_create: :track_created
    after_update :track_updated
The private method `track_created` simply delegates to `track_event`:
    def track_created
      track_event :created, status_was: status
A recordable has many events; again, the same recordable could referenced by multiple recordings, and since events are created when recordings are created, then that's how you end up with multiple events.

OK, great, nice, but what schema would enable such a pattern?  Again, not 100% accurate, but hopefully close enough:

- Buckets (id, bucketable_type, bucketable_id, ...)
- Recordings (id, bucket_id, recordable_type, recordable_id, parent_id, ...)
- Events (id, bucket_id, recording_id, recordable_id, prevous_recordable_id, user_id, ...)

Note: recordables are _untyped_ here, but the type information is actually one join-table away, i.e. it's available inside the linked recording

- Projects (id, ...)
- Messages (id, ...)

And this seems to be it really.

Note: this pattern heavily relies on Rails delegate types (see [rais/rails/pull#39341](https://github.com/rails/rails/pull/39341)), to automatically traverse the recording table to automatically get to the recordable entry.  My first thought was: how will this perform?  How is the extra join operation required to fetch any linked entity going to affect performance?  If it's working for Basecamp, hopefully it's working decently enough for other applications as well, but I guess it also depends on how much the different entities are linked together.

Note: in the end, this is not that different from 'time-based versioning' (read: [Keeping track of graph changes using temporal versioning](https://medium.com/neo4j/keeping-track-of-graph-changes-using-temporal-versioning-3b0f854536fa)), with recordings being the _identity_ nodes, and recordables being the _state_ objects.

Some references:

- [Basecamp 3 Unraveled](https://www.youtube.com/watch?v=tmWlm0M6JSk&t=3060s) -- where an overview of the Bucket/Recording pattern is given
- [On Writing Software Well #3: Using globals when the price is right](https://www.youtube.com/watch?v=D7zUOtlpUPw&t=653s) -- where `Eventable` concern, and the `Event` model are shown
- [On Writing Software Well #5: Testing without test damage or excessive isolation](https://www.youtube.com/watch?v=5hN6OZDyQtk&t=419s) -- where the `Recording` model is shown
- [GoRails: Recording pattern (Basecamp 3)](https://gorails.com/forum/recording-pattern-basecamp-3)

2021-09-06 (permalink)

* `gem install`-d all the things again. I guess the latest `brew upgrade` that I run last week might have broken a few things (and while at it, I also `rvm install --default ruby-3.0.0`)

? implement rainbow-blocks for Vim (see: https://twitter.com/lisperati/status/1434114082903367681). A simple hack would be to define 5/6 syntax rules for `(...)` blocks and configure them so that they can only exist within an outer block, i.e. block1, block2 only inside block1, block3 only inside block2.

2021-08-30 (permalink)

* finished reading [Grokking Simplicity](https://www.amazon.it/Grokking-Simplicity-Software-Functional-Thinking/dp/1617296201/ref=sr_1_1?__mk_it_IT=%C3%85M%C3%85%C5%BD%C3%95%C3%91&dchild=1&keywords=grokking+simplicity&qid=1630331790&sr=8-1)

Grokking Simplicity notes

This page of the book says it all:

We have learned the skills of professionals


Since we're at the end of the book, let's look back at how far we've come and do a high-level listing of the skills we've learned. These are the skills of professional functional programmers. They have been chose for their power and depth.


Part 1: Actions, Calculations, and Data:


- Identifying the most problematic parts of your code by distinguishing actions, calculations, and data

- Making your code more resusable and testable by extracting calculations from actions

- Improving the design of actions by replacing implicit inputs and outputs with explicit ones

- Implementing immutability to make reading data into a calculation

- Organizing and improving code with stratified design


Part 2: First-class abstractions


- Making syntactic operations first-class so that they can be abstracted in code

- Reasoning at a higher level using functional iteration and other functional tools

- Chaining functional tools into data transformation pipelines

- Understanding distributed and concurrent systems by timeline diagrams

- Manipulating timelines to eliminate bugs

- Mutating state safely with higher-order functions

- Using reactive architecture to reduce coupling between cause and effects

- Applying the onion architecture (Interaction, Domain, Language) to design services that interact with the world

Where here is the list of primitives presented in the book to work with timelines.

Serialize tasks with `Queue`:
function Queue(worker) {
  var queue_items = [];
  var working = false;

  function runNext() {
    if(queue_items.length === 0)
    working = true;
    var item = queue_items.shift();
    worker(item.data, function(val) {
      working = false;
      setTimeout(item.callback, 0, val);

  return function(data, callback) {
      data: data,
      callback: callback || function(){}
    setTimeout(runNext, 0);
function calc_cart_worker(cart, done) {
  calc_cart_total(cart, function(total) {

var update_total_queue = Queue(calc_cart_worker);
Serialize tasks and skip duplicate work with `DroppingQueue`:
function DroppingQueue(max, worker) {
  var queue_items = [];
  var working = false;

  function runNext() {
    if(queue_items.length === 0)
    working = true;
    var item = queue_items.shift();
    worker(item.data, function(val) {
      working = false;
      setTimeout(item.callback, 0, val);

  return function(data, callback) {
      data: data,
      callback: callback || function(){}
    while(queue_items.length > max)
    setTimeout(runNext, 0);
function calc_cart_worker(cart, done) {
  calc_cart_total(cart, function(total) {

var update_total_queue = DroppingQueue(1, calc_cart_worker);
Synchronize (i.e. cut) multiple timelines with `Cut`:
function Cut(num, callback) {
  var num_finished = 0;
  return function() {
    num_finished += 1;
    if(num_finished === num)
var done = Cut(3, function() {
  console.log("3 timelines are finished");

done(); // only this one logs
Make sure a function is called only once, with `JustOnce`:
function JustOnce(action) {
  var alreadyCalled = false;
  return function(a, b, c) {
    if(alreadyCalled) return;
    alreadyCalled = true;
    return action(a, b, c);
function sendAddToCartText(number) {
  sendTextAjax(number, "Thanks for adding something to your cart. Reply if you have any questions!");

var sendAddToCartTextOnce = JustOnce(sendAddToCartText);

sendAddToCartTextOnce("555-555-5555-55"); // only this one logs
Encapsulate state inside _reactive_ `ValueCell`s:
function ValueCell(initialValue) {
  var currentValue = initialValue;
  var watchers = [];
  return {
    val: function() {
      return currentValue;
    update: function(f) {
      var oldValue = currentValue;
      var newValue = f(oldValue);
      if(oldValue !== newValue) {
        currentValue = newValue;
        forEach(watchers, function(watcher) {
    addWatcher: function(f) {
var shopping_cart = ValueCell({});

function add_item_to_cart(name, price) {
  var item = make_cart_item(name, price);
  shopping_cart.update(function(cart) {
    return add_item(cart, item);
  var total = calc_total(shopping_cart.val());

Derived values can instead be implemented with `FormulaCell`s:
function FormulaCell(upstreamCell, f) {
  var myCell = ValueCell(f(upstreamCell.val()));
  upstreamCell.addWatcher(function(newUpstreamValue) {
    myCell.update(function(currentValue) {
      return f(newUpstreamValue);
  return {
    val: myCell.val,
    addWatcher: myCell.addWatcher
var shopping_cart = ValueCell({});

var cart_total = FormulaCell(shopping_cart, calc_total);

function add_item_to_cart(name, price) {
  var item = make_cart_item(name, price);
  shopping_cart.update(function(cart) {
    return add_item(cart, item);


Location sharing alternative to the Google built-in one @idea

- It would be fun
- You would not have to share data with Google (why would anyone share it with you instead? My friends and family probably would!)

Streamlining answering the donors questionnaire @idea

"Which cities have you visited during the last 4 weeks?"
"Which countries have you visited over the last 6 months?"

These are some questions donors get asked over and over again before they are are cleared for the donation, and it always catches me off guard.  Can this information be automatically pulled from Google?

What about my wife's data instead, would I have access to it?  During COVID, most of the time it's not just about the place you have visited, but the places other people you are living with had...

2021-08-27 (permalink)

The power of Lisp is its own worst enemy.

[The Lisp Curse](http://www.winestockwebdesign.com/Essays/Lisp_Curse.html)

2021-08-23 (permalink)

* finished reading [The Friendly Orange Glow: The Untold Story of the PLATO System and the Dawn of Cyberculture](https://www.amazon.com/Friendly-Orange-Glow-Untold-Cyberculture/dp/1101871555/ref=tmm_hrd_swatch_0?_encoding=UTF8&qid=1629734304&sr=1-1)

2021-06-28 (permalink)

* finished reading [10 PRINT CHR$(205.5+RND(1)); : GOTO 10](https://www.amazon.it/dp/0262526743?tag=duc01-21&linkCode=osi&th=1&psc=1)

2021-06-19 (permalink)

Remote live coding a Clack application with Swank, and Ngrok

Today I would like to show you how to create a very simple CL Web application with Clack, and how to use Swank and [Ngrok](https://ngrok.com/) to enable remote [live coding](https://en.wikipedia.org/wiki/Live_coding).

Here is what we are going to wind up with:

- Clack application running on `localhost:5000` -- i.e. the Web application
- Swank server running on `localhost:4006` -- i.e. the "live coding" enabler
- Ngrok tunnel to remotely expose `localhost:4006` (just in case you did not have SSH access into the server the app is running on)

As usual, let's start off by naming all the systems we are depending on:

- `clack`, Web application environment for CL
- `ngrok`, CL wrapper for installing and running Ngrok
- `swank`, what enables remote "live coding"
- `uiop`, utilities that abstract over discrepancies between implementations

Now, `ngrok` has not been published to Quicklisp yet, so you will have to manually download it first:
$ git clone https://github.com/40ants/ngrok.git
$ sbcl --noinform
> ...
...and then make sure it is QL:QUICKLOAD-able:
(pushnew '(merge-pathnames (parse-namestring "ngrok/")
Now you can go ahead and load all the required dependencies:
(ql:quickload '("clack" "ngrok" "swank" "uiop"))
Let's take a look at the MAIN function. It starts the Web application, the Swank server, and Ngrok:
(defun main ()
For the Web application:

- We define a special variable, *HANDLER*, to hold the Clack Web application handler
- We start the app, bind it on `localhost:5000`, and upon receiving an incoming request we delegate to SRV
- Inside SRV, we delegate to APP (more to this later)
- Inside APP, we finally process the request and return a dummy message back to the client
(defvar *handler* nil)

(defun clack () (setf *handler* (clack:clackup #'srv :address "localhost" :port 5000)))

(defun srv (env) (app env))

(defun app (env)
  (declare (ignore env))
  '(200 (:content-type "text/plain") ("Hello, Clack!")))
You might be wondering: "why not invoking CLACK:CLACKUP with #'APP? Why the middle-man, SRV?"  Well, that's because Clack would dispatch to whichever function was passed in at the time CLACK:CLACKUP was called, and because of that, any subsequent re-definitions of such function would not be picked up by the framework.  The solution? Add a level of indirection, and so long as you don't need to change SRV, but always APP, then you should be all right!

Let's now take a look at how to setup a Swank server.  First we ask the user for a secret with which to [secure](https://github.com/slime/slime/issues/286) the server (i.e. it will accept incoming connections only from those clients that do know such secret), then we write the secret onto ~/.slime-secret (yes, it's going to overwrite the existing file!), and last we actually start the server:
(defun getenv-or-readline (name)
  "Gets the value of the environment variable, or asks the user to provide
   a value for it."
  (or (uiop:getenv name)
        (format *query-io* "~a=" name)
        (force-output *query-io*)
        (read-line *query-io*))))

(defvar *slime-secret* (getenv-or-readline "SLIME_SECRET"))
(defparameter *swank-port* 4006)

(defun swank ()
  (swank:create-server :port *swank-port* :dont-close t))

(defun write-slime-secret ()
  (with-open-file (stream "~/.slime-secret" :direction :output :if-exists :supersede)
    (write-string *slime-secret* stream)))
Last but not least, Ngrok: we read the authentication token, so Ngrok knows the account the tunnel needs to be associated with, and then we finally start the daemon by specifying the port the Swank server is listening to (as that's what we want Ngrok to create a tunnel for) and the authentication token:
(defvar *ngrok-auth-token* (getenv-or-readline "NGROK_AUTH_TOKEN"))

(defun ngrok () (ngrok:start *swank-port* :auth-token *ngrok-auth-token*))
Give the MAIN function a go, and you should be presented with a similar output:
> (main)
Hunchentoot server is started.
Listening on localhost:5000.
;; Swank started at port: 4006.
 <INFO> [10:13:43] ngrok setup.lisp (install-ngrok) -
  Ngrok already installed, changing authtoken
 <INFO> [10:13:44] ngrok setup.lisp (start) - Starting Ngrok on TCP port 4006
 <INFO> [10:13:44] ngrok setup.lisp (start) -
  Tunnnel established! Connect to the tcp://6.tcp.eu.ngrok.io:12321
Let's test the Web server:
$ curl https://localhost:5000
Hello, Clack!
It's working.  Now open Vim/Emacs, connect to the Swank server (i.e. host: `6.tcp.eu.ngrok.io`, port: `12321`), and change the Web handler to return something different:
(defun app (env)
  (declare (ignore env))
  '(200 (:content-type "text/plain") ("Hello, Matteo!")))
Hit the Web server again, and this time it should return a different message:
$ curl https://localhost:5000
Hello, Matteo!
Note how we did not have to bounce the application; all we had to do was re-define the request handler...that's it!

Happy remote live coding!

PS. The above is also available on [GitHub](https://github.com/iamFIREcracker/cl-clack-swank-ngrok), and on [Replit](https://replit.com/@iamFIREcracker/cl-clack-swank-ngrok).

2021-06-17 (permalink)

* Opened [cl-cookbook/pull/385](https://github.com/LispCookbook/cl-cookbook/pull/385): Don't wrap SWANK:CREATE-SERVER inside BT:MAKE-THREAD

2021-06-11 (permalink)

? That will never work: Netflix https://www.amazon.ca/That-Will-Never-Work-Netflix/dp/0316530204

? No rules Netflix Culture Reinvetion https://www.amazon.ca/No-Rules-Netflix-Culture-Reinvention/dp/1984877860

? book: Designing Data-intensive applications https://www.amazon.ca/dp/1449373321/ref=cm_sw_r_wa_awdb_imm_CPN7ADPYR3CCM6EDZFMC

? book: Microservices Patterns https://www.amazon.ca/dp/1617294543?ref=ppx_pop_mob_ap_share

? book: Grokking algorithms https://www.amazon.ca/dp/1617292230/ref=cm_sw_r_wa_api_glt_i_FCXTQ5MH3YVCRVKBMHYR

2021-06-07 (permalink)

* finished reading [Object-Oriented Programming in COMMON LISP: A Programmer's Guide to CLOS](https://www.amazon.com/Object-Oriented-Programming-COMMON-LISP-Programmers/dp/0201175894)

2021-06-02 (permalink)

Web scraping for fun and profit^W for a registration to get my COVID-19 jab

Rumor says that in a few days everyone in Tuscany, irrespective of their age, could finally register for their first shot of the vaccine; that means "click day", i.e. continuously polling the registration website until your registration category is open.  I am sure a bird will eventually announce when the service will be open, i.e. the time of the day, so no need to stay on the lookout for the whole day, but I figured I could use this as an excuse to do a little bit of Web scraping in Common Lisp, so here it goes.

Dependencies first:
(ql:quickload "drakma")     ; to fire HTTP requests
(ql:quickload "local-time") ; to track of when the scraping last happened
(ql:quickload "st-json")    ; well...you guessed it
(ql:quickload "uiop")       ; read env variables
The next step is writing our MAIN loop:

- Wait for categories to be activated: new categories can be added, or existing ones can be activated
- Send an SMS to notify me about newly activated categories (so I can pop the website open, and try to register)
- Repeat
(defun main ()
    (with-simple-restart (continue "Ignore the error")
WAIT-FOR-ACTIVE-CATEGORIES is implemented as loop form that:

- Checks if categories have been activated, in which case the list of recently activated ones is returned
- Otherwise it saves the current timestamp inside *LAST-RUN-AT* and then goes to sleep for a little while
(defvar *last-run-at* nil "When the categories scraper last run")

(defun wait-for-active-categories ()
  (loop :when (categories-activated-p) :return it
        :do (progn
              (setf *last-run-at* (local-time:now))
Let's now dig into the category processing part.  First we define a special variable, *CATEGORIES-SNAPSHOTS*, to keep track of all the past snapshots of services we scraped so far; next we fetch the categories from the remote website, see if any of them got activated since the last scraping, and last we push the latest snapshot into *CATEGORIES-SNAPSHOTS* and do some _harvesting_ to make sure we don't run out of memory because of all these scraping done so far:
(defvar *categories-snapshots* nil
  "List of categories snapshots -- handy to see what changed over time.")

(defun categories-activated-p ()
  (let ((categories (fetch-categories)))
    (unless *categories-snapshots*
      (push categories *categories-snapshots*))
    (prog1 (find-recently-activated categories)
      (push categories *categories-snapshots*)
Before fetching the list of categories we are going to define a few utilities: a condition signaling when HTML is returned in spite of JSON (usually a sign that the current session expired):
(define-condition html-not-json-content-type () ())
The URLs for the registration page (this will be used later, when generating the SMS) and for the endpoint returning the list of categories:
(defparameter *prenotavaccino-home-url* "https://prenotavaccino.sanita.toscana.it")
(defparameter *prenotavaccino-categories-url* "https://prenotavaccino.sanita.toscana.it/api/index")
A cookie jar to create a _persistent_ connection with the website under scraping:
(defvar *cookie-jar* (make-instance 'drakma:cookie-jar)
  "Cookie jar to hold session cookies when interacting with Prenotavaccino")
With all this defined, fetching the list of categories becomes as easy as firing a request to the endpoint and confirm the response actually contains JSON and not HTML: if JSON, parse it and return it, otherwise, simply try again.  The idea behind "trying again" is that when the response contains HTML, that will most likely contain a redirect to the home page with a newly created session cookie, and since the same cookie jar is getting used, the simple fact that we processed that response should be enough to make the next API call succeed:
(defun parse-json (response)
  (let* ((string (flexi-streams:octets-to-string response)))
    (st-json:read-json string)))

(defun fetch-categories ()
  (flet ((fetch-it ()
           (multiple-value-bind (response status headers)
               (drakma:http-request *prenotavaccino-categories-url* :cookie-jar *cookie-jar*)
             (declare (ignore status))
             (let ((content-type (cdr (assoc :content-type headers))))
               (if (equal content-type "text/html")
                   (error 'html-not-json-content-type)
         (parse-it (response)
           (st-json:getjso "categories" (parse-json response))))
      (handler-case (fetch-it)
        (html-not-json-content-type (c)
          (declare (ignore c))
          (format t "~&Received HTML instead of JSON. Retrying assuming the previous session expired...")
Note: we retry only once, so if two consecutive responses contain HTML instead of JSON, that would result in the debugger to pop open.

Finding all the recently activated categories consists of the following steps:

- for each category just scraped
- compare it with the same in the last snapshot of categories
- a category is considered as _recently_ activated if it's active and it is not present inside the second to last snapshot, or it was present but either it was inactive or if its title has changed (yes, sometimes the title of a category is changed to advertise that new people with different age can now sign up)

Anyways, all the above translates to the following:
(defun category-name (cat) (st-json:getjso "idCategory" cat))
(defun category-title (cat) (st-json:getjso "title" cat))
(defun category-active-p (cat)
  (and (eql (st-json:getjso "active" cat) :true)
       (eql (st-json:getjso "forceDisabled" cat) :false)))

(defun find-recently-activated (categories)
  (flet ((category-by-name (name categories)
           (find name categories :key #'category-name :test #'equal)))
    (let ((prev-categories (first *categories-snapshots*)))
      (loop :for cat :in categories
            :for cat-prev = (category-by-name (category-name cat) prev-categories)
            :when (and (category-active-p cat)
                       (or (not cat-prev)
                           (not (category-active-p cat-prev))
                           (not (equal (category-title cat-prev) (category-title cat)))))
            :collect cat))))
Harvesting is pretty easy: define a maximum length threshold for the list of snapshots and when the list grows bigger than that, start popping items from the back of the list as new values are pushed to the front:
(defparameter *categories-snapshots-max-length* 50
  "Maximum numbers of categories snapshots to keep around")

(defun harvest-categories-snapshots ()
  (when (> (length *categories-snapshots*) *categories-snapshots-max-length*)
    (setf *categories-snapshots*
          (subseq *categories-snapshots* 0 (1+ *categories-snapshots-max-length*)))))
Note: I could have removed "consecutive duplicates" from the list, or prevented these from getting stored in the list to begin with, but I am going to leave this as an exercise for the reader ;-)

Two pieces of the puzzle are still missing: RANDOM-SLEEP, and SEND-SMS.  For RANDOM-SLEEP we decide the minimum number of seconds that the scraper should sleep for, and then add some _randomness_ to it (like the remote site cared that we pretended to try and act like a _human_, but let's do it anyway):
(defparameter *sleep-seconds-min* 60
  "Minimum number of seconds the scraper will sleep for")
(defparameter *sleep-seconds-jitter* 5
  "Adds a pinch of randomicity -- see RANDOM-SLEEP")

(defun random-sleep ()
  (sleep (+ *sleep-seconds-min* (random *sleep-seconds-jitter*))))
For the SMS part instead, we are going to use Twilio; first we define all the parameters required to send SMSs:

- Account SID
- Auth token
- From number
- To numbers (space separated values)
(defun getenv-or-readline (name)
  "Gets the value of the environment variable, or asks the user to provide
  a value for it."
  (or (uiop:getenv name)
        (format *query-io* "~a=" name)
        (force-output *query-io*)
        (read-line *query-io*))))

(defvar *twilio-account-sid* (getenv-or-readline "TWILIO_ACCOUNT_SID"))
(defvar *twilio-auth-token* (getenv-or-readline "TWILIO_AUTH_TOKEN"))
(defvar *twilio-messages-api-url*
  (format nil "https://api.twilio.com/2010-04-01/Accounts/~a/Messages.json" *twilio-account-sid*))
(defvar *twilio-from* (getenv-or-readline "TWILIO_FROM_NUMBER"))
(defvar *twilio-to-list*
  (split-sequence:split-sequence #\Space (getenv-or-readline "TWILIO_TO_NUMBERS")))
Next we assemble the HTTP request, fire it, and do some error checking to signal an error in case it failed to deliver the message to any of the _to_ numbers specified inside *TWILIO-TO-LIST*:
(defun send-sms (&optional (body (sms-body)))
  (flet ((send-it (to)
           (drakma:http-request *twilio-messages-api-url*
                                :method :post
                                :basic-authorization `(,*twilio-account-sid* ,*twilio-auth-token*)
                                :parameters `(("Body" . ,body)
                                              ("From" . ,*twilio-from*)
                                              ("To" . ,to)))))
    (let (failed)
      (dolist (to *twilio-to-list*)
        (let* ((jso (parse-json (send-it to)))
               (error-code (st-json:getjso "error_code" jso)))
          (unless (eql error-code :null)
            (push jso failed))))
      (if failed
          (error "Failed to deliver **all** SMSs: ~a" failed)
Last but not least, the SMS body; we want to include in the message which services got activated since the last time the scraper run, and to do so we:

- Take the latest snapshot from *CATEGORIES-SNAPSHOTS*
- Temporarily set *CATEGORIES-SNAPSHOTS* to its CDR, like the latest snapshot wasn't recorded yet
- Call FIND-RECENTLY-ACTIVATED effectively pretending like we were inside CATEGORIES-ACTIVATED-P and were trying to understand if anything got recently activated (dynamic variables are fun, aren't they?!)
(defun sms-body ()
  (let* ((categories (first *categories-snapshots*))
         (*categories-snapshots* (cdr *categories-snapshots*)))
    (format nil "Ora attivi: ~{~A~^, ~} -- ~a"
            (mapcar #'category-title
                    (find-recently-activated categories))
Note: "Ora attivi:" is the Italian for "Now active:"

And that's it: give `(main)` a go in the REPL, wait for a while, and eventually you should receive an SMS informing you which category got activated!
> (find "LastMinute" (first *categories-snapshots*)
        :key #'category-name
        :test #'string=)
   :ALIST (("idCategory" . "LastMinute") ("active" . :TRUE)
           ("forceDisabled" . :FALSE) ("start" . "2021-06-02")
           ("end" . "2021-07-31") ("title" . "Last<br/>Minute")
            . "<b>ATTENZIONE.</b> Prenotazione degli slot rimasti disponibili nelle prossime 24H.")
           ("message" . "Il servizio aprirà alle ore 19:00 di ogni giorno")
           ("updateMaxMinYear" . :TRUE)))

> (sms-body)
"Ora attivi: Last<br/>Minute -- https://prenotavaccino.sanita.toscana.it"
Happy click day, Italians!

2021-05-31 (permalink)

TIL: Untracked files are stored in the third parent of a stash commit.

Untracked files are stored in the third parent of a stash commit. (This isn't actually documented, but is pretty obvious from The commit which introduced the -u feature, 787513..., and the way the rest of the documentation for git-stash phrases things... or just by doing git log --graph stash@{0})

You can view just the "untracked" portion of the stash via:

git show stash@{0}^3

or, just the "untracked" tree itself, via:

git show stash@{0}^3:

or, a particular "untracked" file in the tree, via:

git show stash@{0}^3:<path/to/file>

Source: [stackoverflow](https://stackoverflow.com/questions/12681529/in-git-is-there-a-way-to-show-untracked-stashed-files-without-applying-the-stas#:~:text=Stash%20entries%20can%20be%20made,as%20part%20of%20the%20diff)

2021-05-27 (permalink)

Few days ago Replit announced support for [every programming language](https://twitter.com/Replit/status/1396915485325860868)!

I never heard of [NixOS](https://nixos.org/) (that's what enables them to support "every programming language"), but I decided to give it a go anyway, and after a little bit of struggles I was then able to put together the following Repls:

- [Common Lisp](https://replit.com/@iamFIREcracker/Common-Lisp) -- of course the first one had to be one about CL
- [Common Lisp w/ SSL](https://replit.com/@iamFIREcracker/Common-Lisp-with-SSL) -- it turns out you have to mess with `LD_LIBRARY_PATH` to get SSL to work with `:cl+ssl`
- [Game of Life in Common Lisp](https://replit.com/@iamFIREcracker/Common-Lisp-greater-Game-of-Life) -- why not...
- [10 PRINT.BAS](https://replit.com/@iamFIREcracker/10PRINTBAS) -- ever heard of the famous one-line Commodore 64 BASIC program to generate mazes?  They even wrote a whole [book](https://10print.org/) about it!

This is pretty cool, well done Replit!

This is the famous one-line Commodore 64 BASIC program to generate random mazes (the Commodore 64 uses the [PETSCII](https://www.c64-wiki.com/wiki/PETSCII) character set, not ASCII):
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
While this is a Common Lisp equivalent:
(loop (princ (aref "/\\" (random 2))))
If the above run a little bit too fast int he REPL (it most likely will, these days), then you can try with this other alternative:
  (princ (aref "/\\" (random 2)))
  (sleep (/ 1 30)))
Happy retro-hacking!

2021-05-21 (permalink)

Example of how to use [AbortController](https://developer.mozilla.org/en-US/docs/Web/API/AbortController) to cancel an action when input changes:
let currentJob = Promise.resolve();
let currentController;

function showSearchResults(input) {
  if (currentController) currentController.abort();
  currentController = new AbortController();
  const { signal } = currentController;

  currentJob = currentJob
    .finally(async () => {
      try {
        const response = await fetch(getSearchUrl(input), { signal });
        await displayResults(response, { signal });
      } catch (err) {
        if (err.name === 'AbortError') return;
      } finally {
Source: [twitter](https://twitter.com/jaffathecake/status/1395682545090633729)

2021-05-16 (permalink)

Configuring lightline.vim to display the quickfix's title

In case anyone was interested in displaying the quickfix title while on quickfix buffers (works with location-list buffers too), all you have to do is loading the following into the runtime:
let g:lightline = {
    \ 'component': {
    \   'filename': '%t%{exists("w:quickfix_title")? " ".w:quickfix_title : ""}'
    \   },
    \ }
NORMAL  [Quickfix List] | -
And after:
NORMAL  [Quickfix List] :ag --vimgrep --hidden --smart-case --nogroup --nocolor --column stream-name | -
Related link: https://www.reddit.com/r/vim/comments/ej2bvx/change_the_quickfix_title/

Lack's Redis session store does not appear to be thread-safe

Steps to reproduce:

- Load the necessary dependencies:
(ql:quickload '(:clack :lack :lack-session-store-redis :cl-redis))
- Setup a dummy "Hello, World" application:
(defparameter *app*
  (lambda (env)
    (declare (ignore env))
    '(200 (:content-type "text/plain") ("Hello, World"))))
- Setup the middleware chain, one that uses Redis as session storage:
(setf *app*
         :store (lack.session.store.redis:make-redis-store :connection (make-instance 'redis:redis-connection
                                                                                      :host #(0 0 0 0)
                                                                                      :port 6379
                                                                                      :auth "auth")))
- Start the app:
> (defvar *handler* (clack:clackup *app*))
Hunchentoot server is started.
Listening on
- Run `wrk` against it:
$ wrk -c 10 -t 4 -d 10
Running 10s test @
  4 threads and 10 connections
Expected behavior:

- The test runs successfully, and the number of requests per second is reported

Actual behavior:

- Lots of errors are signaled; from unexpected errors:
Thread: 7; Level: 1

Redis error: NIL

ERR Protocool error: invalid multibulk length
   [Condition of type REDIS:REDIS-ERROR-REPLY]

  R 0. ABORT - abort thread (#<THREAD "hunchentoot-worker-" RUNNING {100154F693}>)

  F 0.  ((:METHOD REDIS:EXPECT ((EQL :STATUS))) #<unused argument>) [fast-method]
  F 1.  (REDIS::SET "session:fdf15f46896842bcd84ef2e87d2321e97419f70e" "KDpQQ09ERSAxICg6SEFTSC1UQUJMRSAxIDcgMS41IDEuMCBFUVVBTCBOSUwgTklMKSk=")
  F 4.  ((LAMBDA (LACK.MIDDLEWARE.BACKTRACE::ENV) :IN "/Users/matteolandi/Workspace/lack/src/middleware/backtrace.lisp") (:REQUEST-METHOD :GET :SCRIPT-NAME "" :PATH-INFO "/" ...))
... to unexpected end of file:
Thread: 8; Level: 1

Redis error: end of file on #<FLEXI-STREAMS:FLEXI-IO-STREAM {100AF94323}>

  R 0. RECONNECT - Try to reconnect and repeat action.
  R 1. ABORT     - abort thread (#<THREAD "hunchentoot-worker-" RUNNING {1008949253}>)

  F 0.  (REDIS::SET "session:5b391bd4699e86f5b6e757451572c0a3ba825557" "KDpQQ09ERSAxICg6SEFTSC1UQUJMRSAxIDcgMS41IDEuMCBFUVVBTCBOSUwgTklMKSk=")
  F 3.  ((LAMBDA (LACK.MIDDLEWARE.BACKTRACE::ENV) :IN "/Users/matteolandi/Workspace/lack/src/middleware/backtrace.lisp") (:REQUEST-METHOD :GET :SCRIPT-NAME "" :PATH-INFO "/" ...))
A similar error was mentioned on `cl-redis` GitHub's space ([cl-redis/issues#19](https://github.com/vseloved/cl-redis/issues/19)), to which the author replied that one the reasons why this might happen is when trying to share the same connection between multiple threads.  So a took a look the implementation of the Redis store, and it looks like it is indeed sharing the same Redis connection between different HTTP requests (i.e. multiple threads).

The following seems to be fixing the problem though I would like to hear @fukamachi's view on this, before submitting a pull-request:
diff --git a/src/middleware/session/store/redis.lisp b/src/middleware/session/store/redis.lisp
index 9e489d9..3ccfaec 100644
--- a/src/middleware/session/store/redis.lisp
+++ b/src/middleware/session/store/redis.lisp
@@ -54,9 +54,8 @@
 (defun redis-connection (store)
   (check-type store redis-store)
   (with-slots (host port auth connection) store
-    (unless (redis::connection-open-p connection)
-      (setf connection
-            (open-connection :host host :port port :auth auth)))
+    (setf connection
+          (open-connection :host host :port port :auth auth))

 (defmacro with-connection (store &body body)

2021-05-06 (permalink)

Is it possible to imagine a future where “concert programmers” are as common a fixture in the worlds auditoriums as concert pianists? In this presentation Andrew will be live-coding the generative algorithms that will be producing the music that the audience will be listening too. As Andrew is typing he will also attempt to narrate the journey, discussing the various computational and musical choices made along the way. A must see for anyone interested in creative computing.

Andrew Sorensen Keynote: "The Concert Programmer" - OSCON 2014 ([link to video](https://www.youtube.com/watch?v=yY1FSsUV-8c))

2021-05-03 (permalink)

Calling MAKE-INSTANCE with a class name which is only known at run-time

In CL, how can you instantiate a class (i.e. call MAKE-INSTANCE) if you had a string representing its name (and the package it was defined in)?
> (intern "chat.channels.turbo::turbo-channel")

> (intern (string-upcase "chat.channels.turbo::turbo-channel"))

> (read-from-string "chat.channels.turbo::turbo-channel")
At first I gave INTERN ago (I am not exactly sure why though, but that's where my mind immediately went to), but then irrespective of the case of the string I feed into it, it would end up using quote characters (i.e. `|`s).  READ-FROM-STRING on the other hand seemed to get the job done, though I would like to avoid using it as much as possible, especially if the string I am feeding it with is getting generated _externally_ (e.g. sent over a HTTP request).

Well, as it turns out, if you have the strings in `package:symbol` format, then you can parse out the package and symbol names and use FIND-PACKAGE and FIND-SYMBOL to get the symbol you are looking for:
> (find-symbol (string-upcase "turbo-channel") (find-package (string-upcase "chat.channels.turbo")))

> (make-instance *)
PS. @zulu.inuoe on Discord was kind enough to post this utility function that he uses:

this is a thing I use but uhh no guarantees. it also doesn't handle escape characters like | and \

(defun parse-entry-name (name)
  "Parses out `name' into a package and symbol name.

  symbol          => nil, symbol
  package:symbol  => package, symbol
  package::symbol => package, symbol
  (let* ((colon (position #\: name))
         (package-name (when colon (subseq name 0 colon)))
         (name-start (if colon
                         (let ((second-colon (position #\: name :start (1+ colon))))
                           (if second-colon
                               (prog1 (1+ second-colon)
                                 (when (position #\: name :start (1+ second-colon))
                                   (error "invalid entry point name - too many colons: ~A" name)))
                               (1+ colon)))
         (symbol-name (subseq name name-start)))
    (values package-name symbol-name)))

> (multiple-value-bind (package-name symbol-name)
      (parse-entry-name "chat.channels.turbo::turbo-channel")
    (find-symbol (string-upcase symbol-name)
                 (if package-name (find-package (string-upcase package-name)) *package*)))

2021-05-02 (permalink)

Splitting routes into packages, with Caveman

If you use [caveman](https://github.com/fukamachi/caveman) to build your Web applications, sooner or later you will realize that it would not let you create an <APP> instance in one package, and use DEFROUTE in another.  Why? Because the library internally keeps a HASH-TABLE mapping <APP> instances to the package that was active at the time these instances were created, and since DEFROUTE looks up the <APP> instance in this map using *PACKAGE* as key, if the current package is different from the one which was active when <APP> was created then no <APP> instance will be found, and error will be signalled, and you will be prompted to do something about it:
(defmethod initialize-instance :after ((app <app>) &key)
  (setf (gethash *package* *package-app-map*) app))

(defun find-package-app (package)
  (gethash package *package-app-map*))
Of course I was not the first one bumping into this (see [caveman/issues/112](https://github.com/fukamachi/caveman/issues/112)), but unfortunately the one workaround listed in the thread did not seem to work: it's not a matter of carrying the <APP> instance around (which might or might not be annoying), but rather of _changing_ *PACKAGE* before invoking DEFROUTE so that it can successfully look up the <APP> instance inside *PACKAGE-APP-MAP* at macro expansion time.

But clearly you cannot easily change *PACKAGE* before executing DEFROUTE, or otherwise all the symbols used from within the route definition will most likely be unbound at runtime (unless of course such symbols are also accessible from the package in which the <APP> instance was created).  So where do we go from here?  Well, if we cannot _override_ *PACKAGE* while calling DEFROUTE from within a different package, then maybe what we can do is _adding_ *PACKAGE* to *PACKAGE-APP-MAP* and make sure it links to the <APP> instance you already created!
(defun register-routes-package (package)
  (setf (gethash package caveman2.app::*package-app-map*) *web*))
Add a call to REGISTER-ROUTES-PACKAGE right before your first call to DEFROUTE and you should be good to go:
(in-package :cl-user)
(defpackage chat.routes.sessions
  (:use :cl :caveman2 :chat.models :chat.views :chat.routes))
(in-package :chat.routes.sessions)
(register-routes-package *package*)

(defroute "/" ()
  (if (current-user)
    (redirect-to (default-chatroom))
    (render :new :user)))

2021-04-26 (permalink)

On adding Webpack assets to a server-side rendered view (in Common Lisp)

First you need to instruct `webpack` to save compilation metadata such as the paths of the assets being generated, into a file in a well-known location:
// webpack.config.json

class MetaInfoPlugin {
  constructor(options) {
    this.options = { filename: 'meta.json', ...options };

  apply(compiler) {
    compiler.hooks.done.tap(this.constructor.name, stats => {
      const files = {};
      for (let filename of Object.keys(stats.compilation.assets)) {
        const parts = filename.split(/\./);
        const extension = parts[parts.length - 1];
        if (!files[extension]) {
          files[extension] = [ filename ];
        } else {
      const metaInfo = {
      const json = JSON.stringify(metaInfo);
      return new Promise((resolve, reject) => {
        fs.writeFile(this.options.filename, json, 'utf8', error => {
          if (error) {

  plugins: [
    new MetaInfoPlugin({ filename: 'static/dist/meta.json' }),
Run `webpack` and confirm the file is getting generated successfully:
$ cat static/dist/meta.json | json_pp
   "files" : {
      "css" : [
      "js" : [
      "map" : [
Next, somewhere inside your server application, parse the metadata file:
(defparameter *webpack-meta* (merge-pathnames #P"dist/meta.json" *static-directory*))
(defparameter *webpack-assets* (st-json:read-json (uiop:read-file-string *webpack-meta*)))
...define a function to generate proper HTML to include the assets:
(defun webpack-assets ()
    (loop for css in (st-json:getjso* "files.css" *webpack-assets*)
          do (:link :rel "stylesheet" :href (format nil "/dist/~A" css) :data-turbo-track "reload"))
    (loop for js in (st-json:getjso* "files.js" *webpack-assets*)
          do (:script :src (format nil "/dist/~A" js) :data-turbo-track "reload")) ))
...and use it:
(defmacro with-page ((&key (title "Chat!")) &body body)
      (:title ,title)
     (:body ,@body))))
That's it!
<!DOCTYPE html>
<html lang=en>
  <meta charset=UTF-8>
  <link rel=stylesheet
  <script src=/dist/app-8f5ff9499dfe4b9fa887.js

2021-04-16 (permalink)

* finished reading [Smalltalk Best Practice Patterns](https://www.amazon.it/Smalltalk-Best-Practice-Patterns-Kent-dp-013476904X/dp/013476904X)

2021-04-07 (permalink)

Skip "symbol" regions when matching parentheses

Lisp symbols can contain parenthesis when surrounded by vertical bars (i.e. `|Symbol-Name|`), and in case of _unbalanced_ parenthesis matchparen.vim would fail to highlight the "right" pair:
(foo |)-close|)
|     `-- highlighted
`-- cursor position
After "symbol" is added to the list of regions to test against, matchparen.vim will highlight the _expected_ closing pair:
(foo |)-close|)
|             `-- highlighted
`-- cursor position
I know it's very unlikely for one to define a symbol with such a name, but should that ever happen, this fix will make matchparen.vim behave properly.

Anyways, I opened [PR#8079](https://github.com/vim/vim/pull/8079) for this...let's see what Bram and the others think about it.

Looking into a 3 months old PR that I opened to vlime to get it to properly handle char literals like `#\(` or `#\)`

First things first: _disable_ vim-lispindent, or otherwise you are going to start doubting yourself, because unable to reproduce the issue.  Open .vim/pack/iamFIREcracker/start/vim-lispindent/plugin/lispindent.vim, and replace the following lines:
if !empty(g:lispindent_filetypes)
  augroup lispindent_filetypes
    execute 'autocmd FileType ' . g:lispindent_filetypes . ' call lispindent#init()'
  augroup END
if !empty(g:lispindent_filetypes)
  " augroup lispindent_filetypes
  "   autocmd!
  "   execute 'autocmd FileType ' . g:lispindent_filetypes . ' call lispindent#init()'
  " augroup END
Restart vim, have vlime take over lisp files indentation, and you should be good to go (i.e. reproduce the issue the MR is trying to fix).

Now, let's have a look at one of the comments which were left on the PR:

There are some more searchpairpos uses; shouldn't they all use this (or a similar) skip argument? Perhaps we should create one or two helper functions?

These are the functions which make use of `searchpairpos()`:

- `vlime#ui#CurTopExprPos()`
- `vlime#ui#CurOperator()`
- `vlime#ui#SurroundingOperator()`
- `vlime#ui#ParseOuterOperators(3)`
- `vlime#ui#CurArgPos()`

Let's load the following into a buffer and see how each of the above behave:
(defun hello-world()
  (format t "Hello, world!"))

(defun open-parenthesis-p (ch)
  "Some documentation containing unbalanced ("
  (char= ch #\())

(defun close-parenthesis-p (ch)
  "Some documentation containing unbalanced )"
  (char= ch #\)))
Placing the cursor onto the `#\(` character literal, and running `echo vlime#ui#CurTopExprPos('begin')` correctly logs: `[4, 1]`; however, if we pass in `'end'` we get `[10, 17]` instead of the expected `[6, 17]`.  Similarly, if we place the cursor onto the `#\)` character literal and run `echo vlime#ui#CurTopExprPos('end')` we get: `[10, 17]`, but if we use `'begin'` we get `[4, 1]` instead of `[8, 1]`.  Switching to our `searchpairpos()` wrapper seems to be fixing the problem (see [6fe69f3](https://github.com/vlime/vlime/commit/6fe69f330999e392493a3d7164a3b93ca614bb58)).

Placing the cursor onto the `#\(` character literal, and running `echo vlime#ui#CurOperator()` returns an empty string, and interestingly enough adding `skip` to `searchpairpos()` does not seem to make much difference.  Why? Because the function calls `vlime#ui#CurExpr()` which returns `()` instead of the expected `(char= ch #\())`.  Why is `vlime#ui#CurExpr()` returning `()`? Because we are re-using the same logic of matchparen.vim, which intentionally does _not_ try to skip over certain syntax regions if the position the was in at the time the search operation was issued happened to be within one of those syntax regions already (i.e. it will highlight parentheses inside comments).

Question: is this the expected behavior, or should `vlime#ui#CurExpr()` return the actual surrounding expression instead?

In any case, since we have to deal with incomplete expressions anyway, I am proposing that we stop calling `vlime#ui#CurExpr()` here, and simply use the `searchpairpos()` wrapper instead (see [debdd92](https://github.com/vlime/vlime/commit/debdd924d8af2112f3d8a7afad0fe96a0e39f6f9)).

Off to `vlime#ui#SurroundingOperator()`; placing the cursor onto the closing parenthesis right after the `#\(` character literal and calling this function outputs an empty string instead of `char=`.  The same happens if we place the cursor at the closing parenthesis right after the `#\)` character literal of the second function definition.  Using our `searchpairpos()` wrapper here, seems to be fixing the problem (see [dac541a](https://github.com/vlime/vlime/commit/dac541a49414152353516e3c10d7f604479f6aea)).

The next one is `vlime#ui#ParseOuterOperators()`; placing the cursor onto the closing parenthesis right after the `#\(` character literal and calling `vlime#ui#ParseOuterOperators(3)` returns `[['', -1, [6, 15]], ['char=', 2, [6, 3]], ['', 1, [5, 45]]]` instead of the expected `[['char=', 3, [6, 3]], ['defun', 7, [4, 1]]]`.  Again, using our `searchpairpos()` wrapper fixes the problem (see [f8a8777](https://github.com/vlime/vlime/commit/f8a8777eafb3a0cd0a47eeec50dddea2d7278cd7)).

Lastly, for `vlime#ui#CurArgPos()`, let's use these function definitions instead:
(defun open-parenthesis-yoda-p (ch)
  "Some documentation containing unbalanced ("
  (char= #\( ch))

(defun close-parenthesis-yoda-p (ch)
  "Some documentation containing unbalanced )"
  (char= #\) ch))
Placing the cursor onto the `ch` symbol right after the `#\(` character literal and calling `vlime#ui#CurArgPos()` outputs `0` instead of the `2`.  Similarly, doing the same from the `ch` symbol right after the `#\)` character literal outputs `5` instead of `2`.  To fix this, not only we had to use our `searchpairpos()` wrapper, but also change the _parser_ to try to deal with character literals (see [5ea8f32](https://github.com/vlime/vlime/commit/5ea8f32fe74591c2cf42c09adf5629e440ce0027)).

2021-04-06 (permalink)

Smalltalk is one of the simplest, smallest programming languages in the world. Thus, it is supremely easy to learn. Its syntax can fit on a post card!

[Syntax on a Post Card](https://richardeng.medium.com/syntax-on-a-post-card-cb6d85fabf88).

PS. Follows the transcription of that postcard:
exampleWithNumber: x

"A method that illustrates every part of Smalltalk method syntax
except primitives. It has unary, binary, and keyword messages,
declares arguments and temporaries, accesses a global variable
(but not and instance variable), uses literals (array, character,
symbol, string, integer, float), uses the pseudo variables
true false, nil, self, and super, and has sequence, assignment,
return and cascade. It has both zero argument and one argument blocks."

    true & false not & (nil isNil) ifFalse: [self halt].
    y := self size + super size.
    #($a #a "a" 1 1.0)
        do: [:each | Transcript show: (each class name);
                                 show: ' '].
    ^ x < y
PPS. Yes, lately I got curious about Smalltalk, mostly because its highly interactive development experience; having said that, I am not really sold on the idea of being forced into using an IDE to fully _experience_ the language, but who knows, maybe I am going to give it a try...

a system is something you compile and load into the runtime, while a package is something that you access to get a hold of the functions and variables and classes named by symbols inside that package


QL doesn't install packages, it downloads ASDF systems

ASDF then loads these systems, which might create packages in the Lisp image

These were taken from Discord, a while back.  Surely packages, and systems are one of the most confusing things new Common Lisp programmers need to get accustomed to.

Casually chatting on Discord (Lisp/#web) on the reasons why one would try to use React even for a 100% no-JavaScript site:

Yeah, exactly. You can be as dynamic or static as you want, but what I was specifically referring to was to use webpack or whatever else to render the React pages into Plain Old HTML, you can put that on a dumb static file server and it'd Just Work, with no JS required on the client.

You could, on top of that, or instead of it, do as you hinted at before, and have a React engine running server-side which renders React->HTML on-the-fly for a client request. You might choose to do this if you need dynamic (at the time of request) data, but don't want to offload the rendering & IO (to fetch the data) calls to the client

Finally on top of these options, of you can of course serve 'Normal' React (well, compiled into Plain Old Javascript) which is rendered client-side

and if the question is "Why overcomplicate this", it's because you have a lot of freedom of where to put the work, while keeping the same rendering code regardless

It kind of makes sense if you think about it:

- You use React because it will hopefully make it easier to maintain the different pages of the site, even if it's going to be 100% JavaScript free
- In the meanwhile, yourself/the team gets acquaintanced with the piece of technology that most likely you will end up using when adding some sprinkles of JavaScript to the client -- because it will happen...sooner or later you will want to add that to your "app"

Few pointers:

- [Server side rendering with React and NodeJs](https://aparnajoshi.netlify.app/server-side-rendering-with-react-and-nodejs)
- [How to Generate Static Websites with React](https://www.cloudsavvyit.com/5418/how-to-generate-static-websites-with-react/)

? read book: Gödel, Escher, Bach

? read book: leminal thinking

2021-04-05 (permalink)

* finished reading [Structure and Interpretation of Computer Programs](https://mitpress.mit.edu/sites/default/files/sicp/index.html)

2021-03-25 (permalink)

About playing with remote Node.js REPLs; I [recently](https://matteolandi.net/plan.html#day-2021-03-24) said:

I am sure there might be better and more idiomatic ways of dealing with this, but at least it unblocks me for now.

Well, it turns out _there_ is a better way to make sure socket messages are communicated within a given asynchronous context; all you need is a combination of [`Stream.Transform`](https://nodejs.org/api/stream.html#stream_class_stream_transform) and [`Readable.prototype.pipe`](https://nodejs.org/api/stream.html#stream_readable_pipe_destination_options).  From [GitHub](https://github.com/nodejs/node/issues/37866#issuecomment-805578727):

The socket is created before outside of your async scope therefore it's clearly not bound to it.

The repl instance is created within the async scope but it seems it's not modeled as async resource. It operates in caller context which are synchronous calls via socket stream (which is in turn an EventEmitter). So this ends up in the Async context of the socket...


You could try to use a `Transform` stream which passes the data through but switches the async context:

Then the following snippets follow; the custom `Transform` class:
const { Transform } = require("stream");
class MyTransformStream extends Transform {
  constructor(options) {
    this._res = new asyncHooks.AsyncResource("MyTransformStream");
  _transform(chunk, encoding, cb) {
    this._res.runInAsyncScope(cb, null, null, chunk);
and how to use it to "properly" setup the remote REPL:
const myStream = new MyTransformStream();
var repl = require("repl").start({
  input: socket.pipe(myStream), // created within your async scope so ALS propagation works
  output: socket,
Amazing, I can now carry on with my REPL experiments.

2021-03-24 (permalink)

Alright, I think I figured a way to make [this](https://matteolandi.net/plan.html#day-2021-03-23) work by using a combination of `AsyncResoruce.bind` and `Function.prototype.bind`.  First I use `AsyncResource.bind` to force the `socket.on` callback to run inside the "right" asynchronous context, and then I bind the first argument of that _hooked_ callback (i.e. `thisArg`); lastly, I call the _original_ `socket.on` implementation, passing the _hooked_ callback into it.
  .createServer(function (socket) {
    ctx.run((reqId += 1), () => {
      const onOriginal = socket.on;
      socket.on = function onHooked(data, callback) {
        let callbackHooked = asyncHooks.AsyncResource.bind(callback, "REPL");
        // The result of `AsyncResource.bind` is a function expecting
        // its first argument to be `thisArg`, so to properly finish up
        // the wiring we need to bind it before actually using the transformed
        // callback
        callbackHooked = callbackHooked.bind(this, this);
        return onOriginal.apply(socket, [data, callbackHooked]);

      const repl = require("repl").start({ input: socket, output: socket });
      repl.context.ctx = ctx;
      debug("Created repl", ctx.getStore());
debug("Listening on port 8081");
And with this, `AsyncLocalStorage` is not not losing its context anymore.

I am sure there might be better and more idiomatic ways of dealing with this, but at least it unblocks me for now.

2021-03-23 (permalink)

I tried to update to the latest version of Node, but luck, it's still _broken_:
$ node --version
$ node async_bug.js
Listening on port 8081
For the first snippet, the REPL one, from the remote client's terminal:
$ nc localhost 8081
> ctx.getStore()
For the second snippet, the one without the REPL, the one with `socket.on(...)` only:
$ node async_bug.js
Listening on port 8081
Connected 1
[undefined] on-data from-1

Connected 2
[undefined] on-data from-2
I read through the whole [#33723](https://github.com/nodejs/node/issues/33723) thread, and realized that if I create an instance of `AsyncResource` when the connection is established, and run the `socket.on` handler within the context of that resource, then the asynchronous state does not get lost any longer:
  .createServer(function (socket) {
    ctx.run((reqId += 1), () => {
      const res = new asyncHooks.AsyncResource("REPL");
      debug("Connected", ctx.getStore());
      socket.on("data", (data) => {
          () => {
            debug(`[${ctx.getStore()}]`, "on-data", data.toString());
debug("Listening on port 8081");
on the server's stdout:
$ node async_bug.js
Listening on port 8081
Connected 1
[1] on-data from-1

Connected 2
[2] on-data form-2
I quickly hacked the first snippet again, the REPL one, and implemented a man-in-the-middle solution where the socket's input is pumped into the REPL's stdin from inside an `AsyncResource` context, and it's kinda working (i.e. the asynchronous context is _not_ lost, but of course all the evaluation results are dumped on the server's terminal:
  .createServer(function (socket) {
    ctx.run((reqId += 1), () => {
      const res = new asyncHooks.AsyncResource("REPL");
      const repl = require("repl").start({ input: null, output: null });
      socket.on("data", (data) => {
          () => {
            repl.input.emit("data", data);
      repl.context.ctx = ctx;
      debug("Created repl", ctx.getStore());
debug("Listening on port 8081");
If I run this I get:
$ node async_bug.js
Listening on port 8081
> Created repl 1
> Created repl 2
Clearly the _wiring_ left something to be desired, but at least the context was not lost.

I guess I can work-around that ([eventemitter-asyncresource](https://www.npmjs.com/package/eventemitter-asyncresource) maybe?!), but I wonder if it wasn't somehow expected for the created REPL to be bound to the async context of the request.

2021-03-22 (permalink)

`AsyncLocalStorage` losing state when used with `net.Server` and `repl.REPLServer`

I am poking around a tiny wrapper around `repl` to make it possible for someone interact with a running Node.js instance, remotely,  but when I tried to use `AsyncLocalStorage` to attach some state to the active session, such state gets gets lost as soon as the `REPLServer` takes over.

Save the following into a file:
var asyncHooks = require("async_hooks");
var fs = require("fs");
var util = require("util");
var net = require("net");

function debug(...args) {
  fs.writeFileSync(1, `${util.format(...args)}\n`, { flag: "a" });

var ctx = new asyncHooks.AsyncLocalStorage();
var reqId = 0;

  .createServer(function (socket) {
    ctx.run((reqId += 1), () => {
      var repl = require("repl").start({
        input: socket,
        output: socket,
      repl.context.ctx = ctx;
      debug("Created repl", ctx.getStore());
debug("Listening on port 8081");
and run it:
$ node remote-repl.js
Listening on port 8081
Now from a different terminal, connect to port `8081` to get your remote REPL:
$ nc localhost 8081
Now, accessing `ctx` (well, its content) from the REPL should return the ID of the REPL itself (i.e. an increasing number), but `undefined` is returned instead:
> ctx
AsyncLocalStorage {
  kResourceStore: Symbol(kResourceStore),
  enabled: true
> ctx.getStore()
I suspect it might have something to do with the underlying socket or `EventEmitter`, because when I try to access the content of `ctx` from within a `socket.on('data'...)` block, I still get `undefined`.
  .createServer(function (socket) {
    ctx.run((reqId += 1), () => {
      debug("Connected", ctx.getStore());
      socket.on("data", (data) => {
        debug(`[${ctx.getStore()}]`, "on-data", data.toString());
I filed a bug for this, [node/#37866](https://github.com/nodejs/node/issues/37866), so let's wait to hear back from the core developers.

2021-03-19 (permalink)

for each desired change, make the change easy (warning: this may be hard), then make the easy change

Kent Beck, on [Twitter](https://twitter.com/KentBeck/status/250733358307500032)

2021-03-14 (permalink)

? add page on Netmap to matteolandi.net

2021-03-12 (permalink)

Dynamic variables in Node.js (part 2)

The Node.js implementation of dynamic variables that I presented few weeks ago was a bit...lacking, as it did not support for an _external_ actor to interact with (and change) the dynamic environment of a _running_ application.

Imagine if your application exposed a REPL you could [remotely connect to](https://nodejs.org/en/docs/guides/debugging-getting-started/#enabling-remote-debugging-scenarios); now, if you had all the application settings loaded into the dynamic environment, wouldn't it be amazing if you could change those while the application is running and immediately see the effects of such change?  So do all that, connect to the REPL, run `var { env } = require('dynamic_variables.js')`, only to discover that the actual environment is empty...

That's a bit of a bummer, but it also kind of makes sense if you think about it: all the dynamic bindings are done asynchronously, and when you connect to the REPL you are starting off a completely unrelated execution context, and because of that: a) the dynamic environment turns out to be empty, b) there is no way for you to change it and give the application access to the newly created bindings.

To _fix_ this, we will have to change the internal representation of a dynamic environment, and define it in terms of a stack of _frames_, each containing a set of bindings: a _global_ frame, containing all the _top-level_ bindings (i.e. the ones happening outside of any asynchronous context, when the application boots up); and then a list of _dynamic_ frames to keep track of all the asynchronous bindings.

If we want an _external_ actor to interact with the running application and change its dynamic environment, we will also need a way of _mutating_ existing bindings, living in one of the existing frames, without having to create a new one.  In particular, when run by the external actor this new bind operation is going to affect the global frame; but when run inside an asynchronous context it's going to affect the frames in which those bindings were most recently defined / re-defined.  Hopefully the following Common Lisp snippet will help you understand what I am talking about:
(defparameter *x* 5)

(defun rebind ()
  (setf *x* 15))

(defun test1 ()
  "We create a new _frame_ for *x*, so all the SETF operations affect that"
  (let ((*x* 10))
    (assert (= *x* 15)))
  (assert (= *x* 5)))

(assert (= *x* 5))

(defun test2 ()
  "LET here does not create a new _frame_ for *x*, so SETF ends up actually
  changing the _global_ value"
  (let ((whatever 10))
    (assert (= *x* 15)))
  (assert (= *x* 15)))

(assert (= *x* 15))

(defun test3 ()
  "We don't even create a new _frame_ for *x*...so this behaves like TEST2"
  (assert (= *x* 15)))

(setf *x* 5)
(assert (= *x* 15))
Alright then, let's get to it.  We said the new `DynamicEnvironment` was going to need to keep track of _global_ bindings, as well as _dynamic_ ones; for the first we are going to be using an instance of `Bindings` (i.e. a wrapper around `Map`), while for dynamic ones we will be using an array of `Bindings` all wrapped up inside an instance of `AsyncLocalStorage`:
var DynamicEnvironment = function (...flatBindings) {
  this.globalFrame = new Bindings(parseKVPairs(flatBindings));
  this.dynamicFrames = new AsyncLocalStorage();
Note 1: we are not passing `[]` to `AsyncLocalStorage constructor` because reading it back from an _external_ process will result in an _empty_ context, and if we have to deal with `undefined` values anyway then why bother initializing it in the first place?!
Note 2: the `Bindings` type used here is a little bit different from the one I showed [last time](https://matteolandi.net/plan.html#day-2021-02-28); it has a new `Bindings.prototype.has` method to check if a binding for a given variable exists, its `Bindings.prototype.set` method is now a _destructive_ one (i.e. it mutates the current object), it implements the [iteration protocol](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Iteration_protocols) (it's going to come in handy later on):
var Bindings = function (kvpairs) {
  this.data = new Map(kvpairs);
Bindings.prototype.get = function (name) {
  if (!this.has(name)) {
    throw new Error(`Dynamic variable, unbound: '${name.toString()}'`);
  return this.data.get(name);
Bindings.prototype.has = function (name) {
  assert.ok(name, `Dynamic variable name, invalid: ${name.toString()}`);
  return this.data.has(name);
Bindings.prototype.set = function (key, value) {
  return this.data.set(key, value);
Bindings.prototype[Symbol.iterator] = function () {
  return this.data[Symbol.iterator]();
Looking up a binding inside this new environment now translates to finding first the frame, dynamic or global, that _last_ defined a binding for that variable, and then pull the binding value out of it.
DynamicEnvironment.prototype.get = function (name) {
  return this.findBindingFrameOrGlobal(name).get(name);
DynamicEnvironment.prototype.findBindingFrameOrGlobal = function (name) {
  let frame;
  for (let each of this.dynamicFrames.getStore() || []) {
    if (each.has(name)) {
      frame = each;
  return frame || this.globalFrame;
On the topic of setting a new binding: we previously said we wanted to enable users to _create_ new bindings (i.e. create a new frame), as well as _update_ existing ones, as that would enable an external actor to interact with a running application.  To do that, instead of creating two separate methods, we will be having `DynamicEnvironment.prototype.set` check the arguments it receives and figure out what to do based on its length: if it receives an even number of arguments, it means the user specified new bindings only with no body function, in which case the method will proceed and update existing frames; if on the other hand the method is provided with a body function (i.e. odd number of arguments), then a new frame with the specified bindings is going to be added to the stack of frames, and the body function will be executed inside that new dynamic context:
DynamicEnvironment.prototype.set = function (...args) {
  const updatesExistingFrames = args.length % 2 === 0;
  if (updatesExistingFrames) {
    for (let [key, value] of parseKVPairs(args)) {
      this.findBindingFrameOrGlobal(key).set(key, value);
  } else {
    const kvpairs = parseKVPairs(args.slice(0, args.length - 1));
    const body = args[args.length - 1];
    const bindings = new Bindings(kvpairs);
    return this.dynamicFrames.run(
      [bindings, ...(this.dynamicFrames.getStore() || [])],
Lastly I figured it would be nice to make it possible to iterate over all the bindings of a dynamic environment, so the following implements that:
DynamicEnvironment.prototype[Symbol.iterator] = function () {
  const bindings = new Map(this.globalFrame);
  for (let frame of (this.dynamicFrames.getStore() || []).reverse()) {
    for (let [key, value] of frame) {
      bindings.set(key, value);
  return bindings[Symbol.iterator]();
Alright, let's test this out and see if it works.  We are going to create a timer, and inside of it we are going to be logging the content of the dynamic environment:
var env = new DynamicEnvironment('debug', false);

function timer() {

var tid = setInterval((...args) => timer(...args), 2000);
(You should now start seeing a bunch of `false` messages getting logged in the REPL, every 2 seconds)

Next we are going to listen for incoming connections on port `5001`, and every time we receive one, set up a _remote_ REPL to let the external actor play with the running environment:
var net = require("net");
var repl = require("repl");

function createREPL(socket) {
  console.log("Connection received; creating a REPL!");
  const remote = repl.start({
    prompt: "remote> ",
    input: socket,
    output: socket,
    useGlobal: true,

var server = net.createServer((...args) => createREPL(...args));
server.listen(5001, "localhost");
console.log("Remote REPL started on port 5001");
Let's poke around and see what we can do with it:
$ nc localhost 5001
remote> env
DynamicEnvironment {
  globalFrame: Bindings { data: Map(1) { 'debug' => false } },
  dynamicFrames: AsyncLocalStorage {
    kResourceStore: Symbol(kResourceStore),
    enabled: false
It looks like we can get hold of the dynamic environment (well, its global frame at least); let's try and change it:
remote> env.set('debug', true)
If all worked as expected, you should now be seeing a series of `true` messages getting logged in the REPL; _that_ means were successfully able to poke with one application's dynamic environment, from the outside.

"You could have easily done that with global variables" one could argue, and they would be right, we could have.  Let's mix in some more dynamic bindings, to show what the system is capable of.

First we reset the environment and add an additional binding, then we change our timer to create some dynamic bindings and finally print the whole dynamic environment:
env.set('logfile', '/tmp/server.log', 'debug', false);

function timer() {
  env.set("logfile", "/tmp/different.log", "date", new Date(), () =>
    setTimeout(() => {
    }, 500)
The messages logged in the REPL should now look like the following:
[Map Entries] {
  [ 'debug', false ],
  [ 'logfile', '/tmp/different.log' ],
  [ 'date', 2021-03-12T19:37:59.963Z ]
Let's go back to the remote REPL again, and change the value of the `debug` binding again:
remote> env.set('debug', true)
Wait a couple of seconds, and the messages logged on the REPL should now look like this:
[Map Entries] {
  [ 'debug', true ],
  [ 'logfile', '/tmp/different.log' ],
  [ 'date', 2021-03-12T19:40:54.317Z ]
As we can see the dynamic environment from within the `setTimeout` handler properly reflects the changes done from the _remote_ REPL, as well as the ones triggered by the dynamic bind operation.

And that's it!  I have pushed all the above on [GitHub](https://github.com/iamFIREcracker/dynamic_variables.js), and in there I also added an [example](https://github.com/iamFIREcracker/dynamic_variables.js/tree/master/examples/express) showing how dynamic variables can be be used within an Express.js application.

Till next time.

2021-03-11 (permalink)

During World War II, the Navy realized it was losing a lot of aircraft and could better armor its planes to increase their survival. After analyzing where its planes had suffered the most damage, it determined that it needed to reinforce the planes’ wingtips, central body and elevators.

But a statistician named Abraham Wald argued otherwise. He thought the Navy should reinforce the armor of the planes’ nose, engines and mid-body. But why would he suggest that when the planes were taking more damage to the wingtips, central body and elevators? Because in reality, they weren’t. The planes getting shot in the nose area, engines and mid-body were being destroyed from the damage and weren’t making it back to be analyzed.

The Navy thought it had discovered where its planes were suffering the most damage. Instead, it had discovered where its planes could take damage without being destroyed. It wasn’t looking at the whole sample set.

From [Finding the Missing Bullet Holes](https://onebiteblog.com/finding-the-missing-bullet-holes/)

2021-03-05 (permalink)

Am I the only one having a hard time understanding the implementation of Data-Directed Programming, presented in SICP?

A little bit of background first; we are shown how to _tag_ data ([2.4.2 Tagged data](https://mitpress.mit.edu/sites/default/files/sicp/full-text/book/book-Z-H-17.html#%_sec_2.4.2)):
(define (attach-tag type-tag contents)
  (cons type-tag contents))
(define (type-tag datum)
  (if (pair? datum)
      (car datum)
      (error "Bad tagged datum -- TYPE-TAG" datum)))
(define (contents datum)
  (if (pair? datum)
      (cdr datum)
      (error "Bad tagged datum -- CONTENTS" datum)))
...how to implement type predicates:
(define (rectangular? z)
  (eq? (type-tag z) 'rectangular))
(define (polar? z)
  (eq? (type-tag z) 'polar))
...how to implement _packages_:
(define (real-part-rectangular z) (car z))
(define (imag-part-rectangular z) (cdr z))
(define (magnitude-rectangular z)
  (sqrt (+ (square (real-part-rectangular z))
           (square (imag-part-rectangular z)))))
(define (angle-rectangular z)
  (atan (imag-part-rectangular z)
        (real-part-rectangular z)))
(define (make-from-real-imag-rectangular x y)
  (attach-tag 'rectangular (cons x y)))
(define (make-from-mag-ang-rectangular r a)
  (attach-tag 'rectangular
              (cons (* r (cos a)) (* r (sin a)))))

;; similar definitions for the "polar" representation
...and how to implement _generic_ selectors:
(define (real-part z)
  (cond ((rectangular? z)
         (real-part-rectangular (contents z)))
        ((polar? z)
         (real-part-polar (contents z)))
        (else (error "Unknown type -- REAL-PART" z))))
(define (imag-part z)
  (cond ((rectangular? z)
         (imag-part-rectangular (contents z)))
        ((polar? z)
         (imag-part-polar (contents z)))
        (else (error "Unknown type -- IMAG-PART" z))))
(define (magnitude z)
  (cond ((rectangular? z)
         (magnitude-rectangular (contents z)))
        ((polar? z)
         (magnitude-polar (contents z)))
        (else (error "Unknown type -- MAGNITUDE" z))))
(define (angle z)
  (cond ((rectangular? z)
         (angle-rectangular (contents z)))
        ((polar? z)
         (angle-polar (contents z)))
        (else (error "Unknown type -- ANGLE" z))))
Clearly it's not OK for REAL-PART, or IMAG-PART, or any other _generic_ selector, to require change every time a new implementation is added to the system, and this is where "Data-directed programming" is going to come to the rescue.

The idea ([2.4.3 Data-Directed Programming and Additivity](https://mitpress.mit.edu/sites/default/files/sicp/full-text/book/book-Z-H-17.html#%_sec_2.4.3)) is to construct a table mapping operations (e.g. REAL-PART, IMAG-PART) and types (e.g. POLAR, RECTANGULAR) to the actual function in charge of fulfilling the user need, and of course to provide a way to pluck the specific implementation out of the table.  In particular:

- `(put <op> <type> <item>)` will install `<item>` in the table, indexed by `<op>` and `<type>`
- `(get <op> <type>)` will look up the `<op>,<type>` entry in the table

Note: we are not given the actual implementations of PUT and GET.

With this in mind, we can then re-define the rectangular package as follows:
(define (install-rectangular-package)
  ;; internal procedures
  (define (real-part z) (car z))
  (define (imag-part z) (cdr z))
  (define (make-from-real-imag x y) (cons x y))
  (define (magnitude z)
    (sqrt (+ (square (real-part z))
             (square (imag-part z)))))
  (define (angle z)
    (atan (imag-part z) (real-part z)))
  (define (make-from-mag-ang r a)
    (cons (* r (cos a)) (* r (sin a))))
  ;; interface to the rest of the system
  (define (tag x) (attach-tag 'rectangular x))
  (put 'real-part '(rectangular) real-part)
  (put 'imag-part '(rectangular) imag-part)
  (put 'magnitude '(rectangular) magnitude)
  (put 'angle '(rectangular) angle)
  (put 'make-from-real-imag 'rectangular
       (lambda (x y) (tag (make-from-real-imag x y))))
  (put 'make-from-mag-ang 'rectangular
       (lambda (r a) (tag (make-from-mag-ang r a))))
Here is where things get a bit confusing: why, we are _registering_ REAL-PART for the `(rectangular)` type, and not simply `'rectangular` like we are doing for MAKE-FROM-REAL-IMAG?

The only explanation I could think of is that we are giving the `<type>` argument of the PUT call different meanings:

- For REAL-PART, `<type>` represents the list of the types of the arguments expected by the registered procedure (i.e. one argument, of type RECTANGULAR)
- For MAKE-FROM-REAL-IMAG instead, `<type>` represents the type of the _returned_ instance

And why would we do that?  Because otherwise it would not be possible to dispatch to the "Right" implementation in case of generic functions with the same argument types (both the implementation of MAKE-FROM-REAL-IMAG inside the rectangular and polar packages expects 2 arguments of type NUMBER).

Anyways, a similar package for the polar representation is presented, and then finally we are shown the implementation of APPLY-GENERIC, the procedure responsible for invoking the "Right" procedure based on the _types_ of the arguments of the dispatched action:
(define (apply-generic op . args)
  (let ((type-tags (map type-tag args)))
    (let ((proc (get op type-tags)))
      (if proc
          (apply proc (map contents args))
            "No method for these types -- APPLY-GENERIC"
            (list op type-tags))))))
Here more doubts come to my mind: how can we use this with the MAKE-FROM-REAL-IMAG?

Clearly we cannot simply run `(apply-generic 'make-from-real-imag 4 2)`, as that would fail when trying to apply TYPE-TAG to `(4 2)`.  I thought, maybe we pass `(attach-tag 'rectangular (list 4 2))` to APPLY-GENERIC, but then `(map contents args)` would evaluate to `((4 2))` and that is incompatible with the registered procedure, which expects two numbers and not a list of 2 numbers, right?

So where do we go from here?  There _has_ to be a way, but I just cannot find it.

Posted this on [/r/sicp](https://www.reddit.com/r/sicp/comments/ly7wj3/datadirected_programming_and_how_to_invoke/), let's see if anyone can help me out.

2021-02-28 (permalink)

Dynamic variables in Node.js

For those of you unfamiliar with it, Common Lisp has this concept of "dynamic variables", which are nothing more than _global_ variables with _dynamic_ scope; but what does it mean for a variable to have dynamic scope? That you can bind a new a value to it, and all the _subsequent_ accesses to that variable (within the scope of the binding operation), will return _that_ new value instead of the _previous_ one.  Let's take a look at an example, as I hope it will make things a little bit easier to understand (the asterisks around a variable name indeed mark the variable as dynamic / special):
(defvar *x* 5)

(defun foo ()

(defun bar ()
  (let ((*x* 42))

> *x*

> (foo)

> (bar)

> *x*
Step by step explanation:

- `*x*` was initially bound to `5`, so when we access it we get `5` back -- all good
- We call FOO, which all it does is returning the value of `*x*`, and without much surprise we get `5` back -- so all good
- We call BAR, which binds `*x*` to `42` before calling FOO and returning its return value, and interestingly enough we now get back `42` instead of `5` -- there it is the _dynamic_ scope I was talking about earlier
- Lastly, we access `*x*` and we get `5` back -- we are outside BAR's new binding scope, so the value of `*x*` has been restored to its previous value

Let's go back the definition that I gave earlier: _global_ variables, with _dynamic_ scope.  I know that the first rule of global variables is: "thou shalt not use global variables", but it's just that _sometimes_ they appear to be right tool for the job, especially in the context of Web applications; think about the following use cases:

- getting a hold of the currently logged in user
- automatically adding the content of the `X-Request-Id` header to each log trace
- querying the _right_ database, based on the logged in user's tenant

How would you implement these?  Either you shove all this information up into a context object, and pass it around, _everywhere_; or maybe you forget about all the bad things you read about global variables, and consciously and carefully agree to use them where it really matters, where it really makes a difference.

Anyways, enough talking about global variables, I do not want this to be an essay about its pros or cons (I am sure the Internet is full of material and opinions about it); instead, let's go back to our original goal: trying to implement dynamic variables in Node.js.

It all begins with the definition of a type representing a bag of bindings (we are going to wrap the standard `Map` type for this):
var Bindings = function (kvpairs) {
  this.data = new Map(kvpairs);

> new Bindings().data
Map(0) {}

> new Bindings([['a', 1]]).data
Map(1) { 'a' => 1 }

> new Bindings([['a', 1], ['b', 2]]).data
Map(2) { 'a' => 1, 'b' => 2 }
(Don't worry about that _ugly_ syntax for now, we will deal with it later)

Getting a binding (i.e. getting the value bound to a variable), should be as easy as calling `Map.prototype.get` (plus some additional logic to validate user input):
Bindings.prototype.get = function (name) {
  assert.ok(name, `Dynamic variable name, invalid: ${name}`);
  if (!this.data.has(name)) {
    throw new Error(`Dynamic variable, unbound: '${name}'`);
  return this.data.get(name);
First we check that the name of the variable is indeed _valid_, then we confirm that a binding for that variable actually exists, and finally we return the value bound to that variable.  Let's play with it an confirm it's all working fine:
var bb = new Bindings([['a', 1], ['b', 2]])

> bb.get('a')

> bb.get('b')

> bb.get('c')
Uncaught Error: Dynamic variable, unbound: 'c'
    at getBinding (repl:4:11)

> bb.get()
Uncaught AssertionError [ERR_ASSERTION]: Dynamic variable name, invalid: undefined
    at getBinding (repl:2:10)
    at repl:1:1
    at Script.runInThisContext (vm.js:131:18)
    at REPLServer.defaultEval (repl.js:472:29)
    at bound (domain.js:430:14)
    at REPLServer.runBound [as eval] (domain.js:443:12)
    at REPLServer.onLine (repl.js:794:10)
    at REPLServer.emit (events.js:326:22)
    at REPLServer.EventEmitter.emit (domain.js:486:12)
    at REPLServer.Interface._onLine (readline.js:337:10) {
  generatedMessage: false,
  code: 'ERR_ASSERTION',
  actual: undefined,
  expected: true,
  operator: '=='
Lastly, to set new bindings, we will go about and create a new `Binding` object and initialize it with the existing bindings and the newly defined ones..._merged_ together:
Bindings.prototype.set = function (kvpairs) {
  return new Bindings([...this.data, ...kvpairs]);
Again, let's can play with this to confirm that it's all working as expected:
var bb = new Bindings([['a', 1], ['b', 2]])

> bb.set([['c', 3]]).data
Map(3) { 'a' => 1, 'b' => 2, 'c' => 3 }

> bb.set([['a', 3]]).data
Map(3) { 'a' => 3, 'b' => 2 }

> bb.data
Map(2) { 'a' => 1, 'b' => 2 }
All good, great!  Before we dive into the details of the implementation of dynamic variables, let's first implement a couple of functions which will come in handy down the line.  The first one is to simplify the _syntax_ for creating new bindings; we don't want users to specify new bindings via nested lists (i.e. `[['a', 1], ['b', 2]]`); instead, we would like them to use a _flattened_ list instead (i.e. `['a', 1, 'b', 2]`):
function parseKVPairs(flatBindings) {
    flatBindings.length % 2 === 0,
    `Bindings arguments, expected even number of elements, but got: ${flatBindings.length}`

  const kvpairs = [];
  for (var i = 0; i < flatBindings.length; i += 2) {
    kvpairs.push(flatBindings.slice(i, i + 2));
  return kvpairs;
Nothing crazy about this: we first confirm that the number of bindings is indeed even, and then wrap every pair of adjacent elements into a nested list.  Let's give it a go:
> parseKVPairs([])

> parseKVPairs(['a', 1])
[ [ 'a', 1 ] ]

> parseKVPairs(['a', 1, 'b', 2])
[ [ 'a', 1 ], [ 'b', 2 ] ]
Perfect!  The second utility function might look a bit cryptic at first, simply because I am yet to show you what is the problem that it tries to solve, but hopefully soon it will all make more sense.  We want to feed it with a list of elements representing a flattened list of bindings followed by a callback function, and we expect it to return a list whose first element is a list of key-value pairs created from the list of flattened bindings, and the second is the given callback function:
function parseDynamicEnvironmentSetArguments(args) {
  assert.ok(args, `Function arguments, invalid: ${args}`);
    args.length % 2 === 1,
    `Function arguments, expected odd number of elements, but got: ${args.length}`

  const kvpairs = parseKVPairs(args.slice(0, args.length - 1));
  const body = args[args.length - 1];

  return [kvpairs, body];
Let's test it out:
> parseDynamicEnvironmentSetArguments([() => {}])
[ [], [Function (anonymous)] ]

> parseDynamicEnvironmentSetArguments(['a', 1, () => {}])
[ [ [ 'a', 1 ] ], [Function (anonymous)] ]

> parseDynamicEnvironmentSetArguments(['a', 1, 'b', 2, () => {}])
[ [ [ 'a', 1 ], [ 'b', 2 ] ], [Function (anonymous)] ]
Alright, it's all working as expected, and with all of this defined and taken care for, it's time we took a look at a possible implementation for a _dynamic environment_, i.e. environment getting a hold of a bunch of dynamic variables.

The biggest challenge in implementing dynamic variables in Node.js is figuring out a way to persist _state_ across changes of asynchronous context: you set `*x*` to a `5`, invoke `setTimeout`, and when the callback is invoked you expect `*x*` to still be bound to `5`.  Similarly, if two asynchronous operations happen to re-bind the same dynamic variable, you don't any of them to step on each others toes.

Luckily for us, the Node.js core team has been working on this _problem_ for quite some time now, and you can see the result of their effort in the [`async_hooks`](https://nodejs.org/api/async_hooks.html) module.  I am not going to bore you with its implementation details (mostly because I am not familiar with it myself), but for what we are trying to achieve here, all we need to know is that:

- Each piece of running code (_user_ code), can have an ID attached, identifying its asynchronous execution context
- Each piece of running code (_user_ code), can have another ID attached, identifying the asynchronous context that _triggered_, directly or indirectly, the current one (i.e. if you create three nested promises, each callback, when executed, will probably have a different `asyncId` value but same `triggerAsyncId` one)
- There is a low-level API, [`createHooks`](https://nodejs.org/api/async_hooks.html#async_hooks_async_hooks_createhook_callbacks), that can be used to get notified when an asynchronous execution context is created or destroyed; with it, one could think of attaching some _payload_ to the current execution context, and then expose another API for user code to access it
- There is a high-level API, [`AsyncLocalStorage`](https://nodejs.org/api/async_hooks.html#async_hooks_class_asynclocalstorage), that shields the user from all the above complexity, and offers a simple way of running user code with a given piece of _payload_ attached to the current execution context

It goes without saying it that `AsyncLocalStorage` is what we will use to implement our dynamic environment:

- Getting a binding translates to getting a hold of the current execution context's payload (i.e. the bindings), and returning whichever value is currently bound to the given variable name
- Setting a binding translates to creating a new set of bindings, attaching it to the current execution context, and running user code within it -- old bindings will be automatically restored after user code (synchronous or asynchronous) has finished running

Alright, let's get our hands dirty.  Let's start by creating a new type for the dynamic environment:
var { AsyncLocalStorage } = require("async_hooks");

var DynamicEnvironment = function (...flatBindings) {
  this.ctx = new AsyncLocalStorage();
  this.ctx.enterWith(new Bindings(parseKVPairs(flatBindings)));
Here, all we do, is creating the asynchronous context object (i.e. an instance of `AsyncLocalStorage`), and then initialize it with some user defined bindings (e.g. `'a', 1, 'b', 2`).  Let's give it a go to see what happens when we call the constructor (note: `ctx.getStore()` is how you access the _payload_ of the current asynchronous context):
> new DynamicEnvironment().ctx.getStore()
Bindings { data: Map(0) {} }

> new DynamicEnvironment('a', 1).ctx.getStore()
Bindings { data: Map(1) { 'a' => 1 } }

> new DynamicEnvironment('a', 1, 'b', 2).ctx.getStore()
Bindings { data: Map(1) { 'a' => 1, 'b' => 2 } }
Let's now define a method to get the value of a specific binding (note how `Bindings`, our previously defined type, is doing all the heavy lifting here):
DynamicEnvironment.prototype.get = function (name) {
  return this.ctx.getStore().get(name);

var env = new DynamicEnvironment('a', 1, 'b', 2)

> env.get('a')

> env.get('b')

> env.get('c')
Uncaught Error: Dynamic variable, unbound: 'c'
    at Bindings.get (repl:4:11)
    at DynamicEnvironment.get (repl:2:30)

> env.get()
Uncaught AssertionError [ERR_ASSERTION]: Dynamic variable name, invalid: undefined
    at Bindings.get (repl:2:3)
    at DynamicEnvironment.get (repl:2:30)
    at repl:1:5
    at Script.runInThisContext (vm.js:131:18)
    at REPLServer.defaultEval (repl.js:472:29)
    at bound (domain.js:430:14)
    at REPLServer.runBound [as eval] (domain.js:443:12)
    at REPLServer.onLine (repl.js:794:10)
    at REPLServer.emit (events.js:326:22)
    at REPLServer.EventEmitter.emit (domain.js:486:12) {
  generatedMessage: false,
  code: 'ERR_ASSERTION',
  actual: undefined,
  expected: true,
  operator: '=='
The last piece of the puzzle, is a mean of setting a new binding (or bindings), and run some user code within the scope of these new bindings; thanks to `Bindings`, `AsyncLocalStorage`, and the cryptic `parseDynamicEnvironmentSetArguments` I showed you before, this could not have been any easier to implement:
DynamicEnvironment.prototype.set = function (...args) {
  const [kvpairs, body] = parseDynamicEnvironmentSetArguments(args);
  const bindings = this.ctx.getStore().set(kvpairs);
  return this.ctx.run(bindings, body);
First we parse function arguments into key-value pairs and the callback inside of which the new bindings will be active; then we create a new `Bindings` object merging new bindings with any existing ones; lastly we tell `AsyncLocalStorage` to _do its magic_ (i.e. attach new bindings to the execution context, and run user code).  Let's try this out, and see if it works or not:
async function test(body) {
  try {
    await body();
  } catch (err) {

> test(async () => {
  var env = new DynamicEnvironment("x", 5);

  var foo = function () {
    return env.get("x");

  var bar = function () {
    return env.set("x", 42, () => foo());

  assert.equal(env.get("x"), 5);
  assert.equal(foo(), 5);
  assert.equal(await bar(), 42);
  assert.equal(env.get("x"), 5);
Promise { <pending> }
It seems like it _is_ indeed working; but what if we added some asynchronous operations within the scope of the `set` call?
> test(async () => {
  var env = new DynamicEnvironment("x", 5);

  var foo = function () {
    return env.get("x");

  var bar = function () {
    return env.set("x", 42, () => {
      return new Promise((resolve) => {
        setTimeout(() => resolve(foo()), 2000);

  assert.equal(env.get("x"), 5);
  assert.equal(foo(), 5);
  assert.equal(await bar(), 42);
  assert.equal(env.get("x"), 5);
Promise { <pending> }
Still working, great!  What about multiple asynchronous operations at the same time?
> test(async () => {
  var env = new DynamicEnvironment("x", 5);

  var foo = function () {
    return env.get("x");

  var bar = function () {
    return env.set("x", 42, () => {
      return Promise.all([
          () =>
            new Promise((resolve) => {
              setTimeout(() => resolve(foo()), 1000);
          () =>
            new Promise((resolve) => {
              setTimeout(() => resolve(foo()), 2000);

  assert.equal(env.get("x"), 5);
  assert.equal(foo(), 5);
  assert.deepEqual(await bar(), [42, 52, 72]);
  assert.equal(env.get("x"), 5);
Promise { <pending> }
It works, and by the look of it it appears we were indeed able to implement "dynamic variables" in Node.js.

I added all the above into a new repository, [`dynamic_variables.js`](https://github.com/iamFIREcracker/dynamic_variables.js), so feel free to play with it in your REPLs and do let me know if anything breaks for you.

Also, it's worth remembering that `async_hooks` is still considered _experimental_, so its API might suddenly change with a new release of Node.js; well, that and the fact that the current implementation might still contain some nasty bugs that might cause your dynamic bindings to get lost across switches of execution context.  This might not be a big deal if you were just to messing around with this, or if you were just planning to use this to enhance your logging capabilities; but if instead, you were planning anything more _serious_ than that, like selecting the "right" database connection based on the logged-in user's tenant, then I would strongly recommend that you tested as many execution paths as possible and confirmed that no binding got lost in the process.  You know, it works...until it doesn't!

PS. For educational purposes, here I am going to show you a different implementation of a dynamic environment, one that does _not_ use `AsyncLocalStorage` to keep track of re-binds (it does that with a stack of _active_ bindings) and because of that, one that most surely is going to fail the expectations in case of multiple nested, asynchronous, re-binds:
var UnsafeDynamicEnvironment = function (...flatBindings) {
  this.snapshots = [new Bindings(parseKVPairs(flatBindings))];
UnsafeDynamicEnvironment.prototype.get = function (name) {
  return this._getActiveSnapshot().get(name);
UnsafeDynamicEnvironment.prototype.set = function (...args) {
  const [kvpairs, body] = parseDynamicEnvironmentSetArguments(args);
  const bindings = this._getActiveSnapshot().set(kvpairs);
  return this._runWithBindings(bindings, body);
UnsafeDynamicEnvironment.prototype._getActiveSnapshot = function () {
  return this.snapshots[this.snapshots.length - 1];
UnsafeDynamicEnvironment.prototype._runWithBindings = async function (bindings, body) {
  try {
    return await body();
  } finally {

> test(async () => {
  var env = new UnsafeDynamicEnvironment("x", 5);

  var foo = function () {
    return env.get("x");

  var bar = function () {
    return env.set("x", 42, () => foo());

  assert.equal(env.get("x"), 5);
  assert.equal(foo(), 5);
  assert.equal(await bar(), 42);
  assert.equal(env.get("x"), 5);
Promise { <pending> }

> test(async () => {
  var env = new UnsafeDynamicEnvironment("x", 5);

  var foo = function () {
    return env.get("x");

  var bar = function () {
    return env.set("x", 42, () => {
      return new Promise((resolve) => {
        setTimeout(() => resolve(foo()), 2000);

  assert.equal(env.get("x"), 5);
  assert.equal(foo(), 5);
  assert.equal(await bar(), 42);
  assert.equal(env.get("x"), 5);
Promise { <pending> }

> test(async () => {
  var env = new UnsafeDynamicEnvironment("x", 5);

  var foo = function () {
    return env.get("x");

  var bar = function () {
    return env.set("x", 42, () => {
      return Promise.all([
          () =>
            new Promise((resolve) => {
              setTimeout(() => resolve(foo()), 1000);
          () =>
            new Promise((resolve) => {
              setTimeout(() => resolve(foo()), 2000);

  assert.equal(env.get("x"), 5);
  assert.equal(foo(), 5);
  assert.deepEqual(await bar(), [42, 52, 72]);
  assert.equal(env.get("x"), 5);
Promise { <pending> }
AssertionError [ERR_ASSERTION]: Expected values to be loosely deep-equal:


should loosely deep-equal

    at repl:34:14
    at async test (repl:3:9) {
  generatedMessage: true,
  code: 'ERR_ASSERTION',
  actual: [Array],
  expected: [Array],
  operator: 'deepEqual'

2021-02-26 (permalink)

? book: [The Web Application Hacker's Handbook: Finding and Exploiting Security Flaws: Discovering and Exploiting Security Flaws](https://www.amazon.it/Web-Application-Hackers-Handbook-Exploiting/dp/1118026470)

2021-02-19 (permalink)

+ book: [Smalltalk Best Practice Patterns](https://www.amazon.it/Smalltalk-Best-Practice-Patterns-Kent-dp-013476904X/dp/013476904X)

? book: [Refactoring](https://martinfowler.com/books/refactoring.html)

If your Mac refuses to go to sleep, use `pmset -g assertions` to see who's preventing this from happening...and kill it -- the other day it was one of my browser tabs, and only God knows what it was doing it the background...welcome to 2021!

Relevant read: [Sleepless Mac](https://stuff-things.net/2017/05/10/sleepless-mac/)

2021-02-14 (permalink)

* finished reading [The Common Lisp Condition System](https://www.apress.com/gp/book/9781484261330)

The "earnmuff notation" (extracted from the Common Lisp Condition System)

The "earnmuff notation" in CL is the conventional notation for dynamic variables.

Here is a very poor an concise _refresher_ (I should have used _reminder_ there) about how dynamic variables work in CL:
(defvar *x* 5)

(defun bar ()

(defun foo ()
  (let ((*x* 42))

> (foo)

> (bar)

On reporting custom conditions (extracted from The Common Lisp Condition System)

Condition objects are printed in their unreadable form if the dynamic variable `*print-escape*` is bound to T,  However, when that same variable is bound to NIL, printing a restart object causes its _report function_ to be invoked.  That report function accepts a pair of arguments: the condition object itself and the reporting stream that the the report should be written to.  This means that the report function is capable of querying the condition object for all the infromation that is needed for generating a proper report; it is goodf style for condition reporting functions to be self-contained, which means not to depend on any Lisp data other than the condition object itself.

On CERROR, and its signature (extracted from The Common Lisp Condition System)

An interesting feature of CERROR is that the optional arguments passed to the function are used _both_ for constructing the condition object _and_ as format arguments for the passed format string.  Let us analyze the following form:
 (cerror "Continue after signaling a SIMPLE-ERROR ~
          with arguments: ~S ~S ~S ~S"
         :format-control "A simple error signaled with ~A."
         :format-arguments '(42))
 Executing this form will cause the system to signal a SIMPLE-ERROR; we expect the report of that condition object to be `A simple error signaled with 42.`. In addition, we expect a new restart to be bound around the error site; the restart will have a longer report form, reading `Continue after signaling a SIMPLE-ERROR with arguments: ` and then listing all the opotional arguments that CERROR was called with.

On the condition-restart association (extracted from The Common Lisp Condition System)

Let's take a look at the following code snippet:
Debug> (let ((*print-escape* nil))
         (format t "~&~{;; ~W~%~}" (compute-restarts)))

;; Return to level 2 of the debugger.
;; Retry using SETG.
;; Use specified function
;; Return to level 1 of the debugger.
;; Suppy a new value for *X*.
;; Retry evaluating the form.
;; Return to the top level.

Debug> (let ((*print-escape* nil))
         (format t "~&~{;; ~W~%~}" (compute-restarts *debugger-condition*)))

;; Return to level 2 of the debugger.
;; Retry using SETG.
;; Use specified function
;; Return to level 1 of the debugger.
;; Retry evaluating the form.
;; Return to the top level.
Let us differentiate these two lists of restarts via SET-DIFFERENCE to find restarts which appear in the first list but don't in the other:
Debug> (let ((*print-escape* nil))
         (format t "~&~{;; ~W~%~}"
                     (compute-restarts *debugger-condition*)))
;; Suppy a new value for *X*.
We can see that calling COMPUTE-RESTARTS _without_ passing it the UNDEFINED-FUNCTION condition has caused one more restart to appear -- the missing STORE-VALUE restart, with a report `Suppy a new value for *X*.`, which was bound by the STORE-VALUE form which we originally evaluated.  From this observation, we may induce that there must be some sort of relationship between condition objects and restart objects that is checked by COMPUTE-RESTARTS and FIND-RESTART, among other functions.  This relationship in CL is named _condition-restart association_.

On programmatically calling MACROEXPAND (extracted from The Common Lisp Condition System)

We can see that the macro definition has been expanded to include a special _environment_ object inside its lambda list; this object is not directly passed to it by the programmer, but is usually provided by the Lisp implementation as part of calling its macroexpander.  While we will not describe environment objects in detail, we will note their traits which are most relevant to us for the matter at hand.  This environment argument is available only in macros; it holds, among other things, information about local macro definitions -- information which is important for passing into the MACROEXPAND calls of our new functions.  Since these calls must perform macroexpansion of their own, it is important that they receive this environment argument to be able to expand macros correctly.  That is why RESTART-CASE passed the argument to EXPAND-RESTART-CASE, which -- in turn -- passes it into our yet-undefined functions RESTART-CASE-SIGNALING-FORM-P and RESTART-CASE-EXPAND-SIGNALING-FORM.

Refresher on generalized booleans (extracted from the Common Lisp Condition System)

In the second test, the function returned a non-NIL value, which in CL is "as good as" returning true.  CL has generalized booleans, which means that anything that is not NIL counts as a true value, evewn though the symbol T is designated to be the "canonical" representation of truth -- the standard type BOOLEAN, after all, is defined as `(member nil t)`.

2021-02-12 (permalink)

? Look at OSC52 and see if I can make `cb` use it instead of piping data through the SSH tunnel. Some useful links: https://medium.com/free-code-camp/tmux-in-practice-integration-with-system-clipboard-bcd72c62ff7b, https://github.com/fcpg/vim-osc52/blob/master/plugin/osc52.vim

? [The UNIX Hate's Handbook](https://www.amazon.com/UNIX-Haters-Handbook-UNIX-Haters-line/dp/1568842031) @book

? [ASK YOUR DEVELOPER - how to harness the power of software developers and win in the 21st century](https://www.askyourdeveloper.com/) @book

From Hacker News: [Is Y Combinator worth it?](https://news.ycombinator.com/item?id=24616649)

Would you disqualify companies that had bootstrapped successfully, were profitable, and did not take any outside money?

Not a judging question - I understand the value of a good investor's due diligence and backing in improving the odds of sane leadership. But it's interesting to think how that can impact a successful, self-funded copmany's access to talent.

Someone replied:

As a founder, self funding makes a lot of sense, as you avoid dilution and aren’t beholden to a group of people expecting 10x growth every year.

But as a candidate I’d be cautious about taking a job at a self funded startup. They’re probably cash constrained so the base salary might not be as much. And if there is equity, I would want management beholden to people expecting 10x growth.

Is the second paragraph true tough?

As an engineer I would love to work for a company that makes money and pays me from that money.

Exactly, freedom from investors, chained to your customers.  The conversation goes on:

Curious, why does it matter if your paycheck comes from VC money or revenue money? In the end the same amount comes to your bank account.

And eventually someone steps in and explain how things work:

Because VCs work by pushing for high returns on some of their investments, not low returns on all of their investments. They therefore push for business decisions to force high growth, when there can be a better chance of success with a slow burn in some situations. So you end up taking business risks to chase high growth, instead of simply focusing on a niche market to stay profitable, and having a flexible timeframe for growth.

As an engineer, this does hit the paycheck directly - if they go hire more devs to speed product features to market while at the same time increasing sales and marketing... it can change from a bootstrapped model with no end to the runway into a situation where you do have a runway, and it is counted in months, not years. Instead of steady pay with plenty of time to deliver new product, I'm being pushed to deliver quickly, and if we fail, we're unemployed.

And yes, we probably get lower salaries for that model because we are taking less risk. But steady, lower stress work is desired by some of us, and a small bootstrapped group delivers it better than a VC.

The above finds me in absolute agreement -- and I could not have explained it in simpler terms.

2021-02-10 (permalink)

While this was happening, the big sharks sensed an opportunity; larger trading professionals joined the fight but only for their own profit. BlackRock and eight other Wallstreet titans made a combined 16 billion dollars on GameStop in a matter of days. You see? It wasn't just the redditors that made money, and that's a key point in the story: Wallstreet always finds a way to win.

[Reddit vs Wallstreet - GameStop, The Movie](https://www.youtube.com/watch?v=YFQ-v1jCpF0)

2021-02-05 (permalink)

I wanted to fix some of the compilation warnings in my AdventOfCode-solutions-in-Common-Lisp [project](https://iamFIREcracker.github.com/adventofcode), and this one in particular caught my attention.

Given this function:
(defun count-hits (map delta-col delta-row)
  (loop with (rows cols) = (array-dimensions map)
        for row below rows by delta-row
        for col = 0 then (mod (+ col delta-col) cols)
        count (char= (aref map row col) #\#)))
Try and evaluate in the REPL (SBCL/2.0.11 in my case) and you should get the following style warnings back:
;     (LOOP AOC/2020/03::WITH (AOC/2020/03::ROWS
;                              AOC/2020/03::COLS) = AOC/2020/03::DIMS
;           AOC/2020/03::FOR AOC/2020/03::ROW AOC/2020/03::BELOW AOC/2020/03::ROWS AOC/2020/03::BY AOC/2020/03::DELTA-ROW
;           AOC/2020/03::FOR ...)
; ==>
;   NIL
;   The binding of #:LOOP-LIMIT-1 is not a REAL:
;    NIL
;   See also:
;     The SBCL Manual, Node "Handling of Types"
;   The binding of SB-C::DIVISOR is not a REAL:
;    NIL
;   See also:
;     The SBCL Manual, Node "Handling of Types"
; compilation unit finished
;   caught 2 STYLE-WARNING conditions
WARNING: redefining AOC/2020/03::COUNT-HITS in DEFUN
I can avoid the implicit call to DESTRUCTURING-BIND (LOOP-DESTRUCTURING-BIND to be precise, as that is what the `with (rows cols) = ...` expression gets expanded into) by replacing ARRAY-DIMENSIONS with two calls to ARRAY-DIMENSION:
(defun count-hits (map delta-col delta-row)
  (loop with rows = (array-dimension map 0)
        with cols = (array-dimension map 1)
        for row below rows by delta-row
        for col = 0 then (mod (+ col delta-col) cols)
        count (char= (aref map row col) #\#)))
Now _that_ seems to have solved the problem (i.e. no style warning anymore).

What if I wrapped the original LOOP expression into a DESTRUCTURING-BIND form in which I initialize `rows` and `cols`?
(defun count-hits (map delta-col delta-row)
  (destructuring-bind (rows cols) (array-dimensions map)
    (loop for row below rows by delta-row
          for col = 0 then (mod (+ col delta-col) cols)
          count (char= (aref map row col) #\#))))
This version too, does **not** cause the compiler to throw any style warnings.  What exactly is going on?

Let's try and find the minimal snippet to reproduce it:
(defvar arr (make-array '(2 5) :initial-contents '((1 2 3 4 5)
                                                   (6 7 8 9 10))))
(defun test-1 (arr)
  (loop with (rows cols) = (array-dimensions arr)
        for r below rows do
        (loop for c below cols do
              (print (aref arr r c)))))
When I send the above to the REPL I get:
; in: DEFUN TEST-1
;     (LOOP AOC/2020/03::WITH (AOC/2020/03::ROWS
;                              AOC/2020/03::COLS) = (ARRAY-DIMENSIONS
;                                                    AOC/2020/03::ARR)
;           AOC/2020/03::FOR AOC/2020/03::R AOC/2020/03::BELOW AOC/2020/03::ROWS
;           DO (LOOP AOC/2020/03::FOR AOC/2020/03::C AOC/2020/03::BELOW AOC/2020/03::COLS
;                    DO (PRINT
;                        (AREF AOC/2020/03::ARR AOC/2020/03::R AOC/2020/03::C))))
; ==>
;   NIL
;   The binding of SB-C::Y is not a REAL:
;    NIL
;   See also:
;     The SBCL Manual, Node "Handling of Types"
;   The binding of SB-C::Y is not a REAL:
;    NIL
;   See also:
;     The SBCL Manual, Node "Handling of Types"
; compilation unit finished
;   caught 2 STYLE-WARNING conditions
Perfect.  Let's try and expand that LOOP expression, _recursively_, and see if anything stands out:
    (LET* ((#:G692
                                       '(&OPTIONAL ROWS COLS
                                         . #:LOOP-IGNORED-691)))
            (IF #:G692
                (LET ((#:G693 (CAR #:G692)))
                  (PROGN (SETQ #:G692 (CDR #:G692)) #:G693))))
            (IF #:G692
                (LET ((#:G694 (CAR #:G692)))
                  (PROGN (SETQ #:G692 (CDR #:G692)) #:G694))))
           (#:LOOP-IGNORED-691 #:G692))
      (LET ((#:LOOP-LIMIT-690 ROWS) (R 0))
                 (TYPE (AND NUMBER REAL) R)
                 (IGNORABLE #:LOOP-LIMIT-690)
                 (TYPE (AND NUMBER REAL) #:LOOP-LIMIT-690))
          (IF (>= R #:LOOP-LIMIT-690)
              (GO SB-LOOP::END-LOOP))
          (BLOCK NIL
            (LET ((#:LOOP-LIMIT-695 COLS) (C 0))
              (DECLARE (IGNORABLE C)
                       (TYPE (AND NUMBER REAL) C)
                       (IGNORABLE #:LOOP-LIMIT-695)
                       (TYPE (AND NUMBER REAL) #:LOOP-LIMIT-695))
                (IF (>= C #:LOOP-LIMIT-695)
                    (GO SB-LOOP::END-LOOP))
                (PRINT (AREF ARR R C))
                (SETQ C (1+ C))
                (GO SB-LOOP::NEXT-LOOP)
          (SETQ R (1+ R))
          (GO SB-LOOP::NEXT-LOOP)
I am afraid the compiler is right, isn't it?

- LOOP's destructuring bind would not signal an error if the list you are destructuring bind against does not have enough elements; it would bind all the _extra_ variables to NIL
- This means that both `rows` and `cols` can be NIL
- `rows`, however, is used after the keyword `:below`, and because of that it is expected to be a NUMBER
- But NIL is not a valid NUMBER, hence the warning

Technically the compiler is right, but what a pain...

Anyways, I reported this as a bug to sbcl-bugs@lists.sourceforge.net, let's see if they think it's a bug or not.

2021-02-02 (permalink)

"Great flight up... Gotta work on that landing"

SpaceX commenter, at the end of the [Starship SN9 flight test](https://mobile.twitter.com/SpaceX/status/1356699321840721920)

2021-01-20 (permalink)

any legit use case for PROG2 though? It seems very ad-hoc-y

Here is a _legit_ use case for PROG2:
(prog2 (prologue)
It turns out there is a library, `:cl-advice`, that exploits PROG2 in an almost identical way (check this out on [GitHub](https://github.com/szos/cl-advice/blob/3d8653da763ab67b8abda1dba8c7783806a10e64/cl-advice.lisp#L137-L142)):
(prog2 (when ,before
         (apply ,before ,dispatch-args))
       (if ,around
         (apply ,around ,dispatch-args)
         (apply ,main ,dispatch-args))
       (when ,after (apply ,after ,dispatch-args)))))
When perceived in this way, PROG2 is a non-object-oriented `:before`/`:after`

2021-01-17 (permalink)

Advent of Code: [2015/25](https://adventofcode.com/2015/day/25)

The task for today, i.e. Christmas, is to boot up the weather machine -- no idea what that is -- and all we have to do is to read a code from the instruction manual and insert it on the console.  Except, we don't know where the manual is, so need to generate the sequence of codes first.

We are given the rules of how this sequence has to be constructed:
   | 1   2   3   4   5   6
 1 |  1   3   6  10  15  21
 2 |  2   5   9  14  20
 3 |  4   8  13  19
 4 |  7  12  18
 5 | 11  17
 6 | 16

   |    1         2         3         4         5         6
 1 | 20151125  18749137  17289845  30943339  10071777  33511524
 2 | 31916031  21629792  16929656   7726640  15514188   4041754
 3 | 16080970   8057251   1601130   7981243  11661866  16474243
 4 | 24592653  32451966  21345942   9380097  10600672  31527494
 5 |    77061  17552253  28094349   6899651   9250759  31663883
 6 | 33071741   6796745  25397450  24659492   1534922  27995004
First, let's begin by parsing the input to know the position in this _diagonal_ sequence where we will find the code to boot up the weather machine.
(defun parse-target-position (lines)
  (cl-ppcre:register-groups-bind ((#'parse-integer row col))
      ("row (\\d+), column (\\d+)" (first lines))
    (list row col)))
Next, let's implement a generator of all the positions of this _diagonal_ sequence:

- Return `(1 1)` the first time
- Then move `1` up, and `1` right
- If the current row is `0`, we need to wrap around, otherwise there we have the next position of the sequence
- Repeat
(defun next-grid-position (&optional prev)
  (if (not prev)
    (list 1 1)
    (let ((next (mapcar #'+ prev (list -1 1))))
      (if (= (first next) 0)
        (list (second next) 1)
Next, let's implement a generator of all the values of this _diagonal_ sequence (i.e. all the possible codes):

- Start with `20151125`
- Then multiply it by `252533` and take the remainder from dividing that by `33554393`
- Repeat
(defun next-code (&optional prev)
  (if (not prev) 20151125 (mod (* prev 252533) 33554393)))
With these two generators, finding the solution to part 1 should be pretty straightforward:

- Generate the next position
- Generate the next value
- If the position we are at, is what we are looking for, then the current value is our answer
(defun part1 (target)
  (loop for pos = (next-grid-position) then (next-grid-position pos)
        for code = (next-code) then (next-code code)
        when (equal pos target) return code))
No part 2 for Dec 25, so here goes the final plumbing:
(define-solution (2015 25) (pos parse-target-position)
  (values (part1 pos)))

(define-test (2015 25) (2650453))
And that's it:
> (time (test-run))
Success: 1 test, 1 check.
Evaluation took:
  1.226 seconds of real time
  1.189050 seconds of total run time (1.169468 user, 0.019582 system)
  [ Run times consist of 0.026 seconds GC time, and 1.164 seconds non-GC time. ]
  96.98% CPU
  2,821,032,228 processor cycles
  1,175,386,480 bytes consed

2021-01-16 (permalink)

Advent of Code: [2015/23](https://adventofcode.com/2015/day/23)

Little Jane Marie just got her very first computer for Christmas from some unknown benefactor. It comes with instructions and an example program, but the computer itself seems to be malfunctioning. She's curious what the program does, and would like you to help her run it.

The problem description goes on by giving you details about the six instructions that this computer supports (i.e. `hlf`, `tpl`, `inc`, `jmp`, `jie`, and `jio`), the number of registers it has (two, `a` and `b`), how programs are run, and when should they halt (when they jumps to position _outside_ the program).

Given the input program, what's the value of the `b` register after the program has finished running?  Looks like we are going to write a little emulator here...

First off, we are going to need an instruction pointer, two registers, `a` and `b`, and some utility functions to operate on these registers:
(defparameter *ip* 0)
(defparameter *a* 0)
(defparameter *b* 0)

(defun reg (name) (if (string= name "a") *a* *b*))
(defun (setf reg) (value name)
  (if (string= name "a") (setf *a* value) (setf *b* value)))
The next step is to _parse_ our program:

- Split each instruction into its _operator_ and its _operands_
- Based on the value of the _operator_, return a function implementing the required logic, and taking care of updating the instruction pointer accordingly
(defun parse-instruction (line)
  (destructuring-bind (rator . rands) (cl-ppcre:split ",? " line)
    (cond ((string= rator "hlf")
           (lambda ()
             (divf (reg (first rands)) 2)
             (incf *ip*)))
          ((string= rator "tpl")
           (lambda ()
             (mulf (reg (first rands)) 3)
             (incf *ip*)))
          ((string= rator "inc")
           (lambda ()
             (incf (reg (first rands)) 1)
             (incf *ip*)))
          ((string= rator "jmp")
           (lambda ()
             (incf *ip* (parse-integer (first rands)))))
          ((string= rator "jie")
           (lambda ()
             (if (evenp (reg (first rands)))
               (incf *ip* (parse-integer (second rands)))
               (incf *ip*))))
          ((string= rator "jio")
           (lambda ()
             (if (= (reg (first rands)) 1)
               (incf *ip* (parse-integer (second rands)))
               (incf *ip*))))
          (t (error "Unable to parse line: ~A" line)))))

(defun parse-program (lines) (map 'vector #'parse-instruction lines))
All is left to do, is running the program until it _exits_, and as requested, return the value of register `b`:
(defun run-program (a b program)
  (let ((*ip* 0) (*a* a) (*b* b))
    (loop while (array-in-bounds-p program *ip*)
          do (funcall (aref program *ip*)))

(defun part1 (program) (run-program 0 0 program))
For part 2, we are asked to run the program again, this time with register `a` initialized to `1` instead of `0`:
(defun part2 (program) (run-program 1 0 program))
Final plumbing:
(define-solution (2015 23) (program parse-program)
  (values (part1 program) (part2 program)))

(define-test (2015 23) (184 231))
And that's it:
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.000 seconds of real time
  0.000733 seconds of total run time (0.000613 user, 0.000120 system)
  100.00% CPU
  1,797,379 processor cycles
  0 bytes consed

Advent of Code: [2015/24](https://adventofcode.com/2015/day/24)

It's Christmas Eve, and Santa is loading up the sleigh for this year's deliveries. However, there's one small problem: he can't get the sleigh to balance. If it isn't balanced, he can't defy physics, and nobody gets presents this year. No pressure.

We are given a list of the weights of each package, and we are asked to split them into three groups so that each group weights the same; well, not just that, of course we also want to find the combination that:

- Has the fewest **number** of packages in the first group
- (and in case of ties, the one that) Has the smallest "quantum entanglement" (i.e. the product of the weights of the packages in the group)

The input is a set of lines, each representing the weight of a package; a simple call to PARSE-INTEGERS should do just fine.

Now back to the actual problem solution.  Since we are asked to find the configuration that minimizes the number packages in the first group, I figured I could simply focus on that group alone, and find our solution as follows:

- Try and look for all the packages whose weight is equal to the _target_ value (i.e. total weight divided by `3`)
- If none is found, try and look for all pairs of packages whose total weight is equal to the target value
- If none is found, try and look for all triplets of packages whose total weight is equal to the target value

And so on, and so forth, until one or more solutions are found; at which point, I would calculate the "quantum entanglement" of these _winning_ configurations, and return the minimum value.

First let's take care of the easy parts: finding the target weight, and calculating the quantum entanglement of a configuration of packages:
(defun target-weight (weights groups) (/ (reduce #'+ weights) groups))
(defun quantum-entanglement (group) (reduce #'* group))
Now we have to find the configuration for the perfect balance; we know we will iteratively try to find configurations, with increasing number of packages.  But how to find a configuration of `n` packages, whose total weight add up to a specific value?  With a bit of recursion and backtracking:

- If we can select only one package, see if one with the target value exists in the pool of remaining packages, and if it does, then we found ourselves a partial solution
- Otherwise we have to options: add the first available package to the current solution and see if with the remaining packages it's possible to find a solution that adds up to the target value minus the weight of the selected package; or we skip the first available package and try with the next one

This (and some other details that I did not bother mention) translates into the following function:
(defun find-perfect-balance (weights groups
                                     &aux (target (target-weight weights groups)))
  (labels ((recur (n target remaining)
             (cond ((= n 1) (when-let ((found (find target remaining)))
                                  (list (list found))))
                   ((null remaining) '())
                   (t (append
                        (recur n target (rest remaining))
                        (loop for rest in (recur (1- n)
                                                 (- target (first remaining))
                                                 (rest remaining))
                              collect (cons (first remaining) rest)))))))
    (loop for n from 1 for solutions = (recur n target weights)
          when solutions return (reduce #'min solutions
                                        :key #'quantum-entanglement))))
With this, all we have to do for part 1 is calling FIND-PERFECT-BALANCE with the input weights, and the number of groups we want to divide our packages into:
(defun part1 (weights) (find-perfect-balance weights 3))
For part 2, we are asked to repeat the exercise with `4` groups instead of `3` -- turns out we forgot about the trunk of the sleigh:
(defun part2 (weights) (find-perfect-balance weights 3))
Final plumbing:
(define-solution (2015 24) (weights parse-integers)
  (values (part1 weights) (part2 weights)))

(define-test (2015 24) (11266889531 77387711))
And that's it:
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.017 seconds of real time
  0.017210 seconds of total run time (0.016366 user, 0.000844 system)
  100.00% CPU
  41,322,015 processor cycles
  294,912 bytes consed
Note: I made a big assumption here, which is once I find a possible solution for the first group I take it for granted that the remaining packages can be split into 2 groups (3 for part 2), each adding up to the target value.  This assumption got me my 2 stars today, but I guess I got lucky ;-)

2021-01-15 (permalink)

Advent of Code: [2015/22](https://adventofcode.com/2015/day/22)

Another day, another RPG to play (err, implement); this time however, the game logic is a bit more complicated than the one we implemented for [2015/21](https://matteolandi.net/plan.html#day-2021-01-13), with lots of tiny details that if you get wrong, no way you are going to be able to get your 2 stars for the day.

In this game the player does not have any equipments, but only spells; spells cost mana; spells can protect you, deal damage to the boss, or both; spells can have an _effect_ that lasts more than one turn.

Let's have a look at the list of spells available:

- Magic Missile costs `53` mana. It instantly does `4` damage.

- Drain costs `73` mana. It instantly does `2` damage and heals you for `2` hit points.

- Shield costs `113` mana. It starts an effect that lasts for `6` turns. While it is active, your armor is increased by `7`.

- Poison costs `173` mana. It starts an effect that lasts for `6` turns. At the start of each turn while it is active, it deals the boss `3` damage.

- Recharge costs `229` mana. It starts an effect that lasts for `5` turns. At the start of each turn while it is active, it gives you `101` new mana.

I decided to parse this catalog into a LIST of LISTs; LIST of _structured_ types actually, but let's not get lost in the details:
(defstruct (spell (:type list)) name cost damage heal effect)

(defparameter *spells* '(("Magic Missile" 53  4 0)
                         ("Drain"         73  2 2)
                         ("Shield"        113 0 0 (6 0   0 7))
                         ("Poison"        173 0 0 (6 0   3 0))
                         ("Recharge"      229 0 0 (5 101 0 0))))

(defstruct (effect (:type list)) turns mana damage armor)
Let's have a look at the game, and focus on the player first (i.e. you).  The `player` has some _life_ points, _mana_, and certain _effects_:
(defstruct (player (:type list)) life mana effects)
By looking at the list of _active_ effects it should be quite easy to calculate one's total armor, mana, and damage:
(defun player-armor-effect (player)
  (reduce #'+ (player-effects player) :key #'effect-armor))

(defun player-mana-effect (player)
  (reduce #'+ (player-effects player) :key #'effect-mana))

(defun player-damage-effect (player)
  (reduce #'+ (player-effects player) :key #'effect-damage))
One's total mana is equal to the mana the player has, plus all the mana given by any active effect:
(defun player-total-mana (player)
  (+ (player-mana-effect player)
     (player-mana player)))
Similarly, one's total _dealable_ damage is equal to the total damage of any active effect, plus the damage of the spell they might be about to cast:
(defun player-total-damage (player spell)
  (+ (player-damage-effect player) (spell-damage spell)))
Rest is generating a new `player` object, as a result of playing a turn (i.e. casting a spell, or defending an attack):

- Update life points based on the healing power of the spell, or the boss attack
- Update mana taking into account _recharging_ effects, plus the cost of the spell to cast
- Update the list of active effects:
(defun life-next (life spell-heal damage) (- (+ life spell-heal) damage))

(defun mana-next (mana spell-cost) (- mana spell-cost))

(defun effects-next (effects spell-effect)
  (let ((effects (loop for (turns . rest) in effects
                       when (> (1- turns) 0) collect (cons (1- turns) rest))))
    (if spell-effect
      (cons spell-effect effects)

(defun player-next (player damage
                           &optional (spell  '("Dummy" 0 0 0 nil))
                           &aux (player (copy-seq player)))
  (setf (player-life player) (life-next (player-life player)
                                        (spell-heal spell)
        (player-mana player) (mana-next (player-total-mana player)
                                        (spell-cost spell))
        (player-effects player) (effects-next (player-effects player)
                                              (spell-effect spell)))
Now...let's move on to the boss logic: we will first define a structured type for the boss, and some parsing functions to extract our boss specs from our input:
(defstruct (boss (:type list)) life damage)

(defun parse-boss (lines)
  (mapcar #'parse-integer
          (cl-ppcre:all-matches-as-strings "\\d+"
                                           (format nil "~{~A ~}" lines))))
The amount of damage the boss can deal is equal to their damage points minus the player's armor; however, since we play by the boss' rules, the boss is always going to deal at least `1` damage point, irrespective of the player's armor:
(defun boss-attack (boss player)
  (max 1 (- (boss-damage boss) (player-armor-effect player))))
Updating the boss state after a turn is pretty straightforward:
(defun boss-next (boss damage)
  (let ((boss (copy-seq boss)))
    (setf (boss-life boss) (- (boss-life boss) damage))
Alright, now it's time to actually implement the game:

- A _turn_ consists in the player casting a spell **and** the boss attacking us
- There might be multiple _castable_ spells at a given time; we should pick one, cast it, and see where this takes us
- If after the boss' move, the player turns out to be dead, then we should backtrack and try with a different spell
- The list of _castable_ spells is obtained by looking at the player's mana, and the already active effect
- Note: you can cast a spell whose effect is already active, if and only if this is the last turn the effect is active
(defstruct (state (:type list) (:conc-name)) player boss)

(defun not-enough-mana-p (mana spell) (> (spell-cost spell) mana))

(defun effect-in-use-p (effects spell)
  (loop for (turns . rest) in effects
        thereis (and (equal (rest (spell-effect spell)) rest)
                     (> turns 1))))

(defun castable-spells (mana effects)
  (loop for spell in *spells*
        unless (or (not-enough-mana-p mana spell)
                   (effect-in-use-p effects spell))
        collect spell))

(defun player-turn (state spell
                           &aux (player (player state)) (boss (boss state)))
  (let ((boss-damage (player-total-damage player spell)))
    (cons (list (player-next player 0 spell) (boss-next boss boss-damage))
          (spell-cost spell))))

(defun all-player-turns (state &aux (player (player state)))
  (loop for spell in (castable-spells (player-total-mana player) (player-effects player))
        collect (player-turn state spell)))

(defun boss-turn (state)
  (let ((player (player state)) (boss (boss state)))
    (let ((boss-damage (player-damage-effect player))
          (player-damage 0))
      (when (< boss-damage (boss-life boss))
        (setf player-damage (boss-attack boss player)))
      (list (player-next player player-damage) (boss-next boss boss-damage)))))

(defun player-alive-p (player) (> (player-life player) 0))

(defun play (state)
  (loop for (next . cost) in (all-player-turns state)
        for next-next = (boss-turn next)
        when (player-alive-p (player next-next))
        collect (cons next-next cost)))
Lastly, the recursive function responsible for playing the game, and backtrack when it reaches a dead end:
(defun part1 (player boss &aux (best 10000))
  (labels ((recur (state cost-so-far)
             (cond ((player-wins-p state) (setf best (min best cost-so-far)))
                   ((>= cost-so-far best) nil)
                     (loop for (next . cost) in (play state)
                           do (recur next (+ cost-so-far cost)))))))
    (recur (list player boss) 0)
Easy uh?! Anyway, for part 2 we are asked to play a slightly different version of the game, where the player loses 1 life point at the beginning of his turn:

- LIFE-IS-HARD takes care of inflicting the damage to player
- PLAY-HARD stops the game the moment the player has no more life points (note: we need this to happen before any effect takes place)
- PART2 is equal to PART1, except that it calls PLAY-HARD instead of PLAY
(defun life-is-hard (state &aux (state (copy-seq state)))
  (setf (player state) (copy-seq (player state)))
  (decf (player-life (player state)))

(defun play-hard (state &aux (state (life-is-hard state)))
  (when (player-alive-p (player state))
    (play state)))

(defun part2 (player boss &aux (best 10000))
  (labels ((recur (state cost-so-far)
             (cond ((player-wins-p state) (setf best (min best cost-so-far)))
                   ((>= cost-so-far best) nil)
                     (loop for (next . cost) in (play-hard state)
                           do (recur next (+ cost-so-far cost)))))))
    (recur (list player boss) 0)
Final plumbing:
(define-solution (2015 22) (boss parse-boss)
  (values (part1 (list 50 500 nil) boss) (part2 (list 50 500 nil) boss)))

(define-test (2015 22) (900 1216))
And that's it:
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.252 seconds of real time
  0.248577 seconds of total run time (0.212851 user, 0.035726 system)
  [ Run times consist of 0.004 seconds GC time, and 0.245 seconds non-GC time. ]
  98.81% CPU
  581,845,084 processor cycles
  124,414,480 bytes consed

2021-01-13 (permalink)

Advent of Code: [2015/21](https://adventofcode.com/2015/day/21)

Today, we treat ourselves with a simple RPG game to play with:

- It's you versus the "boss"
- The boss starts with 100 hit points and with some pre-defined attack power and armor (all this comes from the input)
- You start with 100 hit points, bare-handed, and without a armor
- There is a shop selling weapons, armors, and rings, and each has a cost (this is not in the input, but part of the description, so I assume this to the same for everybody)
- To fight, you need a weapon, and you can optionally wear an armor, and one or two rings
- The game is played in turns -- you start first -- and the amount of damage dealt is equal to your attack power minus the defendant's armor

What's the minimum you have to spend, to beat the boss?

Let's start with the shop: I am going to use 3 different parameters (one for the weapons, one for the armors, and one for the rings), and inside I will store items as LISTS having the name of the item as first argument, its cost as second, its damage as third, and armor as fourth:
(defparameter *weapons* '(("Dagger"        8     4       0)
                          ("Shortsword"   10     5       0)
                          ("Warhammer"    25     6       0)
                          ("Longsword"    40     7       0)
                          ("Greataxe"     74     8       0)))

(defparameter *armors* '(("Leather"      13     0       1)
                         ("Chainmail"    31     0       2)
                         ("Splintmail"   53     0       3)
                         ("Bandedmail"   75     0       4)
                         ("Platemail"   102     0       5)))

(defparameter *rings* '(("Damage +1"    25     1       0)
                        ("Damage +2"    50     2       0)
                        ("Damage +3"   100     3       0)
                        ("Defense +1"   20     0       1)
                        ("Defense +2"   40     0       2)
                        ("Defense +3"   80     0       3)))
Then we will parse our boss specs:

Hit Points: 100

Damage: 8

Armor: 2

I am going to join the lines back into a single string (DEFINE-SOLUTION would already parse the input file into a list of strings, so I need to _undo_ that first), then extract all all the numeric values from it:
(defun parse-boss (lines)
  (mapcar #'parse-integer
          (cl-ppcre:all-matches-as-strings "\\d+"
                                           (format nil "~{~A ~}" lines))))
Now, the solution I have in mind for "finding the minimum you have to spend to beat the boss" is pretty simple:

- generate all the possible combinations of items you can buy; for each...
- equip your player with those items, and play against the boss
- if you win, keep track of the amount you spent, and minimize it

Let's begin by generating all the possible combinations of items you can buy:

- you need at least one weapon -- or otherwise you are bound to lose, always
- you may or may not wear an armor -- can't have more though
- you may or may not wear one, or two rings -- you can have no rings, one ring, or two

If we consider _optionality_ as an item with no cost, and no damage nor armor, then the above nicely translates to the following function:
(defun all-items-combinations ()
  (loop for w in *weapons* append
        (loop for a in (cons (list "" 0 0 0) *armors*) append
              (loop for r1 in (cons (list "" 0 0 0) *rings*) append
                    (loop for r2 in (cons (list "" 0 0 0) *rings*)
                          collect (mapcar
                                    (rest w)
                                    (rest a)
                                    (rest r1)
                                    (rest r2)))))))
Now, equipping our player with an items combination should be as simple as appending the overall items damage and armor to the user hit points (i.e. `(list 100 damage armor)`).

What about playing the game and see if the player won?

- Calculate how many turns it will take the player to defeat the boss (note: damage / armor stats don't change throughout the game)
- Calculate how many turns it will take the boss to the defeat us
- We win the number of turns it take us to the defeat the boss, is smaller than or equal to the turns it takes to the boss (note: we start first, hence the _or equal_ before)
(defun player-wins-p (player boss)
  (destructuring-bind (p-hits p-damage p-armor) player
    (destructuring-bind (b-hits b-damage b-armor) boss
      (let ((p-turns-to-win (/ b-hits (max (- p-damage b-armor) 1)))
            (b-turns-to-win (/ p-hits (max (- b-damage p-armor) 1))))
        (<= p-turns-to-win b-turns-to-win)))))
All there is left to do at this point is iterate all the combinations, play the game, see if we win, and minimize the cost of the game:
(defun part1 (boss)
  (loop for (cost damage armor) in (all-items-combinations)
        when (player-wins-p (list 100 damage armor) boss)
        minimize cost))
For part 2, the shopkeeper can trick you into buying anything they want, and we are asked to find the maximum we can spend and still be defeated by the boss!

That should be pretty easy:

- generate all the possible combinations of items you can buy; for each...
- equip your player with those items, and play against the boss
- if you lose, keep track of the amount you spent, and maximize it
(defun part2 (boss)
  (loop for (cost damage armor) in (all-items-combinations)
        unless (player-wins-p (list 100 damage armor) boss)
        maximize cost))
Final plumbing:
(define-solution (2015 21) (boss parse-boss)
  (values (part1 boss) (part2 boss)))

(define-test (2015 21) (91 158))
And that's it:
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.004 seconds of real time
  0.003741 seconds of total run time (0.001263 user, 0.002478 system)
  100.00% CPU
  9,978,989 processor cycles
  622,384 bytes consed

2021-01-12 (permalink)

it's quite windy here its side down well though I good it's it's this is oh actually wasn't expected to be this windy hopefully you can actually hear what I'm saying okay great great so this is this is I think the most inspiring thing that I have ever seen and I just like to thank the SpaceX team and the the suppliers and the the people of Brook chic and Brownsville thank you for your support and just like wow what an incredible job by such a great team to build this incredible vehicle so it's like first of all want to stop that I'm just so so so so proud to work with such a great team and it's really ripping here by the way if you're watching this online it is like it was really windy so the the point of this this presentation and this is this event it is really there are two elements to it one is to inspire the public and get people excited about our future in space and and get people fired up about the future the you know what what there are so many things to worry about so many things to be concerned about there's there are many troubles in the world of course and we are important and we need to solve them but we also need things that make us excited to be alive that make us glad to wake up in the morning and be fired up about the future and think yeah the future is gonna be great you know and and this space exploration is one of those things and becoming a spacefaring civilization being out there among the stars this is one of the things that I know makes it makes me glad to be alive I think it makes many people glad to be alive it's one of the best things and this really weird face with a choice which future do you want do you want the future where we become a spacefaring civilization and are in many worlds and now out there among the stars or one where we are forever confined to earth and I say it is the first and I hope you agree with me yeah so so what what the critical breakthrough that's needed for us to become a spacefaring civilization is to make space travel like air travel so with with air travel you could be when you fly a plane you fly that plane many times I mean the risk of stating the obvious it really almost any motor transport whether it's a plane a car a horse the bicycle is reusable you use that motor transport many times and if you had to get a new plane every time you flew somewhere and even get have two planes for a return journey very few people could afford to fly or if you could use a car only once very few people could afford to drive a car so the critical breakthrough that's necessary is a rapidly reusable orbital rocket this is what this is basically the holy grail of space and the fundamental thing that's required

Elon Musk, Starship Update, September 2019 - https://www.youtube.com/watch?v=sOpMrVnjYeY

Advent of Code: [2015/20](https://adventofcode.com/2015/day/20)

Today it's about the elves delivering presents by hand, door to door, and us trying to figure out how many presents each house is going to get.  We have an infinite number of houses, numbered: `1`, `2`, `3`, `4`, `5`, and so on; then we have our elves, each with an assigned number, wandering around and delivering presents as described below:

The first Elf (number `1`) delivers presents to every house: `1`, `2`, `3`, `4`, `5`, ....

The second Elf (number `2`) delivers presents to every second house: `2`, `4`, `6`, `8`, `10`, ....

Elf number `3` delivers presents to every third house: `3`, `6`, `9`, `12`, `15`, ....

Knowing that each elf will deliver at every house they visit presents equal to **10 times** his or her number, what's the first house which is going to receive at least `36000000` presents?

Input parsing was a no-brainer: single line, one integer:
(defun parse-target-presents (lines) (parse-integer (first lines)))
Now, to solve this, I thought about a very naive brute-force:

- For each house, starting from `1`
- Find all the elves which are going to visit it (i.e. find all the _divisors_ of the house number)
- Sum it, and break out of the loop as soon as the sum, times `10`, is bigger than or equal to the input number
(defun divisors (number)
  (loop for divisor from 1 upto (sqrt number)
        when (zerop (mod number divisor))
        if (= (/ number divisor) divisor) collect divisor
        else append (list divisor (/ number divisor))))

(defun part1-slow (target)
  (loop for n from 1 for div-sum = (reduce #'+ (divisors n))
        when (>= (* div-sum 10) target) return n))
Well, calculating all the divisors for all the house numbers turned out to be a very inefficient solution -- simple, but very inefficient:
> (time (part1-slow 36000000))
Evaluation took:
  15.396 seconds of real time
  15.168194 seconds of total run time (15.006065 user, 0.162129 system)
  [ Run times consist of 0.022 seconds GC time, and 15.147 seconds non-GC time. ]
  98.52% CPU
  35,411,623,928 processor cycles
  1 page fault
  183,443,696 bytes consed

Interestingly, _simulating_ this, actually made for an even simpler and definitely more efficient solution:

- For each elf
- For each house the elf is going to visit
- Increase the number of presents it will receive by the elf number times `10`
- At the end, find the first house that got _that_ many presents
(defun part1 (target &aux
                     (all-houses (/ target 10))
                     (houses (make-array all-houses :initial-element 0)))
  (loop for elf from 1 below all-houses do
        (loop for house from elf below all-houses by elf do
              (incf (aref houses house) (* elf 10))))
  (position-if (partial-1 #'>= _ target) houses))
With this:
> (time (part1 36000000))
Evaluation took:
  1.110 seconds of real time
  1.004395 seconds of total run time (0.905748 user, 0.098647 system)
  90.45% CPU
  2,553,495,191 processor cycles
  28,800,016 bytes consed

For part 2, we are asked to answer the same question, except that this time each elf will visit a maximum of `50` houses.  Thanks to CL powerful LOOP macro, all we have to do is adding a `repeat 50` to our inner loop:
(defun part2 (target &aux
                     (all-houses (/ target 10))
                     (houses (make-array all-houses :initial-element 0)))
  (loop for elf from 1 below all-houses do
        (loop repeat 50
              for house from elf below all-houses by elf do
              (incf (aref houses house) (* elf 11))))
  (position-if (partial-1 #'>= _ target) houses))
Final plumbing:
(define-solution (2015 20) (target parse-target-presents)
  (values (part1 target) (part2 target)))

(define-test (2015 16) (831600 884520))
And that's it:
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  1.190 seconds of real time
  1.178205 seconds of total run time (1.166144 user, 0.012061 system)
  98.99% CPU
  2,737,747,940 processor cycles
  57,600,032 bytes consed

2021-01-11 (permalink)

Advent of Code: [2015/19](https://adventofcode.com/2015/day/19)

Rudolph (the Red-Nosed reindeer) is sick, and Santa needs your help to cure him; what exactly are you supposed to do? Come up with a medicine.  We are given a molecule (i.e. a string of chars), and a bunch of _replacement_ rules (e.g. `H => OH`); our task for part 1 is to count all the _distinct_ molecules you can obtain by applying any of your _replacement_ rules to the given molecule.

As usual, let's begin by parsing our input:

- The replacement rules we will parse them into an ALIST
- The molecule, well, that would be a STRING on its own
(defun parse-replacement (string &aux (parts (cl-ppcre:split " " string)))
  (cons (first parts) (third parts)))

(defun parse-replacements (lines) (mapcar #'parse-replacement lines))

(defun parse-input (lines &aux (pos (position "" lines :test #'string=)))
  (cons (parse-replacements (subseq lines 0 pos))
        (nth (1+ pos) lines)))
As per the actual solution to the problem, the following approach should do just fine:

- For each replacement rule `(from . to)`
- For each match of `from` inside our input molecule, replace it with `to` and accumulate the result
- Finally remove all the duplicates, and count the number of distinct molecules you are left with
(defun mreplace (molecule from to pos)
  (format nil "~A~A~A"
          (subseq molecule 0 pos)
          (subseq molecule (+ pos (length from)))))

(defun part1 (input)
  (destructuring-bind (replacements . molecule) input
          (loop for (from . to) in replacements append
                (loop for start in (cl-ppcre:all-matches from molecule) by #'cddr
                      collect (mreplace molecule from to start)))
          :test #'string=))))
Things get a bit more complicated for part 2:

Given the available replacements and the medicine molecule in your puzzle input, what is the fewest number of steps to go from the origin molecule `e` to the medicine molecule?

I tried "the" very naive approach first, but it did not take me long to realize that replacement rules were recursive and that a basic _unbound_ search would have taken **forever** to find all the possible solutions.

My next try was: start from the medicine molecule, then recursively apply rules, _backward_ (e.g. given `(from . to)` replace all the occurrences of `to` with `from`), until you get to `e`.  This too was taking forever, even get to the _first_ solution...

I tried to sort replacement rules and process first the ones that were reducing our molecule the most, but that too did not seem to help much.

Then I thought: what if I processed first, the rules with the least number of matches? My idea was that this could help and minimize the branching factor, to a point where I could actually find a solution -- and if I have a solution, then I can find others by pruning the search space.

Apparently this heuristic seemed to work: I was indeed able to generate few solutions to the problem.  However, generating them **all** would still take forever.

Kind of hopeless, I let the program go for a while, logging each solution encountered so far, and interestingly enough it started logging the same number over and over again -- and guess what, that happened to be the right solution to the problem.

What's going on, exactly? I believe that my input (as well as others') is generated in such a way that all the possible solutions are indeed the same one (i.e. the _same_ in terms of number of steps to get from `e` to the medicine molecule).

Anyway, the solution I am presenting here assumes that all the solutions are equal, and will stop after the first one -- I know, it's not _generic_ in the sense that technically it does not solve the described problem, but it works well with few other users input ([1](https://topaz.github.io/paste/#XQAAAQCzAwAAAAAAAAAgmwAD0h7yydDUnDAkkUzzn2xSRG8GwalWWqou+BD+9mCfhiVF9YMUm1erCKIIOtVpJOTyxBy9CekYRr3r15uwhhFR0xa+k24YkOl2M/AFQHg2hFFyQ1DRqQVxPHOLvDY33XmfR8NUSqtn6caW7O4d5Tf90unTYACfaysDxiG6OopBeGSNGFaNAd85XU8e60kTaPTihi9orqjeHh2pqeU4S4j9pVYovdi/Iv4/Ggv3GNIeSXS7YFYdnwEcklMNIuc7vL6kpIaOGPThW7qW3RD7Q//HNtOd31s7l5XNBTsXuFncUzZeNM9SaF6NK0wSuIG5jO/TaSBVAelcB/Olu8/tkjlLAjyOUlAwyKXtAwVYg5W/iJ067tfw4AVblSdfhip9OVAhzsxD4zfSEUn9BaGOZ4n+Q68OgmysZOkeF0Pkjew7zwnXKyxegU6FrMvTJd/8HfKO6X+/AxF60++RYDqXJ/+DOBgA), and [2](https://topaz.github.io/paste/#XQAAAQDHAwAAAAAAAAAgmwAD0h7yydDUnDAkkUzzn2xSRG8GwalWWqou+BD+9mCfhiVF9YMUm1erCKIIOtVpJOTyxBy9CekYRr3r15uwhhFR0xa+k24YkOl2M/AFQHg2hFFyQ1DRqQVxPHOLvDY33XmfR8NUSqtn6caW7O4d5Tf90unTYACfaysDxiG6OopBeGSNGFaNAd85XU8e60kTaPTihi9orqjeHh2pqeU4S4j9pVYovdi/Iv4/Ggv3GNIeSXS7YFYdnwEcklMNIuc7vL6kpIaOGPThlaCjJRP6mX3vm2DiW3VjWFksMaEgXONssD3iNtoq3uQVPbFpSyovVwj942gIKq/iPaoqz7rOxll8LGYrcSpYhT+RHkDnxWSorTjt/juvXPYr4LaQMF5WwM9DVUa9c09HFRx466lmUEVLfHdKe3nJUsNzbGvQ47/MgLC0LEiv7i0dglb4KYq1MKyjzIUdWhGRuzlly5fJosTTAnB2WW2vWPf/RqnVAA==)), so I can live with that.
(defun number-of-matches (molecule replacement)
  (destructuring-bind (from . to) replacement
    (length (cl-ppcre:all-matches to molecule))))

(defun part2 (input)
  (destructuring-bind (replacements . molecule) input
    (labels ((recur (molecule steps)
               (cond ((string= molecule "e") (return-from part2 steps))
                       (setf replacements
                             (sort (copy-seq replacements) #'<
                                   :key (partial-1 #'number-of-matches molecule)))
                       (loop for (from . to) in replacements
                             for matches = (cl-ppcre:all-matches to molecule)
                             do (loop for start in matches by #'cddr
                                      do (recur (mreplace molecule to from start)
                                                (1+ steps))))))))
      (recur molecule 0))))
Final plumbing:
(define-solution (2015 19) (input parse-input)
  (values (part1 input) (part2 input)))

(define-test (2015 19) (576 207))
And that's it!
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.242 seconds of real time
  0.239820 seconds of total run time (0.237884 user, 0.001936 system)
  99.17% CPU
  557,004,564 processor cycles
  41,759,200 bytes consed

2021-01-10 (permalink)

Advent of Code: [2015/16](https://adventofcode.com/2015/day/16)

Your Aunt Sue has given you a wonderful gift, and you'd like to send her a thank you card. However, there's a small problem: she signed it "From, Aunt Sue". You have 500 Aunts named "Sue".

You are very lucky because you have your "My First Crime Scene Analysis Machine" with you, and you can use it on the gift to detect what's it made of; with that (i.e. the scan result) and a list of all the things you could remember about each of all the 500 aunt Sues, you hope you will be able to figure which aunt sent the gift to you.

The scan result is given -- not part of the _input_, but known upfront anyway -- and I decided to model it using an ALIST mapping each compound to the detected quantity:
(defparameter *mfcsam-message* '((:children . 3)
                                 (:cats . 7)
                                 (:samoyeds . 2)
                                 (:pomeranians . 3)
                                 (:akitas . 0)
                                 (:vizslas . 0)
                                 (:goldfish . 5)
                                 (:trees . 3)
                                 (:cars . 2)
                                 (:perfumes . 1)))
The actual input instead is the list of things we remember about each aunt:

Sue 1: goldfish: 9, cars: 0, samoyeds: 9

Sue 2: perfumes: 5, trees: 8, goldfish: 8

Sue 3: pomeranians: 2, akitas: 1, trees: 5

We are going to parse these into a single ALIST mapping aunts (i.e. their name), to the list of compounds we can remember for them (`:cl-ppcre` to the rescue):
(defun compound-name (string) (make-keyword (string-upcase string)))

(defun parse-compound (string)
  (cl-ppcre:register-groups-bind ((#'compound-name name) (#'parse-integer value))
      ("(\\w+): (\\d+)" string)
    (cons name value)))

(defun parse-compounds (string)
  (mapcar #'parse-compound
          (cl-ppcre:all-matches-as-strings "\\w+: \\d+" string)))

(defun parse-aunt-memories (string)
  (cl-ppcre:register-groups-bind ((#'parse-integer n) (#'parse-compounds compounds))
      ("Sue (\\d+): (.*)" string)
    (cons n compounds)))
(defun aunt-name (aunt) (first aunt))
(defun aunt-compound (aunt name) (rest (assoc name (rest aunt))))

(defun parse-memories (lines) (mapcar #'parse-aunt-memories lines))
With all the input mangling taken care of, we need to figure out which aunt sent the gift to us; we are going to to that iteratively:

- For each compound from our scan result
- For each _remaining_ aunt
- See if what we remember of that aunt matches or not the result of the scan
- If it does, then _keep_ the current aunt in the set of candidates, otherwise, remove it

Note: the list of things we remember for each aunt is **not** complete (we don't remember everything), so when filtering out candidates, we only want to exclude those aunts that have a value for the given compound which is different from the scanned one (if we don't remember, we cannot make any assumptions, so we will keep that aunt in the list in the list of candidates).

Anyway, all this _nicely_ translates to:
(defun absent-or-equal (value) (lambda (v) (or (not v) (= v value))))

(defun part1 (memoirs &aux (remaining memoirs))
  (loop for (name . value) in *mfcsam-message* do
        (setf remaining (remove-if-not
                          (absent-or-equal value)
                          :key (partial-1 #'aunt-compound _ name))))
  (aunt-name (first remaining)))
In part 2 we realize the scanner is not as _precise_ as we thought it was, and that for certain compounds it's only able to tell us if the detected amount is above / below a given threshold:

In particular, the cats and trees readings indicates that there are greater than that many (due to the unpredictable nuclear decay of cat dander and tree pollen), while the pomeranians and goldfish readings indicate that there are fewer than that many (due to the modial interaction of magnetoreluctance).

Here we create a new function, ABSENT-OR-MATCHES-WITH-RANGES, to properly read the new scan result; rest is pretty much unchanged:
(defun absent-or-matches-with-ranges (name value)
  #'(lambda (v)
     (or (not v)
         (case name
           ((:cats :trees) (> v value))
           ((:pomeranians :goldfish) (< v value))
           (t (= v value))))))

(defun part2 (memoirs &aux (remaining memoirs))
  (loop for (name . value) in *mfcsam-message* do
        (setf remaining (remove-if-not
                          (absent-or-matches-with-ranges name value)
                          :key (partial-1 #'aunt-compound _ name))))
  (name (first remaining)))
Final plumbing:
(define-solution (2015 16) (memoirs parse-memories)
  (values (part1 memoirs) (part2 memoirs)))

(define-test (2015 16) (40 241))
And that's it!
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.006 seconds of real time
  0.005962 seconds of total run time (0.003711 user, 0.002251 system)
  100.00% CPU
  15,882,844 processor cycles
  818,768 bytes consed

Advent of Code: [2015/17](https://adventofcode.com/2015/day/17)

The elves bought too much eggnod (`150` liters to be precise), and you are asked to help them put it into smaller containers (your input). How many different _combinations_ of containers can you come up with, to fit all that eggnod?

The input is a list of integers, one per line, so I am just going to use my utility function PARSE-INTEGERS for this.

For the actual solution of this problem, I am going to generate all the possible combinations of containers which add up to `150`, in a recursive fashion:

- Until you have containers left to use, or the remaining eggnod is greater than zero
- Try to use the first available container, and recurse
- Or skip the first container, and recurse
- When the remaining eggnod is zero, there we found a possible solution -- save it somewhere
(defun solve (containers &aux solutions)
  (labels ((recur (containers target solution)
             (cond ((zerop target) (push solution solutions))
                   ((< target 0) nil)
                   ((null containers) nil)
                   (t (recur (rest containers) target solution)
                      (recur (rest containers)
                             (- target (first containers))
                             (cons (first containers) solution))))))
    (recur containers 150 nil)
With this, the solution to part 1 simply becomes applying LENGTH to the result of invoking SOLVE with the given list of containers:
(defun part1 (containers) (length (solve containers)))
For part 2, we are asked to count the solutions that use the least number of containers:
(defun part2 (containers &aux (solutions (solve containers)))
  (let ((min (reduce #'min solutions :key #'length)))
    (count min solutions :key #'length)))
Final plumbing:
(define-solution (2015 17) (containers parse-integers)
  (values (part1 containers) (part2 containers)))

(define-test (2015 17) (1304 18))
And that's it!
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.024 seconds of real time
  0.017904 seconds of total run time (0.006950 user, 0.010954 system)
  75.00% CPU
  56,414,728 processor cycles
  3,440,640 bytes consed

Advent of Code: [2015/18](https://adventofcode.com/2015/day/18)

Looks like we got a little [Game of life](https://en.wikipedia.org/wiki/Conway%27s_Game_of_Life) problem today:

- Given a grid of `1000x1000` lights -- for each, whether it's on or off
- Given a set of rules telling if a specific light should turn on or off based on the state of its 8 adjacent lights -- it turns out the given rules match Conway's

How many lights will be on, after 100 iterations?

Let's begin with parsing the input grid into a bag (i.e. HASH-SET) of coordinates, representing the lights which are on.
(defun parse-grid (lines &aux (grid (make-hset '())))
  (let ((row 1))
    (dolist (string lines grid)
      (loop for ch across string for col from 1
            when (char= ch #\#) do (hset-add (complex col row) grid))
      (incf row))))
With this, and thanks to my utility system, [:gameoflife](https://topaz.github.io/paste/#XQAAAQAbCQAAAAAAAAAUGQimgulVkMqJePdJ12sdAxC/GfcryUc62cNZO2Y9F49pcnga9KUiRymDklgNdDlqcv2dhjGOqX7N8ovjj8SaUTGvr5mN0/ulEICo5hK0220UqGzQVBykFN8pav/Pr4ITd4I3DKGyJRgy7/3T0ryWzgCloLL3GRvf/oikkYBmCALRghWKKt02J3vWavnPlqpnVW/0CYFBYT1CImg7lRpoFmbiwrr4lqIjkFF9HDELYMnxDSBE0fJlw3dgFW6rFaM4YMs0HVdk4wfjJRHUXLkZWm86TDvx+uDMnFWtXK7opquIKAqfRlRKnMzX1aTKqoNyJK/nZ4PRBS+eNkePip+rIxtmHT44/OMoMw+Ib0OR1tm/Wvc0JRb08vblxbpHcX0LLcGntGNYVBxdrKwjoDdRwe8bzTXy8x4o2tL8qyMSWHwsxyAsMwBJQCJSrGjWf0T4E9gkEPAyX8RiAJzcdL9UQDKNSyefq3naHa4/l+bFQlDabeLjVJYh/GDZpW/a5384/Vi/gIFqtFc/5Tlweqwvms+KrkC6ertm0ycdLUNug96wUPyZdA5LkQUb8sCujjboX6FNGfXgLclCf7vmKPj9osR4YueiK9a0DR1q2MvSF3JeFcv/5rLKlt3GY8dSpl9Rbe5fceaQ2pJ2ukOE1pFFqYzHnaSBr+Bf0PMpbba/VoViIzSHb+c3qM1/5PLCtjBPeCvazHVnOU4cI+N3nuzTRnoVItUWvIq4MWAVX6xtnpeRb/s+hq6VImuzAdgpyfUHBzlBq61pCK/qg5h9w+q7plvUXoB35EgDVikCU6wBFIlHz43SAZkaBHBxdNmpb7VKwuL0plBwewcZswzb9cyQnEkJo1Ec67BVaLc05c4mj/rGw+NbA5AL1glqawnAwQUmzKLxgb/sDbSfQcmEMH32YFW5ybZ8v9qkU9KL435TKdPfw8fMzt3teS1QBVFfpP246FC1C2g3clAOnwDNWKNeQ4BgoiXZlpK1S42WA7fM2yT+GC8//i0MgbL/tYtgI+Vl7ILQ9ALTARWez7Pjk9qZnPvA3pVeddhKzy++jwCXP4PP0nQjvoNrQnx5/eSMVasKElJQkV6G+7bfFGQDCh9NgiyD5/2/I+wuvlmCWmMDxCpBsiKhZhGq5rEgPxkAjJU2SxFtd2fLkrsSZJ8ZH9PGfhfTe6clCMmLmrotGQRXtb6LVaTVv3pnhuK+KxKUdzOT3AomFQ1wJyKAIbrMqbeUJrqNBTAKBnzLuEiBXW9HjLz7dJ/oN423vajgAWKyfmoRSIIbbLoozU8/bXZdhzEC3pO4qUAuJpCsQQcnnvb6zeq6ogzuppcq3VjSacH1VaU0czL6LFXVRBZQroPuRol2kHRnjoCpyw8ijSboqRkn117BvJhCOg4P//hNZK8=), that I created to model game of life dynamics, solving this problem should be pretty simple (the only difference with the _classic_ game of life is that here the grid is limited, hence my custom NEIGHBORS function to filter out of bounds positions):
(defparameter *nhood-deltas* '(#C(-1 1) #C(0 1) #C(1 1) #C(-1 0) #C(1 0) #C(-1 -1) #C(0 -1) #C(1 -1)))

(defun neighbors (pos)
  (loop for d in *nhood-deltas* for n = (+ pos d)
        when (and (<= 1 (realpart n) 100) (<= 1 (imagpart n) 100))
        collect n))

(defun part1 (grid)
  (dotimes (n 100 (hset-size grid))
    (setf grid (gol:next grid :neighbors #'neighbors))))
For part 2 we notice the lights placed at the four corners of the grid are _always_ on; how many lights will be on after 100 iterations?  All we have to do, is set those 4 corners on, after each iteration of the game:
(defun turn-corners-on (grid)
  (hset-add #C(1 1) grid)
  (hset-add #C(1 100) grid)
  (hset-add #C(100 1) grid)
  (hset-add #C(100 100) grid)

(defun part2 (grid &aux (grid (turn-corners-on grid)))
  (dotimes (n 100 (hset-size grid))
    (setf grid (turn-corners-on (gol:next grid :neighbors #'neighbors)))))
Final plumbing:
(define-solution (2015 18) (grid parse-grid)
  (values (part1 grid) (part2 grid)))

(define-test (2015 18) (814 924))
And that's it!
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.529 seconds of real time
  0.521719 seconds of total run time (0.515042 user, 0.006677 system)
  [ Run times consist of 0.009 seconds GC time, and 0.513 seconds non-GC time. ]
  98.68% CPU
  1,217,105,812 processor cycles
  280,039,008 bytes consed

2021-01-09 (permalink)

Advent of Code: [2015/15](https://adventofcode.com/2015/day/15)

Today, you set out on the task of perfecting your milk-dunking cookie recipe. All you have to do is find the right balance of ingredients.

Given a list of ingredients (each listing their properties like `capacity`, `durability`, or `calories`), and knowing that there are only `100` teaspoons worth of ingredients left to complete the recipe, which ingredients you are going to use, to maximize the _quality_ of the cookie.

As usual, we will start by parsing the list of ingredients (e.g `Sprinkles: capacity 5, durability -1, flavor 0, texture 0, calories 5`) into -- guess what -- a LIST of properties (note: `capacity` is always the first property, `durability` the second, and so on and so forth, so that makes the parsing a bit easier):
(defun parse-properties (string)
  (mapcar #'parse-integer (cl-ppcre:all-matches-as-strings "-?\\d+" string)))
(defun calories (props) (nth 4 props))

(defun parse-ingredient (string)
  (cl-ppcre:register-groups-bind (name (#'parse-properties properties))
      ("(\\w+): (.*)" string)
    (cons name properties)))
(defun properties (x) (cdr x))

(defun parse-ingredients (lines) (mapcar #'parse-ingredient lines))
We only have `4` ingredients, and a maximum `100` teaspoons to play with, so I am going to brute-force this.

The first step is to list all the possible recipes:

- How many teaspoons of `Sprinkles`? From `0` up to `100`
- How many teaspoons of `PeanutButter`? From `0` up to `100 - sprinkles` (we might have used some teaspoons for sprinkles already, so we need to account for that)
- How many teaspoons of `Frosting`? From `0` up to `100 - sprinkles - peanut`
- How many teaspoons of `Sugar`? `100 - sprinkles - peanut - frosting`

For each of these recipes, make a cookie out of it and calculate its _total score_ (i.e. product of all its _non-negative_ properties, ignoring `calories` for now).  The solution to part 1 will be the maximum of all these scores:
(defun make-cookie (ingredients quantities)
  (assert (= (reduce #'+ quantities) 100))
  (apply #'mapcar
         (loop for i in ingredients for q in quantities
               collect (mapcar (partial-1 #'* q) (properties i)))))

(defun total-score (cookie)
  (reduce #'* (remove-if (partial-1 #'< _ 0) (subseq cookie 0 4))))

(defun part1 (ingredients &aux best)
  (labels ((recur (ingredients-left spoons-left recipe)
             (cond ((= ingredients-left 1) (total-score
                                               (cons spoons-left recipe))))
                   (t (loop for spoons from 0 to spoons-left
                            maximize (recur
                                       (1- ingredients-left)
                                       (- spoons-left spoons)
                                       (cons spoons recipe)))))))
    (recur (length ingredients) 100 '())))
For part 2, we are asked to consider only those cookies that turn out to have `500` calories -- they want to use these as meal they say -- so all we have to do, is change our base case condition and have it return the cookie score if the number of calories is `500`, or `0` otherwise (we will be maximizing the result, so we know `0` would not get in the way):
(defun part2 (ingredients)
  (labels ((recur (ingredients-left spoons-left recipe)
             (cond ((= ingredients-left 1)
                    (let ((cookie (make-cookie
                                    (cons spoons-left recipe))))
                      (if (= (calories cookie) 500)
                        (total-score cookie)
                   (t (loop for spoons from 0 to spoons-left
                            maximize (recur
                                       (1- ingredients-left)
                                       (- spoons-left spoons)
                                       (cons spoons recipe)))))))
    (recur (length ingredients) 100 '())))
Final plumbing:
(define-solution (2015 15) (ingredients parse-ingredients)
  (values (part1 ingredients) (part2 ingredients)))

(define-test (2015 15) (13882464 11171160))
And that's it!
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.280 seconds of real time
  0.275511 seconds of total run time (0.230008 user, 0.045503 system)
  [ Run times consist of 0.005 seconds GC time, and 0.271 seconds non-GC time. ]
  98.57% CPU
  645,482,361 processor cycles
  241,477,936 bytes consed

2021-01-08 (permalink)

Advent of Code: [2015/13](https://adventofcode.com/2015/day/13)

Today, we are challenged with finding the _optimal seating arrangement_ that minimizes those awkward conversations.

Given a **round** table, a list of guests, and how much each of them would love / hate to sit next to every other guest on your list list; what's the seating arrangement that maximises the happiness of the table?

Let's start by looking at the entries of the input we are expected to process:

Alice would lose 75 happiness units by sitting next to David.

Alice would gain 71 happiness units by sitting next to Eric.

We will first parse each entry into 3 elements lists (e.g. `("Alice" "David" -75)`), and then store them all into an association list mapping from guests, to how more / less happier they would be if they were to sit next to another guest:
(defun parse-note (string)
  (cl-ppcre:register-groups-bind (person1 action (#'parse-integer amount) person2)
      ("(\\w+) would (lose|gain) (\\d+) happiness units by sitting next to (\\w+)." string)
    (list person1 person2 (* amount (if (string= action "lose") -1 1)))))

(defun parse-notes (lines &aux notes)
  (loop for string in lines for (person1 person2 delta) = (parse-note string)
        for existing = (assoc person1 notes :test #'string=)
        if existing do (push (cons person2 delta) (rest existing))
        else do (push (list person1 (cons person2 delta)) notes))
(defun all-people (notes) (mapcar #'first notes))
How can we find the optimal arrangement? Well, it turns out there are only 8 people in our guest list, so we might as well try all the possible arrangements and find the best one.

First we define functions to calculate the positive/negative delta in happiness, if two given people happened to be sitting next to one another...
(defun delta-happiness (notes person1 person2)
  (flet ((lookup (person1 person2)
           (let ((row (rest (assoc person1 notes :test #'string=))))
             (rest (assoc person2 row :test #'string=)))))
    (+ (lookup person1 person2) (lookup person2 person1))))
...and to calculate the total happiness associated with a specific seating arrangement:
(defun circular-table-happyness (notes table)
  (loop for prev = (car (last table)) then curr
        for curr in table
        sum (delta-happiness notes prev curr)))
With this, all there is left to do is generating all the possible seating arrangements, and pick the best one (i.e. the one that maximises CIRCULAR-TABLE-HAPPINESS):
(defun part1 (notes)
  (let ((people (all-people notes)))
    (reduce #'max (all-permutations people)
            :key (partial-1 #'circular-table-happyness notes))))
For part 2, you realize you forgot to add your name to the guest list, so you are asked to find the new best seating arrangement, knowing that you would not make anyone happier/sadder by sitting next to them.

At first I thought about literally implementing this: add a new guest to the list, change LOOKUP inside DELTA-HAPPINESS to return `0` if called for the new guest, and try all the possible combinations (there would be `9!` of them, which is still OK); but then I realized that since nobody is neither positively nor negatively affected by me sitting next to them, we can remove the new guest from the picture, and simply imagine the table not to be _circular_ at all!

Say you have this table arrangement: `A B C D A` (it's a circular table, hence the two `A`s), and wanted to see what would happen to the happiness of the table if I, the new guest, were to sit in between `B` and `C` (i.e. `A B C @ D A`); well, since neither `C` nor `D` will be positively / negatively affected by me sitting next to them, we can then calculate the happiness of the arrangement `D A B C`, without the wrapping around part (in a sense, the table has now become linear).

So we don't have to add ourselves to the list, but we simply have to change how the happiness of the table is calculated!
(defun linear-table-happyness (notes table)
  (loop for prev in table for curr in (rest table)
        sum (delta-happiness notes prev curr)))

(defun part2 (notes)
  (let ((people (all-people notes)))
    (reduce #'max (all-permutations people)
            :key (partial-1 #'linear-table-happyness notes))))
Final plumbing:
(define-solution (2015 13) (notes parse-notes)
  (values (part1 notes) (part2 notes)))

(define-test (2015 13) (618 601))
And that's it!
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.485 seconds of real time
  0.444067 seconds of total run time (0.440338 user, 0.003729 system)
  91.55% CPU
  1,117,039,360 processor cycles
  34,144,272 bytes consed

Advent of Code: [2015/14](https://adventofcode.com/2015/day/14)

This year is the Reindeer Olympics! Reindeer can fly at high speeds, but must rest occasionally to recover their energy. Santa would like to know which of his reindeer is fastest, and so he has them race.

This is the kind of input entries we will be dealing with:

Comet can fly 14 km/s for 10 seconds, but then must rest for 127 seconds.

We will parse these into lists listing:

- The name of the reindeer (not really needed, but whatever)
- How fast the reindeer is
- How long it can run for
- How long it has to rest for before start running again
(defun parse-reindeer (string)
  (cl-ppcre:register-groups-bind (name (#'parse-integer speed run-for rest-for))
      ("(\\w+) can fly (\\d+) km/s for (\\d+) seconds, but then must rest for (\\d+) seconds." string)
    (list name speed run-for rest-for)))

(defun parse-reindeers (lines) (mapcar #'parse-reindeer lines))
Each reindeer seems to have a _period_ (given by the sum of how long it can run and how long it has to rest after the run), and during this period they will travel a distance equal to their speed; with this, knowing how far a given reindeer would have traveled in a given amount of time, should become a matter of figuring out the number of activity _cycles_ that fit into the given amount of time, and then multiply that by the reindeer speed (you will also have to deal with partial cycles, but you get the idea):
(defun traveled-distance (time reindeer)
  (destructuring-bind (speed run-for rest-for) (rest reindeer)
    (let* ((period (+ run-for rest-for))
           (cycles (floor time period))
           (remaining (mod time period)))
      (* speed (+ (* cycles run-for) (min remaining run-for))))))

(defun part1 (reindeers)
  (reduce #'max reindeers :key (partial-1 #'traveled-distance 2503)))
For part 2, Santa wants you to tell not the reindeer that traveled the most, but the one that scored the highest number of points (every second, the leading reindeer(s), would get `1` point).

I could not figure out a way to solve this _mathematically_ (I am not even sure such solution exists), so I went on and implemented a _simulation_ of this game:

- `scores` represents the score of each reindeer
- `distances` represents the distance each reindeer has traveled
- `run-fors` represents how long each reindeer has been running for (in the current running activity)
- `rest-fors` represents how long each reindeer has been resting for (again, in the current running activity)
- for each simulation _tick_, see which reindeer can move (and have it move), which one needs to rest (and have it rest), and finally assign `1` point to the leading reindeer or reindeers
(defun part2 (reindeers)
  (loop repeat 2503
        with scores = (make-list (length reindeers) :initial-element 0)
        with distances = (make-list (length reindeers) :initial-element 0)
        with run-fors = (make-list (length reindeers) :initial-element 0)
        with rest-fors = (make-list (length reindeers) :initial-element 0)
        do (loop for (name speed run-for rest-for) in reindeers for index from 0
                 if (< (nth index run-fors) run-for)
                    do (incf (nth index distances) speed)
                       (incf (nth index run-fors))
                 else if (< (nth index rest-fors) rest-for)
                    do (incf (nth index rest-fors))
                 if (= (nth index rest-fors) rest-for)
                    do (setf (nth index run-fors) 0
                             (nth index rest-fors) 0))
        do (loop with furthest = (reduce #'max distances)
                 for d in distances for index from 0
                 when (= d furthest) do (incf (nth index scores)))
        finally (return (reduce #'max scores))))
Final plumbing:
(define-solution (2015 14) (reindeers parse-reindeers)
  (values (part1 reindeers) (part2 reindeers)))

(define-test (2015 14) (2660 1256))
And that's it!
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.003 seconds of real time
  0.003228 seconds of total run time (0.003116 user, 0.000112 system)
  100.00% CPU
  7,435,484 processor cycles
  31,296 bytes consed

2021-01-07 (permalink)

Advent of Code: [2015/10](https://adventofcode.com/2015/day/10)

Today's task, is to implement the [Look-and-say sequence](https://en.wikipedia.org/wiki/Look-and-say_sequence); in particular, given an initial list of digits, we are asked to "play" this game for 40 rounds, and to return the length of the result.

We start off by parsing the input into a list of digits (I am using `(cl-ppcre:split "" ...)` to split the input string into its signle character values):
(defun parse-digits (lines)
(map 'list #'parse-integer (cl-ppcre:split "" (first lines))))
Then we go ahead, and implement the rules of the game (the careful reader would have noticed by now that this is exactly the same as implementing [run-length encoding](https://en.wikipedia.org/wiki/Run-length_encoding)):
(defun look-and-say (digits)
(loop with n = 1
        for (curr . rest) on digits for next = (car rest)
        if (eql curr next) do (incf n)
        else append (list n curr) and do (setf n 1)))

(defun play (digits times)
(dotimes (n times digits)
    (setf digits (look-and-say digits))))
With this, the answer to part 1 becomes a simple:
(defun part1 (digits) (length (play digits 40)))
For part 2 instead, we are asked to apply the same process 50 times. Ok...
(defun part2 (digits) (length (play digits 50)))
Final plumbing:
(define-solution (2015 10) (digits parse-digits)
(values (part1 digits) (part2 digits)))

(define-test (2015 10) (492982 6989950))
And that's it!
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.367 seconds of real time
  0.363077 seconds of total run time (0.337332 user, 0.025745 system)
  [ Run times consist of 0.183 seconds GC time, and 0.181 seconds non-GC time. ]
  98.91% CPU
  844,084,929 processor cycles
  514,162,784 bytes consed

Advent of Code: [2015/11](https://adventofcode.com/2015/day/11)

Santa's password has expired, and we are asked to help him find the next one to use; we know Santa is a lazy guy, and will try _increase_ the current password, one step at a time (i.e. `ay`, `az`, `ba`), until he finds one that matches certain requirements:

Passwords must include one increasing straight of at least three letters, like `abc`, `bcd`, `cde`, and so on, up to `xyz`. They cannot skip letters; abd doesn't count.

Passwords may not contain the letters `i`, `o`, or `l`, as these letters can be mistaken for other characters and are therefore confusing.

Passwords must contain at least two different, non-overlapping pairs of letters, like `aa`, `bb`, or `zz`.

I think a brute-force approach will do just fine:

- Generate the next password
- Check whether if it violates or not, any of the policies
- If it does, repeat, otherwise, there you have Santa's new password

Let's start by implementing functions that will aid with the generation of the next password (note: given how the next password has to be generated, I opted for reversing the character of the password first, that way I could come up with a nice _recursive_ solution):
(defun next-char (ch)
  (if (char= ch #\z) #\a (code-char (1+ (char-code ch)))))

(defun next-password (password)
  (labels ((recur (password &aux
                           (ch (first password)) (ch-next (next-char ch)))
             (cond ((char/= ch-next #\a) (cons ch-next (rest password)))
                   (t (cons ch-next (recur (rest password)))))))
    (recur password)))
The next step is to implement functions to check the 3 different requirements (note: since the characters in my password are in reverse order, the first requirement, three _increasing_ straight characters now becomes checking for three _decreasing_ straight characters instead):
(defun three-decreasing-straight-p (password)
  (loop for ch1 in password for code1 = (char-code ch1)
        for ch2 in (cdr password) for code2 = (char-code ch2)
        for ch3 in (cddr password) for code3 = (char-code ch3)
        thereis (= code1 (1+ code2) (+ code3 2))))

(defun not-confusing-p (password)
  (not (some (lambda (ch) (find ch "ilo" :test #'char=)) password)))

(defun two-different-non-overlapping-pairs-p (password)
  (loop for (ch1 ch2 . rest) on password
        thereis (and (eql ch1 ch2)
                     (loop for (ch3 ch4 ch5) on rest
                           thereis (and (not (eql ch2 ch3))
                                        (eql ch3 ch4)
                                        (not (eql ch4 ch5)))))))

(defun password-valid-p (password)
  (and (three-decreasing-straight-p password)
       (not-confusing-p password)
       (two-different-non-overlapping-pairs-p password)))
Some final glue, and we should be all set:
(defun new-password (string)
  (flet ((string-to-password (string)
           (reverse (mapcar #'parse-char (cl-ppcre:split "" string))))
         (password-to-string (password)
           (format nil "~{~A~}" (reverse password))))
    (loop with curr = (string-to-password string)
          do (setf curr (next-password curr))
          when (password-valid-p curr) return (password-to-string curr))))
For Part 2 we are asked to find the next new password, after the one we just found, so a simple simple `(new-password part1)` will get the job done.

Final plumbing:
(define-solution (2015 11) (password first)
  (let ((part1 (new-password password)))
    (values part1 (new-password part1))))

(define-test (2015 11) ("vzbxxyzz" "vzcaabcc"))
And that's it!
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.092 seconds of real time
  0.077239 seconds of total run time (0.075929 user, 0.001310 system)
  83.70% CPU
  212,365,920 processor cycles
  19,824,640 bytes consed

Advent of Code: [2015/12](https://adventofcode.com/2015/day/12)

Given a string with some JSON in it, what is the sum of all numbers in the document?

I believe you should be able to answer this with the use of regular expressions only, but since I never dealt with JSON in CL before, I figured I would use this as an opportunity to play with [some libraries](https://sabracrolleton.github.io/json-review), and ultimately decided to go with [st-json](https://marijnhaverbeke.nl/st-json/).

`:st-json` conveniently ship. with READ-JSON-FROM-STRING, so parsing the input (i.e. a single JSON string) turned out to be ridicolously easy to do:
(defun parse-jso (lines)
  (st-json:read-json-from-string (first lines)))
Now that I have my JavaScript object, in memory, I need a way to sum all of its numeric values (nested ones included); I am going to solve this recursively, as follows:

- If the current object is a STRING, then return `0`
- If the current object is a NUMBER, then return the object itself
- If the current object is a LIST, then recurse into each element, and sum the result
- Otherwise (it's an object), iterate all its values, recurse into them, and sum the result (note: MAPJSO is to JavaScript objects what MAPHASH is to HASH-TABLEs):
(defun part1 (jso)
  (labels ((recur (x)
             (cond ((stringp x) 0)
                   ((numberp x) x)
                   ((consp x) (reduce #'+ x :key #'recur))
                   (t (let ((sum 0))
                        (st-json:mapjso #'(lambda (k v)
                                           (declare (ignore k))
                                           (incf sum (recur v)))
    (recur jso)))
For part 2, we are told to ignore any object (and all of its children) which has any property with the value `"red"`.  This translates to the following (not the call to RETURN-FROM to break early out of MAPJSO):
(defun part2 (jso)
  (labels ((recur (x)
             (cond ((stringp x) 0)
                   ((numberp x) x)
                   ((consp x) (reduce #'+ x :key #'recur))
                   (t (let ((sum 0))
                        (st-json:mapjso #'(lambda (k v)
                                           (declare (ignore k))
                                           (when (equal v "red")
                                             (return-from recur 0))
                                           (incf sum (recur v)))
    (recur jso)))
Final plumbing:
(define-solution (2015 12) (jso parse-jso)
  (values (part1 jso) (part2 jso)))

(define-test (2015 12) (156366 96852))
That's it!
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.017 seconds of real time
  0.012932 seconds of total run time (0.008439 user, 0.004493 system)
  76.47% CPU
  39,439,061 processor cycles
  1,189,232 bytes consed

2021-01-06 (permalink)

Advent of Code: [2015/09](https://adventofcode.com/2015/day/9)

Given a list of locations and the distances between every pair (of locations), what's shortest distance you have to travel to visit them all?  This smells like classic [TSP](https://en.wikipedia.org/wiki/Travelling_salesman_problem), and by the look of the input file it appears we are dealing with just 8 different locations, which means testing all the possible combinations will not take years to complete.

Anyway, as usual, let's start with parsing the input.  We are given a bunch of lines representing the distance between two locations (e.g. `London to Dublin = 464`), and we would like to store them in such a way that answering questions like: "what's the distance between a and b" would be easy and quick at the same time.

I am going to use an association list (the input is small and I don't think using a HASH-TABLE will make much of a difference) and since the distance between `a` and `b` will always be the same as the distance between `b` and `a`:

- When adding a new a new entry in the alist, we will use the one with the lexicographically smaller name as key, and the other (plus the distance) as value
- When querying for the distance between two locations, we will use the lexicographically smaller location name as alist key

Anyway, the above translates to:
(defun string-min (string1 string2)
(if (string< string1 string2)
    (values string1 string2)
    (values string2 string1)))

(defun parse-connection (string)
(cl-ppcre:register-groups-bind (from to (#'parse-integer distance))
    ("(\\w+) to (\\w+) = (\\d+)" string)
    (multiple-value-bind (a b) (string-min from to)
    (list a b distance))))

(defun parse-connections (lines &aux table)
(loop for string in lines for (from to distance) = (parse-connection string)
        for existing = (assoc from table :test #'string=)
        if existing do (push (cons to distance) (rest existing))
        else do (push (list from (cons to distance)) table))
Like I said earlier, we are going to brute-force all the possible paths, and find the shortest one.  First we need a way to extract the list of locations from the alist of connections:
(defun all-cities (connections)
    (loop for (from . cities) in connections collect from append
        (loop for (to) in cities collect to))
    :test #'string=))
Then, all we have to do is:

- Generate all the possible routes
- For each, calculate the total travelled distance
- Find the shortest one, and return its total distance
(defun distance (connections from to)
(multiple-value-bind (a b) (string-min from to)
    (let ((row (rest (assoc a connections :test #'string=))))
    (rest (assoc b row :test #'string=)))))

(defun route-distance (connections route)
(loop for from in route and to in (rest route)
        sum (distance connections from to)))

(defun part1 (connections &aux (cities (all-cities connections)))
(reduce #'min (all-permutations cities)
        :key (partial-1 #'route-distance connections)))
In part 2, Santa wants us to find the longest path instead (don't ask why).  Anyway, a simple change from `#'min` to `#'max` should do just fine:
(defun part2 (connections &aux (cities (all-cities connections)))
(reduce #'max (all-permutations cities)
        :key (partial-1 #'route-distance connections)))
Final plumbing:
(define-solution (2015 9) (connections parse-connections)
(values (part1 connections) (part2 connections)))

(define-test (2015 9) (141 736))
And that's it!
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
0.404 seconds of real time
0.203766 seconds of total run time (0.161870 user, 0.041896 system)
50.50% CPU
930,626,284 processor cycles
34,111,488 bytes consed

2021-01-03 (permalink)

* Changed [vim-lispindent](https://github.com/iamFIREcracker/vim-lispindent/commit/1d34ede0a1a1a66d55e08215e0c12cb0042b681b) to use a customized `indentexpr` function instead of the combination of 'lisp' mode and `&lispwords`. Note: I did not feel like re-implementing `lispindent` in Vimscript, so what I did instead was to call it with the content of the current buffer, have it indent lines, nicely, and then extract the indentation level for the current line -- not the best in terms of performance, but it gets the job done!

Advent of Code: [2015/07](https://adventofcode.com/2015/day/7)

We are are given a _circuit_ (i.e. wires, gates, and the values of some input signals), and we are asked to figure out the signal being transmitted on a specific wire, `a`.

As usual, let's begin by parsing our input:

123 -> x

456 -> y

x AND y -> d

x OR y -> e

x LSHIFT 2 -> f

y RSHIFT 2 -> g

NOT x -> h

NOT y -> i

Each line represent a connection between a signal and a wire (e.g. `123 -> x`), or a gate and a wire (e.g. `NOT y -> i`); signals are numeric inputs (e.g. `123`), while wires are _alphabetical_ ones (e.g. `x`); finally, gates can have one or two inputs (e.g. `NOT x` and `x LSHIFT 2` respectively).

My plan is parse each line into a list having as elements:

- the output wire -- since gates can have 1 or 2 arguments, I figured I would move the output wire first in the list
- the boolean function the _connection_ is describing (i.e. a function implementing the specific boolean operation)
- the arguments of the function
(defun parse-provision (string)
  (cl-ppcre:register-groups-bind (in out)
      ("^(\\w+) -> (\\w+)" string)
    (list out #'identity in)))

(defun parse-and (string)
  (cl-ppcre:register-groups-bind (in n out)
      ("^(\\w+) AND (\\w+) -> (\\w+)" string)
    (list out #'logand in n)))

(defun parse-or (string)
  (cl-ppcre:register-groups-bind (in n out)
      ("^(\\w+) OR (\\w+) -> (\\w+)" string)
    (list out #'logior in n)))

(defun parse-lshift (string)
  (cl-ppcre:register-groups-bind (in n out)
      ("^(\\w+) LSHIFT (\\w+) -> (\\w+)" string)
    (list out #'ash in n)))

(defun parse-rshift (string)
  (cl-ppcre:register-groups-bind (in n out)
      ("^(\\w+) RSHIFT (\\w+) -> (\\w+)" string)
    (list out #'(lambda (in n) (ash in (- n))) in n)))

(defun parse-not (string)
  (cl-ppcre:register-groups-bind (in out)
      ("^NOT (\\w+) -> (\\w+)" string)
    (list out #'lognot in)))

(defun parse-instruction (string)
  (or (parse-provision string)
      (parse-and string)
      (parse-or string)
      (parse-lshift string)
      (parse-rshift string)
      (parse-not string)))

(defun parse-instructions (lines)
  (mapcar #'parse-instruction lines))
How are we going to measure the signal on a given wire? (`a` in our case) I think a _recursive_ solution will do:

- Find the instruction listing our target wire as its output
- Evaluate the left-hand side of the expression (i.e. first recursively get the signal at its input(s), then apply the boolean function)
- I suspect the input circuit was created in such a way that the same gate would need to be evaluated multiple times, so I am going to keep track of the already processed gates, and their output signal (i.e. memoization)
(defun signal-at (wire instructions &aux (memo (make-hash-table :test 'equal)))
  (labels ((recur (wire &aux (cached (gethash wire memo)))
             (if cached
               (setf (gethash wire memo)
                     (cond ((digit-char-p (char wire 0)) (parse-integer wire))
                             (destructuring-bind (fun . args)
                                 (rest (assoc wire instructions :test #'string=))
                               (apply fun (mapcar #'recur args)))))))))
    (recur wire)))
For part 2 we are asked to override wire `b` with whichever signal we measured on wire `a` for part 1, and then measure wire `a` again, and luckily for us, we did not have to change anything except for the input:
(defun prepare-part2 (instructions part1)
    (list "b" #'identity (format nil "~A" part1))
Final plumbing:
(define-solution (2015 7) (instructions parse-instructions)
  (let ((part1 (signal-at "a" instructions)))
    (values part1
            (signal-at "a" (prepare-part2 instructions part1)))))

(define-test (2015 7) (46065 14134))
And that's it!
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.066 seconds of real time
  0.030142 seconds of total run time (0.009823 user, 0.020319 system)
  45.45% CPU
  152,529,453 processor cycles
  1 page fault
  359,520 bytes consed

Advent of Code: [2015/08](https://adventofcode.com/2015/day/8)

In today's problem we are given a list of strings having specific characters _escaped_, and we are asked to calculate the difference, in bytes, between the escaped and unescaped versions of these strings:

"" is 2 characters of code (the two double quotes), but the string contains zero characters.

"abc" is 5 characters of code, but 3 characters in the string data.

"aaa\"aaa" is 10 characters of code, but the string itself contains six "a" characters and a single, escaped quote character, for a total of 7 characters in the string data.

"\x27" is 6 characters of code, but the string itself contains just one - an apostrophe ('), escaped using hexadecimal notation.

Our input will come in as a list of strings already, so no additional effort will be required to parse the input.

The next step is to calculate the length of a string, knowing that certain characters will come in escaped, and hence need to be _unescaped_(i.e. `\\`, `\"`, and `\x` followed by 2 digits).

My idea was to keep an index, `index`, pointing to the next two characters in the string we should look at, `ch` and `ch1`, and:

- If `ch` is `\` and `ch1` is `x`, move `index` ahead `4` positions
- If `ch` is `\`, move `index` ahead `2` positions
- Otherwise, move `index` ahead `1` position

Count the number of times we do this (i.e. `sum 1`) and at the end of this subtract `2` corresponding to the surrounding quotes:
(defun length-unescaped (string &aux (index 0))
  (- (loop while (< index (length string))
           for ch = (char string index)
            for ch1 = (and (< (1+ index) (length string))
                                (char string (1+ index)))
            sum 1
            if (and (char= ch #\\) (char= ch1 #\x)) do (incf index 4)
            else if (char= ch #\\) do (incf index 2)
            else do (incf index 1))
With this, calculating how much the literla string representation differs from the in memory one should be as simple as:
(defun part1 (strings)
(- (reduce #'+ strings :key #'length)
    (reduce #'+ strings :key #'length-unescaped)))
Part 2 instead asks us to escape input strings first, and then calculate the difference between the escaped and unescaped versions (note: the unescaped version of a string in this case, is equal to the string itself, without modifications).

As it turns out, all we have to do while escaping a string, is:

- Surround the whole thing with quotes
- Escape `\` and `"`

This nicely translates to the following:
(defun length-escaped (string &aux (index 0))
(+ (loop while (< index (length string)) for ch = (char string index)
        sum (case ch ((#\" #\\) 2) (t 1))
        do (incf index))

(defun part2 (connections)
(let ((cities (mapcar #'first connections)))
    (reduce #'max (all-permutations cities)
            :key (partial-1 #'route-distance connections))))
Final plumbing:
(define-solution (2015 9) (connections parse-connections)
(values (part1 connections) (part2 connections)))

(define-test (2015 9) (141 736))
And that's it!
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
0.000 seconds of real time
0.000818 seconds of total run time (0.000714 user, 0.000104 system)
100.00% CPU
1,876,498 processor cycles
32,720 bytes consed

2021-01-02 (permalink)

? make vim-sexp _swap_ operations repeatable with `.`

Advent of Code: [2015/05](https://adventofcode.com/2015/day/5)

Given a list of strings, which ones are _nice_ and which ones are _naughty_?  A string is _nice_ if:

It contains at least three vowels (`aeiou` only), like `aei`, `xazegov`, or `aeiouaeiouaeiou`.

It contains at least one letter that appears twice in a row, like `xx`, `abcdde` (`dd`), or `aabbccdd` (`aa`, `bb`, `cc`, or `dd`).

It does not contain the strings `ab`, `cd`, `pq`, or `xy`, even if they are part of one of the other requirements.

There is not much we need to do to parse the input, as it will come in as a list of STRINGs already; rest is implementing a function to check the above rules, and then count which ones of input strings test positive to this predicate.  Note, I am going to use regular expressions (i.e. `:cl-ppcre` to implement the rule checking logic):
(defun nice-string-p (string)
  (and (>= (length (cl-ppcre:all-matches-as-strings "[aeiou]" string)) 3)
       (cl-ppcre:all-matches-as-strings "(\\w)\\1" string)
       (not (cl-ppcre:all-matches-as-strings "(ab|cd|pq|xy)" string))))

(defun part1 (strings)
  (count-if #'nice-string-p strings))
Part 2 is similar to part 1, except that the rule set is different:

It contains a pair of any two letters that appears at least twice in the string without overlapping, like `xyxy` (`xy`) or `aabcdefgaa` (`aa`), but not like `aaa` (`aa`, but it overlaps).

It contains at least one letter which repeats with exactly one letter between them, like `xyx`, `abcdefeghi` (`efe`), or even `aaa`.

Again, pretty easy to implement using regular expressions:
(defun improved-nice-string-p (string)
  (and (cl-ppcre:all-matches-as-strings "(.)(.).*\\1\\2" string)
       (cl-ppcre:all-matches-as-strings "(.).\\1" string)))

(defun part2 (strings)
  (count-if #'improved-nice-string-p strings))
Final plumbing:
(define-solution (2015 5) (strings)
  (values (part1 strings) (part2 strings)))

(define-test (2015 5) (238 1038736))
And that's it!
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.012 seconds of real time
  0.011163 seconds of total run time (0.009753 user, 0.001410 system)
  91.67% CPU
  27,942,267 processor cycles
  524,160 bytes consed

Advent of Code: [2015/06](https://adventofcode.com/2015/day/6)

Simulation problem: given a 1000x1000 lights grid, and a set of instructions to turn certain lights on or off, or to toggle their state, how many lights will be turned on at the end, after having executed all the instructions?

We start off by parsing the list of instructions, which can be any of the following:

- `turn on 1,2 through 3,4`
- `turn off 5,6 through 7,8`
- `toggle 9,10 through 11,12`

I am going to be using regular expressions again (note: `?:` is the syntax for [non-capturing groups](https://stackoverflow.com/a/3513858)):
(defun parse-coordinate-pair (string)
  (cl-ppcre:register-groups-bind ((#'parse-integer top left bottom right))
      ("(\\d+),(\\d+) through (\\d+),(\\d+)" string)
    (list top left bottom right)))

(defun parse-instruction (string)
  (cl-ppcre:register-groups-bind (action (#'parse-coordinate-pair pair))
      ("(?:turn )?(\\w+) (.+)" string)
    (cons (make-keyword (string-upcase action)) pair)))

(defun parse-instructions (lines)
  (mapcar #'parse-instruction lines))
This will take care of parsing instructions like the ones we have seen above, into:

- `(:on 1 2 3 4)`
- `(:off 5 6 7 8)`
- `(:toggle 9 10 11 12)`

The next step is to run the _simulation_.  I am going to use a two-dimensional ARRAY for the grid, storing T for when the specific light is turn on, and NIL otherwise.
(defun make-lights-grid (instructions)
  (let ((grid (make-array '(1000 1000) :initial-element nil)))
    (loop for (action top left bottom right) in instructions do
          (loop for row from top upto bottom do
                (loop for col from left upto right
                      for already-lit = (aref grid row col) do
                      (ecase action
                        (:toggle (setf (aref grid row col) (not already-lit)))
                        (:on (setf (aref grid row col) t))
                        (:off (setf (aref grid row col) nil))))))
The last step is to count the number of turned on lights (i.e. the number of T in the 2d array):
(defun part1 (instructions)
  (count t (array-elements (make-lights-grid instructions))))
For part 2, it turns out each light has adjustable brightness (it's not just on and off), and the instructions assume a slightly different meaning:

- `:on` means increasing the brightness by 1
- `:off` means decreasing the brightness by 1, without going negative
- `:toggle` means increasing the brightness by 2

After having executed all the revised instructions, what's the total brightness of the grid?

We are still going to use a two-dimensional array for the grid, but this time we will store numbers in it (i.e. the brightness value of the specific light):
(defun make-improved-lights-grid (instructions)
  (let ((grid (make-array '(1000 1000) :initial-element 0)))
    (loop for (action top left bottom right) in instructions do
          (loop for row from top upto bottom do
                (loop for col from left upto right do
                      (ecase action
                        (:toggle (incf (aref grid row col) 2))
                        (:on (incf (aref grid row col) 1))
                        (:off (unless (zerop (aref grid row col))
                                (decf (aref grid row col) 1)))))))

(defun part2 (instructions)
  (reduce #'+ (array-elements (make-improved-lights-grid instructions))))
Final plumbing:
(define-solution (2015 6) (instructions parse-instructions)
  (values (part1 instructions) (part2 instructions)))
(define-test (2015 6) (569999 17836115))
And that's it!
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.698 seconds of real time
  0.593268 seconds of total run time (0.452138 user, 0.141130 system)
  84.96% CPU
  1,606,779,011 processor cycles
  112,239,392 bytes consed

2021-01-01 (permalink)

Advent of Code: [2015/04](https://adventofcode.com/2015/day/4)

AdventCoins mining!

To do this, he needs to find MD5 hashes which, in hexadecimal, start with at least five zeroes. The input to the MD5 hash is some secret key (your puzzle input, given below) followed by a number in decimal. To mine AdventCoins, you must find Santa the lowest positive number (no leading zeroes: 1, 2, 3, ...) that produces such a hash.

Unfortunately there is not much we can do here except for bruteforcing (and before you ask, I did **not** implement MD5 myself...I found a package that did it for me):
(defun parse-secret (lines) (first lines))

(defun five-leading-zeros-p (md5-hash)
  (and (zerop (aref md5-hash 0))
       (zerop (aref md5-hash 1))
       (zerop (ldb (byte 4 4) (aref md5-hash 2)))))

(defun part1 (secret)
  (loop for n from 1 for input = (format nil "~A~D" secret n)
        for hash = (md5:md5sum-string input)
        when (five-leading-zeros-p hash) return n))
Part 2 is pretty much the same, except that now we are asked to find a hash with 6 leading `0`.
(defun six-leading-zeros-p (md5-hash)
  (and (zerop (aref md5-hash 0))
       (zerop (aref md5-hash 1))
       (zerop (aref md5-hash 2))))

(defun part2 (secret)
  (loop for n from 1 for input = (format nil "~A~D" secret n)
        for hash = (md5:md5sum-string input)
        when (six-leading-zeros-p hash) return n))
Final plumbing:
(define-solution (2015 4) (secret parse-secret)
  (values (part1 secret) (part2 secret)))

(define-test (2015 4) (254575 1038736))
And that's it!
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  1.087 seconds of real time
  1.071366 seconds of total run time (1.061611 user, 0.009755 system)
  [ Run times consist of 0.020 seconds GC time, and 1.052 seconds non-GC time. ]
  98.53% CPU
  2,500,813,970 processor cycles
  455,259,792 bytes consed

2020-12-31 (permalink)

Advent of Code: [2015/02](https://adventofcode.com/2015/day/2)

We are given a list presents/boxes dimensions (i.e. `12x34x45` with `12` being the length of the box, `34` being the width, and `45` the height), and we are asked to calculate how much paper is going to be required to wrap them all:

Fortunately, every present is a box (a perfect right rectangular prism), which makes calculating the required wrapping paper for each gift a little easier: find the surface area of the box, which is 2*l*w + 2*w*h + 2*h*l. The elves also need a little extra paper for each present: the area of the smallest side.

We start by parsing input lines into 3-element lists (I am using `:cl-ppcre` to do the splitting, but `:split-sequence` would have done just fine):
(defun parse-dimentions (string)
  (mapcar #'parse-integer (cl-ppcre:split "x" string)))

(defun parse-input (lines) (mapcar #'parse-dimentions lines))
Then we define a function to calculate the amount of required , apply it to each input set of dimensions, and sum each result.
(defun paper (dimensions)
  (destructuring-bind (l w h) dimensions
    (let ((lw (* l w))
          (wh (* w h))
          (hl (* h l)))
      (+ (+ (* 2 lw) (* 2 wh) (* 2 hl))
         (min (* lw) (* wh) (* hl))))))

(defun part1 (list-of-dimentions)
  (reduce #'+ list-of-dimentions :key #'paper))
What present box is it, without some ribbon? For part 2 we are asked to calculate how much ribbon we will need knowing that:

The ribbon required to wrap a present is the shortest distance around its sides, or the smallest perimeter of any one face.

Each present also requires a bow made out of ribbon as well; the feet of ribbon required for the perfect bow is equal to the cubic feet of volume of the present. Don't ask how they tie the bow, though; they'll never tell.

Again, we define a function to calculate how much ribbon is required for each box, apply it to each input set of dimensions, and sum the result:
 (defun ribbon (dimensions)
   (destructuring-bind (l w h) dimensions
     (let ((lw (+ l w))
           (wh (+ w h))
           (hl (+ h l)))
       (+ (* 2 (min lw wh hl))
          (* l w h)))))

 (defun part2 (list-of-dimentions)
   (reduce #'+ list-of-dimentions :key #'ribbon))
Final plumbing:
(define-solution (2015 2) (list-of-dimensions parse-input)
  (values (part1 list-of-dimensions) (part2 list-of-dimensions)))

(define-test (2015 2) (1598415 3812909))
And that's it!
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.004 seconds of real time
  0.003570 seconds of total run time (0.002333 user, 0.001237 system)
  100.00% CPU
  9,252,845 processor cycles
  360,400 bytes consed

Advent of Code: [2015/03](https://adventofcode.com/2015/day/3)

Here we are given a list of instructions Santa is going to follow while delivering the presents to the village houses, and we are asked to figure out how many houses will receive at leat one present (yes, following the instructions might bring Santa back to an already visited house).

Anyway, the input is a list of characters (i.e `^` for north, `v` south, `<` for west, and `>` for east), and I am going to parse these into COMPLEX numbers again, like we did for day 1:
(defun parse-instruction (ch)
  (ecase ch (#\< #c(-1 0)) (#\^ #c(0 1)) (#\> #c(1 0)) (#\v #c(0 -1))))

(defun parse-input (data)
  (map 'list #'parse-instruction (first data)))
Next, starting from position `0` we are going to move santa around, and keep track of the unique list of houses he visited (note: I could have used a HASH-TABLE, but given the input size I figured using a LIST and PUSHNEW would have done just fine):
(defun part1 (instructions)
  (loop with pos = 0 and houses = (list 0)
        for d in instructions
        do (pushnew (incf pos d) houses)
        finally (return (length houses))))
For part 2, the elves decide to pair Santa with a Robot (starting at position `0` with Santa), and split instructions amongst them:

- First instruction to Santa
- Second instruction to the robot
- Third instruction to Santa...
(defun part2 (instructions)
  (loop with santa = 0 and robot = 0 and houses = (list 0)
        for d in instructions for santa-turn-p = t then (not santa-turn-p)
        if santa-turn-p do (pushnew (incf santa d) houses)
        else do (pushnew (incf robot d) houses)
        finally (return (length houses))))
Final plumbing:
(define-solution (2015 3) (instructions parse-input)
  (values (part1 instructions) (part2 instructions)))

(define-test (2015 3) (2565 2639))
And that's it!
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.052 seconds of real time
  0.051087 seconds of total run time (0.048452 user, 0.002635 system)
  98.08% CPU
  120,408,717 processor cycles
  793,776 bytes consed

2020-12-30 (permalink)

Finally fixed some weird indentation problems with my Vim/CL setup

Every time one of my files included a `#\(`, _bad_ things will happen:

- matchit's `%` mappings would land me on the the closing parenthesis immediately _after_ the expected one
- vim-sexp's `vaf` and `vaF` mappings would end up selecting _partial_ forms
- vlime's would start suggesting positive indentation levels even for toplevel expressions

As it turns out all these plugins use `searchpairpos()` behind the scenes to find pairs of matching parentheses, and since that function already supports skipping over configured syntax regions, I _simply_ had to figure out a way to tell it which region to skip:

- matchit: I added `let b:match_skip = 's:comment\|string\|escape\|symbol'` to my vimrc
- vim-sexp: I had to create a [PR](https://github.com/guns/vim-sexp/pull/28)
- vlime: I had to create a [PR](https://github.com/vlime/vlime/pull/64)

No more `(char "(" 0)` hacks!

Advent of Code: [2015/01](https://adventofcode.com/2015/day/1)

We are given a string containing a bunch of `(` and `)` characters representing "move one floor up" and "move one floor down" instructions respectively:

- `(())` means go up two times, and then go down two times
- `(((` means go up 3 times
- `()(` means go up 1 time, then down 1 time, then again up 1 time

We are asked to find the floor we find ourselves at, after having followed all the instructions.

We start off by converting the input into a sequence of _floor movements_ (`(` becomes `1`, and `)` becomes `-1`):
(defun parse-instruction (ch)
  (ecase ch (#\( 1) (#\) -1)))

(defun parse-input (lines)
  (map 'list #'parse-instruction (first lines)))
With this, part 1 simply becomes a matter of summing all the read instructions together:
(defun part1 (instructions)
  (reduce #'+ instructions))
For part 2 instead, we are asked to find the first instruction (i.e. the instruction's _position_ in the list) that takes us to floor `-1`.  Not a big deal; instead of using REDUCE and +, we execute all the instructions inside a LOOP form so we can break out of it as soon as the current floor is `-1`:
(defun part2 (instructions &aux (floor 0))
  (loop for d in instructions and pos from 1
        do (incf floor d)
        when (= floor -1) return pos))
Final plumbing:
(define-solution (2015 1) (instructions parse-input)
  (values (part1 instructions) (part2 instructions)))

(define-test (2015 1) (138 1771))
And that's it!
> (time (test-run))
Success: 1 test, 2 checks.
Evaluation took:
  0.001 seconds of real time
  0.001664 seconds of total run time (0.000725 user, 0.000939 system)
  200.00% CPU
  3,820,178 processor cycles
  178,624 bytes consed

2020-12-29 (permalink)

TIL: the first LOOP's `:thereis` non-NIL form is also going to be the return value of the LOOP form itself
(loop for e in '(1 2 3 5 7)
      thereis (evenp e))
What if you wanted to return return the first _odd_ element, and not just whether such element exist?  You can use AND:
(loop for e in '(1 2 3 5 7)
      thereis (and (evenp e) e))

2020-12-26 (permalink)

Is it possible pass to CHECK-TYPE  a _parametric_ type? i.e. a type that depends on the run-time value of some other function argument?

An example: say I have a DIGITS-TO-NUMBER function accepting as arguments:

- the digits of the number -- i.e. list of integers, each less than...
- a given base

Is this _possible_? I think it should, especially because we are talking about run-time checking, but maybe there is something else I am not properly taking into account.

Anyway, first I defined the predicate/type for this:
(defun list-of-digits-p (digits base)
  (and (every #'numberp digits)
       (every #'(lambda (d) (< d base)) digits)))

(deftype list-of-digits (base)
  (let ((predicate (gensym)))
    (setf (symbol-function predicate)
          #'(lambda (digits) (list-of-digits-p digits base)))
    `(and list (satisfies ,predicate))))
Then I tried to add this to DIGITS-TO-NUMBER, `(check-type digits (list-of-digits base))`, but if you have a look at what CHECK-TYPE expands to, it's clear my attempt would not work:
(DO ()
I was able to achieve what I wanted with the following:
(unless (typep digits `(list-of-digits ,base)) (error "fuuuuu"))
But then I would need to re-implement the restart mechanism which is already in place if you use CHECK-TYPE right?

Well it turns out you can not:

but, the type passed to check-type must be static

if you want a dynamic type, then use assert typep instead

also, satisfies must be a single symbol

This translates to:
(defun digits-to-number (digits &optional (base 10))
  "Return the non-negative integer in base `base` from the list of
  its digits (this function is the inverse of DIGITS). By default,
  the decimal base is assumed.

The order of the digits is such that the `k`th element of the list
refers to the coefficient of `base^k`. In other words, given the input list

    (c0 c1 c2 ... ck)

the returned integer will be constructed as follows:

    c0 + c1*base + c2*base^2 + ... + ck*base^k."
  (check-type base (integer 2))
  (dolist (d digits)
    (assert (typep d `(integer 0 ,(1- base)))))
  (loop :for d :in digits :and c = 1 :then (* c base)
        :sum (* d c)))
And yes, if you want the restart functionality, then you'll need to modify the code a bit.

2020-12-25 (permalink)

Advent of Code: [2020/25](https://adventofcode.com/2020/day/25)


- Bruteforce the "loop-size" param for one of the 2 inputs
- _transform_ the other input with the found "loop-size"
(defparameter *magic* 20201227)

(defun transform (subject loop-size &aux (value 1))
  (dotimes (n loop-size value)
    (setf value (rem (* value subject) *magic*))))

(defun find-loop-size (target &aux (value 1))
  (loop for loop-size from 0
        when (= value target) return loop-size
        do (setf value (rem (* value 7) *magic*))))

(define-solution (2020 25) (integers parse-integers)
  (transform (first integers) (find-loop-size (second integers))))
And that's it, folks!
      --------Part 1--------   --------Part 2--------
Day       Time   Rank  Score       Time   Rank  Score
 25   01:04:57   3328      0   01:05:36   2677      0
 24   00:44:22   2518      0   00:53:15   1519      0
 23   02:41:43   4360      0   10:59:47   5948      0
 22   02:04:15   5221      0   04:08:58   4813      0
 21   04:55:08   5649      0   04:57:55   5338      0
 20   03:52:37   3838      0   05:16:07   1553      0
 19   08:52:01   8253      0   09:28:17   5746      0
 18   03:36:44   7136      0   04:16:03   6268      0
 17   01:46:03   4142      0   01:50:25   3668      0
 16   02:00:52   7462      0   04:35:01   7388      0
 15   01:47:30   7380      0   01:58:04   6301      0
 14   01:49:39   6897      0   07:32:55  11279      0
 13   01:33:50   8042      0   05:25:33   6909      0
 12   01:56:49   8063      0   02:02:13   6403      0
 11   02:34:06   9328      0   03:19:45   8054      0
 10   01:42:33  11666      0   02:18:21   6551      0
  9   01:36:51  11168      0   01:53:55  10491      0
  8   03:03:40  16452      0   03:20:49  13610      0
  7   03:28:16  12620      0   03:38:13  10016      0
  6   02:57:28  15332      0   03:02:12  13914      0
  5   01:35:54  10138      0   01:57:11  10141      0
  4   04:26:24  24047      0   06:03:48  21243      0
  3   01:38:15  11744      0   01:46:12  10883      0
  2   05:26:22  27699      0   05:42:43  26445      0
  1   05:13:06  20406      0   05:17:01  18468      0
PS. I started with AoC in 2018, to learn Common Lisp, and I have been using it since.
PPS. This is the first year that I managed to a) get all the 50 stars by the 25th, b) complete all the problems within 24 hours.
PPPS. I did not know you had to rush and click on the link to get your second star, so basically my part 2 completion time tells you something about how fast/slow I am at reading things ;-)

2020-12-24 (permalink)

Advent of Code: [2020/24](https://adventofcode.com/2020/day/24)


- [Pointy topped](https://www.redblobgames.com/grids/hexagons/) hex grids
- Parse instructions into list of steps, i.e. COMPLEX numbers pointing to the next, adjacent, tile
- Part 1: iterate all the instructions, execute all the steps, and flip the state of the tile you find yourself in at the end
- Part 2: another game of life <3 -- my utility libraries (i.e. [:hset](https://topaz.github.io/paste/#XQAAAQBWAwAAAAAAAAAUGknC0wGe+rxWxGbsuM8XKlXe+zDDeRr4lNRd45fFnhvNSenyD67lPZMOacbfl1vMLbImXQNGkRGm0LFaP9G10YtnY4Zaf8bWX8Td4J0uic8xm16hYOSoVvk8BIkKChEqlP7kbfGlKN9jE/jRjtUMzeK1gyEVQNY+RUDoIYDJANFNu1SZhTUGuQnFKdMGTqLAvG13qGropmbb8XSlBVNxNYatfyPAWFxOvGvcPQ3csBNV9Lu0Ko6fp9yALeEc0SXrRItn9ra7r5RsK6PXuKHSlV9cX8P5+jc4z1RDqgFWSaH/tytsU+skUEWp/akDGwakswDIuMN3HNrjU9UP02bs3FFSMZPm9xkNui00uEY9X0Mv/03HzZiPbzu7dmgSy4su/3sSJpozm9pGsNndqDHw1+NWwNPlH1d76RN2/E3pUBUFKlfkvx5mADArsx1+w8YT48cODPJn1xhJ3oci4Etp5lws5JmT0SM4RKZeC37j1mJ5xfNk1pTCQh/Epa8PbQvCU8gzcIy0A4jgQc9Vz7lUMjyhVaGrxSD0ETVSJs4Ja9ZP7T+W6okQd/t4hTg=) and [:gameoflife](https://topaz.github.io/paste/#XQAAAQAbCQAAAAAAAAAUGQimgulVkMqJePdJ12sdAxC/GfcryUc62cNZO2Y9F49pcnga9KUiRymDklgNdDlqcv2dhjGOqX7N8ovjj8SaUTGvr5mN0/ulEICo5hK0220UqGzQVBykFN8pav/Pr4ITd4I3DKGyJRgy7/3T0ryWzgCloLL3GRvf/oikkYBmCALRghWKKt02J3vWavnPlqpnVW/0CYFBYT1CImg7lRpoFmbiwrr4lqIjkFF9HDELYMnxDSBE0fJlw3dgFW6rFaM4YMs0HVdk4wfjJRHUXLkZWm86TDvx+uDMnFWtXK7opquIKAqfRlRKnMzX1aTKqoNyJK/nZ4PRBS+eNkePip+rIxtmHT44/OMoMw+Ib0OR1tm/Wvc0JRb08vblxbpHcX0LLcGntGNYVBxdrKwjoDdRwe8bzTXy8x4o2tL8qyMSWHwsxyAsMwBJQCJSrGjWf0T4E9gkEPAyX8RiAJzcdL9UQDKNSyefq3naHa4/l+bFQlDabeLjVJYh/GDZpW/a5384/Vi/gIFqtFc/5Tlweqwvms+KrkC6ertm0ycdLUNug96wUPyZdA5LkQUb8sCujjboX6FNGfXgLclCf7vmKPj9osR4YueiK9a0DR1q2MvSF3JeFcv/5rLKlt3GY8dSpl9Rbe5fceaQ2pJ2ukOE1pFFqYzHnaSBr+Bf0PMpbba/VoViIzSHb+c3qM1/5PLCtjBPeCvazHVnOU4cI+N3nuzTRnoVItUWvIq4MWAVX6xtnpeRb/s+hq6VImuzAdgpyfUHBzlBq61pCK/qg5h9w+q7plvUXoB35EgDVikCU6wBFIlHz43SAZkaBHBxdNmpb7VKwuL0plBwewcZswzb9cyQnEkJo1Ec67BVaLc05c4mj/rGw+NbA5AL1glqawnAwQUmzKLxgb/sDbSfQcmEMH32YFW5ybZ8v9qkU9KL435TKdPfw8fMzt3teS1QBVFfpP246FC1C2g3clAOnwDNWKNeQ4BgoiXZlpK1S42WA7fM2yT+GC8//i0MgbL/tYtgI+Vl7ILQ9ALTARWez7Pjk9qZnPvA3pVeddhKzy++jwCXP4PP0nQjvoNrQnx5/eSMVasKElJQkV6G+7bfFGQDCh9NgiyD5/2/I+wuvlmCWmMDxCpBsiKhZhGq5rEgPxkAjJU2SxFtd2fLkrsSZJ8ZH9PGfhfTe6clCMmLmrotGQRXtb6LVaTVv3pnhuK+KxKUdzOT3AomFQ1wJyKAIbrMqbeUJrqNBTAKBnzLuEiBXW9HjLz7dJ/oN423vajgAWKyfmoRSIIbbLoozU8/bXZdhzEC3pO4qUAuJpCsQQcnnvb6zeq6ogzuppcq3VjSacH1VaU0czL6LFXVRBZQroPuRol2kHRnjoCpyw8ijSboqRkn117BvJhCOg4P//hNZK8=)) are finally paying off
(defparameter *steps*
  '(("nw"  #c(-1/2 3/4))
    ("ne" #c(1/2 3/4))
    ("e"  #c(1 0))
    ("se" #c(1/2 -3/4))
    ("sw" #c(-1/2 -3/4))
    ("w"  #c(-1 0))))

(defun parse-step (string)
  (second (assoc string *steps* :test #'string=)))

(defun parse-instruction (string)
  (remove nil (mapcar #'parse-step
                     (cl-ppcre:split "(ne|nw|se|sw|w|e)" string
                                     :with-registers-p t))))

(defun parse-instructions (data)
  (mapcar #'parse-instruction data))

(defun create-floor (instructions)
  (loop with flipped = (make-hset '())
        for steps in instructions for delta = (reduce #'+ steps)
        if (hset-contains-p delta flipped) do (hset-rem delta flipped)
        else do (hset-add delta flipped)
        finally (return flipped)))

(defparameter *neighbors-deltas*
  '(#c(-1/2 3/4) #c(1/2 3/4) #c(1 0) #c(1/2 -3/4) #c(-1/2 -3/4) #c(-1 0)))

(defun neighbors (pos)
  (loop for delta in *neighbors-deltas* collect (+ pos delta)))

(defun should-be-flipped-p (pos n state)
  (or (= n 2) (and (= n 1) (hset-contains-p pos state))))

(define-solution (2020 24) (instructions parse-instructions)
  (let ((floor (create-floor instructions)))
    (values (hset-size floor)
            (dotimes (n 100 (hset-size floor))
              (setf floor
                    (gol:next floor :neighbors #'neighbors
                              :should-be-alive-p #'should-be-flipped-p))))))
PS. I woke up early today because I have errands to do (who doesn't, these days...), and figured I did not want to ruin today as well, especially the ridiculous amount of time I spent solving yesterday's problem. Guess what? You wake up early, the problem is easy, and you end up with a new personal score:
      --------Part 1--------   --------Part 2--------
Day       Time   Rank  Score       Time   Rank  Score
 24   00:44:22   2518      0   00:53:15   1519      0

TIL: you cannot freely use `and`s inside LOOP `for` expressions.. you need to be careful!
[SBCL] CL-USER> (loop for n upto 5 and square = (* n n) collect square)
(0 0 1 4 9 16)

[SBCL] CL-USER> (loop for n upto 5 for square = (* n n) collect square)
(0 1 4 9 16 25)

[SBCL] CL-USER> (loop for n in '(0 1 2 3 4 5) for square = (* n n) collect square)
(0 1 4 9 16 25)

[SBCL] CL-USER> (loop for n in '(0 1 2 3 4 5) and square = (* n n) collect square)
The value
is not of type
when binding SB-KERNEL::X
   [Condition of type TYPE-ERROR]

  R 0. ABORT - Exit debugger, returning to top level.

[SBCL] CL-USER> (loop for n upto 5 and m from 5 downto 0 collect (+ n m))
(5 5 5 5 5 5)

[SBCL] CL-USER> (loop for n in '(0 1 2 3 4 5) and m in '(5 4 3 2 1) collect (+ n m))
(5 5 5 5 5)
So it seems like you can use `and`s when chaining assignments of the _same type_ (i.e. sequence iterations with sequence iterations, number ranges with number ranges...).  From the Lisp Discord server:

AND does bindings and updates "in parallel", like LET/PSETF rather than sequentially like LET*/SETF, so you can't refer to the previous variable (you can in the update form, but it will refer to the previous value rather than the current one)

2020-12-23 (permalink)

Advent of Code: [2020/23](https://adventofcode.com/2020/day/23)


- Create a _circular_ list -- let's thank Quickutil's [NCYCLE](http://quickutil.org/list?q=ncycle)
- Part 1 (play the _damn_ game)
- select the _next_ three cups
- find the destination cup -- it's a circular list, so MEMBER here will do just fine
- update the list -- I used PSETF to update multiple places...in _parallel_
- Part 2 (play the _damn_ game, again, with 1M cups, and this time stop after 10M moves)
- it's _clear_ our calls to MEMBER are the bottleneck we should try to get rid of
- how? create a HASH-TABLE mapping cup labels to their _places_ in the circular list
- fuss around with LIST pointers
(defun parse-cups (data)
  (reverse (digits (parse-integer (first data)))))

(defun play (cups moves &aux (cups-count (length cups))
                  (cups (ncycle cups))
                  (cells (make-hash-table)))
  (labels ((one-less (current)
             (if (> current 1) (1- current) cups-count))
           (destination (cups)
             (loop with (curr cup1 cup2 cup3) = cups
                   for target = (one-less curr) then (one-less target)
                   if (/= target cup1 cup2 cup3)
                   return (gethash target cells))))
    (loop repeat cups-count
          for ref = cups then (cdr ref) do
          (setf (gethash (car ref) cells) ref))
    (loop repeat moves
          for curr = cups for cup1 = (nthcdr 1 cups)
          for cup3 = (nthcdr 3 cups) for after-cup3 = (cdr cup3)
          for dest = (destination cups) for after-dest = (cdr dest) do
          (psetf (cdr curr) after-cup3
                 (cdr cup3) after-dest
                 (cdr dest) cup1
                 cups (cdr cup3))
          ;; propagate above changes into `cells`
          (psetf (gethash (car after-cup3) cells) (cdr curr)
                 (gethash (car after-dest) cells) (cdr cup3)
                 (gethash (car cup1) cells) (cdr dest))
          finally (return (gethash 1 cells)))))

(defun prepare-part2 (cups total-cups)
  (append cups (iota (- total-cups (length cups)) :start 10)))

(define-solution (2020 23) (data)
  (values (loop repeat 8 ;; skip cup with label 1
                for cup in (rest (play (parse-cups data) 100))
                collect cup into digits
                finally (return (parse-integer (format nil "~{~A~}" digits))))
          (let ((cups (play (prepare-part2 (parse-cups data) 1000000) 10000000)))
            (* (cadr cups)
               (caddr cups)))))
How fast does this run?
[SBCL] AOC/2020/23> (time (solution-run))
Evaluation took:
  5.927 seconds of real time
  5.879336 seconds of total run time (5.767486 user, 0.111850 system)
  [ Run times consist of 0.058 seconds GC time, and 5.822 seconds non-GC time. ]
  99.19% CPU
  13,632,853,237 processor cycles
  130,224,736 bytes consed

It's probably not the fastest thing in the world, but it gets the job done!

2020-12-22 (permalink)

Advent of Code: [2020/22](https://adventofcode.com/2020/day/22)


- Nothing crazy, just do as told -- I thought about using a deque to implement a deck, but as it turns out a LIST will do just fine
- Part 1: pop, pop, compare, append, until one of the decks is empty
- Part 2: not as _simple_ as part 1, but close enough
- ALREADY-SEEN-P returns `t` if the current game (i.e. the two decks) has already been _played_ before
- MARK-AS-SEEN marks the current game as played
- PLAYER1-WINS-TURN-P is where most of the logic for part 2 lies (i.e. check if players have enough cards to recurse, then recurse or check the top of their decks)
- The rest is plumbing
(defun parse-decks (data)
  (let (groups current)
    (dolist (string (append data '("")))
      (if (string= string "")
        (setf groups (cons (reverse current) groups) current nil)
        (setf current (cons string current))))
      (mapcar #'parse-integer (rest (second groups)))
      (mapcar #'parse-integer (rest (first groups))))))

(defun play (decks)
  (loop with deck1 = (first decks) and deck2 = (second decks)
        while (and deck1 deck2)
        for c1 = (pop deck1) and c2 = (pop deck2)
        if (> c1 c2) do (setf deck1 (append deck1 (list c1 c2)))
        else do (setf deck2 (append deck2 (list c2 c1)))
        finally (return (values deck1 deck2))))

(defun play-recursive (decks)
  (labels ((already-seen-p (seen deck1 deck2)
             (gethash (list deck1 deck2) seen))
           (mark-as-seen (seen deck1 deck2)
             (setf (gethash (list deck1 deck2) seen) t))
           (player1-wins-turn-p (c1 deck1 c2 deck2)
             (if (or (< (length deck1) c1) (< (length deck2) c2))
               (> c1 c2)
               (recur (subseq deck1 0 c1) (subseq deck2 0 c2))))
           (recur (deck1 deck2)
             (loop with seen = (make-hash-table :test 'equal)
                   while (and deck1 deck2)
                   if (already-seen-p seen deck1 deck2) return deck1 else do
                   (mark-as-seen seen deck1 deck2)
                   (let ((c1 (pop deck1)) (c2 (pop deck2)))
                     (if (player1-wins-turn-p c1 deck1 c2 deck2)
                       (setf deck1 (append deck1 (list c1 c2)))
                       (setf deck2 (append deck2 (list c2 c1)))))
                   finally (return (values deck1 deck2)))))
    (recur (first decks) (second decks))))

(defun score (deck)
  (reduce #'+ (mapcar #'* (reverse deck) (iota (length deck) :start 1))))

(define-solution (2020 22) (decks parse-decks)
  (flet ((winner-score (deck1 deck2)
           (score (or (pr deck1) deck2))))
    (values (multiple-value-call #'winner-score (play decks))
            (multiple-value-call #'winner-score (play-recursive decks)))))
PS. I finally found a use case for [MULTIPLE-VALUE-CALL](http://clhs.lisp.se/Body/s_multip.htm)
PPS. I know this seemed easy, but it took me forever to solve part 2:

- I forgot to pass :test 'equal to my call to MAKE-HASH-TABLE -- making the "already played game" logic useless..
- Fixed that, I still got the "already played game" logic wrong; I know it was clearly stated in the text description, and I remember paying attention to it, still, I ended up caching across all the played games!
- Lastly, when preparing the decks for the nested game, I had a typo that caused my algorithm to use all the remaining decks and not just as many as the value of the card we just read (e.g. I had `(subseq (rest deck1) 0 n1)` instead `(subseq (rest deck1) 0 c1)`, where `c1` is the last read card, and `n1` is the size of the remaining deck), and because of that I started blaming on the used data structure and tried to implement this using a deque (and halfway through the newer implementation, I realized the silly mistake I had made...)

2020-12-21 (permalink)

Advent of Code: [2020/21](https://adventofcode.com/2020/day/21)


- Parse food into a list having the list of ingredients as first element, and the list of allergens as second -- i.e. `'((mxmxvkd kfcds sqjhc nhms) (contains dairy, fish))`
- Then find the `ingredient -> allergen` mapping, using backtracking
- Start with all the foods, and an empty mapping
- For each remaining food, for each of its ingredients, `i`, for each of its allergens, `a`, see if `i -> a` is valid mapping; if it is, update foods by removing all occurrences of `i` and `a` and carry on, otherwise, backtrack
- A new mapping is valid if, for each remaining food, either `a` is not one of its allergens, or if it is, then `i` **has** to be in the list of ingredients
- The stop condition is when there are no more allergens to try
- Part 1: I had my FIND-MAPPING function return the list of foods without a clear mapping, so I went on and counted the remaining ingredients
- Part 2: do as told -- sort the mapping by the allergen name, and concatenated the ingredients
- PS. This naive solution could take forever to complete if you are not careful at selecting which of the remaining foods to try first; I thought it would make sense to try first the foods with the smallest number of ingredients / allergens (see SORT-FOODS), and apparently that did the trick
(defun parse-ingredients (string)
  (cl-ppcre:split " " string))

(defun parse-allergens (string)
  (cl-ppcre:split ", " string))

(defun parse-food (string)
  (cl-ppcre:register-groups-bind ((#'parse-ingredients ingredients)
                                  (#'parse-allergens allergens))
      ("(.*) \\(contains (.*)\\)" string)
    (list ingredients allergens)))

(defun ingredients (food) (first food))
(defun allergens (food) (second food))

(defun parse-foods (data)
  (mapcar #'parse-food data))

(defun find-mapping (foods)
  (labels ((sort-foods (foods)
             (sort foods #'<
                   :key (lambda (food)
                          (* (length (ingredients food))
                             (length (allergens food))))))
           (foods-valid-p (foods) foods)
           (update-foods (foods i a)
             (loop for (ii aa) in foods
                   collect (list (remove i ii :test #'string=)
                                 (remove a aa :test #'string=))
                   when (and (member a aa :test #'string=)
                             (not (member i ii :test #'string=))) return nil))
           (recur (foods mapping &aux (foods (sort-foods foods)))
             (cond ((not (foods-valid-p foods)) nil)
                   ((every #'null (mapcar #'allergens foods))
                    (return-from find-mapping (values mapping foods)))
                     (loop for (ii aa) in foods do
                           (loop for i in ii do
                                 (loop for a in aa do
                                       (recur (update-foods foods i a)
                                              (cons (cons i a) mapping)))))))))
    (recur foods nil)))

(define-solution (2020 21) (foods parse-foods)
  (multiple-value-bind (mapping remaining) (find-mapping foods)
    (values (reduce #'+ (mapcar #'ingredients remaining) :key #'length)
            (format nil "~{~a~^,~}"
                    (mapcar #'first (sort mapping #'string< :key #'cdr))))))

Time to mop up [2020/16](https://matteolandi.net/plan.html#day-2020-12-16)'s solution.

Let's start with part 1:

- For each "nearby" ticket
- For each of its values
- See if it's in range range with any of the defined rules -- and if not, sum these values
(defun value-valid-p (rules value)
  (loop for (name (from1 . to1) (from2 . to2)) in rules
        thereis (or (<= from1 value to1)
                    (<= from2 value to2))))

(defun ticket-errors (rules ticket)
  (loop for value across ticket unless (value-valid-p rules value) sum value))

(defun ticket-scanning-error-rate (doc)
  (reduce #'+ (nearby-tickets doc)
          :key (lambda (ticket) (ticket-errors (rules doc) ticket))))
It's that simple: no arrays, no trying to be clever, KISS!

Now let's move on to part 2:

- First we filter out any _invalid_ ticket
- Then we find the mapping; how? Backtracking
- Start with a _table_ mapping each rule to the ticket positions for which **all** the "nearby" tickets have valid values for that rule, and an empty field mapping
- For each remaining rule `n`, try to use that at one of the _compatible_ ticket positions `p` -- remove `n` from the list of remaining rules, and remove `p` from all the remaining rules _compatible_ positions
- If the new set of rules is invalid -- i.e. there is at least a rule without _compatible_ ticket positions -- it means we reached a dead end, and we need to backtrack
- If we run out of rules to try, it means we found our mapping
- Otherwise, repeat
- PS. To speed things up, of all the remaining rules we will try and evaluate the ones with the least number of _compatible_ ticket positions first
(defun ticket-valid-p (rules ticket)
  (loop for value across ticket always (value-valid-p rules value)))

(defun rules-to-ticket-positions (rules tickets &aux result)
  (loop for pos below (length rules) do
        (loop for (name (from1 . to1) (from2 . to2)) in rules
              for valid = (loop for ticket in tickets
                                for value = (aref ticket pos)
                                always (or (<= from1 value to1)
                                           (<= from2 value to2)))
              for existing = (assoc name result :test #'string=)
              when valid do
              (if existing
                (push pos (second existing))
                (push (list name (list pos)) result)))
        finally (return result)))

(defun find-mapping (rules tickets)
  (labels ((sort-rules (rules)
             (sort rules #'< :key (lambda (x) (length (second x)))))
           (rules-valid-p (rules)
             (loop for (name positions) in rules
                   always (> (length positions) 0)))
           (update-rules (rules n p)
             (loop for (name positions) in rules
                   when (string/= name n)
                   collect (list name (remove p positions))))
           (recur (rules mapping &aux (rules (sort-rules rules)))
             (cond ((not (rules-valid-p rules)) nil)
                   ((null rules) (return-from find-mapping mapping))
                     (loop for (n pp) in rules do
                           (loop for p in pp do
                                 (recur (update-rules rules n p)
                                        (cons (cons n p) mapping))))))))
    (recur (rules-to-ticket-positions rules tickets) nil)))
That's it!
[SBCL] AOC/2020/16> (time (solution-run))
Evaluation took:
  0.005 seconds of real time
  0.004759 seconds of total run time (0.003597 user, 0.001162 system)
  100.00% CPU
  13,419,408 processor cycles
  425,056 bytes consed


2020-12-20 (permalink)

Advent of Code: [2020/20](https://adventofcode.com/2020/day/20)


- Parse input tiles into LISTs having the id as first element, followed by the content of the tile (first row of data, second row of data...) -- `(list 1234 "...#....." ".#.#..#.." ...)`
- Define functions to rotate a tile counterclockwise, and to flip it -- these are enough to generate all the possible transformations...you just have to apply them in sequence: rotate, rotate-flip, rotate-flip-rotate, rotate-flip-rotate-flip...
- Part 1
- Backtracking solution
- place a tile in the top-right corner; what remaining tiles (and their transformations) can be placed to its right?
- try one; what remaining tiles (and their transformations) can be placed to its right?
- carry on until you can, back track when reaching a dead end (i.e. no more compatible tiles)
- Caveat: when placing a new tile you have to check not just the tile on its left, but also the one at its top
- Optimization: before running the algorithm I indexed all the input tiles (and their transformations) by their top border, left border, and top and right border combined, hoping that that would speed things up -- and I guess it did
- Part 2
- Re-assemble the image -- I got this part wrong initially, but only realized it at the end...
- Brute force
- For all the possible transformations of the sea monster pattern -- I accidentally added an extra character in one of its lines, and because of that my pattern matching algorithm was not matching anything at all...
- Generate all the image tiles with the same size as the sea monster _pattern_ we are trying to match...
- Try to _match_ the pattern against the tiles -- yes, I did indeed use regexps for this...I replaced all the whitespaces in the pattern with `.`s
- Lastly, count all the `#`s in the reassembled image, and subtract `15` times the number of matching sea monsters -- correct, I assumed that there would be no overlapping matches, and I guess I was right
(defun parse-tile-id (string)
  (cl-ppcre:register-groups-bind ((#'parse-integer id))
      ("Tile (\\d+):" string)

(defun parse-tile (data)
  (cons (parse-tile-id (first data))
        (rest data)))

(defun id (tile) (car tile))
(defun data (tile) (cdr tile))
(defun data-top (tile) (first (data tile)))
(defun data-bottom (tile) (first (last (data tile))))
(defun data-left (tile) (data-bottom (rotate tile)))
(defun data-right (tile) (data-top (rotate tile)))

(defun parse-tiles (data)
  (let (groups current)
    (dolist (string (append data '("")))
      (if (string= string "")
        (setf groups (cons (reverse current) groups) current nil)
        (setf current (cons string current))))
    (mapcar #'parse-tile groups)))

(defun rotate (tile)
    (id tile)
      (apply #'map 'list (lambda (&rest args) (format nil "~{~a~}" args))
             (data tile)))))

(defun flip (tile &aux flipped)
  (dolist (x (data tile) (cons (id tile) flipped))
    (push x flipped)))

(defun transformations (tile)
  (loop repeat 4
        for x = tile then (rotate x)
        collect x collect (flip x)))

(defun index-key (&key top left) (format nil "~a~a" top left))

(defun create-tiles-index (tiles &aux (index (make-hash-table :test 'equal)))
  (flet ((index-set (k v)
           (multiple-value-bind (existing existsp)
               (gethash k index)
             (if existsp
               (setf (gethash k index) (cons v existing))
               (setf (gethash k index) (list v))))))
    (dolist (tile tiles index)
      (dolist (ttile (transformations tile))
        (let ((top (data-top ttile))
              (left (data-left ttile)))
          (index-set (index-key) ttile)
          (index-set (index-key :top top) ttile)
          (index-set (index-key :left left) ttile)
          (index-set (index-key :top top :left left) ttile))))))

(defun reorder-tiles (tiles &aux (size (isqrt (length tiles))))
  (let ((index (create-tiles-index tiles))
        (image (make-array `(,size ,size) :initial-element nil)))
    (labels ((recur (used row col)
                 ((not (array-in-bounds-p image row col))
                  (return-from reorder-tiles image))
                   (let ((top (and (> row 0) (data-bottom (aref image (1- row) col))))
                         (left (and (> col 0) (data-right (aref image row (1- col))))))
                     (loop with key = (index-key :top top :left left)
                           for tile in (gethash key index)
                           unless (member (id tile) used) do
                           (setf (aref image row col) tile)
                           (if (array-in-bounds-p image row (1+ col))
                             (recur (cons (id tile) used) row (1+ col))
                             (recur (cons (id tile) used) (1+ row) 0))
                           (setf (aref image row col) nil)))))))
      (recur nil 0 0))))

(setf *sea-monster* (list

(defun create-image (tiles)
  (let ((image (make-array '(96 96))))
    (loop for tile-row below 12 do
          (loop for tile-col below 12
                for tile = (aref tiles tile-row tile-col) do
                (loop for row from 0 for string in (subseq (data tile) 1 9)
                      for image-row = (+ (* tile-row 8) row) do
                      (loop for col from 0 for ch across (subseq string 1 9)
                            for image-col = (+ (* tile-col 8) col) do
                            (setf (aref image image-row image-col) ch)))))

(defun crop (image top left height width)
  (loop for row from top below (+ top height) collect
        (loop for col from left below (+ left width)
              collect (aref image row col) into characters
              finally (return (coerce characters 'string)))))

(defun tiles-match-p (tile1 tile2)
  (every (lambda (x1 x2) (cl-ppcre:scan x1 x2)) tile1 tile2))

(defun count-matches (tile image)
  (let ((height (length (data tile)))
        (width (length (first (data tile)))))
    (loop with (image-height image-width) = (array-dimensions image)
          for row from 0 below (- image-height height) sum
          (loop for col from 0 below (- image-width width)
                count (tiles-match-p (data tile)
                                     (crop image row col height width))))))

(define-solution (2020 20) (tiles parse-tiles)
  (let* ((tiles (reorder-tiles tiles))
         (image (create-image tiles)))
      (* (id (aref tiles 0 0))
         (id (aref tiles 0 11))
         (id (aref tiles 11 0))
         (id (aref tiles 11 11)))
      (- (loop for index below (* 96 96)
               count (char= (row-major-aref image index) #\#))
         (* 15
            (loop for tile in (transformations (cons 666 *sea-monster*))
                  for matches = (count-matches tile image)
                  when (> matches 0) return matches))))))
Not the _cleanest_ implementation (lots of hard-coded stuff!), but at that point I was just rushing for the second star -- and believe it or not, today I scored my best placement ever:
      --------Part 1--------   --------Part 2--------
Day       Time   Rank  Score       Time   Rank  Score
 20   03:52:37   3838      0   05:16:07   1553      0

2020-12-19 (permalink)

Advent of Code: [2020/19](https://adventofcode.com/2020/day/19)


- Use input rules to generate a _massive_ regexp -- it took me a couple of hours to realize I could indeed solve this using regexps!
- Match the regexp against the input messags, and count the ones which test positive
- The regexp is created _recursively_
- The regexp for a given rule can be calculated by calculating the regexps of its sub-rules first, and _combining_ them (i.e. `a|b` where `a` is the regexp for the first sub-rule, and `b` is the regexp for the second one)
- The regexp for a given sub-rule can be calculated by _concatenating_ the regexpes of its rules (i.e. `ab` where `a` is the regexp for the first sub-rule, and `b` is the regexp for the second one)
- To make the matching operation more efficient we are compiling the regexp into a scanner first (i.e. `(cl-ppcre:create-scanner regexp)`)
- Part 2 makes two of the input rules recursive, and to avoid my recursive solution to spin forever, I added a `depth` parameter to my MAKE-REGEXP function and decremented it on every recursive call -- and a recursion limit of `20` seems to be enough to generate the right regexp for part 2
(defun parse-sub-rule (string)
  (if (find #\" string)
    (list (string (char string 1)))
    (mapcar #'parse-integer (cl-ppcre:split " " string))))

(defun parse-sub-rules (string)
  (let ((sub-rules (cl-ppcre:split " \\| " string)))
    (mapcar #'parse-sub-rule sub-rules)))

(defun parse-rule (string)
  (cl-ppcre:register-groups-bind ((#'parse-integer id) rest)
      ("(\\d+): (.*)" string)
    (list id (parse-sub-rules rest))))

(defun id (rule) (car rule))
(defun sub-rules (rule) (cadr rule))

(defun parse-input (data)
  (let (groups current)
    (dolist (string (append data '("")))
      (if (string= string "")
        (setf groups (cons (reverse current) groups) current nil)
        (setf current (cons string current))))
      (mapcar #'parse-rule (second groups))
      (first groups))))

(defun rules (input) (car input))
(defun messages (input) (cadr input))

(defparameter *max-recursion* 20)

(defun make-regexp (rules)
  (let ((sub-rules-map (make-hash-table)))
    (dolist (rule rules)
      (setf (gethash (id rule) sub-rules-map) (sub-rules rule)))
    (labels ((recur (id depth &aux (sub-rules (gethash id sub-rules-map)))
                 ((= depth *max-recursion*) "") ; too much recursion, give up!
                 ((not (numberp id)) id)        ; termination rule, return it.
                   (format nil "(~{~A~^|~})"
                           (loop for sub-rule in sub-rules collect
                                 (format nil "~{~A~}"
                                         (loop for id in sub-rule
                                               collect (recur id (1+ depth))))))))))
      (format nil "^~A$" (recur 0 0)))))

(defun count-matches (data &aux (input (parse-input data)))
  (let* ((regexp (make-regexp (rules input)))
         (scanner (cl-ppcre:create-scanner regexp)))
    (count-if (lambda (s) (cl-ppcre:all-matches scanner s)) (messages input))))

(defun prepare-part2 (data)
  (loop for string in data
        if (string= string "8: 42") collect "8: 42 | 42 8"
        else if (string= string "11: 42 31") collect "11: 42 31 | 42 11 31"
        else collect string))

(define-solution (2020 19) (data)
  (values (count-matches data)
          (count-matches (prepare-part2 data))))
PS. My [original solution](https://topaz.github.io/paste/#XQAAAQBcEAAAAAAAAAAUGQimgulVPYrJhMVjyYOkoNMRrSkEDb+VhGpjZScfl0vQffPIveu6zou42QlYi3oXP009ZBKb+73pdKhOT5C+sOCPTrLLxmKoZfx/upwjyRQJVK5QxpWmzVUWY5qy/01i9v9CB49xj+0sI8P96rG17oQR/UE1GaYDQq7CW3x8lQViBxKC0ikxVHqmfbRp7RdQHHBFH3UbsI6Su5ohIHBG4UA9H14+wCjb/qNcyqOQLNlEwX0V8k6JaO24dy2bp/46mg2DlY8nz/WVAcMUGXRZWL6bkK49UKEsediwu46KxfywY5aP/X56cNVSNUEI/CM1TZxuAzw8hZXa237l7P+KP2+tlsHisWpC4LkeKzSIFVSPoSZqALrAHjysHtbirktMiHX3nH2w56zcA0mtHTLWTeU9PyRxKp+UndXzjmlklCRJT5ytgdFJAm/nDgLwe5JdBn1wwMekZEf9yuOGA+I3S7V2e1Pmwhm9LuWjBFAcCabpRLrIgcM16lv1bix8dRVC9l/pnWcME0jJCL6WWAZYoSSDb9TeeduuLH9a14RrHR5x5fK7LIhLVUy7w87lts1yzFkvgwV96z0CndZEluIaqLqiSqtZwcLoqPN3NsAAeSTIFN7hZ2HKAUgUWvtkGV4DXJmTD9MsBGbdM9oPkO7hOE4kD+vL3HFLEIOr5wQPUzLPc3fLZkPJr66RnQN7CuJWddqQaEDf0vdCEMPLA8fKNlUmxl6nJemRYVGFO2YaAWHAMOPqZQ+CKe104YUyB0rTd8MVmBcPsN/jX3P3LfiwSUcnWOOCSV1X5RTMFMnFs+nrD91DTn+inrq3DEuGL3S8nm73l/kN68SA3H4YrhNSz07MUJLrmRzlmvZXoUplaQKiN9DyPMJMXsLpOsSB+iBXOo1/hBmga+cGExX7k4P69Dbg+3J+JINzvwaLRclkrpXkWomgthb5G7/kBd44T+LuquziC7e0sBfFMhJnUyGjmBlsf/G/LCNS+ZjTWf1XIj8XiqhLJmmE/T56I34xPQgnAQlCm3N1+uid2+RQ9Ml/m0Vc0JqFkBM3Cr+7iVEuQIi+/LWDiKzDX1F+XEkp+LpRDzUOjWcFBc2Cmhbg2IkKqd3eytEiQ0QPJZHYtFe5kACz2v1seeU84fGpW+BM11QBn2pLy0tEGoSmnRzuE8Nnk209tXGs2DUhkP13iP07nV1Swdkk9cAR54SbbV0pC+ddPEvznwnigCpV9GRX4Z2N0Sxk7GC6tFRmVYZ4VRtAMEqyL1w9Qs1Y8l7QdJ31F/8jrpo4t6Ad+djOlGRa/ykunLWlu5aoM3wKMRrmbZP/qnzRxaZyDyeOgxlwaU2wActcyKlXvYV7Hjs9I//wCx9Wk8FZvEfcc4lvti3NlF3m3L53CdJCLBpOLc1hZM8Nwn0HI5mFLx5qK6IiXb1LvuK96u7/roJdoqT8HVT2YCCY0vejpFmX8IKj7POD+hYQmj+csMe5Lpu4sjfWwNW/9gB6CGHxWT3lC7D1jUUHgbH3c/FvalIzhj3FdTjAWF3//DZ69g==) implemented the final regexp using a bottom-up approach: start from terminating rules, generate regexps for their parents, then generate regexps for their parents... For part 2 intead, I modified the regexp creator so that, for rule 8 it would output: `(regexp42)+`, while for rule 11 it would output: `((regexp42){1}(regxp31){1}|(regexp42){2}(regxp31){2}|...|(regexp42){10}(regxp31){10})`.
PPS. Clearly I like the recursive solution better!
PPPS. It would be interesting to solve this implementing a [Context-free grammar](https://en.wikipedia.org/wiki/Context-free_grammar) parser.

2020-12-18 (permalink)

Advent of Code: [2020/18](https://adventofcode.com/2020/day/18)


- Parse each expression into an [S-expression](https://en.wikipedia.org/wiki/S-expression)
- Feed each to EVAL
- Sum the results

I implemented the parsing logic by combining together three different parsers:

- PARSE-OPERATOR, responsible for converting the next available character into a symbol (i.e. `+` or `*`)
- PARSE-OPERAND, responsible for parsing a single digit number, or an _expression_
- PARSE-EXPRESSION, responsible for ... the rest

Clearly the bulk of the work is getting done inside PARSE-EXPRESSION:

- First we parse an operand
- Then an operator
- If the operator needs to be evaluated immediately, then parse another operand and create an S-expression (i.e. `'(rator rand1 rand2)`)
- Otherwise, the rest of the expression needs to be evaluated first, so call PARSE-EXPRESSION recursively, and finally create the S-expression (i.e. `'(rator rand1 expr1)`)
- Repeat
(defparameter *operators-with-precedence* nil)

(defun parse-value (ch) (parse-integer (string ch)))

(defun parse-operand (string &aux (ch (char string 0)))
  (cond ((digit-char-p ch) (list (parse-value ch) (subseq string 1)))
        ((char= ch #.(char "(" 0))
         (destructuring-bind (e rest) (parse-expression (subseq string 1))
           (list e (subseq rest 1)))) ; skip closing parenthesis
        (t (error "Unexpected text: ~s" string))))

(defun parse-operator (string &aux (ch (char string 0)))
  (list (find-symbol (string ch)) (subseq string 1)))

(defun parse-expression (string)
  (loop with (expr rest) = (parse-operand string)
        until (or (zerop (length rest)) (char= (char rest 0) #.(char ")" 0)))
        for (rator rator-rest) = (parse-operator rest) do
        (if (find rator *operators-with-precedence*)
          (destructuring-bind (rand rand-rest)
              (parse-operand rator-rest)
            (setf expr (list rator expr rand) rest rand-rest))
          (destructuring-bind (expr1 expr1-rest)
              (parse-expression rator-rest)
            (setf expr (list rator expr expr1) rest expr1-rest)))
        finally (return (list expr rest))))

(defun parse (string &aux (string (remove #\Space string)))
  (first (parse-expression (remove #\Space string))))

(define-solution (2020 18) (data)
    (let ((*operators-with-precedence* '(+ *)))
      (reduce #'+ (mapcar #'parse data) :key 'eval))
    (let ((*operators-with-precedence* '(+)))
      (reduce #'+ (mapcar #'parse data) :key #'eval))))

(define-test (2020 18) (50956598240016 535809575344339))
PS. apparently this is the exact same problem the [Shunting-yard algorithm](https://en.wikipedia.org/wiki/Shunting-yard_algorithm) was designed to solve.

I am finally finding myself with some time to clean up some of my recent Advent of Code solutions, so let's have a look at [2020/09](https://matteolandi.net/plan.html#day-2020-12-09) again.

For part 2, given a list of numbers, we were asked to find a sub-sequence such that the sum of its elements would be equal to the solution for part 1.  This is what I came up with back then.

In words:

- Part 2

- BFS + prefix sums -- I encoded my state as `(psum . seq)` (e.g. `(15 2 3 10)`)

- For each number in the input, try to add it to each of the current states

- If the sum is greater than the target value, discard it

- if the sum is smaller than the target value, add it to the next set of states to evaluate

- If the sum is equal to the target value, there you have your subsequence

In code:
(defun subseq-that-adds-up-to (numbers target &optional frontier)
  (loop with n = (first numbers)
        for (psum . seq) in (cons (cons 0 nil) frontier)
        for psum-next = (+ n psum) for seq-next = (cons n seq)
        if (= psum-next target) return seq-next
        else collect (cons psum-next seq-next) into frontier-next
        finally (return (subseq-that-adds-up-to (rest numbers)
Well, it turns out a way simpler solution exists, and it goes like this; given a subsequence (defined in terms of its `start` index, `end` index, and the sum of its elements):

- When `sum` is less than `target`, move the `end` index ahead (and update `sum`)
- When `sum` is greater than `target`, move the `start` index ahead (and update `sum`)
- When `sum` is equal to `target`, then we found our subsequence!

Take care of initialization details, and you should end up with something like this:
(defun subseq-that-adds-up-to (numbers target
                                       &aux (numbers (coerce numbers 'vector)))
    with start = 0 with end = 0 with sum = (aref numbers start)
    if (= sum target) return (coerce (subseq numbers start (1+ end)) 'list)
    else if (> sum target) do (setf sum (- sum (aref numbers start))
                                    start (1+ start))
    else do (setf end (1+ end) sum (+ sum (aref numbers end)))))

2020-12-17 (permalink)

Advent of Code: [2020/17](https://adventofcode.com/2020/day/17)


- Classic [Conway's Game of Life](https://en.wikipedia.org/wiki/Conway%27s_Game_of_Life) (even the rules were the same); the only difference is that the game evolves in 3 dimensions for part 1 -- cells are cubes! -- and in 4 dimensions for part 2!
- I modeled the game is a set of alive cells, and at each iteration I...
- Get the full list of cells, adjacents to the active ones, without discarding duplicates...
- For each unique cell, count the number of its occurrences (the idea is, if `(x y z)` is mentioned twice, it means two of its adjacents cells are active)
- Use this information (i.e cell position, the number of its neighbors) and the _previous_ cell state, to mark the cell as dead or alive
- PS: Kudos to [Christophe Grand](http://clj-me.cgrand.net/2011/08/19/conways-game-of-life/) for coming up with this nice and concise way of implementing Conway's Game of Life (in Clojure)
- PPS: I was afraid of using LISTs as SETs, so I went on and created a new type, HSET
(defun parse-coords (data)
  (let* ((width (length (first data)))
         (height (length data)))
    (loop for y upto height for row in data nconc
          (loop for x upto width for c across row
                when (eql c #\#) collect (list x y 0)))))

(defun make-hset (values &key (test 'eql))
  (let ((h (make-hash-table :test test)))
    (dolist (v values)
      (setf (gethash v h) t))

(defun hset-contains-p (v hset)
  (multiple-value-bind (unused exists-p)
      (gethash v hset)
    (declare (ignore unused))

(defun hset-values (hset)
  (loop for v being the hash-keys of hset collect v))

(defun hset-size (hset) (hash-table-count hset))

(defparameter *neighbors-deltas* nil)

(defun neighbors-deltas (dimensions)
  (labels ((recur (n)
             (cond ((= n 0) (list nil))
                   (t (loop for d from -1 upto 1 nconc
                            (loop for rest in (recur (1- n))
                                  collect (cons d rest)))))))
    (remove-if (lambda (x) (apply #'= 0 x)) (recur dimensions))))

(defun neighbors (pos)
  (loop for delta in *neighbors-deltas* collect (mapcar #'+ pos delta)))

(defun frequencies (list &key (test 'eql))
  (let ((f (make-hash-table :test test)))
    (dolist (each list)
      (incf (gethash each f 0)))
    (loop for k being the hash-keys of f collect (cons k (gethash k f)))))

(defun next (state)
      for (pos . n) in (frequencies (mapcan #'neighbors (hset-values state)) :test 'equal)
      when (or (= n 3) (and (= n 2) (hset-contains-p pos state)))
      collect pos)
    :test 'equal))

(defun play (coords &aux (dimensions (length (first coords))))
  (let ((*neighbors-deltas* (neighbors-deltas dimensions))
        (game (make-hset coords :test 'equal)))
    (dotimes (n 6 (hset-size game))
      (setf game (next game)))))

(define-solution (2020 17) (coords parse-coords)
    (play coords)
    (play (mapcar #'(lambda (x) (append x (list 0))) coords))))

2020-12-16 (permalink)

Advent of Code: [2020/16](https://adventofcode.com/2020/day/16)


- Parse rules into LISTs having the name of the rule as first argument, and two _ranges_ as second and third element (a range is just an improper list containing two numbers: the min and max values of the range)
- Parse tickets into ARRAY of numbers
- Part 1
- Create an ARRAY of 1000 elements (all my tickets numbers were smaller than 1000)
- Iterate all the rule ranges and mark each possible value of those ranges as valid
- Iterate all the tickets numbers, and sum the ones which are not valid (i.e the numbers which are not in the array above)
- Part 2
- For each _position_ inside a ticket, pre-calculate the list of _compatible_ fields (e.g. the first position is compatible with the "class" rule, the second is compatible with "row" and "seat" rules)
- Find the mapping using backtracking...
- What fields are compatible with the first first position in the ticket? Pick one...
- What fields are compatible with the second position in the ticket, knowing what we fixed the mapping for the first position? Pick one...
- If you reach a dead end, back-track...
- If you manage to place them all, you found your mapping
(defun parse-ticket-rule (string)
  (cl-ppcre:register-groups-bind (name (#'parse-integer from1 to1 from2 to2))
      ("(.+): (\\d+)-(\\d+) or (\\d+)-(\\d+)" string)
    (list name (cons from1 to1) (cons from2 to2))))

(defun parse-ticket-rules (data) (mapcar #'parse-ticket-rule data))

(defun parse-ticket (string)
  (map 'vector #'parse-integer (split-sequence:split-sequence #\, string)))

(defun parse-your-ticket (data) (parse-ticket (second data))) ; skip header

(defun parse-nearby-tickets (data)
  (mapcar #'parse-ticket (rest data))) ; skip header

(defun parse-document (data)
  (let (groups current)
    (dolist (string (append data '("")))
      (if (string= string "")
        (setf groups (cons (reverse current) groups) current nil)
        (setf current (cons string current))))
      (parse-ticket-rules (third groups))
      (parse-your-ticket (second groups))
      (parse-nearby-tickets (first groups)))))

(defun rules (doc) (first doc))
(defun your-ticket (doc) (second doc))
(defun nearby-tickets (doc) (third doc))

(defun ticket-valid-p (ticket valid-numbers)
  (every (lambda (n) (aref valid-numbers n)) ticket))

(defun ticket-errors (ticket valid-numbers)
  (loop for n across ticket unless (aref valid-numbers n) sum n))

(defun validate-tickets (doc)
  (let ((valid-numbers (make-array 1000 :initial-element nil)))
    (loop for (name . ranges) in (rules doc) do
          (loop for (min . max) in ranges do
                (loop for i from min upto max
                      do (setf (aref valid-numbers i) name))))
    (loop for ticket in (nearby-tickets doc)
          sum (ticket-errors ticket valid-numbers) into total-error-rate
          when (ticket-valid-p ticket valid-numbers) collect ticket into valid-tickets
          finally (return (values valid-tickets

(defun calculate-compatible-fields (rules tickets)
  (let ((result (make-array (length rules) :initial-element nil)))
    (dotimes (pos (length rules) result)
      (loop for (name (from1 . to1) (from2 . to2)) in rules
            for valid = (loop for ticket in tickets for n = (aref ticket pos)
                              always (or (<= from1 n to1) (<= from2 n to2)))
            when valid do (push name (aref result pos))))))

(defun find-field-mapping (compatibles &aux (n (length compatibles)))
  (labels ((recur (mapping &aux (pos (length mapping)))
               ((= pos n) (return-from find-field-mapping (reverse mapping)))
               (t (loop for name in (set-difference
                                      (aref compatibles pos)
                                      :test 'string=) do
                        (recur (cons name mapping)))))))
    (recur nil)))

(defun part2 (doc tickets)
  (let* ((compatibles (calculate-compatible-fields (rules doc) tickets))
         (mapping (find-field-mapping compatibles)))
    (loop for name in mapping for part across (your-ticket doc)
          when (search "departure" name) collect part into numbers
          finally (return (reduce #'* numbers)))))

(define-solution (2020 16) (doc parse-document)
  (multiple-value-bind (valid-tickets error-rate)
      (validate-tickets doc)
    (values error-rate (part2 doc valid-tickets))))
All good...except it takes 30 seconds to generate the answer for part 2:
[SBCL] AOC/2020/16> (time (solution-run))
Evaluation took:
  29.244 seconds of real time
  28.687495 seconds of total run time (28.453248 user, 0.234247 system)
  [ Run times consist of 0.149 seconds GC time, and 28.539 seconds non-GC time. ]
  98.10% CPU
  67,259,903,251 processor cycles
  5 page faults
  863,202,720 bytes consed

Surely if CL implemented SETs with trees instead of flat LISTs, things would run a bit faster; however, the problem here is that we are trying to create the mapping, in order, from first position to last, while instead we should pick the next position trying to reduce the branch factor (i.e. if there is a position compatible with a single field only, we should try that immediately).

2020-12-15 (permalink)

Advent of Code: [2020/15](https://adventofcode.com/2020/day/15)


- one to keep track of the last time a given number was spoken
- one to keep track of the second-to-last time a given number was spoken
- for each spoken number:
- if we did not see this before (i.e. the number does not exist in `seen-at-2`), then the next spoken number is `0`
- otherwise, calculate the number _age_ (i.e. `turn-1 - turn-2`), and let that be new new spoken number
- lastly, _rotate_ from one HASH-TABLE to the other: `seen-at-1` into `seen-at-2`, and `turn` (i.e. the current turn) into `seen-at-2`
(defun parse-numbers (data)
  (mapcar #'parse-integer (split-sequence:split-sequence #\, (first data))))

(defun play (numbers turn-max &aux
                     (seen-at-1 (make-hash-table))
                     (seen-at-2 (make-hash-table)))
  (loop for n in numbers for turn from 1 do (setf (gethash n seen-at-1) turn))
  (loop with last = (first (reverse numbers))
        for turn from (1+ (length numbers)) to turn-max
        for turn-1 = (gethash last seen-at-1)
        for turn-2 = (gethash last seen-at-2)
        if (not turn-2) do (setf last 0) else do (setf last (- turn-1 turn-2))
        do (setf (gethash last seen-at-2) (gethash last seen-at-1)
                 (gethash last seen-at-1) turn)
        finally (return last)))

(define-solution (2020 15) (numbers parse-numbers)
  (values (play numbers 2020) (play numbers 30000000)))
That's it -- not the most performant solution, but not that astronomically slow either!
[SBCL] AOC/2020/15> (time (solution-run))
Evaluation took:
  7.049 seconds of real time
  7.023387 seconds of total run time (6.591918 user, 0.431469 system)
  [ Run times consist of 0.466 seconds GC time, and 6.558 seconds non-GC time. ]
  99.63% CPU
  16,212,370,931 processor cycles
  1 page fault
  914,358,672 bytes consed


2020-12-14 (permalink)

Advent of Code: [2020/14](https://adventofcode.com/2020/day/14)


- Parse each "mask = ..." input line into a pair of bit-masks: one for all the "0"s, one for all the "1"s
- "mem[...] = ..." lines instead are parsed into pairs containing the location of the memory to update, and the value to write
- Part 1: mask the value before writing it into memory
- To write 0s, we LOGNOT the _zeros_ mask first, and then LOGAND the value to write with this
- To write 1s instead, we simply LOGIOR the result of the previous step with the _ones_ mask
- Part 2: we generate the list of _masked_ addresses, and then update those memory locations. How? A little bit of dynamic programming
- `dp[i]` represents all the _masked addresses_ considering only the first `i` bits of our mask (the least significant ones)
- Base case: `dp[0] = [0]`; knowing `dp[i]`, `dp[i + 1]` can be calculated as follows
- For each element of `dp[i]` -- let's call it `number`
- If `i`-th bit in the mask is 0, then copy the `i`-th bit of the address into `number`, and append the result to `dp[i + 1]`
- If `i`-th bit in the mask is 1, then set the `i`-th bit of `number` to 1, and append the result to `dp[i + 1]`
- Otherwise, append two new values to `dp[i + 1]`: the first, `number` with its `i`-th bit set to 0, the second with its `i`-th bit set to 1
- Note: each step uses the result of the previous step, so we don't need a LIST/ARRAY to store all the intermediary states
(defun parse-mask (string)
  (cl-ppcre:register-groups-bind (mask)
      ("mask = ([10X]+)" string)
    (let ((zeros 0) (ones 0))
      (loop for c across (reverse mask) for index from 0
            for byte = (byte 1 index)
            when (char= c #\0) do (setf zeros (dpb 1 byte zeros))
            when (char= c #\1) do (setf ones (dpb 1 byte ones)))
      (list :mask (cons zeros ones)))))

(defun zeros (mask) (car mask))
(defun ones (mask) (cdr mask))

(defun parse-mem (string)
  (cl-ppcre:register-groups-bind ((#'parse-integer address value))
      ("mem\\[(\\d+)\\] = (\\d+)" string)
    (list :mem (cons address value))))

(defun address (mem) (car mem))
(defun value (mem) (cdr mem))

(defun parse-line (string)
  (or (parse-mask string) (parse-mem string)))

(defun parse-instructions (data)
  (mapcar #'parse-line data))

(defun value-mask (value mask)
  (logior (logand value (lognot (zeros mask)))
          (ones mask)))

(defun part1 (instructions)
  (loop with mask with mem = (make-hash-table)
        for (type instr) in instructions
        if (eql type :mask) do (setf mask instr)
        else do (setf (gethash (address instr) mem)
                      (value-mask (value instr) mask))
        finally (return (reduce #'+ (hash-table-values mem)))))

(defun masked-addresses (addr mask &aux (dp (list 0)))
  (dotimes (index 36 dp)
    (loop with prev = dp initially (setf dp nil) for number in prev
          for byte = (byte 1 index) do
          (cond ((logbitp index (zeros mask))
                 (push (dpb (ldb byte addr) byte number) dp))
                ((logbitp index (ones mask))
                 (push (dpb 1 byte number) dp))
                (t (push (dpb 0 byte number) dp)
                   (push (dpb 1 byte number) dp))))))

(defun part2 (instructions)
  (loop with mask with mem = (make-hash-table)
        for (type instr) in instructions
        if (eql type :mask) do (setf mask instr)
        else do (loop for addr in (masked-addresses (address instr) mask)
                      do (setf (gethash addr mem) (value instr)))
        finally (return (reduce #'+ (hash-table-values mem)))))

(define-solution (2020 14) (instructions parse-instructions)
  (values (part1 instructions) (part2 instructions)))

2020-12-13 (permalink)

Advent of Code: [2020/13](https://adventofcode.com/2020/day/13)

I knew the `x`s in the input were going to be used for part 2, so I decided to _manually_ parse the input and try to solve this directly from the REPL.

If we take the input timestamp and run the modulus of the timestamp and one of the bus periods, we obtain the time, prior to timestamp, the specific bas was last seen at the bus station; to know when the bus will be back again, we need to subtract this number from the bus period.  Do this for all the buses, find the min, and there you have your solution for part 1:
(- 19 (mod 1001798 19))

(- 41 (mod 1001798 41))

(- 859 (mod 1001798 859))

(- 23 (mod 1001798 23))

(- 13 (mod 1001798 13))

(- 17 (mod 1001798 17))

(- 29 (mod 1001798 29))

(- 373 (mod 1001798 373))

(- 37 (mod 1001798 37))

(* 7 29)
For part 2, as expected, things got really complicated... I immediately tried a _naive_ brute-force solution, but that did not take me anywhere (well, it might have, eventually):
(loop for ts from 100000000000000
      when (and (zerop (mod (+ ts 0) 19))) return ts)

(loop for ts from 100000000000003 by 19
      when (zerop (mod (floor ts 19) 1000000)) do (pr ts)
      when (and (zerop (mod (+ ts 0) 19))
                (zerop (mod (+ ts 9) 41))
                (zerop (mod (+ ts 19) 859))
                (zerop (mod (+ ts 27) 23))
                (zerop (mod (+ ts 32) 13))
                (zerop (mod (+ ts 36) 17))
                (zerop (mod (+ ts 48) 29))
                (zerop (mod (+ ts 50) 373))
                (zerop (mod (+ ts 87) 37))) return ts)
; Zzz
Tried to probe solutions using the _slowest_ bus instead, but that solution too was not converging fast enough:
(loop for ts from 100000000000000
      when (and (zerop (mod (+ ts 19) 859))) return ts)

(loop for ts from 100000000000850 by 859
      when (zerop (mod (floor ts 859) 1000000)) do (pr ts)
      when (and (zerop (mod (+ ts 0) 19))
                (zerop (mod (+ ts 9) 41))
                (zerop (mod (+ ts 19) 859))
                (zerop (mod (+ ts 27) 23))
                (zerop (mod (+ ts 32) 13))
                (zerop (mod (+ ts 36) 17))
                (zerop (mod (+ ts 48) 29))
                (zerop (mod (+ ts 50) 373))
                (zerop (mod (+ ts 87) 37))) return ts)
; Zzz
Then I tried with pen and paper, and after a bit realized that all I was trying to do, was solving the following system of equations:
x + 0  = 0 (mod 19)
x + 9  = 0 (mod 41)
x + 19 = 0 (mod 859)
x + 27 = 0 (mod 23)
x + 32 = 0 (mod 13)
x + 36 = 0 (mod 17)
x + 48 = 0 (mod 29)
x + 50 = 0 (mod 373)
x + 87 = 0 (mod 37)
Searching on DuckDuckGo for "system of modular equations" brought me to this YouTube [video](https://www.youtube.com/watch?v=-hUldZiDUj8) with a very interesting title; I stepped forward a little bit as I had no idea what the guy was talking about, and eventually at [1:38](https://youtu.be/-hUldZiDUj8?t=98) I saw what kind of equation system this theorem is helping you solve, and then I knew that "Chinese Remainder Theorem" was going to help me solve part 2.

I immediately looked for an online solver but the first one I found did not support more than 8 equations, and guess what, our problem had 9!  Luckily, the [next one](https://comnuan.com/cmnn02/cmnn0200a/cmnn0200a.php) did not have such limitation, however, I had to _massage_ my system a little bit before I were able to use the online solver.

First I had to move "time offsets" to the right side of the equation:
x =   0 (mod 19)
x =  -9 (mod 41)
x = -19 (mod 859)
x = -27 (mod 23)
x = -32 (mod 13)
x = -36 (mod 17)
x = -48 (mod 29)
x = -50 (mod 373)
x = -87 (mod 37)
Next I had to make all _negative_ values, _positive_, by adding the bus frequencies until the result were positives:
x =   0 (mod 19)
x =  32 (mod 41)
x = 840 (mod 859)
x =  19 (mod 23)
x =   7 (mod 13)
x =  15 (mod 17)
x =  10 (mod 29)
x = 323 (mod 373)
x =  24 (mod 37)
Lastly, convert the above into the solver's _expected_ input:
1 0 19
1 32 41
1 840 859
1 19 23
1 7 13
1 15 17
1 10 29
1 323 373
1 24 37
Click "Calculate", and you should be presented with the following message:

solution = 905694340256752 + 1361317053288127*n, where n is an integer

Well, the first number right there, `905694340256752`, would be the solution to your part 2 problem!

PS. I want to come back to this, and _implement_ a proper solution, but not today!

2020-12-12 (permalink)

Advent of Code: [2020/12](https://adventofcode.com/2020/day/12)


- Use complex numbers to represent the position and the bearing of the ship
- Turning left by 90 degrees means multiplying your bearing by _i_ (i.e. `#c(0 1)`)
- Turning right by 90 degrees means multiplying your bearing by _-i_ (i.e. `#c(0 -1)`)
- Part 1: movement instructions (e.g. `N`, `S`, `W`, `E`) apply to the ship itself (that was a bit confusing at first, that the the ship could [strafe](https://en.wikipedia.org/wiki/Strafing_(video_games)), but whatever)
Part 2: movement instructions apply to the bearing of the ship
(defun parse-instruction (string)
  (cons (char string 0) (parse-integer string :start 1)))

(defun parse-instructions (data)
  (mapcar #'parse-instruction data))

(defun part1 (instructions)
  (loop with pos = 0 with dir = #c(1 0) for (ch . n) in instructions do
        (ecase ch
          (#\N (incf pos (* n #c(0 1))))
          (#\S (incf pos (* n #c(0 -1))))
          (#\E (incf pos (* n #c(1 0))))
          (#\W (incf pos (* n #c(-1 0))))
          (#\L (mulf dir (expt #c(0 1) (/ n 90))))
          (#\R (mulf dir (expt #c(0 -1) (/ n 90))))
          (#\F (incf pos (* n dir))))
        finally (return (manhattan-distance pos 0))))

(defun part2 (instructions)
  (loop with pos = 0 with dir = #c(10 1) for (ch . n) in instructions do
        (ecase ch
          (#\N (incf dir (* n #c(0 1))))
          (#\S (incf dir (* n #c(0 -1))))
          (#\E (incf dir (* n #c(1 0))))
          (#\W (incf dir (* n #c(-1 0))))
          (#\L (mulf dir (expt #c(0 1) (/ n 90))))
          (#\R (mulf dir (expt #c(0 -1) (/ n 90))))
          (#\F (incf pos (* n dir))))
        finally (return (manhattan-distance pos 0))))

(define-solution (2020 12) (instructions parse-instructions)
  (values (part1 instructions) (part2 instructions)))

2020-12-11 (permalink)

Advent of Code: [2020/11](https://adventofcode.com/2020/day/11)


- Parse the seating layout into a HASH-TABLE (I thought about using a 1D or 2D ARRAY, but figured the HASH-TABLE would make things a bit easier as I would not have to worry about reading out of bounds positions)
- Seat positions are COMPLEX numbers
- Each step I copy the old state, update it based on the problem rules, return the new state
- If the new and the old state are EQUALP, then we found our end state
- Part 1: for each position, simply count the adjacent ones which are occupied
- Part 2: recursively check in the neighboring direction until we find a `#`, `L`, or get out of bounds
(defun parse-layout (data)
  (let* ((layout (make-hash-table))
         (row 0)
    (dolist (string data layout)
      (dotimes (col (length string))
        (setf pos (complex col row)
              (gethash pos layout) (char string col)))
      (incf row))))

(defparameter *neighbors-deltas* '(#C(-1 -1) #C(0 -1) #C(1 -1) #C(-1 0) #C(1 0) #C(-1 1) #C(0 1) #C(1 1)))

(defun neighbors-occupied (seat layout)
  (count #\# *neighbors-deltas*
         :key (lambda (d) (gethash (+ seat d) layout #\.))))

(defun in-sight-occupied (seat layout)
  (labels ((recur (seat d &aux (state (gethash seat layout)))
             (cond ((not state) #\.)
                   ((char= #\. state) (recur (+ seat d) d))
                   (t state))))
    (count #\# *neighbors-deltas*
           :key (lambda (d) (recur (+ seat d) d)))))

(defun next (layout neighbors threshold &aux (result (make-hash-table)))
  (loop for seat being the hash-keys of layout
        for state = (gethash seat layout)
        for occupied-count = (funcall neighbors seat layout)
        do (setf (gethash seat result) state)
        when (and (char= state #\L) (zerop occupied-count)) do (setf (gethash seat result) #\#)
        when (and (char= state #\#) (>= occupied-count threshold)) do (setf (gethash seat result) #\L)
        finally (return result)))

(defun simulate (layout neighbors threshold)
  (loop for prev = layout then next
        for next = (next prev neighbors threshold)
        until (equalp next prev)
        finally (return (count #\# (hash-table-values prev)))))
This works, but it's not that _efficient_ (see later), so whenever I get a chance I would like to try to a) cache recursive calls inside IN-SIGHT-OCCUPIED, b) see if [changing how the problem is modeled](http://clj-me.cgrand.net/2011/08/19/conways-game-of-life/) might help or not.
[SBCL] AOC/2020/11> (time (solution-run))
Evaluation took:
  2.040 seconds of real time
  1.965362 seconds of total run time (1.891473 user, 0.073889 system)
  [ Run times consist of 0.105 seconds GC time, and 1.861 seconds non-GC time. ]
  96.32% CPU
  4,691,074,014 processor cycles
  724,157,584 bytes consed


2020-12-10 (permalink)

Advent of Code: [2020/10](https://adventofcode.com/2020/day/10)


- Part 1
- Sort connectors
- Calculate deltas between adjacents connectors
- Part 2
- Top-down dynamic programming (i.e. recursion + memoization)
- Start from joltage of 0, and call it `p`
- Look at the next adapter `n`: if `n` and `p` differs by less than 3 jolts, then you can...
- Take it, and count all the possible ways you can combine the remaining adapters, knowing that `p` is now `n`
- Or skip it, and count all the possible ways you can combine the remaining adapters, knowing that `p` has not changed
- Stop when there are no more adapters to play with
(defun part1 (adapters)
  (loop for p = 0 then c for c in adapters for delta = (- c p)
        count (= delta 1) into ones
        count (= delta 3) into threes
        finally (return (* ones (1+ threes)))))

(defun part2 (adapters)
  (let ((memo (make-hash-table :test 'equal)))
    (labels ((recur (p rest &aux (n (first rest)) (key (list p n)))
               (cond ((or (null rest) (> (- n p) 3)) 0) ; no luck
                     ((null (rest rest)) 1) ; found it
                     ((gethash key memo) (gethash key memo))
                     (t (setf (gethash key memo)
                              (+ (recur p (rest rest))
                                 (recur n (rest rest))))))))
      (recur 0 adapters))))

(define-solution (2020 10) (numbers parse-integers)
  (let* ((sorted (sort numbers #'<)))
    (values (part1 sorted) (part2 sorted))))
Alternatively, one could still use dynamic programming but go with a bottom-up approach:

- `dp[n]` represents the solution to our problem, considering only the adapters up until `n`
- Base case, the charging outlet: `dp[0] = 1`
- Transformation function: knowing `dp[0]...dp[n]`, the next value `dp[n + 1]` can be calculated simply by summing together the solutions for the 3 previous adapters, as those are the ones in range (i.e. `dp[n]`, `dp[n - 1]`, and `dp[n - 2]`)
(defun part2 (numbers &aux (dp (make-hash-table)))
  (loop initially (setf (gethash 0 dp) 1)
        for n in numbers do
        (setf (gethash n dp) (+ (gethash (- n 1) dp 0)
                                (gethash (- n 2) dp 0)
                                (gethash (- n 3) dp 0)))
        finally (return (gethash n dp))))
I think I like this last one better!

2020-12-09 (permalink)

Advent of Code: [2020/09](https://adventofcode.com/2020/day/09)


- Part 1
- Implement a sliding-window HASH-SET of 25 elements
- For each number after the 25th, check if it's valid, and if not, remove the oldest element from the set, and add the current one
- Checking if a number `n` is valid is simple: for each element `x` in the sliding-set, if `n - x` exists in the set as well then `n` is valid
- Part 2
- BFS + prefix sums -- I encoded my state as `(psum . seq)` (e.g. `(15 2 3 10)`)
- For each number in the input, try to add it to each of the current states
- If the sum is greater than the target value, discard it
- if the sum is smaller than the target value, add it to the next set of states to evaluate
- If the sum is equal to the target value, there you have your subsequence
(defun make-set (numbers)
  (let ((set (make-hash-table)))
    (dolist (x numbers set)
      (setf (gethash x set) t))))

(defun sadd (validator number) (setf (gethash number validator) t))
(defun srem (validator number) (remhash number validator))

(defun validp (validator number)
  (loop for n being the hash-keys of validator
        thereis (gethash (- number n) validator)))

(defun find-first-invalid (numbers)
  (loop with s = (make-set (subseq numbers 0 25))
        for oldest in numbers for newest in (nthcdr 25 numbers)
        if (not (validp s newest)) return newest
        else do (srem s oldest) (sadd s newest)))

(defun subseq-that-adds-up-to (numbers target &optional frontier)
  (loop with n = (first numbers)
        for (psum . seq) in (cons (cons 0 nil) frontier)
        for psum-next = (+ n psum) for seq-next = (cons n seq)
        if (= psum-next target) return seq-next
        else collect (cons psum-next seq-next) into frontier-next
        finally (return (subseq-that-adds-up-to (rest numbers)

(define-solution (2020 9) (numbers parse-integers)
  (let* ((invalid (find-first-invalid numbers))
         (subseq (subseq-that-adds-up-to numbers invalid)))
      (+ (apply 'min subseq) (apply 'max subseq)))))

2020-12-08 (permalink)

Advent of Code: [2020/08](https://adventofcode.com/2020/day/08)


- Instructions are parsed into a plist, e.g. `(:jmp -1)`
- The whole program is an array if instructions -- array, not a list, so I can quickly jump around
- RUN-PROGRAM returns 2 values: the last value of the accumulator, and whether the program exited normally or not
- Part 1: run the program and return the accumulator value
- Part 2: brute force -- toggle instruction, run, on clear exit return the value, otherwise toggle the instruction again and try with the next one
(defun parse-instruction (string)
  (destructuring-bind (name arg)
      (split-sequence:split-sequence #\Space string)
    (cons (make-keyword (string-upcase name)) (parse-integer arg))))

(defun parse-program (data)
  (map 'vector #'parse-instruction data))

(defun run-program (program)
  (loop with ip = 0 with acc = 0
        for (name . arg) = (aref program ip) do
        (incf ip (if (eql name :jmp) arg 1))
        when (eql name :acc) do (incf acc arg)
        when (member ip seen) return (values acc)
        when (= ip (length program)) return (values acc t)
        collect ip into seen))

(defun toggle-instruction (program i)
  (let ((name (car (aref program i))))
    (unless (eql name :acc)
      (setf (car (aref program i))
            (if (eql name :jmp) :nop :jmp)))))

(define-solution (2020 8) (program parse-program)
    (run-program program)
    (loop for i below (length program)
          when (toggle-instruction program i) do
          (multiple-value-bind (acc clear-exit-p)
              (run-program program)
            (when clear-exit-p (return acc)))
          (toggle-instruction program i))))

Regarding my 2020/07 solution:

- Part 1: For each bag type, recursively unfold its content and see if we can find a `:shiny-gold` bag in it

It turns a more efficient solution is to:

- Invert the _bag-x-contains-these-many-bags mappings_ (and get a _bag-x-is-contained-inside-these-other-bags_ mapping)
- Start from `:shiny-gold` and see what's the number of _distinct_ bags you can get to
(defun invert (table)
  (loop with inverted = nil
        for (from mapping) in table do
        (loop for (to . _) in mapping
              for existing = (assoc to inverted) do
              (if existing
                (push from (second existing))
                (push (list to (list from)) inverted)))
        finally (return inverted)))

(defun collect-containing-bags (curr rules)
    (loop for type in (second (assoc curr rules))
          append (cons type (collect-containing-bags type rules)))))
With this, the solution to part 1 simply becomes: `(length (collect-containing-bags :shiny-gold (invert rules)))`

Advent of Code: [2018/17](https://adventofcode.com/2018/day/17)


- Simulation (i.e. let the water spill and see what happens)
- Start from the position of the spring of water and (recursively) try to spill water down, to the left, or to the right
- Base case 1: the position we are at has already been visited
- Base case 2: the position we are at is _below_ the lowest point of the input map
- Mark the current position as _dripping water_ (i.e. `|`)
- Recurse down, and on the way up try to see if any dripping water at the level below can be marked as _still_ (i.e. `~`) -- this is achievede by going left/right, until we reach a block (`#`) or still water
- If the position we just came _up_ from (after we recursed _down_) turns out to be a block or still water, then we recurse to the left or the right, effectively filling up the reservoirs)
(defun parse-clay-vein (string)
  (cl-ppcre:register-groups-bind (x-or-y (#'parse-integer n m o))
      ("(x|y)=(\\d+).*=(\\d+)..(\\d+)" string)
    (if (string= x-or-y "x")
        (loop with col = n for row from m upto o collect (complex n row))
        (loop with row = n for col from m upto o collect (complex col n)))))

(defun parse-map (data)
  (let ((map (make-hash-table)))
    (dolist (vein (mapcar #'parse-clay-vein data) map)
      (dolist (pos vein)
        (setf (gethash pos map) #\#)))))

(defun row-min-max (map)
  (loop for pos being the hash-keys of map
        minimize (imagpart pos) into min
        maximize (imagpart pos) into max
        finally (return (cons min max))))

(defun left (pos) (+ #c(-1 0) pos))
(defun right (pos) (+ #c(1 0) pos))
(defun down (pos) (+ #c(0 1) pos))

(defun should-be-still-water-p (map pos)
  (labels ((recur (next pos &aux (ch (gethash pos map)))
             (when ch
               (or (find ch "#~")
                   (and (char= ch #\|) (recur next (funcall next pos)))))))
    (and (find (gethash (down pos) map) "#~")
         (recur #'left pos)
         (recur #'right pos))))

(defun mark-row-as-still-water (map pos)
  (loop for cur = pos then (left cur) for ch = (gethash cur map)
        until (find ch "#~") do (setf (gethash cur map) #\~))
  (loop for cur = (right pos) then (right cur) for ch = (gethash cur map)
        until (find ch "#~") do (setf (gethash cur map) #\~)))

(defun spill-water (map)
  (destructuring-bind (row-min . row-max) (row-min-max map)
    (labels ((dfs (pos)
               (cond ((gethash pos map))
                     ((> (imagpart pos) row-max))
                     (t (setf (gethash pos map) #\|)
                        (unless (gethash (down pos) map)
                          (dfs (down pos))
                          (when (should-be-still-water-p map (down pos))
                            (mark-row-as-still-water map (down pos))))
                        (when (find (gethash (down pos) map) "#~")
                          (dfs (left pos))
                          (dfs (right pos)))))))
      (dfs (complex 500 row-min)))))

(define-solution (2018 17) (map parse-map)
  (spill-water map)
  (loop for ch being the hash-values of map
        count (char= ch #\|) into dripping
        count (char= ch #\~) into still
        finally (return (values (+ dripping still) still))))

2020-12-07 (permalink)

Advent of Code: [2020/07](https://adventofcode.com/2020/day/07)


- Parse the input into a list of _rules_, where each rule looks like: `(:muted-white ((:posh-chartreuse . 1) (:dim-silver . 1) (:posh-bronze . 4) (:striped-black . 1)))`
- Part 1: For each bag type, recursively unfold its content and see if we can find a `:shiny-gold` bag in it
- Part 2: Starting from the `shiny-gold` bag, recursively unfold its content and count how many bags are in it
(defun bag-type (string)
  (make-keyword (string-upcase (substitute #\- #\Space string))))

(defun parse-bag-content (string)
  (cl-ppcre:register-groups-bind ((#'parse-integer number)
                                  (#'bag-type type)
      ("(\\d+) ([a-z ]+) bags?(.*)" string)
    (cons (cons type number) (parse-bag-content rest))))

(defun parse-rule (string)
  (cl-ppcre:register-groups-bind ((#'bag-type type) rest)
      ("([a-z ]+) bags contain (.*)" string)
    (list type (parse-bag-content rest))))

(defun parse-rules (data)
  (mapcar #'parse-rule data))

(defun contains-shiny-gold-p (rules curr)
  (or (eq curr :shiny-gold)
      (let ((bag-rule (assoc curr rules)))
        (some (partial-1 'contains-shiny-gold-p rules (first _))
              (second bag-rule)))))

(defun count-bags-inside (curr rules)
  (loop for (type . n) in (second (assoc curr rules))
        sum (+ (* (1+ (count-bags-inside type rules)) n))))

(define-solution (2020 7) (rules parse-rules)
    (count-if (partial-1 #'contains-shiny-gold-p rules)
              (remove :shiny-gold rules :key #'first)
              :key #'first)
    (count-bags-inside :shiny-gold rules)))

2020-12-06 (permalink)

Advent of Code: [2020/06](https://adventofcode.com/2020/day/06)


- Parse groups of answers using similar logic used for 2020/04 -- a group of answers is a list of list of characters
- Part 1: reduce the answers of each group using UNION to get all the distinct answers
- Part 2: reduce the answers of each group using INTERSECTION to get all the unanimous answers
(defun parse-groups (data)
  (let (groups current)
    (dolist (string (append data '("")) groups)
      (if (string= string "")
        (setf groups (cons current groups) current nil)
        (setf current (cons (coerce string 'list) current))))))

(define-solution (2020 6) (groups parse-groups)
  (loop for answers in groups
        sum (length (reduce #'union answers)) into part1
        sum (length (reduce #'intersection answers)) into part2
        finally (return (values part1 part2))))

2020-12-05 (permalink)

Advent of Code: [2020/05](https://adventofcode.com/2020/day/05)


- Intuition 1: boarding passes are just binary numbers (e.g. 'F' -> '0', 'B' -> '1')
- Intuition 2: seat ids are just the binary numbers mentioned above -- _seat-row times 8 plus seat-col_
- Convert boarding passes into seat IDs
- Part 1: take the max of these IDs
- Part 2: sort the IDs, iterate through them, in pairs, and find the _missing_ value
(defun parse-seat (string)
    (map 'string
         #'(lambda (ch)
            (ecase ch
              ((#\F #\L) #\0)
              ((#\B #\R) #\1)))
    :radix 2))

(defun parse-seats (data)
  (mapcar #'parse-seat data))

(define-solution (2020 5) (data)
  (loop with seats = (sort (parse-seats data) #'<) with part2
        for curr in seats for next in (rest seats)
        when (= (- next curr) 2) do (setf part2 (1+ curr))
        finally (return (values curr part2))))
PS. Of course I did not realize about intuition 1 and 2 until after I got my 2 stars and started reading other solutions on Reddit.  For part 1 I originally used binary search to figure out where the boarding passes would take me to; for part 2 instead, I marked as seen all the seats that I encountered, and then iterated through them again looking for the missing one (i.e. first find a row with at least one used seat, then look for another one that has one free).

2020-12-04 (permalink)

Advent of Code: [2020/04](https://adventofcode.com/2020/day/04)


- Parse the passports into a plist of fields, skip "cid" ones completely
- Part 1
- Count the passports that have 7 fields (i.e. 14 elements)
- Part 2
- Implement _validators_ for each field type
- Count the number of passports from part1, that also pass these new checks
(defun parse-line (string)
  (loop for part in (split-sequence:split-sequence #\Space string)
        for (name value) = (split-sequence:split-sequence #\: part)
        unless (string= name "cid")
        append (list (make-keyword (string-upcase name)) value)))

(defun parse-passports (data)
  (let (passports current)
    (dolist (string (append data '("")) passports)
      (if (string= string "")
        (setf passports (cons current passports) current nil)
        (setf current (append current (parse-line string)))))))

(defun has-all-required-fields-p (passport)
  (= (length passport) 14)) ; it's a plist, and we skipped the "cid" field

(defun in-range-p (string left right)
  (<= left (parse-integer string) right))

(defun valid-height-p (string)
  (let ((number (subseq string 0 (- (length string) 2)))
        (unit (subseq string (- (length string) 2))))
    (cond ((string= unit "cm") (in-range-p number 150 193))
          ((string= unit "in") (in-range-p number 59 76)))))

(defun valid-hex-color-p (string)
  (and (char= (char string 0) #\#)
       (parse-integer (subseq string 1) :radix 16)))

(defun valid-color-name-p (string)
  (member string '("amb" "blu" "brn" "gry" "grn" "hzl" "oth")
          :test #'string=))

(defun valid-passportid-p (string)
  (and (= (length string) 9) (parse-integer string)))

(defun field-valid-p (name value)
  (handler-case (case name
                  (:byr (in-range-p value 1920 2002))
                  (:iyr (in-range-p value 2010 2020))
                  (:eyr (in-range-p value 2020 2030))
                  (:hgt (valid-height-p value))
                  (:hcl (valid-hex-color-p value))
                  (:ecl (valid-color-name-p value))
                  (:pid (valid-passportid-p value)))
    (error () nil)))

(defun all-fields-valid-p (passport)
  (loop for (name value) on passport by #'cddr
        always (field-valid-p name value)))

(define-problem (2020 04) (data)
  (let ((passports (remove-if-not #'has-all-required-fields-p
                                  (parse-passports data))))
      (length passports)
      (count-if #'all-fields-valid-p passports))))

How to programmatically create keywords in Common Lisp?

All I am trying to do is re-implementing what the _evalutor_ would do when presented with an expression like `:foo`; the thing is, you need to take into account the _reader_ as well, and in particular remember that by default it would STRING-UPCASE all the symbols it reads.

Say you had the following function, responsible for programmatically creating a keyword symbol:
(defun make-keyword (name)
  (intern (string name) :keyword))
Also say that you wanted to use the generated keyword like this (`name` contains the keyword generated with MAKE-KEYWORD above):
(defun field-valid-p (name value)
  (handler-case (case name
                  (:byr (in-range-p value 1920 2002))
                  (:iyr (in-range-p value 2010 2020))
                  (:eyr (in-range-p value 2020 2030))
                  (:hgt (valid-height-p value))
                  (:hcl (valid-hex-color-p value))
                  (:ecl (valid-color-name-p value))
                  (:pid (valid-passportid-p value)))
    (error () nil)))
If you did not STRING-UPCASE your input (e.g. `(make-keyword "byr")`), the _evaluator_ would end up comparing symbols with different case (e.g. `:|byr|` and `:BYR`), and because of that none of the CASE clauses would match.

To make things work one would have to:

- Simulate what the _reader_ would do (i.e. `(make-keyword (string-upcase "byr"))`)
- Change CASE clauses to use bars (e.g. `:|byr|`, `:|iyr|`)

Good stuff!
[SBCL] AOC/2020/04> (defun make-keyword (name)
                      (intern (string name) :keyword))

[SBCL] AOC/2020/04> (make-keyword "foo")

[SBCL] AOC/2020/04> (make-keyword "FOO")
Someone on [StackOverflow](https://stackoverflow.com/questions/211717/common-lisp-programmatic-keyword) suggested to set the value associated with the newly created symbol to the symbol itself, but the [standard](http://www.lispworks.com/documentation/HyperSpec/Body/11_abca.htm) seems to say that this is already automatically done:

The KEYWORD package is treated differently than other packages in that special actions are taken when a symbol is interned in it. In particular, when a symbol is interned in the KEYWORD package, it is automatically made to be an external symbol and is automatically made to be a constant variable with itself as a value.

2020-12-03 (permalink)

Few weeks [ago](https://matteolandi.net/plan.html#day-2020-10-25) I started looking for ways to synchronize my dotfiles / Workspace across multiple computers without being forced to push new changes to the underlying repositories, and I think I might have found a workflow for this.

As I did not particularly like [the way](https://help.dropbox.com/files-folders/restore-delete/ignored-files) Dropbox wants me to tell which directory should be synchronized, and which one should not (mostly because on different OSes the commands to run differ), I started looking for alternatives and stumbled upon `unison`, the file synchronizer ([user manual](https://www.cis.upenn.edu/~bcpierce/unison/download/releases/stable/unison-manual.html)):

- Has clients for MacOS, Windows, and Linux... check
- Supports ways of ignoring directories... check
- Supports ways of ignoring files... check
- Terminal UI... check

`unison` is mostly meant to keep two roots in sync (i.e. the dotfiles on my laptop always in sync with the dotfiles on my work laptop), but what if I wanted my dotfiles synced with a third machine too?  From the user manual:

If you need to do this, the most reliable way to set things up is to organize the machines into a “star topology,” with one machine designated as the “hub” and the rest as “spokes,” and with each spoke machine synchronizing only with the hub. The big advantage of the star topology is that it eliminates the possibility of confusing “spurious conflicts” arising from the fact that a separate archive is maintained by Unison for every pair of hosts that it synchronizes.


What if that _hub_ node was... Dropbox?! We would get our unsaved changes to automatically propagate _for-free_ (so long as you have Dropbox installed of course).  This is exactly what I tried:

- Clone your dotfiles locally -- as you normally would
- Add your laptop / workstation to your Dropbox account -- so it can sync
- Install `unison` -- it's highly recommended that you run the same version across all your clients, even though I am not 100% sure it's a requirement with this setup
- Use `watch` to periodically fire `unison` and sync up your replicas

That's it! I am still experimenting with this, but it seems to be working.

Few tips if you want to start this journey:

- Don't bother syncing .git directories -- manually clone the repositories on each client and sync the files of the working directory instead
- Try not to sync symbolic links (on MacOS at least), or Dropbox might start abusing of your CPU
- If you tell `unison` to skip links (i.e. `-links false`), it will effectively skip them but still exit with a non-zero value
- `watch` seems to [error out](https://stackoverflow.com/questions/62536396/linux-cli-watch-switch-e-errexit-unexpected-exit-with-command-exit-with-a-n) when the program you told it to run generates more lines of output that the current terminal height, even though its exit code is 0
- Lastly, if you intend to start syncing multiple directories, you might want to create your own `unison` wrapper fixing some of these annoyances, and also containing your favourite settings, common `-ignore` rules -- here is mine
# uniosnw
set -u

ROOT1=${1?Need a value for ROOT1}
ROOT2=${2?Need a value for ROOT2}

unison "$ROOT1" "$ROOT2" \
    -auto \
    -batch \
    -confirmbigdel \
    -confirmmerge \
    -prefer newer \
    -links false \
    -ignore 'Name .git' \
    -ignore 'Name .hg' \
    -ignore 'Name .DS_Store' \
    -ignore 'Name .ropeproject' \
    -ignore 'Name .vagrant' \
    -ignore 'Name __pycache__' \
    -ignore 'Name node_modules' \
    -ignore 'Name target' \
    -ignore 'Name .dependenciesSources' \
    -ignore 'Name venv' \
    -ignore 'Name *.db' \
    -ignore 'Name *.dmg' \
    -ignore 'Name *.lock' \
    -ignore 'Name *.log' \
    -ignore 'Name *.pyc' \
    -ignore 'Name *.vdi' \
    -ignore 'Name tags*' \
    -ignore 'Name *.egg-info' \

test $? -le 1 # exit code: 1 means something was skipped but rest went fine
You can then easily "wrap this wrapper" into another script responsible for synchronizing two specific replicas:
# uniosnw-me
set -e

exec >>~/unison-me.log 2>&1 # shut up, `watch`: https://stackoverflow.com/q/62536396

unisonw \
    "$HOME/my-env" \
    "$HOME/Dropbox/unison/my-env" \
    -ignore 'Regex .*/\.mutt/cache' \
    -ignore 'Regex .*/\.mutt/temp' \
    -ignore 'Regex .*/\.vim/spell' \
    -ignore 'Regex .*/\.vim/tmp' \
    -ignore 'Regex .*/\.vim/.netrwbook' \
    -ignore 'Regex .*/\.vim/.netrwhist' \
And then periodically synchronize replicas as follows:
$ watch --interval 60 --beep --errexit --exec unisonw-me
That's it for today!

Advent of Code: [2020/03](https://adventofcode.com/2020/day/03)


- Parse the input into a 2D array
- Start from `row=0,col=0`
- Move as per the given delta, wrapping around when horizontally moving outside of the map
- While doing so (i.e. moving inside the map), count the number of trees you hit
(defun parse-map (data)
  (let ((rows (length data)) (columns (length (first data))))
    (make-array (list rows columns) :initial-contents data)))

(defun count-hits (map delta-col delta-row)
  (loop with (rows cols) = (array-dimensions map)
        for row below rows by delta-row
        for col = 0 then (mod (+ col delta-col) cols)
        count (char= (aref map row col) #\#)))

(define-problem (2020 03) (map parse-map)
    (count-hits map 3 1)
    (loop for (delta-col delta-row) in '((1 1) (3 1) (5 1) (7 1) (1 2))
          collect (count-hits map delta-col delta-row) into hits
          finally (return (reduce #'* hits)))))

2020-12-02 (permalink)

+ change my `unison` wrapper to delete broken links when syncing to Dropbox

Advent of Code: [2020/02](https://adventofcode.com/2020/day/02)


- Parse the input into `(n m ch text)` lists
- Part 1
- count entries for which the number of occurrences of `ch` in `text` is between `n` and `m`
- Part 2
- count the entries for which `text[n] == ch` and `text[m] != ch` or `text[n] != ch` and `text[m] == ch`
(defun parse-password (string)
  (cl-ppcre:register-groups-bind ((#'parse-integer n m) (#'parse-char ch) text)
      ("(\\d+)-(\\d+) (\\w): (\\w+)" string)
    (list n m ch text)))

(defun parse-passwords (data)
  (mapcar #'parse-password data))

(defun valid-password-part1-p (min max ch text)
  (<= min (count ch text) max))

(defun valid-password-part2-p (pos1 pos2 ch text)
  (let ((ch1 (aref text (1- pos1)))
        (ch2 (aref text (1- pos2))))
    (cond ((char= ch1 ch) (char/= ch2 ch))
          ((char= ch2 ch) (char/= ch1 ch)))))

(define-problem (2020 2) (passwords parse-passwords)
  (loop for (n m ch text) in passwords
        count (valid-password-part1-p n m ch text) into part1
        count (valid-password-part2-p n m ch text) into part2
        finally (return (values part1 part2))))

2020-12-01 (permalink)

Advent of Code: [2020/01](https://adventofcode.com/2020/day/01)


- Part 1
- iterate the list of numbers
- for each, see if in the remainder of the list there is one element such that the sum of both is 2020
- Part 2
- iterate the list of numbers
- for each, see if in the remainder of the list there are two elements such that their sum is "2020 minus the currently selected element"
(defun find-pair-that-adds-up-to (target integers)
  (loop for (n . rest) on integers
        for m = (find target rest :key (partial-1 #'+ n))
        when m return (list n m)))

(define-problem (2020 1) (integers parse-integers)
  (values (reduce #'* (find-pair-that-adds-up-to 2020 integers))
          (loop for (n . rest) on integers
                for (m o) = (find-pair-that-adds-up-to (- 2020 n) rest)
                when m return (* n m o))))

2020-11-30 (permalink)

Advent of Code: [2016/24](https://adventofcode.com/2016/day/24)


- First precompute the best path between any two sites (i.e. cells with numeric values) of the map -- this is done using A*
- Next run another A* trying to get the robot to visit each site
- The state is codified as the current position of the robot (i.e. which site it's at), and a bitmask representing what other sites have already been visited
- Part1: no heuristic, and return the final cost
- Part2: use the cost to go back to position 0 as heuristic, and return the final cost plus the gost to go back to position 0
(defstruct blueprint
  (walls (make-hash-table))

(defun char- (a b) (- (char-code a) (char-code b)))

(defun read-blueprint (data)
  (let ((blueprint (make-blueprint))
        (points-of-interest (make-hash-table)))
    (with-slots (walls sites) blueprint
      (loop for row from 0 for string in data do
            (loop for column from 0 for ch across string
                  for pos = (complex column row) do
                    ((char= ch #\#) (setf (gethash pos walls) t))
                    ((char= ch #\.) nil)
                    (t (setf (gethash pos points-of-interest) (char- ch #\0))))))
      (setf sites (make-array (hash-table-count points-of-interest)))
      (maphash #'(lambda (pos poi) (setf (aref sites poi) pos)) points-of-interest))

(defun find-path (walls from to)
  (nth-value 1 (a* from :goal-state to
                   :neighbors (lambda (p)
                                (loop for n in (adjacents p)
                                      unless (gethash n walls)
                                      collect (cons n 1)))
                   :heuristic (partial-1 #'manhattan-distance to))))

(defun precompute-paths (blueprint)
  (with-slots (walls sites) blueprint
    (let* ((n (length sites))
           (paths (make-array (list n n))))
      (loop for i from 0 below n do
            (loop for j from i below n
                  for cost = (find-path walls (aref sites i) (aref sites j)) do
                  (setf (aref paths i j) cost
                        (aref paths j i) cost)))

(defstruct (state (:conc-name)) robot visited)

(defun neighbors (paths site-count state)
  (with-slots (robot visited) state
    (loop for j below site-count
          for cost = (aref paths robot j)
          unless (logbitp j visited)
          collect (cons (make-state :robot j
                                    :visited (logior visited (ash 1 j)))

(defun visit-sites (blueprint)
  (let* ((init-state (make-state :robot 0 :visited 1))
         (site-count (length (blueprint-sites blueprint)))
         (all-sites (1- (ash 1 site-count)))
         (paths (precompute-paths blueprint)))
    (nth-value 1 (a* init-state :goalp (partial-1 #'= (visited _) all-sites)
                     :neighbors (partial-1 #'neighbors paths site-count)))))

(defun visit-sites-part2 (blueprint)
  (let* ((init-state (make-state :robot 0 :visited 1))
         (site-count (length (blueprint-sites blueprint)))
         (all-sites (1- (ash 1 site-count)))
         (paths (precompute-paths blueprint)))
    (multiple-value-bind (end-state end-state-cost)
        (a* init-state :goalp (partial-1 #'= (visited _) all-sites)
            :neighbors (partial-1 #'neighbors paths site-count)
            :heuristic #'(lambda (s)
                          (aref paths (robot s) 0)))
      (+ end-state-cost (aref paths (robot end-state) 0)))))

(define-problem (2016 24) (blueprint read-blueprint)
  (values (visit-sites blueprint)
          (visit-sites-part2 blueprint)))

Advent of Code: [2016/25](https://adventofcode.com/2016/day/25)


- Brute-force
- Use/abuse CL condition system (the new "out" instruction simply SIGNALs the value up, so we can use HANDLER-BIND to inspect the output without unwinding the execution stack)
- For each emitted value
- ...if it's not part of the clock signal sequence, signal an error, and restart the program with the _next_ input (we use HANDLER-CASE here, as we want to unwind the stack and move on)
- ...if it's part of the clock signal sequence, and it's been like that for the last 10 emitted elements (it's an heuristic), then we found our magic input
(define-condition not-clock-signal () ())
(define-condition clock-signal-indeed () ())

(defun generates-clock-signal-p (input program)
  (let ((remaining 10)
        (expected (ncycle (list 0 1))))
        (handler-bind ((assembunnycode:program-output
                         (lambda (c)
                           (if (/= (assembunnycode:output-value c) (car expected))
                             (error 'not-clock-signal)
                               (setf remaining (1- remaining)
                                     expected (cdr expected))
                               (when (zerop remaining)
                                 (signal 'clock-signal-indeed)))))))
          (assembunnycode:run program (list input 0 0 0)))
      (not-clock-signal () nil)
      (clock-signal-indeed () t))))

(define-problem (2016 25) (program assembunnycode:parse-program)
  (loop for n from 0
        when (generates-clock-signal-p n program) return n))

2020-11-29 (permalink)

Advent of Code: [2016/23](https://adventofcode.com/2016/day/23)


- Started from the solution of 2016/12
- Made dynamic variables for the instruction pointer and the program being run, so I can easily access them both from the body of the new instruction, `tgl`
- Implement the new instruction, `tgl`, to use `*ip*` to figure out which program instruction to _toggle_, and then update `*program*` accordingly
- Note 1: as the example shows, `jnz` now supports reading the jump offset from a registry, so its implementation needs to be updated
- Note 2: not just its implementation, but it's parser too! I wasted too much time figuring out why my program was looping...and it turns out it was because I forgot to update PARSE-JNZ
- Part 1: set register `a` to `7`, and run the program
- Part 2: set register `a` to `12`, and run the program...and after only a couple of **minutes** it should spit out the corrent answer
(defvar *regs* nil)
(defvar *ip* nil)
(defvar *program* nil)

(defun reg-name-index (reg)
  (- (char-code reg) (char-code #\a)))

(defun reg (name)
  (aref *regs* (reg-name-index name)))

(defun (setf reg) (value name)
  (setf (aref *regs* (reg-name-index name)) value))

(defun reg-or-value (x)
  (if (numberp x) x (reg x)))

(defun i-cpy (x reg)
  (prog1 1 (setf (reg reg) (reg-or-value x))))

(defun i-inc (reg) (prog1 1 (incf (reg reg))))

(defun i-dec (reg) (prog1 1 (decf (reg reg))))

(defun i-jnz (x y)
  (if (zerop (reg-or-value x)) 1 (reg-or-value y)))

(defun i-tgl (x)
  (prog1 1
    (let* ((offset (+ (reg-or-value x) *ip*)))
      (when (array-in-bounds-p *program* offset)
        (destructuring-bind (fun . args) (aref *program* offset)
          (setf (car (aref *program* offset))
                (ecase (length args)
                  (1 (if (eq fun #'i-inc) #'i-dec #'i-inc))
                  (2 (if (eq fun #'i-jnz) #'i-cpy #'i-jnz)))))))))

(defun parse-value (string)
  (parse-integer string :junk-allowed t))

(defun parse-reg (string)
  (aref string 0))

(defun parse-reg-or-value (string)
  (or (parse-value string) (parse-reg string)))

(defun parse-cpy (string)
  (let ((parts (split-sequence:split-sequence #\Space string)))
    (when (string= (first parts) "cpy")
      (list #'i-cpy (parse-reg-or-value (second parts)) (parse-reg (third parts))))))

(defun parse-inc (string)
  (let ((parts (split-sequence:split-sequence #\Space string)))
    (when (string= (first parts) "inc")
      (list #'i-inc (parse-reg (second parts))))))

(defun parse-dec (string)
  (let ((parts (split-sequence:split-sequence #\Space string)))
    (when (string= (first parts) "dec")
      (list #'i-dec (parse-reg (second parts))))))

(defun parse-jnz (string)
  (let ((parts (split-sequence:split-sequence #\Space string)))
    (when (string= (first parts) "jnz")
      (list #'i-jnz (parse-reg-or-value (second parts)) (parse-reg-or-value (third parts))))))

(defun parse-tgl (string)
  (let ((parts (split-sequence:split-sequence #\Space string)))
    (when (string= (first parts) "tgl")
      (list #'i-tgl (parse-reg-or-value (second parts))))))

(defun parse-instruction (string)
  (or (parse-cpy string) (parse-inc string) (parse-dec string) (parse-jnz string)
      (parse-tgl string)))

(defun parse-program (data)
  (map 'vector #'parse-instruction data))

(defun run (program regs)
  (let ((*regs* regs)
        (*ip* 0)
        (*program* program))
    (loop while (< *ip* (length program))
          for (fun . args) = (aref program *ip*)
          do (incf *ip* (handler-case (apply fun args)
                          (type-error (c)
                            (format t "Error when evaluating ~a with ~a: ~s ~s~%" fun args c c)
    (reg #\a)))

(define-problem (2016 23) (data)
    (run (parse-program data) (make-array 4 :initial-contents (list 7 0 0 0)))
    (swallow (run (parse-program data) (make-array 4 :initial-contents (list 12 0 0 0))))))
Clearly there is a more clever solution to this, especially for part 2, but I did not feel like optimizing my input / implementing a new `mul` instruction (as hinted in the problem description), so I am going to leave that to my future self for now!

2020-11-28 (permalink)

Advent of Code: [2016/22](https://adventofcode.com/2016/day/22)


- Part 1
- nothing crazy, compare each node with every other one and count the pairs which are _viable_
- Note to future self 1: read the text carefully
- Note to future self 2: read the text carefully
- Note to future self 3: read the text carefully
- Note to future self 4: don't make up any additional constraint
- Part 2
- intuition 1: to move data from disk a to c passing from b, you need to create some space on disk b first, move data from a to b, create some space to disk c, and finally move data from b to c
- intuition 2: you can only move data on disks which are _compatible_ with the whichever disk in your input happens to be unused (what in my solution are called: interchangeable nodes)
- A*: starting from the _target_ node, try to move its data to `0,0` -- the state object keeps track of where the target data is at, and which node is empty
- the cost of moving the target data into one of its adjacents disks is `1` plus the cost it takes to _empty that disk first_
- The cost of emptying a specific disk is implemented with another A* in which we _move_ the empty disk to the target location
- Note: when _moving_ the empty disk, make sure you don't touch the node where the target data is currently located at
(defun pos (n) (nth 0 n))
(defun size (n) (nth 1 n))
(defun used (n) (nth 2 n))
(defun avail (n) (nth 3 n))

(defun parse-node (string)
  (cl-ppcre:register-groups-bind ((#'parse-integer c r size used avail))
      ("node-x(\\d+)-y(\\d+)\\s+(\\d+)T\\s+(\\d+)T\\s+(\\d+)" string)
    (list (complex c r) size used avail)))

(defun parse-nodes (data)
  (remove nil (mapcar #'parse-node data)))

(defun viable-pair-p (n1 n2)
  (and (plusp (used n1))
       (not (eq n1 n2))
       (< (used n1) (avail n2))))

(defun viable-pairs (nodes)
  (loop for n1 in nodes
        append (loop for n2 in nodes
                     when (viable-pair-p n1 n2)
                     collect (list n1 n2))))

(defun find-target-node (nodes)
  (let (c-max max)
    (dolist (n nodes max)
      (with-complex-parts (c r) (pos n)
        (when (and (zerop r)
                   (or (null c-max)
                       (> c c-max)))
          (setf c-max c
                max n))))))

(defun find-empty-node (nodes)
  (find 0 nodes :key #'used))

(defun interchangeable-nodes (nodes)
  (loop with empty = (find-empty-node nodes)
        for n in nodes
        when (and (<= (used n) (size empty)))
        collect n))

(defstruct (state (:conc-name)) cur empty)

(defun cost-to-make-space (grid fixed from to)
  (nth-value 1 (a* from
                   :goal-state to
                   :neighbors (search-unit-cost (lambda (pos)
                                                  (loop for n in (adjacents pos)
                                                        when (and (gethash pos grid)
                                                                  (/= pos fixed))
                                                        collect n)))
                   :heuristic (partial-1 #'manhattan-distance to))))

(defun neighbors (state grid)
  (with-slots (cur empty) state
    (loop for next in (adjacents cur)
          for cost = (and (gethash next grid)
                          (cost-to-make-space grid cur empty next))
          when cost collect (cons (make-state :cur next
                                              :empty cur)
                                  (1+ cost)))))

(defun move-data (nodes)
  (let* ((intechangeables (interchangeable-nodes nodes))
         (grid (list-hash-table intechangeables #'pos))
         (init-state (make-state :cur (pos (find-target-node intechangeables))
                                 :empty (pos (find-empty-node intechangeables)))))
    (nth-value 1 (a* init-state
                     :goalp (partial-1 #'= (cur _) 0)
                     :neighbors (partial-1 #'neighbors _ grid)
                     :heuristic (partial-1 #'manhattan-distance (cur _) 0)
                     :test 'equalp))))

(define-problem (2016 22) (nodes parse-nodes)
  (values (length (viable-pairs nodes))
          (move-data nodes)))

2020-11-27 (permalink)

Advent of Code: [2016/21](https://adventofcode.com/2016/day/21)


- Implement scramble operations (tried to be smart about it, and failed...so please don't be me, do the simplest thing possible)
- Parse the input
- Part1: scramble the input
- Part2: almost stared _inverting_ each function, then noticed there were only `8!` possible inputs to try, so I went on and bruteforced it (i.e. generate all inputs, scramble them, and find the one that's equal to "fbgdceah")
(defun swap-position (string x y)
  (let ((copy (copy-seq string)))
    (rotatef (aref copy x) (aref copy y))

(defun swap-letter (string c1 c2)
  (let ((copy (copy-seq string))
        (pos1 (position c1 string))
        (pos2 (position c2 string)))
    (rotatef (aref copy pos1) (aref copy pos2))

(defun rotate-left (string steps &aux (steps (mod steps (length string))))
  (concatenate 'string
               (subseq string steps)
               (subseq string 0 steps)))

(defun rotate-right (string steps)
  (let ((tail-size (- (length string) (mod steps (length string)))))
    (concatenate 'string
                 (subseq string tail-size)
                 (subseq string 0 tail-size))))

(defun rotate-on-position (string c)
  (let ((index (position c string)))
    (rotate-right string (if (>= index 4) (+ index 2) (1+ index)))))

(defun reverse-in-between (string start end)
  (concatenate 'string
               (subseq string 0 start)
               (reverse (subseq string start (1+ end)))
               (subseq string (1+ end))))

(defun move (string x y)
  (let ((with-x-popped (concatenate 'string
                           (subseq string 0 x)
                           (subseq string (1+ x)))))
    (concatenate 'string
                 (subseq with-x-popped 0 y)
                 (string (aref string x))
                 (subseq with-x-popped y))))

(defun parse-swap-position (string)
  (cl-ppcre:register-groups-bind ((#'parse-integer x y))
      ("swap position (\\d+) with position (\\d+)" string)
    (list #'swap-position x y)))

(defun parse-char (string)
  (char string 0))

(defun parse-swap-letter (string)
  (cl-ppcre:register-groups-bind ((#'parse-char c1 c2))
      ("swap letter (\\w) with letter (\\w)" string)
    (list #'swap-letter c1 c2)))

(defun parse-rotate-leftright (string)
  (cl-ppcre:register-groups-bind (dir (#'parse-integer x))
      ("rotate (left|right) (\\d+) step" string)
    (list (if (string= dir "left") #'rotate-left #'rotate-right) x)))

(defun parse-rotate-on-position (string)
  (cl-ppcre:register-groups-bind ((#'parse-char c))
      ("rotate based on position of letter (\\w)" string)
    (list #'rotate-on-position c)))

(defun parse-reverse-in-between (string)
  (cl-ppcre:register-groups-bind ((#'parse-integer start end))
      ("reverse positions (\\d+) through (\\d+)" string)
    (list #'reverse-in-between start end)))

(defun parse-move (string)
  (cl-ppcre:register-groups-bind ((#'parse-integer x y))
      ("move position (\\d+) to position (\\d+)" string)
    (list #'move x y)))

(defun parse-scramble-op (string)
  (or (parse-swap-position string)
      (parse-swap-letter string)
      (parse-rotate-leftright string)
      (parse-rotate-on-position string)
      (parse-reverse-in-between string)
      (parse-move string)
      (error string)))

(defun parse-scramble-script (data)
  (mapcar #'parse-scramble-op data))

(defun scramble (script input)
  (loop for string = input then (apply fun string args)
        for (fun . args) in script
        finally (return string)))

(defun permutations-string (string)
  (mapcar #'(lambda (x) (coerce x 'string))
          (all-permutations (coerce string 'list))))

(define-problem (2016 21) (script parse-scramble-script)
  (values (scramble script "abcdefgh")
          (loop for input in (permutations-string "abcdefgh")
                when (string= (scramble script input) "fbgdceah")
                return input)))
I am not quite happy with how long/verbose the solution turned out to be, but whatever...

2020-11-26 (permalink)

Advent of Code: [2016/20](https://adventofcode.com/2016/day/20)


- Merge overlapping ranges
- At first I thought about using union-find, but ultimately opted for...
- ...sort ranges (smallest start value first, then smallest end value)
- ...scan the sorted list of ranges and merge the _overlapping_ ones
- Part1: take the smallest non-overlapping range, and return `end + 1`
- Part2: iterate non-overlapping ranges, and start from `4294967295` keep on subtracting the size of the current range
(defparameter *ip-max* 4294967295)

(defun parse-range (string)
  (mapcar #'parse-integer (split-sequence:split-sequence #\- string)))

(defun parse-ranges (data)
  (mapcar #'parse-range data))

(defun range< (range1 range2)
  (destructuring-bind (left1 right1) range1
    (destructuring-bind (left2 right2) range2
      (or (< left1 left2)
          (and (= left1 left2)
               (<= right1 right2))))))

(defun merge-overlapping (ranges)
  (let ((sorted (sort (copy-seq ranges) #'range<)))
    (loop with (merged-start merged-end) = (first sorted)
          for (start end) in (rest sorted)
          if (> (1- start) merged-end)
            collect (list merged-start merged-end) into ranges
            and do (setf merged-start start merged-end end)
          else do (setf merged-end (max end merged-end))
          finally (return (append ranges
                                  (list (list merged-start merged-end)))))))

(define-problem (2016 20) (ranges parse-ranges)
  (loop with part1 with part2 = (1+ *ip-max*)
        for (start end) in (merge-overlapping ranges)
        for size = (1+ (- end start)) do
        (when (not part1) (setf part1 (1+ end)))
        (decf part2 size)
        finally (return (values part1 part2))))

2020-11-25 (permalink)

Advent of Code: [2016/19](https://adventofcode.com/2016/day/19)


- I thought I could map the _staling_ operation with some mathematical formula, so then I tried the _naive_ approach (i.e. simulate it), and it just worked!
- Create a _circular_ list containing the index associated with each elf
- Keep a pointer to the _next_ element to be removed
- At each iteration remove such element, update the pointer, and repeat until only one elf is left
- Part 1: the _next_ element to be removed is simply the second element of the circular list
- Part 2: we start from the _middle_ of the list (i.e. which elf is sitting _in front_ of...), remove the _next_ element, and update the pointer _accordingly_
- ...if the size prior to the latest removal was odd, then we move our pointer one step forward (to skip the other elf that happend to be sitting in front of ...)
- ...otherwise, we don't update the pointer (i.e. the next element to remove is right after the current pointer)
(defun parse-num-elfes (data)
  (parse-integer (first data)))

(defun part1 (players)
  (let ((playing (ncycle (loop for i from 1 upto players collect i))))
    (loop repeat players
          for remaining = playing then (cdr remaining) do
          (setf (cdr remaining) (cddr remaining))
          finally (return (first remaining)))))

(defun part2 (players)
  (let ((playing (ncycle (loop for i from 1 upto players collect i))))
    (loop with middle = (nthcdr (1- (floor players 2)) playing)
          for size from players downto 1
          for remaining = middle then (if (evenp size) (cdr remaining) remaining) do
          (setf (cdr remaining) (cddr remaining))
          finally (return (first remaining)))))

(define-problem (2016 19) (elves parse-num-elfes)
    (part1 elves)
    (part2 elves)))

2020-11-24 (permalink)

Advent of Code: [2016/18](https://adventofcode.com/2016/day/18)


- Initially thought about solving this using numbers and bit masks, but then opted for the simpler string-based approach
- Nothing crazy with the implementation, except:
- added two extra "." (i.e. gutters) to the input so I won't have to deal the _edges_
- for the logic to decide whether a tile is safe or it's a trap -- thanks to [Karnaugh](https://en.wikipedia.org/wiki/Karnaugh_map) I was able to simplify the original set of rules to: _are left / right chars the same? then it's safe, otherwise it's not_
(defun add-gutters (string)
  (format nil ".~A." string))

(defun parse-trap-row (data &aux (string (first data)))
  (add-gutters string))

(defun its-a-trap-p (row index)
  ;; Then, a new tile is a trap only in one of the following situations:
  ;; - Its left and center tiles are traps, but its right tile is not.
  ;; - Its center and right tiles are traps, but its left tile is not.
  ;; - Only its left tile is a trap.
  ;; - Only its right tile is a trap.
  ;; In any other situation, the new tile is safe.
  ;; It turns out (thanks Karnaugh) that this can be simplified to:
  ;; - Are left/right the same? Then it's safe
  ;; - Otherwise, it's a trap
  (char/= (aref row (1- index)) (aref row (1+ index))))

(defun next (row)
  (let ((next (make-string (length row) :initial-element #\.)))
    (loop repeat (- (length row) 2)
          for index from 1 do
          (setf (aref next index) (if (its-a-trap-p row index) #\^ #\.)))

(defun generate (row times)
  (loop repeat times
        summing (count #\. row) into safe do
        (setf row (next row))
        finally (return (- safe (* times 2))))) ; offset 2 gutters per row

(define-problem (2016 18) (state parse-trap-row)
    (generate state 40)
    (generate state 400000)))
PS. Even with all these string operations, part2 runs in around around 1.5 second; 8 seconds instead, if we did not apply any Karnaugh magic.

2020-11-23 (permalink)

Alright, alright, the waiting is _almost_ over: a couple of weeks ago [Eric Wastl](http://was.tl/) (creator of Advent of Code) [tweeted](https://twitter.com/ericwastl/status/1328254144545763328) that the 2020 [site](https://adventofcode.com/2020) is finally up -- don't get too excited though, for the first problem you will still have to wait until December 1st, midnight time (EST/UTC-5)!

For those of you not yet familiar with this (taken from the site's [about](https://adventofcode.com/2020/about) page):

Advent of Code is an Advent calendar of small programming puzzles for a variety of skill sets and skill levels that can be solved in any programming language you like. People use them as a speed contest, interview prep, company training, university coursework, practice problems, or to challenge each other.

What do you think?

- Will you participate?  (Yes)
- What programming language would you use? ([Common Lisp](https://github.com/iamFIREcracker/adventofcode) of course)
- Editor / IDE?  ([Vim](https://github.com/iamFIREcracker/dotfiles/tree/master/.vim), the only text editor)

It's going to be fantastic...and brutal at the same time (too many smart cookies here, waking up at 6 am to get ahold of their stars as quickly as possible), but that's the beauty of it!

Speaking of waking up early, it [turns out](https://twitter.com/ericwastl/status/1329683037651673089) they got a DJ to play a live set with a big drop at the exact time the first problem will be revealed, so if you planned on waking up early, or simply happen to live on the right part of Earth, you might want to [tune in](https://t.co/9r8pd95G8v?amp=1).

_In bocca al lupo!_

Advent of Code: [2016/17](https://adventofcode.com/2016/day/17)


- A state is a combination of the current position, and how we got there (e.g. "RLRLDU...")
- Try to move up, down, left, right if
- ...it's not off the map
- ...the door is open
- To see if a door is unlocked, run md5 of the path so far, and extract the first 2 bytes -- nothing crazy, I simply followed instructions
- Stop when you made it to the bottom-right corner -- for Part 1 at least
- Part 2: BFS + pruning
- ...don't stop when you made it to the bottom-right corner
- ...instead check the size of the path, and then _prune_ the result
(defparameter *nhood* '(#C(0 1) #C(0 -1) #C(-1 0) #C(1 0)))
(defparameter *steps* "UDLR")

(defun door-states (seed path)
  (let* ((string (format nil "~A~A" seed path))
         (bytes (md5:md5sum-string string))
         (result (make-array 4 :element-type t :fill-pointer 0)))
    (dotimes (i 2)
      (let ((byte1 (ldb (byte 4 4) (aref bytes i)))
            (byte2 (ldb (byte 4 0) (aref bytes i))))
        (vector-push (> byte1 10) result)
        (vector-push (> byte2 10) result)))

(defstruct (state (:conc-name)) pos path)

(defun neighbors (seed state)
  (with-slots (pos path) state
    (loop for delta in *nhood*
          for step across *steps*
          for openp across (door-states seed path)
          for next = (+ pos delta)
          when (and (<= 0 (realpart next) 3)
                    (>= 0 (imagpart next) -3)
          collect (make-state :pos next
                              :path (format nil "~A~C" path step)))))

(defun part1 (seed)
  (path (bfs (make-state :pos #c(0 0) :path "")
             :goalp #'(lambda (state) (= (pos state) #c(3 -3)))
             :neighbors (partial-1 #'neighbors seed)
             :test 'equalp)))

(defun part2 (seed)
  (let (max)
    (bfs (make-state :pos #c(0 0) :path "")
         :prunep #'(lambda (state cost)
                    (when (null state) (break))
                    (with-slots (pos) state
                      (when (= pos #c(3 -3))
                        (when (or (null max)
                                  (> cost max))
                          (setf max cost))
         :neighbors (partial-1 #'neighbors seed)
         :test 'equalp)

(define-problem (2016 17) (seed first)
    (part1 seed)
    (part2 seed)))

2020-11-22 (permalink)

? vim-sexp's element mappings do not work with COMPLEX

Advent of Code: [2016/14](https://adventofcode.com/2016/day/14)


- Cache md5 hashes -- you don't want to calculate the same hash over and over again
- Implement the cache with a ring buffer -- you don't want to use a hash-table, as there is no need to keep around previously processed hashes
- Rest is implementing logic for finding triplets, and quintets (done using `cl-ppcre`)
(defvar *salt* nil)
(defvar *key-stretching* nil)

(defun circular-cache (size &optional initial-element)
  (let ((cache (make-list size :initial-element initial-element)))
    (ncycle cache)))

(defun generate-hash (index)
  (let ((string (format nil "~A~D" *salt* index)))
    (dotimes (n (1+ *key-stretching*) string)
      (setf string (hexadecimal-string (md5:md5sum-string string))))))

(defun hash (index cache)
  (if-let ((cached (car cache)))
    (setf (car cache) (generate-hash index))))

(defun find-triplet-char (string)
  (when-let ((pos (cl-ppcre:scan "(.)\\1\\1" string)))
    (aref string pos)))

(defun contains-quintet-p (char string)
  (cl-ppcre:scan (format nil "~A{5}" char) string))

(defun keyp (index cache char)
  (loop :repeat 1000
        :for lcache = cache :then (cdr lcache)
        :for lindex = index :then (1+ lindex)
        :for hash = (hash lindex lcache)
        :thereis (contains-quintet-p char hash)))

(defun solve (salt &key (key-stretching 0))
  (let ((*salt* salt)
        (*key-stretching* key-stretching))
    (loop :for cache = (circular-cache 1001) :then (cdr cache)
          :for index = 0 :then (1+ index)
          :for hash = (hash index cache)
          :for char = (find-triplet-char hash)
          :count (and char (keyp (1+ index) (cdr cache) char)) :into keys
          :when (= keys 64) :return index
          :do (setf (car cache) nil))))

(define-problem (2016 14) (salt first)
    (solve salt)
    (solve salt :key-stretching 2016)))
Good, except when we set key-stretching to 2017 as part 2 requires, this implementation takes like _forever_ to generate the _correct_ solution:
[SBCL] AOC/2016/14> (time (problem-run))
Evaluation took:
  377.863 seconds of real time
  335.047006 seconds of total run time (329.634168 user, 5.412838 system)
  [ Run times consist of 13.299 seconds GC time, and 321.749 seconds non-GC time. ]
  88.67% CPU
  869,059,756,917 processor cycles
  106,067,224,416 bytes consed

Instead of:

- Find triplet char
- For each of of the next 1000 hashes, find one that contains a quintet with that char

What if we pre-calculated all the possible quintet characters, and searched the triplet one inside of it?
(defun find-all-quintet-chars (string)
  (mapcar #'(lambda (match) (aref match 0))
          (cl-ppcre:all-matches-as-strings "(.)\\1\\1\\1\\1" string)))

(defun hash (index cache)
  (if-let ((cached (car cache)))
    (let ((hash (generate-hash index)))
      (setf (car cache) (cons hash (find-all-quintet-chars hash))))))

(defun keyp (index cache char)
  (loop :repeat 1000
        :for lcache = cache :then (cdr lcache)
        :for lindex = index :then (1+ lindex)
        :for (hash . quintets) = (hash lindex lcache)
        :thereis (member char quintets)))

(defun solve (salt &key (key-stretching 0))
  (let ((*salt* salt)
        (*key-stretching* key-stretching))
    (loop :for cache = (circular-cache 1001) :then (cdr cache)
          :for index = 0 :then (1+ index)
          :for (hash . ignored) = (hash index cache)
          :for char = (find-triplet-char hash)
          :count (and char (keyp (1+ index) (cdr cache) char)) :into keys
          :when (= keys 64) :return index
          :do (setf (car cache) nil))))
Let's run it and see how long it takes:
[SBCL] AOC/2016/14> (time (problem-run))
Evaluation took:
  281.838 seconds of real time
  276.823709 seconds of total run time (274.384517 user, 2.439192 system)
  [ Run times consist of 10.966 seconds GC time, and 265.858 seconds non-GC time. ]
  98.22% CPU
  648,208,539,419 processor cycles
  104,197,076,032 bytes consed

Well, that's _something_, though it's still taking a bit too much time.

I suspected my dumb `HEXADECIMAL-STRING` might have had something to do with it, so I took a look at [ironclad](https://quickref.common-lisp.net/ironclad.html)'s `BYTE-ARRAY-TO-HEX-STRING` function, and gut the following out of it (did not feel like neither copying that over, in its full length, nor adding an additional dependency):
(defun md5-bytes-string (bytes)
  (let ((hexdigits #.(coerce "0123456789abcdef" 'simple-base-string)))
    (loop :with string = (make-string 32 :element-type 'base-char)
          :for byte :across bytes
          :for index :below 32 :by 2
          :do (setf (aref string index)
                    (aref hexdigits (ldb (byte 4 4) byte))
                    (aref string (1+ index))
                    (aref hexdigits (ldb (byte 4 0) byte)))
          :finally (return string))))
Run the program again, and:
[SBCL] AOC/2016/14> (time (problem-run))
Evaluation took:
  38.225 seconds of real time
  36.210049 seconds of total run time (35.759976 user, 0.450073 system)
  [ Run times consist of 1.091 seconds GC time, and 35.120 seconds non-GC time. ]
  94.73% CPU
  87,914,983,655 processor cycles
  18,883,617,520 bytes consed

Not bad, I can live with that!

PS. For the curious ones wondering how badly `HEXADECIMAL-STRING` might have been implemented: very badly (for this use case at least)
(defun hexadecimal-string (bytes)
  (format nil "~(~{~2,'0X~}~)" (coerce bytes 'list)))

Advent of Code: [2016/15](https://adventofcode.com/2016/day/15)


- Disk #1 has 5 positions and starts at 4, meaning it will be at position 0 at time: 1, 6, 11, 16...
- It will take the capsure 1 second to get to the first disk, which means the button should pressed at any of the instants above, minus 1 second: 0, 5, 10, 15...
- Disk #2 has 2 positions and starts at 1, meaning it will be at position 0 at time: 1, 3, 5, 7...
- The capsule will take 2 seconds to get to disk #2, so in order for the disk to be at position 0 when the capsule reaches it, is at time: -1, 1, 3, 5...
- For each disk, we can then create _generators_ (i.e. returning all the times the disk will be at position 0)
- We start pulling values out of these, until we find a common value
- Note: start pulling from the generator with the lowest value
(defun parse-disk (string)
  (cl-ppcre:register-groups-bind ((#'parse-integer disk size time position))
      ("Disc #(\\d+) has (\\d+) positions; at time=(\\d+), it is at position (\\d+)." string)
    (list disk size time position)))

(defun position-generator (disk size time position)
  (let* ((at-pos-0 (- time position))
         (current (- at-pos-0 disk)))
    (labels ((next ()
               (incf current size)
               (cons current #'next)))
      (cons current #'next))))

(defun make-generators (data)
  (let* ((size (length data))
         (disks (mapcar #'parse-disk data))
         (generators (mapcar (partial-1 #'apply #'position-generator) disks)))
    (make-array size :initial-contents generators)))

(defun minmax (gens)
  (loop :with min :with min-pos :with max :with max-pos
        :for (value . ignored) :across gens
        :for i :from 0
        :when (or (not min) (< value min)) :do (setf min value min-pos i)
        :when (or (not max) (> value max)) :do (setf max value max-pos i)
        :finally (return (cons min-pos max-pos))))

(defun solve (data)
  (loop :with gens = (make-generators data)
        :for (min . max) = (minmax gens)
        :if (= min max) :return (car (aref gens min))
        :else :do (setf (aref gens min) (funcall (cdr (aref gens min))))))

(defun prepare-part2 (data)
  (let* ((new-disk-id (1+ (length data)))
         (new-disk (format nil "Disc #~D has 11 positions; at time=0, it is at position 0." new-disk-id)))
    (append data (list new-disk))))

(define-problem (2016 15) (data)
    (solve data)
    (solve (prepare-part2 data))))

Advent of Code: [2016/16](https://adventofcode.com/2016/day/16)


- Pretty much do as instructed
- Generate random data using an expandable VECTOR...
- ...pre-load it with the input
- ...at each step append "0"
- ...then walk back to front, appending the opposite of the current element
- Calculate the checksum by _reducing_ adjacents elements, until the length is odd...
- ...do all this on the _same_ array, so we don't waste memory
(defun generate-random-data (seed size)
  (let ((array (make-array size
                           :element-type 'base-char
                           :adjustable t
                           :fill-pointer 0)))
    (loop for c across seed do (vector-push c array))
    (loop while (< (length array) size)
          for last = (1- (fill-pointer array)) do
          (vector-push #\0 array)
          (loop for i from last downto 0
                for c = (aref array i) do
                (vector-push (if (char= #\0 c) #\1 #\0) array)))

(defun checksum-reduction (array)
  (loop for i below (length array) by 2
        for j from 0
        for c1 = (aref array i)
        for c2 = (aref array (1+ i)) do
        (setf (aref array j) (if (char= c1 c2) #\1 #\0))
        finally (setf (fill-pointer array) (1+ j)))

(defun checksum (array)
  (prog1 array
    (loop do (checksum-reduction array)
          while (evenp (length array)))))

(define-problem (2016 16) (seed first)
    (checksum (generate-random-data seed 272))
    (checksum (generate-random-data seed 35651584))))

2020-11-20 (permalink)

Advent of Code: [2016/12](https://adventofcode.com/2016/day/12)


- Implement the 4 functions, one per each defined instruction (need to be careful to support value and register arguments)
- Parse the input into an array of functions + respective arguments
- Run the _program_ using a LOOP, `ip` starting from 0, and 4 registers...
- At each iteration, fetch the istruction at `ip` position, and execute it (each instruction returns the offset `ip` should be updated with)
- Stop when `ip` is bigger than the input program
(defvar *regs* nil)

(defun reg-name-index (reg)
  (- (char-code reg) (char-code #\a)))

(defun reg (name)
  (aref *regs* (reg-name-index name)))

(defun (setf reg) (value name)
  (setf (aref *regs* (reg-name-index name)) value))

(defun reg-or-value (x)
  (if (numberp x) x (reg x)))

(defun i-cpy (x reg)
  (prog1 1 (setf (reg reg) (reg-or-value x))))

(defun i-inc (reg) (prog1 1 (incf (reg reg))))

(defun i-dec (reg) (prog1 1 (decf (reg reg))))

(defun i-jnz (x offset)
  (if (zerop (reg-or-value x)) 1 offset))

(defun parse-value (string)
  (parse-integer string :junk-allowed t))

(defun parse-reg (string)
  (aref string 0))

(defun parse-reg-or-value (string)
  (or (parse-value string) (parse-reg string)))

(defun parse-cpy (string)
  (let ((parts (split-sequence:split-sequence #\Space string)))
    (when (string= (first parts) "cpy")
      (list #'i-cpy (parse-reg-or-value (second parts)) (parse-reg (third parts))))))

(defun parse-inc (string)
  (let ((parts (split-sequence:split-sequence #\Space string)))
    (when (string= (first parts) "inc")
      (list #'i-inc (parse-reg (second parts))))))

(defun parse-dec (string)
  (let ((parts (split-sequence:split-sequence #\Space string)))
    (when (string= (first parts) "dec")
      (list #'i-dec (parse-reg (second parts))))))

(defun parse-jnz (string)
  (let ((parts (split-sequence:split-sequence #\Space string)))
    (when (string= (first parts) "jnz")
      (list #'i-jnz (parse-reg-or-value (second parts)) (parse-value (third parts))))))

(defun parse-instruction (string)
  (or (parse-cpy string) (parse-inc string) (parse-dec string) (parse-jnz string)))

(defun parse-program (data)
  (map 'vector #'parse-instruction data))

(defun run (program regs)
  (let ((*regs* regs))
    (loop :with ip = 0
          :while (< ip (length program))
          :for (fun . args) = (aref program ip)
          :do (incf ip (apply fun args)))
    (reg #\a)))

(define-problem (2016 12) (program parse-program)
    (run program (make-array 4 :initial-contents (list 0 0 0 0)))
    (run program (make-array 4 :initial-contents (list 0 0 1 0)))))

Advent of Code: [2016/13](https://adventofcode.com/2016/day/13)


- Part 1: A*
- Part 2: BFS, with prune enabled after 50 steps
- State: position we are in (implemented as COMPLEX)
- Neighbor function: try up, right, down, left, if coordinates are not negative, and the they refer to an open space
(defvar *favorite-number* nil)

(defun magic (x y)
  (+ (* x x)
     (* 3 x)
     (* 2 x y)
     (* y y)))

(defun openp (x y)
  (loop :with number = (+ (magic x y) *favorite-number*)
        :for index :below (integer-length number)
        :count (logbitp index number) :into bits
        :finally (return (evenp bits))))

(defun neighbors (pos)
  (loop :for next :in (adjacents pos)
        :when (and (>= (realpart next) 0)
                   (>= (imagpart next) 0)
                   (openp (realpart next) (imagpart next)))
        :collect next))

(defun part1 (favorite-number end-state)
  (let ((*favorite-number* favorite-number))
    (nth-value 1 (a* #c(1 1) :goal-state end-state
                     :neighbors (search-unit-cost #'neighbors)
                     :heuristic (partial-1 #'manhattan-distance end-state)))))

(defun part2 (favorite-number)
  (let ((*favorite-number* favorite-number))
    (hash-table-count (nth-value 3 (bfs #c(1 1) :neighbors #'neighbors
                                        :prunep #'(lambda (state cost)
                                                   (declare (ignore state))
                                                   (> cost 50)))))))

(define-problem (2016 13) (data)
    (part1 (parse-integer (first data)) #c(31 39))
    (part2 (parse-integer (first data)))))

2020-11-19 (permalink)

? write a page about the keyboard mappings that I use

Changing the message `git` uses when merging branches...is not something you can easily [configure](https://stackoverflow.com/questions/3148863/how-can-i-customize-gits-merge-commit-message); your best bet, is to _override_ it completely.

Place the following somewhere in your `$PATH`
#!/usr/bin/env bash

set -eu

CURRENT_BRANCH=$(git currentbranch)

git merge --no-ff origin/$1 --edit -m "Merge branch '$REMOTE_BRANCH' into '$CURRENT_BRANCH'"
`chmod +x` it, and that's it:
$ git-merge-origin story/cx-532/neo4j-running-out-of-memory-when-taking-backup
Merge made by the 'recursive' strategy.
 system/ansible/roles/neo4j/templates/database-backup.sh.j2 | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
$ git l -n 1
b99eb03303 Merge branch 'story/cx-532/neo4j-running-out-of-memory-when-taking-backups' into 'master' (by Matteo Landi) (HEAD -> master, origin/master)

Advent of Code: [2016/11](https://adventofcode.com/2016/day/11)


- a state contains: the floor the elevator is at, the elements located at each floor
- from a given state, all the _neighbor_ ones can be generated by:
- ...moving the elevator up **or** down
- ...picking 1 **or** 2 items from the current floor (my input contains 5 generators and 5 microchips, so this _naive_ implementation should work just fine)
- ...confirming that no microchip would fry on the floor we are leaving, as well as the one we are getting into
- stop when all the items are at the 4th floor
- Part 2 is the same Part 1, with 4 additional items on the first floor -- and that will sloooooow things down

Easy right?!  Not really, implementing this took me way too long, so let's try to tackle this in steps.  First things first, let's define our _state_ object:

- Two properties, `elevator`, `floors`
- `elevator` is an integer
- `floors` is a LIST containing 4 numbers, bit-sets, each representing which element is currently located at that floor
- we have to deal with 10 items for part 1, and 14 for part 2, so bit-masks of 10/14 bits respectively will get the job done
- in particular, the least significant 5/7 bits will be used for microchips, while the rest for the generators
(defparameter *elements-mapping*
  '(("strontium" 1)
    ("plutonium" 2)
    ("thulium" 4)
    ("ruthenium" 8)
    ("curium" 16)
    ("elerium" 32)
    ("dilithium" 64)))

(defparameter *all-elements* nil)
(defparameter *all-elements-mask* nil)
(defparameter *microchips-mask* nil)

(defstruct (state :conc-name) elevator floors)

(defun floor-generators (floor)
  (ash floor (- *all-elements*)))

(defun floor-microchips (floor)
  (logand *microchips-mask* floor))

(defun parse-generators (string)
  (mapcar (lambda (each)
            (let ((name (first (split-sequence:split-sequence #\Space each))))
              (cadr (assoc name *elements-mapping* :test #'string=))))
          (cl-ppcre:all-matches-as-strings "\\w+ generator" string)))

(defun parse-microchips (string)
  (mapcar (lambda (each)
            (let ((name (first (split-sequence:split-sequence #\- each))))
              (cadr (assoc name *elements-mapping* :test #'string=))))
          (cl-ppcre:all-matches-as-strings "\\w+-compatible" string)))

(defun parse-floor (string)
  (let ((gens (loop :for each :in (parse-generators string)
                    :sum each))
        (ucs (loop :for each :in (parse-microchips string)
                   :sum each)))
    (+ (ash gens *all-elements*)

(defun parse-state (data)
  (make-state :elevator 0
              :floors (make-array 4 :initial-contents (loop :for string :in data
                                                            :collect (parse-floor string)))))
Previously we said that a state is _valid_ if no microchip at each floor is frying; and again, that translates to:

- No generators are on the given floor, or...
- For each microchip on the floor, its generator is there too
(defun nothing-fries-p (floor)
  (let ((generators (floor-generators floor))
        (microchips (floor-microchips floor)))
    (or (zerop generators)
        (= (logand microchips generators) microchips))))
Stop condition: all the items are at the fourth floor... or the fourth floor contains 10 items for part 1 and 14 items for part 2
(defun all-items-at-fourth-p (state)
  (let* ((floor (aref (floors state) 3)))
    (= floor *all-elements-mask*)))
Let's now try to generate all the neighbor states starting from a _known_ one; again, we said that:

- We can go up, or down
- We can pick one, or two items
- So long as nothing fries the floor that we are leaving
- and nothing fries at the floor we are entering into
(defun floor-without (floor index index2)
  (logandc2 floor (logior (ash 1 index)
                          (ash 1 index2))))

(defun floor-with (floor index index2)
  (logior floor (ash 1 index) (ash 1 index2)))

(defun change (floors floor-id floor floor2-id floor2)
  (let ((copy (copy-seq floors)))
    (setf (aref copy floor-id) floor
          (aref copy floor2-id) floor2)

(defun neighbors (state)
  (with-slots (elevator floors) state
    (loop :with floor = (aref floors elevator)
          :for index :below (integer-length floor)
          :when (logbitp index floor)
          :append (loop :for index2 :from index :below (integer-length floor) ;; index2 starts from index, that's how we try to move a single item
                        :when (and (logbitp index2 floor)
                                   (nothing-fries-p (floor-without floor index index2)))
                        :append (loop :for dir :in (list -1 1)
                                      :for elevator2 = (+ elevator dir)
                                      :for floor2 = (and (<= 0 elevator2 3) (aref floors elevator2))
                                      :when (and floor2 (nothing-fries-p (floor-with floor2 index index2)))
                                      :collect (make-state :elevator elevator2
                                                           :floors (change floors
                                                                           (floor-without floor index index2)
                                                                           (floor-with floor2 index index2))))))))
Let's add some final plumbing:
(defun solve (data &optional part2)
  (let* ((*all-elements* (if part2 7 5))
         (*all-elements-mask* (1- (ash 1 (* *all-elements* 2))))
         (*microchips-mask* (1- (ash 1 *all-elements*))))
    (nth-value 1 (bfs (parse-state data)
                      :test 'equalp
                      :goalp #'all-items-at-fourth-p
                      :neighbors #'neighbors))))

(defun prepare-part2 (data)
  (let ((copy (copy-seq data)))
    (setf (nth 0 copy) (format nil "~A ~{~A ~}" (nth 0 copy)
                               (list "elerium generator"
                                     "elerium-compatible microchip"
                                     "dilithium generator"
                                     "dilithium-compatible microchip")))

(define-problem (2016 11) (data)
    (solve data)
    (solve (prepare-part2 data) t)))
It works...
[SBCL] AOC/2016/11> (time (problem-run))
Evaluation took:
  50.306 seconds of real time
  43.853553 seconds of total run time (40.663221 user, 3.190332 system)
  [ Run times consist of 6.507 seconds GC time, and 37.347 seconds non-GC time. ]
  87.17% CPU
  115,699,129,876 processor cycles
  7,316,583,024 bytes consed

...a bit slow, but it works.  Well, that does not surprise me, as NEIGHBORS is very dumb in what it does -- it blindly moves items around, without trying to immediately prune sub-optimal steps.

Anyway, I am fine with this...for now!

2020-11-16 (permalink)

[Codeflows 2020](https://codeflows.io/) qualification round: post-mortem

It was a total disaster: I did not manage to _fully_ finish any of the 6 problems, and ended up scoring 40 points only (I believe 400 points were required to make it to the second round).  I knew I sucked at programming competitions... I just did not know how **badly** I did.

Too many mistakes from my part:

- I took this too lightly: "you have more than 24h to complete this, but we don't anticipate it will take you more than 2 hours" said the email that I received, and since the town where I am living was about to pull another lock-down due to the increased number of COVID cases, I figured I could spend a few more hours outside and start working on this later...in the end, it's just a qualification round, right?!
- I had never used CodeChef before (I did not even have an account when I read the first problem's statement), and it took me a bit of time to get myself acquainted with the platform: _read-from-stdin-and-write-to-stdout_ vs _implement-this-function-that-accepts-these-arguments-and-returns-this-value_ (a la LeetCode), inability to debug failures, _funky_ error messages (see later), lack of commonly available libraries like `SPLIT-SEQUENCE`
- I spent almost an hour trying to figure out why one of my solutions was throwing a [NZEC](https://discuss.codechef.com/t/why-do-i-get-an-nzec/1775), and as am I am writing this, I still have no clue whatsoever of what did I do wrong: I changed my parsers to account for trailing whitespaces in the input (someone suggested it might happen), I changed my input parsing logic to minimize consing, I tried forcing the garbage collector in between runs (I remember I had to do or the Google Code Jam grader would not accept my solution)...but none of these seems to have made any difference
- Kept _switching_ problem the moment I realized the solution I was getting at was not as _straightforward_ as I initially anticipated; this is not new with me (i.e. "it cannot be that complicated, there has to be an easier way..."), and between the clock ticking and the lack of solved problems, I suspect things got even worse as I desperately looked for a low hanging fruit problem with which to build up some confidence
- I was not white-boarding (e.g. pen and paper, Gimp) as much as I should have: "oh, this is easy", I kept saying, and then immediately switched to the editor to hack something up...when clearly I should have spent more time _validating_ the solution, thinking about edge-cases, input constraints, etc.

Let's take a look at the problems, so you can gauge their complexity: the first one, had something to do with an array of integers, and commands that operate over it:

- Increase the values of a given range L-R, by X
- Sum values stored in odd positions of the array
- Sum values stored in even positions of the array

Easy right? Apparently not, as the grader keep on throwing NZECs at me.
(defun read-line-or-eof ()
  (read-line nil nil :eof))

(defun read-integers (n)
  (loop :repeat n
        :collect (read)))

(defun read-heights ()
  (let* ((n (read))
         (heights (read-integers n)))
    (make-array n :initial-contents heights)))

(defun run ()
  (loop :with heights = (read-heights)
        :with q = (parse-integer (read-line-or-eof))
        :repeat q
        :for id = (read)
        :do (cond ((= id 1) (let ((l (read)) (r (read)) (x (read)))
                              (loop :for i :from (1- l) :below r
                                    :do (incf (aref heights i) x))))
                  (t (loop :for i = (- id 2) :then (+ i 2)
                           :while (< i (length heights))
                           :sum (aref heights i) :into sum
                           :finally (format t "~D~%" sum)))))) ;; out of memory exceptions?!
In retrospect, one could have implemented this way more efficiently by simply keeping track of the odd/even buckets, and keep them up to date when tasked to increase the values in a given range, but shouldn't the grader have flagged my solution with a different error message in case it run out of memory, or simply took too much time to complete?  I guess we will never know...

With the second problem, you were given an array containing 0s, 1s, 2s, and were asked to find the maximum number of partitions so that the sum of the numbers in each partition would be divisible by 3.  The realization that I had about this problem was that:

- It takes a 0 to make a partition that is divisible by 3
- It takes a 1 and a 2 to make a partition that is divisible by 3
- It takes three 1s to make a partition that is divisible by 3
- It takes three 2s to make a partition that is divisible by 3

I came up with the following recursive solution, but the grader did not seem to like it (another NZEC error):
(defun read-integers (n)
  (loop :repeat n
        :collect (read)))

(defun solve (s0 s1 s2)
  (labels ((recur (s1 s2)
             (let ((best 0))
               (when (>= s1 3)
                 (setf best (max (1+ (recur (- s1 3) s2)) best)))
               (when (and (>= s1 1) (>= s2 1))
                 (setf best (max (1+ (recur (1- s1) (1- s2))) best)))
               (when (>= s2 3)
                 (setf best (max (1+ (recur s1 (- s2 3))) best)))
    (+ s0 (recur s1 s2))))

(defun run ()
  (loop :with n = (read)
        :repeat n
        :for (s0 s1 s2) = (read-integers 3)
        :do (format t "~D~%" (solve s0 s1 s2))))
I am not sure what the grader was complaining about, but I can only speculate that with big values of `s1` or `s2` my recursive solution would very likely exhaust the stack space.

Do we need to solve this iteratively/recursively, or can we solve this mathematically?

- Under the assumption that the fewer elements we use, to create a group, the better, when you have a chance we should create groups with one 1 and one 2 first
- How many of these groups can we create? `min(s1, s2)`
- With the remaining 1s, `s1 - min(s1, s2)`, we can create groups using three 1s each
- With the remaining 2s, `s2 - min(s1, s2)`, we can create groups using three 2s each

This led me to this other more _efficient_ solution:
(defun solve (s0 s1 s2)
  (let ((min-s1s2 (min s1 s2)))
    (+ s0 min-s1s2 (truncate (- s1 min-s1s2) 3) (truncate (- s2 min-s1s2)))))
Unfortunately the grader did not like this either -- this time, the error message was WA (i.e. wrong answer).  I figured my approach was flawed, so I gave up and moved to the next problem.

Before we do that though, did you spot the _silly_ mistake that I had made? I forgot to pass `3` to the second call to `TRUNCATE`, and unfortunately none of my local tests did catch this...
(defun solve (s0 s1 s2)
  (let ((min-s1s2 (min s1 s2)))
    (+ s0 min-s1s2 (truncate (- s1 min-s1s2) 3) (truncate (- s2 min-s1s2) 3))))
Would the grader like this solution? Hard to say...

The third problem felt more of a mathematical problem, than a programming one: given an array of integers `[A1, A2, ... AN]`, you had to find an array `[B1, B2, ... BN]` such that:

- Each `Bi` is smaller than a very big number
- Each `Ai` divides `Bi`
- Each `Bi` divides `Bi-1 x Bi + 1` (imagine the array is circular, so `Bn+1` is `B1`)

I thought I could manually _solve_ these equations and find the relation between these different constants, but even though I managed to _simplify_ some of these, I did not know how to search for the _correct_ set of values amongst the _possible_ ones, so I gave up on this pretty quickly too.

The fourth problem was about finding two partitions of a binary string (all "0"s and "1"s) such that:

- The first one, is a palindrome (i.e. `s = rev(s)`)
- The second one, is an _antipalindrome_ (i.e. `s = rev(flip(s))`)

I could not immediately _see_ how to solve this, so I tagged this as _too-complicated-for-now_, and moved on...

Fifth problem: you are given the implementation of an inefficient function that, recursively, calculates the **bitwise OR** of the elements of a given array (see later); your task was to permute the input and minimize the total number of function calls.
int calculate_or (int i) {
    if (i == N) return A[i];
    if (calculate_or(i+1) == A[i]) return A[i];
    return calculate_or(i+1) | A[i];
The function calculates the result starting from the tail of the array and _slowly_ moving to the first element; in particular, the _extra_ recursive call is avoided if `calculate_or(i+1) == A[i]`, so if we want to minimize the number of recursive calls, we would have to arrange elements so that the ones with wider 1s mask come firs.

Anyway, since I had to return the minimum number of function calls, I figured I would start with that:
(defun count-fcalls (array &aux (n (length array)))
  (let ((suffix-or (make-array n :initial-contents (loop :for i :from (1- n) :downto 0
                                                         :for value = (aref array i)
                                                         :for each = value :then (logior each value)
                                                         :collect each))))
    (1+ (loop :for i :from 1 :below n
           :for each = (aref array (1- i))
           :for remaining-or = (aref suffix-or i)
           :for remaining-calls = (if (= each remaining-or) 1 2)
           :for acc = remaining-calls :then (* acc remaining-calls)
           :sum acc))))
Next: how could we _smartly_ shuffle the input array to minimize the number of recursive calls?  The first solution that I tried, was to sort elements by the number of raised bits:
(defun num-raised-bits (number)
  (loop :for i :below (integer-length number)
        :count (logbitp i number)))

(defun run ()
  (loop :with n = (read)
        :repeat n
        :for array = (read-integers (read))
        :for sorted = (sort array #'> array :key #'num-raised-bits)
        :do (format t "~D~%" (count-fcalls sorted))))
The grader did not like it.  What about sorting by most-significant bit first, and then by the number of raised bits?
(defun comes-first (n m)
  (let ((len-n (integer-length n))
        (len-m (integer-length m)))
    (or (> len-n len-m)
        (and (= len-n len-m)
             (let ((raised-n (num-raised-bits n))
                   (raised-m (num-raised-bits m)))
               (> raised-n raised-m))))))

(defun run ()
  (loop :with n = (read)
        :repeat n
        :for array = (read-array)
        :for sorted = (sort array #'comes-first)
        :do (format t "~D~%" (count-fcalls sorted))))
I am not sure why I thought this one; was I high or something? Anyway, of course the grader did not accept this too.  Lastly, I came up with the idea of solving this with DFS + memoization, but I could not finish this in time:

- Implement a recursive function to return the minimum number of calls required to calculate the bitwise OR (i.e. the solution to the problem), as well as bitwise OR mask
- I used a bitset to keep track of which element of the input array still had to be processed
- Base case: if a call is made with the same `remaining` bit-mask, return the cached value (memoization)
- Base case: when all the elements have been visited, return `(0 0)` -- 0 remaining function calls, 0 is the bitwise OR for the remaining items (none in this case)
- At each step, try to add one of the elements which are not part of our solution already, and see which of these additions minimize the number of recursive calls
- How? Each recursive calls returns the number of remaining function calls, `fcalls`, as well as the bit-mask of all the remaining elements, `or-suffix`; if the currently tried element is equal to `or-suffix` than we would only call the recursive function once, otherwise twice; this means that adding the currently tried element will require `fcalls` or `2 * fcalls` total function calls respectively
(defun solve (array &aux)
  (let* ((n (length array))
         (all (1- (ash 1 n)))
         (memo (make-hash-table)))
    (labels ((mget (key)
               (gethash key memo))
             (mset (key value)
               (setf (gethash key memo) value))
             (dfs (remaining &aux (cached (mget remaining)))
               (cond (cached cached)
                     ((zerop remaining) (mset remaining (list 0 0)))
                     (t (let (fcalls-best or-suffix-best)
                          (loop :for i :below n
                                :for each = (aref array i)
                                :when (logbitp i remaining)
                                :do (destructuring-bind (fcalls-rest or-suffix-rest)
                                        (dfs (logxor (ash 1 i) remaining))
                                      (let* ((no-extra-call-required (= or-suffix-rest each))
                                             (fcalls-curr (1+ (* (if no-extra-call-required 1 2) fcalls-rest))))
                                        (when (or (not fcalls-best)
                                                  (< fcalls-curr fcalls-best))
                                          (setf fcalls-best fcalls-curr
                                                or-suffix-best (logior each or-suffix-rest))))))
                          (mset remaining (list fcalls-best or-suffix-best)))))))
      (first (dfs all)))))

(defun run ()
  (loop :with n = (read)
        :repeat n
        :for array = (read-array)
        :do (format t "~D~%" (solve array))))
This seems to return the **right** value -- for the inputs I tried this with, at least -- however, it took more than a second to output the answer for `#(1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 29)`, so I hardly believe the grader would accept this...

Anyway, the sixth problem? It had something to do with doing some mathematical calculations on a given input array, but I was immediately put off by the double multiplication at the beginning of the problem description, so ended up not spending much time on this -- I bet this was the easy problem I was desperately looking for...

And this was the last one...what an epic fail ;-)

So much for a _reality check_, uh?!

2020-11-14 (permalink)

I was having a hard-time figuring out the **name** of one of the running applications on my Mac -- I wanted to create a [shortcut](https://github.com/iamFIREcracker/dotfiles/blob/c46cc374d2053d47a0f6664650786f3af7783fd9/.hammerspoon/init.lua#L35) for it, using Hammerspoon -- and it turns the project's [FAQ](https://www.hammerspoon.org/faq/) page had exactly an entry for people facing this problem:

If you are trying to find Microsoft Office applications (e.g. using hs.appfinder.appFromName("Word")), they present a different application name to the OS than they display in the system menu bar. Use "Microsoft Word", "Microsoft Excel", etc. instead.

You can get a list of the real names of all running applications in the Hammerspoon Console with the following snippet:

hs.fnutils.each(hs.application.runningApplications(), function(app) print(app:title()) end)

2020-11-13 (permalink)

Advent of Code: [2016/10](https://adventofcode.com/2016/day/10)


- Parse input and return a hash table containing all the _bots_
- A _bot_ is a struct containing `id`, `logic` (i.e. low-value-microchip goes to bot/output, high-value-microchip goes to bot/output), `microchips` (i.e. list of microchips in the bot's hands)
- Find the first bot which is _unblocked_ (i.e. has 2 microchips)
- Move microchips to other bots/outputs
- Repeat until there are no more bots which are unblocked
(defstruct bot id microchips logic)

(defun bot-microchips-sorted (bot)
  (destructuring-bind (a b) (bot-microchips bot)
    (if (< a b) (list a b) (list b a))))

(defun parse-init-instruction (string)
  (cl-ppcre:register-groups-bind ((#'parse-integer value bot-id))
      ("value (\\d+) goes to bot (\\d+)" string)
    (list value bot-id)))

(defun parse-logic-instruction (string)
  (cl-ppcre:register-groups-bind ((#'parse-integer bot-id)
                                  (#'parse-integer low-to-id)
                                  (#'parse-integer high-to-id))
      ("bot (\\d+) gives low to (bot|output) (\\d+) and high to (bot|output) (\\d+)" string)
    (list bot-id
          (string= low-to-type "bot") low-to-id
          (string= high-to-type "bot") high-to-id)))

(defun parse-bots (data)
  (let ((bots (make-hash-table)))
    (flet ((get-or-create (bot-id)
             (or (gethash bot-id bots)
                 (setf (gethash bot-id bots) (make-bot :id bot-id)))))
      (dolist (string data bots)
        (let (match)
          (cond ((setf match (parse-init-instruction string))
                 (destructuring-bind (value bot-id) match
                   (let ((bot (get-or-create bot-id)))
                     (push value (bot-microchips bot)))))
                ((setf match (parse-logic-instruction string))
                 (let ((bot (get-or-create (first match))))
                   (setf (bot-logic bot) (rest match))))))))))

(defun find-unblocked-bot (bots)
  (loop :for each :being :the :hash-values :of bots
        :for microchips = (bot-microchips each)
        :when (= (length microchips) 2) :return each))

(define-problem (2016 10) (bots parse-bots)
  (loop :with part1 :with outputs = (make-hash-table)
        :for bot = (find-unblocked-bot bots)
        :while bot
        :for (low high) = (bot-microchips-sorted bot)
        :when (and (= high 61) (= low 17)) :do (setf part1 (bot-id bot))
        :do (destructuring-bind (low-to-bot-p low-to-id high-to-bot-p high-to-id)
                (bot-logic bot)
              (if low-to-bot-p
                (push low (bot-microchips (gethash low-to-id bots)))
                (setf (gethash low-to-id outputs) low))
              (if high-to-bot-p
                (push high (bot-microchips (gethash high-to-id bots)))
                (setf (gethash high-to-id outputs) high))
              (setf (bot-microchips bot) nil))
        :finally (return (values part1
                                 (* (gethash 0 outputs)
                                    (gethash 1 outputs)
                                    (gethash 2 outputs))))))

2020-11-12 (permalink)

+ lispindent breaks with cl-ppre group bindings form like `((#'intgeer first second))`

Interesting Rework podcast about PTO policies:

Unlimited paid time off is a common perk in the tech industry, but as one company discovered, an open-ended vacation policy led to confusion and even burnout. Dan Jimenez of Chatbooks comes on Rework to talk about how they shifted from unlimited to mandatory PTO, and how they're recalibrating expectations for work, productivity, and rest during a turbulent time.

Link: [Take Some Time Off (We Mean it!)](https://share.transistor.fm/s/0000e516)

Vim/vim-sexp were acting _weirdly_ when working on a file that contained the following form:
(let* ((match-size (1+ (position #\) string)))
It turns out the plugin was tripping over the `#\)` literal; I could have used `#\right_parenthesis` instead, but unfortunately that's not a portable solution.

So I asked this on the Lisp Discord server, and one suggestion that came in was:

you can use `(char ")" 0)` as a workaround, optionally armed with a `#.` in non-evaluated environments

That's clever, I had not thought about using the compile-time evaluation syntax for this.

Anyway, if you are wondering: yes, that did actually fix my problem, and now Vim is not behaving weirdly anymore -- I am still on the fence whether this looks more readable or not.

2020-11-11 (permalink)

Advent of Code: [2016/08](https://adventofcode.com/2016/day/8)


- Parse instructions
- Implement the screen as a 2d array, and implement _drawing_ functions not caring much about performance (rotate ones in particular, could be implemented without allocating an extra array for the column/row, but I was too lazy to implement that)
- Run all the drawing instructions.  At the end
- Part 1: count lit pixels
- Part 2: _print_ the screen, and _visually_ OCR it :D
(defun parse-draw-rect (string)
  (cl-ppcre:register-groups-bind ((#'parse-integer width height))
      ("rect (\\d+)x(\\d+)" string)
    (list :draw-rect height width)))

(defun parse-rotate-row (string)
  (cl-ppcre:register-groups-bind ((#'parse-integer i shift))
      ("rotate row y=(\\d+) by (\\d+)" string)
    (list :rotate-row i shift)))

(defun parse-rotate-column (string)
  (cl-ppcre:register-groups-bind ((#'parse-integer j shift))
      ("rotate column x=(\\d+) by (\\d+)" string)
    (list :rotate-column j shift)))

(defun parse-single-instruction (string)
  (or (parse-draw-rect string)
      (parse-rotate-row string)
      (parse-rotate-column string)))

(defun parse-instructions (data)
  (mapcar #'parse-single-instruction data))

(defun draw-rect (screen height width)
  (dotimes (i height)
    (dotimes (j width)
      (setf (aref screen i j) #\#))))

(defun rotate-row (screen i shift &aux (n (array-dimension screen 1)))
  (let ((new-values (make-array n :element-type 'character)))
    (loop :for j :below n
          :for k = (mod (+ j shift) n)
          :do (setf (aref new-values k) (aref screen i j)))
    (loop :for v :across new-values
          :for j :from 0
          :do (setf (aref screen i j) v))))

(defun rotate-column (screen j shift &aux (n (array-dimension screen 0)))
  (let ((new-values (make-array n :element-type 'character)))
    (loop :for i :below n
          :for k = (mod (+ i shift) n)
          :do (setf (aref new-values k) (aref screen i j)))
    (loop :for v :across new-values
          :for i :from 0
          :do (setf (aref screen i j) v))))

(defun count-lit-pixels (screen)
  (loop :for i :below (array-total-size screen)
        :count (eql (row-major-aref screen i) #\#)))

(defun print-screen (screen)
  (with-output-to-string (s)
    (terpri s) ; print an additional newline, so I can better format the expected string
    (let ((height (array-dimension screen 0))
          (width (array-dimension screen 1)))
      (dotimes (i height)
        (dotimes (j width)
          (princ (aref screen i j) s))
        (terpri s)))))

(define-problem (2016 08) (instructions parse-instructions)
  (loop :with screen = (make-array (list 6 50) :initial-element #\Space)
        :for (type . args) :in instructions
        :do (case type
              (:draw-rect (apply #'draw-rect screen args))
              (:rotate-row (apply #'rotate-row screen args))
              (:rotate-column (apply #'rotate-column screen args)))
        :finally (return (values (count-lit-pixels screen)
                                 (print-screen screen)))))

Advent of Code: [2016/09](https://adventofcode.com/2016/day/9)


- Part 1: reconstruct the uncompressed message, implementing a recursive descent parser
- when parsing a marker means, one should extract the `size` of the repetition, and the number of `times` the message should be repated -- reconstruct that, and move on
- when parsing _everything else_, simply process until the next marker
(defun parse-message (string)
  (or (parse-marker string)
      (parse-rest string)))

(defun parse-marker (string)
  (cl-ppcre:register-groups-bind ((#'parse-integer size times))
      ("^\\((\\d+)x(\\d+)\\)" string)
    (let* ((match-size (1+ (position #\) string)))
           (seed (subseq string match-size (+ match-size size))))
      (concatenate 'string
                   (format nil "~V@{~a~:*~}" times seed) ;; meh
                   (parse-message (subseq string (+ match-size size)))))))

(defun parse-rest (string)
  (cl-ppcre:register-groups-bind (raw)
      ("^([^(]+)" string)
    (concatenate 'string raw (parse-marker (subseq string (length raw))))))
- Part 2: as the problem statement suggests, a _naive_ solution would not work with this new version of the format
- The intuition is that we don't really care about the message... we only care about its length
- Recursive solution
- The size a compressed portion (i.e. one starting with a marker), is equal the size (i.e. recursive call) of the message after the marker that needs to be repeated, _times_ the number of times that needs to be repeated
- The size of an uncompressed portion is... well the size of that portion
(defun message-length (string)
  (labels ((compressed (string)
             (cl-ppcre:register-groups-bind ((#'parse-integer size times))
                 ("^\\((\\d+)x(\\d+)\\)" string)
               (let ((match-size (1+ (position #\) string))))
                 (+ (* (message-length (subseq string match-size (+ match-size size))) times)
                    (message-length (subseq string (+ match-size size)))))))
           (uncompressed (string)
             (cl-ppcre:register-groups-bind (raw)
                 ("^([^(]+)" string)
               (+ (length raw) (message-length (subseq string (length raw)))))))
    (or (compressed string) (uncompressed string) 0)))
- This works, and we can easily adapt this solution to work for Part 1 too
- The difference is that for part one, we don't _recurse_ when calculating the length of a compressed area
(defparameter *version* 1)

(defun message-length (string)
  (or (message-compressed-length string)
      (message-uncompressed-length string)

(defun message-compressed-length (string)
  (cl-ppcre:register-groups-bind ((#'parse-integer size times))
      ("^\\((\\d+)x(\\d+)\\)" string)
    (let* ((match-size (1+ (position #\) string)))
           (size-uncompressed (if (= *version* 1)
                                (message-length (subseq string match-size (+ match-size size))))))
      (+ (* size-uncompressed times)
         (message-length (subseq string (+ match-size size)))))))

(defun message-uncompressed-length (string)
  (cl-ppcre:register-groups-bind (raw)
      ("^([^(]+)" string)
    (+ (length raw) (message-length (subseq string (length raw))))))

(define-problem (2016 09) (message first)
  (values (message-length message)
          (let ((*version* 2))
            (message-length message))))

2020-11-10 (permalink)

Advent of Code: [2016/05](https://adventofcode.com/2016/day/4)


- Brute-force it
- Let someone else deal with [md5](https://quickref.common-lisp.net/md5.html)
- Check if a hash is _good_ (first 5 characters of the string representation are "0"), then accumulate...
- Part 1: accumulate the 6th character
- Part 2: if 6th character is below "8", write the 7th character on the _right_ spot
- PS: this solution is slow... on my laptop it takes around 40 seconds to run
(defun goodp (md5-hash)
  (and (zerop (aref md5-hash 0))
       (zerop (aref md5-hash 1))
       (zerop (ash (aref md5-hash 2) -4))))

(defun int-hex (n)
  (write-to-string n :base 16))

(defun destruct-hash (md5-hash)
  (let ((valid (goodp md5-hash))
        (part1-digit (aref (string-downcase (int-hex (aref md5-hash 2))) 0))
        (part2-offset (aref md5-hash 2))
        (part2-digit (aref (string-downcase (int-hex (ash (aref md5-hash 3) -4))) 0)))
    (list valid part1-digit part2-offset part2-digit)))

(define-problem (2016 05) (door-id first)
  (loop :with part1 = (make-array 8 :element-type 'character :adjustable t :fill-pointer 0)
        :with part2 = (make-string 8 :initial-element #\_)
        :with remaining = 8
        :for index :from 0
        :for md5-hash = (md5:md5sum-string (mkstr door-id index))
        :for (valid part1-digit part2-offset part2-digit) = (destruct-hash md5-hash)
        :when valid :do (progn
                          (when (< (fill-pointer part1) 8)
                            (vector-push part1-digit part1))
                          (when (and (< part2-offset 8)
                                     (eql (aref part2 part2-offset) #\_))
                            (setf (aref part2 part2-offset) part2-digit
                                  remaining (1- remaining))))
        :when (zerop remaining) :return (values part1 part2)))

Advent of Code: [2016/06](https://adventofcode.com/2016/day/6)


- Parse input
- Part 1: iterate by columns, run `FREQUENCIES`, and return the most frequent char
- Part 2: use the least frequent char instead
(defun parse-recording (data)
  (make-array (list (length data) (length (first data)))
              :initial-contents data))

(defun array-column (array j)
  (loop :for i :below (array-dimension array 0)
        :collect (aref array i j) :into values
        :finally (return (coerce values 'simple-vector))))

(define-problem (2016 06) (recording parse-recording)
  (loop :with columns = (array-dimension recording 1)
        :for j :below columns
        :for column = (array-column recording j)
        :for freqs = (sort (hash-table-alist (frequencies column)) #'> :key #'cdr)
        :collect (car (first freqs)) :into part1
        :collect (caar (last freqs)) :into part2
        :finally (return (values (coerce part1 'string)
                                 (coerce part2 'string)))))

Advent of Code: [2016/07](https://adventofcode.com/2016/day/7)


- Parse IPs into splits, where the even ones represent the hypernet sequences
- Part 1
- ... iterate each split (odd splits **might** contain an ABBA, even ones **should not**)
- ... if it **does** contains an ABBA in an odd split, and **not** contain an ABBA in an even one, then count the entry
(defun parse-ip (string)
  (->< string
       (split-sequence:split-sequence #\[ ><)
       (mapcan (lambda (s) (split-sequence:split-sequence #\] s)) ><)))

(defun parse-ips (data)
  (mapcar #'parse-ip data))

(defun contains-abba-p (string)
  (loop :for i :from 3 :below (length string)
        :for a = (aref string (- i 3))
        :for b = (aref string (- i 2))
        :for c = (aref string (- i 1))
        :for d = (aref string (- i 0))
        :when (and (char= a d) (char= b c) (char/= a b)) :return t))

(defun supports-tls-p (ip)
  (loop :with found-one-abba
        :for part :in ip
        :for hypernetp = nil :then (not hypernetp)
        :if (and hypernetp (contains-abba-p part)) :return nil
        :else :do (unless found-one-abba
                    (setf found-one-abba (contains-abba-p part)))
        :finally (return found-one-abba)))

(define-problem (2016 07) (ips parse-ips)
  (loop :for ip :in ips
        :count (supports-tls-p ip) :into part1
        :finally (return (values part1 part2))))
- Part 2
- ... it's a bit tricker, because there could be multiple ABA inside odd splits, but only one of them might have a BAB in one of the hypernet sequences
- For each ABA inside any of the odd splits, check if the same BAB exist inside any of the hypernet sequences
(defun extract-abas (string)
  (loop :for i :from 2 :below (length string)
        :for a = (aref string (- i 2))
        :for b = (aref string (- i 1))
        :for c = (aref string (- i 0))
        :when (and (char= a c) (char/= a b)) :collect (mkstr a b c)))

(defun aba-bab (aba)
  (mkstr (aref aba 1) (aref aba 0) (aref aba 1)))

(defun contains-bab-p (bab hypernet)
  (some (partial-1 #'search bab _) hypernet))

(defun supports-ssl-p (ip)
  (destructuring-bind (normal hypernet)
      (loop :for split :in ip
            :for i :from 1
            :if (oddp i) :collect split :into normal
            :else :collect split :into hypernet
            :finally (return (list normal hypernet)))
    (loop :for split :in normal
          :for babs = (mapcar #'aba-bab (extract-abas split))
          :when (some (partial-1  #'contains-bab-p _ hypernet) babs) :return t)))

(define-problem (2016 07) (ips parse-ips)
  (loop :for ip :in ips
        :count (supports-tls-p ip) :into part1
        :count (supports-ssl-p ip) :into part2
        :finally (return (values part1 part2))))
- This works, but I think we can try and simplify the implementation a bit
- Let's **always** partition splits into _normal_, and _hypernet_ (not just for part 2)
- Part 1: count IPs with an ABBA inside the _normal_ part, and **not** an ABBA inside the _hypernet_ part
- Part 2: count IPs having at least one ABA part inside the _normal_ part, and a complementary BAB inside the _hypernet_ part
(defun parse-ip-parts (string)
  (->< string
       (split-sequence:split-sequence #\[ ><)
       (mapcan (lambda (s) (split-sequence:split-sequence #\] s)) ><)))

(defun parse-ip (string)
  (loop :for split :in (parse-ip-parts string)
        :for i :from 1
        :if (oddp i) :collect split :into normal
        :else :collect split :into hypernet
        :finally (return (list normal hypernet))))

(defun parse-ips (data)
  (mapcar #'parse-ip data))

(defun contains-abba-p (string)
  (loop :for i :from 3 :below (length string)
        :for a = (aref string (- i 3))
        :for b = (aref string (- i 2))
        :for c = (aref string (- i 1))
        :for d = (aref string (- i 0))
        :when (and (char= a d) (char= b c) (char/= a b)) :return t))

(defun supports-tls-p (normal hypernet)
  (and (some #'contains-abba-p normal)
       (notevery #'contains-abba-p hypernet)))

(defun extract-babs (string)
  (loop :for i :from 2 :below (length string)
        :for a = (aref string (- i 2))
        :for b = (aref string (- i 1))
        :for c = (aref string (- i 0))
        :when (and (char= a c) (char/= a b)) :collect (mkstr b a b)))

(defun contains-bab-p (hypernet)
  (lambda (bab)
    (some (lambda (split) (search bab split)) hypernet)))

(defun supports-ssl-p (normal hypernet)
  (loop :for each :in normal
        :for babs = (extract-babs each)
        :when (some (contains-bab-p hypernet) babs) :return t))

(define-problem (2016 07) (ips parse-ips)
  (loop :for (normal hypernet) :in ips
        :count (supports-tls-p normal hypernet) :into part1
        :count (supports-ssl-p normal hypernet) :into part2
        :finally (return (values part1 part2))))

2020-11-09 (permalink)

Advent of Code: [2016/04](https://adventofcode.com/2016/day/4)


- Parse room specs
- Part 1: calculate the frequency of each character in the encrypted name (dashes excluded)
- ...sort frequency tuples by most frequent first (use lexicographical order to break tie)
- ...find the entries for which the order of the most frequent characters matches the letters defined inside the checksum
- ...sum their IDs
- Part 2: [Caesar cipher](https://rosettacode.org/wiki/Caesar_cipher)
(defun read-single-room-specification (string)
  (cl-ppcre:register-groups-bind (name (#'parse-integer sector) checksum)
      ("([a-z\\-]+)-(\\d+)\\[([a-z-]+)\]" string)
    (list name sector checksum)))

(defun read-specifications (data)
  (mapcar #'read-single-room-specification data))

(defun validp (freqs checksum)
  (setf freqs (sort freqs (lambda (e1 e2)
                            (destructuring-bind ((c1 . f1) (c2 . f2))
                                (list e1 e2)
                              (or (> f1 f2)
                                  (and (= f1 f2)
                                       (char< c1 c2)))))))
  (and (<= (length checksum) (length freqs))
       (loop :for e :across checksum
             :for (c) :in freqs
             :always (eql e c))))

(defun decipher-char (c offset &aux (base (char-code #\a)))
  (->< (char-code c)
    (- >< base)
    (+ offset)
    (mod >< 26)
    (+ base)

(defun caesar-decipher (string offset)
  (map 'string #'(lambda (c)
                   (if (eql c #\-) #\Space (decipher-char c offset)))

(define-problem (2016 04) (specs read-specifications)
  (loop :with part2
        :for (name sector checksum) :in specs
        :for freqs = (hash-table-alist (frequencies (remove #\- name)))
        :for deciphered = (caesar-decipher name sector)
        :when (validp freqs checksum) :sum sector :into part1
        :when (and (not part2) (search "northpole" deciphered)) :do (setf part2 sector)

2020-11-08 (permalink)

Advent of Code: [2016/01](https://adventofcode.com/2016/day/1)


- Nothing crazy: start from `#c(0 0)`, heading `#c(0 1)`, and apply each instruction one after the other
- A left rotation is implemented by multiplying `heading` by `#c(0 1)`; a right rotation is implemented by multiplying `heading` by `#c(0 -1)`
- Part 1: apply all the movements, and finally apply `(MANHATTAN-DISTANCE POS 0)`
- Part 2: apply all the movements, _one step at a time_, and when you land on an already seen position, return `(MANHATTAN-DISTANCE DUPLICATE 0)`
(defun read-instructions (data)
  (mapcar (lambda (step)
            (let ((rotation (if (eql (aref step 0) #\L) #c(0 1) #c(0 -1)))
                  (walk-for (parse-integer (subseq step 1) :junk-allowed t)))
              (cons rotation walk-for)))
          (split-sequence:split-sequence #\Space (first data))))

(define-problem (2016 01) (instructions read-instructions)
    :with pos = #c(0 0) :with part2 :with seen
    :for (change-dir . walk-for) :in instructions
    :for heading = (* #c(0 1) change-dir) :then (* heading change-dir)
    :do (loop
          :repeat walk-for
          :do (progn
                (incf pos heading)
                (when (and (not part2) (member pos seen))
                  (setf part2 pos))
                (push pos seen)))
    :finally (return (values (manhattan-distance pos 0)
                             (manhattan-distance part2 0)))))

Advent of Code: [2016/02](https://adventofcode.com/2016/day/2)


- Create a 2d array representing the keypad, and start from `i = 1`, and `j = 1`
- Move the current position until its out of bounds
- Collect the value on the keypad at the end of each single-key instruction
- Part 1: use the standard 3x3 keypad
- Part 2: use the funky shaped keypad, still represented as a 2d array, with some cells marked as _invalid_
(defparameter *keypad-part1* (make-array '(3 3) :initial-contents '("123"

(defparameter *keypad-part2* (make-array '(5 5) :initial-contents '("--1--"

(defun read-single-key-instructions (string)
  (map 'list (lambda (c)
               (ecase c
                 (#\U '(-1 . 0))
                 (#\R '(0 . 1))
                 (#\D '(1 . 0))
                 (#\L '(0 . -1))))

(defun read-instructions (data)
  (mapcar #'read-single-key-instructions data))

(defun find-code (instructions keypad i j)
    :for single-key-instructions :in instructions
    :do (loop
          :for (di . dj) :in single-key-instructions
          :for ni = (+ i di) :for nj = (+ j dj)
          :when (and (array-in-bounds-p keypad ni nj)
                     (not (eql (aref keypad ni nj) #\-)))
          :do (setf i ni
                    j nj))
    :collect (aref keypad i j) :into numbers
    :finally (return (format nil "~{~a~}" numbers))))

(define-problem (2016 02) (instructions read-instructions)
  (values (find-code instructions *keypad-part1* 1 1)
          (find-code instructions *keypad-part2* 2 0)))

Advent of Code: [2016/03](https://adventofcode.com/2016/day/3)


- Not much to do, expect for do the needed
- Parse the specs into a tuple
- Part 1: count which spec is valid, i.e. `a + b > c` and `a + c > b` and `b + c > a`
- Part 2: _rotate_ the specs first, then count which ones are valid
(defun read-single-triangle-specification (string)
  (cl-ppcre:register-groups-bind ((#'parse-integer a b c))
      ("(\\d+)\\s+(\\d+)\\s+(\\d+)" string)
    (list a b c)))

(defun read-specifications (data)
  (mapcar #'read-single-triangle-specification data))

(defun valid-triangle-p (spec)
  (destructuring-bind (a b c) spec
    (and (> (+ a b) c)
         (> (+ a c) b)
         (> (+ b c) a))))

(defun rotate-specs (specs)
    :for (x y z) :on specs :by #'cdddr
    :for (ax bx cx) = x
    :for (ay by cy) = y
    :for (az bz cz) = z
    :collect (list ax ay az)
    :collect (list bx by bz)
    :collect (list cx cy cz)))

(define-problem (2016 03) (specs read-specifications)
  (values (count-if #'valid-triangle-p specs)
          (count-if #'valid-triangle-p (rotate-specs specs))))

2020-11-05 (permalink)

On _small_ design differences between Android and iOS platforms (and the importance of sticking with OS specific guidelines):

In the Facebook/Android style, a down-pointing chevron is a button you tap to expand more content, and an up-pointing chevron is a button you tap to collapse it. In the iOS/Mac style, a right-pointing chevron (or triangle, depending on the OS) indicates the collapsed state, and a down-pointing chevron indicates the expanded state. The Android way, a down-pointing chevron means “will open, if you tap”; the Mac/iOS way, a down-pointing chevron means “is open, tap to close”. You can argue that the Android style isn’t wrong, per se, just different, but (a) I’ll go to the mat arguing that disclosure chevrons should show state and imply action (the Mac/iOS way), not show action and imply state (the Android/Facebook way); and (b) it’s undeniably wrong for an iOS app — they’re breaking a core platform idiom. For examples that do it the right way, look at the list of accounts at the root level of iOS Mail, or the new collapsible “Pinned” header in iOS 14 Notes.

Source: [Disclosing a Few Thoughts on Facebook’s UI Design](https://daringfireball.net/2020/11/disclosing_a_few_thoughts_on_facebooks_ui_design)

2020-11-03 (permalink)

Here is a short CL function to _clear_ the screen (handy when periodically printing a board or something):
(defun cls ()
  "Clears the screen, and move the cursor to the top.


  - \033[2J: ANSI sequence to clear the screen
  - \033[H: ANSI sequence to move the cursor to the _home_ position

  It does using some FORMAT magic (e.g. `~:*` to move back in the format argument list)"
  (format t "~A[2J~:*~A[H" #\escape))
Reference: [StackOverflow](https://stackoverflow.com/questions/37774983/clearing-the-screen-by-printing-a-character)

2020-11-01 (permalink)

TIL: with [vim-surround](https://github.com/tpope/vim-surround), `cs([` will introduce an additional whitespace when swapping `(` with `[`, while `cs(]` will not.

Finally gave [vim-fugitive](https://github.com/tpope/vim-fugitive)'s `:Gclog` command a try, and I think I am going to stick with it.

Here is a bit of wrapping logic that I built around it:
function! DiffWithMaster() abort " {{{
    if systemlist('git rev-parse --abbrev-ref HEAD')[0] != 'master'
        return 'master..'
endfunction " }}}
nnoremap <leader>gl :Gclog! <C-R>=DiffWithMaster()<CR> -12<CR>
nnoremap <leader>gL :Gclog! <C-R>=DiffWithMaster()<CR><Space>
- The function, `DiffWithMaster()`, returns "master.." when the current branch **is not** "master", or nothing otherwise -- this will come in handy when working on a branch, and willing to list all the commits that have not made it into "master" yet
- The first mapping, `<leader>gl`, runs `:Gclog!` to show the most recent `12` commits _of the current branch_ (i.e. `:Gclog! -12` if on "master", `:Gclog! -12 master..` otherwise)
- The last mapping, `<leader>gL`, _prepares_ a command similar to the one above, except that a) it does not set a limit to the number of commits to show, and b) it waits for you to press enter to run it -- so you can customize it however you want

Finally gave up, and restored some old `*` / `#` mappings that I have -- I cannot stand the position of the cursor jumping around... I usually want to highlight things first, and then jump around using `n` or `N`.
" Smarcase for */# {{{

" When running a forward search, Vim will take you to the first **next**
" occurrence of the pattern; here instead we want to:
" - Go back to where we were (`<C-O>`)
" - Move at the beginning of the current word (`b`): this way, should we decide
"   to change direction and use N instead of n, we would immediately jump to
"   the previous occurrence and not stay where we are)
nnoremap * /\<<C-R>=escape(expand('<cword>'), '*\')<CR>\><CR><C-O>b
" When running a backward search, Vim will take you to the first **previous**
" occurrence of the pattern, which in this case would be the beginning of the
" word under the cursor
nnoremap # ?\<<C-R>=escape(expand('<cword>'), '*\')<CR>\><CR>

" }}}

2020-10-31 (permalink)

* finished reading [Facebook: The Inside Story](https://www.amazon.com/Facebook-Inside-Story-Steven-Levy/dp/0735213151)

? alacritty + vim: cursor shape goes back to block (instead of bar) when using 'cc' on a line that uses indentation

Finally got around to fix some issues with my Javascript's "select-top-level-expression" mapping (previously, it was tripping over when trying to evalute top-level array definitions).

" Custom text object to highlight top-level expressions
" Explanation:
" - 99[( - move to the outmost ( character: that makes it easy to
"   jump to the line where a function invocation starts (not much
"   useful if we are trying to evaluate a function, but for a toplevel
"   function call, this will come in handy
" - 99[{ - move to the outmost { character: that makes it easy to jump
"   to the line where a function definition likely starts
" - V% - toggle linewise-visual, and jump to the 'other side' (this
"   could mean the closing brace, or parenthesis)
vnoremap <buffer> <silent>af :<C-U>silent! normal! 99[(99[{V%<CR>
onoremap <buffer> <silent>af :normal Vaf<CR>
vnoremap <buffer> <silent>if :<C-U>silent! normal! 99[{vi{<CR>
onoremap <buffer> <silent>if :normal vif<CR>
and after:
function! SelectTopLevelExpression(around) abort " {{{
    let motion = ''

    " Let's start by jumping to the end of the line: that's because
    " the cursor might be at the befinning of a function declaration (i.e.
    " with no outermost pair to jump to), so this hopefully moves the cursor to
    " the opening curly brace
    let motion .= 'g_'

    " However, the previous motion might have moved the cursor on a `;` char,
    " one that does not have a pair defined; so in that case, move the cursor
    " one char to the left (i.e. on a closing curly brace)
    let line = getline('.')
    if line[len(line) - 1] == ';'
        let motion .= 'h'

    " Jump to the outmost _pair_; this should gracefully handle the case where
    " the cursor is in the middle of:
    " - array definitions (single, and nested ones)
    " - function parameters (i.e. its signature)
    " - or simply inside a nested block
    let motion .= '99[%'

    if a:around
        let motion .= 'V%'
        let motion .= 'vi{'

    " Finally run the motion.  Note, `normal` instead of `normal!` needs to be
    " used, or otherwise we won't be able to use user-defined mappings
    echom motion
    execute "normal " . motion
endfunction " }}}
vnoremap <buffer> <silent>af :<C-U>call SelectTopLevelExpression(1)<CR>
onoremap <buffer> <silent>af :normal Vaf<CR>
vnoremap <buffer> <silent>if :<C-U>call SelectTopLevelExpression(0)<CR>
onoremap <buffer> <silent>if :normal vif<CR>
I whish I had known about matchit operators before (i.e. `:help [%`).

2020-10-30 (permalink)

Paul Graham on [Twitter](https://twitter.com/paulg/status/1314260316134531075) quoted the below a couple of weeks ago, with a comment: "Every startup founder should read this..."

I don't thinkn STripe is uniformly _fast_. I think teams at Stripe are just _faster than_ most companies, blocked a bit less by peer teams, constrained a tiny bit less by internal tools, etc etc. There are particular projects which have been agonizingly long to ship; literally years after I would have hoped them done. But _across the portfolio_, with now hundreds of teams working, we just _get more done_ than we "should" be able to.

A _stupenduous_ portion of that advantage is just consistently choosing to get more done. That sounds vacuous but hasn't been in my experience. I have seen truly silly improvements occasioned by someone just consistently asking in meetings "Coud we do that faster? What is the minimum increment required to ship? Could _that_ be done faster?" It's the Charge More of management strategy; the upside is so dramatic, the cost so low, and the hit rate so high that you should just invoke it ritualistically

I think I tend to agree with him on this: ultimately, this _ritual_, is meant to keep you on track, focused on what really matters.  However, I am just afraid that, if abused, it would result in _unnecessary_ pressure for both the product, and the development departments.

2020-10-27 (permalink)

Yesterday, someone on the Lisp Discord server while talking about [CLOS](https://en.wikipedia.org/wiki/Common_Lisp_Object_System), mentioned a pretty cool **practical** example that I really think might help people undederstand the concept: Mixins.

From the [Mixin](https://en.wikipedia.org/wiki/Mixin) Wikipedia page:

In object-oriented programming languages, a mixin (or mix-in)[1][2][3][4] is a class that contains methods for use by other classes without having to be the parent class of those other classes. How those other classes gain access to the mixin's methods depends on the language. Mixins are sometimes described as being "included" rather than "inherited".

Now jump to the Examples section of that Wikipedia page, and under Common Lisp you should find the followin.

`object-width` is a generic function with one argument that uses the `+` method combination. This combination determines that all applicable methods for a generic function will be called and the results will be added.
(defgeneric object-width (object)
  (:method-combination +))
`button` is a class with one slot for the button text.
(defclass button ()
  ((text :initform "click me")))
There is a method for objects of class button that computes the width based on the length of the button text. `+` is the method qualifier for the method combination of the same name.
(defmethod object-width + ((object button))
   (* 10 (length (slot-value object 'text))))
A `border-mixin` class. The naming is just a convention. There are no superclasses, and no slots.
(defclass border-mixin () ())
There is a method computing the width of the border. Here it is just 4.
(defmethod object-width + ((object border-mixin))
`bordered-button` is a class inheriting from both `border-mixin` and button.
(defclass bordered-button (border-mixin button) ())
We can now compute the width of a button. Calling `object-width` computes 80. The result is the result of the single applicable method: the method `object-width` for the class `button.`
(object-width (make-instance 'button))
We can also compute the width of a `bordered-button`. Calling `object-width` computes 84. The result is the sum of the results of the two applicable methods: the method `object-width` for the class `button` and the method `object-width` for the class `border-mixin`.
(object-width (make-instance 'bordered-button))
Pretty neat.

2020-10-26 (permalink)

TIL: `git rebase` supports _auto-squashing_ of "squash! ..." and "fixup! ..." commits, and _automatic-stashing_ of local changes.

Add the following to your .gitconfig, and that should be it:
    autosquash = true
    autoStash = true
I can finally use [vim-fugitive](https://github.com/tpope/vim-fugitive)'s fixup/squash mappings without `git` getting in the way because my local repository is _dirty_.

Additional reading:

- [automatically stash save/pop changes on git rebase?](https://stackoverflow.com/a/27117335)
- [Auto-squahsiung Git Commits](https://thoughtbot.com/blog/autosquashing-git-commits)

2020-10-25 (permalink)

+ book: The Common Lisp Condition System: https://www.amazon.com/Common-Lisp-Condition-System-Mechanisms/dp/148426133X

? book: Lisp From Nothing: http://t3x.org/lfn/index.html

I have been thinking about moving my dotfiles directory into Dropbox, lately, but I am still not sure whether that's a good idea or not.

Let's have a look first, at the problem I am trying to solve.  Too frequently I find myself playing with some config file tweaks on my laptop, then ssh into my work virtual machine expecting these config files to include the changes I was just testing out few minutes ago; except, I never pushed those changes to my Git repository to begin with, so I am not sure how would that work -- for the guest to pick up the changes...automatically.  Anyway, every time this happens, I usually go ahead and either commit the changes on my host, push, go back to the guest and pull, or I simply re-apply the changes, manually; clearly none of the mentioned solutions seems to be ideal, so what if there was a better way to deal with this?  What if I moved my dotfiles repository, inside Dropbox, and only pushed changes to GitHub only when actually ready to do so? (i.e. when I am happy with the config changes I am playing with).

As appealing as this solution might be, one biggest problem I see is making sure it all works across different OSes (i.e. MacOS on my personal laptop, Windows 10 on my work laptop, and Linux/Ubuntu inside VirtualBox).  Now, my config files already have OS specific switches, but what about _supporting_ libraries/binaries? For example, I depend on `js-langserver` as LSP server when editing JavaScript files; currently, I simply `npm install` it, but I don't want its node_modules directory to end up getting uploaded on Dropbox as that will be a waste of space/bandwidth, and most importantly, it will very likely not work (e.g. running `js-langserver` from MacOS won't work if I previously run `npm install` from Linux, or even worse, from Windows/Cygwin).

It seems this solution _could_ work if there was a programmatic way for me to tell Dropbox which files to sync, and which files to ignore.  Well, lucky me, there is indeed a way to [tell Dropbox that](https://help.dropbox.com/files-folders/restore-delete/ignored-files), so I guess the next step would be to figure the list of files/directories to exclude from the sync:

- All the node_modules directories
- All the venv directories -- that's the name I usually use for `virtualenv` directories
- Vim's temp/backup files maybe?
- Something else?

2020-10-23 (permalink)

? project idea: create a REPL for cURL. I have been using vim-rest-console, but that requires additional setup, probably too much, especially if I wanted to fire some requests straight from my .plan file; what a curl-repl existed? I mean, I could pop that open into a terminal buffer, and copy requests directly into it. I guess I would have to define a syntax to specify headers, requsts, bodies, and all that stuff.. but it would be interesting.

2020-10-21 (permalink)

At work, over the weekend we had to deal with an outage of our PROD environment, and here is a summary of what had happened:

- Few weeks prior to the outage, our IT department accidentally disabled one of ours distribution lists (unfortunately, the one we were using to receive notifications about the state of our servers)
- At about the same time, the disk usage patterns of our database server had increased -- interestingly, disk usage was increasing faster than we were able to clean up, with our periodic jobs
- Disk usage gets past the warning threshold, triggering some notifications that we never see because the distribution list got accidentally disabled
- Few more days go by, and we get past the critical threshold; more notifications that unfortunately we never get to see
- Eventually, over the weekend, the server runs out of disk space and pretty much all the services hosted on the server stopped working (in particular, for Neo4j, one of the hosted databases wound up with a corrupted transaction log)

Now, after a bit of fire-fighting we were able to:

- recover the corrupted transaction log and unlock that database
- re-enabled the distribution list used for notifications
- lowered notification thresholds even more (more breathing space, should things ever escalate again)

Still we were wondering if there wasn't something else Neo4j could have done to prevent itself from filling up the disk and paint itself into a corner.  What if Neo4j implemented a mechanism that would put the database instance in read-only mode, when disk usage went above a pre-configured threshold?  Elasticsearch implements [something similar](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-cluster.html#disk-based-shard-allocation), and I think it would have come in really handy if it had been there for us to use.

I appreciate it should be responsibility of whoever manages the server that the server itself never runs out space; but bad things happen, and if a service, Neo4j in this case, was able to monitor the situation and defensively shut itself down / enter read-only mode, I believe that would have avoided a lot of pain down the line.

Anyway, brought this up with the Neo4j folks, so let's see what they think about this enhancement.

2020-10-20 (permalink)

I have been poking around with `winston`'s API lately, mostly to look into some unexpected behaviors where the `File` logger would stop writing in case of uncaught exceptions / unhandled rejections, and here is what I found out.

First things first, `winston` ships with some _transport_ settings that can be used to instruct the logger to deal with (or not) uncaught exceptions, and unhandled rejections:

- `handleExceptions`: tells winston to catch and log uncaught exceptions (e.g. `process.on('uncaughtException', log)`)
- `handleRejections`: tells winston to catch and log unhandled rejections (e.g. `process.on('unhandledRejection', log)`)

Previously, unaware that such API existed, we used to manually deal with this as follows:
process.on('uncaughtException', function(error) {
    // TODO: handle uncaught exceptions
    console.log('uncaughtException', error);

process.on('unhandledRejection', function(reason, promise) {
    // TODO: handle error here, all "swallowed" errors get here
    console.log('unhandledRejection', { reason, promise }, reason.stack);
But why keep on doing this, if a library you are already linking and using can deal with this on your behalf?  Exaclty, there is no reason, so we immediately got rid of this custom logic.

Now, annother interesting setting worth mentioning, is `exitOnError`; when set to `true`, winston will fire a `process.exit(1)` and cause the whole application to halt (the idea being... something _unexpected_ happened, so better to exit immediately than continuing in an _unknown_ state).  We have been setting this to `false`, but I wonder if we shouldn't stick with the default value instead -- in the end, should something bad happened and because of that our app crashed, the process manager used to keep the app running should immediately restart it.  Also, we still have to deal with the fact that the `File` transport seems to be shutting itself down in case of uncaught exceptions, and whether this is a bug or expected behavior, `exitOnError: true` seems to be the only workaround available (at least until winston's maintainers don't get back to us on [this](https://github.com/winstonjs/winston/issues/1855)).

Last point, which unfortunately we haven't got around to implement yet: don't bother with on-disk logging, just do all your logging to standard output, and let the process manager (`docker` in our case), deal with that.

2020-10-15 (permalink)

TIL: Express will automatically handle exception thrown inside defined middlewares:

Errors that occur in synchronous code inside route handlers and middleware require no extra work. If synchronous code throws an error, then Express will catch and process it. For example:

I am not sure why this surprises me, in the end the exception is thrown in stack, so that's the **right** thing to do; somehow I was under the impression that `throw`-ing from within a middleware would result in the _uncaught_ exceptions being triggered, instead I was wrong: Express will take care of it, and convert it into 500 response.

Starting with Express 5, route handlers and middleware that return a Promise will call next(value) automatically when they reject or throw an error.

Cool -- again, if you think about, it definitely makes sense.

Source: http://expressjs.com/en/guide/error-handling.html

2020-10-09 (permalink)

TIL: you can force a case-sensitive search on vim, by adding `\C` anywhere on the search pattern (`:help \\C`):
							*/\c* */\C*
When "\c" appears anywhere in the pattern, the whole pattern is handled like
'ignorecase' is on.  The actual value of 'ignorecase' and 'smartcase' is
ignored.  "\C" does the opposite: Force matching case for the whole pattern.
{only Vim supports \c and \C}

2020-10-05 (permalink)

* finished reading [Eloquent JavaScript, 3rd Edition: A Modern Introduction to Programming](https://www.amazon.com/Eloquent-JavaScript-3rd-Introduction-Programming/dp/1593279507)

Few other small things read inside 'Eloquent JavaScript' which I think are worth a note:

The first was about Node.js, the `fs` module in particular, and the fact that not only it ships with the classical callback-based asynchronous interface and with a sycncrhonous version of it, but that since `Promise`s got hot a while back, the core team decided to ship a new promise-based API, with identical names to the callback-based one, but all bundled inside a `promises` namespace.  What's interesting about it? I don't know... it made me think about how difficult it becomes to keep up with new user needs (e.g. `Promise`s are cool, everyone loves them, so we should add support for them) and at the same time supporting an existing product / API: they could have changed the existing API to be promise-based, but that would have introduced a big regression, so they ended up packaging the new API into a new _namespace_ (of course the new package would be less _discoverable_, but it seems like a good trade-off to me).

HTTP Request Method: [MKCOL](http://webconcepts.info/concepts/http-method/MKCOL) -- never heard of it, which I think it suggests it might not be a widely used HTTP method, but I would definitely add this to my TIL list.

Swapping bindings in JavaScript can nicely be done using [destructuring assignment](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Destructuring_assignment): `[b, a] = [a, b]`

2020-09-30 (permalink)

I was reading [Eloquent JavaScript](https://www.amazon.com/gp/product/1593279507/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=1593279507&linkCode=as2&tag=marijhaver-20&linkId=d8642c0457954f03e27c02b0034d0d60) the other night, and learned an interesting and scaring thing about regular expressions:

JavaScript RegExp objects are stateful when they have the global or sticky flags set (e.g. /foo/g or /foo/y). They store a lastIndex from the previous match. Using this internally, exec() can be used to iterate over multiple matches in a string of text (with capture groups), as opposed to getting just the matching strings with String.prototype.match().

This happens with both `.exec()` and `.test()`, meaning that further calls to `.test()` will resume searching from `lastIndex` (which will continue to increase each time `test()` returns `true`).


- [RegExp.prototype.exec()](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp/exec)
- [RegExp.prototype.test()](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp/test): scroll to the "Using test() on a regex with the "global" flag" paragraph

2020-09-28 (permalink)

TIL: on Vim you can `set number` and `set relative number` at the same time, and have the current line number displayed instead of `0`.

With `set relativenumber`:
   3 set ttyfast
   2 set ruler
   1 set backspace=indent,eol,start
   0 set relativenumber
   1 set number
   2 set laststatus=2
   3 set history=1000
With `set number` and `set relativenumber`:
   3 set ttyfast
   2 set ruler
   1 set backspace=indent,eol,start
20   set relativenumber
   1 set number
   2 set laststatus=2
   3 set history=1000

2020-09-25 (permalink)

JSON Web Tokens Suck (https://www.youtube.com/watch?v=JdGOb7AxUo0)

Stateful JWT
- All the user information (username, first last names, email) are stored in the JWT
- Server receives token, validates it, reads it, and uses the information to reply back to the user
Stateless JWT
- We only store a Session-ID inside the JWT
- Server receives token, validates it, looks up the session (DB, cache, whatever), then uses the information to reply back to the user
Session cookies
- Cryptographically signed session identifier stored in a cookie.  Data is stored server-side, and retrieved from a database
- Pretty much similar to Stateless JWT
What's the difference?
- Encoding
- One uses JSON as data format, the other just sends the session ID as a plain string
- HTTP headers
- Key/value storage
- Limited in size (4kb)
- Server uses `Set-Cookie` header to set cookies on the client (e.g. `session=signed(12345)`)
- Flag `HttpOnly`: prevents user code (Javascript) from accessing cookies
- Flag `SameSite=strict`: prevents cross-site request forgery
- Flag `secure`: don't store this date unless we talking on SSL
Local storage:
- Browser based API to implement key/value store
Myths about JWT:
- JWTs are easier to use (than Session cookies): JWTs actually require additional libraries, while session cookies are nowadays supported by all the web servers/frameworks around
- JWTs are more flexible: actually, anything you can define in a JWT, you can also define as a session cookie (the payload would even be more succinct, resulting in less bandwidth usage)
- JWTs are more secure: again, both cryptographically signed, both can be encrypted, except JWT specification is more complicated, and different libraries support/implement it in different ways.  Session cookies on the other hand, have been battle tested for years, and no vulnerabilities have been found so far, so....
- JWTs don't require "cookie consent" -- false... the law requires to put the notification on your application if you are tracking user behavior
- JWTs prevent CSRF (because JWT are usually stored on local storage) -- well, you can still store JWTs with cookies.  If you are using local storage though, you are are opened yourself up to Cross-site scripting (XSS)
- JWTs are better for cross domain (e.g. authentication against www., docs., apps.domain.com)
- JWTs work better on mobile... NO!  Web authentication is handled by Web browsers, and web browsers work the same whether you are running them from Desktop, or Mobile
- JWTs are more efficient... NO!  Same payload, same cryptographic algorithm... in these circumstances the payload of JWT can be 3x larger
- JWTs are easy to revoke -- false, you have to change your signature key.  You can check check the validity of a token... but then if you need to check against a database/cache, why are you using JWTs to begin with?!
- JWTs are easier to "Scale" -- not really.  It's true, if you implement stateful JWT you don't have to interrogate the database, but then you are opening up to other security attacks; however, session management can nowadays be horizontally scaled quite easily, so the argument is weak.
- JWTs are secure by design: actually they were never meant for web authentication; there is always a tradeoff between security and speed, and unless you verify on the server that tokens are still valid (defeating again the purpose of using JWTs), then it won't be impossible for you to prevent malicious users from doing damage
What are JWTs good for?
- 1 time redirect: pay for a book on books., receive a link to download the book on download., the link contains the JWT which then the server internally validates (the JWT is short lived in this case)
- Resetting passwords

2020-09-24 (permalink)

* Submitted a [PR](https://github.com/tbodt/js-langserver/pull/9) for `js-langserver` to switch to the latest version of Tern.js (it simplifies the integration), and add support for `defs` and `loadEgerly`

* Submitted a [PR](https://github.com/sbdchd/neoformat/pull/323) for `neoformat` to try and fix a problem with folds getting destroyed/closed/opened every time `neoformat` is run -- it turns out it was due to the fact that the plugin was always resetting `&filetype`, even though it did not change

2020-09-23 (permalink)

TIL: in Vim, you can disable/enable spell-check based on syntax (`:help spell-syntax`)!
SYNTAX HIGHLIGHTING					*spell-syntax*

Files that use syntax highlighting can specify where spell checking should be

1.  everywhere			   default
2.  in specific items		   use "contains=@Spell"
3.  everywhere but specific items  use "contains=@NoSpell"

For the second method adding the @NoSpell cluster will disable spell checking
again.  This can be used, for example, to add @Spell to the comments of a
program, and add @NoSpell for items that shouldn't be checked.
Also see |:syn-spell| for text that is not in a syntax item.

2020-09-22 (permalink)

Worse is better: http://dreamsongs.com/WorseIsBetter.html

In The Rise of Worse is Better, Gabriel claimed that "Worse-is-Better" is a model of software design and implementation which has the following characteristics (in approximately descending order of importance):

Simplicity: The design must be simple, both in implementation and interface. It is more important for the implementation to be simple than the interface. Simplicity is the most important consideration in a design.

Correctness: The design should be correct in all observable aspects, but It is slightly better to be simple than correct.

Consistency: The design must not be overly inconsistent. Consistency can be sacrificed for simplicity in some cases, but it is better to drop those parts of the design that deal with less common circumstances than to introduce either complexity or inconsistency in the implementation.

Completeness: The design must cover as many important situations as is practical. All reasonably expected cases should be covered. Completeness can be sacrificed in favor of any other quality. In fact, completeness must be sacrificed whenever implementation simplicity is jeopardized. Consistency can be sacrificed to achieve completeness if simplicity is retained; especially worthless is consistency of interface.

2020-09-20 (permalink)

Martin Fowler's essay about [Collection Pipeline](https://www.martinfowler.com/articles/collection-pipeline/) never gets old; should probably come back and re-read it more frequently.

2020-09-16 (permalink)

Lately, I have been exploring the codebase of the product I am involved with at work, looking for repetitive usage patterns.

One in particular that caught my attention, had something to do with [feature toggles](https://www.martinfowler.com/articles/feature-toggles.html):

_Is a given feature X, toggled on for the logged in user? If so, show this area of the screen, otherwise, hide it or show something else instead_

This is how we have been handling this (we use Angular):

- Get the `FeatureToggleStore` injected in the controller of the current screen
- Store somewhere on the controller if the given feature is on/off for the logged in user
- Use that flag inside the template to show/hide (read: `*ngIf`) certain areas of the screen

If you think about it, what we really want is a special [structural directive](https://angular.io/guide/structural-directives), similar to `*ngIf`, that instead of evaluating the input expression and expecting it be either truthy or falsey, it would check that feature against our store, and then manipulate the DOM accordingly.

Easy right?! Let's take a stab at this:
import { Directive, Input, TemplateRef, ViewContainerRef, EmbeddedViewRef } from '@angular/core';
import { FeatureToggleStore } from '@upgraded/services';

  selector: '[ifFeature]',
export class IfFeature {
  private featureToggled = false;
  private thenViewRef: EmbeddedViewRef<any> = null;

      templateRef: TemplateRef<any>,
      private viewContainer: ViewContainerRef,
      private featureToggleStore: FeatureToggleStore
  ) {
    this.thenTemplateRef = templateRef;

  set ifFeature(featureName: string) {
    this.featureToggled = this.isFeatureToggled(featureName);

  private updateView() {
    if (!this.featureToggled) {
    } else {

  private isFeatureToggled(featureName: string) {
    return this.featureToggleStore.hasFeature(featureName);
Expected usage:
<div *ifFeature="beta-feature">
    Hi beta user!
This looks good.  However, the implementation has some problems:

- What if the name of the feature is dynamically updated, and user has access to the new one too? This solution will cause inner part of the DOM (the one protected by the feature), to be re-rendered again even though nothing has changed (we keep on calling `this.viewContainer.createEmbeddedView()`)
- What if we wanted to support an `else` branch, like `*ngIf` does?

Well, for this, we will have to stand on the shoulders of the [giants](https://github.com/angular/angular/blob/master/packages/common/src/directives/ng_if.ts), and you might end up with something like:
import { Directive, Input, TemplateRef, ViewContainerRef, EmbeddedViewRef } from '@angular/core';
import { FeatureToggleStore } from '@upgraded/services';

  selector: '[ifFeature]',
export class IfFeature {
  private featureToggled = false;
  private thenTemplateRef: TemplateRef<any> = null;
  private thenViewRef: EmbeddedViewRef<any> = null;
  private elseTemplateRef: TemplateRef<any> = null;
  private elseViewRef: EmbeddedViewRef<any> = null;

      templateRef: TemplateRef<any>,
      private viewContainer: ViewContainerRef,
      private featureToggleStore: FeatureToggleStore
  ) {
    this.thenTemplateRef = templateRef;

  set ifFeature(featureName: string) {
    this.featureToggled = this.isFeatureToggled(featureName);

  set ifFeatureThen(templateRef: TemplateRef<any>) {
    this.thenTemplateRef = templateRef;
    this.thenViewRef = null;

  set ifFeatureElse(templateRef: TemplateRef<any>) {
    this.elseTemplateRef = templateRef;
    this.elseViewRef = null;

  private updateView() {
    if (this.featureToggled) {
      if (!this.thenViewRef) {
        this.elseViewRef = null;
        if (this.thenTemplateRef) {
          this.thenViewRef = this.viewContainer.createEmbeddedView(this.thenTemplateRef);
    } else {
      if (!this.elseViewRef) {
        this.thenViewRef = null;
        if (this.elseViewRef) {
          this.elseViewRef = this.viewContainer.createEmbeddedView(this.elseTemplateRef);

  private isFeatureToggled(featureName: string) {
    return this.featureToggleStore.hasFeature(featureName);
Additional reading: https://juristr.com/blog/2018/02/angular-permission-directive/

2020-09-08 (permalink)

Small Javascript utility function to create N-d arrays (_create_, not _initialize_):
function createArray(dim, ...dims) {
  if (!dims.length) {
    return new Array(dim);
  } else {
    return Array.from(Array(dim), () => createArray(...dims));
- Accepts the size of the outer layer, and the sizes of the inner ones (if any)
- Recursive solution
- ...when on the last layer, simply return a new Array, with the expected size
- ...otherwise, create a new array with the expected size, and initialize its elements with a recursive call (i.e. each element would now be initialized with a new (N-1)-d array)

Few pointers on writing **good** commit messages:

- https://chris.beams.io/posts/git-commit/
- https://wiki.gnome.org/Git/CommitMessages
- https://www.conventionalcommits.org/en/v1.0.0/

When in doubt, check what the specific project commit guidelines say (if any).

2020-09-05 (permalink)

Great presentation, from Maciej Ceglowski, on how bloated the Web has become -- it's from 2015, but if anything, things have gotten worse since: https://www.youtube.com/watch?v=iYpl0QVCr6U

The annoying thing about HTML emails using CSS magic to convert DIVs into links/buttons, is that when you try and view them with Mutt/w3m, action links would not render correctly; sometimes this is great, there is a reduced risk of phishing attacks, but what if I indeed wanted to click that link?

Two options:

1. Pop Gmail open, and read the email there
2. Configure Mutt/mailcap to use an _external_ browser when explicitly told to

I have been using the former for quite some time, but with the increased influx of HTML email, I have grown tired of it.  Especially because the latter turned out to be pretty simple to implement:
# ~/.mutt/mailcap

text/html; open '%s' & sleep 3; nametemplate=%s.html; needsterminal
text/html; w3m -cols 90 -dump -T text/html '%s'; copiousoutput
The first entry, is used when _sending an attachment to mailcap_ (pop an email open, press `v` to open the attachment menu, select a text/html entry, and press `m`); the latter instead, is automatically used by Mutt to display HTML emails.

PS. The `& sleep 3` is used to make sure the attachment Mutt does not accidentally deletes the attachment before the _external_ browser has finished processing it.
PPS. Want to use HTML emails? Use semantic web, please... seriously.

2020-08-25 (permalink)

* First stab at _converting_ plan files into HTML (previously a plan entry was mostly wrapped inside a `<pre>` block, and that's it): I am now able to format lines differently, based on their type (i.e. completed, fixed, ideas, discarded, and verbatim), but I am not quite happy with the parsing logic (better to use SMUG or something, but still, it got the job done). You can check out the result on: https://matteolandi.net/plan.html

? plan-convert: review parser logic, and maybe use SMUG

? plan-convert: add ability to format inline italic, strong, code, and links

? plan-convert: generic text (i.e. generic notes) is currently getting wrapped inside a <pre> element, or otherwise lists would render on a single line; it all works fone on the browser (where we can style things with CSS), but with `w3m` these long lines do not wrap (as it usually happens with paragraphs) and I am forced to horizontally scroll

2020-08-24 (permalink)

TIL: Elasticsearch will enter read-only mode when running out of disk-space

Elasticsearch started acting out today, and logs were littered with the following error messages:
error: [search] Error when updating documents:
by: [FORBIDDEN/12/index read-only / allow delete (api)];"}}}
Our database box did run out of space earlier today (well, actually, the disk usage got above a specific threshold), so that would explain Elasticsearch's behavior.  Anyway, free-ing up space and bouncing the service is not going to help; you will indeed have to manually unlock the cluster and restore write accesses:
$ curl -XPUT -H "Content-Type: application/json" http://localhost:9201/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}'
After this, things should finally go back to normal.


- https://selleo.com/til/posts/esrgfyxjee-how-to-fix-elasticsearch-forbidden12index-read-only
- https://discuss.elastic.co/t/forbidden-12-index-read-only-allow-delete-api/110282

2020-08-23 (permalink)

? djula: how to use `if`/`ifequal` tags to equate to a _symbol_? For example, say I fed the template with: `:item (list :type :verbatim :content "foo")`, how can I check against type? None of the following seem to work: "verbatim", "VERBATIM", :verbatim. Currently I am forced to pass `:verbatim :verbatim` to the template, and then `{% ifequal item.type verbatim %}` which is a bit...meh

2020-08-22 (permalink)

* finished reading [Patterns of Software: Tales from the Software Community](https://www.amazon.com/Patterns-Software-Tales-Community/dp/0195121236)

2020-08-19 (permalink)

TIL: [standard](ttps://www.npmjs.com/package/standard)

Electron - By Javan Makmali, 2016: https://www.youtube.com/watch?v=nwvourv3nSQ

Basecamp 3, released on Web, iOS, Android, and **desktop** (via Electron)

- The prototype was ready in 1 day
- 6 weeks to get it ready to ship
- You finally can CMD/CTRL-TAB and switch to the app (no more a tab within the browser)
- Having your brand on the dock... that's powerful :)
- Inject some CSS to add icons/items to the Window title bar (not the menu bar)
- History menu, to navigate to previous pages
- Remember your last place, and window size

Quick tour how Electron works:

- Main process: C++
- Opening a tab, spawns a new process (**Renderer** process)
- With Electron, Node.js is simply injected into each Renderer, and into the Main process, and they can communicate together, and access the OS
- Electron is... a set of APIs that you can use to build applications, and interact with the OS

Get started:
$ mkdir hello
$ npm init -y
$ npm install electron-prebuilt --save-dev
$ $EDITOR package.json
# add new "start" script to run: "electron ."
$ $EDITOR index.js
# and add
const { app, BrowserWindow } = require('electron')
app.on('ready', () => {
  let mainWindow = new BrowserWindow()
$ $EDITOR index.html
# add some content
$ npm start
Communicate back to the main process from the renderer:
document.addEventListener('click', (event) => {
  require('electron').remote.app.dock.setBadge(`Clicked! ${event.pageX}`)

- You can inject new items in the menubar (big JavaScript object to define items: item -> click -> userland callback)
- You can create create toast notifications
- You don't have to worry about browser compatibility anymore, and you can use all the experimental features that you want (that are supported by Chromium)
- Mutate the browser like normal extensions would do (i.e. high)
- Intercept every request / proxy it / delete cross-origin headers and render stuff in an iframe even if forbidden
- You can inject a blob of css into fetched page (at Basecamp, they just add a new css-class, and keep all the styles remote)
- Persistence: SQLite, LocalStorage, IndexDB
- You can register system wide key commands (activated when the app is running)
- You can register the app as a protocol handler (e.g. basecamp://...)


- Facilities to bundle/ship Electron applications
- Sign your application with your name
- APIs for automatic updates (Mac apps are always full-updates, while on Windows, only diffs are sent)


- Browser built with Electron + React
- That's how they inspect traffic, and filter out ads
- Open a window, and load a website... simple

2020-08-18 (permalink)

? book: Living with Complexity

The JavaScript That Makes Basecamp Tick, By Javan Makhmali, 2014: https://www.youtube.com/watch?v=mWo3oEwFFzM

Basecamp (2):

- 40 app servers
- 100 total servers
- 1/2 PB file storage
- 1 TB database
- 1 TB caching
- 70 million requests per day
- 50-60ms Rails response times

The Asset Pipeline:

- Tool for compiling, concatenating, and compress asset files

  - desktop.js.coffee.erb -> erb first, then coffee compiler, then javascript...

- Bunch of _require_ comments that bring in assets from somewhere else, and add them to the pipeline
- All of this will later be replaced by webpacker
- You can also configure an `asset_host`, so the generated `<script src.../>` line, will load the asset from there (you will have to manually upload the file on the CDN)

Code structure:

- Lot of small, single-purple, files (see below), mostly to give behavior to elements/group of elements
- Data attributes:J avascript only, not inside CSS (inside CSS, they use class names, and IDs)
// behaviors/element/expandable.js.cofee
selector = "[data-behavior~=exandable]"
expandSelector = "#{selector} [data-behavior~=expand_on_click]"

$(document).on "click", expandSelector, (event) ->

// somewhere else, in the html
<div data-behavior="expandable">
  <div class="expanded_content">...</div>
  <a href="#" data-behavior="expand_on_click">Expand</a>
- Most of the event handlers, are bound to the `document` element: arguably the slowest thing you can do... but it does not really matter -- the end result is pretty good.  Don't need to add bindings when elements are added to the dom; all the events will eventually get to the `document`, which will then dispatch based on the selector.


- Page stacking UI
| Basecamp
| +------------------------
| | Project XXX
| | +----------------------
| | | Todolist YYY
- Refreshing the page, will actually change the layout a bit -- no more stacks, just the one you requested (you can still go back though)
- Pages are cached, so going back will first show you the cached page; the page is still getting fetched, but it will replace the current and cached one, only if-changed (304 status)
- Goodbye: DOMContentLoaded.  Hello: page:beforechange, page:change, page:update
- Custom way of registering global events, that enables them to document what the custom event is for -- that will make for easier debugging (they can enter debug mode, and see each handler how long it took to complete)
bcx.on "page:beforechange", "Set bcx.currentBucket", ->
  if matches = location.pathname.match /\/...

bcx.on "page:update", "Install autoresize behavior on visible elements", ->

bcx.on "page:change", "Enable placeholders in IE", ->
  $("[placeholder]").placeholder() if $.browser.msie
- Inspired by [pjax](https://pjax.herokuapp.com/)
- Lead to [turbolinks](https://github.com/turbolinks/turbolinks)

Assets reload:

- People are not refreshing their pages very often, so they won't always get the latest version of the JavaScript files

  - Every Rails request set `X-Asset-Paths` where all the assests and their digests are listed
  - On the client, whenever a Ajax request is completed (i.e. `ajaxSuccess`), we would compare these digests with the ones currently loaded, and if different, we would trigger a full page reload

Cache all the things:

- Russian doll caching

  - Each individual element is cached: todo, todolist, project
  - When you change a todo, you will touch the todolist, and the project
  - All these cache entries are invalidated
  - Rebuilding the project page is cheap though, as most of the content is already cached

- There is stuff you cannot cache (e.g. timestamps, edit/delete links -- usually available to the author, for 15 minutes)

  - You cache the complete object (as if an Admin, with all the permissions possible, was looking at it)
  - You use JavaScript to taylor the page to the logged in user
  - `<time data-local="time-ago" datetime="${UTC value}">` that will read the UTC value, and render a localized version of it **after** the page has been rendered
  - `data-creator-id=${ID}` and `data-visible-to="admin creator"` data-attributes in the template, then inside JavaScript
$("[data-visible-to]").each ->
  visibleTo = $(this).attr("data-visible-to").split /\s/
  creatorId = parseInt($(this.closest("[data-creator-id]").data("creator-id")))

  return if _.include(visibleTo, "admin") and bcx.currentPerson.isAdmin()
  return if _.include(visibleTo, "creator") and bcx.currentPerson.id is creatorId

  - of course the same checks ought to be done on the server as well :)
def ensure_permission
  head :forbidden unless current_person.can_change?(@comment)

- Basecamp is not a backbone app, but it relies heavily on it on certain areas... like the Calendar!
- When there are a lot of interactions, it's better to start doing things on the client

Eco, templating language on the client:

- Coffeescript, ERB, so people are already familiar with it

"Live" page updates:

How to keep 2 pages in sync:

- Polling: each person with an active session, is polling the polling server (Rack app), every 3 seconds
- When I create a comment, Basecamp sends back to me some JavaScript to a) append the new comment to the thread (e.g. `$("section#comments_for_...").append("...")`), b) highlight it (e.g. `$("article#...").highlight();`)
- Redis queue where page updates pushed, and then delivered to users when they are polling; due to the nature of the updates (they go by section/id), if you are not **that** page, nothing will happen

2020-08-16 (permalink)

* finished reading [Common Lisp Recipes (CLR)](http://weitz.de/cl-recipes/)

2020-08-11 (permalink)

+ read Structure and Interpretation of Computer Programs (SICP)

2020-08-10 (permalink)

TIL: DevOps folks have [design patterns too](https://medium.com/@karthi.net/8-container-design-patterns-that-you-should-know-970a1d6a594c), like the _ambassador_ pattern, described [here](https://www.ctl.io/developers/blog/post/deploying-multi-server-docker-apps-with-ambassadors/) and [here](https://test-dockerrr.readthedocs.io/en/latest/admin/ambassador_pattern_linking/)

2020-08-09 (permalink)

Today I decided to spend some time to scratch one of the itches I have been having lately, when working with CL (Vim + Vlime): closing all the splits except for the current one, the Lisp REPL (terminal buffer), and Vlime's REPL.

When I work on a CL project, I usually arrange my splits as follows:
| file.lisp        | CL-USER>         |
|                  |                  |
|                  |                  |
|                  |                  |
|                  +------------------+
|                  | VLIME REPL       |
|                  |                  |
|                  |                  |
|                  |                  |
Well, that's when I start working... few minutes/hours later, it might as well look similar to the below:
| file.lisp  | file2.lisp | CL-USER>  |
|            |            |           |
|            |            |           |
|            |            |           |
| fil11.lisp | file3.lisp | VLIMEREPL |
|            |            |           |
|            |            |           |
|            |            |           |
So, how can I easily and quickly get from this messy state, to where I started off, with a single file, and the termnial and Vlime REPLs, and Vlime's REPL?  `:only` would unfortunately close all the open splits except for the current one, which would force me to create new splits for the terminal and Vlime REPLs.

I ended up creating the following VimL function:
function! LispCurrentWindowPlusVlimeOnes() abort " {{{
    let focused_win = win_getid()

    for tab in gettabinfo()
        for win in tab.windows
            if win != focused_win
                let bufname = bufname(winbufnr(win))

                if bufname =~? 'vlime'
                    let winnr = win_id2win(win)
                    execute winnr . 'close'
endfunction " }}}
Which can be summarized as follows:

- Get a hold of the focused split's Window-ID
- Iterate all the splits of the current tab

  - If the current split is **not** the focused one
  - and the loaded buffer does not contain 'vlime' (that will match both my terminal REPL, and Vlime's one)
  - then we should close the split

And that's it!  Wire those mappings in (I like to override `:only` ones) and you should be ready to roll:
nnoremap <C-W>o :call LispCurrentWindowPlusVlimeOnes()<cr>
nnoremap <C-W>O :call LispCurrentWindowPlusVlimeOnes()<cr>
nnoremap <C-W><C-O> :call LispCurrentWindowPlusVlimeOnes()<cr>

2020-08-03 (permalink)

Had some troubles updating my brew formulae today:
$ brew cask upgrade java
==> Upgrading 1 outdated package:
Error: Cask 'java' definition is invalid: Token '{:v1=>"java"}' in header line does not match the file name.
It seems like my java cask is [unbelievably old](https://github.com/Homebrew/homebrew-cask/issues/58475):
$ rm -rf "$(brew --prefix)"/Caskroom/java
$ brew cask install java
Updating Homebrew...
==> Auto-updated Homebrew!
Updated Homebrew from 36c10edf2 to 865e3871b.
No changes to formulae.

==> Downloading https://download.java.net/java/GA/jdk14.0.2/205943a0976c4ed48cb16f1043c5c647/12/GPL/openj
Already downloaded: /Users/matteolandi/Library/Caches/Homebrew/downloads/4a16d10e8fdff3dd085a21e2ddd1d0ff91dba234c8c5f6588bff9752c845bebb--openjdk-14.0.2_osx-x64_bin.tar.gz
==> Verifying SHA-256 checksum for Cask 'java'.
==> Installing Cask java
==> Moving Generic Artifact 'jdk-14.0.2.jdk' to '/Library/Java/JavaVirtualMachines/openjdk-14.0.2.jdk'.
🍺  java was successfully installed!

2020-07-28 (permalink)

Oh boy, time-zones are a bitch!

Spent the day looking at the different DB-related libraries I am using in my web app (MITO, CL-DBI, and CL-MYSQl) trying to chase down a problem with DATE fields; these fields, even though stored with the **right** value inside the database, once _inflated_, they ended up with an unexpected time-zone offset (in MITO's terminology, _inflated_ is the action of transforming data, after it's read from the database, so it's easier to work with it inside the controller).

Before we dive into the details, let's make it clear I want my web app to store DATE/DATETIME/TIMESTAMP in UTC (everyone should, really)... and here I thought the following SETF statement will get the job done:
(ql:quickload :local-time)

(setf local-time:*default-timezone* local-time:+utc-zone+)
It all starts with the _model_ trying to save a date object; it reads it as string from the request, converts it to a `local-time:timestamp`, and pass it down:
(defvar date "2020-07-28")

(local-time:parse-timestring date)
MITO will then process the value twice; it will try to _deflate_ it first inside DEFLATE-FOR-COL-TYPE (_deflate_ is the opposite of _inflate_, i.e. make a memory value ready to be processed by the database access layer) -- this step will turn out to be a noop for us, since we are using `local-time:timestamp` already:
(defgeneric deflate-for-col-type (col-type value)
  (:method ((col-type (eql :date)) value)
    (etypecase value
      (string value)
      (null nil)))
Then it will apply some MySQL specific conversions inside CONVERT-FOR-DRIVER-TYPE
(defgeneric convert-for-driver-type (driver-type col-type value)
  (:method (driver-type (col-type (eql :date)) (value local-time:timestamp))
    (local-time:format-timestring nil value
                                  :format *db-date-format*))
This basically converts the `local-time:timestamp` back into a YYYY-MM-DD string:
(defvar *db-date-format*
  '((:year 4) #\- (:month 2) #\- (:day 2)))

(local-time:format-timestring nil **
                              :format *db-date-format*)
And this is how the data is actually getting stored inside the database.  Let's have a look now at what happens when we try to read the data back.

I am using a MySQL database, and CL-DBI internally uses CL-MYSQL to actually talk to the database; CL-MYSQl in particular, tries to convert all the DATE, DATETIME, and TIMESTAMP objects into CL [universal time](http://www.lispworks.com/documentation/lw51/CLHS/Body/26_glo_u.htm#universal_time), and it does it with the following function:
(defun string-to-date (string &optional len)
  (declare (optimize (speed 3) (safety 3))
          (type (or null simple-string) string)
          (type (or null fixnum) len))
  (when (and string (> (or len (length string)) 9))
    (let ((y (parse-integer string :start 0 :end 4))
    (m (parse-integer string :start 5 :end 7))
    (d (parse-integer string :start 8 :end 10)))
      (unless (or (zerop y)
                  (zerop m)
                  (zerop d))
        (encode-universal-time 0 0 0 d m y)))))
Let's apply this function to the value stored inside the database and see whta it returns:
(string-to-date **)
This is what MITO receives, before _inflating_ it; so let's _inflate_ this and see if we get back to where we started:
(local-time:universal-to-timestamp *)
What?!  That's what I thought too.

The first thing I tried was to change STRING-TO-DATE, and force a UTC offset when invoking ENCODE-UNIVERSAL-TIME:
(defun string-to-date (string &optional len)
  (declare (optimize (speed 3) (safety 3))
          (type (or null simple-string) string)
          (type (or null fixnum) len))
  (when (and string (> (or len (length string)) 9))
    (let ((y (parse-integer string :start 0 :end 4))
    (m (parse-integer string :start 5 :end 7))
    (d (parse-integer string :start 8 :end 10)))
      (unless (or (zerop y)
                  (zerop m)
                  (zerop d))
        (encode-universal-time 0 0 0 d m y 0)))))

(string-to-date "2020-07-28")

(local-time:universal-to-timestamp *)
This is better!  What happened? Basically SETF-ing `local-time:*default-timezone*` changes the behavior of the LOCAL-TIME library, but does not change how built-in date-related functions operate; which means that if the runtime thinks it's getting runned in CEST, then CEST is the time-zone it will try and apply unless instructed not not to.

Alright, it kind of makes sense, but how can we fix this?  Clearly we can't change CL-MYSQL and hard-code the UTC time-zone in there, as everyone should be free to use the library under whichever time-zone the like.  Enters the `TZ=UTC` environment variable.
$ TZ=UTC sbcl

(ql:quickload :local-time)

(setf local-time:*default-timezone* local-time:+utc-zone+)

(defvar date "2020-07-28")

(local-time:parse-timestring date)

(defvar *db-date-format*
  '((:year 4) #\- (:month 2) #\- (:day 2)))

(local-time:format-timestring nil **
                              :format *db-date-format*)

(defun string-to-date (string &optional len)
  (declare (optimize (speed 3) (safety 3))
          (type (or null simple-string) string)
          (type (or null fixnum) len))
  (when (and string (> (or len (length string)) 9))
    (let ((y (parse-integer string :start 0 :end 4))
    (m (parse-integer string :start 5 :end 7))
    (d (parse-integer string :start 8 :end 10)))
      (unless (or (zerop y)
                  (zerop m)
                  (zerop d))
        (encode-universal-time 0 0 0 d m y)))))

(string-to-date **)

(local-time:universal-to-timestamp *)
_Lovely time-zones..._

2020-07-18 (permalink)

? lispindent: backquote templates are not properly indented

This is the output:
(defmacro with-tansaction (conn &body body)
  `(let ((*connection* ,conn))
    (cl-dbi:with-transaction *connection*
While I would expect:
(defmacro with-tansaction (conn &body body)
  `(let ((*connection* ,conn))
     (cl-dbi:with-transaction *connection*
Note how `(cl-dbi:with-transaction ...)` and its body should be indented by an additional space

2020-07-17 (permalink)

* finished reading [Lisp Hackers](https://leanpub.com/lisphackers)

* finished reading [Full Stack Lisp](https://leanpub.com/fullstacklisp)

2020-07-15 (permalink)

TIL: `cal` can be instructed to display not just the current month, but also surrounding ones, with `-$N` (if `$N` is 3, `cal` will show the current month, the previous, and the next one):
$ cal -3
     June 2020             July 2020            August 2020
Su Mo Tu We Th Fr Sa  Su Mo Tu We Th Fr Sa  Su Mo Tu We Th Fr Sa
    1  2  3  4  5  6            1  2  3  4                     1
 7  8  9 10 11 12 13   5  6  7  8  9 10 11   2  3  4  5  6  7  8
14 15 16 17 18 19 20  12 13 14 15 16 17 18   9 10 11 12 13 14 15
21 22 23 24 25 26 27  19 20 21 22 23 24 25  16 17 18 19 20 21 22
28 29 30              26 27 28 29 30 31     23 24 25 26 27 28 29
                                            30 31
Source: https://drewdevault.com/2020/07/14/March-2nd-1943.html

2020-07-14 (permalink)

* finished reading [Lisp for the Web](https://leanpub.com/lispweb)

* finished reading [Lisp Web Tales](https://leanpub.com/lispwebtales)

RailsConf 2016 - Turbolinks 5: I Can’t Believe It’s Not Native! by Sam Stephenson: https://www.youtube.com/watch?v=SWEts0rlezA


- 18 months
- 200+ screens
- 5 platforms (desktop web, mobile web, android native, ios native, and email)
- 6 rails devs, 2 android, 2 ios
- built 3 open source frameworks: Trix (editor), Action Cable, and Turbolinks 5

Turbolinks for iOS (or Android):

- Intercepts link-click events, and creates a new view on the stack
- There is a single WebView only, and that's shared by all the windows on the stack (smaller memory footprint)
- You can customize the user agent, and on the backend for example, hide certain areas which would not look good on the APP
- Native (kind-of) app in 10 minutes
- You can go fully native on specific screens -- hybrid approach
- You can create native menus, by simply extracting metadata from the received page

2020-07-12 (permalink)

TIL: `git` fixups / autosquashes: https://fle.github.io/git-tip-keep-your-branch-clean-with-fixup-and-autosquash.html

* finished reading [Sails.js in Action 1st Edition](https://www.amazon.com/Sails-js-Action-Mike-McNeil/dp/1617292613)

2020-07-06 (permalink)

Rails Conf 2013 Patterns of Basecamp's Application Architecture by David Heinemeier Hansson: https://www.youtube.com/watch?v=yhseQP52yIY&list=PL01Ur3GaFSxxOQWCxypbFBqb2s11Yy_N0&index=8

HTML: document based programming, or delivery mechanism?  Google maps vs Blogs (and to some extend, SPAs vs server-side rendered applications, or hybrid applications)

To make things fast:

1. Key-based cache expiration (or generational caching)

  - Model name + id / "new" + last-updated-at (if available)

2. Russian Doll nested caching

  - Comment is inside a post, a post is inside a project
  - When you change a comment, the system automatically touches the parent, then the parent touches its parent... (where "touches" really means update the last-modified-date timestamp)
  - The MD5 of a partial, is included in the cache key, which means every time you update a partial, the new view will be rendered (and cached again); if the partial is not updated, the old, cached value would still be safe to use
  - Templates are inspected, and caches are invalidates based on `render` calls
  - What if you want to show local time?  You add UTC to the template, add `data-time-ago` attribute, inject some JavaScript on the client to display the utc value into local value
  - What if you have admin links (Edit, Delete, Copy, Move) which are only visible to a subset of users?  We cannot access the logged-in user when rendering/caching the view, so a similar solution to the problem above: annotate the link with `data-visible-to`, and then some JavaScript will take care of removing the link which is not visible

3. Turbolinks process persistence

  With Turbolinks, your app could be as fast as the backend is able to serve responses: 50/150 ms?  The experience is great; higher than that, and you are not going to notice much of a difference from a complete page-refresh.

4. Polling for updates

2020-07-04 (permalink)

Bit of vim g* niceties:

g8: show the ASCII code of the character under the cursor
g<: see the output of the last command
gF: got to file, and line number (./path/file.js:23) -- handy if you have a stack-trace in a buffer
gv: re-select the last selection
gi: go back to last insert location
gq: wrap command (e.g. break at the specified text-width)

2020-07-03 (permalink)

A little bit off-topic from the usual stuff I write about, but I just want to drop here some links to videos about DJ techniques (don't ask why I got into these):

* Strobing/Strobe Technique: https://www.youtube.com/watch?v=2TokF34LUtI

* Another video about Strobing/Chasing: https://www.youtube.com/watch?v=vXaq5gTHsME

* Back-spinning: https://www.youtube.com/watch?v=WTx34IsDIJU

* Finally the original DJ battle video that got me into this rabbit hole: https://www.youtube.com/watch?v=XymUZKLTjW4

TIL: Ansible won't re-create a container, when you remove one of its ENV variables -- and apparently, that's by design!

Apparently ansible would not recreate the container when ENV variables
are removed, because the condition of the remaining ENV variables still
being set would still hold true.


The work-around, is to inject a dummy ENV variable, and keep on updating
it's value every time we want to force ansible to recreate the container
(for whatever reason).

2020-07-01 (permalink)

? cb.vim: can it work like a standard register?

+ cb.vim: vim-fugitive: ["x]y<C-G> copies the current git-object, but how can I do the same with ,y and send it to cb?

2020-06-30 (permalink)

? ap: parsing errors are not loggedf to stdout ... and there is no exit status too

? book: the medici effect

? book: the innovator's dilemma

2020-06-20 (permalink)

$ curl wttr.in
Weather report: Pisa, Italy

    \  /       Partly cloudy
  _ /"".-.     20 °C
    \_(   ).   ← 6 km/h
    /(___(__)  10 km
               0.0 mm
┌──────────────────────────────┬───────────────────────┤  Sat 20 Jun ├───────────────────────┬──────────────────────────────┐
│            Morning           │             Noon      └──────┬──────┘     Evening           │             Night            │
│    \  /       Partly cloudy  │    \  /       Partly cloudy  │    \  /       Partly cloudy  │    \  /       Partly cloudy  │
│  _ /"".-.     22 °C          │  _ /"".-.     24..25 °C      │  _ /"".-.     22..25 °C      │  _ /"".-.     20 °C          │
│    \_(   ).   ↑ 6-8 km/h     │    \_(   ).   → 10-12 km/h   │    \_(   ).   → 12-15 km/h   │    \_(   ).   → 9-14 km/h    │
│    /(___(__)  10 km          │    /(___(__)  10 km          │    /(___(__)  10 km          │    /(___(__)  10 km          │
│               0.0 mm | 0%    │               0.0 mm | 0%    │               0.0 mm | 0%    │               0.0 mm | 0%    │
┌──────────────────────────────┬───────────────────────┤  Sun 21 Jun ├───────────────────────┬──────────────────────────────┐
│            Morning           │             Noon      └──────┬──────┘     Evening           │             Night            │
│    \  /       Partly cloudy  │    \  /       Partly cloudy  │    \  /       Partly cloudy  │    \  /       Partly cloudy  │
│  _ /"".-.     23..24 °C      │  _ /"".-.     25..26 °C      │  _ /"".-.     23..25 °C      │  _ /"".-.     22 °C          │
│    \_(   ).   ↑ 4-5 km/h     │    \_(   ).   → 9-10 km/h    │    \_(   ).   → 11-14 km/h   │    \_(   ).   ↘ 6-10 km/h    │
│    /(___(__)  10 km          │    /(___(__)  10 km          │    /(___(__)  10 km          │    /(___(__)  10 km          │
│               0.0 mm | 0%    │               0.0 mm | 0%    │               0.0 mm | 0%    │               0.0 mm | 0%    │
┌──────────────────────────────┬───────────────────────┤  Mon 22 Jun ├───────────────────────┬──────────────────────────────┐
│            Morning           │             Noon      └──────┬──────┘     Evening           │             Night            │
│    \  /       Partly cloudy  │    \  /       Partly cloudy  │    \  /       Partly cloudy  │    \  /       Partly cloudy  │
│  _ /"".-.     25..27 °C      │  _ /"".-.     27..28 °C      │  _ /"".-.     24..26 °C      │  _ /"".-.     22 °C          │
│    \_(   ).   ↑ 4-5 km/h     │    \_(   ).   → 8-10 km/h    │    \_(   ).   → 10-14 km/h   │    \_(   ).   ↘ 6-11 km/h    │
│    /(___(__)  10 km          │    /(___(__)  10 km          │    /(___(__)  10 km          │    /(___(__)  10 km          │
│               0.0 mm | 0%    │               0.0 mm | 0%    │               0.0 mm | 0%    │               0.0 mm | 0%    │
This is pretty dope!

2020-06-11 (permalink)

TIL: only the Quartz Java scheduler (i.e. the one used by Jenkins)[0] supports the H character to _evenly_ distribute jobs across a defined time range [1]

[0] https://en.wikipedia.org/wiki/Cron
[1] https://stackoverflow.com/a/47302727

2020-06-07 (permalink)

* Changed my `lispindent` for to read `.lispwords` from the current directory, from its parent, from its parent parent... until the process gets past the user home directory.

* scmindent: it leaves trailing empty spaces when keywords are surrounded with empty lines

* scmindent: skip commented out lines

* Created Vim plugin, vim-lispindent, to: 1. automatically `set equalprg` for lisp and scheme files, 2. parse `.lispwords` and 3. set Vim's `lispwords` accordingly

? vim-lispindent: download `lispindent` if not found on $PATH

~ vim-lispindent: support rules with LIN = 0 (we want to remove these from `lispwords`)

? vim-lispindent: README + doc/

? cg: pass the last executed command (e.g '$!') and use that to _influence_ the order of suggested commands to execute

? scmindent: cases inside `HANDLER-CASE` are not _properly_ indented

2020-06-01 (permalink)

Been finally giving [vim-sexp](https://github.com/guns/vim-sexp) a try, and I gotta be honest, I am quite liking it.

I have also been reading about _structural code editing_, and landed on this [SO page](https://stackoverflow.com/a/30178499) about _barfing_ and _slurping_:
slurping and barfing are the essential operations/concepts to use one of the modern structural code editors. after getting used to them I'm completely incapable of editing code without these. Of the ~20 people that sit with me writing clojure all day all of them use these all the time. so saying that they are "helpful for lisp coders" is a very tactful and polite understatement.

slurp: to include the item to one side of the expression surrounding the point into the expression

barf: to exclude either the left most or right most item in the expression surrounding the point from the expression
That's an alright definition, but when would you actually use it?  Another [answer](https://stackoverflow.com/a/30190612) to the same question tries to shed some light on this:
so a session might be:

    (some-fn1 (count some-map))
    (some-fn2 (count some-map))

aha, a let could come in here to refactor the (count some-map):

    (let [c (count some-map)]|)
    (some-fn1 c)
    (some-fn2 c)

But the let isn't wrapping the 2 calls, so we want to pull in (slurp) the next 2 forms inside the let s-exp, so now at the cursor position, slurp twice which will give after first:

    (let [c (count some-map)]
      (some-fn1 c)|)
    (some-fn2 c)

and then on second:

    (let [c (count some-map)]
      (some-fn1 c)
      (some-fn2 c)|)
It all starts to make sense.  Let's now have a look at Tpope's [vim-sexp-mappings](https://github.com/tpope/vim-sexp-mappings-for-regular-people), and in particular what he thought some sane mappings should be for slurpage and barfage:
" Slurping/Barfing
" angle bracket indicates the direction, and the parenthesis
" indicates which end to operate on.
nmap <silent><buffer>   <(   <Plug>(sexp_capture_prev_element)
nmap <silent><buffer>   >)   <Plug>(sexp_capture_next_element)
nmap <silent><buffer>   >(   <Plug>(sexp_emit_head_element)
nmap <silent><buffer>   <)   <Plug>(sexp_emit_tail_element)
So going back the example above where the cursor was on the outermost closed paren:
    (let [c (count some-map)]|)
    (some-fn1 c)
    (some-fn2 c)
To slurp the next expression in, you would end up pushing the closing paren the cursor is on to right (past to the next element), and the chosen mapping, `>)`, really tries to caption that...

...again it all starts to make sense...

* Changed `lispindent` to support reading ./.lispwords: https://github.com/ds26gte/scmindent/issues/7

* Changed `lispindent` to properly format FLET, MACROLET, and LABELS forms: https://github.com/ds26gte/scmindent/issues/8

2020-05-31 (permalink)

? add a Vim operator for neoformat: this requires the current formatter to support reformatting of partial areas, in which case, one could surround the area to re-format with markers (e.g. comments, that the formatter is likely to ignore), and then use these markers to extract the formatted area of interest and stick it back into the buffer

2020-05-29 (permalink)

~ small utility to create RSS feed URL given another URL (i.e. give it a YouTube URL, and it will generate a feed for it; give it a GitHub URL, and it will generate a feed for it)

2020-05-25 (permalink)

On indenting lisp files with Vim.

First things first, Vim has Lisp mode (`:h lisp`), an that alone should give you good enough indentation for lisp files.
au FileType lisp setlocal lisp
However, Vim's default way of indenting s-expressions is not always perfect, so if you happen to use [Vlime](https://github.com/vlime/vlime) (and if you don't, I highly reccommend that you do), then you can switch to Vlime's `indentexpr` by simply disabling lisp mode (the plugin will do the rest):
" Weirdly, you have to set nolisp for (the correctly working) indentexpr to work.
" If lisp (the vim option) is set, then it overrules indentexpr.
" Since an indentexpr is set by vlime (and working!) it would be great to add nolisp to the default configuration, I guess.
" https://github.com/l04m33/vlime/issues/26#issuecomment-343761050
au FileType lisp setlocal nolisp
Lastly, you can go back to using lisp mode, but tell Vim to use an external program like [scmindent](https://github.com/ds26gte/scmindent) to invoke when indenting s-expressions (make sure `lispindent` is in your `$PATH`):
au FileType lisp setlocal lisp
au FileType lisp setlocal equalprg=lispindent
You might prefer this last approach more, especially when collaborating with others, as it does not rely on a specific conbination of editor / plugin, but on an external tool that others can easily configure in their workflow.

Few final words about Vim `lispwords` (`:h lisowords`), and particular a question that has bugged me for quite some time: do you need to set them up with using Vlime's `indentexpr`, or an external program to indent your lisp files?  Yes, especially if you care for Vim properly indenting s-expressions as you type them: `indentexpr` is only used when in conjunction with the `=` operator (i.e. when manually indenting blocks of code), while `lispwords` are used when typing in new code.

2020-05-24 (permalink)

Random quote from a podcast I recently listened to:
A lot of people say, just do a support model... and I think everybody in the business room looks at this and laughs... because you don't trade time for money... only poor people do that... rich people, they trade products for money.

2020-05-23 (permalink)

Simplified Poor's man yankstack implementation to to use `localtime()` instead of timers.

The idea is pretty simple, everytime you invoke it:

- Take a timestamp
- Check that timestamp another one (buffere variable) representing when the function was last executed
- If the two timestamps are more than 5 seconds apart, it means the user wants to start from register 0, otherwise 1, 2, 3...

Change SendToTerminal (STT), to strip out command prompts (if configured)

Sometimes you just want to _send_ a line like the following one, without worrying about stripping the `> ` bit out of it.
$ fortune
So I changed STT to automatically strip it out for me!

2020-05-22 (permalink)

? book: tools for thought

2020-05-21 (permalink)

Fucking fixed bash completion for `g`, my `git` alias.
# For God knows what reason, on Linux, `git` completion functions would not be
# loaded until you manually trigger them:
#   > git <tab>
# So here we manually source the file, if available
if [ -f /usr/share/bash-completion/completions/git ]; then
  source /usr/share/bash-completion/completions/git
function g() { git "$@"; }
__git_complete g __git_main

2020-05-19 (permalink)

* Finally added a new page on my website, to talk about .plan files. Check it out: https://matteolandi.net/plan.html

2020-05-16 (permalink)

Alright, I think I figured out what I had to do in order to get my CL programs, on Window, to properly output stuff to stdout.  It turns out you have to explicitly tell deploy[1], the system I use do deploy standalone CL applications, that you want your Windwos application to be a _console_ application, or otherwise deploy will go with a GUI application (and GUI applications that don't usually get a standard output to write to).

To do so, we will have to set the feature flag `:deploy-console`, before building the application:
(pushnew :deploy-console *features*)
(let ((deploy:*status-output* t))
  (asdf:make :cg :force t))
And that's it, your '.exe' program will finally print stuff to stdout, irrespective of the terminal you run it from:
mlandi at pisa299 in /home/mlandi
$ cg --help
Guess commands to run from stdin, and print them to stdout.

Available options:
  -h, --help               print the help text and exit
  -v, --version            print the version and exit
  -d, --debug              parse the RC file and exit
[1] https://github.com/Shinmera/deploy

Run into another problem with Travis-CI the other day; when deploying artifacts to GitHub, it would error out with the following message:
/home/travis/.travis/functions: eval: line 109: unexpected EOF while looking for matching ``'
/home/travis/.travis/functions: eval: line 110: syntax error: unexpected end of file
After a bit of digging I finally found this `dpl` issue [1], where the OP says that there were some problems with his password/token; that made me think about the token I had used, and whether I got it right or not (I generate tokens with `cb | travis encrypt`, so what if I accidentally copied something else inthe OS clipboard, overriding GitHub's code?).

The new token turned out to be different from what I had inside my .travis.yml file, so yeah...yet another PEBCAK (even though, the guys at Travis-CI could have logged a better error message!)

[1] https://github.com/travis-ci/dpl/issues/634
[2] https://en.wikipedia.org/wiki/User_error#Acronyms_and_other_names

2020-05-12 (permalink)

Did a little bit of debugging as to why **all** my Common Lisp Windows executables were not printing things out on the stdout as expected, and it seems to have something to do with redirection...
mlandi at pisa299 in /home/mlandi
$ cg --help

mlandi at pisa299 in /home/mlandi
$ cg --help | cat
Guess commands to run from stdin, and print them to stdout.

Available options:
  -h, --help               print the help text and exit
  -v, --version            print the version and exit
  -d, --debug              parse the RC file and exit
I am not sure what's going on here... but at least I narrowed down the problem a bit.  I will try something else during the following days, but I might wind up asking directly to the cygwin mailing list.

2020-05-05 (permalink)

Been plaing with `ap` lately, trying to find a _good-enough_ heuristic, but it's really hard.  I even tried with implementing a depth-first-search algorithm, and even though with pruning it converges to a good-enough solution pretty quickly, it won't ever reach the optimum (the search space is simply too vaste).  The solution?  Add a command line switch, `--good-enough`, to tell `ap` to settle for a sub-optimal solution, and to do it ASAP.

To recap:

- Optimal solution: A* without heuristic (might take forever, or even blow up)
- Good enough solution -- best between

  - A* with very naive heuristic (one that can easily overestimate the remaining cost)
  - Bounded DFS -- i.e. one that will stop after having evaluated 50000 states

* finished reading [Shape Up: Stop Running in Circles and Ship Work that Matters](https://basecamp.com/shapeup)

2020-04-24 (permalink)

Previously I had the following in my `.gitconfig`:
    b = branch -v
Now, and after a bit of fiddling around, I have:
    b = "!f() { git branch --color --verbose | tr -d '\\r'; }; f"
Why?  Because GitLab merge commits can contain `^M` caracters, and those can fuck up the output of `git branch --verbose`.

The solution?  Manually strip carriage returns escapes, and forcibly enable colours -- `git` knows it's output is being piped, so it would turn colors off by default.

2020-04-21 (permalink)

This `docker` trick will never get old:
$ docker run --rm -i --name bash-rm -v $(pwd):/pwd-host bash rm ...

2020-04-19 (permalink)

Finally spent some time trying to figure out a solution for `ap` claiming activities too _early_ (i.e. before all their dependencies were satisfied).

I wound up keeping track of when each activity is completed; this way, when worker `w` who has been busy for `t` time tries to claim activity `a`, the algo would check and confirm that all `a` dependencies have been completed before `t` (and if not, it would skip the activity and move to the next one).

It's a bit inefficient at the moment (well, the whole codebase leaves a lot to be desired) but before cleaning that up I need first to figure out a good heuristic; and if I cannot find one, I might as well change how search state are represented, or decide to abandon the project altogether.

Let's see.

2020-04-18 (permalink)

Wanted to try and setup some _automatic checkins_ for the team; as people work remotely, I would like to experiment with reducing the number of scheduled meetings, but before implementing that and say, kill every daily standup and have people write their daily status update, I thought we should first give people a place where they could talk about anything, and stay up to date with each other lives (if they want).  So rather than start asking people what they will be working on during the coming week, I thought we should start with something lighter, something social, something like: How was your weeked like?

The task is quite easy: every Monday morning at 8am, have Jenkins fire an email (later published on Teams) to ask people: what have you guys done over the weekend?  People into sharing, would then reply to the thread, or simply read what others have been up to.  Nothing creazy, nothing creepy, just a bit of sharing to make people we work with a bit more interesting.

All right, let's talk implementation:

every Monday morning at 8am

We have people working from: Jakarta, Noida, Hyderabad, Pisa, Scotland, and New York.  So what that 8am really means here?  Well, ideally we would want people to process this as soon as they get in (i.e. skim through inbox / Teams to see if there is anything critical, then spend a few minutes _sharing_), so the notification needs to be there when Jakarta people get in.

have Jenkins

Jenkins lets you easily schedule when jobs need to run (Build Triggers > Build periodically > Schedule), and it supports custom timezones too:
# Every Monday morning, at 8am
* 0 8 * * 0

fire an email ...

Nothing could have been easier: on a Jenkins job configuration page, Build > Add build step > Execute shell, then copy the following into the 'Command' field (replace `$TO_EMAIL` with the email address associated with the specific MSFT Teams channel):
#!/bin/bash -ex

/bin/mutt -s 'How was your weekend?' $TO_EMAIL <<EOF
Reply to this thread and let the others know what have you been up to.
And that's it, really!

2020-04-16 (permalink)

Source: https://twitter.com/_tessr/status/626076327133577216
how to remember tar flags 🎏

tar -xzf 👈 eXtract Ze Files!
tar -czf 👈 Compress Ze Files!
This never gets old!

2020-04-13 (permalink)

Flipped the order of notes in my .plan file...again!

The main reason why switched from _most-recent-last_ to _most-recent-first_ last August, was that I thought that would make it easier for a casual reader to access the most recent entry; however, that made it really difficult for someone, to dump the whole file and read it in chronological order.

Anyway, now that I can create a Feed RSS off my .plan file (see: https://matteolandi.net/plan.xml), there no more need for adding new notes at the top of my .plan file, so I decided to flip the order of my notes to _most-recent-last_.

This is what I last used last August, to reverse the order of all the notes:
" Select all the end of lines *not* preceding a new day note
/\v\n(# [0-9]{4})@!
" Change all the end of lines with a § -- unlikely to use character
" Sort all the collapsed notes (! is for reverse :help sort)
" Change § back to \r
It still does not properly support the scenario where a `§` character is being used in a note, but I checked  my plan file before running it, and it's going to be all right.

* finished reading [It Doesn't Have to Be Crazy at Work](https://www.amazon.com/Doesnt-Have-Be-Crazy-Work/dp/0062874780/)

2020-04-12 (permalink)

* finished reading [The 4-Hour Work Week: Escape the 9-5, Live Anywhere and Join the New Rich](https://www.amazon.com/4-Hour-Workweek-Expanded-Updated-Cutting-Edge-ebook/dp/B002WE46UW)

2020-04-11 (permalink)

Ode to [checklists](https://medium.com/javascript-scene/managing-remote-teams-use-checklists-301aae93e7a5):

Good checklists are:

- Short enough to memorize
- Only include the keypoints

For example (quite relevant these days):

- Use soap
- Wet both hands completely
- Rub the soap until it forms a thick lather covering both hands completely
- Wash hands for at least 20 seconds [not included in this study, but we know now it takes at least 20 seconds to break down viral bugs including Coronavirus so I’m putting it here for posterity]
- Completely rinse the soap off

Other examples, closer to the software development industry

Before asking peers to review your pull request, make sure that:

- [ ] Successfully tested on Jenkins
- [ ] Product team has signed off the implemented workflow
- [ ] Bonus: Screenshots/screencast demo included

Before approving a pull request and merging that into master, make sure that:

- [ ] Code is readable
- [ ] Code is tested
- [ ] The features are documented
- [ ] Files are located and named correctly
- [ ] Error states are properly handled
- [ ] Successfully tested on Jenkins

Before pushing a new version out, make sure that:

- [ ] Look into failing tests (if any), and confirm if intermittent
- [ ] Announce on whichever social media you are using, the main features / bugfixes / changes being released

After a successful release, make sure that:

- [ ] Do some smoke-test to confirm it's all good
- [ ] Run any post-release action (e.g. database migration, if not automatic, support tickets updated)
- [ ] Announce on social media that the release has completed successfully
- [ ] Archive completed stories

When adding a new Defefct to the backlog, make sure it includes:

- [ ] Description
- [ ] Steps to reproduce
- [ ] Expected behavior
- [ ] Actual behabior
- [ ] Bonus: Screenshot/video demonstrating the bug
- [ ] Bonus: any additional pre-analysis

I believe we can go on forever, but this seems promising.

2020-04-08 (permalink)

Is it possible to implement Basecamp's automatic check-ins with Teams?  I think it is, with some duct tape of course.

- Each Teams channel has an email address associated with it; you send an email to it, and a few seconds later that message would pop up on teams
- A Jenkins job can be configured to run periodically (e.g. every day @ 9am)
- You can can easily send email from the command line as follows: `echo 'body' | mutt -s 'Subject' your.address@foo.com`


1. Gather your Microsoft Teams' channel email
2. Set up your script:
#!/bin/bash -ex

/bin/mutt -s 'Automatic check-in: What will you be working on this week?' $TO_EMAIL <<EOF
Reply to this thread and let the others know what your plan for the week is.

- Things change, so don't worry if you wind up working on something completely different
- Few bullet points, a poem, a mockup, anything that might help you sending the message across
- That's it
3. Tell Jenkins to run this every Monday morning, at 9am

And that's it!  Everyone, every Monday morning will have a new notification in the specified channel, and could inform the others about their plans for the week.

? tmux / macos: when piping a pane into a program, trailing whitespaces are added to some of the lines (who is generating them? is it the command that generated the output, is it tmux itself?)

? ap: implement person-activity preferences<