To Factor and Back Again

Warning: bad Factor code contained within.

As an escape from AI, I recently started programming in Factor. As well as being a very interesting departure from C-like languages, AI kind of sucks at writing in it - keeping me from reaching for that crutch.

After messing around a little bit, getting used to the syntax (which was much easier than I expected thanks to its simplicity and the resulting consistency), I decided upon a first project: I would resurrect this blog, once powered by Jekyll, with a custom generator.

The first hurdle

No package manager… and no markdown package.

So… I sat down and started writing a parser from scratch. This was a good exercise, and it was during this that the concatenative style really clicked for me. One eureka moment was realising how much more readable and expressive code could be when following the sorts of functional patterns we often use in Swift: map, reduce, etc (I had initially reached for recursion 🤷‍♂️).

:: parse-md-line ( string -- md-line )
  ! ( -- line-type slice )
  all-line-types
  [
    line-type-pattern string
    swap <regexp>
    first-match
  ]
  map-find
  swap
  
  ! ( slice -- string )
  1array missing-slices
  "" join

  md-line boa
;

Rescued by the standard library

While Factor doesn’t have a package manager, what it does have is a great standard library.

After beginning to get frustrated by my lack of progress, I discovered the fantastic EBNF package after watching Factor: an extensible interactive language by Factor’s creator, Slava Pestov. With this vocab (library) at my disposal, I binned all my work up to this point, and started again 🙂. In a fraction of the time I had spent on the previous version I had ~90% of a parser.

EBNF: markdown [=[
  text = [^*_[\n]+
    => [[ >string text swap 2array ]]
  char = [^\n]
    => [[ 1string text swap 2array ]]
  link-text = [^*_[\]\n]+
    => [[ >string text swap 2array ]]
  link-char = [^\]\n]
    => [[ 1string text swap 2array ]]
  bold = "*"~ (!("*") inline)+:xs "*"~
    => [[ xs >array bold swap 2array ]]
  italic = "_"~ (!("_") inline)+:xs "_"~
    => [[ xs >array italic swap 2array ]]
  link-inline = bold | italic | link-text | link-char
  link = "["~ link-inline+:label "]"~ "("~ [^)]+:url ")"~
    => [[ label >array url >string link -rot 3array ]]
  inline = bold | italic | link | text | char
  h1 = "#"~ " "+~ [^\n]+:t "\n"~
    => [[ t >string h1 swap 2array ]]
  h2 = "##"~ " "+~ [^\n]+:t "\n"~
    => [[ t >string h2 swap 2array ]]
  h3 = "###"~ " "+~ [^\n]+:t "\n"~
    => [[ t >string h3 swap 2array ]]
  ulitem = "*"~ " "+~ inline+:xs "\n"~
    => [[ xs >array ul-item swap 2array ]]
  olitem = [0-9]+~ "."~ " "+~ inline+:xs "\n"~
    => [[ xs >array ol-item swap 2array ]]
  list = (ulitem|olitem)+
    => [[ >array list swap 2array ]]
  paragraph = inline+:xs "\n"~
    => [[ xs >array paragraph swap 2array ]]
  blank = " "*~ "\n"~
    => [[ ignore ]]
  header = h1 | h2 | h3
  block = header | list | paragraph
  document = (blank|block)+
    => [[ [ ignore = ] reject >array ]]
]=]

For templating, I simply took advantage of Factor’s ability to dynamically evaluate a string of Factor code. Strings wrapped in %% become quotations (closures) that are expected to push a string onto the stack when called (returning a string). Post metadata uses %%%, whose quotation instead relies on side effects to set metadata and leaves the stack unchanged.

USING: kernel splitting parser strings math sequences ;
IN: template.parser

<PRIVATE

: parse-quotation ( slice -- quotation )
  >string
  split-lines
  parse-lines
;

: quote-text ( slice -- quotation )
  >string [ ] curry
;

: parse-body ( string -- blocks )
  "%%" split-subseq
  [
    2 mod 0 = [
      quote-text
    ] [
      parse-quotation
    ] if
  ] map-index
;

PRIVATE>

TUPLE: template meta body ;

: parse-template ( string -- template )
  "%%%\n" split1
  dup [ swap ] unless
  parse-body
  [
    dup [ parse-quotation ] when
  ] dip
  template boa
;

This post was rendered by Ruby

Factor is incredibly powerful. A quotation is just a collection of words (functions), providing similar flexibility to that of Lisp’s homoiconicity. Coupled with its seamless support for macros - it’s a language which will be whatever you want it to be.

Factor is beautiful. I realised during the project that its concatenative style is a purer, cleaner way of expressing some of my favourite patterns in Swift and PowerShell: method chaining and piping. Being point-free, it’s physically quicker to make changes to your ‘chains’ than either of those two languages.

The tooling is amazing. You’re provided with one of the best REPLs I’ve ever used, and are encouraged to conduct all your development through it: scaffold modules and tests, play with and edit code, search documentation. I haven’t done much ‘REPL-driven programming’ before, but the listener has converted me.

It’s also fast.

However…

You could make most of these points about Ruby too… maybe not about speed 🙂, but certainly power, its metaprogramming capabilities placing it firmly in the same league as Factor - Ruby’s power is just derived from its Smalltalk lineage rather than from Forth (Factor has great OO facilities too - but they don’t feel as seamless as Ruby’s to me).

I was also becoming concerned about Forth/Factor’s reputation as being ‘write-only’ languages. Readability would likely improve with experience, but even coming back to code I had written the previous evening could prove difficult. Thinking Forth presents Forth’s stack and point-free style as an aid to modularity:

Forth eliminates from our programs the details of how words are invoked and how data are passed. What’s left? Only the words that describe our problem.

But the same coupling to arguments in C-like languages’ functions exists here too, it’s just hidden. Forth words are coupled to the stack order - the order of their positional arguments. I believe the more explicit coupling allowed in Ruby, of non-positional argument names, is more accommodating to change and re-use.

In the end

I tired of fixing edge cases in my parser, and the allure of Ruby got the better of me (or perhaps I’m just fickle). I grabbed Kramdown and ERB. Much less work… after spending an afternoon configuring Pry and Neovim to provide a fraction of the listener experience 😅.

I found myself omitting a lot of brackets.

(much to Rubocop’s disgust)