lbatista 3 days ago

Convolutional Neural Networks in APL

  blog←{⍺×⍵×1-⍵}
  backbias←{+/,⍵}
  logistic←{÷1+*-⍵}
  maxpos←{(,⍵)⍳⌈/,⍵}
  backavgpool←{2⌿2/⍵÷4}⍤2
  meansqerr←{÷∘2+/,(⍺-⍵)*2}
  avgpool←{÷∘4{+/,⍵}⌺(2 2⍴2)⍤2⊢⍵}
  conv←{s←1+(⍴⍵)-⍴⍺⋄⊃+/,⍺×(⍳⍴⍺){s↑⍺↓⍵}  ⊂⍵}
  backin←{(d w in)←⍵⋄⊃+/,w{(⍴in)↑(-⍵+⍴d)↑⍺×d}  ⍳⍴w}
  multiconv←{(a ws bs)←⍵⋄bs{⍺+⍵ conv a}⍤(0,(⍴⍴a))⊢ws}
https://dl.acm.org/doi/pdf/10.1145/3315454.3329960
  • dbcurtis 3 days ago

    A friend once described APL as a “write-only language”. You make his case well :)

    It is pretty easy to write unmaintainable APL, it seems to me.

    • mlochbaum 3 days ago

      These are written in a generally basic and clean style (avoiding tacit programming which is sometimes considered hard to understand, e.g. function {s↑⍺↓⍵} instead of the train (s↑↓)). They're nice to read and I'd have no trouble maintaining them. You just don't know the language.

      conv looks like the hardest. s←1+(⍴⍵)-⍴⍺ is the result shape, number of subarrays with the length of ⍺ that will fit in ⍵ in each dimension. Looks like (⍳⍴⍺){s↑⍺↓⍵}¨⊂⍵ is missing the ¨; the inner function s↑⍺↓⍵ drops ⍺ elements (left argument) and then takes the first s, so it gets a length-s window of ⍵. This is called on each possible index into ⍺, together with the whole of ⍵, so it ends up getting all such windows. Presumably s is expected to be larger than ⍴⍺ so this is the more efficient way to slice things. ⊃+/,⍺× multiplies by ⍺ and sums, ravelling with , before applying +/ to collapse the dimensions and sum them all at once. Each element is an array of shape s, so summing them gives a result of shape s. There you go, multidimensional convolution!

      • lbatista 3 days ago

        Thanks for pointing the missing glyph. I will paste the correct code.

          backbias←{+/,⍵}
          logistic←{÷1+*-⍵}
          maxpos←{(,⍵)⍳⌈/,⍵}
          backavgpool←{2⌿2/⍵÷4}⍤2
          meansqerr←{÷∘2+/,(⍺-⍵)*2}
          avgpool←{÷∘4{+/,⍵}⌺(2 2⍴2)⍤2⊢⍵}
          conv←{s←1+(⍴⍵)-⍴⍺⋄⊃+/,⍺×(⍳⍴⍺){s↑⍺↓⍵}¨⊂⍵}
          backin←{(d w in)←⍵⋄⊃+/,w{(⍴in)↑(-⍵+⍴d)↑⍺×d}¨⍳⍴w}
          multiconv←{(a ws bs)←⍵⋄bs{⍺+⍵ conv a}⍤(0,(⍴⍴a))⊢ws}
    • 7thaccount 3 days ago

      I'd bet money that I could figure out what this code is doing easier than the equivalent in Python, which is my daily driver. You just need to look up the symbols and piece it together. Yes, that takes an extra step, but so did learning any new language. Also, I can see all of this in one spot.

      • RodgerTheGreat 3 days ago

        The value of being able to see the entire implementation of a nontrivial program at once cannot be overstated. This is the real magic of APLs: no unnecessary abstraction, boilerplate, or structural fluff, just algorithms composed from general high-level building-blocks.

        • rbanffy 2 days ago

          The right level of abstraction is key. If your problem maps neatly to the primitives offered by APL, then the code will be readable idiomatic APL. If, however, they don’t, and you are using the APL abstractions in ways they were not intended to, and you are building new abstractions on top of them, then you’ll need to be a skilled programmer to keep it readable.

    • xelxebar 3 days ago

      Hard disagree. I wrote a YAML parser in APL[0], set it aside for 8 months, and then one day decided to read through it while getting a haircut. I can happily report that the full 22 lines (with zero library deps) were very grokkable. In fact, I even found a bug and now have several ideas for a rearchitecture I'd like to work on.

      APL just looks unfamiliar. Don't confuse that with unreadability. IMHO, the ergonomics for expressing and communicating high-level specifications just blows other languages out of the water. And those specifications also happen to also be executable implementations with leading-edge performance to boot.

      Admittedly, though, the learning curve is painfully steep. I think it's worth it though.

      [0]:https://github.com/xelxebar/dayaml

    • userbinator 3 days ago

      Would you say the same about Chinese, Japanese, Korean, or some other non-Latin-script human language?

      Just because it's not some C-derivative language doesn't mean there aren't those who can read and write it well.

      • dbcurtis 3 days ago

        No, not that at all. It is more a reflection of APL’s extreme density. C-derivatives tend to devolve into ASCII-salad, which is no better.

        Also, with APL, the code for something that can be expressed in linear algebra is reasonably natural, but when APL gets used to code other things it can be hard to reverse out what the author was thinking when they found a linear algebra expression of a problem for which that is not a natural mapping.

    • redrobein 2 days ago

      Hate to quote theprimeagen on this but people have to stop mixing up readability with familiarity.

    • JKCalhoun 3 days ago

      > A friend once described APL as a “write-only language”.

      They meant regex.

      • MarkusWandel 2 days ago

        And IBM script, and troff, and vi macros, and...

        What these all have in common is that the computer wasn't that clever (yet) compared to humans, so it was worth learning a cryptic but efficient (in terms of code size, or execution speed) language. Or maybe these "line noise" languages were just fashionable.

        Now, JIT-ing something eminently readable is essentially free, so there is no more point in these old things (I don't personally use Matlab, but all the theory people at work do, and it seems to be the spiritual successor to APL).

    • chewxy 2 days ago

      Not really, this is actually pretty readable

  • hello_computer 3 days ago

    This is how it should have been done to begin with.

  • kragen 2 days ago

    this won't work in 01975 apl

dang 3 days ago

Related. Others?

APL Demonstration (1975) [video] - https://news.ycombinator.com/item?id=32118279 - July 2022 (1 comment)

APL Demonstration (1975) [video] - https://news.ycombinator.com/item?id=21104698 - Sept 2019 (51 comments)

APL Demonstration From 1975 [video] - https://news.ycombinator.com/item?id=7817593 - May 2014 (7 comments)

APL demonstration 1975 [video] - https://news.ycombinator.com/item?id=6155639 - Aug 2013 (1 comment

meken 3 days ago

That’s so weird how he backspaces to overlay two characters on top of one another to achieve an operator

  • PhilipRoman 3 days ago

    Apparently this used to be a common thing, groff has a feature to output bold text for TTY devices by backspacing and re-printing each character twice. You can get underlined text in a similar way using backpace and underscore. It doesn't work out of the box on terminal emulators, but less(1) does this rendering for man pages, etc.

  • sevensor 2 days ago

    That's how we did it with typewriters, back in the day. You could backspace and overstrike with another character, to make a composite symbol.

julienchastang 3 days ago

I've watched this video in the past. If I recall correctly, he does not make a single typo or fat-fingering during the entire 27 minutes.

  • llm_trw 3 days ago

    When a typo means you've waster 24 hours waiting for the batch job to come back (if you're lucky) you learn pretty quickly not to make them.

  • ziml77 3 days ago

    I think that's largely because he wasn't typing very fast and was looking at the keys.