Inner and Outer products

This part will cover

• Inner products
• Outer products
• Matrix inverse

---

Recall the Rank ⍤ operator which specified what rank cells a function acts on,

      M ← ? 5 5 ⍴ 10
3 2  2 9 5
9 2 10 8 2
3 4  6 5 2
9 3  5 3 2
8 8  6 8 8
      (⊂⍤2)M
┌──────────┐
│3 2  2 9 5│
│9 2 10 8 2│
│3 4  6 5 2│
│9 3  5 3 2│
│8 8  6 8 8│
└──────────┘
      (⊂⍤1)M
┌─────────┬──────────┬─────────┬─────────┬─────────┐
│3 2 2 9 5│9 2 10 8 2│3 4 6 5 2│9 3 5 3 2│8 8 6 8 8│
└─────────┴──────────┴─────────┴─────────┴─────────┘      
      (⊂⍤0)M
3 2  2 9 5
9 2 10 8 2
3 4  6 5 2
9 3  5 3 2
8 8  6 8 8

As can be seen above, a rank-n cell of a rank-r array obtained by specifying the first (r-n) indices of the array.

      M[;]
3 2  2 9 5
9 2 10 8 2
3 4  6 5 2
9 3  5 3 2
8 8  6 8 8
      M[1;]
3 2 2 9 5
      M[1;1]
3

This specifies the cells to act on monadically; for dyadic functions, two ranks must be specified, to be interpreted as matching up cells of a certain rank from the left argument with certain cells of the right argument.

      v ← ? 5 ⍴ 5
      v
3 4 4 3 2
      M ← 5 5 ⍴ ⍳25
      M
 1  2  3  4  5
 6  7  8  9 10
11 12 13 14 15
16 17 18 19 20
21 22 23 24 25

      ⍝ See pairing of 0-cells of v and 1-cells of M
      v({⍺,'-',⍵}⍤(0 1))M
3 -  1  2  3  4  5
4 -  6  7  8  9 10
4 - 11 12 13 14 15
3 - 16 17 18 19 20
2 - 21 22 23 24 25

      ⍝ Pairing of 1-cells of v and 1-cells of M
      v({⍺,'-',⍵}⍤(1 1))M
3 4 4 3 2 -  1  2  3  4  5
3 4 4 3 2 -  6  7  8  9 10
3 4 4 3 2 - 11 12 13 14 15
3 4 4 3 2 - 16 17 18 19 20
3 4 4 3 2 - 21 22 23 24 25

      N ← 5 5 ⍴ +\⍳25
      N
  1   3   6  10  15
 21  28  36  45  55
 66  78  91 105 120
136 153 171 190 210
231 253 276 300 325

      ⍝ Pairing of 1-cells of N and 1-cells of M
      N({⍺,'-',⍵}⍤(1 1))M
  1   3   6  10  15 -  1  2  3  4  5
 21  28  36  45  55 -  6  7  8  9 10
 66  78  91 105 120 - 11 12 13 14 15
136 153 171 190 210 - 16 17 18 19 20
231 253 276 300 325 - 21 22 23 24 25

One commonly used operation is matching every pair of elements from two vectors, one way to write this is using each and rank ⍤

      v1 ← ? 5 ⍴ 5
5 5 3 4 1
      v2 ← ⎕A[? 5 ⍴ 26]
HTOKY
      v1(,⍤(0 1))v2
5 HTOKY
5 HTOKY
3 HTOKY
4 HTOKY
1 HTOKY
      v1(,¨⍤(0 1))v2
┌───┬───┬───┬───┬───┐
│5 H│5 T│5 O│5 K│5 Y│
├───┼───┼───┼───┼───┤
│5 H│5 T│5 O│5 K│5 Y│
├───┼───┼───┼───┼───┤
│3 H│3 T│3 O│3 K│3 Y│
├───┼───┼───┼───┼───┤
│4 H│4 T│4 O│4 K│4 Y│
├───┼───┼───┼───┼───┤
│1 H│1 T│1 O│1 K│1 Y│
└───┴───┴───┴───┴───┘

There is a specific operator for this operation called the outer product (∘.f), this operator is special in that it takes a right function argument.

      v1(∘.,)v2
┌───┬───┬───┬───┬───┐
│5 H│5 T│5 O│5 K│5 Y│
├───┼───┼───┼───┼───┤
│5 H│5 T│5 O│5 K│5 Y│
├───┼───┼───┼───┼───┤
│3 H│3 T│3 O│3 K│3 Y│
├───┼───┼───┼───┼───┤
│4 H│4 T│4 O│4 K│4 Y│
├───┼───┼───┼───┼───┤
│1 H│1 T│1 O│1 K│1 Y│
└───┴───┴───┴───┴───┘

      'POP' 'HEAVY' 'ALT' 'SYNTH' ∘., 'ROCK' 'METAL' 'PUNK' 'WAVE'
┌─────────┬──────────┬─────────┬─────────┐
│POPROCK  │POPMETAL  │POPPUNK  │POPWAVE  │
├─────────┼──────────┼─────────┼─────────┤
│HEAVYROCK│HEAVYMETAL│HEAVYPUNK│HEAVYWAVE│
├─────────┼──────────┼─────────┼─────────┤
│ALTROCK  │ALTMETAL  │ALTPUNK  │ALTWAVE  │
├─────────┼──────────┼─────────┼─────────┤
│SYNTHROCK│SYNTHMETAL│SYNTHPUNK│SYNTHWAVE│
└─────────┴──────────┴─────────┴─────────┘

For higher rank arrays, the outer product works similarly, matching each element of both arrays

      ⍳2 2
┌───┬───┐
│1 1│1 2│
├───┼───┤
│2 1│2 2│
└───┴───┘

      ((⍳2 2)⌷¨¨⊂⊂'AB')
┌──┬──┐
│AA│AB│
├──┼──┤
│BA│BB│
└──┴──┘

      (⍳2 2)∘.,((⍳2 2)⌷¨¨⊂⊂'AB')
┌──────┬──────┐
│1 1 AA│1 1 AB│
├──────┼──────┤
│1 1 BA│1 1 BB│
└──────┴──────┘
┌──────┬──────┐
│1 2 AA│1 2 AB│
├──────┼──────┤
│1 2 BA│1 2 BB│
└──────┴──────┘

┌──────┬──────┐
│2 1 AA│2 1 AB│
├──────┼──────┤
│2 1 BA│2 1 BB│
└──────┴──────┘
┌──────┬──────┐
│2 2 AA│2 2 AB│
├──────┼──────┤
│2 2 BA│2 2 BB│
└──────┴──────┘

In many applications, it is useful to reduce over the diagonal of the outer product. The inner product of vectors +.× is an example of this, which APL generalizes using the inner product f.g operator.

      v1 ← ⍳5
      v2 ← 'ABCDE'

      v1∘./v2
┌─────┬─────┬─────┬─────┬─────┐
│A    │B    │C    │D    │E    │
├─────┼─────┼─────┼─────┼─────┤
│AA   │BB   │CC   │DD   │EE   │
├─────┼─────┼─────┼─────┼─────┤
│AAA  │BBB  │CCC  │DDD  │EEE  │
├─────┼─────┼─────┼─────┼─────┤
│AAAA │BBBB │CCCC │DDDD │EEEE │
├─────┼─────┼─────┼─────┼─────┤
│AAAAA│BBBBB│CCCCC│DDDDD│EEEEE│
└─────┴─────┴─────┴─────┴─────┘

      ⍝ Diagonal
      (1 1)∘⍉(v1∘./v2)
┌─┬──┬───┬────┬─────┐
│A│BB│CCC│DDDD│EEEEE│
└─┴──┴───┴────┴─────┘

      ⍝ The following are equivalent
      ,/(1 1)∘⍉(v1∘./v2)
┌───────────────┐
│ABBCCCDDDDEEEEE│
└───────────────┘
      v1,./v2
┌───────────────┐
│ABBCCCDDDDEEEEE│
└───────────────┘

For arbitrary arrays, the inner product X(f.g)Y is the same as taking the outer product ∘.(f/ (g¨)) of the rows of the left argument (trailing vectors) (⊂[⍴⍴x]X) and the columns of the right argument (leading vectors) ⊂[1]Y, then removing a layer of depth ⊃⍤1. For example, the elements of X(+.×)Y are the sum of the product ∘.(+/ (ר)) of a row of X and a column of Y, which is exactly matrix multiplication.

      ⎕←NUM←5 5⍴0 0 0 1 2 1 1 2 3
0 0 0 1 2
1 1 2 3 0
0 0 1 2 1
1 2 3 0 0
0 1 2 1 1
      ⍝ The trailing vectors (rows) of NUM
      ⊂[⍴⍴NUM]NUM
┌─────────┬─────────┬─────────┬─────────┬─────────┐
│0 0 0 1 2│1 1 2 3 0│0 0 1 2 1│1 2 3 0 0│0 1 2 1 1│
└─────────┴─────────┴─────────┴─────────┴─────────┘

      ⎕←ALPHA←5 5⍴⎕A
ABCDE
FGHIJ
KLMNO
PQRST
UVWXY
      ⍝ The leading vectors (columns) of ALPHA
      ⊂[1]ALPHA
┌─────┬─────┬─────┬─────┬─────┐
│AFKPU│BGLQV│CHMRW│DINSX│EJOTY│
└─────┴─────┴─────┴─────┴─────┘

      (⊂[⍴⍴NUM]NUM) (∘.(/¨)) ⊂[1]ALPHA 
┌─────────────┬─────────────┬─────────────┬─────────────┬─────────────┐
│┌┬┬┬─┬──┐    │┌┬┬┬─┬──┐    │┌┬┬┬─┬──┐    │┌┬┬┬─┬──┐    │┌┬┬┬─┬──┐    │
│││││P│UU│    │││││Q│VV│    │││││R│WW│    │││││S│XX│    │││││T│YY│    │
│└┴┴┴─┴──┘    │└┴┴┴─┴──┘    │└┴┴┴─┴──┘    │└┴┴┴─┴──┘    │└┴┴┴─┴──┘    │
├─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤
│┌─┬─┬──┬───┬┐│┌─┬─┬──┬───┬┐│┌─┬─┬──┬───┬┐│┌─┬─┬──┬───┬┐│┌─┬─┬──┬───┬┐│
││A│F│KK│PPP││││B│G│LL│QQQ││││C│H│MM│RRR││││D│I│NN│SSS││││E│J│OO│TTT│││
│└─┴─┴──┴───┴┘│└─┴─┴──┴───┴┘│└─┴─┴──┴───┴┘│└─┴─┴──┴───┴┘│└─┴─┴──┴───┴┘│
├─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤
│┌┬┬─┬──┬─┐   │┌┬┬─┬──┬─┐   │┌┬┬─┬──┬─┐   │┌┬┬─┬──┬─┐   │┌┬┬─┬──┬─┐   │
││││K│PP│U│   ││││L│QQ│V│   ││││M│RR│W│   ││││N│SS│X│   ││││O│TT│Y│   │
│└┴┴─┴──┴─┘   │└┴┴─┴──┴─┘   │└┴┴─┴──┴─┘   │└┴┴─┴──┴─┘   │└┴┴─┴──┴─┘   │
├─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤
│┌─┬──┬───┬┬┐ │┌─┬──┬───┬┬┐ │┌─┬──┬───┬┬┐ │┌─┬──┬───┬┬┐ │┌─┬──┬───┬┬┐ │
││A│FF│KKK│││ ││B│GG│LLL│││ ││C│HH│MMM│││ ││D│II│NNN│││ ││E│JJ│OOO│││ │
│└─┴──┴───┴┴┘ │└─┴──┴───┴┴┘ │└─┴──┴───┴┴┘ │└─┴──┴───┴┴┘ │└─┴──┴───┴┴┘ │
├─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤
│┌┬─┬──┬─┬─┐  │┌┬─┬──┬─┬─┐  │┌┬─┬──┬─┬─┐  │┌┬─┬──┬─┬─┐  │┌┬─┬──┬─┬─┐  │
│││F│KK│P│U│  │││G│LL│Q│V│  │││H│MM│R│W│  │││I│NN│S│X│  │││J│OO│T│Y│  │
│└┴─┴──┴─┴─┘  │└┴─┴──┴─┴─┘  │└┴─┴──┴─┴─┘  │└┴─┴──┴─┴─┘  │└┴─┴──┴─┴─┘  │
└─────────────┴─────────────┴─────────────┴─────────────┴─────────────┘

      ⍝ Making the outer product clearer by adding labels
      (' ',⊂[⍴⍴NUM]NUM),(⊂[1]ALPHA)⍪((⊂[⍴⍴NUM]NUM)(∘.(/¨))⊂[1]ALPHA)
┌─────────┬─────────────┬─────────────┬─────────────┬─────────────┬─────────────┐
│         │AFKPU        │BGLQV        │CHMRW        │DINSX        │EJOTY        │
├─────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤
│0 0 0 1 2│┌┬┬┬─┬──┐    │┌┬┬┬─┬──┐    │┌┬┬┬─┬──┐    │┌┬┬┬─┬──┐    │┌┬┬┬─┬──┐    │
│         │││││P│UU│    │││││Q│VV│    │││││R│WW│    │││││S│XX│    │││││T│YY│    │
│         │└┴┴┴─┴──┘    │└┴┴┴─┴──┘    │└┴┴┴─┴──┘    │└┴┴┴─┴──┘    │└┴┴┴─┴──┘    │
├─────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤
│1 1 2 3 0│┌─┬─┬──┬───┬┐│┌─┬─┬──┬───┬┐│┌─┬─┬──┬───┬┐│┌─┬─┬──┬───┬┐│┌─┬─┬──┬───┬┐│
│         ││A│F│KK│PPP││││B│G│LL│QQQ││││C│H│MM│RRR││││D│I│NN│SSS││││E│J│OO│TTT│││
│         │└─┴─┴──┴───┴┘│└─┴─┴──┴───┴┘│└─┴─┴──┴───┴┘│└─┴─┴──┴───┴┘│└─┴─┴──┴───┴┘│
├─────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤
│0 0 1 2 1│┌┬┬─┬──┬─┐   │┌┬┬─┬──┬─┐   │┌┬┬─┬──┬─┐   │┌┬┬─┬──┬─┐   │┌┬┬─┬──┬─┐   │
│         ││││K│PP│U│   ││││L│QQ│V│   ││││M│RR│W│   ││││N│SS│X│   ││││O│TT│Y│   │
│         │└┴┴─┴──┴─┘   │└┴┴─┴──┴─┘   │└┴┴─┴──┴─┘   │└┴┴─┴──┴─┘   │└┴┴─┴──┴─┘   │
├─────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤
│1 2 3 0 0│┌─┬──┬───┬┬┐ │┌─┬──┬───┬┬┐ │┌─┬──┬───┬┬┐ │┌─┬──┬───┬┬┐ │┌─┬──┬───┬┬┐ │
│         ││A│FF│KKK│││ ││B│GG│LLL│││ ││C│HH│MMM│││ ││D│II│NNN│││ ││E│JJ│OOO│││ │
│         │└─┴──┴───┴┴┘ │└─┴──┴───┴┴┘ │└─┴──┴───┴┴┘ │└─┴──┴───┴┴┘ │└─┴──┴───┴┴┘ │
├─────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤
│0 1 2 1 1│┌┬─┬──┬─┬─┐  │┌┬─┬──┬─┬─┐  │┌┬─┬──┬─┬─┐  │┌┬─┬──┬─┬─┐  │┌┬─┬──┬─┬─┐  │
│         │││F│KK│P│U│  │││G│LL│Q│V│  │││H│MM│R│W│  │││I│NN│S│X│  │││J│OO│T│Y│  │
│         │└┴─┴──┴─┴─┘  │└┴─┴──┴─┴─┘  │└┴─┴──┴─┴─┘  │└┴─┴──┴─┴─┘  │└┴─┴──┴─┴─┘  │
└─────────┴─────────────┴─────────────┴─────────────┴─────────────┴─────────────┘

      ⍝ Reducing over the result using catenate ,
      ((⊂[⍴⍴NUM]NUM)(∘.(,/(/¨)))⊂[1]ALPHA)
┌─────────┬─────────┬─────────┬─────────┬─────────┐
│┌───┐    │┌───┐    │┌───┐    │┌───┐    │┌───┐    │
││PUU│    ││QVV│    ││RWW│    ││SXX│    ││TYY│    │
│└───┘    │└───┘    │└───┘    │└───┘    │└───┘    │
├─────────┼─────────┼─────────┼─────────┼─────────┤
│┌───────┐│┌───────┐│┌───────┐│┌───────┐│┌───────┐│
││AFKKPPP│││BGLLQQQ│││CHMMRRR│││DINNSSS│││EJOOTTT││
│└───────┘│└───────┘│└───────┘│└───────┘│└───────┘│
├─────────┼─────────┼─────────┼─────────┼─────────┤
│┌────┐   │┌────┐   │┌────┐   │┌────┐   │┌────┐   │
││KPPU│   ││LQQV│   ││MRRW│   ││NSSX│   ││OTTY│   │
│└────┘   │└────┘   │└────┘   │└────┘   │└────┘   │
├─────────┼─────────┼─────────┼─────────┼─────────┤
│┌──────┐ │┌──────┐ │┌──────┐ │┌──────┐ │┌──────┐ │
││AFFKKK│ ││BGGLLL│ ││CHHMMM│ ││DIINNN│ ││EJJOOO│ │
│└──────┘ │└──────┘ │└──────┘ │└──────┘ │└──────┘ │
├─────────┼─────────┼─────────┼─────────┼─────────┤
│┌─────┐  │┌─────┐  │┌─────┐  │┌─────┐  │┌─────┐  │
││FKKPU│  ││GLLQV│  ││HMMRW│  ││INNSX│  ││JOOTY│  │
│└─────┘  │└─────┘  │└─────┘  │└─────┘  │└─────┘  │
└─────────┴─────────┴─────────┴─────────┴─────────┘

      ⍝ Removing a layer of depth
      ⊃⍤0⊢((⊂[⍴⍴NUM]NUM)(∘.(,/(/¨)))⊂[1]ALPHA)
┌───────┬───────┬───────┬───────┬───────┐
│PUU    │QVV    │RWW    │SXX    │TYY    │
├───────┼───────┼───────┼───────┼───────┤
│AFKKPPP│BGLLQQQ│CHMMRRR│DINNSSS│EJOOTTT│
├───────┼───────┼───────┼───────┼───────┤
│KPPU   │LQQV   │MRRW   │NSSX   │OTTY   │
├───────┼───────┼───────┼───────┼───────┤
│AFFKKK │BGGLLL │CHHMMM │DIINNN │EJJOOO │
├───────┼───────┼───────┼───────┼───────┤
│FKKPU  │GLLQV  │HMMRW  │INNSX  │JOOTY  │
└───────┴───────┴───────┴───────┴───────┘

      NUM,./ALPHA
┌───────┬───────┬───────┬───────┬───────┐
│PUU    │QVV    │RWW    │SXX    │TYY    │
├───────┼───────┼───────┼───────┼───────┤
│AFKKPPP│BGLLQQQ│CHMMRRR│DINNSSS│EJOOTTT│
├───────┼───────┼───────┼───────┼───────┤
│KPPU   │LQQV   │MRRW   │NSSX   │OTTY   │
├───────┼───────┼───────┼───────┼───────┤
│AFFKKK │BGGLLL │CHHMMM │DIINNN │EJJOOO │
├───────┼───────┼───────┼───────┼───────┤
│FKKPU  │GLLQV  │HMMRW  │INNSX  │JOOTY  │
└───────┴───────┴───────┴───────┴───────┘

Using the matrix inverse ⌹ function to verify the multiplication

      M ← +\ 5 5 ⍴ ⍳7
      M
1  3  6 10 15
6 13 14 16 19
4  9 15 22 23
2  5  9 14 20
7  8 10 13 17

      N ← +\ 5 5 ⍴ ⍳6
      N
1  3  6 10 15
6  7  9 12 16
5 11 12 14 17
4  9 15 16 18
3  7 12 18 19

      L ← M +.× N
      L
134 285 435  560  630
275 540 789 1010 1185
290 599 891 1124 1292
193 406 615  790  895
208 423 633  820  960

      L+.×⌹N
1  3  6 10 15
6 13 14 16 19
4  9 15 22 23
2  5  9 14 20
7  8 10 13 17

      ⌹M+.×L
1  3  6 10 15
6  7  9 12 16
5 11 12 14 17
4  9 15 16 18
3  7 12 18 19

The matrix inverse ⌹ also takes the pseudoinverse of a matrix, if the inverse does not exist, which can be used to get least squares solutions of systems of linear equations when a unique solution is not possible. Take the example of a bakery, wanting to make the most out of their ingredients

⍝ Recipes
⍝           Flour   Milk    Sugar   Butter  Eggs
Cake    ←   450     0       700     500     6
Pancake ←   200     300     50      50      1
Cupcake ←   150     125     150     50      0
Cookies ←   280     0       250     200     2

Available ← 2200 1000 2200 1600 19

Since there are more ingredients than recipes, there will not be a unique solution to this problem. The system of equations here is

$$\text{Cakes}\cdot\begin{bmatrix}450\\ 0\\ 700\\ 500\\ 2 \end{bmatrix}+\text{Pancake}\cdot\begin{bmatrix}200\\ 300\\ 50\\ 50\\ 1 \end{bmatrix}+\text{Cupcakes}\cdot\begin{bmatrix}150\\ 125\\ 150\\ 50\\ 0 \end{bmatrix}+\text{Cookies}\cdot\begin{bmatrix}280\\ 0\\ 250\\ 200\\ 2 \end{bmatrix}=\begin{bmatrix} 2200\\ 1000\\ 2200\\ 1600\\ 19 \end{bmatrix}$$

which can be written in matrix form as

$$\begin{bmatrix} 450 & 200 & 150 & 280 \\ 0 & 300 & 125 & 0 \\ 700 & 50 & 150 & 250 \\ 500 & 50 & 50 & 200 \\ 2 & 1 & 0 & 2 \end{bmatrix} \begin{bmatrix} \text{Cakes} \\ \text{Pancake} \\ \text{Cupcake} \\ \text{Cookies} \end{bmatrix} = \begin{bmatrix} 2200 \\ 1000 \\ 2200 \\ 1600 \\ 19 \end{bmatrix}$$

which can be solved by obtaining the pseudoinverse of the matrix (represented below using the + symbol), and multiplying it by the target vector.

$$\begin{bmatrix} \text{Cakes} \\ \text{Pancake} \\ \text{Cupcake} \\ \text{Cookies} \end{bmatrix} = \begin{bmatrix} 450 & 200 & 150 & 280 \\ 0 & 300 & 125 & 0 \\ 700 & 50 & 150 & 250 \\ 500 & 50 & 50 & 200 \\ 2 & 1 & 0 & 2 \end{bmatrix}^+ \begin{bmatrix} 2200 \\ 1000 \\ 2200 \\ 1600 \\ 19 \end{bmatrix}$$

       Goods ← ⍉ ↑ Cake Pancake Cupcake Cookies
       Goods
450 200 150 280
  0 300 125   0
700  50 150 250
500  50  50 200
  6   1   0   2

       ⍝ (Pseudo)inverse multiplied by the target vector 
       (⌹Goods)+.×Available  
1.99585722 2.919959252 0.992099059 2.032347529

Then, the closest solution is baking roughly 2 cakes, 3 batches of pancakes, 1 batch of cupcakes, and 2 batches of cookies.