in conversation, michal wallace suggested that we would profit from having a tool which makes it easy to rewrite a long function as a short function with calls to short subfunctions.

michal is currently working on such a factoring tool which uses a gui and which doesn't depend on k.  i decided that it wouldn't be difficult to add this functionality to the 1010 k parser.  the k tool would take a text file containing a marked-up function as input and produce an executable k script containing the factored code:  the transformed input function and the several subfunctions indicated by the markup code.

here's an example.

the input file is a k script, f.k:

    get:{[u;c;r;m;q]
     if[1=#^m[];:(m;nl q)]                                  / exit on empty trace table
     I::q _sm"*<!-- explain:*"                              / ??? need a better way
    [r1:get1
     q::[I;q;xo gq qp q]                                    / get xml of query at path
     o::[I;query q;init q]                                  / query
     m::[I;m;@[m;_n;@[;&-1<*:'m`i]]]                        / ignore recursive ops
     r::[I;row[u;o;`REAL_INDEX]r;r]                         / row (zero-based)
     o:-4_ o                                                / drop last four ops
     j:find[m]c                                             / find c in trace matrix
     i:m[`i;j]                                              / index of op generating c
     e:m[`e;j]                                              / expression generating c
     k:m[`f;j]                                              / crosstab tuples or function
     l:ni i                                                 / nested index
     f:*a:o . l                                             / op to explain
    [
     if[f _in done;:(m;nl q)]                               / early exit!!
    [r2:get2
     v:ops[f][u;m;q;c;r;k]a 1                               / get selection value for next query
     p:*n:next[o;m;i;l]f                                    / get next (o;m)
     m:n 1                                                  / next trace
     q:nl xo p,ox ri,hi,ex[c;e]se v                         / append
    [
     (m;q)}                                                 / return trace, next query and its result

the markup consists of pairs left square brackets.  the opening left bracket contains code:

    <result>:<subfunction>

everything between paired brackets is extracted as a subfunction with the name <subfunction>.  the marked-up code is replaced with a call to <subfunction>, the result is assigned to the <result> variable.

the output script for this example is g.k.  the function 'get' is from mdb/expain.k:

    / (gets;sets)
    gsv:{a:*k:lst cde x;t:p k 1;n:nip t;b:ass t;(n _dvl:[#a;a;+,"xyz"],b;?b,a _dvl,"")}                 / (gets;sets)
    cde:{:[LCU=*x;1_-1_ x;x]}                                                                           / {..} -> ..
    lst:{:[LBR=*x;(cut[SEP]1_ i#x;(1+i:x?RBR)_ x);("";x)]}                                              / argument list

    / factor
    q:{[f]{x,(,,BLA),y}/{(,x),y}.{vfg[*x;x 1]}/(f;())}                                                  / factor function
    vfg:{[f;F]:[#i:&LBR=*:'f;vre[f;F]. 2#i;(f;F)]}                                                      / exit when no sub-blocks
    efo:{[u;g]:[eex q:p g;'":x detected!";ufx q;'"_f detected!";g]}                                     / :x, _f
    eex:{:[4:x;0;~2=#x;|/_f'x;(,COL)~*x;1;|/_f'x]}                                                      / :x
    ufx:{:[~4:x;|/_f'x;"_f"~*x;1;0]}                                                                    / _f
    fbu:1_,/(SEP,dlb dtb unc@)'                                                                         / flatten blanks uncomment
    cls:{:[~4:y;?,/_f[x]'y;LCU~*y;int[x;*gsv 1_-1_ y];()]}

    vre:{[f;F;s;e]
     v:e _ f;v@:&~LBR=*:'v;v:nip p@-1_ fbu v                                                            / refs after sub
     j:s+!1+e-s;g:f -1_ j;f:f _di 1_ j                                                                  / fun-sub, sub
     u:gsv(1+u?COL)_ u:fbu[s#f],RCU                                                                     / vars before sub
     w:gsv efo[u]h:fbu 1_ g                                                                             / vars of sub
     w[1],:cls[u 1;p h]_dvl w 1                                                                         / vars of closures
     a:w 1;b:int[a]v;c:int[,/w]u 1                                                                      / results
     i:LBR,(1_,/SEP,'c),RBR                                                                             / header
     r:(k:m?COL)#m:1_ f s                                                                               / result
     f[s]:m,i,,/{SEP,x,COL,r,BLA,y}'[b;$!#b]                                                            / fun calls sub
     g[0]:((k+1)_ m),COL,LCU,i                                                                          / header of sub
     g,:,:[1=#b;COM,*b;LPA,(1_,/SEP,'b),RPA],RCU                                                        / return of sub
     (f;F,,g)}                                                                                          / append and continue

g.k is produced from f.k this way:

    "g.k"0:q@0:"f.k"

the k parser q function sequentially processes marked up sections of the input function.  at each step q needs to figure out a few things:  which variables in the input function does the subfunction require (including variables used in closures), and which variables from the subfunction does the input function require?

subfunctions must adhere to certain rules:  no early exits (:x), no recursive calls using _f, and a subfunction cannot split a multi-line conditional or if/while/do statement or a multi-line list expression.  the tokenizer will choke on the multi-line violations with an unbalanced error.

the code for q is, and is found in the k parser, bs/kparse.k:

    / (gets;sets)
    gsv:{a:*k:lst cde x;t:p k 1;n:nip t;b:ass t;(n _dvl:[#a;a;+,"xyz"],b;?b,a _dvl,"")}                 / (gets;sets)
    cde:{:[LCU=*x;1_-1_ x;x]}                                                                           / {..} -> ..
    lst:{:[LBR=*x;(cut[SEP]1_ i#x;(1+i:x?RBR)_ x);("";x)]}                                              / argument list

    / factor
    q:{[f]{x,(,,BLA),y}/{(,x),y}.{vfg[*x;x 1]}/(f;())}                                                  / factor function
    vfg:{[f;F]:[#i:&LBR=*:'f;vre[f;F]. 2#i;(f;F)]}                                                      / exit when no sub-blocks
    efo:{[u;g]:[eex q:p g;'":x detected!";ufx q;'"_f detected!";dcx[u 1]q;'"closure detected!";g]}      / :x, _f, closure detection
    dcx:{:[~4:y;|/_f[x]'y;LCU~*y;|/x _lin*gsv 1_-1_ y;0]}                                               / closure
    eex:{:[4:x;0;~2=#x;|/_f'x;(,COL)~*x;1;|/_f'x]}                                                      / :x
    ufx:{:[~4:x;|/_f'x;"_f"~*x;1;0]}                                                                    / _f
    fbu:1_,/(SEP,dlb dtb unc@)'                                                                         / flatten blanks uncomment

    vre:{[f;F;s;e]
     v:e _ f;v@:&~LBR=*:'v;v:nip p@-1_ fbu v                                                            / refs after sub
     j:s+!1+e-s;g:f -1_ j;f:f _di 1_ j                                                                  / fun-sub, sub
     u:gsv(1+u?COL)_ u:fbu[s#f],RCU                                                                     / vars before sub
     w:gsv efo[u]fbu 1_ g                                                                               / vars of sub
     a:w 1;b:int[w 1]v;c:int[,/w]u 1                                                                    / results
     h:LBR,(1_,/SEP,'c),RBR                                                                             / header
     r:(i:m?COL)#m:1_ f s                                                                               / result
     f[s]:m,h,,/{SEP,x,COL,r,BLA,y}'[b;$!#b]                                                            / fun calls sub
     g[0]:((i+1)_ m),COL,LCU,h                                                                          / header of sub
     g,:,:[1=#b;COM,*b;LPA,(1_,/SEP,'b),RPA],RCU                                                        / return of sub
     (f;F,,g)}                                                                                          / append and continue