Core-Style Arguments for Script Commands

Cyan Ogilvie
Ruby Lane, Inc.
cyan@rubylane.com

Beyond 2 or 3 positional parameters, readability suffers:

proc searchdb {progcs progq usecache errorsev spid sites style db
    ipp id cs ss page uid readahead _results {regioncode ""}
    {minprice ""} {maxprice ""} {types ""} {redtagpercentoff 0}
    {addedsince ""} {sortby ""} {trialsiteid ""} {trialclickrate ""}
    {showsoldprice 0} {onsale ""} {lane ""} {website ""} {facets ""}
} {
   ...
}

searchdb "" $ss 0 notice "" "" $style $db $maxresults "" "" "" 1 \
    "" 0 progresults "" "" "" "" 0 "" "" $userid $newtestrate 0 "" \
    "" $website

Core commands typically use optional, named parameters:

glob -nocomplain -type f -tails -directory $spooldir *@*
lsort -index 0 -stride 2 -dictionary $search_counts
entry .pw -show * -textvariable pw -width 15

The majority of core commands with more than 3 parameters use this scheme:

binary, chan, clock, exec, fconfigure, fcopy, file, glob, interp, load, lsearch, lsort, namespace, package, puts, read, regexp, regsub, return, socket, source, string, subst, switch, unload, unset, zlib, pack, clipboard, place, event, wm, focus, font, winfo, grab, selection, send, grid, tk, bell and all Tk widget constructors and instance commands.

The only exceptions I could find were [trace remove] and [lreplace] or control structures

But script-defined commands are second class citizens

  • No support provided for these patterns to scripts
  • proc, apply, TclOO's constructors and methods support:
    • Positional parameters
    • Default values
    • Variadic arguments
  • coroutines support only variadic arguments
Tcl scripts can implement core-style argument parsing but:
  • it's slow - a problem for hot code
  • clutters proc implementations with tangential code
  • obscures proc signatures
  • stack traces less clear when incorrect args are passed

Therefore it is very seldom done

Conventions established by the core

-foo a boolean toggle “foo” is enabled, its absence means that the toggle is disabled (e.g. -nocase, -all)
-foo bar an argument named “foo” is assigned the value “bar”. In some cases, not specifying the argument means that it takes a default value (e.g. regexp -start), in other cases that triggers behaviour different to all possible values (e.g. lsort -command)
-foo / -bar / -baz a set of boolean-style arguments that are mutually exclusive and select a value for a single logical argument (e.g. lsort -ascii / -integer / -dictionary)
-- signals the end of the named options, further arguments are interpreted as positional parameters even if they would have matched a named argument (not universal)
When contradicting arguments are given, later arguments override earlier ones:
lsort -increasing -ascii -dictionary -decreasing $list
uses dictionary comparison, decreasing order (possibly not universal)

parse_args

Extension that provides support for these patterns to scripts

Primary design goals:

  • Feel like Tcl
  • Fast
  • Parameter signatures should be obvious at a glance
  • Terse
proc glob args {
    parse_args $args {
        -directory  {}  
        -join       {-boolean}
        -nocomplain {-boolean}
        -path       {}  
        -tails      {-boolean}
        -types      {-default {}} 
        args        {-name patterns}
    }

    if {$join} {
        set patterns [list [file join {*}$patterns]]
    }

    if {[llength $patterns] == 0 && $nocomplain} return

    foreach pattern $patterns {
        if {[info exists directory]} {
            ...
        }
    }
    ...
}
proc regexp args {
    parse_args $args {
        -about      {-boolean}
        -expanded   {-boolean}
        -indices    {-boolean}
        -line       {-boolean}
        -linestop   {-boolean}
        -lineanchor {-boolean}
        -nocase     {-boolean}
        -all        {-boolean}
        -inline     {-boolean}
        -start      {-default 0}
        exp         {-required}
        string      {-required}
        matchvar    {}  
        args        {-name submatchvars}
    }   
    ... 
}
proc lsearch args {
    parse_args $args {
        -exact      {-name matchtype -multi}
        -glob       {-name matchtype -multi}
        -regexp     {-name matchtype -multi}

        -sorted     {-boolean}
        -all        {-boolean}
        -inline     {-boolean}
        -not        {-boolean}
        -start      {-default 0}

        -ascii      {-name compare_as -multi -default ascii}
        -dictionary {-name compare_as -multi}
        -integer    {-name compare_as -multi}
        -real       {-name compare_as -multi}

        -nocase     {-boolean}

        -decreasing {-name order -multi}
        -increasing {-name order -multi -default increasing}
        -bisect     {-boolean}

        -index      {}
        -subindices {-boolean}
    }

    if {$sorted && [info exists matchtype] && $matchtype in {glob regexp}} {
        error "-sorted is mutually exclusive with -glob and -regexp"
    }
    if {![info exists matchtype]} {set matchtype glob}
}

Performance

Two design goals are in conflict:

  • intuitive definitions (easy for humans)
  • high performance (easy for machines)
parse_args $args {
    -directory  {}
    -join       {-boolean}
    -nocomplain {-boolean}
    -path       {}
    -tails      {-boolean}
    -types      {-default {}}
    args        {-name patterns}
}
Tcl_Obj IntRep:
  • Tcl_GetIndexFromObj lookup for named params
  • default values
  • required params
  • validators
  • enum choices

Using the intrep of a custom Tcl_ObjType to store this:

  • Efficiently caches the parse info
  • Gets lifecycle management for free
  • Prevents cached Tcl_Objs from being shared across threads
proc native {t_a title c_a category  w_a wiki {r_a rating} {rating 1.0}} {
    list $title $category $wiki $rating
}   

proc using_parse_args args {
    parse_args $args {
        -title      {-required}
        -category   {-default {}} 
        -wiki       {-required}
        -rating     {-default 1.0 -validate {string is double -strict}}
    }   
    list $title $category $wiki $rating
}
            | microseconds
tcl parsing | 24.540
     native |  0.535
 parse args |  0.838

Beyond [proc] argument parsing

coroutine foo apply [list {} {
    set res     {}
    set options {-code 0 -level 0}
    while 1 {
        catch {
            parse_args [yieldto return -options $options $res] {
                -foo    {-default xyzzy}
                -count  {-required}
            }

            ... generate next value
        } res options
    }
}]

Future Work

  • C API (or perhaps Ns_ParseObjv is a better fit?)
    Ns_ObjvSpec opts[] = {
        {"-cache",       Ns_ObjvTime,   &ttlPtr,  NULL},
        {"-nocache",     Ns_ObjvBool,   &nocache, INT2PTR(NS_TRUE)},
        {"-tcl",         Ns_ObjvBool,   &tcl,     INT2PTR(NS_TRUE)},
        {"--",           Ns_ObjvBreak,  NULL,     NULL},
        {NULL, NULL, NULL, NULL}
    };
    Ns_ObjvSpec args[] = {
        {"file",  Ns_ObjvString, &file,  NULL},
        {"?args", Ns_ObjvArgs,   &nargs, NULL},
        {NULL, NULL, NULL, NULL}
    };
    if (Ns_ParseObjv(opts, args, interp, 1, objc, objv) != NS_OK) {
        return TCL_ERROR;
    }
  • Support positional parameters interspersed with named parameters
  • Documention
  • Introspection