more stuff

Author: ablaom

Project.toml

@@ -4,6 +4,7 @@ authors = ["Anthony D. Blaom <anthony.blaom@gmail.com>"]
 version = "0.1.0"
 
 [deps]
+InteractiveUtils = "b77e0a4c-d291-57a0-90e8-8db25a27a240"
 Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
 
 [extras]

docs/make.jl

@@ -9,11 +9,11 @@ makedocs(;
     pages=[
         "Introduction" => "index.md",
         "Anatomy of an Implementation" => "anatomy_of_an_implementation.md",
-        "Common Implementation Patterns" => "common_implementation_patterns.md",
         "Reference" => "reference.md",
         "Fit, update and ingest" => "fit_update_and_ingest.md",
         "Predict and other operations" => "operations.md",
         "Model Traits" => "model_traits.md",
+        "Common Implementation Patterns" => "common_implementation_patterns.md",
     ],
     repo="https://$REPO/blob/{commit}{path}#L{line}",
     sitename="LearnAPI.jl"

docs/src/anatomy_of_an_implementation.md

@@ -5,9 +5,10 @@
 > dispatched on the model type; `predict` is an example of an **operation** (another is
 > `transform`). In this example we also implement an **accessor function**, called
 > `feature_importance`, returning the absolute values of the linear coefficients. The
-> ridge regressor has a target variable and one trait declaration flags the output of
-> `predict` as being a [proxy](@ref scope) for the target. Other traits articulate the
-> model's training data type requirements and the input/output type of `predict`.
+> ridge regressor has a target variable and `predict` makes literal predictions of the
+> target (rather than, say, probablistic predictions); this behaviour is flagged by the
+> `target_proxies` model trait.  Other traits articulate the model's training data type
+> requirements and the input/output type of `predict`.
 
 We begin by describing an implementation of LearnAPI.jl for basic ridge
 regression (no intercept) to introduce the main actors in any implementation.
@@ -58,7 +59,8 @@ function LearnAPI.fit(model::MyRidge, verbosity, X, y)
 
         # process input:
         x = Tables.matrix(X)  # convert table to matrix
-        features = Tables.columnnames(X)
+        s = Tables.schema(X)
+        features = s.names
 
         # core solver:
         coefficients = (x'x + model.lambda*I)\(x'y)
@@ -140,28 +142,28 @@ nothing # hide
 Another example of an accessor function is [`training_losses`](@ref).
 
 
-## [Model traits](@id traits) 
+## [Model traits](@id traits)
 
 Our model has a target variable, in the sense outlined in [Scope and undefined
 notions](@ref scope), and `predict` returns an object with exactly the same form as the
 target. We indicate this behaviour by declaring
 
 ```@example anatomy
-LearnAPI.target_proxy(::Type{<:MyRidge}) = (; predict=LearnAPI.TrueTarget())
+LearnAPI.target_proxies(::Type{<:MyRidge}) = (; predict=LearnAPI.TrueTarget())
 nothing # hide
 ```
 Or, you can use the shorthand
 
 ```@example anatomy
-@trait MyRidge target_proxy = (; predict=LearnAPI.TrueTarget())
+@trait MyRidge target_proxies = (; predict=LearnAPI.TrueTarget())
 nothing # hide
 ```
 
 More generally, `predict` only returns a *proxy* for the target, such as probability
 distributions, and we would make a different declaration here. See [Target proxies](@ref)
 for details.
 
-`LearnAPI.target_proxy` is an example of a **model trait**. A complete list of traits
+`LearnAPI.target_proxies` is an example of a **model trait**. A complete list of traits
 and the contracts they imply is given in [Model Traits](@ref).
 
 > **MLJ only.** The values of all traits constitute a model's **metadata**, which is

docs/src/index.md

@@ -41,6 +41,11 @@ model. Probability distributions, confidence intervals and survival functions ar
 of [Target proxies](@ref). LearnAPI provides a trait for distinguishing such models based
 on the target proxy.
 
+LearnAPI does not provide an interface for data access or data resampling, and could be
+used in conjunction with one or more such interfaces (e.g.,
+[Tables.jl](https://github.com/JuliaML/MLUtils.jl),
+[MLJUtils.jl](https://github.com/JuliaML/MLUtils.jl)).
+
 ## Methods
 
 In LearnAPI.jl a *model* is just a container for the hyper-parameters of some machine
@@ -56,17 +61,17 @@ The following methods, dispatched on model type, are provided:
 
 - `ingest!` for incremental learning
 
-- **operations**, such as `predict`, `transform` and `inverse_transform` for applying the
-  model to data not used for training
+- **operations**, `predict`, `predict_joint`, `transform` and `inverse_transform` for
+  applying the model to data not used for training
 
 - common **accessor functions**, such as `feature_importances` and `training_losses`, for
-  extracting, from training outcomes, information common to different types of models
+  extracting, from training outcomes, information common to some models
 
-- **model traits**, such as `target_proxy(model)`, for promising specific behaviour
+- **model traits**, such as `target_proxies(model)`, for promising specific behaviour
 
 There is flexibility about how much of the interface is implemented by a given model
 object `model`. A special trait `functions(model)` declares what has been explicitly
-implemented or overloaded to work with `model`, excluding traits.
+implemented to work with `model`, excluding traits.
 
 Since this is a functional-style interface, `fit` returns model `state`, in addition to
 learned parameters, for passing to the optional `update!` and `ingest!` methods. These
@@ -77,12 +82,13 @@ component (important for models that do not generalize to new data).
 Models can be supervised or not supervised, can generalize to new data observations, or
 not generalize. To ensure proper handling by client packages of probabilistic and other
 non-literal forms of target predictions (pdfs, confidence intervals, survival functions,
-etc) the kind of prediction can be flagged appropriately; see more at "target" below.
+etc) the output of `predict` and `predict_joint` can be flagged appropriately; see more at
+"target" below.
 
 
 ## [Scope and undefined notions](@id scope)
 
-The Learn API provides methods for training, applying, and saving machine learning models,
+LearnAPI.jl provides methods for training, applying, and saving machine learning models,
 and that is all. *It does not specify an interface for data access or data
 resampling*. However, LearnAPI.jl is predicated on a few basic undefined notions (in
 **boldface**) which some higher-level interface might decide to formalize:
@@ -115,16 +121,16 @@ resampling*. However, LearnAPI.jl is predicated on a few basic undefined notions
 
 ## Contents
 
-Our opening observations notwithstanding, it is useful to have a guide to the interface,
-linked below, organized around common *informally defined* patterns or "tasks". However,
-the definitive specification of the interface is the [Reference](@ref) section.
+It is useful to have a guide to the interface, linked below, organized around common
+*informally defined* patterns or "tasks". However, the definitive specification of the
+interface is the [Reference](@ref) section.
 
 - [Anatomy of an Implementation](@ref) (Overview)
 
-- [Common Implementation Patterns](@ref) (User Guide)
-
 - [Reference](@ref) (Official Specification)
 
+- [Common Implementation Patterns](@ref) (User Guide)
+
 - [Testing an Implementation](@ref)
 
 !!! info

docs/src/model_traits.md

@@ -1,12 +1,17 @@
 # Model Traits
 
-In this table, `Table` and `Continuous` are names owned by the package
-[ScientificTypesBase.jl](https://github.com/JuliaAI/ScientificTypesBase.jl/).
+Ordinary traits are available for overloading by an new model implementation. Derived
+traits are not.
+
+## Ordinary traits
+
+In the examples column of the table below, `Table` and `Continuous` are names owned by the
+package [ScientificTypesBase.jl](https://github.com/JuliaAI/ScientificTypesBase.jl/).
 
 | trait                                            | fallback value        | return value  | example |
 |:-------------------------------------------------|:----------------------|:--------------|:--------|
 | [`LearnAPI.functions`](@ref)`(model)`  | `()`                  | implemented LearnAPI functions (traits excluded) | `(:fit, :predict)` |
-| [`LearnAPI.target_proxy`](@ref)`(model)`    | `NamedTuple()`                  | details form of target proxy output | `(; predict=LearnAPI.Distribution()` |
+| [`LearnAPI.target_proxies`](@ref)`(model)`    | `NamedTuple()`                  | details form of target proxy output | `(; predict=LearnAPI.Distribution()` |
 | [`LearnAPI.position_of_target`](@ref)`(model)`   | `0`                   | † the positional index of the **target** in `data` in `fit(..., data...; metadata)` calls | 2 |
 | [`LearnAPI.position_of_weights`](@ref)`(model)`  | `0`                   | † the positional index of **observation weights** in `data` in `fit(..., data...; metadata)` | 3 |
 | [`LearnAPI.descriptors`](@ref)`(model)`          | `()`                  | lists one or more suggestive model descriptors from `LearnAPI.descriptors()` | (:classifier, :probabilistic) |
@@ -35,3 +40,10 @@ is understood to exclude the variable, but note that `fit` can have multiple sig
 varying lengths, as in `fit(model, verbosity, X, y)` and `fit(model, verbosity, X, y,
 w)`. A non-zero value is a promise that `fit` includes a signature of sufficient length to
 include the variable.
+
+## Dervied Traits
+
+| trait                                  | return value              | example |
+|:---------------------------------------|:--------------------------|:--------|
+| [`LearnAPI.name`](@ref)`(model)`       | model type name as string | "PCA"   |
+| [`LearnAPI.ismodel`](@ref)`(model)`    | `true` if `functions(model)` is not empty | `true` |

docs/src/operations.md

@@ -2,8 +2,8 @@
 
 > **Summary** Methods like `predict` and `transform`, that generally depend on learned
 > parameters, are called **operations**. All implemented operations must be included in
-> the output of the `methods` model trait. When an operation returns a [target
-> proxy](@ref scope), it must make a `target_proxy` declaration.
+> the output of the `functions` model trait. When an operation returns a [target
+> proxy](@ref scope), it must make a `target_proxies` declaration.
 
 An *operation* is any method with signature `some_operation(model, fitted_params,
 data...)`. Here `fitted_params` is the learned parameters object, as returned by
@@ -23,17 +23,14 @@ ŷ, predict_report = LearnAPI.predict(some_model, fitted_params, Xnew)
 [`LearnAPI.transform`](@ref)         | no          | none     |             |
 [`LearnAPI.inverse_transform`](@ref) | no          | none     | `transform` |
 
-> **† MLJ only.** MLJBase provides fallbacks for `predict_mode`, `predict_mean` and
-> `predict_median` by broadcasting methods from `Statistics` and `StatsBase` over the
-> results of `predict`.
 
 ## General requirements
 
 - Only implement `predict_joint` for outputing a *single* multivariate probability
   distribution for multiple target predictions, as described further at
   [`LearnAPI.predict_joint`](@ref).
 
-- Each operation explicitly implemented or overloaded must be included in the return value
+- Each operation explicitly overloaded must be included in the return value
   of [`LearnAPI.functions`](@ref).
 
 ## Predict or transform?
@@ -91,27 +88,33 @@ have no fields.
 | `LearnAPI.SurvivalFunction`     | survival function (possible requirement: observation is single-argument function mapping `Real` to `Real`) |
 | `LearnAPI.SurvivalDistribution` | probability distribution for survival time (possible requirement: observation have type `Distributions.ContinuousUnivariateDistribution`) |
 
-> **† MLJ only.** To avoid [ambiguities in
-> representation](https://github.com/alan-turing-institute/MLJ.jl/blob/dev/paper/paper.md#a-unified-approach-to-probabilistic-predictions-and-their-evaluation),
-> these options are disallowed, in favour of preceding alternatives.
+† Provided for completeness but discouraged to avoid [ambiguities in
+representation](https://github.com/alan-turing-institute/MLJ.jl/blob/dev/paper/paper.md#a-unified-approach-to-probabilistic-predictions-and-their-evaluation).
+
 
 !!! warning
 
 	The "possible requirement"s listed are not part of LearnAPI.jl.
 
 An operation with target proxy as output must declare a `TargetProxy` instance using the
-[`LearnAPI.target_proxy`](@ref), as in
+[`LearnAPI.target_proxies`](@ref), as in
+
+```julia
+LearnAPI.target_proxies(::Type{<:SomeModel}) = (predict=LearnAPI.Distribution(),)
+```
+
+which has the short form
 
 ```julia
-LearnAPI.target_proxy(::Type{<:SomeModel}) = (predict=LearnAPI.Distribution(),)
+LearnAPI.@trait target_proxies = (predict=LearnAPI.Distribution(),)
 ```
 
-If `predict_joint` is implemented, then a `target_proxy` declaration is also
+If `predict_joint` is implemented, then a `target_proxies` declaration is also
 required, but the interpretation is slightly different. This is because the output of
 `predict_joint` is not a number of observations but a single object. The trait value
 should be an instance of one of the following types:
 
-|          type                   | form of output of `predict_joint(model, _, data)`
+|          type                   | form of output of `predict_joint(model, fitted_params, data)`
 |:-------------------------------:|:--------------------------------------------------|
 | `LearnAPI.Sampleable`      | object that can be sampled to obtain a *vector* whose elements have the form of target observations; the vector length matches the number of observations in `data`. |
 | `LearnAPI.Distribution`    | explicit probability density/mass function whose sample space is vectors of target observations;  the vector length matches the number of observations in `data` |

docs/src/reference.md

@@ -1,7 +1,7 @@
 # Reference
 
-Here we give the definitive specification of interface provided by LearnAPI.jl. For a more
-informal guide see [Common Implementation Patterns](@ref).
+Here we give the definitive specification of the interface provided by LearnAPI.jl. For a
+more informal guide see [Common Implementation Patterns](@ref).
 
 ## Models
 

src/LearnAPI.jl

@@ -1,6 +1,7 @@
 module LearnAPI
 
 using Statistics
+using InteractiveUtils
 
 include("tools.jl")
 include("models.jl")

src/model_traits.jl

@@ -4,7 +4,7 @@
 const DERIVED_TRAITS = (:name, :ismodel)
 const ORDINARY_TRAITS = (
     :functions,
-    :target_proxy,
+    :target_proxies,
     :position_of_target,
     :position_of_weights,
     :descriptors,
@@ -62,7 +62,22 @@ See also [`LearnAPI.Model`](@ref).
 """
 functions(::Type) = ()
 
-target_proxy(::Type) = NamedTuple()
+target_proxies() = subtypes(TargetProxy)
+
+"""
+    target_proxies(model)
+
+Return a named tuple of target proxies, keyed on operation name, applying to `model`. For
+example, a value of
+
+    (predict=LearnAPI.Distribution(),)
+
+means that `LearnAPI.predict` returns probability distributions, rather than actual values
+of the target. View all target proxy types with `target_proxies()`. For more information
+on target variables and target proxies, refer to the LearnAPI documentation.
+
+"""
+target_proxies(::Type) = NamedTuple()
 
 position_of_target(::Type) = 0
 

src/operations.jl

@@ -1,9 +1,7 @@
-const PREDICT_OPERATIONS = (
-
 const OPERATIONS = (:predict, :predict_joint, :transform, :inverse_transform)
 
 const DOC_NEW_DATA =
-    "Here `report` contains ancilliary byproducts of the computation, or "*
+    "The `report` contains ancilliary byproducts of the computation, or "*
     "is `nothing`; `data` is a tuple of data objects, "*
     "generally a single object representing new observations "*
     "not seen in training. "
@@ -16,7 +14,7 @@ const DOC_NEW_DATA =
 
 Return `(ŷ, report)` where `ŷ` are the predictions, or prediction-like output (such as
 probabilities), for a machine learning model `model`, with learned parameters
-`fitted_params`, as returned by a preceding call to [`LearnAPI.fit`](@ref)`(model, ...)`.
+`fitted_params` (first object returned by [`LearnAPI.fit`](@ref)`(model, ...)`).
 $DOC_NEW_DATA
 
 
@@ -36,13 +34,13 @@ implementation itself promises, by making an optional [`LearnAPI.output_scitypes
 declaration.
 
 If `predict` is computing a target proxy, as defined in the MLJLearn documentation, then a
-[`LearnAPI.target_proxy`](@ref) declaration is required, as in
+[`LearnAPI.target_proxies`](@ref) declaration is required, as in
 
 ```julia
-LearnAPI.target_proxy(::Type{<:SomeModel}) = (predict=LearnAPI.Distribution,)
+LearnAPI.target_proxies(::Type{<:SomeModel}) = (predict=LearnAPI.Distribution,)
 ```
 
-Do `LearnAPI.target_proxy()` to list the available kinds.
+Do `LearnAPI.target_proxies()` to list the available kinds.
 
 By default, it is expected that `data` has length one. Otherwise,
 [`LearnAPI.input_scitypes`](@ref) must be overloaded.
@@ -96,10 +94,10 @@ For a supervised learning model, return `(d, report)`, where `d` is some represe
 the *single* probability distribution for the sample space ``Y^n``. Here ``Y`` is the
 space in which the target variable associated with `model` takes its values, and `n` is
 the number of observations in `data`. The specific form of the representation is given by
-`LearnAPI.target_proxy(model)`.
+[`LearnAPI.target_proxies(model)`](@ref).
 
-Here `fitted_params` are the model's learned parameters, as returned by a preceding call
-to [`LearnAPI.fit`](@ref). $DOC_NEW_DATA.
+Here `fitted_params` are the model's learned parameters (the first object returned by
+[`LearnAPI.fit`](@ref)). $DOC_NEW_DATA.
 
 While the interpretation of this distribution depends on the model, marginalizing
 component-wise will generally deliver *correlated* univariate distributions, and these will
@@ -109,10 +107,10 @@ generally not agree with those returned by `LearnAPI.predict`, if also implement
 
 Only implement this method if `model` has an associated concept of target variable, as
 defined in the LearnAPI.jl documentation. A trait declaration
-[`LearnAPI.target_proxy`](@ref), such as
+[`LearnAPI.target_proxies`](@ref), such as
 
 ```julia
-LearnAPI.target_proxy(::Type{SomeModel}) = (; predict_joint=Sampleable())
+LearnAPI.target_proxies(::Type{SomeModel}) = (; predict_joint=Sampleable())
 ```
 
 is required. Here the possible kinds of target proxies are `LearnAPI.Sampleable`,
@@ -129,9 +127,9 @@ function predict_joint end
     LearnAPI.transform(model, fitted_params, data...)
 
 Return `(output, report)`, where `output` is some kind of transformation of `data`,
-provided by `model`, based on the learned parameters `fitted_params`, as returned by a
-preceding call to [`LearnAPI.fit`](@ref)`(model, ...)` (which could be `nothing` for
-models that do not generalize to new data, such as "static transformers"). $DOC_NEW_DATA
+provided by `model`, based on the learned parameters `fitted_params` (the first object
+returned by [`LearnAPI.fit`](@ref)`(model, ...)`). The `fitted_params` could be `nothing`,
+in the case of models that do not generalize to new data. $DOC_NEW_DATA
 
 
 # New model implementations
@@ -167,7 +165,7 @@ the map
 data -> first(transform(model, fitted_params, data))
 ```
 
-For example, if `transform` corresponds to a projection, `inverse_transform` is the
+For example, if `transform` corresponds to a projection, `inverse_transform` might be the
 corresponding embedding.