Rethink custom/extensible validations

[mlms13]

The Problem:

One pervasive challenge in defining a decode library is the fact that OCaml/Reason's type system allows for different types than what JSON allows. This can be seen in the intFromNumber and date decoders, which impose rules on top of JSON values that JSON itself doesn't define.

It becomes an even more complex issue when dealing with Reason's "variants" which have no obvious mapping to/from JSON. In the past, bs-decode has provided two solutions to this:

• the ExpectedValidOption error, which is sort of a catch-all, but it carries no useful error information
• the ParseError.ResultOf and Decode.Make module functors, which is a powerful but complex solution that allows extending the underlying error type

Variant decoders have gotten easier to express with alt/oneOf and the recent addition of literal decoders, but the foundational literal decoders use ExpectedValidOption which means that don't give great error messages on failure.

The Proposal:

I haven't fully thought through this, but I'm considering adding a FailedValidation('e) error constructor.

Int and Date decoding would take advantage of this (eliminating the need for ExpectedInt and ExpectedValidDate). Literal decoders could use this, eliminating the need for ExpectedValidOption, and since the error carries a polymorphic payload, it would be easy to extend, hopefully without needing to get module functors involved.

The big thing I haven't fully figured out is how error reporting would work. Since we'll be using these validations internally, I think that means the 'e type will actually be an open polymorphic variant. For functions like ParseError.toString, you'll need to tell us how to convert your extensions to a string, but hopefully not the entire variant. :> might get involved, which is too bad, but I still think this is worth trying.

If implemented, we should:

• Add the constructor in ParseError
• Add a new validate function that is sort of like a combination of map and flatMap that lets the user return their own custom validations
• Use validate for ints (and remove ExpectedInt)
• Use validate for dates (and remove ExpectedValidDate)
• Use validate for literal decoders (and remove ExpectedValidOption)
• Probably remove ExpectedTuple (also see Deprecate tuple decoders #121)
• Document usage with really clear examples

[mlms13]

I haven't pushed anything yet (or even committed it, honestly), but I started playing around with how this could work, and I'm excited about the possibilities. At the moment, it's starting to look something like:

// in ParseError.re

module Error = {
  type t('a) =
    | Value(valueError, Js.Json.t)
    | Array(arrError('a), list(arrError('a)))
    | Object((string, objError('a)), list((string, objError('a))))
    | Validation('a, Js.Json.t)
  and valueError =
    | ExpectedNull
    | ExpectedBool
    | ExpectedNumber
    | ExpectedString
    | ExpectedArray
    | ExpectedObject
  and arrError('a) = {
    position: int,
    error: t('a),
  }
  and objError('a) =
    | MissingField
    | InvalidFieldValue(t('a));

  let expectedNull = json => Value(ExpectedNull, json);
  let expectedBool = json => Value(ExpectedBool, json);
  let expectedNumber = json => Value(ExpectedNumber, json);
  let expectedString = json => Value(ExpectedString, json);
  let expectedArray = json => Value(ExpectedArray, json);
  let expectedObject = json => Value(ExpectedObject, json);

  let arrError = (position, error) => {position, error};
  let arrayErrorSingleton = (position, error) =>
    Array({position, error}, []);
  let arrayErrors = (first, rest) => Array(first, rest);

  let objError = (field, error) => (field, error);
  let objectErrorSingleton = (field, error) => Object((field, error), []);
  let objectErrors = (first, rest) => Object(first, rest);
  let missingField = field => objectErrorSingleton(field, MissingField);
  let invalidFieldValue = (field, error) =>
    objectErrorSingleton(field, InvalidFieldValue(error));

  let validationError = (error, json) => Validation(error, json);
};

// in the actual decoders
let validate = (f, decode, json) =>
  decode(json)
  |> Result.flatMap(a =>
       f(a) |> Result.mapError(Error.validationError(_, json))
     );

let isValidInt = num =>
  float(int_of_float(num)) == num
    ? Ok(int_of_float(num)) : Error(`InvalidInt);

let int = validate(isValidInt, number);

// in a downstream project
let naturalNotThirteen =
  int
  |> validate(a => a >= 0 ? Ok(a) : Error(`InvalidNaturalNumber))
  |> validate(a => a == 13 ? Error(`UnluckyNumber) : Ok(a));