create FlowLog event from FlowRemoved

[adferguson]

we now have a basic stub to accept FlowRemoved events from the NetCore runtime. these should be exposed as events within FlowLog, like switch_port, switch_down, etc.

one design point is how to connect these events to FlowLog programs. ideally, it should be very natural, at a high-level, and not expose programmers to the details of OpenFlow switches.

for now, we can probably expose the raw match / pattern which expired. the FlowLog program will be responsible for interpreting it.

if we want to connect things at a higher level, we will probably need to associate the OpenFlow0x01_Core.pattern = OpenFlow0x01.Match.t which is returned by the FlowRemoved event with the original predicate (NetCore_Types.pred) we sent in the Seq(Filter(pred), Action(...)) partial evaluation.

to convert a NetCore_Types.pred to the low-level OpenFlow match, we can call NetCore_Compiler.CompilePol.compile_pred which will give us a "BoolClassifier". roughly speaking, this is a list of NetCore patterns which accept or reject packets. to be specific: the type of BoolClassifier.t is (NetCore_Pattern.t * bool * NetCore_Types.ruleMeta) list. finally, a call to NetCore_Pattern.to_match0x01 will convert that NetCore_Pattern.t to the OpenFlow match we received in the FlowRemoved message.

[tnelson]

(1) Started looking at the naive version of this (exposing match, and letting the Flowlog program interpret it) today. It doesn't look bad on the surface, but then I ran into a problem. Suppose we expose an expiring rule as an event, like we did for switch_down, etc. What type does the match have? It's not a Flowlog type, so we need some sort of construct to interpret it. E.g.,

ON expired_flow(exp):
    DELETE (x, y, z) FROM nat WHERE
        z = exp. // <--- what goes here?

One way to do this is to make some more built-in functions for matches, so we could say:

WHERE exactTpSrc(z, exp.match)

but the XSB conversion isn't straightforward. Maybe there's a smarter way?

(2) Do we want to re-assert expiring rules, to make sure that the switch policy remains in synch with the controller state? (For instance, what would happen if a MAC-learning rule expired?)

(3) What is the high-level abstraction that we want to hit? Is it that tuples in state tables have a lifetime (in that, if all rules they contribute to don't get used, they will eventually be removed)? Or something different?

[adferguson]

for (1), I think we could just lift the interesting fields (nwsrc, nwdst, tpsrc, tpdst, etc. as we'd need for NAT) out of the match into the FlowLog event. since we don't support a hierarchy within the fields of an event, we do that already with ip_packet -- we have ip_packet.nwsrc, ip_packet.nwdst, but not ip_packet.tos or ip_packet.frag, etc.

for (2), no, we do not want to re-assert the expiring rules automatically; rather, we would want the controller state to be updated by the expiring rules (the synch goes the other way). for example, with NAT, when the rule expires, the tuples in the database which caused that rule to be generated are to be removed (same would apply for MAC learning). this really gets to the heart of your point (3), though.

as for (3), that's a really great question, and I don't know off-the-bat -- can you make a note of it to discuss in person with Shriram when I return? I think it will require us to think of some more examples in which we want to use FlowRemoved and see what the abstraction should be. I think giving the tuples in the state tables a lifetime as you describe is a really interesting idea, although I also think that's not quite what we're doing with the NAT example. in the NAT example, we are need to release limited resources (transport ports available for the NAT rules), and we can only use timeouts as a way to approximate when the resources can be released (even if we could match on TCP FIN packets, that would just be an optimization)

[tnelson]

Pushed a basic version of this. But I'm not sure if I'm testing properly.
I'm trying to use dpctl del-flows to force deletion of several rules, but
it isn't triggering the event (or the watchdog printf, either). Should
deleting flows manually be triggering the event? What other ways are there
to test?

Note that since Flowlog has no option types, I'm using the value 0 to
denote a wildcard. This is likely bad; if necessary I can add flags for
wildcards to the event.

On Thu, Jan 2, 2014 at 11:22 PM, Andrew Ferguson
notifications@github.comwrote:

for (1), I think we could just lift the interesting fields (nwsrc, nwdst,
tpsrc, tpdst, etc. as we'd need for NAT) out of the match into the FlowLog
event. since we don't support a hierarchy within the fields of an event, we
do that already with ip_packet -- we have ip_packet.nwsrc, ip_packet.nwdst,
but not ip_packet.tos or ip_packet.frag, etc.

for (2), no, we do not want to re-assert the expiring rules automatically;
rather, we would want the controller state to be updated by the expiring
rules (the synch goes the other way). for example, with NAT, when the rule
expires, the tuples in the database which caused that rule to be generated
are to be removed (same would apply for MAC learning). this really gets to
the heart of your point (3), though.

as for (3), that's a really great question, and I don't know off-the-bat
-- can you make a note of it to discuss in person with Shriram when I
return? I think it will require us to think of some more examples in which
we want to use FlowRemoved and see what the abstraction should be. I think
giving the tuples in the state tables a lifetime as you describe is a
really interesting idea, although I also think that's not quite what we're
doing with the NAT example. in the NAT example, we are need to release
limited resources (transport ports available for the NAT rules), and we can
only use timeouts as a way to approximate when the resources can be
released (even if we could match on TCP FIN packets, that would just be an
optimization)

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/41#issuecomment-31504635
.

[adferguson]

great Tim, thanks! I will give this a try later today. it looks like it will be enough for us to get started.

regarding the option types / wildcards -- this is clearly one of the wrinkles we will have to consider when designing the abstraction and determining how to do this "properly" in Flowlog.

and yes, deleting permanent flows manually should not trigger the event if you are using my current version of NetCore, unmodified. OpenFlow has a flag designated for whether or not the switch should send a FlowRemoved notification (OFPFF_SEND_FLOW_REM), and I am only setting that flag to true if it is a flow with an IdleTimeout; see should_notify in the controller: https://github.com/adferguson/frenetic/blob/netcore/lib/NetCore_Controller.ml#L325

make sense?