global meta fields

[smolkaj]

Update: Most of this PR has already been merged (#521). The remaining bits are low priority.

This implements a feature suggested and discussed in #511. It introduces "meta fields" (or "logical fields"). They behave like other fields, but are erased during compilation.

They come in 4 versions: mutable or immutable, and initialized with a constant or initialized with the value of another field:

	constant	alias
immutable	let <meta> := <int64> in p	let <meta> := <field> in p
mutable	var <meta> := <int64> in p	var <meta> := <field> in p

Here is a typical use case:

(* create immutable meta field that carries original port value *)
let ingress := port in

(* by default, forward packets to controller *)
port := pipe("controller");

(* overwrite default behavior ... *)
begin
  if ingress=1 then port:=2 else
  if ethDst=10.0.0.1 then port:=1 else
  id
end

It compiles to the following table:

+------------------------------------------------------+
| Switch 0 | Pattern         | Action                  |
|------------------------------------------------------|
| InPort = 1                 | Output(2)               |
|------------------------------------------------------|
| EthDst = 00:00:00:00:00:01 | Output(1)               |
|------------------------------------------------------|
|                            | Output(Controller(128)) |
+------------------------------------------------------+

Note how the ingress field has been erased.

A more detailed discussion of the motivation for this feature can be found in #511. More examples can be found in examples/meta-fields/

Note that this feature is just syntax sugar, i.e. it does not increase the expressivity of the language: any program with meta fields can be rewritten to an equivalent program without meta fields (in fact, that's what the compiler does). However, meta fields are a useful language abstraction that sometimes allow for more modular, more concise, and more readable programs. The virtual compiler (with meta fields vswitch and vport) is a powerful examples of this.

TODO:

• fix automatic field ordering
• global compilation of meta fields -> replace adhoc vswitch and vport fields with meta fields
• document code a little bit, especially Env
• use inline records for Let constructor (see Support inline records of OCaml 4.03? janestreet/ppx_sexp_conv#9 (comment))
• allow using ip/mac addresses and masks for meta fields (done: ppx #527)
• syntactic convention for meta field names? (done: ppx #527)
• any suggestions? syntax?

[smolkaj]

Since I will be offline for a while, I merged in the code that's ready to go (namely, the local compiler now supports meta fields #521) and rebased this branch. This addresses feature-request #511.

Global compilation is a little more involved (and less important), so I will address this once I'm back.

[calebvoss]

I found some weird behavior. As of right now, when matching on or modifying meta fields, we have to use vanilla integers (maybe that will change eventually, especially to allow matching with masks). But there is an issue when using an integer that's at least 2^31 (IP >= 128.0.0.0) Here is some code that complains because my integer 0x80000000 is too big for int32 due to sign issues.

(* Fails to compile *)
let orig_srcip := ip4Src in (
   filter switch=1;
   filter ip4Src=10.1.1.0/24;
   filter not orig_srcip=0x80000000 (* 0x7fffffff would be fine *)
)

Error message:

Uncaught exception:

  (Failure
   "conversion from int64 to int32 failed: 2147483648 is out of range")

Raised at file "pervasives.ml", line 30, characters 22-33
Called from file "lib/Frenetic_Fdd.ml", line 438, characters 42-57
...

But the weirder part is that under some circumstances, there is no error:

(* Compiles with no mask on ip4Src *)
let orig_ip4Src := ip4Src in (
   filter switch=1;
   filter ip4Src=10.1.1.0;
   filter not orig_ip4Src=0x80000000
)

(* Compiles with no negation of meta field filter *)
let orig_ip4Src := ip4Src in (
   filter switch=1;
   filter ip4Src=10.1.1.0/24;
   filter orig_ip4Src=0x80000000
)

[jnfoster]

Thanks for the bug report.

This must be a fat-fingered numeric operation somewhere.

-N

On Tue, Aug 2, 2016 at 10:42 AM, Caleb Voss notifications@github.com
wrote:

I found some weird behavior. As of right now, when matching on or
modifying meta fields, we have to use vanilla integers (maybe that will
change eventually, especially to allow matching with masks). But there is
an issue when using an integer that's at least 2^31 (IP >= 128.0.0.0) Here
is some code that complains because my integer 0x80000000 is too big for
int32 due to sign issues.

(* Fails to compile )
let orig_srcip := ip4Src in (
filter switch=1;
filter ip4Src=10.1.1.0/24;
filter not orig_srcip=0x80000000 ( 0x7fffffff would be fine *)
)

Error message:

Uncaught exception:

(Failure
"conversion from int64 to int32 failed: 2147483648 is out of range")

Raised at file "pervasives.ml", line 30, characters 22-33
Called from file "lib/Frenetic_Fdd.ml", line 438, characters 42-57
...

But the weirder part is that under some circumstances, there is no error:

(* Compiles with no mask on ip4Src *)
let orig_ip4Src := ip4Src in (
filter switch=1;
filter ip4Src=10.1.1.0;
filter not orig_ip4Src=0x80000000
)

(* Compiles with no negation of meta field filter *)
let orig_ip4Src := ip4Src in (
filter switch=1;
filter ip4Src=10.1.1.0/24;
filter orig_ip4Src=0x80000000
)

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#517 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABwi0g8t5IAqFDaFIOU8LapC87c1-cvGks5qb4F_gaJpZM4JND3h
.

[smolkaj]

The problem occurs because OCaml's builtin integers (int, int32, int64) are signed. So the largest int32 is actually 2^31-1, not 2^32-1. Thus an error is thrown when the compiler tries to convert an int64 in [2^31, 2^32) to an int32.

I'm actually not sure what the best way to go about this is, because OCaml doesn't come with native support for unsigned integers. Any ideas, @jnfoster / @arjunguha / @basus ? I would expect this sort of problem would have come up before?

Update:
To be clear, all we really need is a function Int64.to_uint32 : int64 -> int32 that converts a int64 in the interval [0, 2^32-1] to a int32 (basically by just taking the lower 32 bits).

[basus]

Ctypes seems to have an Untyped module that might be what you need: https://ocaml.janestreet.com/ocaml-core/112.17/doc/ctypes.0.3.4/_build/src/ctypes/unsigned/?

[smolkaj]

@basus: It doesn't seem to have an usigned conversion function (see my updated post).

[smolkaj]

By the way, I checked and the parser suffers from the same bug. It uses Int32.of_string and Int64.of_string, but these functions throw an exception for integers greater than 2^31-1 and 2^63-1, respectively.

[jnfoster]

Perhaps this?

https://github.com/andrenth/ocaml-stdint

[smolkaj]

@calebvoss: You can now assign ip addresses and mac addresses directly to meta fields, see https://github.com/frenetic-lang/frenetic/blob/master/ppx/test/main.ml#L17-L18. We still need to fix the bug you mentioned (see #536), but this should do as a workaround.