A few months ago, I took a look at Nathan Marz's Storm project. As someone who has used and implemented Actor-like systems a few times over the years, always fun to see another take. Although Storm is aimed at the "cloud", it's easy to look at it and see how you might be able to use it for local computation. In fact, I have a problem area I'm working in right now where something like Storm might be useful, but I just need it to work in a single process. In JavaScript.
Another recent problem area I've been looking at is async, and
using Promises to help with that mess. I've been looking at
Kris Kowal's q
specifically.
It's a nice package, and there's a lot to love.
But as I was thinking about how I was going to end up using them, I imagined
building up these chained promises over and over again for some of my
processing. As live objects. Which, on the face of it, is insane. If
I can build a static structure and have object flow through them instead.
Should be able to cut down on the amount of live objects created, and it will
probably be easier to debug.
So, with that in mind, I finally sat down and reinterpreted Storm this evening, with my dustup project.
Some differences from Storm, besides the super obvious ones (JavaScript vs Java/Clojure, and local vs distributed):
One of the things that didn't seem right to me in Storm was differentiating spouts from bolts. So I only have bolts.
I'm also a fan of the PureData real-time graphical programming environment, and so borrowed some ideas from there. Namely, that bolts should have inlets where data comes in, and outlets where they go out, and that to hook bolts together, that you'll connect an inlet to an outlet.
Designed with CoffeeScript in mind. So you can do nice looking things like this:
x = new Bolt outlets: stdout: "standard output" stderr: "standard error"which generates the following JavaScript:
x = new Bolt({ outlets: { stdout: "standard output", stderr: "standard error" } })and this:
Bolt.connect a: boltA, b: boltBwhich generates the following JavaScript
Bolt.connect({a: boltA, b: boltB})Even though I designed it for and with CoffeeScript, you can of course use it in JavaScript, and it seems totally survivable.
Stopping there left me with something that seems useful, and still small. I hope to play with it in anger over the next week or so.
Here's an example of a topology that just passes data from one bolt to another,
in CoffeeScript:
(here's the JavaScript
for you CoffeeScript haters)
In the lines 1-2, we get access to the dustup package and
the Bolt class that it exports.
On lines 4-6, we're creating a new bolt that only has a single outlet, named
"a"
(the string value associated with the property a - "outlet a" - is ignored).
On lines 8-11, we're creating a new bolt that only has a single inlet, named
"b". Inlets need to provide a function which will be invoked when the
inlet receives data from an outlet. In this case, the inlet function will
write the data to "the console".
On line 13, we connect the a outlet from the first bolt to the b inlet of
the second bolt. So that when we send a message out the a outlet, the
b inlet will receive it.
Which we test on line 15, by having the first bolt emit some data. Guess what happens.
next
The next obvious thing to look at, is to see how asynchronous functions, timers, and intervals fit into this scheme.
