Composability and copy/paste safety

[brucou]

Just lost a long message I was just writing to you so I guess I should make this shorter.

I am the creator and maintainer of Kingly, a state machine library catering mostly to user interfaces, so I am pretty interested in the subject. Follows my feedback.

First of all, wow. You integrated miscellaneous technologies and tools into what seems to be a cohesive tool so congratulations for the end result, it looks good.

On visualization: I like to use yed as a visual graph editor for its nice user interface and impressive array of quality automated visualization algorithms. Now I do have some frustrations with the tool, but I found that better than the PlantUml editor, [state-machine-cat]9https://github.com/sverweij/state-machine-cat) and Cytoscape on real-world graphs. Automatically achieveing an insightful visualization of a graph is a difficult problem and I found that most of the aforementioned tools had troubles with self-loops and backloops to compound states. For small simple graphs though they are great. Cytoscape is open-source and probably the most advanced tool, specially in conjunction with an Elk layout.

References:

• The Elk layout algorithm, and its Cytoscape plugin

On textual languages vs. visual languages: there is a case for textual languages for sure. A great advantage of textual languages is the ease of copy/paste and replacing functionality, not mentioning navigating/search (and tooling) abilities. However I would believe the advantage is a bit dampened when it comes to state machines, as the latter does not compose so nicely. You can't copy a portion of the graph and paste it elsewhere and be done. There is a range of pitfalls there. The great thing is that you seem to check semantic contracts, which would effectively support copy/pasting as long as the contracts guarantee against the pitfalls. But still, a state links computations to given context. If you copy/paste the state, you are not copy/pasting the context -- which might be exactly what you want, or might be the next bug in your design. Contract checking I think would probably only cover against bad syntax and obvious coherency rules, leaving some room for other mistakes.

References:

• Kiel: Textual and Graphical Representations of Statecharts
• Statechart Development Beyond WYSIWYG

I recommend you test the language on actual examples. You will find some in Kingly tutorials and examples.

Here for instance are machines for a two-player chessgame, evolving from:

to:

It would be great to see how a textual format supports more easily the modification of a design with a view to remove, modify and add features.

I have other examples that are not public yet. If you are interested in that I can send them to you out-of-band.

[DrSensor]

^{Just want to make it easier to read 🙂}

On topic

However I would believe the advantage is a bit dampened when it comes to state machines, as the latter does not compose so nicely.

I think composability can be achieved using hierarchical state + import system 🤔 (hopefully)

You can't copy a portion of the graph and paste it elsewhere and be done. There is a range of pitfalls there.

Copy/paste safety is something that I've not think before, this is will be challenging. There is several way I can think of to accommodate this:

• Using type checking that also can be used as a schema similar how Dhall achieve this. However, I really-really want to avoid having type system since it will be cumbersome to implement and I don't know if it will fit with Scdlang semantics.
• Use the query syntax (Separate grammar for query syntax #14) as a schema. I think this is the most feasible way and fit with the abstraction layer I create. However, I need this pest feature to be implemented first.
• Via linter. This might be the easiest but also cumbersome. The user sure can write the rules in the linter configuration but they might miss some part of it because linter need to be explicit on everything.

Out of topic

On visualization: I like to use yed as a visual graph editor for its nice user interface and impressive array of quality automated visualization algorithms.

Thanks for the info, I will explore yed !! (I wonder if it can open/import SCXML or XMI file 🤔)

Automatically achieving an insightful visualization of a graph is a difficult problem and I found that most of the aforementioned tools had troubles with self-loops and back-loops to compound states. For small simple graphs though they are great. Cytoscape is open-source and probably the most advanced tool, specially in conjunction with an Elk layout.

This project is focus on code generation so drawing and automatic layout is out of scope. However, it still allow integrating with other tools as long I doesn't need to define the scalar type (e.g width, height, distance) and the layout. Seems like the author of state-machine-cat had experiment with various graph engines before (see here). I wonder if there is a CLI (server) or DSL that use Cytoscape 🤔

Now I do have some frustrations with the tool, but I found that better than the PlantUml editor, state-machine-cat and Cytoscape on real-world graphs.

I actually want to support PlantUML too since it accept graphviz/dot format as an input. However, I can't find the official stand alone CLI for PlantUML, only the jar file and node-plantuml (maybe I should support both in the future.

By the way, this is the current integration looks like under the hood 😂

I am the creator and maintainer of Kingly, a state machine library catering mostly to user interfaces

I think I want to make Kingly as additional target since the FSM definition is similar to the AST schema of State Machine Cat. Let me know if Kingly has some sort of schema definition or typescript declaration for that, it will help me to generate the object model 🥺.

I recommend you test the language on actual examples. You will find some in Kingly tutorials and examples.

Absolutely! I wonder what insight I can gain when writing FSM for UI with the aid of both state diagram, transition table, and timing diagram 😎

I have other examples that are not public yet. If you are interested in that I can send them to you out-of-band.

Many thanks. You can send that to sensorfied@gmail.com

[brucou]

I think composability can be achieved using hierarchical state + import system 🤔 (hopefully)

It may not be that easy. To give you an example, a suspense functionality could be described by a machine, and you may want to reuse this all over the place as it is pretty generic. I demoed it with Kingly with the following machine:

So this machine has one input and two outputs. The input is the default one. The outputs, when you reuse become inputs of what comes after. So you may want to parameterize the outputs (so you can rename the end states), add your suspense machine textual description and add a few lines doing the linking outputA -> inputX, outputB -> inputY. In short your reusable graph is a function, and you need to identify inputs and outputs, and have syntactic support in your description language for aliasing and reassignment. I quickly looked at Dhall, and given it has functions, maybe a similar technique could be used to solve this. It is a question of experimenting. In what you currently proposed (import system -- I looked also very quickly so apologies if I misunderstood), I see how you rename the input of the reusable graph but not the outputs.

I am a bit theoretical here, so the best is probably to take that graph, remove the spinning state, and then you have the state machine for a promise. Then try to reuse that promise in places.

Seems like the author of state-machine-cat had experiment with various graph engines before (see here).

The link seems to be old. Cytoscape has since seen a large number of improvements.

I wonder if there is a CLI (server) or DSL that use Cytoscape 🤔

There is a CLI but I don't know if it does what you want. Check the documentation. I assume you want it to take a graph as input and output a svg or other format. You can pass inputs from the CLI but I am not sure it outputs a graph. Cytoscape has a much broader scope than graph visualization. However there is a REST interface where maybe the graph visualization could be obtained. Cytoscape also has a scripting language. Don't know, it is about scouring through the docs. The tool is great and there are a ton of visualizations that you can get for free and which will be useful for real-life graphs.

I think I want to make Kingly as additional target since the FSM definition is similar to the AST schema of State Machine Cat.

That's great.

Let me know if Kingly has some sort of schema definition or typescript declaration for that, it will help me to generate the object model 🥺.

The fsm definition forrmat is by design simple with redundancy built-in to allow for cross-checking. There are some types here. An excerpt:

/**
 * @typedef {Object} FSM_Def
 * @property {FSM_States} states Object whose every key is a control state admitted by the
 * specified state machine. The value associated to that key is unused in the present version of the library. The
 * hierarchy of the states correspond to property nesting in the `states` object
 * @property {Array<EventLabel>} events A list of event monikers the machine is configured to react to
 * @property {Array<Transition>} transitions An array of transitions the machine is allowed to take
 * @property {*} initialExtendedState The initial value for the machine's extended state
 * @property {{updateState :: Function(ExtendedState, ExtendedStateUpdate) : ExtendedState}} updateState function
 * which update the extended state of the state machine
 */

Absolutely! I wonder what insight I can gain when writing FSM for UI with the aid of both state diagram, transition table, and timing diagram 😎

The transition table and timing diagram links do not work. By timing diagram are you referring to stuff like this:

There is a good example of transition table in a deep-dive article I published giving a 10,000 feet view of the state machine methodology.

Many thanks. You can send that to sensorfied@gmail.com

Just published an example of routing with state machines:

[DrSensor]

So this machine has one input and two outputs. The input is the default one. The outputs, when you reuse become inputs of what comes after... I quickly looked at Dhall, and given it has functions, maybe a similar technique could be used to solve this... In what you currently proposed (import system -- I looked also very quickly so apologies if I misunderstood), I see how you rename the input of the reusable graph but not the outputs.

My bad not writing the draft clearly since I try to design that to be used in conformance test or alike 😅. Base on that diagram, here is how it looks like when written in Scdlang:

/// suspense.scl
state Suspense {
  initial -> Pending |> startTimer
  Pending -> Spinning @ TimerExpired |> renderFallback
}

/// main.scl
state { Suspense } = import`./suspense.scl`

initial -> [Suspense] @ "START" |> runOperation
[Suspense] -> Error @ Error |> renderErrorComponent
[Suspense] -> Done @ Ready |> renderComponent

I've not thinking about the aliasing for elements inside the compound state yet. If it's necessary, then it might be looks like this:

state { Suspense } = import`./suspense.scl`

[override]
state Suspense {
  action renderFallback = renderErrorComponent
}

this will replace renderFallback with renderErrorComponent.

---

There is a CLI but I don't know if it does what you want. Check the documentation. I assume you want it to take a graph as input and output a svg or other format...

I've installed and tried Cytoscape with only small luck 😢. The format I can produce that only accepted by Cytoscape is GraphML (cytoscape-docs) which is produced by graph-easy. After I import it, it only render the transition and not provide the label name for the event (maybe because GraphML is old format). It's a bit unfortunate that Cytoscape doesn't support Graphviz/DOT language. However, the good part is they support table (CSV and others) but I don't know how to map them. More specifically, how the transition table is looks like so that it can be imported in Cytoscape.

My fail attempt

High level overview on how I do it

---

The fsm definition forrmat is by design simple with redundancy built-in to allow for cross-checking. There are some types here. An excerpt:

Thanks 👍. Since it's JSDoc, let me know if you found tools to convert JSDoc into Typescript or JSON so that I can pass it to https://quicktype.io

---

The transition table and timing diagram links do not work. By timing diagram are you referring to stuff like this:

My bad. Here it is which link to this project issues:

• Transition Table Generate state transition table #24
• Timing Diagram Generate timing diagram #27. There is also a proposal to support this in PlantUML (here)

Btw, that "Image search scenario" you share is a Sequence Diagram which will be handy for visualizing interaction between several State Machine (or State Machine that invoke a service).

---

There is a good example of transition table in a deep-dive article I published giving a 10,000 feet view of the state machine methodology.

Just read it a while ago and I wonder how the table will looks like when entry, exit, and activity was involved 🤔

---

Just published an example of routing with state machines:

Nice!

[brucou]

ok got you.

Transition Table #24

I was a bit confused by your transition table. A transition table should have a shot of before and after. So I put before(control state, extnded state, input) and after (control state, extended state, output). But you seem to put two events? I guess there may be another column for the guard? But two events in the same row I don't understand the semantics of it if any.

Timing Diagram

wow. You know a bunch of nice visualization tools. Yeah I guess that could be nice, though I am not sure so far I needed a visualization like that. What do you analyze like that? Performance? What else would you use the duration information for?

Since it's JSDoc, let me know if you found tools to convert JSDoc into Typescript

There are a few. Typescript actually accepts JSDoc 'types'. But they all have caveats. I don't know so much about typescript specially the latest releases. But for somebody who knows it should not be too hard, I don't use anything fancy, and JSDoc anyways is much less expressive than TS. You said you need to generate an object model. How is that linked to converting your format to Kingly's? Actually I guess you parse this file right? What does the output of the parser look like?

If you look at the Suspense machine, in Kingly format it would be:

// Array of strings
export const events = [TIMER_EXPIRED, SUCCEEDED, FAILED, START];
// Object whose hierarchy is that of the states. Values for keys are always empty string
const states = {
  [INIT]: "",
  [SUSPENSE]: {
    [PENDING]: "",
    [SPINNING]: "",
  },
  [ERROR]: "",
  [DONE]: "",
};
// String
const initialControlState = INIT;
// Object
const initialExtendedState = {};
// Array of transitions -- that is the most complex format
// That's where the JSDoc comes in handy
// Here there is only the standard transitions (no guards) showcased
const transitions = [
  { from: INIT, event: START, to: SUSPENSE, action: runOperation },
  { from: SUSPENSE, event: INIT_EVENT, to: PENDING, action: startTimer },
  { from: PENDING, event: TIMER_EXPIRED, to: SPINNING, action: renderFallback },
  { from: SUSPENSE, event: SUCCEEDED, to: DONE, action: renderSucceeded },
  { from: SUSPENSE, event: FAILED, to: ERROR, action: renderError },
];

// That is the machine definition
export const fsmDef = {
  initialControlState,
  initialExtendedState,
  states,
  events,
  transitions,
// Don't mind that
  updateState
};

Just published an example of routing with state machines:

Nice!

Did you read it? What do you think? Is it understandable?

I've installed and tried Cytoscape with only small luck

I can't help you here, I only use the Cytoscape JS library, not the CLI. So the CLI only takes graphml? I have a visualizer here which uses cytoscape. I don't remember what is the name of the format I use but:

• edges are array of { data: { id, source, target, type, label } all (id, label, etc.) being strings
• nodes are array of {data: {id, label, parent}}

Example code here

Now to the hardest part to explain, composability:

Your example works. Actually I understand better your format with it. It is pretty concise though it takes some time to read (parse, gotta get used to the conventions)

But that is not what I was referring to. I am referring to using the whole Suspense machine (e.g. the whole thing you wrote in your format) within a bigger machine. The Suspense behaviour in the end needs an event to start, but it ends in two control states from which there is no further transitions right? The idea is to plug this suspense machine into a bigger machine which reuses it in different places. so what you want to replace here (or parameterize) are the terminal states.

From your dhal example I was thinking more about a function:

SuspenseMachine (INITIAL, DONE_STATE, ERROR_STATE):
  state { Suspense } = import`./suspense.scl`

  INITIAL -> [Suspense] @ "START" |> runOperation
  [Suspense] -> ERROR_STATE @ Error |> renderErrorComponent
  [Suspense] -> DONE_STATE @ Ready |> renderComponent

Then you can plug the machine anywhere in another machine by replacing the initial state with some other state from the larger machine, and the terminal states by two other states from the larger machine.

Note that your example works nicely because the Suspense compound control state does not have edges arriving or leaving INSIDE the compound state. If you start to need to know the inside of the compound state, e.g. the implementation details of the compound state, then it gets complicated I think to keep a nice import syntax. Or am I wrong?

The Movie Database Interface has such examples with the Movie Querying control state. and the Movie Detail control state. The Movie Detail control state needs to know the internals of the Movie Querying control state. That makes sense because they are indeed coupled through the id of the movie being selected. That does not happen in suspense because the fetching data behaviour is completely independent from the rest of the machine.

Also

I wonder how the table will looks like when entry, exit, and activity was involved

Entry and exit actions are actions, so you can add them in the actions columns like any other actions. Activity are also actions, so same thing. Actually they are usually implemented like that. An activity is mapped to an entry action for a control state which is to start the activity, and an exit action which is to stop the activity. I don't have entry, exit, and activities actions in Kingly as I don't want to have a complex syntax due to all the related sugar. The current syntax already allows all this without needing anything else. It is just a bit more verbose but then much simpler.

[DrSensor]

I was a bit confused by your transition table. A transition table should have a shot of before and after. So I put before(control state, extnded state, input) and after (control state, extended state, output). But you seem to put two events? I guess there may be another column for the guard? But two events in the same row I don't understand the semantics of it if any.

Let's discuss it in #24

---

wow. You know a bunch of nice visualization tools. Yeah I guess that could be nice, though I am not sure so far I needed a visualization like that. What do you analyze like that? Performance? What else would you use the duration information for?

My background is electrical engineering so it might get influence from that. There are several way to analyze FSM in ASIC design (see here). Let's move the discussion in #27.

---

There are a few. Typescript...

Would you mind to specify? 🙂

---

... Typescript actually accepts JSDoc 'types'. But they all have caveats. I don't know so much about typescript specially the latest releases. But for somebody who knows it should not be too hard, I don't use anything fancy, and JSDoc anyways is much less expressive than TS. You said you need to generate an object model. How is that linked to converting your format to Kingly's? Actually I guess you parse this file right? What does the output of the parser look like?

Just want to make my life easier. Converting from one schema to Rust manually is a bit daunting 😅. Well, I still can use the object/ast model of State Machine Cat but I still need to edit them to fit Kingly schema.
To answer your question: Nope, I don't parse that file. Here if you want to know how the object model looks like (in Rust).

---

Just published an example of routing with state machines:

Nice!

Did you read it? What do you think? Is it understandable?

Honestly, it's like reading a schematic diagram 😅. Let's discuss it in brucou/kingly#2

---

I can't help you here, I only use the Cytoscape JS library, not the CLI. So the CLI only takes graphml? I have a visualizer here which uses cytoscape. I don't remember what is the name of the format I use but:

Well, the CLI itself is just a wrapper for the GUI so I can't integrate it in this project. The Cytoscape program itself (not library) can accept various type (see here) but I can't produce none of them except graphml.

I think I will refrain from digging this Cytoscape things since it's more like data visualization rather than diagramming tools (don't ask 😋). I see a pattern where languages specific FSM framework (e.g xstate, django-fsm, sismic) tend to provide visualization tools. I think I will hitchike that framework just like how I target State Machine Cat to generate state diagram on many format (including ascii art). Hope there is a framework that provide state diagram visualization using Cytoscape 😏.

[DrSensor]

Entry and exit actions are actions, so you can add them in the actions columns like any other actions. Activity are also actions, so same thing. Actually they are usually implemented like that. An activity is mapped to an entry action for a control state which is to start the activity, and an exit action which is to stop the activity. I don't have entry, exit, and activities actions in Kingly as I don't want to have a complex syntax due to all the related sugar. The current syntax already allows all this without needing anything else. It is just a bit more verbose but then much simpler.

Activity is not same as action. Also, overuse of entry and exit can cause double execution. I will put the details in #27 (stay tune 😉)

[DrSensor]

From your dhal example I was thinking more about a function:... Then you can plug the machine anywherein another machine by replacing the initial state with some other state from the larger machine, and the terminal states by two other states from the larger machine.

After thinking for a while, this is exactly what macro is in programming language 😮. Though it's more like C-macro rather than hygienic macro. Alright let's try this:

macro SuspenseMachine {
  state { Suspense } = import`./suspense.scl`

  INITIAL -> [Suspense] @ "START" |> runOperation
  [Suspense] -> ERROR_STATE @ Error |> renderErrorComponent
  [Suspense] -> DONE_STATE @ Ready |> renderComponent
}

I approach it by making the token (e.g ERROR_STATE, INITIAL) implicit (auto infer type) so it's easy to type.

state { Terminate } = import:scl`git@github.com:org/project.git/terminate`
macro { SuspenseMachine } = import`./suspense-machine.scl`

Ready @ Refresh |> reload // internal transition

use SuspenseMachine {
  state INTIAL = Ready,
	DONE_STATE = finish,
	ERROR_STATE = [Terminate]
}

initial -> Ready
Ready -> [ComplexMachine] @ Loaded // this surely cause confusion but whatever 😋

However, when using the macro, it should be explicit what type is being replaced.

Well, I think SuspenseMachine is a bad example for this. Compound state is much more preferable for this case.