Graph exploration tool

Hi Robert,

This is a fantastic question and something we always had in our backlog. As you have mentioned, we did some experimentation in the past but are now re-exploring this feature again, but we are hoping to understand the various use cases better first. I was wondering if you had some sample use cases where exploring via a graph would be more intuitive than the existing way in Data Explorer in Fusion (where users go from Asset → other resources and so forth via a tabular UI)?

Thanks,

David

Hi David, 

The clearest use is one which the tabular view might cover neatly as well. Relationships are directional, and so the app-developer needs insight into that directionality. The lack of this feature adds friction.

For the app-developer or the data-scientist the relationship itself is also an object of interest that you want surfaced in your graph-exploration. Most notably for the Labels associated with a given relationship. Which is a necessary filter for us for performance reasons.

We are often designing traverses in the graph. And, I don’t want the app-developer to have a note-pad up and noting the path as they explore. The  table view cannot replicate this feature.

Lastly, an DAG is much more naturally explored via nodes and edges. A table makes sense for a tree, which our model is not.

As it stands now the app-developer or designer cannot explore the model on their own. And, must involve a subject-matter-expert and data-scientist for each new feature adding a graph-traversal. We would like to remove the data-scientist from this feedback-loop. 

Hi Robert,

These are all great use cases where a 2D graph viewer would be much more efficient and effective than the current tabular view. We have added all of the details you have provided into our backlog. This is something many other users of CDF has asked about, so we definitely are aware of the importance of this feature. We have started a new team focusing on Development Experience so we really want to prioritise when we look at our longer term goals. I will definitely keep you in the loop regarding this feature and provide more details in January 2022.

Thank you so much for all the details and use cases, this is incredibly useful and will help us push for prioritizing such a feature!

David

Hi David,

Thanks for your response.We’re currently working on a prototype graph exploration API internally at Statnett. If you wish, we can invite you in for a session on our experiences once we have something we use internally.

Secondly, it’s important to note that the graph-traversal tool is not only central in our vision for the Developer Experience, as you say. We will (probably) be changing our end-user facing UX in 2022 to be graph-based and not tree-based. @Ola Øyan  and @Morten Knutsen  are stakeholders here.

This has implications for the scope of the graph exploration service, especially as it pertains to filtering and traversal of the graph. I.e a view of the model from line to substation might include switchgear etc. that we don’t want to show the end user. Does that filtering happen client or server side?

Look forward to hearing your thoughts,

Robert

Hi Robert, 

I would love go hear your experience and see what the graph exploration API is all about. I was curious, would that also mean that y’all are building also a graph exploration UI prototype as well at Statnett?

Additionally, on the graph based exploration for Statnett as the primary way to explore data, this makes a ton of sense based on how data is modelled currently, and we definitely want to help Statnett have a smooth experience building such a UI for the end users. It’s also great you have mentioned the goal being in 2022!

As for the filtering, we are hoping that the visualization component should be as de-coupled from the business logic as possible. Hence it should be the builders of the frontend app to decide to whether include that logic of filtering in the server (or middleware) or the application logic. Then, whatever the result of after the filtering has been completed, the visualization component should be able to handle and display the result. This is also where we would love to iterate with you and make the configuration aspect of the component really smooth. 

David

Yeah bud! 

We’re currently iterating on a GraphQl API as a middleware between CDF and a typescript frontend. For now we’re exploring how well HighCharts handles graphs. It’s a nascent project, and we’re budgeting a week or two of developer time in before years end.

We are in agreement in that we should de-couple the visualization from the business logic. This is where we found the need for the middleware API. It follows that a major point of interest for us is what you envision for the data-structures that the visualization consumes. As you say, we’re excited for the iteration. 

Robert

Hi Robert, 

I will schedule up a follow up next week with you, Ola and Morten to gather your thoughts and share some of our thinking as well. The interface for the visualization is something we will want to iterate with you on in building this feature.

A quick question on HighCharts, do you mean visualization with something like this: https://www.highcharts.com/docs/chart-and-series-types/network-graph? 

David

Yep, that’s where we started. And a quick update before we meet tomorrow is that we’ve pivoted from HightCharts and towards Cytoscape.js for visualization. The render engine broke under the back of a couple hundred nodes, and a few thousand edges.