Comprehensive graph analysis & visualization for nodely environments

[henry-p]

Overview

This PR introduces powerful graph analysis and visualization capabilities for Nodely environments, enabling developers to understand complex data flow structures through both statistical analysis and visual representations.

Key Features

Graph Analysis & Statistics

• Complete dependency extraction from all node types (value, leaf, branch, sequence)
• Recursive pattern detection for >and, >or, and >cond branch structures
• Comprehensive statistics generation including node counts, edge counts, and dependency analysis
• Embedded node support for detailed branch structure analysis

{:total-nodes 23, :total-edges 18
 :node-types {:value 9, :leaf 8, :branch 4, :sequence 2}
 :dependency-counts {:user-age 3, :user-profile 2, :threshold 1, ...}
 :max-dependencies 4}

DOT Graph Visualization

• DOT format generation compatible with Graphviz
• Smart edge labeling showing control flow relationships (condition, truthy, falsey, cond-expr, etc.)
• Embedded node visualization for complex branch structures
• Value preview system with HTML escaping and truncation
• Reflection-based function name extraction using var metadata and source file analysis

Example Visualization

Example of a complex nodely environment with branches, conditions, and embedded nodes visualized as a directed graph.

Usage Example

(comment
  (require '[nodely.analysis.visualize :as viz]
           '[nodely.analysis.visualize-test :refer [comprehensive-test-env]]
           '[clojure.java.io :as io])

  (let [analysis (viz/analyze-nodely-env comprehensive-test-env)
        dot-format (:dot-format analysis)
        filepath "tmp/graph.dot"]
    (println "Stats:" (:statistics analysis))
    (io/make-parents filepath)
    (spit filepath dot-format)))

;; Convert dot to png: $ dot -Tpng tmp/graph.dot -o tmp/graph.png
;; on macOS open like this: $ open "tmp/graph.png"

API Modes:

• (analyze-nodely-env env) - auto-detects source file using reflection
• (analyze-nodely-env env :disabled) - skips function name extraction entirely

Architecture

Analysis Pipeline

1. Node Extraction → Extract all nodes with type information and dependencies
2. Pattern Detection → Identify >and, >or, >cond patterns in branch structures
3. Embedded Node Analysis → Extract inline expressions from complex branches
4. Statistics Generation → Compute comprehensive metrics

Visualization

DOT Generation → Create valid Graphviz format with semantic edge labels and professional styling

Advanced Capabilities

• Pattern Recognition: Automatically detects >cond, >and, >or structures with nested branch support
• Type-Safe Previews: Handles all Clojure data types with smart truncation and HTML escaping
• Edge Semantics: Distinguishes between different control flow relationships

Benefits

• Understand Dependencies: See exactly which nodes depend on what
• Documentation: Generate visual documentation of data flows
• Code Review: Visual graphs make complex logic easier to review

Breaking Changes

None - this is purely additive functionality with zero external dependencies.

Key Functions:

• graph/extract-graph-structure - Main analysis entry point
• viz/analyze-nodely-env - Analysis + visualization with flexible API modes
• viz/auto-detect-source-file - Reflection-based source file detection

[aredington]

Thanks for your interest in contributing to Nodely!

On a first review I think this demonstrates a lot of thoughtful work and a good understanding of our under documented data model.

I've left a few specific comments but on reading through more of the PR here's some changes I'd like to see generally applied to proceed:

• When interacting with the data model of Nodely nodes, please only use functions from the nodely.data namespace. This space is not comprehensive, nor sufficient for the use cases you've demonstrated with this work, so expanding what's provided there is reasonable scope for this feature. Let's consider keywords with :nodely.data a code smell for the analysis namespaces.
• Toward the end in analysis we see a lot of creation of an anonymous function closing over arguments, and, then immediate invocation of the anonymous function. I think this is to establish some recursive calls? Can these be reformulated to a tail call optimizable structure? If we can TCO do we still need recursion depth constants?
• The same few pieces of information are extracted in the headers of these functions. These seem like good candidates for destructuring.

I want to keep the feedback sizes digestible, so, let's focus on these changes at first and then we will evaluate further. I think this is a strong basis for us to include these features in Nodely!

[henry-p]

@aredington I addressed a bunch of your comments. Hope that gets it closer.

[aredington]

Few minor requests, but this is looking pretty good!

To set expectations, @sovelten is out and right now we're planning to hold on a merge until she can take a look, but overall this is some great work. Thanks for taking the time with it!

[henry-p]

any updates on this?