HaverLab
/
nwa
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Fixing Version Branch Tools

This commit is contained in:
John Haverlack 2026-07-04 19:51:53 -08:00
parent b15bc074ae
commit 5af215dc81
6 changed files with 37 additions and 540 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

112
docs/CHANGELOG.md
View File

@ -1,118 +1,14 @@
# Software Defined Laboratory (SDL) # Node Web App (NWA)
| Attribute | Value | | Attribute | Value |
| --- | --- | | --- | --- |
| **Author** | John Haverlack | | **Author** | John Haverlack |
| **Copyright** | 2026 John Haverlack | | **Copyright** | 2026 John Haverlack |
| **License** | MIT | | **License** | MIT |
| **Version** | 0.3.10 | | **Version** | 0.1.10 |
| **Date** | 2026-02-08 | | **Date** | 2026-02-08 |
## v0.3.10 - 2026-02-08 (ALPHA) ## v0.1.0 - 2026-02-08 (ALPHA)
- Dash Staging Jobs
- Dash Full sdl_id
- Updating main Web UI Page
- Start v0.3.10
## v0.3.9 - 2026-02-08 (ALPHA)
- v0.3.9
- Refactor Cluster Status Dash
- Refactor Cluster Status Dash
- System Columnn Display
- Start v0.3.9
## v0.3.8 - 2026-02-01 (ALPHA)
- v0.3.8
- Refactoing Worker Dashboard
- Refactoing Worker Dashboard
- Refactored Cluster Status
- Refactor of telemetry format + hw detection
- Switched Panel Icon to Bootstrap Icon
- Updating Worker Telemetry
- CP
- Fixing Wget/Curl Command Installer Port
- Fixing Wget/Curl Command Installer Port
- Fixing Wkr Start / Stop
- Update stop-sdl-wkr.sh
- WIP Install / Start / Stop
- WIP Install / Start /Stop
- Systemd Detectiion in Start/Stop/Status
- WIP: Start / Stop Scripts
- Non Systemd start stop scripts
- WIP: start stop scripts
- Updating Task List
- Adding Screeshot to README
- Start v0.3.8
## v0.3.7 - 2026-02-01 (ALPHA)
**Summary**
sdl-wkr Telemetry
- v0.3.7
- Fixing sdl-wkr RAM detection
- Start v0.3.7
## v0.3.6 - 2026-02-01 (ALPHA)
**Summary**
Light/Dark Mode, Wkr Install, Cluster Stats
- v0.3.6
- Adding Cluster Telemetry
- WIP: Worker Display
- Tweaks
- Start v0.3.6
- CP Updates
- Dash: Total Cluster Stats
- Publishing to cluster/workers
- Light/Dark Mode Toggle
- Added light/dark theme CSS
## v0.3.5 - 2026-02-01 (ALPHA)
- v0.3.5
- Fixing sdl-wkr install script
- Start v0.3.5
## v0.3.4 - 2026-01-31 (ALPHA)
- Working sdl-wkr self update
- Working sdl-wkr self update
- Start v0.3.4
## v0.3.3 - 2026-01-31 (ALPHA)
- WIP: sdl-wkr updates
- WIP: sdl-wkr updates
- Start v0.3.3
## v0.3.2 - 2026-01-31 (ALPHA)
- v0.3.2
- v0.3.2
- Squetching Logs
- WORKING: join/authz
- WIP: Join Worker
- WIP: Join Process
- sdl-wkr Cluster Join
- Fixing Service Module Render Bug
- Start v0.3.2
## v0.3.1 - 2026-01-31 (ALPHA)
**Summary**
README Cleanup
- v0.3.1
- Start v0.3.1
- Removing duplicate LICENSE file
- README Cleanup
## v0.3.0 - 2026-01-31 (ALPHA)
**Summary** **Summary**
Initial Codebase Initial Codebase
@ -121,6 +17,6 @@ Initial Codebase
Initial GitHub Release Initial GitHub Release
- Preparing GitHub Version - Preparing GitHub Version
- Start v0.3.0 - Start v0.1.0
- Initial commit - Initial commit

305
docs/DESIGN.md
View File

@ -1,4 +1,4 @@
# Software Defined Laboratory (SDL) # Node Web (NWA)
| Attribute | Value | | Attribute | Value |
| --- | --- | | --- | --- |
@ -10,305 +10,4 @@
## Overview ## Overview
The Software Defined Laboratory (SDL) project provides a generalized distributed computing platform for managing parallel computational workflows across a cluster of heterogeneous distributed nodes. SDL is a minimalistic High Performance Compute (HPC) platform focused on extreme minimizaton of technical debt associated configuration, deployability, and maintenance of the cluster. TBD
# Roadmap
## Prioritized Tasks
MVP - Minimum Viable Product Task List
**Core Infrastructure** ✅ (Complete)
- [x] sdl-mgr MQTT Broker
- [x] sdl-mgr UDP Beacon
- [x] sdl-mgr Web UI MQTT WebSocket Pub/Sub
- [x] sdl-wkr UDP Listener
- [x] SDL_ID Node ID
- [x] Version Updater Script
- [x] Deploy sdl-mgr to $SDL_HOME/sdl-mgr
- [x] Create SDL Worker Dist Build Process
- [x] Create SDL Worker Install to $SDL_HOME
- [x] SDL Install Script
- [x] Create SDL Worker Auto Update Process
- [x] Create Worker Join Process
- [x] Create Worker Telemetry Process
**Worker Monitoring & Telemetry** 🔄 (In Progress)
- [x] Worker Hardware Inventory Detection
- [x] Physical vs Logical CPU cores (hyperthreading)
- [x] Physical vs VM detection (hypervisor flag)
- [x] GPU VRAM capacity aggregation
- [x] Memory capacity (bytes → GB conversion)
- [x] Worker Real-time Load Status (pure Node.js)
- [x] CPU usage percentage (`os.cpus()` times)
- [x] RAM usage (total/free/used via `os.totalmem()/freemem()`)
- [x] GPU usage (exec-based: nvidia-smi, rocm-smi)
- [ ] Disk IO (optional: exec-based)
- [x] Worker Uptime Tracking
- [x] SDL process uptime (`process.uptime()`)
- [x] OS uptime (`os.uptime()`)
- [ ] Worker Display Improvements
- [x] Show OS distro + version in Platform column (e.g., "Debian 12 / x64")
- [x] Show system type (Physical 🖥️ / VM 💠)
- [ ] Show GPU VRAM in GPU column (e.g., "2 GPUs (48 GB)")
- [x] Show worker uptime in dashboard
- [x] Display last seen as elapsed time in dashboard, not date
- [ ] Track Username for sdl process
- [ ] Web UI
- [ ] Progresive Web App
- [ ] Editable Dashboard
- [x] Panel Icon
- [ ] Capacity Detection
- [ ] GPU VRAM capacity
- [ ] CPU capacity
- [ ] RAM capacity
- [ ] Network Negotiontion speed
- [ ] S3 (MinIO) storage capacity
- [ ] SW: Python, Rust, C++, etc
- [ ] PyTorch, Tensorflow, etc
- [ ] Ollamma, Models, etc
**Resource Management**
- [ ] CPU Reserve Capacity Allocation
- [ ] Config option: `reserve_cpus: N` (keep N cores for system)
- [ ] Track: `total`, `reserved`, `allocated`, `available`, `used`
- [ ] Display reserved capacity in dashboard
- [ ] Memory Reserve Capacity (optional, future)
- [ ] GPU Reserve Capacity (optional, future)
**Service Management**
- [x] Start/Stop shell scripts (for systems without systemd user services)
- [x] `start-sdl-mgr.sh` / `stop-sdl-mgr.sh`
- [x] `start-sdl-wkr.sh` / `stop-sdl-wkr.sh`
- [ ] Combined systemd service
- [ ] `sdl.service` to manage both sdl-mgr and sdl-wkr together
- [ ] Support for system-level (sudo) and user-level services
- [ ] SDL-MGR auto-installs SDL-WKR
- [ ] Manager node becomes worker by default
- [ ] Config flag to disable: `install_worker_on_manager: false
- [x] Pass HTTP Port to SDL-WKR Curl installer
- [ ] Update Install Summary
- [ ] Add Quite Mode to sdl-wkr install
**Data & Storage**
- [ ] Data Storage Organizational Structure
- [ ] Define directory hierarchy for tasks, results, logs
- [ ] Implement file naming conventions
- [ ] Add cleanup/retention policies
- [ ] MinIO S3 Storage Server Integration
- [ ] Install and configure MinIO
- [ ] S3 bucket structure for SDL data
- [ ] Worker access to S3 storage
- [ ] Web UI S3 browser integration
**Projects**
- [ ] Project Management Org Structure
- [ ] Define project metadata schema
- [ ] Project lifecycle states (draft, active, archived)
- [ ] Permission/access control model
- [ ] Experiment Organization under Org Structure
- [ ] Experiment templates
- [ ] Parameter tracking
- [ ] Result linking to experiments
- [ ] HPC Experiment Binary Development
- [ ] sdl-wkr Capablity detection (phyton, etc)
- [ ] Binary packaging format
- [ ] Dependency management
- [ ] Version control for binaries
**Dashboard Enhancements**
- [x] Dark/Light theme polish (CSS refinements)
- [ ] Real-time resource graphs (Chart.js)
- [ ] Worker status history timeline
- [x] Cluster health indicators
- [x] Alert/notification system for worker failures
- [ ] Add a Clock to the Dashboard (ISO 8601)
- [ ] WOrker Cards instead of table
- [ ] Search bar for workers, experiments, projects
**Testing & Documentation**
- [ ] Cross-platform testing (Debian, Ubuntu, Arch, Fedora, Win, OSx, BSD)
- [ ] Installation documentation
- [ ] Configuration examples
- [ ] Troubleshooting guide
- [ ] Validate IP change Stablity
---
# Design
## Design Goals
- Cross Platform (lnx, mac, win, bsd, rpi) Linux First, support for others later.
- Node.JS based Manager and Worker code bases
- Extreme Minimal OS Dependancies. sdl-mgr self hosts all the necessary code to deploy and coordinate a cluster. Including distrubution of Node binaries.
- Automate Update Workflows
- Zero Technical Debt (ideal)
- Sane handling of sdl-workers comming and going, being added or removed.
- Zero sdl-wkr Configuration (ideal) - Install > Run > Listen to UDP Bcast > Auto Join MQTT for cluster command and control.
### Security
- Security considerations TLS, Auth, etc. are future scopes of work. At this stage we are focused on getting the manager and workers up and running as a Minimum Viable Product (MVP) for a cluster of SDL workers. But we are also thinking forward about security and cross platform support so those features will be easy to implement later without a major code refactor.
- The current assumption is that SDL runs in a segmented subnet with restricted access. And that we do not have malicious actors trying to compromise the network or poison the open MQTT broker.
# Architecture
Software Defined Laboratry (SDL)
## Directory Structure
### SDL Source Code Structure
```
../sdl
├── docs
├── sdl-mgr
│   ├── app
│   │   └── modules
│   │   ├── mqtt
│   │   ├── nwa-lib
│   │   ├── sdl-mgr
│   │   ├── template
│   │   └── web
│   ├── conf
│   │   └── modules
│   ├── html
│   │   ├── conf
│   │   ├── css
│   │   ├── img
│   │   ├── js
│   │   └── md
│   └── scripts
├── sdl-wkr
│   ├── app
│   │   └── modules
│   │   ├── nwa-lib
│   │   ├── sdl-wkr
│   │   └── template
│   ├── conf
│   │   └── modules
│   └── scripts
└── tools
```
### SDL Installation Structure
```
~/.sdl/
├── conf
├── data
│   ├── jobs
│   └── proj
├── dist
├── logs
├── nodejs
│   └── current -> /home/jehaverlack/.sdl/nodejs/node-v22.22.0-linux-x64
├── sdl-mgr
│   ├── current -> /home/jehaverlack/.sdl/sdl-mgr/sdl-mgr_0.2.7
│   └── sdl-mgr_0.2.7
│   ├── app
│   │   └── modules
│   │   ├── mqtt
│   │   ├── nwa-lib
│   │   ├── sdl-mgr
│   │   ├── template
│   │   └── web
│   ├── conf
│   │   └── modules
│   ├── html
│   │   ├── conf
│   │   ├── css
│   │   ├── img
│   │   ├── js
│   │   └── md
│   └── scripts
└── sdl-wkr
```
### SDL Manager Install Script
```
$ git clone https://github.com/jehaverlack/sdl.git
$ cd sdl
$ ./install-sdl.sh
```
This will install both sdl-mgr and sdl-wkr on the Manager host.
### SDL Worker Install Script
Once the sdl-mgr is running, it broadcasts the install command over UDP. On your worker host, run the following command to listen for the install command on your worker.
```
nc -u -l -k 10101 | jq -r '.msg.sdl_wkr_install_cmd[]'
```
> **NOTE**: If you have a firewall, you must allow SDL Worker to listen on UDP port 10101.
The install command will look like the following, both curl and wget commands are supported.
```
curl -s http://10.0.0.144:8081/dist/install-sdl-wkr.sh | bash -s 10.0.0.144:8081
```
### SDL Worker Auto Update Process
Once a worker is joined to a manager's MQTT broker, it will automatically update to the latest version of SDL deployed by the manager.
### Worker Join
- UDP Bcast
- MQTT Join
### Worker Telemetry
Workers publish telemetry to the manager MQTT broker. The manager will collect this telemetry and publish to the web UI over MQTT.
### SDL Storage
- MinIO S3 Storage Server
- this is a non node.js codebase
```
Project
└── Experiment
├── L0 Raw Results
└── L1 Analysis Results
```
```
sdl/
└── projects/
└── <project_id>/
└── experiments/
└── <experiment_id>/
└── runs/
└── <run_id>/
├── l0/
└── l1/
```
# Projects
- Need to define structure to manage projects.
- Projects will be managed with JSON files
- We web UI management Edit interface is nice but will come later. Just need to get things working first.
## Experiments
- Within each Project Scope experiments can be conducted (run) as a specific configuration or permuation of variable to test.
- Each experiment represents a set of parrallized jobs for the cluster to execute on.
- L0 Raw Results will be collected and stored in a run S3 Bucket.
- L1 Analysis Results will be collected and stored in a run S3 Bucket
- L1 Analysis represents an other set of parrallized jobs for the cluster to execute on.
### Run Engine (RE)
- Run Engines (RE): May be a Python, Rust, C++, etc. executable/script that is forked by the sdl-wkr Node.JS process.
- RE get input configuration from the local sdl-wkr process.
- RE communicate only with the sdl-wkr process.
- But RE processes may read and write direclty to S3
- the sdl-wkr process communicates with the SDL Manager (sdl-mgr) process to notify on status of jobs.

119
docs/INSTALL.md
View File

@ -1,119 +1,2 @@
# Node Web App (NWA)
# Software Defined Laboratory (SDL)
> NOTE: This is a work in progress. SDL has only been tested on Debian based
> Linux systems. But has been designed to be cross platform. So running on
> other platforms should not be a heavy lift.
Other than basic OS tools, SDL downloads all dependencies idempotently and installs
everything into a single SDL home directory (~/.sdl).
---
## SDL Manager (sdl-mgr)
#### Installation
```
git clone <SDL_REPO_URL>
```
```
cd sdl
```
```
./install-sdl.sh
```
#### Upgrading
```
cd sdl
```
```
git pull origin main
```
```
./install-sdl.sh
```
---
## SDL Worker (sdl-wkr)
#### Prerequisites
(Debian, Ubuntu, Linux Mint, Zorin OS, Raspbian)
```
sudo apt -y install gawk coreutils curl jq grep sed tar unzip netcat-openbsd
```
#### Installation
(Linux, OSX, Windows(WSL2))
Listen to UDP Broadcast on port 10101
- To get the **sld-wkr** curl or wget install command
```
nc -u -l -k 10101
```
OR
```
nc -u -l -k 10101 |jq
{
"ts": "2026-01-31T05:20:31.815Z",
"sdl_id": "906a027c-52f0-4cbb-b3e0-5eb84856d50a",
"role": "sdl-mgr",
"host": "haverlab-sld-mgr",
"type": "udp-beacon",
"msg": {
"info": {
"desc": "Software Defined Laboratory: Manager (sdl-mgr)",
"version": "0.2.10",
"copyright": "2026 John Haverlack"
},
"cluster": {
"id": "haverlab",
"name": "Haverlab",
"desc": "John Haverlack's Home Lab"
},
"mqtt": {
"host": "10.0.0.178",
"port": 1883
},
"web": {
"proto": "http",
"host": "10.0.0.178",
"port": 8081,
"api_config": "/api/config",
"config_url": "http://10.0.0.178:8081/api/config"
},
"sdl_wkr_install_cmd": {
"curl": "curl -s http://10.0.0.178:8081/dist/install-sdl-wkr.sh | bash -s 10.0.0.178",
"wget": "wget -O - http://10.0.0.178:8081/dist/install-sdl-wkr.sh | bash -s 10.0.0.178"
}
}
}
```
```
curl -s http://<SDL_MGR_IP>:<WEB_PORT>/dist/install-sdl-wkr.sh | bash -s <SDL_MGR_IP>
```
#### Upgrading
SDL Workers continuously monitor the SDL Manager’s advertised version via MQTT cluster status messages and will automatically re-install and restart when a newer version is detected.
Alternatively you can upgrade manually by rerunning the install-sdl-wkr.sh script.
```
curl -s http://<SDL_MGR_IP>:<WEB_PORT>/dist/install-sdl-wkr.sh | bash -s <SDL_MGR_IP>
```

4
metadata.json
View File

@ -15,6 +15,7 @@
}, },
"META_MAPS": { "META_MAPS": {
"package.json": { "package.json": {
"description": "description",
"version": "version", "version": "version",
"version_date": "version_date", "version_date": "version_date",
"author": "author", "author": "author",
@ -35,7 +36,6 @@
"MANIFEST": [ "MANIFEST": [
"README.md", "README.md",
"docs/DESIGN.md", "docs/DESIGN.md",
"sdl-mgr/app/package.json", "nwa/app/package.json"
"sdl-wkr/app/package.json"
] ]
} }

6
nwa/app/package.json
View File

@ -1,8 +1,8 @@
{ {
"name": "nwa", "name": "nwa",
"version": "0.1.0", "version": "0.1.1",
"version_date": "2026-02-08", "version_date": "2026-07-04",
"description": "Node Web App (NWA)", "description": "Node Web App",
"main": "index.js", "main": "index.js",
"type": "module", "type": "module",
"scripts": { "scripts": {

31
tools/metadata-update.sh
View File

@ -1,13 +1,13 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# metadata-update.sh - Update SDL metadata in package.json and Markdown files # metadata-update.sh - Update NWA metadata in package.json and Markdown files
set -euo pipefail set -euo pipefail
# ----------------------------- # -----------------------------
# Resolve paths # Resolve paths
# ----------------------------- # -----------------------------
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SDL_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)" NWA_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
META_FILE="$SDL_ROOT/metadata.json" META_FILE="$NWA_ROOT/metadata.json"
command -v jq >/dev/null || { echo "jq required"; exit 1; } command -v jq >/dev/null || { echo "jq required"; exit 1; }
command -v sed >/dev/null || { echo "sed required"; exit 1; } command -v sed >/dev/null || { echo "sed required"; exit 1; }
@ -26,15 +26,34 @@ escape_regex() {
# ----------------------------- # -----------------------------
# Update package.json # Update package.json
# ----------------------------- # -----------------------------
# update_package_json() {
# local file="$1"
# local tmp
# tmp="$(mktemp)"
# jq \
# --slurpfile meta "$META_FILE" \
# '
# . as $pkg
# | ($meta.META_MAPS["package.json"] | to_entries) as $maps
# | reduce $maps[] as $m ($pkg;
# .[$m.key] = $meta.METADATA[$m.value]
# )
# ' "$file" > "$tmp"
# mv "$tmp" "$file"
# echo "Updated $file"
# }
update_package_json() { update_package_json() {
local file="$1" local file="$1"
local tmp local tmp
tmp="$(mktemp)" tmp="$(mktemp)"
jq \ jq \
--argfile meta "$META_FILE" \ --slurpfile meta_file "$META_FILE" \
' '
. as $pkg ($meta_file[0]) as $meta
| . as $pkg
| ($meta.META_MAPS["package.json"] | to_entries) as $maps | ($meta.META_MAPS["package.json"] | to_entries) as $maps
| reduce $maps[] as $m ($pkg; | reduce $maps[] as $m ($pkg;
.[$m.key] = $meta.METADATA[$m.value] .[$m.key] = $meta.METADATA[$m.value]
@ -68,7 +87,7 @@ update_markdown() {
# Main # Main
# ----------------------------- # -----------------------------
jq -r '.MANIFEST[]' "$META_FILE" | while read -r rel_path; do jq -r '.MANIFEST[]' "$META_FILE" | while read -r rel_path; do
file="$SDL_ROOT/$rel_path" file="$NWA_ROOT/$rel_path"
if [[ ! -f "$file" ]]; then if [[ ! -f "$file" ]]; then
echo "WARN: $rel_path not found, skipping" echo "WARN: $rel_path not found, skipping"