RoboXR VIROO — Interactivity Framework

Foundation Always-Active Layer (VIROO Actions — not tied to any VR Builder step)

These systems run independently of scenario process flow. They are persistent across all scenarios and driven by VIROO Actions rather than VR Builder step logic.

System	Mechanism	Package / Tag	Notes
AGVmotion (HFV routing)	AGVWaypointFollower.cs + UnityEvents	AGVmotion	HFVs loop between cells A/B/C, pause at pickup waypoints, fire OnArrived / OnDeparted events
Conveyor belts	Animated material scroll + trigger collider	Conveyor	Parts move along belt; fire events at infeed/outfeed endpoints
Emergency Stop	VIROO Action on E-stop button collider	VIROO	Broadcasts halt to robot + conveyor + AGV simultaneously
Safety zone floor	Nested colliders → material swap	VIROO	Green / Yellow / Red zones always visible; drives Scenario 3 proximity logic
Info tooltips	Gaze hover (1.5 s dwell) on any machine	VIROO	Works in all scenarios; no step dependency

AGVmotion Waypoint Path Structure

Main corridor (Z ≈ 8.5, X = -12 → -22) [MainCorridorWP_1] ────── [MainCorridorWP_2] ────── [MainCorridorWP_3] │ │ │ ▼ ▼ ▼ [CellA_DoorWP] [CellC_DoorWP] [CellB_DoorWP] X=-12.96, Z=7.67 X=-14.75, Z=12.34 X=-20.23, Z=7.67 │ │ │ ▼ ▼ ▼ [CellA_PickupWP] [CellC_PickupWP] [CellB_PickupWP] (near arm(2)) (near arm) (near arm(1)) X=-14.33, Z=5.97 X=-16.97, Z=12.91 X=-21.17, Z=5.46

Door waypoints sit at the fence gate gaps — not through fence panels. AGV slows, turns, and enters through the opening before proceeding to the pickup point beside the robot.

Scenario 1 AI-Assisted Industrial Layout Planning (AAXLP)

Scene starts completely empty. The learner builds the robot cell from scratch using voice commands and AI assistance.

Target audience: Industrial engineering students, production planners (EQF 4–8)
Packages: AAXLP · AGVmotion · VIROO

Step	Interaction	Mechanism	Package
1	Voice: "new layout" → select 4+ wall points → name the layout	WallPlacer + RoomGenerator	AAXLP
2	Voice: "spawn robot", "spawn conveyor", "spawn CNC machine" → items appear at gaze point	ItemHandler + Whisper STT + CommandWheel	AAXLP
3	Gaze + hand to move/rotate items; boundary check: red = invalid, green = valid	ItemHandler + BoundaryChecker	AAXLP
4	Voice: "add connection" between two machines → flow path drawn	PathVisualizer (Manage Connections)	AAXLP
5	Voice: "suggest layout" → AI proposes optimal machine/robot positions (ChatGPT)	ChatGPTController → onResponseFinishedProcessing	AAXLP
6	Voice: "show AGV paths" → PathVisualizer draws AGV routes over the layout	PathVisualizer	AAXLP
7	Live AGV test-run: HFV follows proposed route in real-time over the new layout	AGVWaypointFollower.cs triggered by onLayoutSaved	AGVmotion
8	Results panel: cycle time estimate, space utilization %, collision risk score	Custom UI hooked to AAXLP onResponseFinishedProcessing + onItemSpawn	AAXLP VR Builder

Key AAXLP events to hook: onItemSpawn, onConnected, onLayoutSaved → trigger AGVmotion preview path.

Scenario 2 Robotic Machine Tending (REACH + AGVmotion + Conveyor)

Scene starts with full cell: robot idle, CNC machine closed, conveyor empty. AGV approaches with raw parts.

Target audience: Robot operators, CNC technicians, mechatronics trainees (EQF 4–6)
Packages: REACH · AGVmotion · VR Builder · VORAUS (optional)

Step	Interaction	Mechanism	Package
1	AGV arrives at CellX_PickupWP with raw part on pallet	AGVWaypointFollower → fires OnArrived UnityEvent	AGVmotion
2	Panel: choose Local Control (VIROO) or VORAUS Core Control	VR Builder Push Button condition → branches process	VR Builder
3	Pull power lever → robot + conveyor + machine activate	VR Builder Pull Lever condition	VR Builder
4	User grabs raw part from AGV pallet	VirooXRSimpleInteractable grab (Allow Direct Interaction)	VIROO
5	User places part on infeed conveyor snap zone	VR Builder Snap Zone condition	VR Builder
6	Conveyor moves part automatically to machine loading position	Conveyor animation triggered by snap zone filled event	Conveyor
7	User opens CNC machine door + places part in fixture	VR Builder Grip Button condition	VR Builder
8	CNC door must close before cycle start is enabled (interlock)	VR Builder condition chain — door state gates next step	VR Builder
9	User presses cycle start button	VR Builder Push Button condition	VR Builder
10	Robot arm performs tending cycle — gaze select target, ghost preview, confirm	REACH MoveTarget behavior + gaze 1.5s dwell + pinch confirm	REACH
11	Voice: "set target position" → robot physically moves to committed pose	REACH Detect Command condition + MoveTarget	REACH
12	CNC machine runs: spindle animation, door indicator lights	Execute VIROO Actions behavior	VIROO
13	Finished part placed on outfeed conveyor → conveyor moves it to AGV pickup	Conveyor endpoint event → AGV OnDeparted	Conveyor AGVmotion
14	AGV departs with finished part, travels to next cell	AGVWaypointFollower resumes route	AGVmotion
15	Emergency stop at any point → everything halts simultaneously	VIROO Action broadcast: robot + conveyor + AGV	VIROO
16	Summary panel: steps completed, safety violations flagged	VR Builder End Chapter → results UI	VR Builder

VORAUS mode difference: Steps 10–12 send real WebSocket commands to VORAUS Core (127.0.0.1:8765) instead of local VIROO animations. Robot joint angles and process states are bidirectional.

Scenario 3 Safety & Human–Robot Interaction (VIROO + REACH Safety)

Scene starts with robot running a normal cycle. AGV is also moving through the corridor. Safety zones are visible on the floor.

Target audience: Safety officers, supervisors, mechatronics students (EQF 4–7)
Packages: VIROO · REACH · VR Builder

Step	Interaction	Mechanism	Package
1	Robot runs cycle, AGV loops, safety zones (green/yellow/red) visible on floor	Always-active VIROO Actions + AGVWaypointFollower	VIROO AGVmotion
2	User walks into Yellow zone → robot slows 50%, amber light, soft beep	Collider Hover Entered → Execute VIROO Action (speed modifier + audio)	VIROO
3	User walks into Red zone → robot stops, red indicator light, warning sound	Nested collider → Execute VIROO Action (stop robot)	VIROO
4	User exits Red zone → robot resumes slow speed in Yellow zone	Collider exit event → restore speed modifier	VIROO
5	AGV approaches robot pickup point — learner observes safe docking sequence	AGVmotion timed approach — visual/observational in this scenario	AGVmotion
6	User presses E-stop → robot + conveyor + AGV all halt immediately	VIROO Action broadcast to all objects simultaneously	VIROO
7	Reset protocol: user enters cell, clears parts, voice: "enable gaze control"	REACH Detect Command condition	REACH
8	User moves robot to safe start point via gaze dwell + left pinch (Z) + right pinch near head (commit)	REACH gaze interaction: 1.5s dwell → ghost preview → commit	REACH
9	User presses reset button → system fully restarts	VR Builder Push Button condition → End Chapter / loop back	VR Builder

Interactivity by Input Type — All Scenarios

Gaze Dwell (1.5 s)

All scenarios

Info tooltips on any machine. Robot target selection in REACH. Activates VirooXRSimpleInteractable Hover events.

Hand Grab

Scenario 2

Pick raw/finished parts from AGV pallet. Place in CNC machine fixture. VirooXRSimpleInteractable (Allow Direct Interaction).

Left Pinch

Scenario 2, 3

Moves REACH robot ghost target along Z-axis (depth). Used in combination with gaze dwell.

Right Pinch Near Head

Scenario 2, 3

Safety commit gesture — confirms robot moves to ghost position. Must be performed near head to prevent accidental trigger.

Voice Commands

All scenarios

AAXLP layout commands (spawn, move, suggest). REACH robot commands (14 directions + set target). Powered by Whisper STT + ChatGPT.

Push Button

Scenario 2, 3

Cycle start, E-stop, reset, mode select (Local/VORAUS). VR Builder Push Button condition.

Pull Lever

Scenario 2

Power on sequence — activates robot + conveyor + machine. VR Builder Pull Lever condition.

Proximity Walk-In

Scenario 3

Trigger safety zone state changes (green → yellow → red). Collider-based, drives robot speed modifier.

Snap Zone

Scenario 2

Part placement on conveyor, machine fixture, AGV pallet. VR Builder Snap Zone condition gates next process step.

Slider / Knob

Scenario 2 (optional)

Machine parameter controls (speed, feed rate). VR Builder Set Slider / Turn Knob conditions.

Recommended Build Order

AGVmotion — Write AGVWaypointFollower.cs, place waypoints in scene, validate Cell A → Cell B loop first.
Conveyor system — Animated material scroll + trigger collider + UnityEvent infeed/outfeed endpoints.
Scenario 2 VR Builder process — Most complete flow; tests REACH + AGV + conveyor + CNC interlock together.
Scenario 3 proximity/zone system — Builds on Scenario 2 robot setup; add collider zones + speed modifiers.
Scenario 1 AAXLP integration — Standalone; hook AAXLP Unity Events to AGVmotion preview path.

Package & Script Quick Reference

Package	Key Scripts / Components	Main Role
AAXLP	ItemHandler, WallPlacer, RoomGenerator, BoundaryChecker, PathVisualizer, ChatGPTController, RunWhisper	Scenario 1 — voice-driven layout planning + AI suggestion
REACH	REACHInteractionManager, VoiceProcessor, MoveTarget (VR Builder behavior), CustomIkSolver.exe	Scenarios 2 & 3 — robot arm control via gaze + voice + ghost preview
AGVmotion	AGVWaypointFollower.cs (custom), waypoint empties in scene	All scenarios — HFV routing between cells, pickup/dropoff events
VR Builder	Push Button, Pull Lever, Grip Button, Snap Zone, Detect Command, Set Slider, Turn Knob, MoveTarget, Execute VIROO Actions, End Chapter	All scenarios — step sequencing, conditions, process logic
VIROO	VirooXRSimpleInteractable, XRGazeInteractor, VIROO Actions, Network Event Propagation, Permission Handler	All scenarios — grab, gaze, network sync, safety zones, always-active actions
VORAUS	VorausCoreWebsocket.cs, WebSocket 127.0.0.1:8765	Scenario 2 (optional) — live robot control via real industrial system