Hong Kong · +86 181 1551 0311 · y-yuxuan yang@po yu edu hk
Po t o o ssuu com/yuxuanyang · L nked n n yuxuanyang0311 · G Hub YYX
Lead nte act on p oduc de ign o ndust y‒academ a o n -venture pro ec s Acce Group col abo at on b dg ng esea ch methods and ma ke - ac ng del ve y. ‒ndependent y de gned he end-to-end UX o a sma t e de ly-care moni or ng ecosys em a fam y mob le app w h a five- er a a m h erarchy a 72-sc een a -de ect on compan on app and a careg ve web con o e
Mentor eme g ng des gn a en s across the end-to-end d gi a -product l ecyc e w h emphas s on human‒computer nte act on and user-cen red des gn methodo ogies
PROJECTS
Jun or Des gner — Shangha Huitu Heyue Arch tectural Design Co , Ltd Dec 2022 ‒ Aug 2024 ar - me Remo e Video Soc a P at orm Fu -Link Restructuring (Goog e UX p act cal s udy
Large-scale s e mode ng and spat a pa ametr c tud es n Rh no / SketchUp / Au oCAD suppo ted 10+ andscape and bu -env ronment p o ect wi h h gh-prec s on 3D mode s ‒A -ass sted decons ruct on o deep nav ga on og c produced a 20+ page compe i ve / naviga ion ana ys s a h ghfide y p o otype and a de a ed n erac on-spec fica ion documen
Three pro ects one research tra ectory rom the c in cal ab to pub ic-safety dep oyment
RESEARCH FOCUS
V r ua rea y and human‒compu e nte ac ion or cogn ve ehab l a on and vu ne ab e users remo -tolerant n eract
Conduc ed age- r end y acce s b ty esea ch a so des gned the U and n erac on og c o a wa er-sa e y mon o ng system produc ng suppor ing ha dware documentat on and vi ua mate a s
‒Des gned and bu t a work ng VR p o otype rans at ng es ab shed heory cogn t ve oad wo k ng memory mo o mpa rment nto three nterac on-des gn deci ions o cogn t ve y- and mo o - mpa red u e s
‒Eng nee ed an N=3 t me-th esho d input fi e that e ec s t emor no se wh le prese v ng de be a e mot on and an N=5 wo k ng-memory-capac ty fin te state mach ne ha scaffo ds task-swi ching
‒Bu a oca , m i econd-p ec ion behav ou a - ogg ng p pe ne CSV a a founda on o d g a phenotyp ng ve fied el ab l y and sens v ty o prac ce effects n deve oper se -tes ng ‒Code g hub com YYX311 ‒
Co e des gne s te survey concept deve opment and 3D pa ia -nar at ve mode ng Awa ded NCDA Hube Regiona
VR AR & p ogramm ng Un y (VR/AR C# Un ty sc p ng) G t G tHub HTML5 / CSS3 / JavaScr p
Des gn & 3D F gma (expe t Axu e, Rhino (advanced Ske chUp Blende Adobe CC Au oCAD
Research & A workflows A -ass s ed proto yp ng & deve opment behavioura da a ogg ng Gem n / Cha GPT, S tch Pome l
AWARDS & CERT F CAT ONS
Thi d P i e ‒13th NCDA Nat ona Co ege D g a Ar & De ign Compet ion Prov
LANGUAGES
Ch nese nat ve Eng sh p o ess ona wo k ng profic ency wo-yea MSc augh and assessed ent ely n Eng sh, Un vers y Co ege Co k e and
Designing for the Most Vulnerable
Theoretical Framework: From Established Theory to Design Decisions
Three established theoretical lenses (Cognitive Load, Working Memory, Motor Impairment) directly translate into three design decisions.
COLUMN
1: THEORETICAL ANCHOR
Cognitive Load Theory (Sweller, 1988)
COLUMN 2: USER CONSTRAINTS
Tremor and Working Memory
School of Computer Science &Information Technology University College Cork, Ireland
Research Assistant, Metaverse+ Lab
The Hong Kong Polytechnic University (since Oct 2025)
Thesis Achievement 72/100 Distinction band (Irish Grading)
Forensic psychiatric patients face a steep cognitive decline when transitioning from secure wards to community life. Existing VR systems are either entertainment-focused (lacking clinical safety) or clinically rigid (lacking user-centered design). This research asks: how can we design a VR system that is simultaneously tremortolerant, cognitively-scaffolded, and data-transparent for clinical use?
THE RESEARCH
GAP
This portfolio presents a Master’s-level pilot prototype built in Unity 2021 for the Meta Quest 3, alongside developer self-testing data (n=1, 3 sessions).
The clinical validation of these design choices with real forensic psychiatric populations is the central proposal of the four-year PhD research described on Page 6.
COLUMN 3: DERIVATION
Three Design Decisions Emerge
Two Contributions
Two engineered, implemented, and self-verified design contributions: temporal filtering for tremor + state machine for cognitive scaffolding.
System Architecture: From Sensor Input to Behavioral Logs
Engineering rigor：full Unity stack with data integrity, performance optimization, and forensic-aware safety design.
CONTRIBUTION 1
N=3 Time-Threshold Filtering Algorithm
Represents standard collision physics with intentionality confirmation. Implemented in TrashCan.cs (Thesis Appendix A.4), the algorithm discards high-frequency tremor noise while preserving deliberate sweeping motion.
Externalizes prospective memory into the system architecture. Implemented in TrashBinCapacity.cs (Thesis Appendix A.3), the FSM enforces a Sweep → Check Bin → Empty Bin loop, training task-switching within the user’s working memory limit.
DATA INTEGRITY (SOFTWARE ENGINEERING) ABOUT THE ARCHITECTURE
This architecture enforces a strict data flow from sensor input to offline analysis. No network communication or cloud services are used during operation. All logs are locally stored as CSV files and retrieved offline for clinical analysis.
The system reliably captures sub-second behavioral micro-events and detects practice effects, establishing readiness for clinical validation.
1) 6-STEP USER SCENARIO FLOW (HONESTLY TESTED BY THE DEVELOPER)
2) PRACTICE EFFECT IN IRI (REAL DATA FROM 3 SESSIONS)
3) THREE KEY METRICS (AGGREGATED ACROSS 3 SESSIONS)
C (20251227_145416)
TESTING SETUP — IMPORTANT CONTEXT
data logging integrity at millisecond granularity, and (c) the sensitivity of behavioral metrics to learning effects.
4) WHAT THIS DATA DEMONSTRATES 5) WHAT THIS DATA DOES NOT SHOW
The N=3 filter does not block intentional inputs (19 successful collections out of 19 attempts) Generalization to forensic psychiatric populations (no impaired subjects tested)
The N=5 FSM enforces task switching without errors (no HitWhileFull events across 3 sessions)
The CSV logger captures sub-second events reliably (millisecond-precision timestamps verified)
The system is sensitive to practice effects (60.6%
Detection of cognitive deficits (no comparison with clinical baselines)
Clinical efficacy of the intervention (no controlled trial conducted)
Research (2027–2031)
Impact (2031+)
From Pilot to Program: A Four-Year Research Agenda No.2 From Ward to Living Room
The Master's pilot established technical feasibility. This portfolio's final page proposes how that foundation could scale into a publishable, clinically-relevant PhD research program.
ML-driven adaptation
• Multi-modal sensing
The N=3/N=5 design choices were derived from theory but tested only on the developer. The first task of a PhD is to validate these choices with the target population.
RESEARCH QUESTIONS
RQ1.1: Do the N=3 filter parameters generalize to more-impairment-free psychiatric users?
RQ1.2: Do PMFR (Prospective Memory Failure Rate) and CFI (Cognitive Fatigue Index) correlate with established clinical instruments (MoCA, MMSE)?
METHODOLOGY (PROPOSED)
Cross-sectional study, n=20–30 patients Within-subject design across 4 weeks of training
5.
IEEE EMBc (engineering medicine) or Chi Health A validated digital phenotyping framework for VR-based cognitive assessment in secure psychiatric settings If the proposed PhD program succeeds, the resulting framework contributes to three important fields:
Forensic Psychiatry — first VR cognitive rehabilitation framework validated for this population
HCI for Vulnerable Users — design principles for tremorand cognition-impaired interaction
Digital Phenotyping — open-source metrics (PMFR, CFI) for behavioral assessment in immersive environments
closed-loop dynamically adjusted (DDA) framework for VR rehabilitation.
The current N=3 and N=5 are static. Real rehabilitation requires per-patient personalization to the "flow channel" (Ksksentimihaly 1990).
RESEARCH QUESTIONS
RQ2.1: Can a reinforcement learning agent learn personalized difficulty curves from behavioral data?
RQ2.2: Does adaptive difficulty improve engagement and outcomes compared to fixed parameters?
The Master's pilot is the foundation. The PhD is the path. The vision is a future where neuro-rehabilitation is personalized, evidence-based, and accessible to those who need it most.
Hand interaction logs alone underutilize the Quest Pro's sensor suite. Eyetracking can reveal attentional patterns invisible to interaction logs (Hutton et al. 2008).
RESEARCH QUESTIONS
RQ3.1: Can eye-tracking + interaction logs estimate cognitive load in real-time?
RQ3.2: Do gaze patterns predict task completion success?
METHODOLOGY (PROPOSED)
Hardware upgrade to Meta Quest Pro Multi-modal data fusion pipeline Real-time cognitive load estimation
Designing a multi-stakeholder health-monitoring ecosystem for ageing in place
Role Lead Designer & System
PolyU-Accel Group Metaverse+ Joint Laboratory
Carewell Digital Health Deployed 2023—
System architecture Interface prototypes · Service blueprint Field pilot Evaluation report
How might we connect hospital-grade monitoring with everyday life at home— across fragmented data, diverse user needs, and privacy-sensitive contexts—to support proactive care and ageing in place?
Three Users, One Data Stream
The same evidence-to-decision structure as Project 01: documented user constraints on the left translate, column by column, into three named design principles on the right.
The Family App: Designing Calm
Principles P1–P3 in shipped form. Three decisions, each traceable from screen component to intended behaviour
COLUMN 1: USER CONSTRAINTS
ageing does to interaction
— Role-based information architecture.
One data stream, three interface densities; complexity is a property of the role, not the system.
“Should call?” (threshold + context)
alert numbness
“Rank this vs 30 others” (comparative triage)
“Is the sensor OK?” (device state)
Missedcritical events Silentdata-quality decay
In eldercare, most alerts are
P2 — Glanceable safety state.
Every screen shows a verdict before any number.
P1 Safety First Surface risks early, act immediately
P2 Calm by Design Reduce noise, increase clarity
P3 Share, Don’t Alarm Inform without interrupting
Verdict-first cards surface what matters most. Permetric normal ranges are visual, not textual. Events are grouped by type and time. Resolution chips show closure at a glance. Tiered modal communicates severity. Acknowledgement builds calm and trust. Three primary actions. Big targets. Usage high, errors low Health overview At-a-glance, risk-first
D2 IN FULL — THE FIVE-TIER ALARM HIERARCHY (P3)
Tier Tier Trigger (examples)
User-triggered SOS Detected fall
absence from mattress
P3 —
Five severity tiers with distinct channels, colours, and escalation paths.
+ push, one-tap acknowledge
notification, log entry
entry, badge, auto-clears on re-wear
status chip only
One App, Designed Twice
My person-centred proposal (V1) and the device-centred release that shipped (V2) — the most instructive gap in this portfolio, kept visible on purpose.
V1 · PERSON-CENTRED PROPOSAL — 72-SCREEN DESIGN (IN ITERATION)
ANATOMY OF THE PROPOSED CRITICAL ALERT
Plain-language event statement
“Fall detected — are you okay?” The elder is addressed directly, no severity codes.
Entry = identity. Elder home = a verdict. V1’s T1 alert — the interaction V2 deferred. “I’M SAFE” as the largest target Manual SOS bypass Visible 15-second countdown
who you are — Elder, Family, Caregiver — three densities of one system.
“All Clear” before any number; nothing below the fold is needed to feel safe.
The escalation clock is shown, not hidden; inaction has a stated consequence.
— the false-positive path gets the biggest button, because every false escalation taxes the credibility of true ones.
— for the conscious caller who needs help now.
Immediate, affirmative response for safety.
V2 · DEVICE-CENTRED RELEASE — ITERATED WITH AGT ENGINEERING FOR RAPID DEPLOYMENT
The Caregiver Console: Same Data, Different Density
The web console inverts every mobile decision on purpose: where the app hides numbers behind verdicts, the console exposes them — because the caregiver’s job is comparison, not reassurance
2,646 historical alarm records in production at time of writing; unresolved items surface first, with a daily to-handle counter — the alarm hierarchy of page 13, operationalised at facility scale.
HR/BP ranges, sampling and positioning frequency are configured per person and synced to the device — the seed of the individualised-alerting research question below.
GPS + Wi-Fi + cellular fused location per wristband on a facility map; geofences raise a T3 event when a resident with dementia leaves a safe zone.
An LLM assistant answers caregivers’ questions over residents’ longitudinal data and drafts care suggestions from trends.
“Who are you?” — safety verdict
Full-screen takeover; elder can cancel or escalate
First-class user (simplified mode)
Derived from user constraints (RQ2)
The device fleet Facility contracts run on device fleets and short procurement cycles
Connected-device list + battery %
Alert record with status chips, for family & staff
Inherited from the AGT family app’s four tabs Monitored subject; family operates
One IA shared with AGT app cuts training and support cost; reusing its device / alert / control modules saved months.
The customer — the facility — manages devices, not roles. A rational optimisation for a different objective.
We documented the trade-off rather than hiding it.
: role-based entry elder-facing simplified mode · the countdown-cancel interaction. The design argument: in a safetycritical alert the cancel affordance is itself a safety feature — and the person on the floor, not the person watching, should be the first user the alert addresses.
Interrupt sparingly
Now today
Anxiety, numbness
: plain-language alert copy · SOS quick-contact chain exceptions events / logs separation (a trace of the T1‒T5 ladder).
One household Alarms Auto-resolved To-handle Median handle time Verdict
Sortable raw values
Queue everything, rank hard
Trends, weeks, cohort reports
The person, by role Whole facility (30‒300 residents)
Missed triage, liability
interfaces designed — family app falldetection companion (V1 proposal, 72 screens) — V2 shipped release caregiver console (18 modules)
Alarm Outcomes (at time of writing)
The caregiver’s job is comparison, not reassurance. The console removes ambiguity by exposing volume, context, and trade-offs.
Information architecture of the 18-module console (home → anomaly → devices → parameters → mattress → fall radar → geofence → reports → roles)
Interaction logic for triage states (pending → acknowledged → resolved) and per-resident parameter sync IA & UX
The 20-page V.3 operator manual — every workflow description on this page is drawn from documentation wrote
Frontend implementation, backend, AI models — engineering team (Accel Group)
Alarm mix by tier (design intent)
device classes integrated — 2 wristbands smart mattress · UWB positioning fall radar PTZ camera alarm records visible in the production triage queue at documentation time— the system runs at real scale documentation shipped and iterating — app manual V.2 (2026-02-28), console manual V.3 (2026-03-03), authored by me
V1 treats the elder as the first user in a safety-critical moment; V2 treats devices as the first-class objects in an operations system. Both are legitimate within their goals; the open research question (RQ2) is how to reconcile these models without sacrificing either safety or scale.
Exactly as in Project 01, this list is not an apology — it is the research agenda. Each “X ”above converts into research question on the right.
A deployed multi-device eldercare ecosystem four stakeholders thousands of real alarm events the design principles (P1‒P3) that keep it humane — and the open questions it cannot answer without research
Validated digital phenotyping and individualized, fatigue-aware alerting for vulnerable users — lab-grade behavioural metrics (Project 01) evaluated inside a real care ecosystem (Project 02), with elderly participants, under the industry‒academia infrastructure of the Metaverse+ Lab where both this project and already live.
PM & Logic
No.3
When Seconds Count
Normal swimming
No action required. Green dot on the plan; state card reads normal.
Suspected drowning Amber dot + alarm card. The operator must verify at the lane immediately
Drowning Red dot + alarm card + sound. Dispatch the lifeguard; rescue before anything else.
Redundant encoding. Every state is expressed three ways at once — dot color on the plan, a plainlanguage text card, and an audible alarm — so a glance, a read, or a sound each suffice under stress.
A mixed-sensing population as design constraint. Wristband users are tracked precisely; walk-in swimmers without a band are seen only by video AI. The interface keeps both groups — and the gap between their counts — visible at all times.
Three Design Decisions
The home view boots masked: 「視訊畫⾯已隱藏（隱
私防窺）」 Live video opens only when the operator deliberately enters 「緊急模式」 — which first replays the last five minutes, then cuts to live — and must be exited after use, restoring the mask. Watching swimmers becomes an auditable act tied to an incident, not a default condition of the room.
The 「傳感器」 view concentrates everything a duty decision needs into a single screen: operating mode, three headcounts, per-swimmer IMU data, and a live UWB pool plan with color-coded dots ( ). The operator never navigates away to assess danger — and every alarm points back to a lane, not a log entry
Three headcounts sit side by side and must reconcile — a mismatch is itself a signal that someone in the water is untracked:
WHY THIS IS A MEDICAL-TECHNOLOGY HCI PROBLEM
Physiological sensing (heart rate, motion), automated risk classification, and a human who must trust an alarm enough to act on it in seconds — the same triad that defines clinical monitoring, here in a public-safety setting.
Deliberately boring by default — alarm salience is budgeted, not maximized
A screen watched for hours must make normality unremarkable — flat greens, no motion, no numbers demanding attention — so that the one moment of color change is impossible to miss.
Alarm salience is budgeted, not maximized.
Data custody follows the same logic. Recordings and personal swim reports export only after a locally-stored download password; destructive actions — clearing a swimmer’s history — are irreversible and explicitly warned before they run.
「視頻緩存」 recordings cached in 15-minute segments named by date / camera / start time; downloading any segment or personal report requires the operator’s password.
SENSING LAYER
From Alarm to Action
Wristband user, confirmed
Four alert scenarios, one operator playbook — the interface and the manual designed as a single safety system.
STATE LANGUAGE (THE THREE-STATE MODEL)
Normal swimming
No action required. Green dot on the plan; state card reads normal.
Suspected drowning
Amber dot + alarm card. The operator must verify at the lane immediately
Drowning
Red dot + alarm card + sound. Dispatch the lifeguard; rescue before anything else.
Amber card the swimmer’s dot turns amber on the plan. Response: read the lane off the plan, verify poolside at once; dismiss only after confirming safety
Red card + sound: the dot turns red. Response: dispatch the lifeguard to that lane immediately; the alarm is dismissed only after the scene is safe.
— rescue first, record later.
WHY THIS IS A MEDICAL-TECHNOLOGY
HCI PROBLEM
Physiological sensing (heart rate, motion), automated risk classification, and a human who must trust an alarm enough to act on it in seconds — the same triad that defines clinical monitoring, here in a public-safety setting.
Detected by video AI — no dot exists for this swimmer, so the plan may stay green. Response: open on the matching camera to locate the person visually, then dispatch.
DESIGN QUESTION
Duty staff must act within seconds of a drowning event — yet swimmers deserve privacy from constant camera surveillance, and the people on shift are property staff, not engineers. How can one console deliver unambiguous danger signals, keep video hidden by default, and make every data export accountable?
One card names both victim types at once. Response: full mobilization — the plan locates the wristband user, emergency video locates the other; all staff deployed in parallel.
OPENING RULE OF THE MANUAL’S EMERGENCY CHAPTER — THE INTERFACE NEVER PUTS DATA ENTRY BEFORE A RESCUE
Two Layers of Evidence
From bench simulation to the pool deck — what was tested, where, and what each layer can honestly claim
Dismissing an alarm closes the prompt — it never asserts the danger is over. The manual makes this distinction explicit to prevent premature dismissal.
Outside opening hours (default 19:00‒09:00) the pool should be empty, so the system inverts: any presence is the alarm. A center-screen intrusion popup fires with a timestamp; the operator verifies by camera, then chooses:
— dismiss this popup once, or「5
— snooze for 5 minutes only: a deliberate friction so a live intrusion can Wnever be silenced permanently
S3 is the matrix’s hardest cell: the most vulnerable swimmer is the one the plan view cannot draw. The playbook routes the operator to video precisely — and only — when the sensor map goes blind. The intrusion alert interrupts at screen center — unlike drowning cards, which dock at the side to keep the plan visible. Interruption strength encodes threat model.
KEY TAKEAWAY
The matrix ensures every case has a place to look and a line to follow. The workflow enforces one opening rule. The afterhours model protects against the second threat.
Together they form a single coherent safety system: alarm → locate → verify → rescue → resolve.
An interruption hierarchy, by design Drowning cards dock at the edge because the operator still needs the plan to act; intrusion popups block the center because, in an empty pool, there is nothing else the screen should be doing. Where an alert appears is itself information.
Honest about system limits Switching between working and nonworking modes takes ~1 minute to reconnect positioning. The UI and the manual both say so — rather than pretending the system is instant — and instruct staff not to toggle repeatedly while it reconnects.
Wristband worn at the bench; the green dot tracks on the plan in real time.
A wristband-wearing tester drives the full alarm loop — normal → alarm → verify → resolve — against the live
Field session at the pool: hardware and wristbands deployed in the real environment the system is designed for.
Testing with the wristband in actual pool water — the acoustic, optical and RF conditions no bench can reproduce. The team observes system behaviour poolside, checking that lane positions and states match what is happening in the water.
Field footage from an on-site test session. This tier moves the evidence from the lab to the deployment environment: real water, real distances, real lighting. It demonstrates environmental viability — structured detection-accuracy trials remain future work (Page 20).
Full alarm loop — normal → alarm → verify → resolve
Real water, RF attenuation, glare, ambient noise
Detection-accuracy statistics (fit miss / false-alarm rates)
Why two tiers matter Bench tests isolate the interaction loop; on-site tests expose everything the bench hides — water attenuating radio signals, glare on camera lenses, ambient noise competing with alarms.
Claiming only what each tier can support is the same evidentiary discipline as the VR pilot on Pages 8‒9.
A safety system’s real interface is whatever the operator reaches for at 2 a.m. — often, that is the manual.
not engineers, and structured by situation rather than by feature.
Daily duty — the three states & colors,privacy mode, the sensor view, headcount reconciliation
Emergency playbook — the fourdrowning scenarios + intrusion response; flaggedfrom the cover; in an emergency, turn here first
Evidence & upkeep — clipdownloadcustody,personal swim reports, a troubleshooting table mapping 8 symptoms to causes, support contacts
Quick reference — plain-language glossary + a one-page color/state lookupcardfor the dutydesk
Safety-critical lines carry inlineduty tags
— andevery UI label isquotedverbatim in 「 」 so screen andpage can be matched word-for-word under stress. Known constraints (the ~1-minute mode-switch reconnection, the unrecoverable downloadpassword) aredocumentedas constraints, not hidden.
Design position. In safety-critical HCI, the artefact boundary is not the screen. Alarm semantics, dismissal rules, andprivacy obligations weredesignedonce andexpressedtwice — in the interface and in the manual — so neither can contradict the other.
