A real-time deep-learning platform that learns normal pedestrian motion from skeleton trajectories and detects risky behaviors at railroad grade crossings.
ST-GCN
Pose Estimation
Anomaly Detection
Edge Deployment
STGCN-PADSpatial-temporal motion intelligence
Safety problem
Recognize risky motion, not only pedestrian presence.
Lingering, squatting, and other unexpected behaviors near tracks require a system that understands motion patterns and can localize the person at risk.
Learning strategy
Train only on normal walking trajectories.
OpenPose converts pedestrian behavior into skeleton-coordinate sequences. STGCN-PAD learns their normal spatiotemporal structure and uses reconstruction error to detect unfamiliar motion.
Field demonstration / 01
From live video to localized alert
Camera input, skeleton extraction, motion assessment, and visual/audio alerts in railroad grade-crossing environments.
Performance / 02
Accuracy-speed tradeoff for field deployment
The MLP-Mixer reconstruction option substantially reduces inference time while preserving comparable anomaly-detection performance.
88.2%
Frame-level AUC CNN reconstruction
3.63 ms
Trajectory inference MLP-Mixer
10×
Faster inference than prior work
~20 FPS
Online field-test processing rate
Pipeline / 03
STGCN-PAD workflow
01
Pose extraction
Represent each pedestrian as a sequence of skeleton keypoints rather than raw appearance.
02
Spatiotemporal learning
Model joint relationships within each frame and across consecutive frames using ST-GCN layers.
03
Motion reconstruction
Reconstruct normal trajectories through a CNN or faster MLP-Mixer network.
04
Risk alert
Detect unfamiliar motion from reconstruction error and localize the corresponding pedestrian.
Research role / 04
My contribution
Methodology, formal analysis, and original manuscript; co-contributor to the study concept and investigation.
Three journal articles document the progression from skeleton-trajectory detection and localization to semi-supervised learning and spatial-temporal graph modeling.
