SayLens is both a useful camera-based learning product and a realistic mobile performance laboratory. Its workload combines a live camera, on-device machine learning, native memory ownership, concurrent Kotlin workers, React rendering, and UI-thread animation.
The public story should not be “React Native reached 30 FPS.” It should show how the system was measured, where each frame spends time, which resource trade-offs were accepted, and how the implementation changes when the evidence changes.
This project can demonstrate:
The repository history already forms a useful engineering narrative:
The current debug snapshot on a Samsung SM-M536B improved the camera preview from about 10 FPS to about 30 FPS and aggregate detector throughput from about 5 FPS to about 30 FPS. The same configuration used about 324% to 339% process CPU and 529 to 533 MB resident memory. These numbers prove throughput on one device; they do not yet prove efficiency, release performance, or sustained thermal behavior.
The application exposes exactly two profiles instead of a device-class taxonomy:
Maximum performance deliberately spends CPU, memory, and battery. The only budget it respects is the one that makes the overlay feel attached to the camera image: worker counts that raise throughput but also raise result age are rejected, because a fresher detection matters more than a larger FPS number.
The J6 has eight logical cores, 1.8 GB of memory, a Mali-T830, and runs the application as a 32-bit process. Release-build measurements at a 720x480 detector buffer with the camera preview negotiated at 30 FPS:
| CPU workers | Throughput | Latency | Result |
|---|---|---|---|
| 1 (power saving) | 1.7-2.2 inf/s | 554-600 ms | stable |
| 4 | 7.2-8.2 inf/s | 468-530 ms | stable, 238 MB PSS |
| 6 | 9.6-11.0 inf/s | 492-636 ms | stable, 301 MB RSS |
| 8 | 11.4-12.0 inf/s | 598-745 ms | stable, results 150-250 ms older |
Six workers are the measured optimum. Eight add no reliable throughput and make each result noticeably older, which is exactly what makes a moving overlay feel detached from the image, so the ceiling stays at six and leaves two cores to the camera and UI pipelines.
An earlier revision ramped the worker pool at runtime and kept the smallest count within 95% of peak throughput. Two measurements retired it:
SIGBUS
(BUS_ADRERR) inside libmediapipe_tasks_jni.so. The ramp from four to
eight workers crashed the app every time it was attempted.Both profiles now build their entire worker pool up front. Detector creation is serialized behind one lock and every CPU worker is prewarmed before the first camera inference, which costs about 2.4 seconds when the camera screen opens for the first time and then holds a steady pool for the rest of the session.
MediaPipe creates an EGL context per detector instance even for CPU inference,
and a delegate that aborts the process cannot be caught in Kotlin: the J6’s
32-bit graphics stack crashed inside libmediapipe_tasks_jni.so when the GPU
path was forced. The GPU worker is therefore offered only when the process is
64-bit, the device reports OpenGL ES 3.1 or newer, and no previous probe failed.
The first GPU inference on a device is bracketed by a synchronously persisted
marker: if the process never returns from it, the next launch counts an
incomplete probe and blocks the delegate after the second one. A recoverable GPU
failure falls back to CPU and blocks the delegate immediately.
The next phase optimizes for a balanced real-time experience rather than the largest isolated FPS number. Every change follows this loop:
The preferred configuration should maintain a smooth preview and fresh, readable overlays while consuming materially less CPU and memory than the current maximum-throughput setup.
Each published run should capture the following together:
No public before-and-after claim should mix debug and release builds, different devices, different scenes, or different measurement durations.
The first benchmark protocol should use a physical Android device and record:
Development builds remain useful for React and JavaScript diagnosis. Public performance comparisons must use a release or explicitly profileable build without Metro or development instrumentation.
Acceptance: the same commit and device can run the protocol five times and produce comparable machine-readable results.
The current detector clears each worker’s inputBitmap reference after an
inference. Treat retaining and reusing one correctly sized bitmap per worker as
a hypothesis, then verify allocation rate, GC activity, memory, and throughput
before changing the ownership model permanently. Also measure the compact RGBA
copy path independently from MediaPipe inference.
Acceptance: allocation and memory evidence explains whether bitmap and buffer reuse improves sustained behavior without corrupting frames or detector output.
Compare one through four workers at the supported detector resolutions. Capture preview smoothness, result freshness, latency percentiles, CPU, memory, battery, and thermal drift for every configuration.
Acceptance: the default is selected from an evidence-backed efficiency curve, not from the largest throughput value. A future adaptive scheduler may reduce workers when the device becomes hot or when the scene is stable.
Instrument frame sequence, capture time, inference completion, native publish, React acceptance, and visible presentation. Replace label-plus-array-index IDs with stable tracks based on label and bounding-box overlap, and expire stale tracks deterministically.
Acceptance: overlays do not swap identity when detector ordering changes, and the published result-age distribution remains inside the chosen interaction budget.
Profile the exact live-detection flow with React DevTools. Record why the camera screen and each object card render, inspect the heaviest commit, and verify that interpolation frames stay on the UI thread without creating React commits.
Acceptance: the evidence distinguishes detector updates, React reconciliation, layout work, and Reanimated interpolation instead of treating them as one FPS number.
Run ten- and thirty-minute sessions with the selected worker configuration. Capture thermal status, battery delta, CPU, memory growth, preview smoothness, and detector freshness over time. Include at least one less-capable Android device before making a broad hardware claim.
Acceptance: no unbounded memory growth occurs, degradation over time is documented, and the default can be justified for its target device class.
Measure cold startup to first usable camera frame, analyze the JavaScript bundle and release artifact, test R8 and resource shrinking, restrict release ABIs as appropriate, and verify every packaged native library for 16 KB page alignment.
Acceptance: release startup and size have reproducible baselines, optimizations are validated on device, and native-library alignment is checked before public distribution.
The performance case study is ready to publish when it contains:
The current 30 FPS detector snapshot remains valuable as the origin of the case study. It should be presented as the high-throughput baseline whose efficiency the next experiments will improve.