How to Detect Phone Use in Class with AI (YOLO + Computer Vision)

Detecting phone use in class with AI means running a standard webcam feed through a YOLO object detection model that checks three signals at once — a visible phone, hands near the lap, and a tilted-down head — and only flags a session once those signals combine into a high enough confidence score, then logs the start time, end time and duration automatically.

Why build this instead of just telling students to put phones away

The goal of this system isn't to catch students for the sake of catching them. It's to protect classroom time. If a student is scrolling during a lesson, they're not listening to the teacher, not learning the material, and not doing the writing or classwork that actually moves them forward. A camera that can quietly flag when that's happening — with real data instead of a teacher's guess — turns an enforcement problem into a measurement problem, which is a much easier problem to solve.

How the detection actually works

The core of the system is a Python script built on top of YOLO, an object detection model developed by another team and widely available on GitHub. A standard webcam captures live video, and the model doesn't just look for "a rectangle that might be a phone." It checks three things together: whether a phone is visible in frame, whether the hands are positioned down near the lap (the classic phone-under-the-desk posture), and whether the head is tilted down.

Combining those three signals is what keeps the system from crying wolf. A student who glances down at a textbook or scratches their leg isn't using all three signals at once, so the confidence score stays low and nothing gets logged. Only when the signals line up does the system treat it as an actual phone-use event.

What gets logged, and how good is the data

Every detected session gets logged with a start time, an end time, the total duration, and a confidence score. In testing, one session ran about 7 seconds with a strong confidence score, and a second ran about 8 seconds at roughly 0.85 confidence — close to a 1.0 maximum, meaning the system was highly certain a phone was actually in use. The closer that score sits to 1.0, the more precise the detection is for that specific session.

This isn't a vague "the student looked distracted" alert. It's a timestamped event with a measurable confidence level attached, which is exactly the kind of data a school needs before acting on something as sensitive as monitoring student behavior.

From detection to a report on someone's desk

Catching the behavior is only useful if it reaches the right person without extra manual work. Once a session is logged, it can be exported automatically as a CSV or sent straight into a Google Sheet, and from there it can be routed directly to a principal's or a teacher's desktop. Nobody has to sit and review raw camera footage; they just see the event report.

Real limitations, and what would fix them

It took about a week to build this, through a fair amount of trial and error, and it's honest to call it roughly 95% production-ready rather than a finished commercial product. The biggest limiting factor is hardware: this version ran on a 2017 MacBook Pro's built-in RGB camera, which is a modest, outdated sensor by today's standards.

Even with that limitation, detection worked well when the camera faced the student head-on, since the phone reads as a flatter, clearer shape from that angle — and it still picked up phone use reasonably well even when the phone was held at an angle. Three upgrades would push accuracy further: positioning the camera at the corner of the classroom instead of a laptop lid, adding infrared so detection still works when students dim the lights, and adding depth sensing so the system can better separate a 3D person and object from the background. All three of those are camera and sensor upgrades, not changes to the detection logic itself — the YOLO-based approach already does a decent job within the constraints of the hardware it's given.

Built with an AI coding agent, and headed to GitHub

The system itself was built using an AI agent IDE builder to speed up the Python development — not a requirement for building something like this, but a real time-saver during a week of trial and error. The underlying YOLO model, specifically the YOLO Pro variant, is open source and available on GitHub, and the plan is to publish this project's code there too, so other schools or developers can inspect, reuse or extend it rather than treating it as a black box.

Privacy by design: metadata, not surveillance footage

It's worth being direct about what this system does and doesn't capture. It doesn't store a video reel of a student for a teacher to scroll through later. It logs session metadata — a start time, an end time, a duration, and a confidence score — and that's what gets exported to a CSV or Google Sheet and routed to a principal or teacher. That distinction matters for any school considering something like this: the output is an auditable event log, not a surveillance archive.

Where this goes next

The natural next step is scaling from "one webcam watching one desk" to "one system watching a whole classroom," by mapping out individual desk zones so the same detection logic can track multiple students at once instead of just the one directly in front of the camera. That's a hardware and calibration problem more than a model problem — the underlying YOLO-based detection already generalizes well once it has a clear enough view.

Who this is actually for

This started as a project for a single classroom, but the same underlying detection logic — three combined signals feeding a confidence score, logged as timestamped events rather than raw footage — isn't specific to phones or to classrooms. Schools in particular are a natural fit because the problem (a distraction that's genuinely hard to catch consistently by a teacher glancing around a room) is common and the solution (consistent, unbiased, logged detection) directly addresses it.

Key takeaways

• The system combines three signals — visible phone, hand position, head tilt — into one confidence score, instead of trusting any single cue on its own.
• It's built on YOLO, an existing object detection model, running against a standard webcam feed through a Python script.
• Every session logs start time, end time, duration and a confidence score, so the data is specific and auditable, not just a vague alert.
• Reports can export automatically as CSV or straight to a Google Sheet, landing directly with a teacher or principal with no manual review step.
• The biggest limitation right now is camera hardware; better camera placement, infrared and depth sensing are the clearest paths to improving accuracy.
• The logical next step is mapping desk zones to monitor an entire classroom instead of a single seat.

Want something like this built for your school or business?

A computer vision system like this one can be adapted to other monitoring needs beyond a single classroom camera.