Commit Graph

3492 Commits

Author SHA1 Message Date
Thomas Hallock 73a8314ed9 refactor(worksheet-parsing): centralize state with context + reducer
Major refactoring of worksheet parsing to use centralized state management:

New architecture:
- WorksheetParsingContext: React context provider for parsing state
- state-machine.ts: Typed reducer with actions for streaming lifecycle
- sse-parser.ts: Shared SSE parsing utility for OpenAI Responses API
- usePartialJsonParser.ts: Progressive JSON extraction during streaming

Streaming UI improvements:
- ParsingProgressOverlay: Dark overlay on photo tile during parsing
- ParsingProgressPanel: Collapsible reasoning text panel
- ProgressiveHighlightOverlay: Problem boxes light up as LLM parses
- New streaming API routes: /parse/stream and /parse-selected/stream

Bug fixes during testing:
- Fix TypeScript error: cast event.response for id access in sse-parser
- Fix reparse reasoning display: preserve "processing" status for reparse
- Fix concurrent parsing: revert previous attachment status when switching
- Fix problem count: track dispatched problems to prevent duplicates

Components updated to use context:
- SummaryClient: Wrapped with WorksheetParsingProvider
- OfflineWorkSection: Uses context instead of local streaming state
- PhotoViewerEditor: Uses context for coordinated parsing

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 21:40:36 -06:00
Thomas Hallock 1a4da71f08 feat(worksheet-parsing): wire up focus review callbacks
Connect onApproveProblem, onFlagProblem, and onFocusReviewComplete
callbacks from PhotoViewerEditor to useUpdateReviewProgress hook.

This enables the one-at-a-time problem review flow where users can:
- Approve individual problems
- Flag problems for later review
- Complete the review workflow

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 18:37:09 -06:00
Thomas Hallock b3e2566856 fix(photo-editor): always load OpenCV when entering edit mode
DocumentAdjuster needs OpenCV for the perspective transform preview,
but loadOriginalForEditing was only loading OpenCV when corners
weren't already saved. This caused photos with saved corners to get
stuck on "Loading editor..." forever.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 18:36:48 -06:00
Thomas Hallock ad321d39c5 refactor(practice): extract photo management and worksheet review components
SummaryClient Refactoring:
- Extract SessionAttachmentResponse type to src/types/attachments.ts
- Create usePhotoManagement hook consolidating 10+ callbacks and 7+ state vars
- Create usePhotoViewer hook for photo viewer modal state
- Extract CameraModal and DocumentAdjustmentModal components
- Add getPendingAttachmentId helper to useWorksheetParsing

Worksheet Review UI:
- Add ProblemReviewCard component for individual problem editing
- Add ProblemReviewFlow for swipeable problem navigation
- Add ReviewMiniMap for visual problem overview
- Add ReviewToolbar for review mode controls
- Add ReparseHintsModal for reparse with hints
- Add LLMDebugPanel for viewing raw LLM responses
- Add unapprove endpoint to revert approved worksheets

Storybook:
- Add worksheet parsing stories with sample images
- Update preview configuration

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 18:24:55 -06:00
Thomas Hallock f87733b59c perf(practice): reduce bundle size by 54KB via code splitting
Break the import chain that bundled arcade-specific code (useRoomData,
GameModeProviderWithHooks) with practice pages:

- Create PreviewModeContext.tsx to avoid pulling in full GamePreview module
- Refactor GameModeContext to receive player/room data as props
- Create GameModeProviderWithHooks wrapper that calls the heavy hooks
- Use dynamic imports in arcade layouts for GameModeProviderWithHooks
- Update GamePreview.tsx to dynamically import GameModeProviderWithHooks

Result: Practice summary page First Load JS reduced from 467KB to 413KB.
useRoomData and arcade room code now only loaded by arcade pages.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 12:33:07 -06:00
Thomas Hallock 33fa2f324a fix(know-your-world): resolve infinite loop in MapRenderer useEffect deps
The `interaction` object from useInteractionStateMachine() was included
in several useEffect dependency arrays. However, this hook returns a
useMemo'd object whose dependencies include objects like cursorPosition
and magnifierPosition that get new references on every state change
(even if values are the same).

This caused an infinite loop:
1. State change → new interaction object reference
2. useEffect runs → dispatches event → state change
3. Repeat → "Maximum update depth exceeded" error

Fix: Extract specific stable values from interaction (dispatch, setMode,
state.mode) before using them in dependency arrays, instead of using
the whole interaction object.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 11:28:58 -06:00
Thomas Hallock 1842bd3cf1 perf: lazy-load not-found pages to save ~4.7MB per page load
The fancy interactive 404 page was importing AbacusReact and other heavy
dependencies, causing ~4.7MB of JavaScript to be loaded on EVERY page
(Next.js App Router preloads not-found chunks for soft navigation).

Changes:
- Extract heavy 404 content to InteractiveNotFound component
- Use next/dynamic with ssr: false to lazy-load only when 404 occurs
- Simplify practice/[studentId]/not-found.tsx by removing PageWithNav
- Remove unused manifest.json reference from layout.tsx (was 404ing)
- Simplify /api/classrooms/mine to use getDbUserId helper

Also includes worksheet parsing review progress feature:
- Add useReviewProgress hook for tracking review state
- Add WorksheetReviewSummary component for summary display
- Add review-progress API endpoint
- Integrate into OfflineWorkSection and SummaryClient

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 08:22:41 -06:00
github-actions[bot] 9c3d2b96ea 🎨 Update template examples and crop mark gallery
Auto-generated fresh SVG examples and unified gallery from latest templates.
Includes comprehensive crop mark demonstrations with before/after comparisons.

Files updated:
- packages/templates/gallery-unified.html

🤖 Generated with GitHub Actions

Co-Authored-By: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-01-04 02:48:15 +00:00
Thomas Hallock d4a09e6a90 fix(worksheet-parsing): make cancel actually stop parsing from overwriting
- Fix dual cancel button bug: clear preview state before starting
  re-parse mutation, not after
- Add cancellation check to parse route: re-read status from DB after
  LLM completes, discard results if status was reset to null
- Add same cancellation check to parse-selected route for consistency
- Set parsingStatus to 'processing' at start of parse-selected so
  cancellation can be detected

Previously, clicking cancel would reset the DB status to null, but
when the LLM call completed, it would overwrite with new results.
Now both routes check if parsing was cancelled before writing.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 17:15:33 -06:00
Thomas Hallock 518d9c6cb9 feat(worksheet-parsing): add cancel button for parsing and re-parsing
- Add cancel button to gallery thumbnails when parsing in progress
- Add cancel button to fullscreen PhotoViewerEditor when parsing
- Add cancel button for re-parsing in progress (fullscreen view)
- Track reparsingPhotoId to show correct status per-photo in both views
- Gallery shows "Re-parsing..." badge on specific photo being re-parsed
- DELETE endpoint resets parsing status for immediate retry

Also includes codebase-wide formatting from biome.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 12:24:28 -06:00
Thomas Hallock 2bb22dd1f2 feat(worksheet-parsing): add cancel button for parsing in progress
Users can now immediately cancel a parsing operation instead of waiting
for it to complete or fail. This prevents attachments from getting stuck
in "processing" state with no way to recover.

Changes:
- Added DELETE endpoint to parse API route to cancel/reset parsing
- Added useCancelParsing mutation hook with optimistic updates
- Added cancel button (✕) to the "Analyzing..." badge in OfflineWorkSection
- Cancel button resets the attachment to unparsed state

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 06:53:42 -06:00
Thomas Hallock 63dcca13cf fix(worksheet-parsing): prevent attachments from getting stuck in processing
Two fixes:

1. Outer catch block now updates status to "failed" if an error occurs
   before the parsing starts (e.g., file read error). Previously, the
   attachment would stay in "processing" state forever.

2. Allow retry if attachment has been stuck in "processing" for > 5 mins.
   The `parsedAt` timestamp is now set when entering "processing" state
   to track how long it's been processing.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 06:25:55 -06:00
Thomas Hallock b208213b66 fix(worksheet-parsing): remove .optional() for OpenAI strict mode compatibility
OpenAI's strict JSON schema mode requires ALL properties to be in the
`required` array. The `notes` and `excluded` fields used `.optional()`
which caused them to be excluded from `required`, resulting in API error:
"Missing 'notes'" in required array.

Fix:
- `notes`: Changed from `.nullable().optional()` to just `.nullable()`
- `excluded`: Changed from `.optional()` to `.default(false)`

Both fields are now properly included in the required array while still
allowing null/false values as defaults.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-02 22:13:19 -06:00
Thomas Hallock f7dccb3e0b fix(docker): add llm-client package.json to Dockerfile
The Docker build was failing because packages/llm-client/package.json
was not being copied during the dependency installation stages. This
caused zod and other llm-client dependencies to not be installed,
resulting in "Cannot find module 'zod'" errors during the build.

Added the COPY command for llm-client/package.json in both:
- base stage (line 30) - for build dependencies
- deps stage (line 81) - for production dependencies

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-02 21:48:21 -06:00
Thomas Hallock 67f25cc6f1 chore: update lockfile for llm-client dependencies
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-02 21:42:31 -06:00
Thomas Hallock cf6d47ff4e fix(llm-client): move zod to dependencies for Docker builds
The zod package was listed as peerDependency which isn't automatically
installed in Docker production builds. Moving it to regular dependencies
ensures the package builds correctly in CI.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-02 21:41:13 -06:00
Thomas Hallock 111e934129 feat(worksheet-parsing): add bulk exclude/restore and improve approve flow
- Remove per-problem Exclude/Restore buttons from EditableProblemRow
- Add bulk "Exclude Selected" and "Restore Selected" buttons to selection toolbar
- Add toast notifications for approve success/failure
- Close viewer and refresh page after successful approve to show updated session
- Fix mutation to properly await res.json() before returning

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-02 21:39:43 -06:00
Thomas Hallock 137b31f206 fix(worksheet-parsing): improve UI stability and use global debug toggle
- Fix thumbnail thrashing when adjusting bounding boxes by splitting
  generation into two effects: initial load vs incremental updates
- Use fixed 48x32px thumbnail container with objectFit: contain to
  maintain aspect ratio while preventing layout shifts
- Move selection toolbar outside scrollable area to prevent row jumps
- Replace local debug toggle with global visual debug context
- LLM debug panel now only shows when visual debug is enabled

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-02 20:51:23 -06:00
semantic-release-bot 990b573baa chore(abacus-react): release v2.21.0 [skip ci]
# [2.21.0](https://github.com/antialias/soroban-abacus-flashcards/compare/abacus-react-v2.20.0...abacus-react-v2.21.0) (2026-01-03)

### Bug Fixes

* **practice:** add fallback error message when photo upload is blocked ([33efdf0](33efdf0c0d))
* **vision:** hide detection overlay when auto-detection disabled ([995cb60](995cb60086))
* **vision:** remote camera connection and session management ([8a45415](8a454158b5))

### Features

* add LLM client package and worksheet parsing infrastructure ([5a4c751](5a4c751ebe))
* **observer:** responsive session observer layout ([9610ddb](9610ddb8f1))
* **worksheet-parsing:** add parsing UI and fix parent access control ([91aaddb](91aaddbeab))
* **worksheet-parsing:** add selective re-parsing and improve UI ([830a48e](830a48e74f))
2026-01-03 02:42:27 +00:00
Thomas Hallock 830a48e74f feat(worksheet-parsing): add selective re-parsing and improve UI
Selective Re-parsing:
- Add parse-selected API endpoint for re-parsing specific problems
- Support user-adjusted bounding boxes that persist across re-parses
- Add crop-utils for extracting problem regions from worksheet images

LLM Metadata Tracking:
- Store JSON schema, prompt, and raw response in database
- Add debug panel in PhotoViewerEditor to inspect LLM details
- Add migrations for llm_metadata, llm_prompt, llm_json_schema columns

UI Improvements:
- Remove selection mode toggle - problems always selectable
- Show checkboxes on hover only (no layout jump)
- Move selection toolbar to fixed footer outside scrollable area
- Add BoundingBoxOverlay component for visual problem selection
- Add EditableProblemRow with hover-based checkbox visibility
- Unified hover highlighting across checkbox and problem cells

Also includes:
- Fix approve route to handle excluded problems correctly
- Add DebugContentModal for viewing prompts/responses
- Update LLM client to return metadata in responses

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-02 20:41:09 -06:00
Thomas Hallock 33efdf0c0d fix(practice): add fallback error message when photo upload is blocked
When canUpload is false but there's no specific remediation available
(e.g., due to a bug in access control), show a generic "Unable to upload
photos" banner instead of silently hiding the upload buttons.

This ensures users see feedback when access is unexpectedly denied,
rather than being confused by missing UI elements.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-02 14:14:20 -06:00
Thomas Hallock 91aaddbeab feat(worksheet-parsing): add parsing UI and fix parent access control
Worksheet Parsing UI (Slices 1-2):
- Add parse button to OfflineWorkSection thumbnails and PhotoViewerEditor
- Create ParsedProblemsList component to display extracted problems
- Add useWorksheetParsing hook with mutations for parse/review/approve
- Add attachmentKeys to queryKeys for cache management
- Wire up parsing workflow in SummaryClient

Fix parent upload access:
- Change /api/players/[id]/access to use getDbUserId() instead of getViewerId()
- Guest users' guestId was not matching parent_child.parent_user_id
- Parents can now see upload/camera buttons in offline work section

Fix curriculum type errors:
- Add missing 'advanced' property to createFullSkillSet()
- Fix enabledRequiredSkills -> enabledAllowedSkills in problem-generator
- Remove incorrect Partial<> wrapper from type casts

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-02 14:10:29 -06:00
Thomas Hallock 5a4c751ebe feat: add LLM client package and worksheet parsing infrastructure
Part A: @soroban/llm-client package
- Multi-provider support (OpenAI, Anthropic) via env vars
- Zod schema validation for structured LLM responses
- Retry loop with validation error feedback in prompt
- Progress indication hooks for UI feedback
- Vision support for image analysis

Part B: Worksheet parsing feature
- Zod schemas for parsed worksheet problems
- LLM prompt builder for abacus workbook images
- Parser using llm.vision() with retry logic
- Session converter to create SlotResults for BKT
- Database migration for parsing columns
- API routes: /parse, /review, /approve workflow

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-02 08:49:48 -06:00
Thomas Hallock 9610ddb8f1 feat(observer): responsive session observer layout
- Make session observer modal/page fully responsive for all screen sizes
- Replace absolute positioning with flex layout for problem + abacus
- Create MobileResultsSummary component for compact results on small screens
- Full-screen modal on mobile, centered dialog on desktop
- Stack problem and abacus vertically on small screens (<640px)
- Reduce vertical spacing to eliminate scrolling on mobile
- Hide desktop results panel on mobile, show compact summary chip

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 21:32:20 -06:00
Thomas Hallock d80601d162 chore: add vision planning doc, storybook story, and update gitignore
- Add VISION_DOCK_INTEGRATION_PLAN.md for vision dock architecture
- Add VisionCameraControls.stories.tsx for storybook
- Update .gitignore to exclude venv, uploads, and training data

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 20:43:06 -06:00
Thomas Hallock 995cb60086 fix(vision): hide detection overlay when auto-detection disabled
Add ENABLE_AUTO_DETECTION flag to ObserverVisionFeed.tsx to hide the
useless detection overlay that always showed "---" and "0%" since
auto-detection is globally disabled. This matches the pattern already
used in DockedVisionFeed.tsx.

Also includes minor formatting fixes from Biome.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 20:37:37 -06:00
Thomas Hallock 8a454158b5 fix(vision): remote camera connection and session management
- Fix race condition in useRemoteCameraDesktop where session ID wasn't
  saved before socket connection check, preventing auto-reconnect
- Same fix in useRemoteCameraPhone for phone-side connection
- Fix "new session" button in RemoteCameraQRCode - properly clears old
  session and creates new one using prevRef to detect state changes
- Show full QR code UI with copyable URL (removed compact mode)
- Redesign AbacusVisionBridge UI: camera feed as hero, toolbar on feed,
  collapsible crop settings, source selector as tabs

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 19:01:46 -06:00
semantic-release-bot 41aa7ff33f chore(abacus-react): release v2.20.0 [skip ci]
# [2.20.0](https://github.com/antialias/soroban-abacus-flashcards/compare/abacus-react-v2.19.0...abacus-react-v2.20.0) (2026-01-02)

### Bug Fixes

* **vision:** clear config when switching camera sources ([ff59612](ff59612e7b))
* **vision:** hide flip camera button when only one camera available ([7a9185e](7a9185eadb))
* **vision:** include remote camera in isVisionSetupComplete check ([a8fb77e](a8fb77e8e3))
* **vision:** remote camera persistence and UI bugs ([d90d263](d90d263b2a))

### Features

* **vision:** add activeCameraSource tracking and simplify calibration UI ([1be6151](1be6151bae))
* **vision:** add CV-based bead detection and fix remote camera connection ([005140a](005140a1e7))
* **vision:** add TensorFlow.js column classifier model and improve detection ([5d0ac65](5d0ac65bdd))
* **vision:** broadcast vision frames to observers (Phase 5) ([b3b769c](b3b769c0e2))
* **vision:** disable auto-detection with feature flag ([a5025f0](a5025f01bc))
* **vision:** integrate vision feed into docked abacus ([d8c7645](d8c764595d))
2026-01-02 00:02:33 +00:00
Thomas Hallock 1be6151bae feat(vision): add activeCameraSource tracking and simplify calibration UI
- Add explicit activeCameraSource field to VisionConfig to track which
  camera is in use (local vs phone), fixing button visibility bugs when
  switching between camera sources
- Simplify calibration UI by removing the confusing "Auto/Manual" mode
  toggle, replacing with a cleaner crop status indicator
- Remove calibration requirement from isVisionSetupComplete for local
  camera since auto-crop runs continuously when markers are detected
- Update DockedVisionFeed to use activeCameraSource instead of inferring
  from which configs are set

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 18:01:12 -06:00
Thomas Hallock 70b363ce88 refactor(vision): combine setup modal into single draggable experience
- Merge VisionSetupModal and AbacusVisionBridge into unified UI
- Remove two-step configuration process (no more "Configure Camera" button)
- Add vision control props to AbacusVisionBridge:
  - showVisionControls, isVisionEnabled, isVisionSetupComplete
  - onToggleVision, onClearSettings callbacks
- Add Enable/Disable Vision and Clear Settings buttons to bridge footer
- Simplify VisionSetupModal from ~257 to ~93 lines
- Modal is now draggable via framer-motion (built into AbacusVisionBridge)

User experience: Open modal → immediately see camera feed and all controls
in one place. Drag modal anywhere. Configure, enable/disable, close.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 17:30:46 -06:00
Thomas Hallock d90d263b2a fix(vision): remote camera persistence and UI bugs
- Fix camera source switching: clear remoteCameraSessionId in context when
  switching to local camera so DockedVisionFeed uses the correct source
- Fix modal drag during calibration: disable framer-motion drag when
  calibration overlay is active to allow handle dragging
- Fix initial camera source: pass initialCameraSource prop to
  AbacusVisionBridge so it shows phone camera when reconfiguring remote
- Extend session TTL from 10 to 60 minutes for remote camera sessions
- Add localStorage persistence for remote camera session IDs
- Add auto-reconnect logic for both desktop and phone hooks
- Add comprehensive tests for session-manager, useRemoteCameraDesktop,
  and useRemoteCameraPhone hooks
- Guard test setup.ts for node environment (HTMLImageElement check)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 17:23:27 -06:00
Thomas Hallock 43524d8238 test: add unit tests for vision broadcast feature
- VisionIndicator.test.tsx: tests for rendering, status indicator, click behavior, accessibility
- ObserverVisionFeed.test.tsx: tests for image display, detected value, live/stale indicator
- useSessionBroadcast.vision.test.ts: tests for sendVisionFrame socket emission
- useSessionObserver.vision.test.ts: tests for visionFrame receiving and cleanup
- MyAbacusContext.vision.test.tsx: tests for vision config state and callbacks

Also fixes:
- useSessionObserver: clear visionFrame and transitionState on stopObserving
- test/setup.ts: add canvas Image mock to prevent jsdom errors with data URIs

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 16:08:51 -06:00
Thomas Hallock a5025f01bc feat(vision): disable auto-detection with feature flag
- Add ENABLE_AUTO_DETECTION flag (set to false) in DockedVisionFeed
- Conditionally import detection modules for tree-shaking when disabled
- Guard all detection processing, loops, and value handlers
- Hide detection overlay when auto-detection is disabled
- Remove vision toggle button from ActiveSession (no longer needed)
- Clean up unused imports and code
- Format fixes from biome

The camera feed still works for observation mode, but the ML/CV
bead detection is disabled until accuracy is improved.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 15:55:50 -06:00
Thomas Hallock a8fb77e8e3 fix(vision): include remote camera in isVisionSetupComplete check
The isVisionSetupComplete flag was only checking for local camera
setup (cameraDeviceId + calibration), which caused remote camera
mode to be treated as "not configured" even when connected.

Now considers vision setup complete if either:
- Local camera: has camera device AND calibration
- Remote camera: has remote session ID (phone handles calibration)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 15:36:27 -06:00
Thomas Hallock e80ef04f45 chore(vision): clean up debug console.log statements
Remove unnecessary debug logging from vision components
that was used during development.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 15:31:06 -06:00
Thomas Hallock b3b769c0e2 feat(vision): broadcast vision frames to observers (Phase 5)
Wire up the vision broadcast pipeline:

1. DockedVisionFeed captures rectified frames from canvas and emits
   them at 5fps via the context's emitVisionFrame callback

2. PracticeClient wires setVisionFrameCallback to call sendVisionFrame
   from useSessionBroadcast, connecting the context to the socket

3. useSessionBroadcast sends VisionFrameEvent to the session channel
   with imageData, detectedValue, and confidence

4. socket-server relays vision-frame events to observers

5. useSessionObserver receives and stores visionFrame for display

6. SessionObserverModal shows ObserverVisionFeed when visionFrame
   is available, replacing the interactive AbacusDock with the
   student's live camera feed

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 15:28:59 -06:00
Thomas Hallock ff59612e7b fix(vision): clear config when switching camera sources
When switching between local and phone camera, clear the other
source's configuration to prevent stale data.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 15:13:20 -06:00
Thomas Hallock d8c764595d feat(vision): integrate vision feed into docked abacus
- Add vision state management to MyAbacusContext (camera, calibration,
  remote session, enabled state)
- Add VisionIndicator component showing vision status on dock header
- Add VisionSetupModal for configuring camera and calibration
- Add DockedVisionFeed component that replaces SVG abacus when vision
  is enabled, with:
  - Continuous ArUco marker detection for auto-calibration
  - OpenCV perspective correction via VisionCameraFeed
  - Real-time bead detection and value display
  - Support for both local camera and remote phone camera
- Wire AbacusVisionBridge to save config to context via
  onConfigurationChange callback
- Update MyAbacus to conditionally render DockedVisionFeed vs
  AbacusReact based on vision state

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 15:05:58 -06:00
Thomas Hallock 005140a1e7 feat(vision): add CV-based bead detection and fix remote camera connection
- Add beadDetector.ts with intensity-profile-based bead detection (CV approach)
- Integrate CV pipeline for both local camera and remote phone camera feeds
- Add processImageFrame() to frameProcessor for remote camera image processing
- Fix React 18 Strict Mode duplicate session creation in RemoteCameraQRCode
- Add debug logging to remote camera hooks for connection troubleshooting
- Add VisionStatusIndicator for remote camera feed in AbacusVisionBridge

The duplicate session bug was caused by React 18 Strict Mode double-mounting
components and running effects twice with fresh state, which called
createSession() twice and created two different sessions - phone joined
one, desktop subscribed to the other.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 11:29:02 -06:00
Thomas Hallock 5d0ac65bdd feat(vision): add TensorFlow.js column classifier model and improve detection
- Add trained CNN model for abacus column digit classification
  - model.json: TensorFlow.js layers model (fixed for Keras 3 compatibility)
  - group1-shard1of1.bin: quantized model weights (~2.2MB)

- Improve detection performance and stability
  - Throttle inference to 5fps (was running every animation frame)
  - Lower stability threshold: 3 consecutive frames (was 10)
  - Lower confidence threshold: 50% (was 70%)

- Clean up debug logging from development

Note: Model trained on synthetic data, accuracy on real images is limited.
Future work: retrain on real abacus photos for better accuracy.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 22:59:40 -06:00
Thomas Hallock 7a9185eadb fix(vision): hide flip camera button when only one camera available
Only show camera controls section when there's something to display:
- Flip button: only if multiple cameras
- Torch button: only if torch available
- Whole section: only if either button would be shown

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 22:35:52 -06:00
semantic-release-bot da97ad0675 chore(abacus-react): release v2.19.0 [skip ci]
# [2.19.0](https://github.com/antialias/soroban-abacus-flashcards/compare/abacus-react-v2.18.0...abacus-react-v2.19.0) (2026-01-01)

### Features

* **vision:** add physical abacus column setting and fix remote flash toggle ([b206eb3](b206eb3071))
* **vision:** improve remote camera calibration and UX ([8846cec](8846cece93))
2026-01-01 04:35:16 +00:00
Thomas Hallock b206eb3071 feat(vision): add physical abacus column setting and fix remote flash toggle
Physical Abacus Columns Setting:
- Add physicalAbacusColumns to AbacusDisplayConfig (default: 4)
- Add database column with migration 0054
- Add slider UI in AbacusDisplayDropdown (range 1-21)
- Update AbacusVisionBridge to use setting instead of calculating from problem

Remote Camera Flash Toggle Fix:
- Add socket events for torch sync (set-torch, torch-state)
- Phone reports torch state to desktop on change/connection
- Desktop can control phone's torch remotely
- Add torch button in AbacusVisionBridge for phone camera mode
- Both local and remote flash toggles now work correctly

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 22:33:58 -06:00
Thomas Hallock 8846cece93 feat(vision): improve remote camera calibration and UX
- Fix remote camera autocrop rotation by swapping ArUco corners for phone camera
  (detectMarkers assumes Desk View orientation which is 180° rotated)
- Add rotate left/right buttons to CalibrationOverlay for manual calibration
- Fix mode switching bug: switching to auto mode now clears desktop calibration
  on phone via new 'remote-camera:clear-calibration' socket event
- Add copy button to QR code URL with visual feedback
- Fix text selection spanning into video feed with userSelect: none
- Add flip camera and torch controls to local camera UI
- Add session persistence for remote camera reconnection
- Fix 4:3 aspect ratio for cropped abacus output

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 22:04:06 -06:00
semantic-release-bot 937223e318 chore(abacus-react): release v2.18.0 [skip ci]
# [2.18.0](https://github.com/antialias/soroban-abacus-flashcards/compare/abacus-react-v2.17.0...abacus-react-v2.18.0) (2026-01-01)

### Bug Fixes

* allow teacher-parents to enroll their children in other classrooms ([52df7f4](52df7f4697))
* **bkt:** handle missing helpLevelUsed in legacy data causing NaN ([b300ed9](b300ed9f5c))
* **camera:** handle race condition in camera initialization ([2a24700](2a24700e6c))
* **classroom:** auto-transition tutorial→session observation + fix NaN display ([962a52d](962a52d756))
* **classroom:** broadcast digit-by-digit answer and correct phase indicator ([fb73e85](fb73e85f2d))
* **dashboard:** compute skill stats from session results in curriculum API ([11d4846](11d48465d7))
* **db:** add missing is_paused column to session_plans ([9d8b5e1](9d8b5e1148))
* **db:** add missing journal entries for migrations 0041-0042 ([398603c](398603c75a))
* **docker:** add canvas native deps for jsdom/vitest ([5f51bc1](5f51bc1871))
* **docker:** override canvas with mock package for Alpine/musl ([8be1995](8be19958af))
* **docker:** skip canvas native build (optional jsdom dep) ([d717f44](d717f44fcc))
* **observer:** seed results panel with full session history ([aab7469](aab7469d9e))
* only show session stats when there are actual problems ([62aefad](62aefad676))
* **practice:** allow teachers to create student profiles ([5fee129](5fee1297e1))
* **practice:** always show add student FAB button ([a658414](a6584143eb))
* **practice:** real-time progress in observer modal + numeric answer comparison ([c0e63ff](c0e63ff68b))
* **practice:** show active sessions for teacher's own children ([ece3197](ece319738b))
* **practice:** use Next.js Link for student tiles + fix session observer z-index ([6def610](6def610877))
* **seed:** accurate BKT simulation for developing classifications ([d5e4c85](d5e4c858db))
* **share:** use getShareUrl for correct production URLs ([98a69f1](98a69f1f80))
* **vision:** fix manual calibration overlay not showing on remote camera ([44dcb01](44dcb01473))
* **vision:** fix remote camera calibration coordinate system ([e52f94e](e52f94e4b4))
* **vision:** swap corners diagonally for webcam orientation ([dd8efe3](dd8efe379d))

### Features

* API authorization audit + teacher enrollment UI + share codes ([d6e369f](d6e369f9dc))
* **camera:** auto-start camera when opening camera modal ([f3bb0ae](f3bb0aee4f))
* **camera:** fullscreen modal with edge-to-edge preview ([db17c96](db17c96168))
* **chart:** add grouped structure to chart hover tooltip ([594e22c](594e22c428))
* **chart:** improve skill classification visual hierarchy with colors and patterns ([c9518a6](c9518a6b99))
* **classroom:** add active sessions API endpoint ([07f6bb7](07f6bb7f9c))
* **classroom:** add real-time enrollment/unenrollment reactivity ([a0693e9](a0693e9084))
* **classroom:** add session broadcast and active session indicators ([9636f7f](9636f7f44a))
* **classroom:** add unified add-student modal with two-column layout ([dca696a](dca696a29f))
* **classroom:** add unified TeacherClassroomCard with auto-enrollment ([4d6adf3](4d6adf359e))
* **classroom:** complete reactivity fixes (Steps 7-11) ([2015494](2015494c0e))
* **classroom:** consolidate filter pill to single-row design ([78a63e3](78a63e35e3))
* **classroom:** implement enrollment system (Phase 4) ([1952a41](1952a412ed))
* **classroom:** implement entry prompts system ([de39ab5](de39ab52cc))
* **classroom:** implement real-time enrollment updates ([bbe0500](bbe0500fe9))
* **classroom:** implement real-time presence with WebSocket (Phase 6) ([629bfcf](629bfcfc03))
* **classroom:** implement real-time session observation (Step 3) ([2feb684](2feb6844a4))
* **classroom:** implement real-time skill tutorial observation ([4b73879](4b7387905d))
* **classroom:** implement teacher classroom dashboard (Phase 3) ([2202716](2202716f56))
* **classroom:** implement teacher-initiated pause and fix manual pause ([ccea0f8](ccea0f86ac))
* **classroom:** implement two-way abacus sync for session observation (Step 5) ([2f7002e](2f7002e575))
* **classroom:** improve enrollment reactivity and UX ([77336be](77336bea5b))
* **classroom:** integrate create student form into unified add-student modal ([da92289](da92289ed1))
* **classroom:** integrate Enter Classroom into StudentActionMenu ([2f1b9df](2f1b9df9d9))
* **dashboard:** add skill progress chart with trend analysis and timing awareness ([1fc8949](1fc8949b06))
* enable parents to observe children's practice sessions ([7b82995](7b82995664))
* **family:** implement parent-to-parent family code sharing (Phase 2) ([0284227](02842270c9))
* improve session summary header and add practice type badges ([518fe15](518fe153c9))
* **observer:** add live active session item to history list ([91d6d6a](91d6d6a1b6))
* **observer:** add live results panel and session progress indicator ([8527f89](8527f892e2))
* **observer:** implement shareable session observation links ([3ac7b46](3ac7b460ec))
* **practice:** add auto-rotation for captured documents ([ff79a28](ff79a28c65))
* **practice:** add document adjustment UI and auto-capture ([473b7db](473b7dbd7c))
* **practice:** add document scanning with multi-quad tracking ([5f4f1fd](5f4f1fde33))
* **practice:** add fixed filter bar, sticky headers, and shared EmojiPicker ([0e03561](0e0356113d))
* **practice:** add intervention system and improve skill chart hierarchy ([bf5b99a](bf5b99afe9))
* **practice:** add mini start practice banner to QuickLook modal ([d1176da](d1176da9aa))
* **practice:** add Needs Attention to unified compact layout ([8727782](8727782e45))
* **practice:** add photo attachments for practice sessions ([9b85311](9b853116ec))
* **practice:** add photo editing with rotation persistence and auto-detect ([156a0df](156a0dfe96))
* **practice:** add smooth fullscreen transition from QuickLook to dashboard ([cb8b0df](cb8b0dff67))
* **practice:** add student organization with filtering and archiving ([538718a](538718a814))
* **practice:** add StudentActionMenu to dashboard + fix z-index layering ([bf262e7](bf262e7d53))
* **practice:** compact single-student categories and UI improvements ([0e7f326](0e7f3265fe))
* **practice:** implement measurement-based compact layout ([1656b93](1656b9324f))
* **practice:** implement retry wrong problems system ([474c4da](474c4da05a))
* **practice:** parent session observation + relationship UI + error boundaries ([07484fd](07484fdfac))
* **practice:** polish unified student list with keyboard nav and mobile UX ([0ba1551](0ba1551fea))
* **seed:** add category field to all mock student profiles ([f883fbf](f883fbfe23))
* **session-summary:** redesign ProblemToReview with BKT integration and animations ([430c46a](430c46adb9))
* **storybook:** add TeacherClassroomCard stories ([a5e5788](a5e5788fa9))
* **vision:** add AbacusVisionBridge for physical soroban detection ([47088e4](47088e4850))
* **vision:** add ArUco marker auto-calibration for abacus detection ([9e9a06f](9e9a06f2e4))
* **vision:** add remote phone camera support for abacus detection ([8e4975d](8e4975d395))

### Performance Improvements

* reduce practice page dev bundle from 47MB to 115KB ([fd1df93](fd1df93a8f))
2026-01-01 02:42:45 +00:00
Thomas Hallock e52f94e4b4 fix(vision): fix remote camera calibration coordinate system
- Fix hook dependency issues in AbacusVisionBridge by using destructured
  stable function references instead of remoteCamera object
- Add proper container dimension tracking for remote camera calibration
  overlay to fix coordinate mismatch
- Add rotate180 option to perspectiveTransform to support both Desk View
  (camera pointing down, needs 180° rotation) and phone cameras (no rotation)
- Phone camera cropping now uses direct mapping without rotation

The main issues fixed:
1. Hook dependencies were causing effects to run repeatedly when navigating
   to remote camera URL in a different window
2. CalibrationOverlay was using hardcoded fallback dimensions instead of
   actual container dimensions for remote camera
3. Perspective transform was applying 180° rotation which is wrong for
   phone cameras held at normal angles

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 20:41:25 -06:00
Thomas Hallock 957ff71cf1 chore: polish vision and practice components
- CalibrationOverlay: formatting fixes
- useDeskViewCamera: add SSR guard and focusMode setting
- ActiveSession: integrate vision mode changes
- DocumentAdjuster/PhotoViewerEditor: component updates
- SummaryClient: updates for practice summary
- Drizzle meta: formatting consistency

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 19:07:49 -06:00
Thomas Hallock 44dcb01473 fix(vision): fix manual calibration overlay not showing on remote camera
- Re-run container dimension effect when isCalibrating changes
- Add height checks in addition to width checks for overlay visibility
- Add delayed dimension update to handle layout settling

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 19:01:42 -06:00
Thomas Hallock 8e4975d395 feat(vision): add remote phone camera support for abacus detection
Enable using a phone as a remote camera source for abacus vision detection.
The phone handles calibration and perspective correction locally, then
streams cropped frames to the desktop via Socket.IO.

Features:
- QR code generation for easy phone connection
- Auto-calibration with ArUco markers on phone
- Manual calibration option
- Proper OpenCV perspective transform (not bounding box crop)
- Real-time frame streaming with frame rate display
- LAN address support via NEXT_PUBLIC_LAN_HOST env var

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 18:59:24 -06:00
Thomas Hallock 9e9a06f2e4 feat(vision): add ArUco marker auto-calibration for abacus detection
- Add js-aruco2 library for pure JavaScript marker detection
- Create /create/vision-markers page for downloading printable marker PDFs
- Add auto/manual calibration mode toggle to AbacusVisionBridge
- Implement automatic quad detection from 4 corner markers (IDs 0-3)
- Fix 180° rotation for Desk View camera orientation
- Add loading state for marker library initialization

🤖 Generated with [Claude Code](https://claude.ai/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-31 18:16:48 -06:00