[rover0811]

Summary

Introduces a structured, zero-cost observability layer across the food-crawling pipeline. All instrumentation is implemented via pure Python dataclasses and enums — no external APM, no additional AWS services, no added cost. Slack notifications are consolidated to one summary message per date instead of per-restaurant, reducing noise and API calls.

Key Changes

Structured Observability Contracts

• Added typed outcome enums and dataclass payloads (ObservationResult, ScrapingOutcome, SchedulingOutcome) for all pipeline stages
• JSON-serializable observation records emitted to CloudWatch Logs for zero-cost structured querying
• Five outcome categories tracked per scrape: SUCCESS, PARTIAL, EMPTY, SKIPPED, FAILED

Safe Exception and Retry Handling

• Hardened parsing utility fallbacks to prevent unhandled exceptions from propagating silently
• Instrumented existing client retry logic with structured outcome recording
• Sanitized restaurant exception payloads to strip PII and raw stack traces before logging

Consolidated Slack Notifications

• Aggregated scraping and scheduling outcomes into a single Slack summary per date
• Eliminated per-restaurant Slack calls; one message per pipeline run
• Terminal failure notifications sanitized to remove internal paths and secrets

Step Functions Correlation

• Normalized handler events for both scraping and scheduling Lambda entry points
• Execution context (Step Functions execution ARN, state name) correlated into each observation record

Infrastructure Hardening

• Public API Gateway endpoint removed; Lambda invocation secured via IAM
• CloudWatch Log group retention set to 30 days
• Existing Lambda log groups must be imported into Terraform state before first deployment (see Deployment Note)

Validation

Check	Result
Unit + integration tests	✅ 148 passed, 0 failed
LSP diagnostics	✅ 0 errors, 0 warnings
Python compile (all modules)	✅ Clean
Step Functions ASL validate	✅ Valid
SAM validate	✅ Valid
SAM build	✅ Successful

Deployment Note

Not yet deployed. Before applying Terraform:

1. Import existing Lambda log groups into Terraform state to avoid destroy/recreate:

   terraform import aws_cloudwatch_log_group.scraper /aws/lambda/<scraper-function-name>
   terraform import aws_cloudwatch_log_group.scheduler /aws/lambda/<scheduler-function-name>
   

2. Known warning: Lambda layer architecture mismatch may surface during sam build on Apple Silicon — this is a local build artifact and does not affect deployed runtime behavior.

Commits (15)

• 4304b2c Add structured observability contracts
• 687a87e Sanitize restaurant exception payloads
• ba323f2 Reset observability dependency context
• 9ed7b71 Harden parsing utility fallbacks
• edd36c7 Record time slot strategy outcomes
• 972a5ad Classify scraper source outcomes
• c0643b4 Summarize Slack notifications by date
• 5ba1bb3 Instrument existing client retries
• 6411ac9 Aggregate scraping outcomes
• c102115 Aggregate scheduling outcomes
• 23a37a7 Normalize scheduling handler events
• 731e14c Normalize scraping handler events
• ad54111 Sanitize terminal failure notifications
• 77abecd Secure workflow infrastructure contracts
• 774d050 Expand deterministic observability CI coverage

[gemini-code-assist]

Code Review

This pull request refactors the application architecture by removing public API Gateway endpoints in favor of IAM-authorized direct Lambda invocations and EventBridge schedules, adding CloudFormation-managed log groups with a 30-day retention policy, and introducing a robust observability and exception sanitization framework. Additionally, scrapers and clients were updated to track slot-level processing outcomes with structured logging. The review feedback highlights two key improvement opportunities: optimizing the dormitory scraping handler by avoiding asyncio.run() inside a loop to prevent resource exhaustion, and using get_text(strip=True) in the HTML table parser to ensure clean dictionary keys.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

Calling asyncio.run() inside a loop creates and destroys a new event loop for every single iteration. This is highly inefficient and can lead to resource exhaustion or connection issues if the HTTP client/connection pool is tied to the loop lifecycle. Consider wrapping the loop in an async helper function and running it once using asyncio.run(), utilizing asyncio.gather for concurrent execution.

        async def send_all_summaries():
            await asyncio.gather(*(
                notification_service.send_date_summary(
                    _summary_from_parsed(parsed_menu)
                )
                for parsed_menu in parsed_menus
            ))
        asyncio.run(send_all_summaries())

[rover0811]

Fixed in commit 8491ea4. Introduced a single async helper _dormitory_scrape_and_notify that handles both the scrape and all notification sends in one coroutine, so the handler calls asyncio.run() exactly once regardless of how many menu slots are returned. Sends remain sequential (one await notification_service.send_date_summary() per parsed menu) rather than concurrent — this avoids Slack burst/rate-limit issues that asyncio.gather could trigger when multiple summaries arrive simultaneously. Two new tests assert the single-event-loop contract: test_dormitory_scraping_handler_uses_single_asyncio_run (success path, 2 menus → 1 asyncio.run call) and test_dormitory_scraping_handler_domain_error_uses_single_asyncio_run (domain error path → still 1 asyncio.run call). Both pass.

[gemini-code-assist]

td_tag.get_text() returns the raw text of the table cell, which may contain leading/trailing whitespaces or newlines depending on the HTML formatting. Using get_text(strip=True) ensures clean keys for the dictionary.

Suggested change

	menu_nm_dict[td_tag.get_text()] = tr_tag
	menu_nm_dict[td_tag.get_text(strip=True)] = tr_tag

[rover0811]

Fixed in commit 6d67b88. Applied get_text(strip=True) as suggested — the one-character change eliminates leading/trailing whitespace and newlines from dictionary keys. Added a regression test (test_parse_table_to_dict_strips_whitespace_from_keys) that covers both multi-line indented text and HTML entity newlines (
), asserting that keys like '중식1' and '석식1' are present without surrounding whitespace. All 26 tests in test_utils.py + test_lambda_handlers.py pass.