DevWatch

The sample reports / 01—03

Read the whole report.

What shipped. How demanding it was. What needs fixing.
Three projects, with the analysis and evidence to back it up.

Real projects
Names anonymized
DevWatch / Project assessmentReport 01

Jan 2026 – Aug 2026 · Outside agency

AI answering & quoting for small businesses

Confidence: medium

Period score

72/100
Output across the full period

Difficulty

68/100
How demanding the work was

Quality

52/100
Assessment of the code

Findings

6
Issues & questions to follow up

Scores are AI estimates from code changes. Read the evidence and limits below.

The short version

A substantial product. Too much depends on one person.

The later work is careful and well tested. The next priorities are rotating exposed credentials and getting a second engineer familiar with the calls, payments, and deployment.

Built

An AI receptionist that can quote, book, and follow up.

Phone, email, and text agents; customer and admin dashboards; calendar, billing, and CRM integrations.

Improved

The most demanding work came with stronger tests.

The final stretch added two interchangeable voice providers and safeguards against duplicate calls and charges.

Needs attention

Secrets in history. Knowledge concentrated in one developer.

Committed credentials and certificate keys need rotation. Two agent tools also appear to call the wrong address.

01

What the code shows

The full analysis

Over seven months this repository went from empty to a working AI answering and quoting service for small businesses.

It now holds the screens customers use (dashboard, calendar, call and email inboxes, agent setup), a separate internal admin console for your own team, the behind-the-scenes systems that run everything, a public documentation site, and connections to a voice-AI provider and Twilio for phone calls, Stripe for billing, Google and Microsoft for calendars and mailboxes, and HubSpot and an industry CRM for pushing leads into customers' existing sales tools.

The biggest single pieces built were: a self-service signup and agent-training wizard; a quoting engine that prices moves from hourly, flat, mileage and hybrid rules including overtime, travel time and fuel; an AI agent that answers phone calls, texts and emails, books and reschedules jobs, and hands off to a human; and, in July, a system that lets you run the same phone agent on either of two voice providers and switch a customer between them safely while calls are in flight.

About 280,000 lines of hand-written change landed, alongside roughly another 277,000 lines of generated or copied-in content (automatically produced database and routing files, a vendor's email templates, encoded logo images) that does not represent original work and is excluded from the assessment. Nearly all of this came from one person, working under two email addresses, in concentrated bursts across 51 active days.

The last three months of work in particular is careful, well-tested engineering. The main concerns are that credentials and website certificate private keys were committed into the repository early on and remain in its history, that several very large single commits are effectively unreviewable, and that the project has no second pair of eyes on any of it.

02

Delivery over time

Selected weeks

A closer look at four weeks from the report. These are selected snapshots, not a complete timeline.

Week of Feb 9, 202685/100

Dashboard & agent foundations

The heaviest early week: a complete new dashboard (about 24,000 lines, part boilerplate), automatic synchronisation of the AI agent's instructions using a background job queue, fixes to social and email login, dashboard analytics built on database views, and deletion of the old 26,000-line web app. A run of trial-and-error deployment commits also sits here.

Week of Mar 30, 202685/100

A working quoting engine

Biggest week of the first half. The quoting engine landed with its own pricing rules (hourly, flat, mileage, hybrid, overtime, travel, fuel, minimum floors) and tests, metered access for outside software, usage reporting, service-type management screens in two applications, subscription plan management and a documentation site. Also a password reset fix and recovery of previously uncommitted database changes.

Week of Jun 8, 202688/100

CRM, email & calendar integrations

The largest week by delivered scope: dedicated HubSpot and an industry CRM lead delivery with a central lead create-or-update pipeline (about 28,000 lines), mailbox synchronisation for Gmail and Microsoft with an AI email agent that can quote and reschedule, a hand-written calendar replacing the third-party one, and dynamic pricing plans in signup. Includes a deployment safety check for the production database.

Week of Jul 27, 202692/100

Two voice providers, one call system

Exceptional week. One commit of about 81,000 lines introduced the switchable phone-call system with two voice providers, safeguards against processing the same call or charge twice, cost tracking, and real database and cache integration tests. On top of that: the admin organisations area rebuilt, staff impersonation for support, protection against a class of server-side request attacks, and repairs to lead delivery.

Reviewers rated this the most demanding work of the period.

03

Contribution & concentration

Who did what

Contributions visible in the repository. Hours are estimates of hands-on effort, not recorded time.

Developer A

about 99% of the difficulty-weighted work

Commits
213
Active days
37
Estimated hours
850–1280

Built essentially the entire product over seven months: the customer dashboard, a separate internal admin dashboard, the behind-the-scenes systems, the database, and the integrations with a voice-AI provider, Twilio, Stripe, Google, HubSpot and an industry CRM.

Delivered the self-service signup and agent-training wizard, the quoting engine with its own pricing rules and metered access for outside software, the email and SMS agents with mailbox synchronisation, and the switchable phone-call system with the safety controls that stop calls or payments from being processed twice.

Also handled deployment, monitoring, backups, and a security hardening pass that added request throttling, tightened access to internal connection points, and signed the login handshakes with outside services. Roughly 26,000 lines of automated checks were added, most of them from June onwards.

Developer B

negligible, under 1%

Commits
1
Active days
1
Estimated hours
0

One single-line change to an image on the signup page, on 7 February.

04

Evidence to act on

Findings 6

Some findings need a fix; others need a question answered. The detail distinguishes past issues, possible defects, and risks that still need checking.

  1. 01

    Committed credentials

    Passwords and secret keys were saved into the project's files where anyone with repository access can read them, and they stay readable in the project's history even after being deleted.

    Reviewers found: a plaintext PostgreSQL password in docker-compose configuration (January, repeated later), a plausible real authentication secret in .env.example, a voice-AI provider webhook secret in backend source and in a WEBHOOK_TESTS.md file that was later deleted, Let's Encrypt certificate private keys (infra/nginx/ssl/privkey.pem, grafana-privkey.pem) committed and repeatedly replaced, a live quoting access key in scripts/quoting-test.md, provider-key-looking values and real phone numbers in examples/pooling-numbers.json, and a plaintext Prometheus authorization token in infra/monitoring/prometheus/prometheus.yml.

    All of these should be treated as compromised and rotated.

  2. 02

    Session data written to logs

    For a period, the system wrote its own diagnostic output into logs including complete request headers, which can contain login session cookies. If those logs were retained anywhere, they could be used to impersonate users. This appeared in the login/authorization layer and the shared client code in February and was cleaned up during the May hardening pass, but the earlier deployment window is worth confirming with the developer.

  3. 03

    Probable live defect

    Two of the AI agent's abilities may be calling the wrong address and therefore silently not working. A configuration change on 24 July appears to have copy-pasted one address into the appointment-booking and text-a-link tool definitions, so both point at /api/tools/tally instead of their own connection points. Quick to verify and fix.

  4. 04

    Unreviewable commit sizes

    Some changes are too large for a second person to check. Several commits bundle a month of work into one submission, including one of roughly 81,000 lines covering the voice-call system, billing, database changes, admin screens and infrastructure at once. Reviewers also noted duplicate identical commits on 30 July, which usually indicates a mistake in how branches were merged rather than extra work.

    This does not make the code wrong, but it means no one else has realistically reviewed it.

  5. 05

    Volume inflated by generated and copied content

    Reported line counts are much larger than the work behind them. Roughly 277,000 of the added lines are machine-generated or copied from elsewhere: automatically produced database type and routing files, a vendor's sample email templates complete with Notion/Stripe/Vercel images, duplicated AI assistant documentation stored three times over, and logo images encoded as text. These were excluded from the effort assessment.

  6. 06

    Single-person dependency

    One person holds all working knowledge of the system. Across 237 commits, all but one line came from a single developer using two email addresses. If that person becomes unavailable, there is no one with context on the phone-call handling, the payment flows or the deployment process.

05

Recommended action

What to do next

The work is real and, from late May onwards, of good quality. Two things are worth acting on.

First, commission a one-time security review focused on secrets: database passwords, a monitoring token, a live quoting access key, a service webhook secret and Let's Encrypt certificate private keys were all committed at various points between January and June. Removing them from current files does not remove them from history, so they should be rotated regardless of whether the repository is private.

Second, address the single-person concentration. One developer holds all knowledge of the phone-call system, the payment handling and the deployment process, and several commits (one of about 81,000 lines) are too large for anyone else to review. Getting a second engineer to read through the voice-call and Stripe handling, and asking for smaller commits going forward, would reduce the risk cheaply.

Beyond that, one small defect flagged by review is worth a five-minute check: two of the AI agent's tools appear to call the wrong address.

06

Coverage & uncertainty

Scope & limits

The counts describe the material reviewed. They are not a measure of developer productivity.

Commits reviewed
237
Days with commits
52 of 214
Contributor identities in the repository
3
Lines added / removed
+280,208 / −102,358
Test lines added
26,613
Generated or copied lines excluded from scoring
277,510
Report confidence
medium

What this report can’t tell you

This report is an AI estimate built from the project's git history — every commit and its full code changes were reviewed. It is strong evidence, not surveillance truth: git does not show infrastructure work, code review, meetings, or debugging that ends in a small fix. Low scores are a reason to look closer, never proof on their own.

Contributor identities can include multiple email addresses for one person, owner accounts, and server accounts. They are not necessarily separate developers.