Form Infrastructure & Compatibility

[alaca]

Right now the analytics table has a form_id column but we never fill it in, so all forms on a list get together. This phase gives each form a stable ID, wires it through tracking, and adds a forms table for titles and lifecycle.

What we already have

wp_mailchimp_sf_form_analytics has form_id.
increment_views() / increment_submissions() take $form_id - callers pass nothing.
The block is rendered on server, so a new attribute won't trigger block validation errors.

Form IDs

Block forms: a UUID stored as a form_id block attribute. edit.js generates it on mount when empty (heal every empty block, not just the clicked one).
Shortcode and widget forms, legacy. Aggregate together
No render time fallback. Legacy blocks with no UUID keep writing to the shared bucket until the post is re-saved and gets a UUID.

Forms table wp_mailchimp_sf_forms

form_id PK, list_id, title, post_id, status (active|removed), created_at, updated_at
Title: a form_title block attribute edited in the block sidebar, capped at 50 chars, upserted on save_post.
Removed: on save_post, diff form_ids in content vs table and mark missing ones removed. Re-adding the same UUID flips back to active. Guard against autosaves, revisions, and partial saves.
Copy-paste: copies share the UUID and merge into one form / one title (last save wins). Accepted as known behavior, no regeneration.

Page-source (blog vs footer breakdown): out of scope here, needs separate discussion.

Code changes

block.json: add form_id and form_title attributes.
edit.js: generate UUID on mount when empty. Add the form name sidebar field.
markup.php: output data-form-id and a hidden mailchimp_sf_form_id field from the attribute.
mailchimp_widget.php: same, hardcoded to legacy.
assets/js/mailchimp.js: include form_id from data-form-id in the view ping.
class-mailchimp-analytics-data.php: handle_form_view() reads/validates form_id (^[A-Za-z0-9_-]{1,50}$) and passes to increment_views(), track_submission() passes form_id to increment_submissions().
class-mailchimp-form-submission.php: read the hidden field, validate, do_action( 'mailchimp_sf_form_submission_success', $list_id, $form_id )
New: forms table create/update + the save_post sync/diff.

[iamdharmesh]

Thanks a lot for the research and for sharing it here, @alaca.

A UUID stored as a block attribute is the only thing that's unique, stable, and requires no migration.

I agree with this.

Old block forms that predate this and were never re-saved → b_<post_id>_<n>

How will this be matched or connected when a user edits the post and the form block receives a UUID-based form ID?

There are still quite a few things that need to be finalized:

1. Form name/title

   We need to store a human-readable form title/name so it can be displayed in the Analytics dashboard. Users should be able to edit the form title (perhaps from the block sidebar).

   Since we'll need to query and group analytics data by form and display that data under the corresponding form title, we need to decide how the title should be stored. A dedicated wp_mailchimp_sf_forms table might make sense, as it would provide a clean, queryable, and join-friendly structure for the Analytics dashboard.

2. Copy-paste or duplicate block handling

   Copying and pasting a block into another post (or even within the same post) will copy the same UUID. As a result, analytics for both placements would be merged into a single data series.

   We need to decide whether we want to solve this problem or accept it and document it as a known limitation in the help/docs.

3. Removed blocks

   How should we handle legacy analytics data for a form block that has been removed?

   This could be handled cleanly if forms are stored in a dedicated wp_mailchimp_sf_forms table with a status field. In that case, the form record could remain available for analytics purposes even after the block has been removed.

We may uncover additional questions once implementation begins, but these are the main areas that need to be finalized for now.
cc: @vikrampm1 @builtbytay

[builtbytay]

Chiming in here. @iamdharmesh @alaca
On Dharmesh's points:

1. Yes please on the form title field. We should also consider limiting the characters here, because this will display on the eventual per-form analytics chart.
2. I think we should keep this as a know behavior (not a limitation). reason: Someone might have their forms labeled by which type of page they have it on (ie - blog form, footer form, etc.) and want to aggregate in that way per title. Later on (if possible) when we add the complexity of page-source, this makes the differentiation a lot more useful as well. Happy to discuss this one more if needed!
3. "This could be handled cleanly if forms are stored in a dedicated wp_mailchimp_sf_forms table with a status field. In that case, the form record could remain available for analytics purposes even after the block has been removed." -- I like this approach to maintain the data integrity. Removing a form usually does not mean they want to remove the form data Marketers cherish their data.

[alaca]

Thanks both. Going with a dedicated wp_mailchimp_sf_forms table.

1. Title: agreed. Table-backed, edited from the block sidebar (form_title attribute synced on save_post), capped at 50 chars since it shows on the chart.

Table:
form_id PK, list_id, title, post_id, status (active|removed), created_at, updated_at

2. Copy-paste: good to keep as known behavior, no UUID regeneration (simpler code). One thing to confirm: copies share a UUID, so they always count as one form with one title. Renaming a copy just overwrites that shared title (last save wins), it doesn't split it into a separate form? For the blog and footer distinction you raised, @builtbytay, I think that's a separate page source concept rather than something that we can do with title, we will probably need more discussion on how to approach this.

3. Removed blocks: agreed on the status field. On save_post I'll diff the form_ids in the post against the table and mark missing ones status = removed instead of deleting, so data and title stay. Re-adding the same UUID (undo, paste back, revision restore) flips it back to active. I'll also guard the save_post diff against autosaves, revisions, and partial saves so they don't wrongly flag forms.

On the b_<post_id>_ fallback you flagged @iamdharmesh: good point, that data would never connect to the UUID anyway, so it is simpler to drop it. Legacy blocks that haven't been re-saved keep writing to the shared bucket like today, then switch to their UUID the first time the post is saved.

I'll update the specs.

[iamdharmesh]

Thanks for the input, @builtbytay and @alaca.

Based on the discussion here, I think we have enough information to reach a conclusion. Please review this when you have a chance, and we can discuss them further during today's call.

1. Summary

Give each block-based signup form a stable identity so the existing analytics dashboard can break views/submissions down per form, not just per list. A UUID formId + human readable formTitle are added as persistent block attributes, emitted into the rendered form, threaded through the existing view/submission tracking, and recorded in a new forms registry table that supplies labels and lifecycle (active/removed) to the dashboard. Shortcode and legacy-widget forms are intentionally unchanged and continue to aggregate under a "Legacy (untracked)" series.

2. What already exists

Phase 1 shipped a forward-compatible analytics foundation, so this is a smaller change than it looks. The fact table wp_mailchimp_sf_form_analytics already has a form_id varchar(50) DEFAULT '' column with UNIQUE KEY (list_id, form_id, event_date), and increment_views() / increment_submissions() / track_submission() plus the mailchimp_sf_form_submission_success hook already exist (currently $list_id-only). What's missing is everything that populates form_id: the block doesn't emit it, the view/submission handlers ignore it, there's no forms registry, and the dashboard (fetch_rows/aggregate in class-mailchimp-form-performance.php) is list-level only with no per-form path. This brief fills that gap.

3. Scope

In scope

• formId (UUID) + formTitle block attributes; editor backfill + optional title prompt/Inspector field. (Provided sample UI but @builtbytay can help confirming it)

Sample inline notice	sample settings in sidebar

• Emit data-form-id + hidden mailchimp_sf_form_id in block render only.
• Thread form_id through view JS → handle_form_view() → increment_views, and through submission POST → success hook → track_submission → increment_submissions.
• New wp_mailchimp_sf_forms registry table + migration; upsert + soft-removal reconcile on host save/trash.
• Dashboard per-form filter, per-form chart series, removed-form handling, title fallback chain.

Out of scope

• Shortcode (mailchimp_sf_signup_form()) and legacy widget rendering/tracking — unchanged; stay in the form_id = '' "Legacy (untracked)" bucket.
• Auto-migrating existing published content. Existing forms get a formId only when their host is next opened/saved in the editor, and start per-form tracking from that point — no migration sweep.
• Changing the analytics fact table schema (already form-aware).
• Cron/full-site scans (save-time reconcile only).
• Auto-deduping copy-paste collisions (accept it and document it)

4. Proposed change

4.1 Block attributes + editor (block.json, edit.js)

• Add to block.json: "formId": { "type": "string", "default": "" }, "formTitle": { "type": "string", "default": "" }.
• edit.js on mount: if formId empty → generate UUID v4 → setAttributes({ formId }) (silent; persists next save).
• If formId present but formTitle empty → non-blocking inline notice (pre-fill from header); Save title sets the attribute + upserts the registry row; Skip dismisses for the session. Title is never required to publish (Use fallback if not provided).
• Inspector "Form identity" panel with an editable TextControl for formTitle (rename anytime).

4.2 Write path — markup & client

• markup.php (block only): add data-form-id="{formId}" to <form> and a hidden <input name="mailchimp_sf_form_id" value="{formId}">.
  • ⚠ Name collision: markup.php:29 (and the legacy widget) already define $form_id = wp_unique_id( $list_id . '_' ) — an ephemeral per-request DOM id for the <form id> / #mc_signup_* anchor, unrelated to analytics. Thread the new attribute through a distinct variable (e.g. $mc_analytics_form_id); do not overwrite the DOM id.
  • If formId is empty (not yet saved) → omit both attributes → tracked as legacy form_id=''.
• assets/js/mailchimp.js: read data-form-id; append form_id to the analytics FormData; dedupe key becomes listId|formId (forms without data-form-id keep list-only dedupe).

4.3 Write path — server handlers

• handle_form_view(): read form_id from POST, validate it (see below), then increment_views( $list_id, $form_id ).
• Submission: handle_form_submission() reads $_POST['mailchimp_sf_form_id']; append the arg to the hook → do_action( 'mailchimp_sf_form_submission_success', $list_id, $form_id ) (additive — existing 1-arg listeners keep working); update track_submission( $list_id, $form_id = '' ) → increment_submissions( $list_id, $form_id ).
• form_id validation (new): add a guard mirroring is_valid_list_id() — reject anything that isn't a well-formed UUID and cap at 50 chars — so malformed/junk IDs can't pollute the analytics or registry tables. Invalid → treat as '' (legacy bucket), don't error.

4.4 Registry table + migration

New table following the analytics-table pattern, with a mailchimp_sf_forms_db_version option and a create_table() invoked on plugins_loaded.

CREATE TABLE {$prefix}mailchimp_sf_forms (
    form_id     varchar(50)  NOT NULL,
    title       varchar(255) NOT NULL DEFAULT '',   -- from block formTitle (use value of 'header' attribute if skipped/unset)
    host_type   varchar(20)  NOT NULL DEFAULT '',   -- post, page, wp_template, wp_template_part, wp_block, widget
    host_id     bigint(20)   unsigned NOT NULL DEFAULT 0,
    status      varchar(10)  NOT NULL DEFAULT 'active',  -- active | removed
    removed_at  datetime     NULL DEFAULT NULL,
    updated_at  datetime     NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    PRIMARY KEY (form_id),
    KEY host_lookup (host_type, host_id, status)
) {$charset_collate};

Note: Table schema is for the information purpose only to provide idea, final schema need to be finalized at the time of implementation.
*host_type column exists because widget is not a post type, and host_id alone wouldn't be enough to identify it.

Open question for discussion:

• How to handle the same formId in multiple posts (copy-paste to another post)?

4.5 Upsert + reconcile on save

On host save: parse_blocks() + recursive walk → extract all mailchimp/mailchimp formIds → upsert each (title, header snapshot, host metadata, status=active, removed_at=NULL); for registry rows matching this host whose form_id is absent from the extracted set → status=removed, removed_at=NOW(). Rows are never hard-deleted (historical analytics labels must keep resolving).

• Hooks: save_post (posts, pages, wp_template, wp_template_part, wp_block), wp_trash_post, plus the block-widget-area save hook (the exact widget hook is a build-time call).
• Reconcile runs only on the storage host. A consumer page embedding a synced pattern via core/block does not inline the inner formId, so it won't falsely reconcile it away — verify in tests.

4.6 Read path / dashboard (class-mailchimp-form-performance.php)

• New get_by_list(): derive the form set for the selected list(s) and date range from the analytics table (distinct form_id where list_id matches and rows fall in range), then join the registry for labels.
• Per-form mode: GROUP BY form_id → series map (one chart line per form). Default (no form selected) = current list-level aggregate — the Phase-1 view is unchanged.
• Removed forms: included in the filter when they have rows in the selected range; labelled "… (removed)".
• Title fallback chain (per label/series): (1) registry title (2) host post/page title (posts table via host_id) + list name, e.g. "About Us · Newsletter"; (4) truncated form_id, e.g. "Form a3f2…8b1c".
• (Optional) Location hint: derived at query time from host_type + host post title, e.g. "Site Footer · Template part".
• Shortcode/legacy forms aggregate under the virtual "Legacy (untracked)" series (form_id='').

4.7 Collision handling — documented limitation

Copy-pasting a block to another post copies the same formId; analytics for both placements merge into one series. This is accepted as a known limitation, documented in help/docs — no editor-side paste handling or server-side dedupe. (Template/synced-pattern sharing of one formId across many renders is intended aggregation, not a collision.)

4.8 Legacy shortcode / widget — unchanged

No data-form-id, no hidden input; tracking stays form_id=''. The existing shortcode→block transform (includes/blocks/mailchimp/index.js) remains the optional upgrade path.

5. Acceptance criteria

AC	How it's met
Each saved block form has a stable formId surviving edits/updates	UUID block attribute, backfilled in edit.js, serialized in the block delimiter.
Views/submissions attribute to the right form	data-form-id/hidden input → handlers → increment_*( $list_id, $form_id ).
Dashboard breaks down per form; default view unchanged	New per-form series + filter; list-level default preserved.
Removed forms keep historical data and remain selectable	Soft status=removed; analytics rows never deleted; fallback labels.
Shortcode/legacy unaffected	No render/tracking changes; aggregate as Legacy.
Existing 1-arg submission listeners keep working	Hook arg is additive.
Malformed form_id can't pollute analytics	UUID/length guard in handlers.

6. Test plan

1. New block → formId minted on first save; survives edit + page update unchanged.
2. View + submit a block form → analytics rows carry the correct form_id; dashboard shows a per-form series.
3. Two forms, same list, one page → view dedupe keyed listId|formId records both correctly.
4. Legacy block (no formId) before save → tracked as form_id=''; after editor save → real UUID tracking begins.
5. Shortcode + legacy widget → still form_id='', appear only under "Legacy (untracked)".
6. Remove a block + save host → registry row removed; historical chart still resolves its label; form still selectable in range.
7. Trash host → its rows reconcile to removed.
8. Form in footer template part / synced pattern → one aggregated formId across renders; consumer pages do not reconcile it away.
9. Copy-paste duplicate → both placements merge into one series (documented behaviour).
10. Title fallback: title set (fallback header) → host title+list → truncated id, in order, as each prior source is emptied.
11. Migration: fresh install + upgrade both create wp_mailchimp_sf_forms; mailchimp_sf_forms_db_version gates re-creation.
12. form_id validation rejects over-length / non-UUID junk (falls back to legacy bucket).

7. Risks & considerations

• Reconcile on every save_post parses block content; fine, but confirm cost on very large/nested documents.
• Widget-area save hook is a build-time decision: validate which of update_option on widget_block / sidebars_widgets and/or the REST widget save endpoint fires reliably for block-widget areas, and reconcile there.
• No auto-migration: existing forms aggregate as Legacy until their host is re-saved — set expectations in docs.

[alaca]

@iamdharmesh on your open question (same formId in multiple posts): I think the answer is to introduce another table for placements, instead of carrying a single host_id on the registry row.

The problem with a single host_id is that a form can legitimately live on multiple pages (footer, template part, synced pattern, or deliberate copy-paste), and we want to support that as one aggregated form. With one host_id, the save time reconcile can mark a live form as removed depending on save order. Delete it from one container and save, and reconcile flips status while it's still live elsewhere.

So I'd split placements out:

wp_mailchimp_sf_forms            
  form_id PK, list_id, title, status (active|removed), created_at, updated_at

wp_mailchimp_sf_form_placements   
  form_id, container_type, container_id   (unique)
  -- container_type: post | page | wp_template | wp_template_part | wp_block | widget

Status becomes derived, not stored per container, and active if the form has at least one placement, removed if zero. On save we reconcile only the current container (add placements present, drop ones absent, recompute status). Synced patterns and template parts land as a single placement and render everywhere. C/P lands as one placement per post. Both aggregate to one form, and status stays correct.

[iamdharmesh]

Thanks @alaca, the design looks good to me. I have two thoughts:

Title overwrite for shared form IDs

This is really the same limitation as duplicate form_ids. If the same form_id exists in multiple posts with different titles, the stored title will always reflect whichever post was saved last.

Since this only happens when a block is copied/pasted or a post is duplicated, I'd document it as part of the same limitation rather than treating it separately. Something like: "Duplicated forms share the same record, so they also share the same title. The most recently saved title wins."

Thinking out loud before we build the placements table

The placements table seems to exist only to track form status (active/removed). Since analytics are aggregated by form_id only, I'm wondering whether we actually need to track status at all in this phase.

• If not, we could simplify things by dropping the status field and the placements table entirely, and just store:

  (form_id PK, title, created_at, updated_at)

  We'd then only insert/update records on save_post, which removes the need for deletion tracking, cleanup logic, and status recalculation.

• If we do need active/removed status, then the placements table makes sense because it's the only reliable way to track whether a form is still in use.

Just wanted to discuss with you and @builtbytay: if we're okay dropping status and container information (the post ID), we could move forward with the simpler approach.

Either way, I think the title-overwrite behavior is a known limitation that we should document.

[alaca]

Agreed on both, @iamdharmesh.

Title overwrite: yes, it is the same root cause as the shared form_id, so let's document it together as known behavior, not a separate item. Your wording works: "Duplicated forms share the same record, so they also share the same title. The most recently saved title wins."

Dropping status + placements: okay. Status only ever drove a cosmetic "(removed)" badge, it doesn't touch the numbers or the data. And dropping it actually makes "same form on multiple pages" simpler, since there's no save order reconcile to get wrong, the form just aggregates on form_id. So for this phase:

wp_mailchimp_sf_forms
  form_id PK, title, created_at, updated_at

Upsert on save_post, never delete. No status, no placements

[builtbytay]

@iamdharmesh @alaca I don't see a need for a status so I definitely agree with no status. Placements, I agree can stay out of scope for this, but I want to make sure that in the future, we're able to expand to include this so that we can add another chart (hypothetically) that breaks down individual form performance by page (slug/placement).

I also want to note that the shared records of duplicate forms with the same ID is intentional from my end as well. That's how the analytics data is displayed. The only thing we need it separated for is what I mentioned above (future work).