Skip to main content

AI Agent Summaries & Usage Artifacts

Every AI Agent session produces two artifacts, stored against the parent room:

ArtifactType valuePurpose
AI_AGENT_SESSION_SUMMARY11A structured JSON report of the session: what was learned, what was practised, quiz results, workspace activity, and the continuation context for the next session.
AI_AGENT_USAGE10Token and audio usage, broken down by role and modality, with estimated costs.

This page describes both, with the exact field names used in plugnmeet_room_artifacts.proto.

Under Active Development

The AI Agent engine and its artifact formats are under active development. Field names and fields that are not yet listed may change before general release. One-click UI integration for Moodle, WordPress, and Joomla plugins is planned once the feature is finalized and released.

How You Receive Artifacts​

  1. The session ends, and the server writes both artifacts to the parent room.
  2. Your server receives an artifact_created webhook for each artifact.
  3. Your server retrieves the JSON with /auth/artifact/fetch, filtering by type (AI_AGENT_SESSION_SUMMARY or AI_AGENT_USAGE).
Use the Webhook as the Trigger

Do not poll for artifacts. Listen for artifact_created, check room_artifact.type, and then fetch only the artifact you need.

AI_AGENT_SESSION_SUMMARY​

The summary file is JSON, generated with the proto field names (snake_case). Empty strings, empty lists, and zero values are included, so every field is always present. Optional fields are omitted when they are not set.

Top-Level Fields​

FieldJSON typeDescription
summary_sourcestringllm when a text model refined the summary. local_fallback when the summary was built from session telemetry only.
generated_atstringISO-8601 timestamp of when the summary was created.
user_infoobjectThe learner or visitor for this session. Always present. See user_info.
sessionobjectSession facts: times, duration, end reason, topic, and counts.
evaluationobjectMastery and participation assessment.
quiz_evaluationobjectQuiz and poll performance.
progress_logarrayConcept-by-concept status over the session.
workspace_activityobjectWhich tools and workspaces were used.
continuationobjectContext and focus for the next session.
narrative_summarystringA short, human-readable paragraph for teachers, sales reps, or CRM notes.
usagemapToken usage per role: session, delegation, and summary. See usage.
Schema Version

The summary does not include a schema_version field. Detect changes by the field names in the table above, and by the artifact's metadata in the artifact record.

user_info​

FieldJSON typeDescription
user_idstringThe internal participant ID used in the session.
namestringDisplay name, if known. Optional.
ex_user_idstringYour system's user ID, passed when the participant joined. Use this to match records. Optional.
extra_datastringThe participant's custom metadata, as a JSON object string. Optional.

session​

FieldJSON typeDescription
room_idstringThe parent room ID.
room_sidstringThe LiveKit room SID for the parent room.
session_idstringUnique ID of this AI session.
agent_display_namestringName the agent used, for example Math Tutor.
topicstringThe configured topic.
allowed_subtopicsstring[]The configured subtopics.
started_atstring (int64)Start time, Unix milliseconds. Protobuf encodes 64-bit integers as JSON strings.
ended_atstring (int64)End time, Unix milliseconds.
duration_secondsstring (int64)Session length in seconds.
end_reasonstringWhy the session ended, such as host_stopped, empty_room, or budget.
off_topic_redirects_countnumberCumulative count of off-topic redirects for the session.
turns_countnumberNumber of conversational turns.

evaluation​

FieldJSON typeDescription
overall_scorenumberAn overall score for the session, as set by the evaluation step.
mastery_levelstringA short descriptor, such as developing, proficient, or mastered.
participation_levelstringA short descriptor of how actively the participant engaged.
concepts_masteredstring[]Concepts the participant demonstrated.
concepts_practicedstring[]Concepts covered but not yet secure.
struggled_withstring[]Concepts the participant found difficult.
vocabulary_or_key_termsstring[]Key terms or vocabulary used in the session.
rubric_notesstringExplanation of how the evaluation was reached.

The mastery_level and participation_level values are free text generated by the summary step. Build filters on overall_score and the concept arrays, not on exact label strings.

quiz_evaluation​

FieldJSON typeDescription
total_quizzesnumberNumber of quizzes or polls with correct answers in the session.
total_responsesnumberNumber of answers submitted.
correct_responsesnumberNumber of correct answers.
accuracy_percentnumbercorrect_responses / total_responses as a percentage.
questionsarrayOne entry per question.

Each entry in questions has:

FieldJSON typeDescription
poll_idstringThe poll or quiz ID.
questionstringThe question text.
correct_optionstringThe correct answer text.
total_votesstring (uint64)Total votes received.
correct_votesstring (uint64)Votes for the correct answer.
passedbooleanWhether the question met the passing rule for the quiz.

progress_log​

Each entry describes one concept:

FieldJSON typeDescription
conceptstringThe concept name.
statusstringThe status at the end of the session, such as practiced or mastered.
notesstringOptional note about the concept.

workspace_activity​

FieldJSON typeDescription
tools_usedmap<string, number>How many times each tool was called, for example whiteboard_draw_diagram: 3.
whiteboard_usedbooleanWhether the whiteboard was used.
notepad_usedbooleanWhether the shared notepad was used.

continuation​

FieldJSON typeDescription
next_session_focusstringRecommended focus for the next session.
recommended_warm_upstringA short opening activity for the next session.
unfinished_subtopicsstring[]Subtopics not yet covered.
previous_session_context_promptstringA ready-made context prompt. Pass it to the next session as previous_session_context.

Usage by Role​

The usage map has one entry per role:

KeyCovers
sessionThe realtime voice model: the live conversation, including audio.
delegationThe text model that handles tool calls and delegated turns. Present for providers that use a separate delegation model.
summaryThe text model that wrote the end-of-session summary.

Each entry is a token usage object, described in the next section.

Example Summary​

The example below is a realistic summary for a 20-minute study session. Values are illustrative.

{
"summary_source": "llm",
"generated_at": "2026-03-14T10:42:19Z",
"user_info": {
"user_id": "pnm_user-7f3a",
"name": "Ana Rivera",
"ex_user_id": "student-204",
"extra_data": "{\"class\":\"Grade 9B\"}"
},
"session": {
"room_id": "study-room-spanish",
"room_sid": "RM_kR8w2sXq1ZbA",
"session_id": "ai-7c1e9b40",
"agent_display_name": "Lucía",
"topic": "Everyday Spanish conversation",
"allowed_subtopics": ["greetings", "ordering food", "travel vocabulary"],
"started_at": "1773484800000",
"ended_at": "1773486000000",
"duration_seconds": "1200",
"end_reason": "host_stopped",
"off_topic_redirects_count": 1,
"turns_count": 34
},
"evaluation": {
"overall_score": 72,
"mastery_level": "developing",
"participation_level": "high",
"concepts_mastered": ["greetings", "ordering with 'quisiera'"],
"concepts_practiced": ["travel vocabulary"],
"struggled_with": ["subjunctive after 'quiero que'"],
"vocabulary_or_key_terms": ["la cuenta", "el menú", "por favor"],
"rubric_notes": "Confident in greetings and food vocabulary. Needs more practice with the subjunctive."
},
"quiz_evaluation": {
"total_quizzes": 2,
"total_responses": 6,
"correct_responses": 4,
"accuracy_percent": 67,
"questions": [
{
"poll_id": "poll-1a2b",
"question": "Which phrase politely asks for the bill?",
"correct_option": "La cuenta, por favor",
"total_votes": "1",
"correct_votes": "1",
"passed": true
},
{
"poll_id": "poll-3c4d",
"question": "Choose the correct form: Quiero que tú ___ conmigo.",
"correct_option": "vengas",
"total_votes": "1",
"correct_votes": "0",
"passed": false
}
]
},
"progress_log": [
{ "concept": "greetings", "status": "mastered", "notes": "Used formal and informal forms correctly." },
{ "concept": "subjunctive after 'quiero que'", "status": "struggling", "notes": "Needed two corrections." }
],
"workspace_activity": {
"tools_used": {
"whiteboard_draw_shapes": 2,
"whiteboard_write_content": 5,
"notepad_write": 3,
"create_quiz": 2
},
"whiteboard_used": true,
"notepad_used": true
},
"continuation": {
"next_session_focus": "Subjunctive after expressions of desire",
"recommended_warm_up": "Order a meal using a role-play with three menu items",
"unfinished_subtopics": ["travel vocabulary"],
"previous_session_context_prompt": "Previous session (Everyday Spanish): mastered greetings and ordering; struggled with the subjunctive after 'quiero que'. Start with a short warm-up, then practise the subjunctive."
},
"narrative_summary": "Ana practised greetings and ordering food with confidence. She answered most quiz questions correctly, but the subjunctive after 'quiero que' needs more practice. Start the next session with a meal role-play, then focus on the subjunctive.",
"usage": {
"session": {
"prompt_tokens": 18420,
"completion_tokens": 3110,
"total_tokens": 21530,
"breakdown": {
"session_prompt_tokens": "18420",
"session_audio_prompt_tokens": "9800",
"session_completion_tokens": "3110",
"session_audio_completion_tokens": "2600",
"audio_millis": "1200000"
}
},
"delegation": {
"prompt_tokens": 2950,
"completion_tokens": 640,
"total_tokens": 3590,
"breakdown": {
"delegation_prompt_tokens": "2950",
"delegation_completion_tokens": "640"
}
},
"summary": {
"prompt_tokens": 4100,
"completion_tokens": 520,
"total_tokens": 4620,
"breakdown": {
"summary_prompt_tokens": "4100",
"summary_completion_tokens": "520"
}
}
}
}

Breakdown values are strings in the JSON because protobuf encodes int64 as strings. Parse them as integers in your code.

Field Names in the Example

The example uses the field names listed in this page. Some fields, such as the breakdown keys inside usage, can differ between providers. Read the values you need, and ignore keys you do not recognise.

AI_AGENT_USAGE​

The usage artifact is stored as RoomArtifactMetadata with a token_usage payload. Its metadata fields are:

Metadata fieldDescription
token_usage.prompt_tokensTotal prompt tokens across all roles.
token_usage.completion_tokensTotal completion tokens across all roles.
token_usage.total_tokensPrompt plus completion tokens.
token_usage.breakdownPer-role and per-modality counts, described below.
token_usage.prompt_tokens_estimated_costEstimated prompt cost. Optional. Omitted for BYOK sessions.
token_usage.completion_tokens_estimated_costEstimated completion cost. Optional. Omitted for BYOK sessions.
token_usage.total_tokens_estimated_costEstimated total cost. Optional. Omitted for BYOK sessions.
reference_artifact_idThe artifact ID of the matching AI_AGENT_SESSION_SUMMARY. Use it to link the two.
user_infoThe same learner identity as the summary. Included when the summary has one.
occurred_in_breakout_roomtrue when the session ran in a breakout room.
source_room_table_idThe database table ID of the room where the session ran.

Breakdown Keys​

The breakdown map uses a prefix for each role: session_, delegation_, or summary_. For example, session_prompt_tokens and delegation_prompt_tokens.

Each role can report these keys, when the value is greater than zero:

Key suffixMeaning
_prompt_tokensInput tokens.
_completion_tokensOutput tokens.
_cached_prompt_tokensInput tokens served from cache.
_cache_write_prompt_tokensInput tokens written to cache.
_text_prompt_tokensText input tokens.
_audio_prompt_tokensAudio input tokens.
_image_prompt_tokensImage input tokens.
_cached_text_prompt_tokens, _cached_audio_prompt_tokens, _cached_image_prompt_tokensCached input, per modality.
_cache_write_text_prompt_tokens, _cache_write_audio_prompt_tokens, _cache_write_image_prompt_tokensCache writes, per modality.
_text_completion_tokensText output tokens.
_audio_completion_tokensAudio output tokens.
_thinking_completion_tokensReasoning output tokens, when the model reports them.

Session-wide keys:

KeyMeaning
audio_millisSpeech duration in milliseconds, used for audio-priced and minute-priced sessions.
summary_prompt_tokensInput tokens used to write the summary.
summary_completion_tokensOutput tokens used to write the summary.
custom_provider_<provider>Set to 1 for BYOK sessions. For example, custom_provider_openai.

Per-Modality Cost Accounting​

Costs are estimated from the pricing table in your server's config.yaml. Each modality has its own rate:

  • Text input and output use the text rates.
  • Audio input and output use the audio rates, which are usually higher.
  • Image input uses the image rate.
  • Cached input uses the cached rate when set. Without a cached rate, cached tokens are priced as uncached input.
  • Cache writes use the cache-write rate when set.
  • Minute-priced live speech (OpenAI Live) is estimated from audio_millis and the price_per_minute rate.

The estimate is an accounting aid. It does not replace your provider's invoice. Keep your pricing values in line with your provider contract.

Parent-Room Attribution​

Both artifacts are stored under the parent room, even when the session ran in a 1-on-1 breakout room. Use these fields to tell them apart:

  • occurred_in_breakout_room: true identifies a session that ran in a breakout room.
  • source_room_table_id identifies the room where the session was served.
  • reference_artifact_id links usage to its summary.

For example, a school can report usage per class by grouping on the parent room ID, and still see which sessions were per-student breakouts.

BYOK Billing Suppression​

When a room uses its own provider through custom_provider (BYOK), the server treats the session as outside platform billing:

  • The three *_estimated_cost fields are not included in the usage artifact.
  • The breakdown includes custom_provider_<provider>: 1. Your billing or webhook code can read this flag to skip platform charges.
  • Token counts are still recorded, so you can track usage even when you pay the provider directly.

Use this flag in your billing logic:

const usage = artifact.metadata.token_usage;
const isBYOK = Object.keys(usage.breakdown || {}).some(k => k.startsWith('custom_provider_'));

if (!isBYOK) {
// Charge the platform cost for this room, using usage.total_tokens_estimated_cost
}

Using Artifacts in Your Systems​

GoalFields to use
Show a teacher a student reportevaluation, quiz_evaluation, continuation, narrative_summary
Feed a CRMuser_info.ex_user_id, session.duration_seconds, quiz_evaluation.accuracy_percent, narrative_summary
Continue the next sessioncontinuation.previous_session_context_prompt
Track platform costAI_AGENT_USAGE, total_tokens_estimated_cost, and the absence of custom_provider_*
Find which tools were usedworkspace_activity.tools_used

For the continuity flow, see Multi-Session Continuity. For configuring prices, see Pricing Tables.

Privacy Notes​

  • The summary contains learner and visitor information, and quiz answers. Treat it as personal data under your privacy policy.
  • The summary is an evaluation, not a transcript. Transcripts are a separate artifact type.
  • Your server controls how long artifacts are kept.

See the Security Overview for how AI service artifacts are stored and retained.