in lots of instances, it is advisable to examine what is occurring with KPIs. It is whether or not the dashboard is responding to abnormalities or whether or not the numbers are up to date each day. Primarily based on my years of expertise as a KPI analyst, I estimate that over 80% of those duties are pretty commonplace and could be solved just by following a easy guidelines.
It is a high-level plan to research KPI modifications (For extra info, please see the article “An irregular root trigger evaluation 101”):
- Estimate modifications within the topline of a metric To know the magnitude of the shift.
- Test the standard of your information Be certain the numbers are correct and dependable.
- Accumulate the context About inner and exterior occasions that will have influenced the change.
- Slice metrics and cube Establish which segments contribute to metric shifts.
- Combine your findings in an government abstract It consists of hypotheses and estimates of the influence on key KPIs.
With a transparent plan to carry out, such duties might be automated utilizing AI brokers. The code brokers we mentioned lately could also be appropriate for that as they assist us effectively analyze information earlier than and after minimal capability to jot down and execute code. So let’s construct such an agent utilizing the Huggingface Smolagents framework.
Whereas engaged on the duty, we’ll clarify extra superior options of the Smolagents framework.
- A way for fine-tuning any sort of immediate to make sure the specified habits.
- We are going to construct a multi-agent system that explains KPI modifications and hyperlinks them to the basis trigger.
- Add reflections to the circulate utilizing the Supplementary Plan step.
MVP to clarify KPI modifications
As at all times, we’ll take an iterative method and begin with a easy MVP, specializing in the slicing and dicing steps of the evaluation. Analyze modifications in easy metrics (revenues) divided by one dimension (nation). Use the dataset from the earlier article, “What does KPI change imply?”
Load the info first.
raw_df = pd.read_csv('absolute_metrics_example.csv', sep = 't')
df = raw_df.groupby('nation')[['revenue_before', 'revenue_after_scenario_2']].sum()
.sort_values('revenue_before', ascending = False).rename(
columns = {'revenue_after_scenario_2': 'after',
'revenue_before': 'earlier than'})Subsequent, let’s initialize the mannequin. For a easy activity, I selected Openai GPT-4O-Mini. Nonetheless, the Smolagents framework support Because it’s every kind of fashions, you should use the mannequin you want. Subsequent, it is advisable to create an agent to supply duties and datasets.
from smolagents import CodeAgent, LiteLLMModel
mannequin = LiteLLMModel(model_id="openai/gpt-4o-mini",
api_key=config['OPENAI_API_KEY'])
agent = CodeAgent(
mannequin=mannequin, instruments=[], max_steps=10,
additional_authorized_imports=["pandas", "numpy", "matplotlib.*",
"plotly.*"], verbosity_level=1
)
activity = """
Here's a dataframe exhibiting income by phase, evaluating values
earlier than and after.
May you please assist me perceive the modifications? Particularly:
1. Estimate how the entire income and the income for every phase
have modified, each in absolute phrases and as a share.
2. Calculate the contribution of every phase to the entire
change in income.
Please spherical all floating-point numbers within the output
to 2 decimal locations.
"""
agent.run(
activity,
additional_args={"information": df},
)The agent returned a fairly believable consequence. We obtained detailed statistics on the modifications in metrics for every phase and their influence on topline KPIs.
{'total_before': 1731985.21, 'total_after':
1599065.55, 'total_change': -132919.66, 'segment_changes':
{'absolute_change': {'different': 4233.09, 'UK': -4376.25, 'France':
-132847.57, 'Germany': -690.99, 'Italy': 979.15, 'Spain':
-217.09}, 'percentage_change': {'different': 0.67, 'UK': -0.91,
'France': -55.19, 'Germany': -0.43, 'Italy': 0.81, 'Spain':
-0.23}, 'contribution_to_change': {'different': -3.18, 'UK': 3.29,
'France': 99.95, 'Germany': 0.52, 'Italy': -0.74, 'Spain': 0.16}}}Let us take a look at the code generated by the agent. It is wonderful, however there’s one potential drawback. LLM recreated the dataframe based mostly on the enter information fairly than instantly referring to the enter information. This method isn’t splendid as it may result in elevated errors and token utilization (particularly when working with giant information units).
import pandas as pd
# Creating the DataFrame from the supplied information
information = {
'earlier than': [632767.39, 481409.27, 240704.63, 160469.75,
120352.31, 96281.86],
'after': [637000.48, 477033.02, 107857.06, 159778.76,
121331.46, 96064.77]
}
index = ['other', 'UK', 'France', 'Germany', 'Italy', 'Spain']
df = pd.DataFrame(information, index=index)
# Calculating complete income earlier than and after
total_before = df['before'].sum()
total_after = df['after'].sum()
# Calculating absolute and share change for every phase
df['absolute_change'] = df['after'] - df['before']
df['percentage_change'] = (df['absolute_change'] /
df['before']) * 100
# Calculating complete income change
total_change = total_after - total_before
# Calculating contribution of every phase to the entire change
df['contribution_to_change'] = (df['absolute_change'] /
total_change) * 100
# Rounding outcomes
df = df.spherical(2)
# Printing the calculated outcomes
print("Whole income earlier than:", total_before)
print("Whole income after:", total_after)
print("Whole change in income:", total_change)
print(df)It’s price fixing this challenge earlier than continuing to constructing extra complicated techniques.
Alter the immediate
LLM solely follows the directions given to it, so tweaking the immediate addresses this challenge.
Initially I attempted to make the duty immediate extra clear and clearly inform LLM to make use of the variables supplied.
activity = """Here's a dataframe exhibiting income by phase, evaluating
values earlier than and after. The information is saved in df variable.
Please, use it and do not attempt to parse the info your self.
May you please assist me perceive the modifications?
Particularly:
1. Estimate how the entire income and the income for every phase
have modified, each in absolute phrases and as a share.
2. Calculate the contribution of every phase to the entire change in income.
Please spherical all floating-point numbers within the output to 2 decimal locations.
"""It did not work. So the subsequent step is to have a look at the system immediate and see why it really works like this.
print(agent.prompt_templates['system_prompt'])
#...
# Listed here are the foundations it's best to at all times observe to resolve your activity:
# 1. All the time present a 'Thought:' sequence, and a 'Code:n```py' sequence ending with '```<end_code>' sequence, else you'll fail.
# 2. Use solely variables that you've outlined.
# 3. All the time use the appropriate arguments for the instruments. DO NOT cross the arguments as a dict as in 'reply = wiki({'question': "What's the place the place James Bond lives?"})', however use the arguments instantly as in 'reply = wiki(question="What's the place the place James Bond lives?")'.
# 4. Take care to not chain too many sequential device calls in the identical code block, particularly when the output format is unpredictable. As an illustration, a name to look has an unpredictable return format, so should not have one other device name that relies on its output in the identical block: fairly output outcomes with print() to make use of them within the subsequent block.
# 5. Name a device solely when wanted, and by no means re-do a device name that you simply beforehand did with the very same parameters.
# 6. Do not title any new variable with the identical title as a device: as an illustration do not title a variable 'final_answer'.
# 7. By no means create any notional variables in our code, as having these in your logs will derail you from the true variables.
# 8. You should use imports in your code, however solely from the next checklist of modules: ['collections', 'datetime', 'itertools', 'math', 'numpy', 'pandas', 'queue', 'random', 're', 'stat', 'statistics', 'time', 'unicodedata']
# 9. The state persists between code executions: so if in a single step you've got created variables or imported modules, these will all persist.
# 10. Do not hand over! You are in command of fixing the duty, not offering instructions to resolve it.
# Now Start!On the finish of the immediate, there’s an instruction "# 2. Use solely variables that you've outlined!". This may be interpreted as a strict rule of not utilizing different variables. So I modified it "# 2. Use solely variables that you've outlined or ones supplied in extra arguments! By no means attempt to copy and parse extra arguments."
modified_system_prompt = agent.prompt_templates['system_prompt']
.change(
'2. Use solely variables that you've outlined!',
'2. Use solely variables that you've outlined or ones supplied in extra arguments! By no means attempt to copy and parse extra arguments.'
)
agent.prompt_templates['system_prompt'] = modified_system_promptThis variation alone did not assist. Subsequent, I seemed up the duty messages.
╭─────────────────────────── New run ────────────────────────────╮
│ │
│ Here's a pandas dataframe exhibiting income by phase, │
│ evaluating values earlier than and after. │
│ May you please assist me perceive the modifications? │
│ Particularly: │
│ 1. Estimate how the entire income and the income for every │
│ phase have modified, each in absolute phrases and as a │
│ share. │
│ 2. Calculate the contribution of every phase to the entire │
│ change in income. │
│ │
│ Please spherical all floating-point numbers within the output to 2 │
│ decimal locations. │
│ │
│ You could have been supplied with these extra arguments, that │
│ you may entry utilizing the keys as variables in your python │
│ code: │
│ {'df': earlier than after │
│ nation │
│ different 632767.39 637000.48 │
│ UK 481409.27 477033.02 │
│ France 240704.63 107857.06 │
│ Germany 160469.75 159778.76 │
│ Italy 120352.31 121331.46 │
│ Spain 96281.86 96064.77}. │
│ │
╰─ LiteLLMModel - openai/gpt-4o-mini ────────────────────────────╯There are directions associated to utilizing extra arguments "You could have been supplied with these extra arguments, you can entry utilizing the keys as variables in your python code". We are able to make it extra particular and clear. Sadly, this parameter isn’t uncovered externally so I needed to discover it in Source code. To seek out the trail to a Python package deal, you should use the next code:
import smolagents
print(smolagents.__path__)Then I discovered it brokers.py I filed this line and altered it to incorporate extra particular directions.
self.activity += f"""
You could have been supplied with these extra arguments out there as variables
with names {",".be part of(additional_args.keys())}. You may entry them instantly.
Here's what they include (only for informational functions):
{str(additional_args)}."""It was a little bit of a hack, however that is what occurs within the LLM framework sometimes. Remember to reload the package deal later, and we’re OK to go. Check it to see if it really works now.
activity = """
Here's a pandas dataframe exhibiting income by phase, evaluating values
earlier than and after.
Your activity will likely be perceive the modifications to the income (after vs earlier than)
in numerous segments and supply government abstract.
Please, observe the next steps:
1. Estimate how the entire income and the income for every phase
have modified, each in absolute phrases and as a share.
2. Calculate the contribution of every phase to the entire change
in income.
Spherical all floating-point numbers within the output to 2 decimal locations.
"""
agent.logger.stage = 1 # Decrease verbosity stage
agent.run(
activity,
additional_args={"df": df},
)Hooray! The issue has been fastened. Agent not copies enter variables and references df As a substitute, a direct variable. That is the newly generated code.
import pandas as pd
# Calculate complete income earlier than and after
total_before = df['before'].sum()
total_after = df['after'].sum()
total_change = total_after - total_before
percentage_change_total = (total_change / total_before * 100)
if total_before != 0 else 0
# Spherical values
total_before = spherical(total_before, 2)
total_after = spherical(total_after, 2)
total_change = spherical(total_change, 2)
percentage_change_total = spherical(percentage_change_total, 2)
# Show outcomes
print(f"Whole Income Earlier than: {total_before}")
print(f"Whole Income After: {total_after}")
print(f"Whole Change: {total_change}")
print(f"Proportion Change: {percentage_change_total}%")Now we’re prepared to maneuver on to constructing an actual agent that solves our duties.
AI Agent for KPI Story
Lastly, it is time to work on AI brokers that designate KPI modifications and allow you to create an government abstract.
Our brokers observe this plan for root trigger evaluation.
- Estimate modifications to topline KPIs.
- To know which segments drive the shift, we slice and psycho the metrics.
- Seek for occasions within the change log to see if you happen to can clarify the metric modifications.
- Consolidate all of the findings of a complete government abstract.
After a whole lot of experiments and a few tweaks, I’ve arrived at promising outcomes. Right here is the vital adjustment I made (extra on it later):
- I’ve leveraged Multi-agent setup A changelog agent that lets you entry the changelog and assist clarify KPI modifications by including one other crew member.
- I did an experiment A extra highly effective mannequin Like
gpt-4oandgpt-4.1-miniSincegpt-4o-miniIt wasn’t sufficient. Utilizing a extra highly effective mannequin not solely improved outcomes, but additionally considerably diminished the variety of steps.gpt-4.1-miniWe received the ultimate consequence after simply 6 steps in comparison with 14-16 stepsgpt-4o-mini. This implies that funding in dearer fashions could also be invaluable to the agent workflow. - Agent supplied Advanced instruments Analyze KPI modifications for easy metrics. This device performs all of the calculations, however LLM solely interprets the outcomes. In a earlier article, we mentioned intimately the method to KPI change evaluation.
- I performed the immediate very a lot Full the step-by-step information To assist brokers get again on monitor.
- Added Planning Process This encourages LLM brokers to first take into account their method and rethink their plans with each three iterations.
In any case changes, I obtained the next abstract from the agent: That is fairly good.
Govt Abstract:
Between April 2025 and Might 2025, complete income declined sharply by
roughly 36.03%, falling from 1,731,985.21 to 1,107,924.43, a
drop of -624,060.78 in absolute phrases.
This decline was primarily pushed by vital income
reductions within the 'new' buyer segments throughout a number of
international locations, with declines of roughly 70% in these segments.
Essentially the most impacted segments embrace:
- other_new: earlier than=233,958.42, after=72,666.89,
abs_change=-161,291.53, rel_change=-68.94%, share_before=13.51%,
influence=25.85, impact_norm=1.91
- UK_new: earlier than=128,324.22, after=34,838.87,
abs_change=-93,485.35, rel_change=-72.85%, share_before=7.41%,
influence=14.98, impact_norm=2.02
- France_new: earlier than=57,901.91, after=17,443.06,
abs_change=-40,458.85, rel_change=-69.87%, share_before=3.34%,
influence=6.48, impact_norm=1.94
- Germany_new: earlier than=48,105.83, after=13,678.94,
abs_change=-34,426.89, rel_change=-71.56%, share_before=2.78%,
influence=5.52, impact_norm=1.99
- Italy_new: earlier than=36,941.57, after=11,615.29,
abs_change=-25,326.28, rel_change=-68.56%, share_before=2.13%,
influence=4.06, impact_norm=1.91
- Spain_new: earlier than=32,394.10, after=7,758.90,
abs_change=-24,635.20, rel_change=-76.05%, share_before=1.87%,
influence=3.95, impact_norm=2.11
Primarily based on evaluation from the change log, the principle causes for this
pattern are:
1. The introduction of recent onboarding controls applied on Might
8, 2025, which diminished new buyer acquisition by about 70% to
stop fraud.
2. A postal service strike within the UK beginning April 5, 2025,
inflicting order supply delays and elevated cancellations
impacting the UK new phase.
3. A rise in VAT by 2% in Spain as of April 22, 2025,
affecting new buyer pricing and inflicting increased cart
abandonment.
These components mixed clarify the outsized detrimental impacts
noticed in new buyer segments and the general income decline.LLM brokers additionally generated many instance charts (they had been a part of our development rationalization device). For instance, this exhibits the influence throughout the mixture of nation and maturity.
The outcomes look actually thrilling. So let’s dig deeper into the precise implementation and perceive the way it works below the hood.
Multi-AI Agent Setup
Begin with the Changelog Agent. This agent queries the change log and makes an attempt to establish potential root causes of modifications to the metrics it observes. This agent is applied as a device name lingering agent, because it doesn’t require sophisticated operations. This agent known as by one other agent, so you will need to outline that agent. title and description attribute.
@device
def get_change_log(month: str) -> str:
"""
Returns the change log (checklist of inner and exterior occasions which may have affected our KPIs) for the given month
Args:
month: month within the format %Y-%m-01, for instance, 2025-04-01
"""
return events_df[events_df.month == month].drop('month', axis = 1).to_dict('data')
mannequin = LiteLLMModel(model_id="openai/gpt-4.1-mini", api_key=config['OPENAI_API_KEY'])
change_log_agent = ToolCallingAgent(
instruments=[get_change_log],
mannequin=mannequin,
max_steps=10,
title="change_log_agent",
description="Helps you discover the related info within the change log that may clarify modifications on metrics. Present the agent with all of the context to obtain data",
)The supervisor agent calls this agent and has no management over which queries it receives. So I made a decision to vary the system immediate to incorporate extra context.
change_log_system_prompt = '''
You are a grasp of the change log and also you assist others to clarify
the modifications to metrics. If you obtain a request, search for the checklist of occasions
occurred by month, then filter the related info based mostly
on supplied context and return again. Prioritise probably the most possible components
affecting the KPI and restrict your reply solely to them.
'''
modified_system_prompt = change_log_agent.prompt_templates['system_prompt']
+ 'nnn' + change_log_system_prompt
change_log_agent.prompt_templates['system_prompt'] = modified_system_promptTo permit the first agent to delegate duties to the Changelog Agent, it should merely be specified. managed_agents Subject.
agent = CodeAgent(
mannequin=mannequin,
instruments=[calculate_simple_growth_metrics],
max_steps=20,
additional_authorized_imports=["pandas", "numpy", "matplotlib.*", "plotly.*"],
verbosity_level = 2,
planning_interval = 3,
managed_agents = [change_log_agent]
)Let’s examine the way it works. First, you may see the brand new system immediate for the first agent. It now consists of details about crew members and directions on ask them for assist.
You can too give duties to crew members.
Calling a crew member works the identical as for calling a device: merely,
the one argument you can provide within the name is 'activity'.
On condition that this crew member is an actual human, you need to be very verbose
in your activity, it ought to be an extended string offering informations
as detailed as crucial.
Here's a checklist of the crew members you can name:
```python
def change_log_agent("Your question goes right here.") -> str:
"""Helps you discover the related info within the change log that
can clarify modifications on metrics. Present the agent with all of the context
to obtain data"""
```The execution log exhibits that the first agent efficiently delegated the duty to the second agent and obtained the next response:
<-- Major agent calling the change log agent -->
─ Executing parsed code: ───────────────────────────────────────
# Question change_log_agent with the detailed activity description
ready
context_for_change_log = (
"We analyzed modifications in income from April 2025 to Might
2025. We discovered giant decreases "
"primarily within the 'new' maturity segments throughout international locations:
Spain_new, UK_new, Germany_new, France_new, Italy_new, and
other_new. "
"The income fell by round 70% in these segments, which
have outsized detrimental influence on complete income change. "
"We wish to know the 1-3 most possible causes for this
vital drop in income within the 'new' buyer segments
throughout this era."
)
rationalization = change_log_agent(activity=context_for_change_log)
print("Change log agent rationalization:")
print(rationalization)
────────────────────────────────────────────────────────────────
<-- Change log agent execution begin -->
╭──────────────────── New run - change_log_agent ─────────────────────╮
│ │
│ You are a useful agent named 'change_log_agent'. │
│ You could have been submitted this activity by your supervisor. │
│ --- │
│ Activity: │
│ We analyzed modifications in income from April 2025 to Might 2025. │
│ We discovered giant decreases primarily within the 'new' maturity segments │
│ throughout international locations: Spain_new, UK_new, Germany_new, France_new, │
│ Italy_new, and other_new. The income fell by round 70% in these │
│ segments, which have outsized detrimental influence on complete income │
│ change. We wish to know the 1-3 most possible causes for this │
│ vital drop in income within the 'new' buyer segments throughout │
│ this era. │
│ --- │
│ You are serving to your supervisor resolve a wider activity: so ensure that to │
│ not present a one-line reply, however give as a lot info as │
│ potential to present them a transparent understanding of the reply. │
│ │
│ Your final_answer WILL HAVE to include these elements: │
│ ### 1. Activity consequence (quick model): │
│ ### 2. Activity consequence (extraordinarily detailed model): │
│ ### 3. Extra context (if related): │
│ │
│ Put all these in your final_answer device, all the things that you simply do │
│ not cross as an argument to final_answer will likely be misplaced. │
│ And even when your activity decision isn't profitable, please return │
│ as a lot context as potential, in order that your supervisor can act upon │
│ this suggestions. │
│ │
╰─ LiteLLMModel - openai/gpt-4.1-mini ────────────────────────────────╯Utilizing the Smolagents framework, you may simply arrange a easy multi-agent system the place supervisor brokers coordinate and delegate duties to crew members with particular expertise.
Repeat the immediate
We began with a really excessive stage immediate outlining the objectives and ambiguous instructions, however sadly it did not work persistently. LLMS isn’t sensible sufficient to grasp the method your self. So I created an in depth step-by-step immediate, explaining the whole plan and together with detailed specs of the expansion story instruments I’m utilizing.
activity = """
Here's a pandas dataframe exhibiting the income by phase, evaluating values
earlier than (April 2025) and after (Might 2025).
You are a senior and skilled information analyst. Your activity will likely be to grasp
the modifications to the income (after vs earlier than) in numerous segments
and supply government abstract.
## Observe the plan:
1. Begin by udentifying the checklist of dimensions (columns in dataframe that
usually are not "earlier than" and "after")
2. There is likely to be a number of dimensions within the dataframe. Begin high-level
by taking a look at every dimension in isolation, mix all outcomes
collectively into the checklist of segments analysed (do not forget to save lots of
the dimension used for every phase).
Use the supplied instruments to analyse the modifications of metrics: {tools_description}.
3. Analyse the outcomes from earlier step and maintain solely segments
which have outsized influence on the KPI change (absolute of impact_norm
is above 1.25).
4. Test what dimensions are current within the checklist of great phase,
if there are a number of ones - execute the device on their combos
and add to the analysed segments. If after including a further dimension,
all subsegments present shut different_rate and impact_norm values,
then we will exclude this break up (regardless that impact_norm is above 1.25),
because it would not clarify something.
5. Summarise the numerous modifications you recognized.
6. Attempt to clarify what's going on with metrics by getting data
from the change_log_agent. Please, present the agent the total context
(what segments have outsized influence, what's the relative change and
what's the interval we're taking a look at).
Summarise the data from the changelog and point out
solely 1-3 probably the most possible causes of the KPI change
(ranging from probably the most impactful one).
7. Put collectively 3-5 sentences commentary what occurred high-level
and why (based mostly on the information obtained from the change log).
Then observe it up with extra detailed abstract:
- Prime-line complete worth of metric earlier than and after in human-readable format,
absolute and relative change
- Listing of segments that meaningfully influenced the metric positively
or negatively with the next numbers: values earlier than and after,
absoltue and relative change, share of phase earlier than, influence
and normed influence. Order the segments by absolute worth
of absolute change because it represents the facility of influence.
## Instruction on the calculate_simple_growth_metrics device:
By default, it's best to use the device for the entire dataset not the phase,
because it offers you the total details about the modifications.
Right here is the steerage interpret the output of the device
- distinction - absolutely the distinction between after and earlier than values
- difference_rate - the relative distinction (if it is shut for
all segments then the dimension isn't informative)
- influence - the share of KPI differnce defined by this phase
- segment_share_before - share of phase earlier than
- impact_norm - influence normed on the share of segments, we're
in very excessive or very low numbers since they present outsized influence,
rule of thumb - impact_norm between -1.25 and 1.25 is not-informative
For those who're utilizing the device on the subset of dataframe bear in mind,
that the outcomes will not be aplicable to the total dataset, so keep away from utilizing it
except you wish to explicitly have a look at subset (i.e. change in France).
For those who determined to make use of the device on a selected phase
and share these leads to the chief abstract, explicitly define
that we're diving deeper into a selected phase.
""".format(tools_description = tools_description)
agent.run(
activity,
additional_args={"df": df},
)Explaining all of those particulars has been a really tough activity, but when constant outcomes are wanted, it’s crucial.
Planning Process
The Smolagents framework lets you add planning steps to an agent circulate. This encourages brokers to begin with a plan and replace after a specified variety of steps. From my expertise, this reflection is extraordinarily useful in sustaining a give attention to the issue, adjusting the actions, and aligning with the preliminary planning and objectives. It is strongly recommended to make use of it when complicated inference is required.
Setup is so simple as specifying planning_interval = 3 For code brokers.
agent = CodeAgent(
mannequin=mannequin,
instruments=[calculate_simple_growth_metrics],
max_steps=20,
additional_authorized_imports=["pandas", "numpy", "matplotlib.*", "plotly.*"],
verbosity_level = 2,
planning_interval = 3,
managed_agents = [change_log_agent]
)that is it. The agent then gives reflections that start with fascinated by the preliminary planning.
────────────────────────── Preliminary plan ──────────────────────────
Listed here are the details I do know and the plan of motion that I'll
observe to resolve the duty:
```
## 1. Info survey
### 1.1. Info given within the activity
- We've a pandas dataframe `df` exhibiting income by phase, for
two time factors: earlier than (April 2025) and after (Might 2025).
- The dataframe columns embrace:
- Dimensions: `nation`, `maturity`, `country_maturity`,
`country_maturity_combined`
- Metrics: `earlier than` (income in April 2025), `after` (income in
Might 2025)
- The duty is to grasp the modifications in income (after vs
earlier than) throughout totally different segments.
- Key directions and instruments supplied:
- Establish all dimensions besides earlier than/after for segmentation.
- Analyze every dimension independently utilizing
`calculate_simple_growth_metrics`.
- Filter segments with outsized influence on KPI change (absolute
normed influence > 1.25).
- Study combos of dimensions if a number of dimensions have
vital segments.
- Summarize vital modifications and interact `change_log_agent`
for contextual causes.
- Present a last government abstract together with top-line modifications
and segment-level detailed impacts.
- Dataset snippet exhibits segments combining international locations (`France`,
`UK`, `Germany`, `Italy`, `Spain`, `different`) and maturity standing
(`new`, `present`).
- The mixed segments are uniquely recognized in columns
`country_maturity` and `country_maturity_combined`.
### 1.2. Info to search for
- Definitions or descriptions of the segments if unclear (e.g.,
what defines `new` vs `present` maturity).
- Possible not obligatory to proceed, however might be requested from
enterprise documentation or change log.
- Extra particulars on the change log (accessible through
`change_log_agent`) that might present possible causes for income
modifications.
- Affirmation on dealing with mixed dimension splits - how precisely
`country_maturity_combined` is fashioned and ought to be interpreted in
mixed dimension evaluation.
- Information dictionary or description of metrics if any extra KPI
apart from income is related (unlikely given information).
- Dates affirm interval of study: April 2025 (earlier than) and Might
2025 (after). No must look these up since given.
### 1.3. Info to derive
- Establish all dimension columns out there for segmentation:
- By excluding 'earlier than' and 'after', probably candidates are
`nation`, `maturity`, `country_maturity`, and
`country_maturity_combined`.
- For every dimension, calculate change metrics utilizing the given
device:
- Absolute and relative distinction in income per phase.
- Influence, phase share earlier than, and normed influence for every
phase.
- Establish which segments have outsized influence on KPI change
(|impact_norm| > 1.25).
- If a number of dimensions have vital segments, mix
dimensions (e.g., nation + maturity) and reanalyze.
- Decide if mixed dimension splits present significant
differentiation or not, based mostly on delta fee and impact_norm
consistency.
- Summarize course and magnitude of KPI modifications at top-line
stage (combination income earlier than and after).
- Establish high segments driving constructive and detrimental modifications
based mostly on ordered absolute absolute_change.
- Collect contextual insights from the change log agent relating to
possible causes tied to vital segments and the Might 2025 vs
April 2025 interval.
## 2. Plan
1. Establish all dimension columns current within the dataframe by
itemizing columns and excluding 'earlier than' and 'after'.
2. For every dimension recognized (`nation`, `maturity`,
`country_maturity`, `country_maturity_combined`):
- Use `calculate_simple_growth_metrics` on the total dataframe
grouped by that dimension.
- Extract segments with calculated metrics together with
impact_norm.
3. Mixture outcomes from all single-dimension analyses and filter
segments the place |impact_norm| > 1.25.
4. Decide which dimensions these vital segments belong
to.
5. If multiple dimension is represented in these vital
segments, analyze the mixed dimension fashioned by these
dimensions (for instance, mixture of `nation` and `maturity`
or use present mixed dimension columns).
6. Repeat metric calculation utilizing
`calculate_simple_growth_metrics` on the mixed dimension.
7. Study if the mixed dimension splits create significant
differentiation - if all subsegments present shut difference_rate
and impact_norm, exclude the break up.
8. Put together a abstract of great modifications:
- Prime-line KPIs earlier than and after (absolute and relative
modifications).
- Listing of impactful segments sorted by absolute absolute_change
that influenced general income.
9. Present the checklist of segments with particulars (values earlier than,
after, absolute and relative change, share earlier than, influence,
impact_norm).
10. Utilizing this summarized info, question `change_log_agent`
with full context:
- Embody vital segments, their relative modifications, and
intervals (April to Might 2025).
11. Course of the agent's response to establish 1-3 most important possible
causes of the KPI modifications.
12. Draft government abstract commentary:
- Excessive-level overview of what occurred and why, based mostly on log
data.
- Detailed abstract together with top-line modifications and
segment-level metrics influence.
13. Ship the ultimate reply utilizing `final_answer` device containing
the above government abstract and data-driven insights.Then, in every of the three steps, the agent revisits and updates the plan.
────────────────────────── Up to date plan ──────────────────────────
I nonetheless want to resolve the duty I used to be given:
```
Here's a pandas dataframe exhibiting the income by phase,
evaluating values earlier than (April 2025) and after (Might 2025).
You are a senior and skilled information analyst. Your activity will likely be
perceive the modifications to the income (after vs earlier than) in
totally different segments
and supply government abstract.
<... repeating the total preliminary activity ...>
```
Listed here are the details I do know and my new/up to date plan of motion to
resolve the duty:
```
## 1. Up to date details survey
### 1.1. Info given within the activity
- We've a pandas dataframe with income by phase, exhibiting
values "earlier than" (April 2025) and "after" (Might 2025).
- Columns within the dataframe embrace a number of dimensions and the
"earlier than" and "after" income values.
- The objective is to grasp income modifications by phase and supply
an government abstract.
- Steerage and guidelines about analyze and interpret outcomes
from the `calculate_simple_growth_metrics` device are supplied.
- The dataframe incorporates columns: nation, maturity,
country_maturity, country_maturity_combined, earlier than, after.
### 1.2. Info that we now have discovered
- The scale to investigate are: nation, maturity,
country_maturity, and country_maturity_combined.
- Analyzed income modifications by dimension.
- Solely the "new" maturity phase has vital influence
(impact_norm=1.96 > 1.25), with a big detrimental income change (~
-70.6%).
- Within the mixed phase "country_maturity," the "new" segments
throughout international locations (Spain_new, UK_new, Germany_new, France_new,
Italy_new, other_new) all have outsized detrimental impacts with
impact_norm values all above 1.9.
- The mature/present segments in these international locations have smaller
normed impacts beneath 1.25.
- Nation-level and maturity-level phase dimension alone are
much less revealing than the mixed nation+maturity phase
dimension which highlights the brand new segments as strongly impactful.
- Whole income dropped considerably from earlier than to after, largely
pushed by new segments shrinking drastically.
### 1.3. Info nonetheless to search for
- Whether or not splitting the info by extra dimensions past
nation and maturity (e.g., country_maturity_combined) explains
additional heterogeneous impacts or if the sample is uniform.
- Rationalization/context from change log about what prompted the most important
drop predominantly in new segments in all international locations.
- Confirming whether or not any nation inside the new phase behaved
otherwise or mitigated losses.
### 1.4. Info nonetheless to derive
- A concise government abstract describing the top-level income
change and figuring out which segments clarify the declines.
- Rationalization involving the change log agent with abstract of
possible causes for these outsized reductions in income within the
new segments throughout international locations for April-Might 2025.
## 2. Plan
### 2.1. Confirm if including the extra dimension
'country_maturity_combined' splits the impactful "new" segments
into subsegments with considerably totally different impacts or if the
change charges and normed impacts are comparatively homogeneous. If
homogeneous, we don't achieve deeper perception and will disregard
additional splitting.
### 2.2. Summarize all vital segments recognized with
outsized impact_norm ≥ 1.25, together with their earlier than and after
values, absolute and relative modifications, phase shares earlier than,
influence, and normalized influence, ordered by absolute worth of the
change.
### 2.3. Question the change_log_agent with the total context:
vital segments are the brand new country_maturity segments with
giant detrimental modifications (~ -70%), timeframe April 2025 to Might 2025,
and request high 1-3 most possible causes for the KPI income drop
in these segments.
### 2.4. Primarily based on the change log agent's response, synthesize a
3-5 sentence high-level commentary explaining what occurred
broadly and why.
### 2.5. Draft an in depth government abstract together with:
- Whole income earlier than and after in human-readable format with
absolute and relative change.
- A listing of great segments driving these modifications, so as
by absolute influence, with detailed numbers (earlier than, after, absolute
and relative change, phase share earlier than, influence, normed influence).
### 2.6. Use the `final_answer` device to supply the finalized
government abstract report.I actually like the best way brokers are inspired to repeat the primary activity and give attention to the principle challenge. Common reflections like this can be helpful in actual life. Groups are sometimes caught within the course of and lose sight of the rationale behind what they’re doing. It is fairly cool to see administration greatest practices built-in into the agent framework.
that is it! We constructed a code agent that might analyze KPI modifications for easy metrics and investigated all of the vital nuances of the method.
Yow will discover the whole code and working logon github.
abstract
I did a whole lot of experiments with code brokers and am prepared to attract a conclusion. In our experiments, we used the Huggingface Smolagents framework for our code brokers. It is a very great tool set.
- Easy integration with totally different LLMs (from native fashions through oramas to public suppliers akin to humanity and Openai),
- Excellent logging that makes it simpler to grasp the whole thought technique of brokers and debugging issues,
- The flexibility to construct complicated techniques that make the most of the setup or planning capabilities of multi-AI brokers with out a lot effort.
Smolagents is my favourite agent framework proper now, however there are limitations.
- Generally it may be missing in flexibility. For instance, I needed to instantly modify the supply code immediate to get the required habits.
- It solely helps hierarchical multi-agent setups (the place one supervisor can delegate duties to different brokers), however doesn’t cowl sequential workflows or consensus-based decision-making processes.
- It doesn’t help long-term reminiscence out of the field. In different phrases, you begin from scratch for each activity.
Thanks for studying this text. I hope this text was insightful to you.
reference
This text isCode Agent hugging her face, hugging her faceBrief programs by deeplearning.ai.
